Task #15476
Remove from persistent runs <= 247
Description
As discussed in January ,
runs in v2data and earlier are not needed in the PNFS persistent area.
This is runs <= 247.
Remove these using the archiver/transient script,
as persistent space usage iscurrently over 90%
History
#1 Updated by Arthur Kreymer almost 3 years ago
Please confirm that it is OK to remove RawData runs <= 247
from /pnfs/annie/persistent/raw/raw.
The files will remain in /pnfs/annie/raw/raw.
#2 Updated by Arthur Kreymer almost 3 years ago
Date: Tue, 7 Feb 2017 01:31:35 +0000
From: Benjamin Richards <b.richards@qmul.ac.uk>
To: redmine@fnal.gov
Cc: Arthur Kreymer <kreymer@fnal.gov>, basnet@physics.org, rhatcher@fnal.gov, msanchez@fnal.gov, wetstein@fnal.gov
Hi,
Sounds good to me
thanks
Ben
#3 Updated by Arthur Kreymer almost 3 years ago
- % Done changed from 10 to 90
https://cdcvs.fnal.gov/redmine/issues/15476 GET LIST OF RUNS ssh annieraw@annie01.fnal.gov RUNS=$(ls /pnfs/annie/persistent/raw/v2data) echo ${RUNS} 0058 0062 0068 0100 0101 0102 0103 0104 0105 0106 0107 0108 0109 0110 0111 0112 0113 0114 0116 0119 0120 0121 0122 0123 0124 0125 0126 0127 0128 0129 0130 0131 0132 0133 0134 0135 0136 0137 0138 0139 0140 0141 0142 0143 0144 0145 0146 0147 0148 0149 0150 0151 0152 0153 0154 0155 0156 0157 0158 0159 0160 0161 0162 0163 0164 0165 0166 0167 0168 0169 0172 0173 0174 0175 0176 0177 0178 0179 0180 0181 0182 0183 0184 0186 0188 0189 0190 0191 0192 0194 0195 0196 0197 0198 0199 0200 0203 0204 0205 0206 0207 0208 0209 0210 0211 0212 0213 0214 0215 0216 0217 0218 0219 0220 0221 0222 0223 0224 0225 0226 0227 0228 0229 0230 0231 0234 0235 0236 0237 0238 0243 0244 0245 0246 0247 echo ${RUNS} | wc -w 135 VERIFY AGAINST RAW/RAW RUNSR=$(ls /pnfs/annie/persistent/raw/raw | head -135) printf "${RUNS}\n" > /dev/shm/RUNS printf "${RUNSR}\n" > /dev/shm/RUNSR diff /dev/shm/RUNS /dev/shm/RUNSR DRY RUN for RUN in ${RUNS} ; do ${HOME}/archiver/transient raw ${RUN} ; done REAL RUN du -sh /pnfs/annie/persistent/raw/raw 31T /pnfs/annie/persistent/raw/raw for RUN in ${RUNS} ; do ${HOME}/archiver/transient raw ${RUN} do ; done RUN transient raw 0058 Version 20161225 Wed Feb 8 23:09:08 CST 2017 58647 /pnfs/annie/persistent/raw/raw/0058 removed `/pnfs/annie/persistent/raw/raw/0058/RAWDataR58S139923473476680p0.root' removed `/pnfs/annie/persistent/raw/raw/0058/RAWDataR58S139923473476680p1.root' ... removed `/pnfs/annie/persistent/raw/raw/0247/RAWDataR247S2p97.root' removed `/pnfs/annie/persistent/raw/raw/0247/RAWDataR247S2p98.root' DONE Wed Feb 8 23:18:45 CST 2017 du -sh /pnfs/annie/persistent/raw/raw 11T /pnfs/annie/persistent/raw/raw SUMMARY Usage is reduced by 20 TB, about 40% Will verify this on the web status page tomorrow. http://fndca3a.fnal.gov/cgi-bin/du_cgi.py
#4 Updated by Arthur Kreymer almost 3 years ago
Space usage remains high, in spite of file removal :
http://fndca3a.fnal.gov/cgi-bin/du_cgi.py
shows 45 TB still in use, no reduction.
Yujun - any idea why ?
#5 Updated by Arthur Kreymer over 2 years ago
- Status changed from Assigned to Resolved
- % Done changed from 90 to 100
Completed Feb 2017.
Space reports took a while to catch up, due to scale of removals.