Project

General

Profile

Task #15475

Remove v2data runs converted to v5data

Added by Arthur Kreymer almost 3 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
High
Start date:
02/06/2017
Due date:
02/10/2017
% Done:

100%

Estimated time:
3.00 h
Spent time:
Duration: 5

Description

As agreed in Ops meetins, v2data runs which exist in v5data
can be removed from /pnfs/annie/persistent.
They remain in /pnfs/annie/raw, for an archive.

This needs to be done soon, persistent usage is over 90%.
That is unhealthy, and even worse if the disk fills.

History

#1 Updated by Arthur Kreymer almost 3 years ago

  • % Done changed from 10 to 50
  • Estimated time changed from 2.00 h to 3.00 h

I scanned v3 vs v5 files, using an archiver/v3inv5 script on annie01.

This counts files over 1 MB, and tries to match file names,
changing V5D to V3D under v5data, and changing T1 to 1 under v3data.

Only 3 runs have matching files, 0248 0249 0250

I have run the transient script on those areas :

RUNS='0248 0249 0250'

for RUN in ${RUNS} ; do ./transient v3data ${RUN} ; done

That looked ok, so ran the removal :

for RUN in ${RUNS} ; do ./transient v3data ${RUN} do ; done

See archiver/transient.log

Two runs seem to be partly converted.
v3inv5 shows file counts, net size in MBytes,
and prints MATCH if file names match.

$ ./v3inv5

0248 607 files
302854 /pnfs/annie/persistent/raw/v3data/0248
331086 /pnfs/annie/persistent/raw/v5data/0248 MATCH

0249 706 files
352933 /pnfs/annie/persistent/raw/v3data/0249
386013 /pnfs/annie/persistent/raw/v5data/0249 MATCH

0250 786 files
394547 /pnfs/annie/persistent/raw/v3data/0250
431494 /pnfs/annie/persistent/raw/v5data/0250 MATCH

0251 count mismatch 981 979

0252 count mismatch 518 330
...

#2 Updated by Arthur Kreymer almost 3 years ago

This freed only about 1 TB. We need several more.

I will probably have to proceed with the originally planned
removal from persistent of RawData files through run 247,
as previously discussed. This could free up 22 TB.

I will open a separate issue for that.

#3 Updated by Arthur Kreymer about 2 years ago

  • Status changed from Assigned to Resolved

Resolved Feb 2017.

#4 Updated by Arthur Kreymer about 2 years ago

  • % Done changed from 50 to 100


Also available in: Atom PDF