Project

General

Profile

Support #5640

stage /pnfs/minos/neardet_data

Added by Arthur Kreymer over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Start date:
03/12/2014
Due date:
% Done:

100%

Estimated time:
4.00 h
Spent time:
Duration:

Description

Per rhatcher request, prestage Minos ND data to DCache for use in Minerva overlays.

History

#1 Updated by Arthur Kreymer over 5 years ago

Tested 1 month, nothing found to do

mindata@minos27

ST=/grid/fermiapp/common/tools/test/stage
LOGD=/nusoft/app/web/htdoc/minos/dcache/stage
mkdir ${LOGD}

Initial month test, all was on disk

set nohup
${ST} -q -L ${LOGD} minos /pnfs/minos/neardet_data/2010-10

set nohup
${ST} -q -L ${LOGD} minos /pnfs/minos/neardet_data
less /nusoft/app/web/htdoc/minos/dcache/stage/_pnfs_minos_neardet_data.20140312-1425.log

STARTING Wed Mar 12 14:25:59 CDT 2014

93868 in directory /pnfs/minos/neardet_data

The list of files had not yet been built the next day.
Cancelled the job.

#2 Updated by Arthur Kreymer over 5 years ago

  • % Done changed from 0 to 50
  • Estimated time set to 4.00 h

To avoid possible threading issues,
o cut default dcdisk threads to 1,
o copied this and stage to /grid/fermiapp/minos/scripts

Restarted the stage, mindata@minos27

ST=/grid/fermiapp/minos/scripts/stage
LOGD=/nusoft/app/web/htdoc/minos/dcache/stage
set nohup
${ST} -q -L ${LOGD} minos /pnfs/minos/neardet_data

less /nusoft/app/web/htdoc/minos/dcache/stage/_pnfs_minos_neardet_data.20140316-0834.log

see new log of files needed,
less /nusoft/app/web/htdoc/minos/dcache/stage/_pnfs_minos_neardet_data.20140316-0834.files

93955 in directory /pnfs/minos/neardet_data
83905 on disk
10050 off disk
Staging 10050 files
/grid/fermiapp/minos/scripts/stage: line 72: Staging 10050 files \n: command not found
...
10050 off disk
7687 off disk
5232 off disk
2361 off disk
250 off disk
250 off disk
Sun Mar 16 21:28:24 CDT 2014
...

The missing files are all in directories
neardet_data/NEWDAQ
neardet_data/2012-12

Examples
NEED neardet_data/NEWDAQ/N02000388_0000.mdaq.root
NEED neardet_data/2012-12/N00022886_0005.mdaq.root

These are on tape, using dc_stat in ~kreymer/minos/scripts

./dc_stat /pnfs/minos/neardet_data/NEWDAQ/N02000388_0000.mdaq.root
...
LAYER 4
VP6333
0000_000000000_0005925
37759
neardet_data
/pnfs/minos/neardet_data/NEWDAQ/N02000388_0000.mdaq.root

0000AD93F403E0634CE3B586720B88F8B17B

CDMS135837741400000
enmvr029:/dev/rmt/tps4d0n:1310265181
4207252136

MINOS27 > ./dc_stat N00022886_0005.mdaq.root
...
LAYER 4
VP6333
0000_000000000_0005077
62157650
neardet_data
/pnfs/fnal.gov/usr/minos/neardet_data/2012-12/N00022886_0005.mdaq.root

0000F25B9D8C5A45436FAD533188CD4812D6

CDMS135474841800000
enmvr030:/dev/rmt/tps4d0n:9311180992
3264857355

The raw pools show 2 restores active at
http://fndca.fnal.gov:2288/queueInfo

tape VP6333 is LT04 CD-LTO4F1

Copies started again :
250 off disk Mon Mar 17 10:43:13 CDT 2014
110 off disk Mon Mar 17 11:25:04 CDT 2014
...
1 off disk
no file listed, nothing in the file list.

But one file is still needed, though never in the NEED list in the *.files list :
Spotted it in /tmp/mindata-_pnfs_minos_neardet_data.20140316-0834.log

/pnfs/minos/neardet_data/2013-06/N00049492_0007.mdaq.root

Killing the global restore, will work on 2013-06 for quicker debugging.
And fix that stray Staging message.

#3 Updated by Arthur Kreymer over 5 years ago

  • Status changed from New to Resolved
  • % Done changed from 50 to 100

Fixed the stage script to include the last file in the list.
And to clean up that print statement.

All near files are on disk now.



Also available in: Atom PDF