Feature #8304
FTS for use with Roundup
70%
Description
Configure FTS for use with Roundup.
Document in https://cdcvs.fnal.gov/redmine/projects/minos/wiki/FTS
History
#1 Updated by Arthur Kreymer almost 6 years ago
Sunday 04/12 configured to write to /pnfs/minos/scratch
filetypes = test_files nearsntp
[filetype nearsntp]
scan-dirs = /minos/data/minfarm/ftsnearcat
scan-interval = 600
scan-delay = 1
scan-file-patterns = *sntp*root
scan-exclude-file-patterns = *.json
extract-metadata = False
#transfer-to = enstore:/pnfs/minos/scratch/users/minospro/ftstest/reco_near/dogwood0/sntp_data/
transfer-to = enstore:/pnfs/minos/scratch/users/minospro/ftstest
#2 Updated by Arthur Kreymer almost 6 years ago
The first file has been written !
Date: Tue, 14 Apr 2015 14:53:51 -0500
From: Michael Gheith <mgheith@fnal.gov>
To: Arthur E Kreymer <kreymer@fnal.gov>
Subject: RE: MINOS FTS
I think you should be OK now Arthur.
Mike D. found a typo in the sam_cp.cfg file which was causing the correct proxy from being set.
#3 Updated by Arthur Kreymer over 5 years ago
- Status changed from Assigned to Accepted
- % Done changed from 0 to 70
This is tested now generating the path automatically,
minospro@minossamgpvm01
fake script moves and renames/redeclared a file from
OP=/pnfs/minos/reco_near/elm4/sntp_data/2015-03
to
NP=/minos/data/minfarm/ftsnearcat
and archive to
PO=/pnfs/minos/scratch/users/minospro/ftstest
See logs in
FTSL=/minos/app/home/minospro/FTS/minossamgpvm01.fnal.gov/fts_logs
SCL=/minos/app/home/minospro/FTS/minossamgpvm01.fnal.gov/fts_logs/sam_cp.log
source /grid/fermiapp/products/common/etc/setups.sh
setup sam_web_client
export SAM_EXPERIMENT=minos
export X509_USER_PROXY=/var/tmp/minossamgpvm01.fnal.gov_fts_proxy
cd ${HOME}/FTS/minossamgpvm01.fnal.gov
./stop
OFILE=N00061107_0000.spill.sntp.elm4.0.root
./start
ls -lR $PO
less ${FTSL}/fts.2015-06-11.log
This all works, next step to to write somewhere backed by tape,
and be sure the cleanup works appropriately.
#4 Updated by Arthur Kreymer over 5 years ago
Writing to tape as been tested.
Tested to dogwood0 cand, so the tape can easily be recycled.
transfer-to = enstore:/pnfs/minos/reco_near/${app_version}/cand_data/${year}-${month}/
Found up to 5 copies running at once, counterproductive
This is limited by CPN locks during this test.
Looking for a way to limit this when going through this host.