Project

General

Profile

Feature #8304

FTS for use with Roundup

Added by Arthur Kreymer almost 5 years ago. Updated almost 5 years ago.

Status:
Accepted
Priority:
High
Assignee:
Arthur Kreymer
Start date:
04/12/2015
Due date:
% Done:

70%

Estimated time:
Duration:

Description

Configure FTS for use with Roundup.

Document in https://cdcvs.fnal.gov/redmine/projects/minos/wiki/FTS

History

#1 Updated by Arthur Kreymer almost 5 years ago

Sunday 04/12 configured to write to /pnfs/minos/scratch

filetypes = test_files nearsntp

[filetype nearsntp]

scan-dirs = /minos/data/minfarm/ftsnearcat
scan-interval = 600
scan-delay = 1
scan-file-patterns = *sntp*root
scan-exclude-file-patterns = *.json

extract-metadata = False

#transfer-to = enstore:/pnfs/minos/scratch/users/minospro/ftstest/reco_near/dogwood0/sntp_data/
transfer-to = enstore:/pnfs/minos/scratch/users/minospro/ftstest

#2 Updated by Arthur Kreymer almost 5 years ago

The first file has been written !

Date: Tue, 14 Apr 2015 14:53:51 -0500
From: Michael Gheith <>
To: Arthur E Kreymer <>
Subject: RE: MINOS FTS

I think you should be OK now Arthur.
Mike D. found a typo in the sam_cp.cfg file which was causing the correct proxy from being set.

#3 Updated by Arthur Kreymer almost 5 years ago

  • Status changed from Assigned to Accepted
  • % Done changed from 0 to 70

This is tested now generating the path automatically, 

minospro@minossamgpvm01

fake script moves and renames/redeclared a file from

OP=/pnfs/minos/reco_near/elm4/sntp_data/2015-03
to
NP=/minos/data/minfarm/ftsnearcat
and archive to
PO=/pnfs/minos/scratch/users/minospro/ftstest

See logs in
FTSL=/minos/app/home/minospro/FTS/minossamgpvm01.fnal.gov/fts_logs
SCL=/minos/app/home/minospro/FTS/minossamgpvm01.fnal.gov/fts_logs/sam_cp.log

source /grid/fermiapp/products/common/etc/setups.sh

setup sam_web_client
export SAM_EXPERIMENT=minos

export X509_USER_PROXY=/var/tmp/minossamgpvm01.fnal.gov_fts_proxy

cd ${HOME}/FTS/minossamgpvm01.fnal.gov

./stop
OFILE=N00061107_0000.spill.sntp.elm4.0.root
./start

ls -lR $PO
less ${FTSL}/fts.2015-06-11.log

This all works, next step to to write somewhere backed by tape,
   and be sure the cleanup works appropriately.

#4 Updated by Arthur Kreymer almost 5 years ago

Writing to tape as been tested.

Tested to dogwood0 cand, so the tape can easily be recycled.

transfer-to = enstore:/pnfs/minos/reco_near/${app_version}/cand_data/${year}-${month}/

Found up to 5 copies running at once, counterproductive
This is limited by CPN locks during this test.
Looking for a way to limit this when going through this host.

Also available in: Atom PDF