Project

General

Profile

Review Request #6152

Review Request #6135: Data Management Workflow Umbrella Task

Remove direct cpn calls

Added by Gabriel Perdue over 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Immediate
Start date:
05/06/2014
Due date:
08/12/2014
% Done:

100%

Estimated time:
(Total: 0.00 h)
Spent time:
(Total: 1.00 h)
Duration: 99

Description

Remove all instances of the use of the deprecated “cpn” locking package and replace them with the equivalent calls to the supported “ifdh” patches.


Subtasks

Necessary Maintenance #6181: Replace cpn with ifdh for MINOS filesClosedMarc Mengel

Review Request #6186: Review non-MinosDataAlg explicit cpn callsClosedCheryl Patrick

History

#1 Updated by Philip Rodrigues over 5 years ago

  • % Done changed from 0 to 10

I grepped the code for direct cpn calls and got this:

minervagpvm05:/grid/fermiapp/minerva/software_releases/v10r7p3/minerva/MINERVA/MINERVA_v10r7p3

find . -type f -name 'x86_64-slc5-gcc43*' -prune -o -print | xargs grep -iI cpn | tee /scratch/minerva/rodriges/minerva-v10r7p3-explicit-cpns

./Ana/CCQEAntiNu/ana/make_plots/CCQEAntiNu_Utils.h: system(("/grid/fermiapp/common/tools/cpn "+filename+" "+directory).c_str());
./Ana/CCQEAntiNu/ana/make_plots/CCQEAntiNu_Utils.h: std::cout<<"cpn took "<<startTime-endTime<<" seconds "<<std::endl;
./Ana/CCQEAntiNu/scripts/CCQEPlottingUtils.C: system(("/grid/fermiapp/minos/scripts/cpn "+filename+" "+directory).c_str());
./Ana/CCQEAntiNu/scripts/CCQEPlottingUtils.C: cout<<"cpn took "<<startTime-endTime<<" seconds "<<endl;
./Ana/CCQuasiElastic/scripts/oneTrack/Utils.C: system(("/grid/fermiapp/minos/scripts/cpn "+filename+" "+directory).c_str());
./Ana/CCQuasiElastic/scripts/oneTrack/Utils.C: cout<<"cpn took "<<startTime-endTime<<" seconds "<<endl;
./Ana/TestBeamAna/scripts/root-piratios.C: TH1F mcpNumEvents = (TH1F)fmcp->Get("hnumEventsProcessed");
./Ana/UnfoldUtils/tmp/linuxx8664gcc/RooUnfoldMap_LinkDef.h:extern wchar_t *wcpncpy (wchar_t *_dest, __const wchar_t *_src, size_t _n)
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: char cpn_cmd528;
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: static const char * CPN = "/grid/fermiapp/minos/scripts/cpn"; //maybe we can get this from environmental?
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: sprintf( cpn_cmd, "%s %s %s", CPN, (ENERGYRECTOOLSROOT+"/data/argoneut_lookup_table.root").c_str(), CONDOR_DIR_INPUT );
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: << " " << cpn_cmd
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: const int rval = system( cpn_cmd );
./Rec/EnergyRecTools/src/ArgoneutEnergyCorrection.cpp: error() << "Attempt to CPN the ArgoNeuT Lookup Table to the worker node failed with return value: " << rval << endmsg;
./Tools/ControlRoomTools/nearline_scripts/nearline_bluearc_copy.sh: cpn $file $NEARLINE_BLUEARC_SWAP_AREA
./Tools/ControlRoomTools/nearline_scripts/nearline_bluearc_copy.sh: cpn $file /minerva/app/users/condor-tmp/nearonline/
./Tools/ControlRoomTools/nearline_scripts/nearline_bluearc_copy.sh: cpn $file $NEARLINE_BLUEARC_GMPLOTTER_AREA/www/
./Tools/ControlRoomTools/nearline_scripts/nearline_bluearc_copy.sh: cpn $file $NEARLINE_BLUEARC_GMPLOTTER_AREA/plotter/
./Tools/CondorUtils/python/CondorJobBuilder.py: # production accounts should use CPN so they own their output files.
./Tools/CondorUtils/src/CondorInput.cxx: static const TString cpn("cpn");
./Tools/CondorUtils/src/CondorInput.cxx: //! Execute the CPN and return its exit code
./Tools/CondorUtils/src/CondorInput.cxx: const TString cpnCmd = cpn + " " + fname + " " + *condorDirInput
;
./Tools/CondorUtils/src/CondorInput.cxx: return system( cpnCmd );
./Tools/MinosInterface/src/MinosDataAlg.cpp: //note: If we are running in a condor environment these filenames will be in CONDOR_DIR_INPUT, so we simply open the file with no CPN
./Tools/MinosInterface/src/MinosDataAlg.cpp: //! If we are on the grid, CPN the SNTP to the local host's CONDOR_DIR_INPUT area
./Tools/MinosInterface/src/MinosDataAlg.cpp: char cpn_cmd528;
./Tools/MinosInterface/src/MinosDataAlg.cpp: static const char * CPN = "/grid/fermiapp/minos/scripts/cpn"; //maybe we can get this from environmental?
./Tools/MinosInterface/src/MinosDataAlg.cpp: sprintf( cpn_cmd, "%s %s %s", CPN, itDBEntry->fname.c_str(), CONDOR_DIR_INPUT );
./Tools/MinosInterface/src/MinosDataAlg.cpp: << " " << cpn_cmd
./Tools/MinosInterface/src/MinosDataAlg.cpp: const int rval = system( cpn_cmd );
./Tools/MinosInterface/src/MinosDataAlg.cpp: error() << "Attempt to CPN the MINOS SNTP file to the worker node failed with return value: " << rval << endmsg;
./Tools/ProductionScripts/data_scripts/DataRunProcessor.py: self.n_locks = 0 #keep track of the number of cpn locks used
./Tools/ProductionScripts/data_scripts/DataRunProcessor.py:print "Number of CPN locks used:",job.n_locks
./Tools/ProductionScripts/data_scripts/DataRunProcessorSAM.py: self.n_locks = 0 #keep track of the number of cpn locks used
./Tools/ProductionScripts/data_scripts/DataRunProcessorSAM.py:print "Number of CPN locks used:",job.n_locks
./Tools/ProductionScripts/ongrid_scripts/GaudiRunProcessor.py: self.n_locks = 0 #keep track of the number of cpn locks used
./Tools/ProductionScripts/ongrid_scripts/GaudiRunProcessor.py:print "Number of CPN locks used:",job.n_locks
./Tools/ProductionScripts/ongrid_scripts/GaudiRunProcessorSAM.py: self.n_locks = 0 #keep track of the number of cpn locks used
./Tools/ProductionScripts/ongrid_scripts/GaudiRunProcessorSAM.py:print "Number of CPN locks used:",job.n_locks
./Tools/ProductionScripts/rawdata_scripts/PedTableProduction.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/rawdata_scripts/PedTableProductionSamweb.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/rawdata_scripts/RawDataToRawDigits.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/rawdata_scripts/RawDataToRawDigitsSamweb.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/rawdata_scripts/RawDigitsToMinMonitoring.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/rawdata_scripts/RawDigitsToSupDigits.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/samweb/PedTableProductionDEV.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);
./Tools/ProductionScripts/samweb/RawDataToRawDigitsDEV.py: os.system("/grid/fermiapp/minos/scripts/cpn "+logfile+" "+blogfile);

Most of the culprits are in raw data scripts in ProductionScripts. I'm aware of the MinosDataAlg one, and want to fix it as part of replacing MinosFileDB with something less horrible. I didn't look into any others.

#2 Updated by Gabriel Perdue over 5 years ago

  • Assignee set to Cheryl Patrick

Change all cpn calls to ifdh

#3 Updated by Gabriel Perdue over 5 years ago

  • Status changed from New to Assigned

#4 Updated by Daniel Ruterbories about 5 years ago

  • Status changed from Assigned to Resolved

#5 Updated by Daniel Ruterbories over 4 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF