Support #19599
art job influenced by random files
0%
Description
Hi,
It looks like the behavior of an art job can be influenced by "random"
rogue files on disk, which are not findable via PATH, LD_LIBRARY_PATH,
etc). This happens even when there is no '.' anywhere in the
environment. This is a bug because jobs should be defined by their
release area and explicit inputs, and not affected by other random
files.
Andrei
1) prepare a test file
ssh mu2egpvm01.fnal.gov setup mu2e source /cvmfs/mu2e.opensciencegrid.org/Offline/v6_5_2/SLF6/prof/Offline/setup.sh mkdir -p /mu2e/app/users/$(whoami)/20180406-dict-breakage cd /mu2e/app/users/$(whoami)/20180406-dict-breakage cp -pr /mu2e/app/users/gandr/20180406-dict-breakage/inputs . mu2e -c inputs/testjob.fcl # takes a couple of minutes
2) Verify that data products can be listed without complaints:
mu2e -c Print/fcl/dumpDataProducts.fcl carbon_muons_hits.art > /dev/null
(no complaints)
3) Drop a bomb:
mv inputs/RecoDataProducts .
4) Re-run the data product listing:
mu2e -c Print/fcl/dumpDataProducts.fcl carbon_muons_hits.art > /dev/null
In file included from libmu2e_RecoDataProducts_dict dictionary payload:27:
./RecoDataProducts/inc/StereoHit.hh:5:20: error: typedef redefinition with different types ('mu2e::ComboHit' vs 'mu2e::StereoHit')
typedef ComboHit StereoHit;
...
...
fatal error: too many errors emitted, stopping now [-ferror-limit=]
History
#1 Updated by Kyle Knoepfel about 1 year ago
Paul Russo wrote:
Oh,
This is ROOT behavior, check the definition of your ROOT_INCLUDE_PATH environment variable.
You are picking up this header file through it:
./RecoDataProducts/inc/StereoHit.hh:5:20: error: typedef redefinition with different types ('mu2e::ComboHit' vs 'mu2e::StereoHit')
typedef ComboHit StereoHit;
#2 Updated by Kyle Knoepfel about 1 year ago
Rob then wrote:
Thanks Paul,
Can you explain a bit more about what is going on? Does the action happen in the JIT? At dictionary-load-time? At dictionary-use-time?
I take it that the problem is that the jit can’t find the header file because it has been moved?
Or am I way out in left field? Waveland Avenue? Wisconsin?
Rob
#3 Updated by Kyle Knoepfel about 1 year ago
- Status changed from New to Feedback
- Assignee set to Paul Russo
#4 Updated by Andrei Gaponenko about 1 year ago
I checked on Paul's suggestion. The offending file can not be found via ROOT_INCLUDE_PATH. The value of that variable is shown below. Note that it does not include the /mu2e/app/users/gandr/20180406-dict-breakage directory or any of its parents. I also checked that there is no '/mu2e/app' anywhere in the environment (besides the PWD), and no dot anywhere.
Andrei
20180406-dict-breakage$ echo $ROOT_INCLUDE_PATH | tr ':' '\n' /cvmfs/mu2e.opensciencegrid.org/Offline/v6_5_2/SLF6/prof/Offline /cvmfs/mu2e.opensciencegrid.org/artexternals/cry/v1_7i/Linux64bit+2.6-2.12-e15-prof/cry_v1.7/include /cvmfs/mu2e.opensciencegrid.org/artexternals/mu2e_artdaq_core/v1_02_01e/include /cvmfs/mu2e.opensciencegrid.org/artexternals/art/v2_10_02/include /cvmfs/mu2e.opensciencegrid.org/artexternals/fhiclcpp/v4_06_05/include /cvmfs/mu2e.opensciencegrid.org/artexternals/cetlib/v3_02_00/slf6.x86_64.e15.prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/boost/v1_66_0/Linux64bit+2.6-2.12-e15-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/artdaq_core/v3_01_05/include /cvmfs/mu2e.opensciencegrid.org/artexternals/artdaq_core/v3_01_05/include /cvmfs/mu2e.opensciencegrid.org/artexternals/TRACE/v3_13_04/include /cvmfs/mu2e.opensciencegrid.org/artexternals/art/v2_10_02/include /cvmfs/mu2e.opensciencegrid.org/artexternals/canvas_root_io/v1_01_02/include /cvmfs/mu2e.opensciencegrid.org/artexternals/xrootd/v4_8_0a/Linux64bit+2.6-2.12-e15-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/mysql_client/v5_5_58/Linux64bit+2.6-2.12-e15/include /cvmfs/mu2e.opensciencegrid.org/artexternals/postgresql/v9_6_6a/Linux64bit+2.6-2.12-p2714b/include /cvmfs/mu2e.opensciencegrid.org/artexternals/pythia/v6_4_28j/Linux64bit+2.6-2.12-gcc640-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/gsl/v2_4/Linux64bit+2.6-2.12-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/fftw/v3_3_6_pl2/Linux64bit+2.6-2.12-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/canvas/v3_02_02/include /cvmfs/mu2e.opensciencegrid.org/artexternals/range/v3_0_3_0/include /cvmfs/mu2e.opensciencegrid.org/artexternals/clhep/v2_3_4_5c/Linux64bit+2.6-2.12-e15-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/messagefacility/v2_01_06/include /cvmfs/mu2e.opensciencegrid.org/artexternals/tbb/v2018_2/Linux64bit+2.6-2.12-e15-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/fhiclcpp/v4_06_05/include /cvmfs/mu2e.opensciencegrid.org/artexternals/cetlib/v3_02_00/slf6.x86_64.e15.prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/boost/v1_66_0/Linux64bit+2.6-2.12-e15-prof/include /cvmfs/mu2e.opensciencegrid.org/artexternals/cetlib_except/v1_01_06/include
#5 Updated by Paul Russo about 1 year ago
This is an example of auto-parse behavior. We should re-examine the question of whether or not this should be disabled in ROOT at art startup, and configurable by fhicl.
#6 Updated by Rob Kutschke about 1 year ago
Hi Paul,
Can you explain "auto-parse behaviour".Thanks,Rob#7 Updated by Kyle Knoepfel about 1 year ago
- Tracker changed from Bug to Support
#8 Updated by Kyle Knoepfel about 1 year ago
Based on discussions with Rob and Andrei, the next step is to experiment with disabling ROOT's auto-parsing behavior. This must be one before opening the first input file: disabling auto-parsing at module or service construction would take effect before the first input file is opened.
We are willing to experiment with this, but your testing of disabling auto-parsing would be more elucidating for your use case.
#9 Updated by Raymond Culbertson about 1 year ago
You probably already discussed this with Rob and Andrei, but could
reply with a summary of what root is trying to achieve with auto-parsing
and what functionality might be lost now and in future versions of root
if it is disabled. Thanks