file merge showstopper
A homogeneous dataset of art files was generated starting with the
EmptyEvent source and using fcl configs that differ only in random
number seeds and subrun numbers. The next stage of processing handles
individual output files fine. However an attempt to use multiple
first stage files in a single second stage jobs leads to
%MSG-s ArtException: PostCloseFile 01-Apr-2015 17:56:03 CDT PostEndRun cet::exception caught in art ---- UnimplementedFeature BEGIN Cannot merge file '/mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00006/data_g4s1_mubeam.art' due to a branch mismatch. Contact the framework group. cet::exception caught in EventProcessor and rethrown ---- UnimplementedFeature END %MSG
This is a showstopper for Mu2e production. Note that some files can
be merged, but others can not.
- build Mu2e Offline git ce7a550b6fd5ff52e1ad3f6842a20a907f5d859e
- verify that you can analyze individual files:
mu2e -c JobConfig/cd3/beam/beam_g4s2.fcl /mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00005/data_g4s1_mubeam.art
mu2e -c JobConfig/cd3/beam/beam_g4s2.fcl /mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00006/data_g4s1_mubeam.art
- Try to process both of the above files in a single job and observe
The fcl configuration used to run the first stage and the log files
can be found in the same directories as the data files.
#1 Updated by Andrei Gaponenko about 5 years ago
An update: Kyle provided a critical tool to identify the discrepancy. The difference between the files is due to one job making an extra produces<...>() call compared to the other. That produces<>() call is conditional and the condition depends on an uninitialized variable in Mu2e Offline code. I still have to test a fix, but it looks like the problem is not in the framework but in the user code.
#2 Updated by Kyle Knoepfel about 5 years ago
- Tracker changed from Bug to Support
- Status changed from New to Feedback
- Assignee set to Kyle Knoepfel
I'm glad we were able to diagnose the likely culprit. I have left this issue in "feedback" because I believe the debug printout (where the branch name and brand ID is printed out for each
produces call) should be provided/providable either via a
messagefacility logging call or with a dedicated
ProductTracker service. I will discuss this with the art team, and we will determine the proper approach, at which point I'll close the issue.