Question about fastCloning
We have a question about the fastCloning option in the outputs configuration. Normally if we are reading a file that has a product with older format, fastCloning will be turned off automatic. But we have encountered the following problem.
ssh dunegpvm01 source /grid/fermiapp/products/dune/setup_dune.sh setup dunetpc v06_24_00 -q e10:prof lar -o test.root noana.fcl single_dunefd_gen_reco.root #both noana.fcl and single_dunefd_gen_reco.root are attached to this ticket.
It gives the warning message:
%MSG-i fastCloneTree: PostProcessPath end_path 15-Feb-2017 12:42:39 CST PostProcessEvent INFO: Unable to fast clone tree Events INFO: ROOT reason is: INFO: One of the export sub-branches (recob::Tracks_pmtrajfit__Reco.obj.fCovEnd.fRep.fArray) is not present in the import TTree. INFO: Processing will continue, tree will be slow cloned. %MSG
noana.fcl does not do anything but copying the input file to an output file. It seems to run fine and produce an output file test.root.
However, the output file seems to be corrupted:
lar -c eventdump.fcl test.root %MSG-i MF_INIT_OK: lar 15-Feb-2017 12:45:45 CST JobSetup Messagelogger initialization complete. %MSG 15-Feb-2017 12:45:47 CST Initiating request to open input file "test.root" 15-Feb-2017 12:45:48 CST Opened input file "test.root" Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 15-Feb-2017 12:45:50 CST PRINCIPAL TYPE: Run PROCESS NAME | MODULE_LABEL | PRODUCT INSTANCE NAME | DATA PRODUCT TYPE | SIZE SinglesGen.. | generator... | ..................... | sumdata::RunData. | ...- Total products (present, not present): 1 (1, 0). %MSG-s ArtException: PostPathEndRun end_path 15-Feb-2017 12:45:50 CST PostEndRun cet::exception caught in art ---- LogicError BEGIN Principal::getForOutput A product with a status of 'present' is not actually present. The branch name is art::TriggerResults_TriggerResults__Reco. Contact a framework developer. cet::exception caught in EventProcessor and rethrown ---- LogicError END %MSG Art has completed and will exit with status 65.
If we uncomment 'fastCloning: false' in noana.fcl, everything seems fine.
If we run noana.fcl without 'fastCloning: false' on a different file that was generated with the same old format:
lar -o test1.root -c noana.fcl /pnfs/dune/scratch/dunepro/v06_18_00/reco2/prodgenie_nu_dune10kt_1x2x6/12774876_0/prodgenie_nu_dune10kt_1x2x6_667_20161210T015117_detsim2_9ac488e2-6d45-4a85-bfac-41eb457db664_reco1_reco2.root -n 1 We see the following: %MSG-i FastCloning: RootOutput:out1@Construction 15-Feb-2017 12:52:49 CST ModuleConstruction Initial fast cloning configuration (from default): true %MSG Begin processing the 1st record. run: 20000001 subRun: 668 event: 66701 at 15-Feb-2017 12:52:49 CST 15-Feb-2017 12:52:49 CST Opened output file with pattern "test1.root" %MSG-w FastCloning: PostProcessPath end_path 15-Feb-2017 12:52:49 CST PostProcessEvent Fast cloning deactivated for this input file due to information in FileBlock. %MSG
and the output file was fine. There seems to be a difference in the fastCloning behaviour.
Here is additional information on how the input file was created:
source /grid/fermiapp/products/dune/setup_dune.sh setup dunetpc v06_22_00 -q e10:prof lar -n 1 -c prodsingle_dunefd.fcl lar -c standard_reco_dune10kt.fcl single_dunefd_gen.root lar -c eventdump.fcl single_dunefd_gen_reco.root
Thanks for any insights.
Tom & Tingjun
#4 Updated by Kyle Knoepfel over 2 years ago
- Category set to I/O
- Status changed from Accepted to Feedback
- Assignee set to Kyle Knoepfel
- % Done changed from 0 to 100
- SSI Package art added
The problem is understood. In art 2.03.00, the fast-cloning rules were relaxed so that if fast-cloning failed, then slow-cloning would ensue without
art throwing an exception. However, the change from fast-cloning to slow-cloning was not recorded properly by the
RootOutput system. The result was that in cases where fast-cloning failed when it was originally requested, the
Event-level branches were filled with the presence bit set to
This error has been fixed with the following commit art:3060c880 and has been included on the HEAD of
develop branch. We would like to know if it would be necessary to back-port this change to an
art 2.05-series bug-fix release, or if it is sufficient to create an
art 2.06.02 release with this fix included.