Bug #15561

Question about fastCloning

Added by Tingjun Yang about 4 years ago. Updated about 4 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
Spent time:
Occurs In:
SSI Package:


Dear Artists,

We have a question about the fastCloning option in the outputs configuration. Normally if we are reading a file that has a product with older format, fastCloning will be turned off automatic. But we have encountered the following problem.

ssh dunegpvm01
source /grid/fermiapp/products/dune/
setup dunetpc v06_24_00 -q e10:prof
lar -o test.root noana.fcl single_dunefd_gen_reco.root    #both noana.fcl and single_dunefd_gen_reco.root are attached to this ticket.

It gives the warning message:

%MSG-i fastCloneTree:  PostProcessPath end_path 15-Feb-2017 12:42:39 CST  PostProcessEvent
INFO: Unable to fast clone tree Events
INFO: ROOT reason is:
INFO: One of the export sub-branches (recob::Tracks_pmtrajfit__Reco.obj.fCovEnd.fRep.fArray[15]) is not present in the import TTree.
INFO: Processing will continue, tree will be slow cloned.

noana.fcl does not do anything but copying the input file to an output file. It seems to run fine and produce an output file test.root.

However, the output file seems to be corrupted:

lar -c eventdump.fcl test.root
%MSG-i MF_INIT_OK:  lar 15-Feb-2017 12:45:45 CST JobSetup
Messagelogger initialization complete.
15-Feb-2017 12:45:47 CST  Initiating request to open input file "test.root" 
15-Feb-2017 12:45:48 CST  Opened input file "test.root" 
Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 15-Feb-2017 12:45:50 CST
SinglesGen.. | generator... | ..................... | sumdata::RunData. | ...-

Total products (present, not present): 1 (1, 0).

%MSG-s ArtException:  PostPathEndRun end_path 15-Feb-2017 12:45:50 CST  PostEndRun
cet::exception caught in art
---- LogicError BEGIN
  A product with a status of 'present' is not actually present.
  The branch name is art::TriggerResults_TriggerResults__Reco.
  Contact a framework developer.
  cet::exception caught in EventProcessor and rethrown
---- LogicError END
Art has completed and will exit with status 65.

If we uncomment 'fastCloning: false' in noana.fcl, everything seems fine.

If we run noana.fcl without 'fastCloning: false' on a different file that was generated with the same old format:

lar -o test1.root -c noana.fcl /pnfs/dune/scratch/dunepro/v06_18_00/reco2/prodgenie_nu_dune10kt_1x2x6/12774876_0/prodgenie_nu_dune10kt_1x2x6_667_20161210T015117_detsim2_9ac488e2-6d45-4a85-bfac-41eb457db664_reco1_reco2.root -n 1
We see the following:
%MSG-i FastCloning:  RootOutput:out1@Construction 15-Feb-2017 12:52:49 CST  ModuleConstruction
Initial fast cloning configuration (from default): true
Begin processing the 1st record. run: 20000001 subRun: 668 event: 66701 at 15-Feb-2017 12:52:49 CST
15-Feb-2017 12:52:49 CST  Opened output file with pattern "test1.root" 
%MSG-w FastCloning:  PostProcessPath end_path 15-Feb-2017 12:52:49 CST  PostProcessEvent
Fast cloning deactivated for this input file due to information in FileBlock.

and the output file was fine. There seems to be a difference in the fastCloning behaviour.

Here is additional information on how the input file was created:

source /grid/fermiapp/products/dune/
setup dunetpc v06_22_00 -q e10:prof
lar -n 1 -c prodsingle_dunefd.fcl
lar -c standard_reco_dune10kt.fcl single_dunefd_gen.root
lar -c eventdump.fcl single_dunefd_gen_reco.root

Thanks for any insights.

Tom & Tingjun

noana.fcl (385 Bytes) noana.fcl Tingjun Yang, 02/15/2017 12:54 PM
single_dunefd_gen_reco.root (859 KB) single_dunefd_gen_reco.root Tingjun Yang, 02/15/2017 12:54 PM


#1 Updated by Kyle Knoepfel about 4 years ago

  • Description updated (diff)

#2 Updated by Kyle Knoepfel about 4 years ago

  • Tracker changed from Support to Bug
  • Status changed from New to Accepted

At first glance, the input file does not appear to be corrupt. We will investigate further.

#3 Updated by Kyle Knoepfel about 4 years ago

The behavior you observe has been reproduced with the files you provided.

#4 Updated by Kyle Knoepfel about 4 years ago

  • Category set to I/O
  • Status changed from Accepted to Feedback
  • Assignee set to Kyle Knoepfel
  • % Done changed from 0 to 100
  • SSI Package art added

The problem is understood. In art 2.03.00, the fast-cloning rules were relaxed so that if fast-cloning failed, then slow-cloning would ensue without art throwing an exception. However, the change from fast-cloning to slow-cloning was not recorded properly by the RootOutput system. The result was that in cases where fast-cloning failed when it was originally requested, the Event-level branches were filled with the presence bit set to false.

This error has been fixed with the following commit art:3060c880 and has been included on the HEAD of art's develop branch. We would like to know if it would be necessary to back-port this change to an art 2.05-series bug-fix release, or if it is sufficient to create an art 2.06.02 release with this fix included.

#5 Updated by Kyle Knoepfel about 4 years ago

Gianluca, Erica, your opinions would be appreciated regarding a new release of art that incorporates this bug fix. See previous message on this issue.

#6 Updated by Lynn Garren about 4 years ago

Please note the separate request for a new build of fftw at #15597. This will also need to be folded into the decision.

#7 Updated by Tingjun Yang about 4 years ago

Thanks for solving the problem!

We are happy to wait for the next tagged art release as the problem only happens when mixing releases with different data products.

#8 Updated by Kyle Knoepfel about 4 years ago

  • Status changed from Feedback to Resolved

Per stakeholder discussion, a bug-fix release of 2.05 is not necessary at this time. A bug-fix release of 2.06 will be released, but it is not urgent.

#9 Updated by Kyle Knoepfel about 4 years ago

  • Status changed from Resolved to Closed
  • Target version set to 2.06.02

Also available in: Atom PDF