Bug #21316

v2_11 doesn't fix provenance from v2_10

Added by Raymond Culbertson almost 2 years ago. Updated over 1 year ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
4.00 h
Occurs In:
SSI Package:


In this issue
a provenance problem was found in art v2_10 and fixed in v2_11.
Now we are testing art v2_11_01 and still see a form
of the problem. However we are not sure if the fact that
the input files were created in v2_10 means that they will
always be corrupt, or they should be read cleanly by v2_11.
Below is the how to reproduce the current error. Is this
expected? Is there a way to repair files or suppress the error?

In process 1:
source /cvmfs/
source /cvmfs/
(sets up art v2_10_04)
export IN=/pnfs/mu2e/tape/phy-sim/dig/mu2e/CeEndpoint/MDC2018b/art/13/a3/
mu2e -n 10 -c JobConfig/common/artcat.fcl -s $IN -o

In process 2
source /cvmfs/
source /cvmfs/
(sets up art v2_11_01)
mu2e -s -c Print/fcl/print.fcl
produces error: Product reported as not present, but is pointed to nonetheless!

Note that if the steps in process 1 are done in process2, creating, the result will be read cleanly by v2_11. If
is read and written by v2_11, then the provenance is read in v2_11, it
produces the error, so v2_11 isn't repairing the bad file. Is this expected?

The releases can be created locally by
git clone
git checkout v7_1_2 (or v7_1_2a)
source /cvmfs/
source Offline/ (instead of cvmfs)
scons -j 20


#1 Updated by Kyle Knoepfel almost 2 years ago

  • Status changed from New to Feedback

Please update to art 2.11.03 to determine if there is still a problem. Issue #19809 addresses a similar issue that was resolved for art 2.11.02.

#2 Updated by Kyle Knoepfel over 1 year ago

From Ray:

This problem as described in the original report still persists when use art v2_11_04 as the "process 2" art version.

#3 Updated by Kyle Knoepfel over 1 year ago

  • Status changed from Feedback to Assigned
  • Assignee set to Kyle Knoepfel


#4 Updated by Kyle Knoepfel over 1 year ago

  • Occurs In 2.11.04 added
  • Occurs In deleted (2.11.01)

#5 Updated by Kyle Knoepfel over 1 year ago

  • Category set to Infrastructure
  • Status changed from Assigned to Feedback
  • % Done changed from 0 to 100
  • Estimated time set to 4.00 h

The problem is understood. Although the changes made for art 2.11.02 addressed the persistency issues when using the RootOutput module, the workflow here uses a ProvenanceDumper-based module, which is independent of RootOutput and was not updated for art 2.11.02. In addition there were some mistaken assumptions regarding how art calculates the availability of the product.

The following commits resolve these problems:

Does Mu2e request a bug-fix release (e.g. 2.11.05), or shall we just include the fix for the next planned release of art, which is version 3.02.00? I suspect the former.

#6 Updated by Kyle Knoepfel over 1 year ago

  • Target version set to 2.11.05

#7 Updated by Kyle Knoepfel over 1 year ago

  • Status changed from Feedback to Resolved

Per discussion with Rob, the fix for this bug will be included with art 2.11.05.

#8 Updated by Kyle Knoepfel over 1 year ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF