Project

General

Profile

Bug #16192

Provenance is not retained when parent product is only accessed via a Ptr

Added by Rob Kutschke almost 3 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
04/13/2017
Due date:
% Done:

100%

Estimated time:
Spent time:
Occurs In:
Scope:
Internal
Experiment:
Mu2e
SSI Package:
art
Duration:

Description

I have some code that reads in two data products via event mixing: GenParticles and SimParticles.

A downstream producer module does an event.get on the SimParticles and produces a data product.
There are a few more modules in the job. In the end we drop all data products except the last
one in the chain. The provenance for the SimParticles is correctly written to the output file - it
is indeed part of the ancestry of the product that is written out.

Here is the problem. SimParticles that represent primary particles have Ptrs that point to
the corresponding GenParticle (For secondary SImParticles this Ptr is null). In the module
discussed above, the code follows the Ptr to a GenParticle and uses information found in the
GenParticle. I had expected that art would recognize from this that GenParticles are
part of the ancestry of the newly produced data product - but it does not. So the provenance
for the GenParticles does not appear in the output file.

I have a working example on cluck but it's not a compact example. When I have some
more time I can explain how to run it.

We are trying to isolate another problem and we are not sure if this is related.
If it's not related this is a low priority request but it might well be related.

History

#1 Updated by Kyle Knoepfel almost 3 years ago

  • Status changed from New to Accepted

We would like to schedule a meeting to discuss this issue in more detail.

#2 Updated by Kyle Knoepfel almost 3 years ago

We have discussed this issue at length. At this point, we would need a sample job that demonstrates the behavior you are running into to diagnose further.

#3 Updated by Kyle Knoepfel almost 3 years ago

  • Status changed from Accepted to Feedback

We will visit you to discuss more details.

#4 Updated by Raymond Culbertson over 2 years ago

Here is, I believe, a reproducible example
on mu2egpvm01
source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
source /cvmfs/mu2e.opensciencegrid.org/Offline/v6_3_2/SLF6/prof/Offline/setup.sh
mu2e -c Print/fcl/dumpDataProducts.fcl -s /mu2e/app/users/rlc/cosm/garageRun/sim.owner.cosmic-g4s2-garage.version.sequencer.art
Segmentation fault

#5 Updated by Raymond Culbertson over 2 years ago

Please use the file here, it will be stable, the other will not..
mu2e -c Print/fcl/print.fcl -s /mu2e/data/users/rlc/sim.owner.cosmic-g4s2-garage.version.sequencer.art

#6 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from Feedback to Assigned
  • Assignee set to Kyle Knoepfel

The original issue observes that parentage is not modified due to an art::Ptr de-reference. That is intended behavior, and there is not a proposal to adjust it.

Is the segmentation-violation report related to this behavior?

#7 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from Assigned to Feedback

#8 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from Feedback to Assigned

The segmentation violation is confirmed. Unfortunately, reconstructing the problem within the art test suite is difficult. For now, a way of avoiding the segmentation violation is to specify:

dumper: {
  module_type: DataProductDump
  wantResolvedOnly: true
}

There does seem to be an issue with provenance persistency in the case of art::Ptrs, but constructing the specific set of circumstances is delicate.

#9 Updated by Kyle Knoepfel over 2 years ago

Ray, we need to be able to run each of the stages to diagnose further. A couple requests:

  • Can you provide a pared-down example that demonstrates the same segmentation violation?
  • Can you provide us access to all of the files we would need (including conditions *.txt files) to reproduce this?

It is fairly trivial to fix the code where the segmentation violation is occurring. However, I am concerned that it may be a symptom of a more systemic problem.

#10 Updated by Kyle Knoepfel over 2 years ago

Please disregard the previous note--I have been able to reproduce the error within the art test suite.

#11 Updated by Raymond Culbertson over 2 years ago

I can confirm that the suggested work-around does work for us.

#12 Updated by Kyle Knoepfel over 2 years ago

A fix within the art framework is in place--commit art:366943c8. I will test it on the specific Mu2e example given above. Once the fix is released with a new version of art, the workaround should no longer be necessary.

#13 Updated by Kyle Knoepfel over 2 years ago

  • % Done changed from 0 to 70

#14 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 70 to 100

An additional commit has been made that handles backwards compatibility (art:f99207c). I have confirmed that this commit fixes the segmentation violation. Although this fix avoids the error, it cannot determine what the provenance should be for the dropped product in question. That is a problem that has been resolved by the previous commit, allowing correct provenance reading for any files written in the future.

I will mark this issue has resolved since the example that demonstrates the failure can run successfully with the fixed art. However, there may be issues related to art::Ptr provenance that the fix does not address. I propose that as those issues arise, a new bug report/feature request be logged.

Do you prefer a 2.09-series bug-fix release that incorporates the above fix? Or is it sufficient to wait for a 2.10 release, using the aforementioned workaround until then?

#15 Updated by Raymond Culbertson over 2 years ago

We can't justify an urgent patch - please proceed at your convenience. It was only causing a minor inconvenience (as far as we detected) and the work-around even removed that. Thanks very much

#16 Updated by Kyle Knoepfel over 2 years ago

  • Category set to Infrastructure
  • Target version set to 2.10.00
  • Occurs In 2.07.03 added
  • SSI Package art added

#17 Updated by Kyle Knoepfel about 2 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF