Feature #5182
Need to be able to merge two currently unmerge-able files.
Description
Hello,
I get
cet::exception caught in art ---- UnimplementedFeature BEGIN Cannot merge file '25.root' due to a branch mismatch. Contact the framework group. cet::exception caught in EventProcessor and rethrown ---- UnimplementedFeature END
when trying to merge files. The data products that are retained
have the same types/module labels/instance names/process names.
The exception above is thrown even after all metadata are supposed
to be dropped.
A simple way to reproduce is to use the attached fcl file and do
mu2e -c rewrite.fcl /mu2e/data/tdr/beam/g4s3p2/tdr.beam.g4s3.ds23.1009a_1025a_1223a.13872813/good/00000/beamFlashOnMothers.root mv rewrite.root 09.root mu2e -c rewrite.fcl /mu2e/data/tdr/beam/g4s3p2/tdr.beam.g4s3.ds23.1025a_1025a_1223a.14019207/good/00000/beamFlashOnMothers.root mv rewrite.root 25.root mu2e -c rewrite.fcl 09.root 25.root
The last command fails with the exception above.
Andrei
Associated revisions
History
#1 Updated by Christopher Green about 7 years ago
- Category set to I/O
- Status changed from New to Accepted
- Target version set to 493
- Estimated time set to 16.00 h
- SSI Package art added
- SSI Package deleted (
)
Time estimate is for a preliminary analysis only. A better estimate of the time to fix the problem is not possible at this time.
#2 Updated by Christopher Green almost 7 years ago
- Status changed from Accepted to Assigned
- Assignee set to Christopher Green
I have started to investigate this issue, and have succeeded in building the Mu2e software against a private (and therefore changeable) version of art in debug mode, and duplicating the issue. Tuesday afternoon I hope to be able to investigate exactly what is happening.
#3 Updated by Christopher Green almost 7 years ago
- % Done changed from 0 to 20
- Estimated time changed from 16.00 h to 40.00 h
This problem is now understood. In order to continue to detect files which really are incompatible, we must be very careful in proceeding with a fix, and several new tests must be constructed carefully in order to avoid the introduction of new problems.
Basically, "Drop metadata for dropped products" does not deal with the list of lists of BranchID
for each process, which is the basis for assignment of ProductID
for Ptr
in products. In order to remedy this issue, we must carefully check that any product depending on a product with dropped metadata is also dropped in order to be sure that there can be no inconsistencies with references to missing or incorrect products in a particular event.
#4 Updated by Christopher Green almost 7 years ago
- Tracker changed from Bug to Feature
- Subject changed from file merge UnimplementedFeature to Need to be able to merge two currently unmerge-able files.
- Status changed from Assigned to Accepted
- Assignee deleted (
Christopher Green) - Priority changed from High to Normal
Tests for current inconsistency checking implemented with 9febe38.
Unfortunately, as we discussed, the aspects of your two files that make them unmerge-able really do make them unmerge-able, in that the histories of the single product in the two files are different, rendering safe use of Ptr
in the current scheme impossible. I understand you are not using Ptr
here, but we are not in a position to detect the lack of use and relax the rules within the current scheme for Ptr
.
Implementation of this feature will have to wait for the transition and rationalization of the metadata to SQLite. Please contact us if you need help with a workaround in the meantime.
#5 Updated by Christopher Green almost 7 years ago
- Target version changed from 493 to 1.09.00
#6 Updated by Christopher Green almost 7 years ago
- Target version changed from 1.09.00 to 521
#7 Updated by Rob Kutschke almost 7 years ago
Can we have a clear description of the rules that must be followed in order for files to be merge-able. I am looking for an explanation pitched at an expert or a near-expert.
#8 Updated by Kyle Knoepfel over 3 years ago
- Status changed from Accepted to Resolved
- Assignee set to Kyle Knoepfel
- Target version deleted (
521) - % Done changed from 20 to 100
#9 Updated by Kyle Knoepfel over 3 years ago
- Target version set to 2.08.00
#10 Updated by Kyle Knoepfel over 3 years ago
- Status changed from Resolved to Closed
Event-shape tests for issue #5182, and some splitting of CMakeLists.txt