ART does not seem to handle object evolution well
I added a data member called fTrackID to our sim::Electrons object in LArSoft yesterday and a user attempting to read in a file made a week ago reports this error:
%MSG-s ArtException: PostOpenFile 26-Apr-2011 09:02:48 CDT BeforeEvents
cet::exception caught in art
---- FatalRootError BEGIN
Fatal Root Error: @SUB=TTreeCloner::CollectBranches
One of the export sub-branches (sim::Electronss_driftel__GenieGen.obj.fTrackID) is not present in the import TTree.
cet::exception caught in EventProcessor and rethrown
---- FatalRootError END
I would have expected it to gracefully accept the new data member and fill it with the default value in the constructor rather than complaining.
I understand that we can't really go the other way, removing data members and expecting it to work, but adding one shouldn't be an issue. ROOT seems to handle the addition of data members OK. The problem is that if we can't handle the addition of a data member, then every time we decide an object needs a new data member we will end up remaking files. Ultimately that should happen, but in the transition period it would be nice to be able to read in both sets of files.
#1 Updated by Christopher Green over 8 years ago
- Category set to Navigation
- Status changed from New to Resolved
- Assignee set to Christopher Green
- % Done changed from 0 to 100
We've discussed this on chat, but I wanted to get it down somewhere permanent.
- Schema evolution is the purview of the product class-writer, not ART.
- The immediate cause of the error is that you are doing fast cloning, so ART is not able to update the object with the new member because it is never actually read in. You can solve your immediate problem either by turning off fast cloning in the output, or by dropping the old branches, respectively:
fastCloning: false; or
outputCommands: [ "keep *", "drop sim::Electronss_driftel_*_GenieGen" ]
You may wish to expand the wildcarding depending on how many branches you need to drop.
- If it was a new object (default constructed) prior to being reconstructed, then the member will be unchanged and therefore be default constructed (if you initialized it properly in the constructor) or uninitialized (and therefore contain whatever dirty bits were there before).
- If you are re-using the object, then the new member will contain whatever was there before. This can be handled in formal schema evolution, but what I just described is what happens if you don't deal with it explicitly.
You can specify a particular value for defaulted members by putting instructions into the
classes_def.xml file. I will put an explanation and instructions for schema evolution using modern versions of ROOT onto the ART Wiki tomorrow.
#2 Updated by Christopher Green over 8 years ago
- Target version set to 0.07.00
Please see the ART Wiki article Facilitating Schema Evolution for Data Products for a full treatment of schema evolution and management in the context of ART. In addition, the next version of ART will provide a script source:tools/checkClassVersion that will verify the consistency of the
classes_def.xml file with respect to the class definition. Please read this article and comment or ask questions as you wish.