Project

General

Profile

Idea #15169

Support splitting data products by schema evolution

Added by Gianluca Petrillo over 2 years ago. Updated over 2 years ago.

Status:
Accepted
Priority:
Normal
Assignee:
-
Category:
I/O
Target version:
-
Start date:
01/17/2017
Due date:
% Done:

0%

Estimated time:
Experiment:
-
SSI Package:
art
Duration:

Description

The request is for a chance to keep backward compatibility with data when splitting a data product in parts.
I will try to describe this with terms everybody can digest.

The first scenario is where I have a product "bakery" with std::vector<ApplePie> of ApplePie objects version 10. In my new version, I decide to take out some component that does not really belong to ApplePie, and I aim to have two data products: std::vector<ApplePie> version 11 and std::vector<Cinnamon> (say also version 11 for clarity). With the new code I expect art to give me a handle to std::vector<ApplePie> when I ask event.getByLabel<std::vector<ApplePie>>("bakery"), and similarly for std::vector<Cinnamon> in case I really needed that, reading it from the stored std::vector<ApplePie>.

A second, derivative scenario is where I have a product "keepgoing" of type std::vector<CoffeeAndCream>, and I decide to redesign it into two separate data products std::vector<Coffee> and std::vector<Cream>. I want art to give me std::vector<Coffee> when I ask event.getByLabel<std::vector<Coffee>>("keepgoing"), and similarly for std::vector<Cream>, and to throw an exception (ProductNotFound-like) when asking event.getByLabel<std::vector<CoffeeAndCream>>("keepgoing").

History

#1 Updated by Gianluca Petrillo over 2 years ago

I have discussed with Philippe Canal of this, except that we were not including the std::vector wrapper.

First of all, ROOT schema evolution in principle does allow that use case. In fact, the syntax for a I/O rule (IOREAD) even allows for a source class different from the target one.

Philippe explained me, and I try to report it right, that, given a tree with the old object, something like:

ApplePie* pApplePie = nullptr;
tree->SetBranchAddress("ApplePies_bakery__bla", &pApplePie);
tree->GetEntry(0);
// ... and at a different getByLabel()...
Cinnamon* pCinnamon = nullptr;
tree->SetBranchAddress("ApplePies_bakery__bla", &pCinnamon);
tree->GetEntry(0);
should summon the right magic. Details might need to be fixed.

The next question is how to let art know that when I ask for std::vector<Cinnamon> I really mean to read it from std::vector<ApplePie> (when I ask for std::vector<ApplePie>, art already knows).
That information is known by the person who is also writing the I/O rules to perform the conversion. It is possible to add attributes to the XML section describing a class, something along the line of

<class name="Cinnamon" ClassVersion="11" source="ApplePie:11;" >
  <version ClassVersion="11" checksum="4256037352"/>
</class>
where both the name source and the syntax ApplePie:11; have just been made up.
The framework might follow a convention where after not finding a branch with std::vector<Cinnamon>, it would follow the cue from the attribute and look for a branch with std::vector<ApplePie> (here the std::vector wrapping is indeed a complication). The cue can be extracted from the dictionary information. Philippe seemed to think that that attribute can't be tied to the <version> tag.

#2 Updated by Marc Paterno over 2 years ago

  • Tracker changed from Feature to Idea

Tracker changed from feature request to idea, because of the complexity of the issue.

#3 Updated by Marc Paterno over 2 years ago

  • Description updated (diff)

#4 Updated by Kyle Knoepfel over 2 years ago

  • Status changed from New to Accepted


Also available in: Atom PDF