Project

General

Profile

Feature #16546

Find all products by type

Added by Nathaniel Tagg almost 2 years ago. Updated 9 months ago.

Status:
Closed
Priority:
Low
Assignee:
Target version:
Start date:
05/15/2017
Due date:
% Done:

100%

Estimated time:
Spent time:
Scope:
Internal
Experiment:
MicroBooNE
SSI Package:
gallery
Duration:

Description

This isn't exactly needed, since I can hack it from the user level, but it might be useful to others.

I want to be able to get all data products of type recob::Wire from the current file.

Here's how I can hack it with custom code:

template<typename T>
vector<std::pair<string,art::InputTag>>  findByType(TTree* fTree)
{
  vector<std::pair<string,art::InputTag>> retval;
  // Leaf names for std::vector<recob::Wire> get renamed
  // to the art version of "recob::Wires" by this. 
  std::string pattern = art::TypeID(typeid(T)).friendlyClassName();
  std::cout << "Looking for leaf of type " << pattern << "_label_instance_process." << endl;
  TObjArray* list = fTree->GetListOfBranches();

  // Look through every branch name.

  // We're looking for:
  // OBJ_LABEL_INSTANCE_PROCESSNAME.obj_
  // Where OBJ is something like "recob::Wires" 
  // Where LABEL is something like "pandora" 
  // INSTANCE is usually (always?) blank
  // Where PROCESSNAME is something like "McRecoAprStage1" 

  for(int i=0;i<list->GetEntriesFast();i++) {
    TObject* o = list->At(i);
    TBranch* br = dynamic_cast<TBranch*>(o);
    std::string found = br->GetName();

    // Does it start with our object type?
    if(found.find(pattern)!=0) continue;
    // std::cout << "--Candidate: " << found << std::endl;

    // Does it end with '.'?
    string::size_type p3 = found.find_last_of('.'); 
    if(p3!=found.length()-1 || p3==0) continue;
    // std::cout << "p3 " << found.substr(p3) << std::endl;

    // Tokenize by underscore.
    string::size_type p2 = found.rfind("_",p3-1);
    if(p2==string::npos || p2==0) continue;
    // std::cout << "p2 " << found.substr(p2) << std::endl;

    string::size_type p1 = found.rfind("_",p2-1);
    if(p1==string::npos || p1 ==0 ) continue;
    // std::cout << "p1 " << found.substr(p1) << std::endl;

    string::size_type p0 = found.rfind("_",p1-1);
    if(p0==string::npos || p0==0) continue;
    // std::cout << "p0 " << found.substr(p0) << std::endl;

    art::InputTag tag(found.substr(p0+1,p1-p0-1),  found.substr(p1+1,p2-p1-1), found.substr(p2+1,p3-p2-1));
    retval.push_back( make_pair( found, tag ));
  }
  return retval;

}

This code simply parses the Event tree branch list to find matching objects, and parses out the label/instance/process handles to create art::InputTags as a list. Then the calling code can do something like the following:

typedef vector<recob::Wire> thing_t;
for(auto item: findByType<thing_t>(ev.getTTree())) {
  cout<< "Branch: " << item.first << std::endl;
  cout<< "Tag   : " << item.second << std::endl;
  gallery::Handle< thing_t > handle;
  ev.getByLabel(item.second,handle);
  if(handle.isValid())
    std::cout << "Found " << handle->size() << " wires." << std::endl;      
}

Looking at the ART source code, it would appear that this sort of parsing should be robust, since underscores are otherwise not allowed in InputTag elements.

History

#1 Updated by Nathaniel Tagg almost 2 years ago

Sorry, here's the top code in easier-to-read fashion:

template<typename T>
vector<std::pair<string,art::InputTag>>  findByType(TTree* fTree)
{
  vector<std::pair<string,art::InputTag>> retval;
  // Leaf names for std::vector<recob::Wire> get renamed
  // to the art version of "recob::Wires" by this. 
  std::string pattern = art::TypeID(typeid(T)).friendlyClassName();
  std::cout << "Looking for leaf of type " << pattern << "_label_instance_process." << endl;
  TObjArray* list = fTree->GetListOfBranches();

  // Look through every branch name.

  // We're looking for:
  // OBJ_LABEL_INSTANCE_PROCESSNAME.obj_
  // Where OBJ is something like "recob::Wires" 
  // Where LABEL is something like "pandora" 
  // INSTANCE is usually (always?) blank
  // Where PROCESSNAME is something like "McRecoAprStage1" 

  for(int i=0;i<list->GetEntriesFast();i++) {
    TObject* o = list->At(i);
    TBranch* br = dynamic_cast<TBranch*>(o);
    std::string found = br->GetName();

    // Does it start with our object type?
    if(found.find(pattern)!=0) continue;

    // Does it end with '.'?
    string::size_type p3 = found.find_last_of('.'); 
    if(p3!=found.length()-1 || p3==0) continue;

    // Tokenize by underscore.
    string::size_type p2 = found.rfind("_",p3-1);
    if(p2==string::npos || p2==0) continue;

    string::size_type p1 = found.rfind("_",p2-1);
    if(p1==string::npos || p1 ==0 ) continue;

    string::size_type p0 = found.rfind("_",p1-1);
    if(p0==string::npos || p0==0) continue;

    art::InputTag tag(found.substr(p0+1,p1-p0-1),  found.substr(p1+1,p2-p1-1), found.substr(p2+1,p3-p2-1));
    retval.push_back( make_pair( found, tag ));
  }
  return retval;

}

#2 Updated by Kyle Knoepfel almost 2 years ago

  • Description updated (diff)

#3 Updated by Kyle Knoepfel almost 2 years ago

  • Status changed from New to Feedback

Is a facility that matches the semantics of art::Event::getManyByType (in the art package) the one that is desired? If so, we would match the gallery interface to the corresponding interface in art.

#4 Updated by Kyle Knoepfel almost 2 years ago

From Nathaniel:

First, I thought getManyByType was too hard, since I requested it about a year ago by email and didnt' get any response...? Looking at the source code, it wouldn't be easy to hack.

Second, getManyByType doesn't do exactly what I want: I not only want the data products, but also the InputTag information, so I can output to the user. Maybe ART has that semantic in there somewhere.

But you could easily wrap my suggestion to also give the same result as getManyByType.. that would work fine for most people.

#5 Updated by Kyle Knoepfel almost 2 years ago

First, I thought getManyByType was too hard, since I requested it about a year ago by email and didnt' get any response...? Looking at the source code, it wouldn't be easy to hack.

We are sorry the email request was lost. Please submit future requests through the issues tracker system so they do not get lost.

Second, getManyByType doesn't do exactly what I want: I not only want the data products, but also the InputTag information, so I can output to the user. Maybe ART has that semantic in there somewhere.

In order to keep the reading speed of gallery as fast as possible, Provenance information is not available. This is the information required to provide reliable InputTag construction.

But you could easily wrap my suggestion to also give the same result as getManyByType.. that would work fine for most people.

Although we are able to provide the gallery::Event::getManyByType facility, unlike the art::Handle, the gallery::Handle does not provide provenance information. Given the lack of access to provenance information, is adding this facility to gallery still of interest?

#6 Updated by Nathaniel Tagg almost 2 years ago

Yes.

Here's the use case: an event viewer or simply a file cataloger. You know the file might contain Hits, but you want to list all the possible Hit lists the file contains so that the user can select the one they want (or put the correct name into an ART job so they can find them).

Again, simply adding my solution as a function in gallery::Event might be good enough for most users, and this is low priority for me (since you nicely provide a pointer to the raw Tree for me to operate on).

#7 Updated by Christopher Green almost 2 years ago

  • Status changed from Feedback to Assigned
  • Assignee set to Kyle Knoepfel
  • SSI Package gallery added

#8 Updated by Nathaniel Tagg 10 months ago

No update for 11 months?

#9 Updated by Kyle Knoepfel 10 months ago

See #16547-7. Looks like I dropped the ball on this one--I probably should have put this back in "Accepted". As discussed in the aforementioned reference, we should have the ability to look at our backlog of issues in the next week or so.

#10 Updated by Kyle Knoepfel 9 months ago

  • Target version set to 1.10.00

We intend to release gallery 1.10.00 on the timescale of 1 or 2 weeks.

#11 Updated by Kyle Knoepfel 9 months ago

  • Status changed from Assigned to Resolved
  • % Done changed from 0 to 100

This feature has been implemented with commits gallery:66cc1ef and gallery:a332366. Some significant restructuring of gallery was necessary to provide the feature. The salient points are:

  • getManyByType has been implemented with similar interface to art's getManyByType
  • The id() accessor has been added to the gallery::Handle and gallery::ValidHandle class templates, enabling product description retrieval.

Example of its use:

std::vector<gallery::Handle<MyProduct>> hs;
e.getManyByType(hs);

for (auto const& h : hs) {
  // Handles retrieved via 'getManyByType' are guaranteed to be valid
  assert(h.isValid());

  // Do something with the product
  auto const& prod = *h;

  // Where did the product come from?
  art::ProductID const id{h->id()};
  std::cout << e.getProductDescription(id).inputTag() << '\n';
}

#12 Updated by Kyle Knoepfel 9 months ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF