Support for arbitrary, random-file access
Originally raised on ART-USERS list, March 27:
I'm attempting to use ART (namely larsoft) in a non-standard mode. I want to use the executable as a service to retrieve event data from arbitrary files and events.
That is: I want an event loop like this:
art::Event* event = get_event()
One way: build a service that runs in a normal event loop, but then tells the InputSource where to go next. I've looked at nutools/EventDisplay/EventDisplay_service.cc, and this gets about halfway there: it allows for navigation between events, but not between files (as far as I can tell). Also, it's not entirely clear how a service hooks into the state machine; there appears to be no documentation on designing services.
Second way: don't bother invoking the ART EventProcessor and the full-fledged state machine, but instead just start in main(), instantiate my own input source, or perhaps even RootInputFile (which is what I really want), but these rely upon art "internals" that are even less well documented.
Although my current system works, there are problems with speed, robustness, and maintenance, which would be improved by actually using the framework to read the data instead of using ROOT to interrogate the Events tree directly.
I've been trying to directly instantiate RootInput or RootInpuFile myself from main(), but this is fraught with hazards due to uninitialized framework services/singletons. I've been trying to replicate my own version of EventProcessor which would do the same job, but I don't really know what I'm doing so it's hard to debug.
Chris Backhouse points out that I should simply create my own RootInput InputSource, but it's unclear to me how to create that object from main() without the framework set up around it. It occurs to me that one way forward would be to make a simple service, then inside BeginJob just goes and makes it's own InputSource independent from the EventProcessor, and captures job execution until finished. I haven't tried that yet.
Herb Greenlee pointed out that the CatalogInterface and FileTransfer service interfaces do something like this, which might be my way in.
#1 Updated by Christopher Green over 5 years ago
- Status changed from New to Feedback
- Scope changed from External to Internal
I think we'd like to change the definition of this ticket to more accurately reflect what you really want, which is (I believe) for art to provide support specifically for event displays. We'd like to discuss this with you and the other interested experiments who have raised this subject in the past, get some requirements written down and actually schedule the work. Please let us know if the slight re-targeting of this ticket is acceptable to you.
#2 Updated by Nathaniel Tagg over 5 years ago
Sure, this is fine.
In successive experiments, I've found that the problem of the event loop is actually superficial: a bigger problem it appears is that the framework does not allow successive loading of files with different data products. This is also killer for my application.
#3 Updated by Nathaniel Tagg over 5 years ago
One more note:
As a final, last-ditch hack, I tried to simply loop the core of main(): making a pset with my file, creating an event processor, running it (over 1 event) then deleting the event processor and starting over.
This fails, because ART uses a singleton (!) ProductMetaData, which has no destroyer. That is, ART explictly disallows a programmer to look at files with different data products in the same job.
I do not understand why the reading of different data products in different files would be something ART doesn't explicitly support out of the box!
So, I wander off into the sunset, the mighty walls of the framework keeping me from my damsel in distress. I will continue with my previous efforts to employ sappers to expand my secret network of tunnels under the castle.