Support #22624

Way to read SQLITE/larsoft config without using a global

Added by Nathaniel Tagg 4 months ago. Updated 4 months ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


I'm trying to read the configuration of a larsoft file, which uses an SQLITE backing store. I'm doing this in a random-access environment under Gallery, with multiple open files being accessed by different threads. However, when I look at the sample code, I find that the only way to do this import is through a ParameterSetRegistry, which seems to be a singleton.

I can't fathom why a singleton is used here. Why not have a generic Registry object and then make a special instantiation as a singleton?

Is there a usable workaround?


#1 Updated by Kyle Knoepfel 4 months ago

  • Status changed from New to Feedback
  • Description updated (diff)

Can you give more information? What problem have you encountered that requires a workaround? Are you running into data races? Segfaults?

#2 Updated by Kyle Knoepfel 4 months ago

From Nathaniel:

I have the current problems:

- I can only read the Pset from one file at a time. I have to have a global mutex that disallows reading any other file or part of the same file at the same time. That's because it uses a singleton. Again, I can see no reason why a singleton is needed here. If FHICLCPP is to be used outside the ART context, this should be addressed.

New problems I've found on implementation:
- Using the config_dumper code as a template, I find this is incompatible with Gallery. Both codes attempt to SetBranchAddress for elements of the MetaData tree, causing a segfault.

- It's super slow. It takes an entire second to retrieve one element from a standard uboone reco file. That's comperable to, say, reading all hits.

#3 Updated by Kyle Knoepfel 4 months ago

Hi Nathaniel,

Thanks for your response. Although I agree with some of your concerns, it's unlikely we'll be able to change the current design for the following reasons:

  • The ParameterSetRegistry class is a carryover from CMSSW. Although the art developers have taken great steps to remove many global variables, this one is more difficult to remove as (almost) all ParameterSet creation relies on the ParameterSetRegistry to implicitly store its values. We could conceive of making a ParameterSetRegistry object an argument that must be specified to the ParameterSet constructor. That, however, is a breaking change that would require stakeholder input. Regardless, finding a way to change the design takes effort that we currently do not have.
  • Although we have profiled the fhiclcpp library in the past, we typically do not expend a lot of effort in optimizing it. We routinely tell users that all configuration-processing and -reading should be done as rarely as possible, which, in the context of art, is at the beginning of a job. We would have to understand your use case better as to why you are reading the on-disk configurations, and how you are using them once they are in memory. It may be that there is a better solution that meets your need than having to read the RootFileDB.
  • It's not clear to what extent gallery is a necessary library for your use case. Assuming that it is, keep in mind that gallery is designed for fast reading of data products, not its provenance. Expanding its scope to provide more provenance can result in a performance hit elsewhere, which would be at odds with the intent of the library. Note, also, that no effort has been made to ensure thread-safety of the gallery library.

If simple improvements can be identified and implemented, then we are supportive of pursuing them. However, based on the current fhiclcpp design, we do not expect simple improvements to sufficiently address your request. The SciSoft team has also had its effort assigned almost exclusively to LArSoft, reducing the likelihood of us redesigning this part of the system.

I am happy to continue the discussion. At the very least, if you can describe your use case more fully, we may be able to find a different solution.


#4 Updated by Nathaniel Tagg 4 months ago

Hi Kyle,
Thanks for the thoughtful reply.

My problem is actually quite deep. As I wrote in the Larsoft forum the other day:
There is no way to tell what experiment a file came from, or what detector the file came from, from looking at the data file.

To my mind, this is absolutely imperative that this sort of information be accessible to analyzers. In my case, I'm trying to build a general-purpose event viewer that can random-access data files (hence Gallery). At present, the ONLY way to even guess what detector a file came from is to look at the job configuration and attempt to figure out what channel mapping service or geometry file was used when the file was created. This is obviously a sucky workaround, but there doesn't seem to be any other.

This is complicated by the fact that this metadata is so obscurely stored (i.e. the SQLITE blob inside a ROOT file) that it's nearly impossible to get out in a robust, fast way. SAM metadata is only useful if one is looking at a file declared to SAM AND you know what experiment the file is from. It's also complicated by the global object, when my binary is running on a forked-or-threaded system so that multiple users can access files at the same time. (That in turn is a workaround required because starting up a new binary process for each request takes up to 10 minutes, when I am aiming at a server time of under 5 seconds.)

(This would all have been trivially solved if ARTists had used Robert Hatcher and Nick West's work on MINOS, which solved these problems through the notion of a ValidityContext which mapped out multi-parameter space for event origins, but instead ART uses only run, subrun, and event numbers and trusts that these will be unique. That assumption is clearly violated by any realistic way in which we run DUNE or SBN. But now we are stuck with this model, and need to find a workaround.)

There are other reasons I want to get at this data, but this gives you the jist.

This obviously is beyond the scope of fhiclcpp, but it highlights how underlying libraries should not make build assumptions. To be honest, I haven't delved deeply enough why a ParameterSetRegistry class is even required - the scant documentation seems to suggest it's a holder of ParameterSet objects, and I still don't get why that needs to be a global.

My workaround is to set a global mutex, clear the registry, copy the useful parts of the parameter sets into a different form (JSON), throw away the parameter set, unlock the mutex, and simply ignore the entire FHICL framework. Will this work? Or will the global Registry object mean that successive reads of different files can't be parsed correctly?

Also available in: Atom PDF