Recovering Parameter Sets used to Configure Services
I remember talking about this but I was not able to find a discussion on redmine. So I am opening one. If there is already one please point me to it.
Mu2e would like to be able to open a file of Mu2e event data and ask for the configuration of the Geometry Service that was used when a particular step in the processing chain was run. We know how to navigate the parameter sets using
We know how to find the parameter sets associated with the modules that made particular data products.
Suppose I run a chain of three jobs with cleverly named proces_names: Step1, Step2 and Step3. Then I run an analysis job on the output of Step3. Further presume that in this chain, all 4 jobs instantiate the geometry service. In my analysis job I can navigate the parameter set registry and I will discover 4 parameter sets used to configure the geometry service. As best I can tell, there is no way to tell which was used for Step1, which for Step2 and so on. And we have the same issue for other services.
Mu2e is planning a multi-month MC production run starting in May. We would like to have a solution for this in place before we start this job.
Our fallback position is to trick out our Services with methods to discover their own parameter set IDs and to stash these IDs somewhere. But we would prefer to use a solution provided by art.
Can this be provided by May 1?
A few more details on the fall back solution: I think we would write a module that interrogates each service to learn its parameter set ID. Then we could stash these IDs in a Run scope data product; or we could stash them in the internal file scope data base. If we need to do execute this plan, do you see any issue we need to consider?
#1 Updated by Christopher Backhouse almost 8 years ago
I ran into this recently. Specifically, I'd like to be able to see the configuration of services in the config_dumper output.
Other than this, one could take the output of config_dumper and use it as a fcl script that would recreate the original job, right?
#2 Updated by Christopher Green over 7 years ago
- Due date set to 05/01/2013
- Category set to Metadata
- Target version set to 1.09.00
- Start date deleted (
- Estimated time set to 16.00 h
- Scope set to Internal
- Experiment - added
- SSI Package art added
We believe this is possible now that issue #3906 is resolved, and should be able to be incorporated into config_dumper.
#4 Updated by Thomas Junk about 7 years ago
I ran into the same issue, where config_dumper does not dump fcl parameters for services.
Specifically, checking a geometry selection:
I was checking a file of only partially known provenance, and wanted to know more about how it
was generated. I ran config_dumper on it but did not get a complete listing of fcl parameters
supplied to the original job. It would be intuitive if config_dumper gave a similar (ideally exactly the same)
output as an art program running with ART_DEBUG_CONFIG turned on. As it is, I did not get the
services block printed out by config_dumper, and that contained the filename for the detector geometry
gdml, among other things.
#6 Updated by Christopher Green almost 7 years ago
- Status changed from Accepted to Feedback
- Assignee set to Christopher Green
- Target version changed from 521 to 1.09.03
- % Done changed from 0 to 20
Improvements to config_dumper functionality added with c37e0a4.
Rob, do you still need the in-code functionality?
#10 Updated by Thomas Junk over 6 years ago
I'd like to re-open this issue. config_dumper still does not dump services configuration parameters in art v1_09_04, specifically this installed version:
has the same behavior as before -- no services parameters get dumped.