Project

General

Profile

Feature #3639

Recovering Parameter Sets used to Configure Services

Added by Rob Kutschke over 6 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
High
Category:
Metadata
Target version:
Start date:
Due date:
05/01/2013
% Done:

100%

Estimated time:
16.00 h
Spent time:
Scope:
Internal
Experiment:
-
SSI Package:
art
Duration:

Description

I remember talking about this but I was not able to find a discussion on redmine. So I am opening one. If there is already one please point me to it.

Mu2e would like to be able to open a file of Mu2e event data and ask for the configuration of the Geometry Service that was used when a particular step in the processing chain was run. We know how to navigate the parameter sets using

fhicl::ParameterSetRegistry::get()

We know how to find the parameter sets associated with the modules that made particular data products.

Suppose I run a chain of three jobs with cleverly named proces_names: Step1, Step2 and Step3. Then I run an analysis job on the output of Step3. Further presume that in this chain, all 4 jobs instantiate the geometry service. In my analysis job I can navigate the parameter set registry and I will discover 4 parameter sets used to configure the geometry service. As best I can tell, there is no way to tell which was used for Step1, which for Step2 and so on. And we have the same issue for other services.

Mu2e is planning a multi-month MC production run starting in May. We would like to have a solution for this in place before we start this job.

Our fallback position is to trick out our Services with methods to discover their own parameter set IDs and to stash these IDs somewhere. But we would prefer to use a solution provided by art.

Can this be provided by May 1?

A few more details on the fall back solution: I think we would write a module that interrogates each service to learn its parameter set ID. Then we could stash these IDs in a Run scope data product; or we could stash them in the internal file scope data base. If we need to do execute this plan, do you see any issue we need to consider?


Related issues

Related to art - Bug #3906: Main parameter set not saved in RootFileDBClosed2013-05-17

Related to art - Feature #1460: Dump configuration by process name from data file.Closed2011-07-152013-09-30

Related to art - Bug #6801: Possible failure of config_dumper to dump services configuration.Closed2014-08-14

Has duplicate art - Feature #5690: FCL file parameters not properly trackedClosed2014-03-18

History

#1 Updated by Christopher Backhouse over 6 years ago

I ran into this recently. Specifically, I'd like to be able to see the configuration of services in the config_dumper output.
Other than this, one could take the output of config_dumper and use it as a fcl script that would recreate the original job, right?

#2 Updated by Christopher Green about 6 years ago

  • Due date set to 05/01/2013
  • Category set to Metadata
  • Target version set to 1.09.00
  • Start date deleted (03/31/2013)
  • Estimated time set to 16.00 h
  • Scope set to Internal
  • Experiment - added
  • SSI Package art added

We believe this is possible now that issue #3906 is resolved, and should be able to be incorporated into config_dumper.

#3 Updated by Christopher Green about 6 years ago

  • Status changed from New to Accepted

#4 Updated by Thomas Junk over 5 years ago

I ran into the same issue, where config_dumper does not dump fcl parameters for services.
Specifically, checking a geometry selection:

I was checking a file of only partially known provenance, and wanted to know more about how it
was generated. I ran config_dumper on it but did not get a complete listing of fcl parameters
supplied to the original job. It would be intuitive if config_dumper gave a similar (ideally exactly the same)
output as an art program running with ART_DEBUG_CONFIG turned on. As it is, I did not get the
services block printed out by config_dumper, and that contained the filename for the detector geometry
gdml, among other things.

#5 Updated by Christopher Green over 5 years ago

  • Target version changed from 1.09.00 to 521

#6 Updated by Christopher Green over 5 years ago

  • Status changed from Accepted to Feedback
  • Assignee set to Christopher Green
  • Target version changed from 521 to 1.09.03
  • % Done changed from 0 to 20

Improvements to config_dumper functionality added with c37e0a4.

Rob, do you still need the in-code functionality?

#7 Updated by Christopher Green over 5 years ago

  • % Done changed from 20 to 100

Resolved per discussion with Rob.

#8 Updated by Christopher Green over 5 years ago

  • Status changed from Feedback to Resolved

#9 Updated by Marc Paterno over 5 years ago

  • Status changed from Resolved to Closed

#10 Updated by Thomas Junk about 5 years ago

I'd like to re-open this issue. config_dumper still does not dump services configuration parameters in art v1_09_04, specifically this installed version:

/grid/fermiapp/products/larsoft/art/v1_09_04/slf6.x86_64.nu.e5.prof/bin/config_dumper

has the same behavior as before -- no services parameters get dumped.



Also available in: Atom PDF