Feature #4187
Reading in metadata from input files
0%
Description
Currently, the metadata facilities only exist to add or get metadata produced within a job. When running a job which takes in input files, there needs to be a way to access any metadata associated with that file. Then, experiments could write a module/service which makes the appropriate decision of whether or not to carry that metadata over into the output file.
History
#1 Updated by Marc Paterno about 6 years ago
- Status changed from New to Feedback
I'm not sure to what extent, if at all, this may meet your needs, but have you looked at the "provenance" information available through the art::Handle (or art::ValidHandle)? Each can provide access to an object of type art::Provenance, which provides access to the metadata for the object associated with the Handle. You can find the header for Provenance at https://cdcvs.fnal.gov/redmine/projects/art/repository/revisions/develop/entry/art/Framework/Principal/Provenance.h. If you have not already done so, please take a look and let us know to what extent that meets your needs.
#2 Updated by Rob Kutschke about 6 years ago
You can also look at some mu2e code to see how to access provenance information:
For the parameter sets in the provenance of a given data product:
http://cdcvs.fnal.gov/cgi-bin/public-cvs/cvsweb-public.cgi/mu2e/Offline/Analyses/src/ViewProvenance_module.cc?rev=1.3&content-type=text/x-cvsweb-markup
To see all parameter sets stored in the metadata:
http://cdcvs.fnal.gov/cgi-bin/public-cvs/cvsweb-public.cgi/mu2e/Offline/Analyses/src/ViewAllProvenances_module.cc?rev=1.1&content-type=text/x-cvsweb-markup
#3 Updated by Adam Aurisano about 6 years ago
I see that it is possible to get the parameter set used to configure the module which created a given data product; however, I'm trying to get metadata associated with the job as a whole. This metadata was injected into the art file using the FileCatalogMetadata service. There was a metadata module (analyzer) whose job was to extract metadata from a parameter set in the FHiCL file and insert it use the FileCatalogMetadata service, but there was not data product added to the event by the metadata module. Would it be possible to recover the parameter set of the metadata module even though it did not add any data products to the event?
#4 Updated by Christopher Green almost 6 years ago
- Scope set to Internal
- Due date set to 09/30/2013
- Category set to Infrastructure
- Status changed from Feedback to Accepted
- Target version set to 1.09.00
- Start date deleted (
06/20/2013) - Estimated time set to 32.00 h
- Experiment NOvA added
- SSI Package art added
We believe we can provide read-only access to the SQLite database from the currently open input file for as long as that file is open. Some finer design questions remain, but I think they can be resolved fairly quickly.
#5 Updated by Christopher Green over 5 years ago
- Target version changed from 1.09.00 to 521
#6 Updated by Kyle Knoepfel over 1 year ago
- Target version deleted (
521)