Project

General

Profile

Bug #3520

Inserted metadata not appearing in output

Added by Adam Aurisano over 6 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Category:
I/O
Target version:
Start date:
02/22/2013
Due date:
% Done:

100%

Estimated time:
Occurs In:
Scope:
Internal
Experiment:
-
SSI Package:
Duration:

Description

I attempted to add metadata in a NOvA process. In one module, I added this in the module's destructor:

art::ServiceHandle< art::FileCatalogMetadata > metadata;
metadata->addMetadata( "testMetadata", "success!" );

In a downstream module, I added this to extract and print the metadata:

art::ServiceHandle<art::FileCatalogMetadata > metadata;
std::vector< std::pair< std::string, std::string > > md;
metadata->getMetadata( md );

for (std::vector< std::pair< std::string, std::string > >::iterator it = md.begin(); it != md.end(); ++it)
  {
     mf::LogInfo("SimpleTransport") << "Metadata: Key = " << it->first << " Value = " << it->second;
  }

When these modules are run in the context of a single jop, the metadata is correctly printed out when the second module destructs; however, if these two modules are run in two subsequent jobs (passing the output file from the first job to the second), the metadata is not found.

In addition, when I try running the metadata dumper, I find:

sam_metadata_dumper gen.root 

File catalog metadata from file gen.root:

No file catalog Metadata rows found - table is missing or empty

*** Break *** segmentation violation

History

#1 Updated by Christopher Green over 6 years ago

  • Description updated (diff)

#2 Updated by Christopher Green over 6 years ago

  • Status changed from New to Resolved
  • Assignee set to Christopher Green
  • Target version set to 1.03.06
  • % Done changed from 0 to 100

As we discussed in person, the primary problem is that you were not entering the data at the correct time for it to make it into a file: modules are destroyed only after all data processing is complete and files have been closed.

With the caveat that currently one cannot add metadata specific to one output module, using FileCatalogMetadata::addMetadata() with a class-scope "Kilroy" variable at the produce() stage, or even just once at the construction or beginJob() stage would probably work, but beware of the consequences of having multiple instances of the same module.

In addition, there were some consequences of an internal change to SQLite that affected the way we were deciding whether to try to write the DB back to the ROOT file or not. This was corrected with 3c42a229657de19848b2a864fe81dbb272b389ab. Also, dictionary warnings have been suppressed from the output of sam_metadata_dumper and DB access error checking and reporting has been improved.

I will enter a separate feature request, linked to this one, for the wider issue of output-specific metadata.

#3 Updated by Christopher Green over 6 years ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF