Project

General

Profile

History

Wiki ยป

ART Suite Release Notes 1.10.00 (2014/06/10).

ART 1.10.00.

The individual issues addressed with this release are enumerated below, with further details as appropriate. However, the details of all SAM-releated changes and improvments are detailed in their own section.

New features:

  • #3743: FileCatalogMetadata service doesn't set file format.
  • #3744: Missing sam metadata.
  • #3745: Art should not require metadata that sam considers to be optional or non-existent.
  • #3746: Art should support generating per-file user-specified sam metadata.
  • #3774: Improved ability to generate output file names. Full details of all format specifiers for output file substitution may be found in the wiki article Output file renaming for ROOT files.
    • Sequence no.: %#.
    • Last-opened input file ...
      • Base name without extension: %ifb.
      • Fully-resolved path without filename: %ifd.
      • Extension: %ife.
      • Base name with extension: %ifn.
      • Fully-resolved path with filename: %ifp.
      • Regex-based substitutions (ECMAScript with %-delimiter): %ifs%<match>%<format>%[ig]%.
  • #3945: ActivityRegistry should have callbacks for when output files are opened and closed.
    • PostOpenOutputFile(std::string const & mod_label).
    • PreCloseOutputFile(std::string const & mod_label).
    • PostCloseOutputFile(art::OutputFileInfo const & info).
  • #3962: FileCatalogMetadata service improvements.
  • #4679: Relax FileCatalogOptionsHandler constraint on when SAM metadata are required.
  • #6396: Additional goodies for artmod.
    • Destructor is left to the compiler rather than being declared or defined: the vast majority of module programmers will never need to implement one, and those that do will know how to do it.
    • Plugins should in general never be copied or assigned: these functions are now deleted using ISO C++2011 syntax.
    • Most plugin types (producers, filters, analyzers, sources) now include the main messagefacility header in order to facilitate the use of managed log messages and warnings.
  • *BREAKING CHANGE* #6418: Ability to obtain Ptr into collections not yet in event from input source.
    The advent of the above feature requires implementors of detail classes for the Source template to change the signature of their constructors. Upon an attempt to compile old code against this version of art, a #error message will be generated giving instructions which include a perl, "one-liner" to solve the compilation problem.
    In a nutshell: art::PrincipalMaker has been replaced by art::SourceHelper, which has the function make_ptr(...) (but see known issues below). The required art::TypeLabel should have been cached from the return of the art::ProductRegistryHelper::reconstitutes(...) calls made in the detail class' constructor.
  • *BREAKING CHANGE* Plugin management (art::LibraryManager and friends) has been migrated to cetlib, to enable its use in packages other than art and those that depend upon it. An experiment may now design its own category of plugins, although one is strongly recommended to use existing plugin types if they are suitable. If you use LibraryManager directly you will need to reseat your header and account for the change of namespace from art to cet.

Bug fixes:

  • #4730: Configuration post-processing system sometimes checks command line options for prerequisites when it should be checking configuration.
  • #6346: registerSecondaryFileNameProvider and fileNames parameter.
    Due to a typo, the fileNames parameter was required erroneously for a mixing module even if the detail object provided a registerSecondaryFileNameProvider() function.
  • #6353: While mixing, the registered secondary filename provider is called for the first time before respondToOpenInputFiles.
    An internal reorganization of the mixing facility was required to allow the registerSecondaryFileNameProvider() function to be called late enough that information it might require to operate would be available.
  • #6420: Assns::size_type inaccessible.
    Bi-directional associations with a payload (art::Assns<A, B, D>) did not have this typedef available.

SAM-releated changes and improvments.

  • The primary key for metadata entries in the SQLite DB has been fixed and now autoincrements as originally intended. Therefore when human-readable output is selected for sam_metadata_dumper (see below), the previous "0:" is now replaced by the row number from the DB.
  • *BREAKING CHANGE* The header for art::OutputFileInfo (used by file-based watchpoints such as PostCloseOutputFile(...)) has moved from art/Framework/Core to art/Utilities for reasons of dependency sanity. You may need to execute the following command to fix your code:
    ack -l art/Framework/Core/OutputFileInfo | xargs perl -wapi\~ -e 's&art/Framework/Core/OutputFileInfo&art/Utilities/OutputFileInfo&'
  • *BREAKING CHANGE* The sam_metadata_dumper utility now puts out JSON-format information by default. Example output:
    {
  "test/Integration/TestMetadata_plugin_t.d/out.root": {
    "process_name": "DEVEL",
    "key1": "value1",
    "key2": "value2",
    "file_format": "artroot",
    "file_format_era": "ART_2011a",
    "file_format_version": 5,
    "start_time": "2014-06-10T13:59:22",
    "end_time": "2014-06-10T13:59:22",
    "event_count": 1,
    "first_event": [ 1, 0, 1 ],
    "last_event": [ 1, 0, 1 ]
  },
  "test/Integration/SAM_metadata.d/out.root": {
    "applicationFamily": "Ethel",
    "applicationVersion": "v0.00.01a",
    "file_type": "MC",
    "run_type": "MCChallenge",
    "group": "MyGang",
    "process_name": "SAMMetadataW",
    "testMetadata": "success!",
    "dataTier": "The one with the thickest frosting",
    "streamName": "o1",
    "file_format": "artroot",
    "file_format_era": "ART_2011a",
    "file_format_version": 5,
    "start_time": "2014-06-10T13:59:22",
    "end_time": "2014-06-10T13:59:22",
    "runs": [ [ 1, 0, "MCChallenge" ] ],
    "event_count": 1,
    "first_event": [ 1, 0, 1 ],
    "last_event": [ 1, 0, 1 ]
  }
}
    This document has passed JSON verification at http://jsonformatter.curiousconcept.com/ and should be readable with any JSON parser (see http://json.org/ for details). The original human-readable output may be obtained by using the -H, --hr or --human-readable options, but be advised that any textual data provided as part of a tuple (the runs item, for example) have been surrounded by double-quotes.
  • The following new automatically-generated SAM metadata items are provided when appropriate:
    • file_format. Also file_format_era and file_format_version, although these are not recognized as special by SAM.
    • runs -- user must provide the run type for this to be generated (see below). This repesents all run and subrun records that have been seen, regardless of whether any events were attached.
    • event_count.
    • first_event (actually, lowest EventID seen).
    • last_event (actually, highest EventID seen).
    • start_time (output file open time).
    • end_time (the time the metadata were written to the about-to-be-closed file).
  • There is a new plugin type (FileCatalogMetadataPlugin) whose entry points are invoked by the output stream for which it is configured at appropriate times in the life cycle of that stream. A full description of this feature may be found in the wiki article: FileCatalogMetadataPlugin.
  • The validation of required metadata items has changed to be more in line with SAM's requirements:
    • Certain items may be specified with either command-line options or FHiCL directives; previously the verification tripped unless these items were specified on the command line.
    • If SAM input is selected, or any of the below items are provided, then all of them are required (except file_type, see below):
      • --sam-application-family, --sam-app-family, services.FileCatalogMetadata.applicationFamily;
      • --sam-application-version, --sam-app-version, services.FileCatalogMetadata.applicationVersion;
      • --sam-file-type, services.FileCatalogMetadata.fileType;
      • --sam-group, services.FileCatalogMetadata.group.
    • The file_type item may be provided in code via the FileCatalogMetadata::addMetadata() service function or the FileCatalogMetadataPlugin interface, but will default to "unknown" if not specified (but see known issues below).

Known issues:

  • The naming of public functions in art::SourceHelper is inconsistent (makeXXXPrincipal() vs make_ptr). See Release Notes 1.10.00a for the resolution.
  • The dataTier and streamName SAM metadata items are still inconsistent with their names as recognized by SAM. See Release Notes 1.10.00a for the resolution.
  • file_type, if specified as a command-line option or FHiCL specification, is not honored. See Release Notes 1.10.00a for the resolution.

messagefacility 1.11.08 (new version).

fhicl-cpp 2.19.04 (new version).

cetlib 1.06.01 (new version).

cpp0x 1.04.04 (in since ART 1.09.03).