Project

General

Profile

Feature #7081

SAM Process ID in metadata

Added by Dominick Rocco about 5 years ago. Updated 10 days ago.

Status:
Assigned
Priority:
Normal
Assignee:
-
Category:
Metadata
Target version:
-
Start date:
09/29/2014
Due date:
% Done:

0%

Estimated time:
Scope:
Internal
Experiment:
NOvA
SSI Package:
Duration:

Description

Robert Illingworth pointed out that it would be helpful to store the SAM process ID in the file metadata. It would be nice if ART could pick this up and add it. NOvA could build this into it's own metadata module, but Robert thought it is an appropriate all-experiment feature. The art_sam_wrap.sh shell script stores the process id in a variable called $consumer_id, so the art code could try to key off that. Otherwise, it seems appropriate to pass it in as an environment variable, art_sam_wrap.sh could be modified to pass it in.

History

#1 Updated by Christopher Green about 5 years ago

  • Status changed from New to Feedback

According to the code, the services.FileCatalogMetadata.processID parameter (settable via the --sam-process-id option) is already added to the file catalog metadata as process_id. Please let us know whether this item is present in the output of sam_metadata_dumper for an output file that you know used SAM as input, as the absence of this item would indicate a bug which should be found and fixed.

#2 Updated by Dominick Rocco about 5 years ago

It appears that the --sam-process-id flag can only be passed in the presence of --sam-web-uri. In our case, art_sam_wrap.sh does the fetching of the file, so we don't pass in --sam-web-uri. I think that the desired behavior is to require the process ID in the presence of a web URI, but not vice versa. Is that a sensible alternative? Here's the output I see when --sam-web-uri is not included.

$ nova -c /grid/fermiapp/nova/novaart/novasvn/releases/development/ProductionScripts/fhicl/prod_reco_numi_job.fcl /nova/ana/users/rocco/tmp/neardet_genie_fhc_nonswap_geantonly_1000_r01000095_s54_S14-08-01_v1_20140620_170832_fnpc2058.fnal.gov_1404203105_4746_0_sim.pid.root --sam-process-id 99
OptionsHandler caught a cet::exception calling art::FileCatalogOptionsHandler::doProcessOptions()
---- Configuration BEGIN
  configurations services.user.CatalogInterface.webURI (--sam-web-uri) and
  services.FileCatalogMetadata.processID (--sam-process-id) must be specified
  together or not at all.
---- Configuration END

Art has completed and will exit with status 7002.

When I do pass them together, I get a different exception, but it may be simply because of a poorly formed URI when trying to run on a local file.


$ nova -c /grid/fermiapp/nova/novaart/novasvn/releases/development/ProductionScripts/fhicl/prod_reco_numi_job.fcl /nova/ana/users/rocco/tmp/neardet_genie_fhc_nonswap_geantonly_1000_r01000095_s54_S14-08-01_v1_20140620_170832_fnpc2058.fnal.gov_1404203105_4746_0_sim.pid.root --sam-process-id 99 --sam-web-uri stuff --sam-data-tier=out1:pid --sam-stream-name=out1:0
%MSG-s ArtException:  nova 01-Oct-2014 09:27:25 CDT JobSetup
cet::exception caught in art
---- OtherArt BEGIN
  ServiceCreation
  ---- Type mismatch BEGIN
    processID
    ---- Type mismatch BEGIN
      can't obtain atom from sequence
    ---- Type mismatch END
  ---- Type mismatch END
  cet::exception caught during construction of service type art::FileCatalogMetadata:
---- OtherArt END
%MSG
Art has completed and will exit with status 1.

#3 Updated by Kyle Knoepfel over 4 years ago

  • Target version set to 521

#4 Updated by Kyle Knoepfel over 3 years ago

  • Status changed from Feedback to Assigned
  • Assignee set to Paul Russo

#5 Updated by Kyle Knoepfel about 2 years ago

  • Target version deleted (521)

#6 Updated by Kyle Knoepfel 10 days ago

  • Assignee deleted (Paul Russo)


Also available in: Atom PDF