Project

General

Profile

Support #8226

file merge showstopper

Added by Andrei Gaponenko over 4 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Immediate
Assignee:
Category:
-
Target version:
-
Start date:
04/01/2015
Due date:
% Done:

100%

Estimated time:
Spent time:
Scope:
Internal
Experiment:
Mu2e
SSI Package:
art
Duration:

Description

Hello,

A homogeneous dataset of art files was generated starting with the
EmptyEvent source and using fcl configs that differ only in random
number seeds and subrun numbers. The next stage of processing handles
individual output files fine. However an attempt to use multiple
first stage files in a single second stage jobs leads to

%MSG-s ArtException:  PostCloseFile 01-Apr-2015 17:56:03 CDT PostEndRun
cet::exception caught in art
---- UnimplementedFeature BEGIN
  Cannot merge file '/mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00006/data_g4s1_mubeam.art' due to a branch mismatch.
  Contact the framework group.
  cet::exception caught in EventProcessor and rethrown
---- UnimplementedFeature END
%MSG

This is a showstopper for Mu2e production. Note that some files can
be merged, but others can not.

To reproduce:

- build Mu2e Offline git ce7a550b6fd5ff52e1ad3f6842a20a907f5d859e

- verify that you can analyze individual files:

   mu2e -c JobConfig/cd3/beam/beam_g4s2.fcl /mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00005/data_g4s1_mubeam.art

   mu2e -c JobConfig/cd3/beam/beam_g4s2.fcl /mu2e/data/users/gandr/cd3/20150331a.g4s1/good/00006/data_g4s1_mubeam.art

- Try to process both of the above files in a single job and observe
the breakage.

The fcl configuration used to run the first stage and the log files
can be found in the same directories as the data files.

Andrei

History

#1 Updated by Andrei Gaponenko over 4 years ago

An update: Kyle provided a critical tool to identify the discrepancy. The difference between the files is due to one job making an extra produces<...>() call compared to the other. That produces<>() call is conditional and the condition depends on an uninitialized variable in Mu2e Offline code. I still have to test a fix, but it looks like the problem is not in the framework but in the user code.

Andrei

#2 Updated by Kyle Knoepfel over 4 years ago

  • Tracker changed from Bug to Support
  • Status changed from New to Feedback
  • Assignee set to Kyle Knoepfel

I'm glad we were able to diagnose the likely culprit. I have left this issue in "feedback" because I believe the debug printout (where the branch name and brand ID is printed out for each produces call) should be provided/providable either via a messagefacility logging call or with a dedicated ProductTracker service. I will discuss this with the art team, and we will determine the proper approach, at which point I'll close the issue.

#3 Updated by Kyle Knoepfel over 4 years ago

  • Status changed from Feedback to Closed
  • % Done changed from 0 to 100
  • SSI Package art added
  • SSI Package deleted ()


Also available in: Atom PDF