Project

General

Profile

Bug #2976

mu2e -S filelist runs on more than the filelist

Added by Andrei Gaponenko over 7 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Christopher Green
Category:
Application
Target version:
1.11.00
Start date:
09/28/2012
Due date:
09/30/2013
% Done:

100%

Estimated time:
8.00 h
Spent time:
2.00 h
Occurs In:
Scope:
Internal
Experiment:
Mu2e
SSI Package:
art
Duration: 368

Description

Hi,

There is a file list with just one file:

$ cat mars-monfnal3-rand1.list 
/data/mu2e/extmon/mars/valexakh/monfnal3/andr-uc/rand1/000_fort.87

The fcl file has

source: {
    module_type: FromEMFMARSFileWeighted
    fileNames: [ "/data/mu2e/extmon/mars/valexakh/monfnal3w1/andr-test/398_fort.86", "/data/mu2e/extmon/mars/valexakh/monfnal3w1/andr-test/399_fort.86" ]

    runNumber          : 55
    firstSubRunNumber  : 0
    firstEventNumber   : 1

    maxEvents: -1
}

When running

$ mu2e -c Sources/test/FromEMFMARSFileWeighted.fcl -S ./mars-monfnal3-rand1.list

I see that the job processes two files:

/data/mu2e/extmon/mars/valexakh/monfnal3/andr-uc/rand1/000_fort.87

(requested by the cmdline list) and

/data/mu2e/extmon/mars/valexakh/monfnal3w1/andr-test/399_fort.86

(which looks like a leftover from what is in the fcl).

While the job uses a custom source module, the detail class does no
handling of the list of files.  So the problem is in the generic
source implementation in the framework and should be reproducible
without my specific code (which is not in CVS at the moment).

Andrei

Associated revisions

Revision 96c0f778 (diff)
Added by Christopher Green over 5 years ago

Issue #2976.

History

#1 Updated by Christopher Green over 6 years ago

  • Due date set to 09/30/2013
  • Category set to Application
  • Status changed from New to Accepted
  • Target version set to 1.09.00
  • Estimated time set to 8.00 h
  • Scope set to Internal
  • Experiment - added
  • SSI Package art added

We should implement a test for this behavior if it does not already exist. It is possible that this issue was resolved as a byproduct of re-working argument-handling to support SAM requirements.

#2 Updated by Christopher Green about 6 years ago

  • Target version changed from 1.09.00 to 521

#3 Updated by Andrei Gaponenko over 5 years ago

Hello,

art v1_08_09 still does not handle input file lists correctly.
Someone got wrong outputs today because of it. A grid job
got an empty list of input files (zero length file as an argument to the
"-S" art's command line option). Instead of either crashing or producing
no outputs art ran on a test data file that was mentioned in the fcl config.

To reproduce:

$ cat inputListTest.fcl
process_name : inputListTest
source: { module_type: RootInput fileNames : [ "nonExistingFile.root" ] }

$ ls nonExistingFile.root
ls: cannot access nonExistingFile.root: No such file or directory

$ ls nofiles.txt
ls: cannot access nofiles.txt: No such file or directory

$ touch nofiles.txt
$ mu2e c inputListTest.fcl -S nofiles.txt
...
cet::exception caught in art
--- CatalogServiceError BEGIN
Input file not found: nonExistingFile.root.
---- CatalogServiceError END

#4 Updated by Christopher Green over 5 years ago

  • Status changed from Accepted to Resolved
  • Assignee set to Christopher Green
  • Target version changed from 521 to 1.11.00
  • % Done changed from 0 to 100
  • Experiment Mu2e added
  • Experiment deleted (-)

With 96c0f77:

  1. An source list file (empty or otherwise) will always override specification in FHiCL.
  2. A source file list specification is incompatible with command-line specification of file names (either with -s, --source or as non-option arguments).

This functionality is now tested to prevent future regression.

Please advise soonest if this is not desired behavior.

#5 Updated by Christopher Green over 5 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF