Project

General

Profile

Bug #14160

Ordering of input files to source being sorted in middle of chain

Added by Johnny Ho almost 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
Normal
Category:
-
Target version:
Start date:
10/13/2016
Due date:
% Done:

100%

Estimated time:
4.00 h
Spent time:
Occurs In:
Scope:
Internal
Experiment:
LArSoft
SSI Package:
art
Duration:

Description

Dear artists,

We are having a problem in lariatsoft where the input files to a source are not being opened in the order in which they are given.

To reproduce the problem, create a list of files with the following:

/pnfs/lariat/raw/000/992/lariat_r006916_sr0001.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0002.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0003.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0004.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0005.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0006.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0007.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0008.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0009.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0010.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0011.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0012.root

as files.list. Then run a simple source, DoNothingInput_source, that only prints the name of the opened file:

source /grid/fermiapp/larsoft/products/setup
export PRODUCTS=/grid/fermiapp/products/lariat:${PRODUCTS}:/grid/fermiapp/products/common/db
setup lariatsoft v06_11_00 -q e10:prof
art -c do_nothing_input_source.fcl -S files.list > test.log 2>&1 && grep -E 'file|root' test.log

This produces the following output:

You have specified the following 12 input file(s):
/pnfs/lariat/raw/000/992/lariat_r006916_sr0001.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0002.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0003.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0004.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0005.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0006.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0007.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0008.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0009.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0010.root
/pnfs/lariat/raw/000/992/lariat_r006916_sr0011.root
/pnfs/lariat/raw/000/991/lariat_r006916_sr0012.root
Opened input file /pnfs/lariat/raw/000/991/lariat_r006916_sr0003.root
Opened input file /pnfs/lariat/raw/000/991/lariat_r006916_sr0005.root
Opened input file /pnfs/lariat/raw/000/991/lariat_r006916_sr0006.root
Opened input file /pnfs/lariat/raw/000/991/lariat_r006916_sr0009.root
Opened input file /pnfs/lariat/raw/000/991/lariat_r006916_sr0012.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0001.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0002.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0004.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0007.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0008.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0010.root
Opened input file /pnfs/lariat/raw/000/992/lariat_r006916_sr0011.root

It looks like the files are being sorted at some point before we get to readFile in the source. It is important to note that the sorting is done on the full file path.

Thanks,
Johnny

Associated revisions

Revision ed2ac51d (diff)
Added by Christopher Green almost 3 years ago

Remove unwanted std::sort() on filenames, per issue #14160.

History

#1 Updated by Johnny Ho almost 3 years ago

Oops, the instructions to reproduce the problem is missing a line!

source /grid/fermiapp/larsoft/products/setup
export PRODUCTS=/grid/fermiapp/products/lariat:${PRODUCTS}:/grid/fermiapp/products/common/db
setup lariatsoft v06_11_00 -q e10:prof
art -c do_nothing_input_source.fcl -S files.list > test.log 2>&1 && grep -E 'file|root' test.log

#2 Updated by Christopher Green almost 3 years ago

  • Description updated (diff)
  • Status changed from New to Resolved
  • Assignee set to Christopher Green
  • Target version set to 2.05.00
  • % Done changed from 0 to 100
  • Estimated time set to 4.00 h

Description corrected per previous comment.

Problem solved after unwanted sort found, per art:ed2ac51

#3 Updated by Kyle Knoepfel almost 3 years ago

  • Project changed from cet-is to art
  • Status changed from Resolved to Closed


Also available in: Atom PDF