Feature #5443

fhicl file search function

Added by Wesley Ketchum about 6 years ago. Updated about 3 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


Something from the backlog of suggestions...

It would be nice to have a function/script that, given a fhicl file name, will search though $FHICL_FILE_PATH and find the location of the fhicl file it would use if given that name. Even better, it could list all found files, and rank the order in which they would be used. This will help people (1) understand where fhicl files are, and (2) help unnecessary headache in debugging, as users can immediately know if the input file they are changing is actually the one being used.

Would appreciate other comments on ways to increase utility of such a tool. (3 KB) Gianluca Petrillo, 02/14/2014 01:41 PM (3.63 KB) Gianluca Petrillo, 02/20/2014 06:31 PM


#1 Updated by Wesley Ketchum about 6 years ago

I should also say ... if something like this exists already and I don't know about it, please just point me to it!

#2 Updated by Erica Snider about 6 years ago

  • Status changed from New to Assigned
  • Assignee set to Erica Snider

Hi Wes,
I have a tool like this. It actually finds all instances. The first one listed will be the one that gets used, but it's sometimes helpful to know what other versions are out there.

Here is what is does:

echo $FHICL_FILE_PATH | sed -e "s/\:/\n/g" | xargs -I {} ls {}/${1} 2>/dev/null

Maybe not the most elegant line of bash, but it gets the job done. The problem at the moment is trying to figure out where this sort of thing should live so that everyone can use it. We need to discuss this a bit.

There are a few other utilities we need in order to debug fcl files. One of these is something that will tell you the fcl file in which the final value of a parameter is set. This is necessary because you can overwrite parameters without any warning or complaint from the parser. The art team had a utility that will help provide input to such a script, but do not have exactly this.

Thanks for poking me on this. It's so hard to figure out what is going on in a chain of fcl includes, we really need a set of good utilities asap.

So more to come.


#3 Updated by Wesley Ketchum about 6 years ago

I figured somebody would have something like this already!

As to where-to-implement/location ... it's probably not the prettiest, but we could just make a file with shell functions that one could source. Source-ing that file could be added to "" and the lbne setup script. It could be put in the setups directory of the main larsoft repository? It would just require that script being included in the larsoft install then.

We would have an immediate use for something like this, so I would hesitate to let the perfect solution outdo a decent one...

#4 Updated by Erica Snider about 6 years ago

Completely agree on the last point. Let me think about it a bit.

#5 Updated by Gianluca Petrillo about 6 years ago

I attach the script I use... which, for the principle of the duplication of the wheels, is based on the same core as Erica's.
Run with '--help' for options.

#6 Updated by Erica Snider about 6 years ago

Hi Wes,
I just spoke to the art guys. They pointed me to two things, neither of which is quite what we need, but both of which are helpful until something better comes along.

  1. If you "export ART_DEBUG_CONFIG=<filename>" before running lar, then art will create the file <filename> that contains the final FHICL configuration. In other words, it will create a single fcl file that will have the same result as the one you provided on the command line, but without any includes or prologs.
  2. There is a command in cetlib called 'inc-expand' that will print out an entire fcl include chain as if it were one ginormous file. (The cetlib product gets set up when you set up uboonecode or larsoft.) The only problem with the current implementation is that (a) it does not print the filename when it drops into or pops out of an include file, and (b) it does not seem to work in the version that gets set up for uboonecode v1_00_02. The demo in Marc Paterno's office worked fine, so maybe I'm just doing something wrong...

The art guys might be willing to fix inc-expand so that it prints file names. We could also try to write a perl script that does this. I don't think it would be hard, but there might be something I'm not considering.

A second tool that would be useful is one that would just tell you where the final value of some parameter is defined. Getting the details of this correct is hard because it requires parsing the fcl in order to determine scope, but simple searches might still be useful. It would be easy to write a perl script to do this. We could then assess whether the error rate was too high to be of use.


#7 Updated by Gianluca Petrillo about 6 years ago

(an updated version with grep abilities)

#8 Updated by Lynn Garren over 5 years ago

We suggest a larutil product which will contain useful scripts. Some existing scripts may migrate from larsoft to larutil. larutil will be one of the few products which is declared current.

#9 Updated by Ruth Pordes over 4 years ago

  • Status changed from Assigned to Feedback

Need for this needs to be reassessed. Is provided by fichl art authors maybe - is there documentation?

#10 Updated by Wesley Ketchum about 3 years ago

A number of tools exist now:
  • fhicl-expand
  • fhicl-dump
  • (in ubutil)

Annotate options on the first two help locate fhicl files.

I think this can be closed.

#11 Updated by Katherine Lato about 3 years ago

  • Status changed from Feedback to Closed

Also available in: Atom PDF