User guide » History » Version 21

« Previous - Version 21/86 (diff) - Next » - Current version
Herbert Greenlee, 01/12/2015 12:36 PM


Larsoft common batch and workflow tools are contained in ups product larbatch (this redmine), which is built and distributed as part of larsoft. Larbatch tools are built on top of Fermilab jobsub_client batch submission tools. For general information about jobsub_client and the Fermilab batch system, refer to articles on the jobsub wiki and the fife wiki.

No other part of larsoft is dependent on larbatch, and larbatch is not setup as a dependent of the larsoft umbrella ups product. Rather, larbatch is intended to be a dependent of experiment-specific ups products (see this article for instructions on configuring larbatch for a specific experiment.

After setting up ups product larbatch, several executable scripts and python modules are available on the execution path and python path. Here is a list of the more important ones.

    An executable python script that is the the main entry point for user interation. More information can be found below.
    A python module, imported by, that implements some of the workflow functionality. End users would not normally interact directly with this module. However, a significant aspect of is that is supplies hooks for providing experiment-specific implementations of some functionality, as described in an accompanying article on this wiki.
    The main batch script. is a general purpose script that manages a single invocation of an art framework program (lar executable). sets up the run-time environment, fetches input data, interacts with sam, and copies output data. It is not intended that end users will directly invoke However, one can get a general idea of the features and capabilities of by viewing the built-in documentation by typing " -h, or reading the file header.

Using is used in conjunction with a xml format project definition file. The concept of a project, as understood by, and as defined by the project definition file, is a multistage linear processing chain involving a specified number of batch workers at each stage.

The general structure of the project file is that it contains global options, a single project block (enclosed in "<project name=project-name>...</project>"), and one or more processing stage blocks (enclosed in "<stage name=stage-name>...</stage>" inside the project block. More information about project files can be found below.

In a typical invocation of, one specifies the project file (via option --xml), tha stage name (via option --stage), and one or more action options. Here are some use cases for invoking

  • -h or --help
    Print built-in help (lists all available command line options).
  • -xh or --xmlhelp
    Print built-in xml help (lists all available constructs that can be included in project definition file).
  • --xml xml-name --status
    Print global summary status of the project.
  • --xml xml-name --stage stage-name --submit
    Submit batch jobs for specified stage.
  • --xml xml-name --stage stage-name --check
    Check results from specified stage (identifies failed jobs). This action assumes that the art program produces an artroot output file.
  • --xml xml-name --stage stage-name --checkana
    Check results from specified stage (identifies failed jobs). This version of the check action skips some checks done by --check that only make sense if the art program produces an artroot output file. Use this action to check results from an analyzer-only art program.
  • --xml xml-name --stage stage-name --makeup
    Submit makeup jobs for failed jobs, as identified by a previous --check or --checkana action.
  • --xml xml-name --stage stage-name --clean
    Delete output for the specified stage and later stages. This option can be combined with --submit.
  • --xml xml-name --stage stage-name --declare
    Declare successful artroot files to sam.
  • --xml xml-name --stage stage-name --upload
    Upload successful artroot files to enstore.
  • --xml xml-name --stage stage-name --define
    Create sam dataset definition.
  • --xml xml-name --stage stage-name --audit
    Check the completeness and correctness of a processing stage using sam parentage information. For this action to work, input and output files must be must be declared to sam.

Project File Structure

Examples (Microboone)

Example project files in ubutil

Internal documentation

Refer to header of or type " --xmlhelp".

XML Entities

Global options

Project Block

Stage Blocks