Project

General

Profile

Document build system

1 Overview

The lbne-sci-opp document can be built in an automated fashion. The build has two independent layers:

plots
Everything can be built at top-level which involves running ROOT, Python and other programs in order to rebuild some of the plots. The plots are "installed" such that they can be used by the next layer. The directories holding the plotting code are at top level (but do not include ./lbne-sci-opp/ which holds the document itself)
doc
A subset of the full build is just running the usual LaTeX tools (pdflatex). This can be done independently of the larger build and involves just the ./lbne-sci-opp/ sub-directory.

2 Checking out the plotting code and document

A full checkout is required to get the plotting code and the document itself.

$ svn checkout svn+ssh://p-lbne-sci-opp@cdcvs.fnal.gov/cvs/projects/lbne-sci-opp

Take note that the document LaTeX is under the doubly named sub-directory:

lbne-sci-opp/lbne-sci-opp/

3 Building the plots automatically with waf

The plots can be built in an automated fashion with the included waf tool. The steps to do this are:

  1. Waf is included and needs to be run once to unpack itself:
$ ./waf --version
  1. Configure the build with a command like:
$ ./waf --prefix=lbne-sci-opp --subdirs=bv,etw configure
  1. Process the subdirs to produce plots
$ ./waf
  1. Check the "tmp/<subdir>" directory to assure plots are good.
  2. Finally, install them into where the document expects to find them
$ ./waf install

This install step will place each subdir of plots into a specific subdir

lbne-sci-opp/<subdir>fig/
  1. Build the document (see below)
  2. When all is okay, commit.

Information on Adding plotting code is given below.

4 Building the document automatically with waf

To build the document itself, check out the repository as above. Then perform the following steps:

  1. Go into the sub-directory holding the document LaTeX files:
$ cd lbne-sci-opp/lbne-sci-opp/
  1. Perform a one-time unpacking if this was not yet done:
$ ../waf --version
  1. Configure the things. This is needed only once or if ever the wscript file changes
$ ../waf configure
  1. Finally build:
../waf

When rebuilding is needed after the LaTeX or other content changes simply rerun this last command.

The resulting document will be found at:

tmp/lbne-sci-opp.pdf

4.1 Draft document

A draft version of the document may also be built. This should build more quickly at the expense of some features and the inclusion of elements like line numbers. To build this version add the --draft flag to waf:

$ ../waf --draft configure build

The resulting document will be found at:

tmp/lbne-sci-opp-draft.pdf

4.2 Producing the arXiv submission

After one produces the desired results, the tar file for submission to the arXiv can be produced with:

$ waf distcheck

FIXME: this works but the result is not yet tested against an actual arXiv submission.

4.3 Troubleshooting

When using waf the output is terse and when something goes wrong it may not be immediately obvious what is broken. Some ways to figure this out:

  • Check the associated tmp/*.log file
  • Run waf in verbose mode by adding -vvv (not always useful)
  • Run the failing command manually. This is best done inside the tmp/ directory and not in the source area.

A clean build area can be made with:

$ waf clean

To fully nuke build files which will then require a new configure step one can do:

$ waf distclean configure

5 Manual document build

Note, what is described in this section is essentially what the automated build, described above, performs.

5.1 Building the bibliography file

The arXiv will not run bibtex for us but does accept .bbl files. A special version of the main document is used to generate the needed lbne-sci-opp-bib.bbl file.

$ pdflatex lbne-sci-opp-bib
$ bibtex lbne-sci-opp-bib

Note: the -draft (single-dash) option may be given to pdflatex to slightly speed up this step.

5.2 Building the "glossy" version

After this .bbl file is generated the "real" document may be built with the normal double-pump of pdflatex.

$ pdflatex lbne-sci-opp
$ pdflatex lbne-sci-opp

5.3 Building the "draft" version

To manually build the draft version, first assure the lbne-sci-opp-bib.bbl file has been built as described in 5.1. Then the draft version is build similarly as the "glossy" version but using a different main file:

$ pdflatex lbne-sci-opp-draft
$ pdflatex lbne-sci-opp-draft

5.4 Producing the arXiv submission

No manual support exists for producing the arXiv tar file.

6 Adding plotting code.

Add plotting code requires these steps:

  • pick a name for the set of plots (suggest the author's name or the domain of interest)
  • make a top-level directory under the top-level lbne-sci-opp/ using this name
  • populate the sub-directory with the plotting code and adapt it to the styling
  • instead of reproducing the plots and .root or .C file holding a TCanvas can be the starting point
  • write a wscript_build file to hook it into the greater build
  • add the sub-directory to the subdirs list in the top-level wscript file
  • commit

Some of these steps are explained more in the following sections

6.1 Plotting code

It's best if your code requires minimal processing time. It should not do much more than layout a canvas with pre-existing histograms, graphs, etc. If using ROOT, these histograms may be committed to the repository as a .root file (see ./etw/) or an entire TCanvas can be stored in a .C file (see ./blake/). Be conscious of bloating the repository with overly large files and/or generated files which have not yet become stable.

Your code must run following these requirements:

  • It must be able to run in a directory different from its source directory
  • Any input files and the output plot file(s) must be specified on a shell command line
  • The default document style should be applied
  • Each TCanvas to print must be "washed" as described below.

A Python "shim" program may be provided to assure this.

6.2 Plot styling

An attempt to centralize styling is done using the lbneplot Python package. So far, it supports only ROOT plots. It works by providing:

  • a default ROOT TStyle set to gStyle
  • a "washing" mechanism to apply to ROOT TCanvas and associated objects
  • a set of semantically named colors

6.2.1 Washing

Not all style elements can be set via gStyle and must be applied after objects are created and maybe Draw()'n. To handle this, all TCanvas should be "washed". This will descend the graphical hierarchy looking for objects that should have various fonts, sizes, colors or fill styles applied. To perform a washing one uses code like:

from lbneplot import style, wash
canvas = ...  # get from somewhere
ds = style.document_style()
wash.canvas(canvas, ds)

Examples of this usage are here and here.

6.2.2 Python vs. C++

Washing is done with Python code. If the plotting code is written in C++ and it can be arranged to produce a single TCanvas instance then it can still be "washed". See ./blake/washer.py as an example. If C++ code can not be made to fit this pattern something will be done.

6.2.3 Semantic styles

In order for different plots to use consistent style for a given semantic element (eg, all νμ plots using the same line thickness and color) a set of style elements indexed by semantic names is provided.

FIXME: t.b.d.

6.3 The wscript_build file

This file tells waf how to build and install the plots. Examples are here and here.

To run a build command one uses

bld(rule=..., source=..., target=...)
source
a list of input files
target
a list of output files
rule
how to turn input into output

The rule can be either a string giving the command to run or it can be a Python function. In the former case one can access the source and target list via, for example ${SRC[0].abspath()}. In the later, the function takes a single task argument which has task.inputs and task.outputs lists. In both cases the "files" are waf nodes. See the waf book and ref docs for details.

Installing files is done like:

bld.install_files('${PREFIX}/NAMEfig', 'the_plot.pdf')

Replace NAME with the name of the plot source subdirectory. The the_plot.pdf is the file to install and this can also be a list. When final installation is done as above this file can be refered to in the LaTeX with something like:

\includegraphics[width=\textwidth]{NAMEfig/the_plot.pdf}

7 Author list

The author list is held in ./lbne-sci-opp/lbneauthor-payload-authblk.tex and is retrieved from:

https://lbne.bnl.gov/web/members/export/lbneauthor-payload-authblk.tex

This is file is generated on the fly based on a database. It is important that any fixes be made to the database and this file re-retrieved. Do NOT make hand-edits to this file.

8 Updating this file

This file exists in the lbne-sci-opp Redmine project's document repository and is installed into its HTML tab. To update it, check out the repository, load the file into emacs with org-mode installed and do:

C-cC-ehh

Then copy the resulting HTML file to Redmine with:

$ scp README_build.html p-lbne-sci-opp@cdcvs.fnal.gov:html/index.html

Author: Brett Viren

Created: 2014-02-12 Wed 15:22

Emacs 24.3.1 (Org mode 8.2.3c)

Validate