NOvA offline 3Sep2010


1. build system - SRT-like
2. features of current FMWK
3. database

For the frameworks in NOVA. Timescale important. Resigned to the fact that they will
want to switch (average vote).

Issue 1 - build system

ability of private / public contexts. Like the base / development release concept.
code generation step is needed to do the ROOT.
Demand is that the SRT-like system in place.
How about early next week?
Can larsoft be done first?
They have ~50 or so packages in NOvA.

We described the new way that we will be managing software using a more
fragmented approach. set up more independent units of code.

Can we connect a person into the SRT problem in order to more forward quickly?

Need an SRT build-like system for NOvA to handle the offline build system.

Issue 2a

update of configuration interactively. want interactive debugging of algorithms,
including interactive control of the event loop. FMWK has this facility (control
of event loop and configuration parameters). Mark uses and likes this feature.
The modules that are active are fixed.
Mostly control of event to be processed and reconfiguration.

Reconfiguration is not so difficult.
Provenance is not interesting here because it is for developing algorithms.

Mark - finds 20 events that are good for algorithm evaluation and configuration
tuning. Tweak the configuration until the algorithms do the right thing for the 20
events, then hit "save the current configuration", which emits a file that can be reused.
This is interactive configuration building.
FMWK environment has two executables, one for event display mode and one for production.

Issue 2b

inheritance within products - be careful here

upcast on read. They want queries of products in their base class form.
They want modules to take the base class form of the object and process them.
Want products to be queryable by their base type.

Desire to use inheritance for composition purposes only - no virtual functions allowed!
We talked about the problems of doing this from the persistence problem - slicing
and versioning.
Concern is for the large load of everything. Event display wants to see the base class
only with no evidence of the derived class.

ROOT allow for overlays of read data onto base classes with no knowledge of derived
classes. This a low-level method of slicing.

watch out - our current model is to load all the dictionaries upon start-up. All the
dict object libraries have a dependency on the data object library.

We do not have an operation that can go through many collections and get
the collected items without knowing the collection types or the item types within
those collections.

Marc asked about using the ROOT files without experiment libraries. Brian and Mark
say that the data libraries are necessary.

Rob points out that the data libraries are needed to do processing.

Issue 2c

I/O file formats - DAQ and offline. Need to be able to do transformations from
binary records in a simple file into real EDM objects. The raw data comes in
the form of vectors of int that map into simple structs.

Issue - databases

What database interactions are needed?
calibration / alignment data, beam spill, conditions data.
assume one central server here at Fermilab.
validity contexts are used. data keyed by run/subrun and data keyed by
timestamp. the result is just a large block of rows.

Are all the rows uniform with numbers in columns?
Do all constants come in?

Size of calibration is about 300,000 channels * 5 alignment number * 5-7 numbers
for calibration information.
Changes on a run by run basis, no more than hours (maximum rate of change),
weekly changes probable. ROOT file will probably have the calibration numbers
in it, but the database will be used to identify everything and possibly be the source
of the data in the ROOT calibration file.

get the spill information out of the database.
correlate the beam using GPS information.

Need further discussion.
Other systems they used had prefetching capabilities because the database acces
is slow.
Most of the data is the near detector.

described the database tables in XML.
could read calibration data from flat files so DB was not necessary.

schema release tied to software release (released together).

Marc mentioned the frontier system.