November 2017 too many spills per subrun issue

the lariat offline - online common package is lariatfragment

Lariat offline is currently using art v2_08_04 e14, and artdaq_core v1_07_12.

The online is currently using art v2_01_02 e10
Bill and I will assume we can use

The the lariatonline point-of-view, the SW packages starting point is:

[lariat@lariat-gateway00 ~]$ ups active
Active ups products:
art               v2_01_02        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
artdaq            v1_13_02        -f Linux64bit+2.6-2.12  -q debug:e10:eth:s35 -z /grid/fermiapp/products/lariat/
artdaq_core       v1_05_07        -f Linux64bit+2.6-2.12  -q debug:e10:s35   -z /grid/fermiapp/products/lariat/
artdaq_mfextensions  v1_01_01        -f Linux64bit+2.6-2.12  -q debug:e10:s35   -z /grid/fermiapp/products/lariat/
artdaq_utilities  v1_00_09        -f Linux64bit+2.6-2.12  -q debug:e10:s35   -z /grid/fermiapp/products/lariat/
boost             v1_61_0         -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
caencomm          v1_2a           -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/products/lariat/
caendigitizer     v2_9_1          -f Linux64bit+2.6-2.12                     -z /home/nfs/lariat/local_product_installation
caenvme           v2_50           -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/products/lariat/
canvas            v1_04_02        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
cetlib            v1_18_02        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
cetpkgsupport     v1_10_02        -f NULL                                    -z /grid/fermiapp/products/lariat/
clhep             v2_3_2_2        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
cppunit           v1_12_1d        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
fftw              v3_3_4          -f Linux64bit+2.6-2.12  -q debug           -z /grid/fermiapp/products/lariat/
fhiclcpp          v3_20_01        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
gcc               v4_9_3a         -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/products/lariat/
lariatfragments   v06_53_00daq    -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
lariatonline      v05_00_00       -f Linux64bit+2.6-2.12  -q debug:e10       -z /home/nfs/lariat/local_product_installation
libxml2           v2_9_3          -f Linux64bit+2.6-2.12  -q debug           -z /grid/fermiapp/products/lariat/
messagefacility   v1_16_28        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
mpich             v3_2_0          -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
python            v2_7_11         -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/products/lariat/
qt                v5_6_1a         -f Linux64bit+2.6-2.12  -q e10             -z /grid/fermiapp/products/artdaq
root              v6_06_04a       -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
sqlite            v3_12_02_00     -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/products/lariat/
tbb               v4_4_3a         -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
TRACE             v3_06_07        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
ups               v5_2_0          -f Linux64bit+2.6-2.12                     -z /grid/fermiapp/larsoft/products
xmlrpc_c          v1_39_07        -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/
xrootd            v3_3_4e         -f Linux64bit+2.6-2.12  -q debug:e10       -z /grid/fermiapp/products/lariat/

Going up to art v2_08_04 would mean:

woof :^) ups depend "art" "v2_08_04" -q"e14:prof" 
art v2_08_04 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
|__cetpkgsupport v1_13_00 -f NULL -z /products -g current
|__canvas v3_00_03 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |__messagefacility v2_01_01 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |__fhiclcpp v4_06_01 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |  |__cetlib v3_01_01 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |  |  |__cetlib_except v1_01_03 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |  |  |  |__gcc v6_3_0 -f Linux64bit+3.10-2.17 -z /products
   |  |  |  |__boost v1_64_0 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |  |  |__sqlite v3_16_02_00a -f Linux64bit+3.10-2.17 -z /products
   |  |  |__python v2_7_13d -f Linux64bit+3.10-2.17 -z /products
   |  |__tbb v2017_7 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |__root v6_10_04d -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |  |__fftw v3_3_6_pl2 -f Linux64bit+3.10-2.17 -z /products -q prof
   |  |__libxml2 v2_9_4b -f Linux64bit+3.10-2.17 -z /products -q prof
   |  |__xrootd v4_5_0c -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |__clhep v2_3_4_4 -f Linux64bit+3.10-2.17 -z /products -q e14:prof
   |__cppunit v1_13_2 -f Linux64bit+3.10-2.17 -z /products -q e14:prof

With knowledge that the latest artdaq_core v3_00_00 should allow for reading in the v1_07_12 environment, we
will proceed with updating to the latest software: artdaq v2_03_03 - artdaq_core v3_00_00.

The lariat daq HW interface code produce one data blob per spill. As this travel through the BoardReader, it
eventually gets broken into 4 Fragments and the eventbuilder is configure to have 4 fragments per event.

The bottom line is there should be one file per spill and this will also mean each spill is a subrun.

This is also considered an event -- each spill.

The data per spill (data per file) -- so the file sizes can range for .5 GB to 2 GB.
There was a limit when writing the root files --- we don't know if this still exists -- I will do a quick test...

Example of the issue as seen by looking at output files:

[lariat@lariat-daq00 daqdata]$ ls -lt *.root dropbox/*.root | grep 'Nov 16 17:'
-rw-r--r-- 1 lariat    t-1034  133589417 Nov 16 17:13 RootOutput-609d-89a5-a53f-ac8f.root
-rw-r--r-- 1 lariat    t-1034      78567 Nov 16 17:12 lariat_r-_sr-_8_8_8_8_8.root
-rw-r--r-- 1 lariat    t-1034  267297872 Nov 16 17:12 dropbox/lariat_r014306_sr0004.root
-rw-r--r-- 1 lariat    t-1034      78567 Nov 16 17:10 lariat_r-_sr-_6_6_6_6_6_6.root
-rw-r--r-- 1 lariat    t-1034  267962688 Nov 16 17:10 dropbox/lariat_r014306_sr0003.root
-rw-r--r-- 1 lariat    t-1034      78567 Nov 16 17:08 lariat_r-_sr-_4_4_4_4_4.root
-rw-r--r-- 1 lariat    t-1034  267319931 Nov 16 17:08 dropbox/lariat_r014306_sr0002.root
-rw-r--r-- 1 lariat    t-1034      78567 Nov 16 17:06 lariat_r-_sr-_2_2_2_2_2_2_2_2_2.root
-rw-r--r-- 1 lariat    t-1034  190495565 Nov 16 17:06 dropbox/lariat_r014306_sr0001.root