Project

General

Profile

Calibration tags » History » Version 45

Version 44 (Keith Matera, 01/20/2017 02:36 PM) → Version 45/49 (Keith Matera, 03/01/2017 12:07 PM)

h1. Calibration tags

Official location of backup calibration CSV files in the case of database failures (starting Apr 25 2016):

*/nova/ana/calibration/calibcsvs/*

Official location of (outmoded) backup calibration CSV files (pre-Apr 25 2016; left in place for now in case anyone's code relies upon it):

*/nova/ana/calibration/csvs/*

Self-explanatory directory structure.

Details on how to create a calibration UPS product are given in this wiki page: [[calibration-tagging]]. Input files for calibration can be monitored "here":http://nusoft.fnal.gov/nova/production/sandbox/d3_raw2root_ana.html for artdaq files and "here":http://nusoft.fnal.gov/nova/production/sandbox/d3_pclist_ana.html for pc-list files.

h2. Latest full production versions (prod2)

| |FarDet|NearDet|NDOS |
|Data|v9 |v9 |Legacy|
|MC |v9 |v9 |Legacy|

(Setting a newer tag falls back to an older version where necessary)

The results of latest attenuation fits are displayed for each channel on this "webpage":http://nusoft.fnal.gov/nova/calibration/

h2. Latest mini-production versions (prod3 miniproduction)

| |FarDet|NearDet|NDOS |
|Data|v10 |v10 |Legacy|
|MC |v10 |v10 |Legacy|

h2. Legacy (pre Validity Database)

* Attenuation constants for NDOS data and MC, ND and FD MC
* Absolute calibration hardcoded into Calibrator
* MC calibrations are for untuned MC
* No corrections for threshold effects applied
* Exact datasets used lost in the mists of time

h2. v1

* Attenuation and absolute calibration constants for FD data and MC
* No drift constants
* MC calibrations are for untuned MC
* Data calibration are for diblocks 1 and 2, from runs 11496-11555 (cold, high gain), using Kanika's good runs list
* Threshold effect corrections included, but taken from untuned MC
* Confusion around updated PE/ADC conversions leads to 20% shift in absolute scale

h2. v2

* Attenuation and absolute constants for FD MC
* No drift constants, no data constants
* Calibrations are for tuned MC (S14-03-05)
* Datasets in SAM: "prodcalib_S14-03-05_FDCry_pclist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prodcalib_S14-03-05_FDCry_pclist%20%29 "prodcalib_S14-03-05_FDCry_pcliststop":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prodcalib_S14-03-05_FDCry_pcliststop%20%29
* Reco/True flat to ~1% except beyond +/-700cm, approx 2% low of unity

h2. v3

* MC constants unchanged from v2
* New attenuation and absolute calibration constants for FD data
* Attenuation using corrections determined from v2 tuned MC
* Constants for the first 4 diblocks based on data from run 13153 (Feb 15 2014) - 14029 (March 18 2014)

* Datasets in SAM: "prodcalib_S14-03-24_FD_data_cosmic_pclist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pclist%20)%20 and "prodcalib_S14-03-24_FD_data_cosmic_pcliststop":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pcliststop%20)%20
* Snapshots: "prodcalib_S14-03-24_FD_data_cosmic_pclist_snap14-03-28":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pclist_snap14-03-28%20)%20 and "prodcalib_S14-03-24_FD_data_cosmic_pcliststop_snap14-04-09":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pcliststop_snap14-04-09%20)%20
* PCList Dimensions: _file_type importedSimulated and data_tier pclist and simulated.generator cry and nova.label beta and calibration.base_release 'S14-03-06'_ and the snapshot id is 15034
* PCListStop Dimensions: _ischildof: (isparentof: (snapshot_id 15034)) and data_tier pcliststop and calibration.base_release S14-03-24"_ and the snapshot id is 15191

h2. v3.1

Same as v3, but with "averaged" data attenuation constants included, for use in diblocks that are not yet calibrated.

h2. v4

h3. FD data

Data from runs 14592 (start of 10Hz triggering) to 15585 (last run before P1 has diblocks missing from the middle)

* Datasets in SAM: "prodcalib_S14-03-24_FD_data_cosmic_pclist_for_v4":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pclist_for_v4%20)%20 and "prodcalib_S14-03-24_FD_data_cosmic_pcliststop_for_v4":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pcliststop_for_v4%20)%20
* Snapshots: "prodcalib_S14-03-24_FD_data_cosmic_pclist_for_v4_snap14-05-11":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pclist_for_v4_snap14-05-11%20)%20 and "prodcalib_S14-03-24_FD_data_cosmic_pcliststop_for_v4_snap14-05-11":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-03-24_FD_data_cosmic_pcliststop_for_v4_snap14-05-11%20)%20

The processed attenprof files: "S14-06-09_FD_data_attenprofs_for_v4":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20S14-06-09_FD_data_attenprofs_for_v4%20)%20

Absolute calibration and averaged constants carried over from previous versions

h3. ND MC

* Datasets in SAM: "prodcalib_S14-05-12_ndcry_pclist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-05-12_ndcry_pclist%20)%20 and "prodcalib_S14-05-12_ndcry_pcliststop":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prodcalib_S14-05-12_ndcry_pcliststop%20)%20

Absolute energy scale determined "by hand" using reco/true of simulated cosmic ray hits because the production group are in a hurry.

h2. v5

h3. ND data

Data from runs 10408 (full cooled detector) to 10493 (most recent processed at time of calibration)

* Datasets in SAM: "prod_pclist_S14-09-29_nd_DDActivity1_keepup_snapshot25oct2014_forv5":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prod_pclist_S14-09-29_nd_DDActivity1_keepup_snapshot25oct2014_forv5%20)%20 and
"prod_pcliststop_S14-09-29_nd_DDActivity1_keepup_snapshot25oct2014_forv5":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prod_pcliststop_S14-09-29_nd_DDActivity1_keepup_snapshot25oct2014_forv5%20)%20

The processed attenprof files: "S14-10-15_ND_data_attenprofs_for_v5":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20S14-10-15_ND_data_attenprofs_for_v5%20)%20

h3. ND MC

* Datasets in SAM: "prod_pclist_S14-09-29_nd_cry":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prod_pclist_S14-09-29_nd_cry%20)%20 and "prod_pcliststop_S14-09-29_nd_cry":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prod_pcliststop_S14-09-29_nd_cry%20)%20

The processed attenprof files: "S14-09-29_ND_mc_attenprofs_for_v5":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20S14-09-29_ND_mc_attenprofs_for_v5%20)%20

h2. v5.1

Same as v5 except that ND data absolute constants are set to be those for (v5) ND MC. This should resolve the 9% shift between data and MC that's observed in the Near Detector. The underlying problem is the selection of a lot of showering events in the ND data (docdb 12435). This is only a temporary patch until that is fixed.

h2. v6

h3. ND data

Update to attenuation calibration only to fit cell-by-cell in the main body of the detector and to keep averages only for the muon catcher.

Data from runs 10408 (full cooled detector) to 10602 (most recent processed at time of calibration)

* Datasets in SAM: "prod_pclist_S14-09-29_nd_DDActivity1_keepup_snapshot_dec92014":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20prod_pclist_S14-09-29_nd_DDActivity1_keepup_snapshot_dec92014%20)%20

h2. v7

Updates:
New cosmic muon selection for pchit files to improve the selection of cosmic muons dramatically.
New attenuation formula representing a small correction of the original.
New fit strategy for the muon catcher where fits are forced to converge using lowess.
FD MEU values found using Gaussian method before and mean now.

h3. ND data

* Datasets in SAM:
"ndpchitdata_v7_snap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20ndpchitdata_v7_snap%20)%20
"ndpcliststop_data_v7_snap24Mar15":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20ndpcliststop_data_v7_snap24Mar15%20)%20

h3. ND mc

* Datasets in SAM:
"ndpchitmc_v7_snap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20ndpchitmc_v7_snap%20)%20
"ndpclist_stop_mc_v7_snap24mar15":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20ndpclist_stop_mc_v7_snap24mar15%20)%20

h3. FD data

* Datasets in SAM:
"fddatapclist_v7_snap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20fddatapclist_v7_snap%20)%20
"fddatapcliststop_v7_snap24Mar15":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20fddatapcliststop_v7_snap24Mar15%20)%20

h3. FD mc

* Datasets in SAM:
"fdmcpclist_v7_snap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20fdmcpclist_v7_snap%20)%20
"fdmc_pcliststop_v7_snap24Mar15":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20(%20defname%3A%20fdmc_pcliststop_v7_snap24Mar15%20)%20

h2. v7.1

Identical to v7 except the ND data absolute energy scale has been adjusted. Data hits will now be 3.7% higher in energy.

See "docdb 13355":http://nova-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=13355 for the source of this number (+3.7%=1/0.964). Other calibration cross-checks show values scattered around a similar shift ("docdb 13373":http://nova-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=13373 for an overview).

This is the calibration version used for the First Analysis.

h2. v8

These are the calibrations produced prior to the second analysis or prod2. These calibrations are the first since a major re-tune of the simulated light levels in the detector. As a result the final calibrated energy for a given ADC signal has gone up by around 40%. Details can be found in "docdb-14727":http://nova-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=14727 and references therein.

For the first time this calibration includes validity periods. That is to say two FD calibrations are provided, one for low gain (100) data/MC and a second for high gain (150) data/MC. As a result new naming conventions were brought in to differentiate between periods, as was new calibrator logic. This logic is explained on this page: [[calibration-tagging]]. The CSV files that match these calibrations are in the UPS product *v08.02* (note this is not the same thing as a calibration tag - although the two are often tightly coupled). In the process of creating this product two earlier versions were made:

* v08.00 - *has the wrong FD data calibrations in and should not be used*.
* v08.01 - *has a bug in the whitespace for the absolute calibration and should not be used*.

When the dust has settled and we have a validated product these bad products will be removed.

This is the calibration used for the reco-preview files for prod 2.

h3. Mini production datasets used are as following

h3. ND data

* Datasets in SAM:
"prod_pclist_R15-11-17-miniprod.a_nd_ddactivity_period2":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.a_nd_ddactivity_period2%20%29

h3. ND MC

* Datasets in SAM
"prod_pclist_R15-11-17-miniprod.b_nd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.b_nd_cry_all%20%29

h3. FD data (low gain)

* Dataset in SAM
"prod_pclist_R15-11-17-miniprod.a_fd_cosmic_preshutdown":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.a_fd_cosmic_preshutdown%20%29

h3. FD data (high gain)

* Dataset in SAM
"prod_pclist_R15-11-17-miniprod.a_fd_cosmic_epoch3b":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.a_fd_cosmic_epoch3b%20%29

h3. FD MC (low gain)

* Dataset in SAM
"prod_pclist_R15-11-17-miniprod.b_fd_cry_all_ideal_gain100":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.b_fd_cry_all_ideal_gain100%20%29

h3. FD MC (high gain)

* Dataset in SAM
"prod_pclist_R15-11-17-miniprod.b_fd_cry_all_ideal_gain150":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R15-11-17-miniprod.b_fd_cry_all_ideal_gain150%20%29

h2. v9

These are the second analysis calibration files. They have all of the major changes included in v8. The difference with respect to v8 is that a simulation bug related to split tracks has been fixed.

h3. FD Data

All Far Detector data are divided into different periods; period 1, 2 and 3 as
described in the "Period and Epoch Naming":https://cdcvs.fnal.gov/redmine/projects/novaart/wiki/Period_and_Epoch_Naming production page. Here period
3 corresponds to the FD high gain era. The calibration has been performed separately for different periods.

Datasets used for the NOvA second analysis are,

h4. Period 1: "prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period1_quarter_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period1_quarter_goodruns%20%29
h4. Period 2: "prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period2_quarter_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period2_quarter_goodruns%20%29
h4. Period 3 epoch 3b: "prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3b_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3b_goodruns%20%29
h4. Period 3 epoch 3c: "prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3c_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3c_goodruns%20%29

h3. FD Monte Carlo

The Far Detector MC are divided into low gain and high gain era.
Datasets used for the FD MC calibration are,

h4. Low gain: "prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain100":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain100%20%29

h4. High gain: "prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain140":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain140%20%29

h3. ND Data

As in the FD, the ND data are also divided into different periods, period 2 and 3. We do not use the period 1 for Near Detector calibration because there are not enough data in that period. We use the period 2 attenuation constants for period 1 runs. Datasets for different are,

h4. Period 2: "prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_period2_v2_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_period2_v2_goodruns%20%29

h4. Period 3 epoch 3b: "prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3b_v3_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3b_v3_goodruns%20%29

h4. Period 3 epoch 3c: "prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3c_v2_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3c_v2_goodruns%20%29

h3. ND MC

A single dataset is used for ND MC second analysis calibration because of no changes in the Near Detector configurations. Dataset used for the ND MC calibration is,

h4. "prod_pclist_R16-01-27-prod2calib.e_none":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_R16-01-27-prod2calib.e_none%20%29

h2. v10 (currently: v10.01)

These are the calibrations produced prior to the third analysis or prod3. They were employed by the pre-prod3 miniproduction, which ran over the fileset defined here:

http://nusoft.fnal.gov/nova/production/miniprod2016/

h3. Major players and conveners in the v10 calibration

* Medbh Campbell (attenuation corrections) <medbh.campbell.15@ucl.ac.uk>
* Tyler Alion (absolute corrections) <tylerdalion@gmail.com>
* Pengfei Ding and Tyler Alion (fiber brightness services module) <dingpf@fnal.gov>, <tylerdalion@gmail.com>
* Keith Matera (calibration convener) <kmatera@fnal.gov>
* Ryan Nichol (calibration convener) <r.nichol@ucl.ac.uk>
* Chris Backhouse (retired calibration convener and consultant) <bckhouse@caltech.edu>
* Alex Radovic (retired calibration convener and consultant) <aradovic@wm.edu>
* Alex Himmel (production convener) <ahimmel@fnal.gov>
* Bruno Zamorano (production convener) <B.Zamorano@sussex.ac.uk>

h3. What files can this calibration version be used for

* ND MC files
* ND data files
* FD MC files (as of v10.01)
* FD data files (as of v10.01)

h3. What tags are associated with this calibration version

* In Calibrator.fcl:
** Tag: v10
** AttenEpochTag: v2.5
** AbsEpochTag: v2.5
** TimingEpochTag: v2.5
** TimingTag: tv4

* In nova-offline-ups-externals-development, install_nova_externals_development, novasoft/ups/product_deps
** Version: v10.01

h3. Validity contexts

The following validity contexts were used to generate calibration constants. Each *bolded phrase*, separated from others by a comma, is a validity context. Each validity context has its own set of attenuation and absolute calibration constants.

FD MC: Full, *Gain 100*, and *Gain 140* (two sets of constants)
ND MC: *Full (one set of constants)*

FD Data: *Epoch2b*, and *Epoch 3b* (two sets of constants)
ND Data: *Period 1*, *Period 2*, and *Period 3* (three sets of constants)

The runs covered by each set of calibration constants are given below. In numbers of the form r#-r#, if the first # is missing, that means that the low run is 0 (the earliest possible run). If the second # is missing, that means that the high run is infinity (the latest possible run).

*FD MC Gain 100*: r-r20752
*FD MC Gain 140*: r20753-r,1,20753

*ND MC*: r-r

*FD Data Epoch 2b*: r-r20752
*FD Data Epoch 3b*: r20753-r

*ND Data Period 2*: r-r11228
*ND Data Period 3*: r11229-r

N.B. that we often extrapolate constants derived in one validity context to another (e.g., the FD Epoch2b constants are used to cover FD Periods 1 & 2, and the ND Period 2 constants are used to cover ND Periods 1 & 2). This is intentional. If you suspect that these validity contexts are being extrapolated wrongly, please let your calibration convener know.

h3. Major changes introduced between v9 and v10

* The geometry gdml files for both near and far detectors were changed. The biggest effect of this is to shrink the cell interior, increasing PE/cm by ~2% ("docdb 15421":http://nova-docdb.fnal.gov:8080/cgi-bin/ShowDocument?docid=15421)
* Cherenkov light was added to the MC simulation ("docdb 16349":http://nova-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=16349&filename=CherenkovMC_dEdx_20102016.pdf&version=3 as a starting point)
* Fiber brightnesses were added to the MC simulation ("docdb 15419":http://nova-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=15419&filename=FiberBrightnessModeling_ArtImplementation.pdf&version=3 and "docdb 16347":http://nova-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=16374&filename=FibreBrightnessMC_collabmeeting2016.pdf&version=1 as a starting point)
* MC files are calibrated by fiber brightness in the following fashion:
** A single x and y plane of shadowing/threshold corrections are generated for each detector
** An x and y plane of attenuation corrections is generated for _each fiber brightness_, for each detector (9 FB values x 2 views = 18 planes per detector)
** The muon catcher is treated as its own detector for the purposes of shadowing/threshold and attenuation corrections, giving another 18 planes (bringing the total to 36 for the Near Detector)
** There is now one atten*.csv file for each fiber brightness in the MC, _e.g._, from calib_atten_consts.nd.mc.v10.r-r.csv --> calib_atten_consts.nd.mc.v10.r-r.fb{0-8}.csv. This reflects that we now load a vector of AttenCache objects to apply the attenuation calibrations, each of which is represented by its own set of constants. See /grid/fermiapp/products/nova/externals/calibcsvs/v10.00/NULL/csv/v10/ for a full set of .csv files.
* Data is calibrated in the same fashion as in earlier calibrations

h3. Hacks, deviations, and exceptions to the above *Major changes* section

* The muon catcher was calibrated using the v9 calibration constants. The v9 muon catcher attenuation constants files (just one, for each the x and y views, giving a total of 2 atten_consts and atten_points files) were copied over 9 times, and labeled according to the fb labeling described in the previous section (changing .csv --> .fb{0-8}.csv)

h3. Changes between v10.00 (Dec 2016) and v10.01 (Jan 2016)

* FD data calibration constants added, allowing for the calibration of FD data and FD MC files. (Why FD MC files only now? The FD MC cosmics overlay files use data hits, which must be calibrated using data calibration constants.)

h3. Plans in the works for Feb 2017 and beyond

* Make new Calibration/Calibrator code backwards compatible with pre-v10 calibration constants files
* Update muon catcher calibration constants to use v10 constants rather than v9 constants
* Reconsider using separate threshold/shadowing corrections for different fiber brightness values



h2. v11 (currently v11.02) (speculative)

h3. The v11 calibration product does not exist yet


This section details the expected properties of the v11 calibration product. The generation of the v11 calibration product has not yet begun. The generation of v11 calibration constants has not yet begun. The generation of v11 calibration constants has not yet begun. Did you hear that? The generation of v11 calibration constants has not yet begun. If you are confused into trying to use the v11 calibration constants, it is your own fault.

h3. When the v11 calibration product does exist

These are the calibrations produced for the 2017 analysis production cycle, or prod3. They were employed by the prod3calib production cycle, which ran over the calibration samples defined here:

http://nusoft.fnal.gov/nova/production/miniprod2016/

h3. Major players and conveners in the v11 calibration

* Medbh Campbell (attenuation corrections) <medbh.campbell.15@ucl.ac.uk>
* Tyler Alion (absolute corrections) <tylerdalion@gmail.com>
* Keith Matera (calibration convener) <kmatera@fnal.gov>
* Ryan Nichol (calibration convener) <r.nichol@ucl.ac.uk>
* Chris Backhouse (retired calibration convener and consultant) <bckhouse@caltech.edu>
* Alex Radovic (retired calibration convener and consultant) <aradovic@wm.edu>
* Alex Himmel (production convener) <ahimmel@fnal.gov>
* Bruno Zamorano (production convener) <B.Zamorano@sussex.ac.uk>

h3. What files can this calibration version be used for

* ND MC files
(expected starting v11.00)
* ND data files (expected starting v11.00)
* FD MC files (expected starting v11.01)
* FD data files (expected starting v11.01)

h3. What tags are associated with this calibration version

* In Calibrator.fcl:
** Tag: v11
** AttenEpochTag: v2.6 v2.5
** AbsEpochTag: v2.6 v2.5
** TimingEpochTag: v2.5
** TimingTag: tv4

* In nova-offline-ups-externals-development, install_nova_externals_development, novasoft/ups/product_deps
** Version: v11.02

v11.00

h3. Validity contexts

The following validity contexts were used to generate calibration constants. Each *bolded phrase*, separated from others by a comma, is a validity context. Each validity context has its own set of attenuation and absolute calibration constants. The runs covered by each set of calibration constants are provided after the colon. In numbers of the form r#-r#, if the first # is missing, that means that the low run is 0 (the earliest possible run). If the second # is missing, that means that the high run is infinity (the latest possible run).

* *ND Data Periods 1&2*: r-r11228
* *ND Data Period 3*: r11229-r11628
* *ND Data Period 4*: r11632-r11925
* *ND Data Period 5*: r11926-r

* *ND MC*: r-r

* *FD Data Period 1&2: r-r20752* 1*: r-r17890
* *FD Data Period 2*: r17891-r20752
* *FD Data Period
3*: r20753-r23419
* *FD Data Period 4*: r23420-r24613
* *FD Data Period 5*: r24614-r

* *FD MC Gain 100*: r-r20752
* *FD MC Gain 140*: r20753-r

Each validity context defined above uses all events which are part of the calibration samples defined here: http://nusoft.fnal.gov/nova/production/miniprod2016/ , and which also that occur within that context. This calibration process was not statistics-limited -- if we use less than a full period's worth of a dataset, it is because we only needed that many events. We then extrapolate to cover the full period, assuming consistency in detector behavior. If you disagree with a particular use of this extrapolation, please contact a Calibration convener with specific references (pages included) to the documentation that you believe supports abandoning this strategy.

h3. Major changes introduced between v10 and v11

* Shadowing and threshold corrections are now divided by fiber brightness
** MC files are calibrated by fiber brightness in the following fashion:
*** An x and a y plane of shadowing/threshold corrections are generated for _each fiber brightness_, for each detector (9 FB values x 2 views = 18 planes per detector)
*** An x and a y plane of attenuation corrections is generated for _each fiber brightness_, for each detector (9 FB values x 2 views = 18 planes per detector)
*** The muon catcher is treated as its own detector for the purposes of shadowing/threshold and attenuation corrections, giving another 18 planes (bringing the total to 36 for the Near Detector)
*** There is still one atten*.csv file for each fiber brightness in the MC, _e.g._, from calib_atten_consts.nd.mc.v10.r-r.csv --> calib_atten_consts.nd.mc.v10.r-r.fb{0-8}.csv.
** Data is calibrated in the same fashion as MC calibrations, with these exceptions:
*** Each cell's attenuation corrections are generated individually. This does not apply to shadowing and threshold corrections, which are MC-driven and applied via mapping to a single x and y plane of corrections, as described above

* The range for muon-catcher attenuation fits has been corrected
** This means that muon-catcher attenuation correction functions can be fitted more reliably, without resulting in large numbers of uncalibrated channels due to poor fits.

* Absolute Drift calibration now derives constants for x and y view independently corrections have been added
** And then takes This should ensure more uniform FEB response over time (correcting for hardware swaps, hardware deterioration, DSO scans changing the mean as the calibration scale to actually apply pedestal limits, etc.)

h3. Hacks, deviations, and exceptions to the above *Major changes* section

* None expected

h3. Plans in the works for the next calibration cycle

* Drift calibration corrections may be added
** This should ensure more uniform FEB response over time (correcting for hardware swaps, hardware deterioration, DSO scans changing the pedestal limits, etc.)
Unknown