Project

General

Profile

Official Datasets » History » Version 80

Matthew Tamsett, 02/17/2015 04:48 AM
Move ND cry samples from "real-detector conditions" section to ideal as they were wrongly placed.

1 11 Dominick Rocco
h1. Official Datasets
2 11 Dominick Rocco
3 60 Matthew Tamsett
This page documents the most up to date datasets produced by the production group. Information on older legacy files is kept on the [[NOvA-SAM:LegacyDatasets]] page.
4 1 Gavin Davies
5 60 Matthew Tamsett
h2. Introduction 
6 1 Gavin Davies
7 60 Matthew Tamsett
All modern NOvA data and MC resides within a data handling system called Sequential Access Metadata (SAM).  The files are stored on a tape-backed disk array called dCache.  Locations for the files are tracked using a database with a web interface, accessible through the @samweb@ command line client utility.  The [[NOvA-SAM:SAM_web_cookbook]] provides an introduction to using the system.  All of the datasets listed below are sam "definitions". These are a collection of file meta-data based logical queries which together serve to define a set of files within the SAM system.
8 1 Gavin Davies
9 60 Matthew Tamsett
As a throwback to the old days, some of the data is also stored on bluearc.  CAFs are a notable group of files of which the most recent are almost always stored on bluearc.  There are also small, selected samples of the most recent keep-up (see below) files available on bluearc in:
10 54 Gavin Davies
11 60 Matthew Tamsett
<pre>
12 60 Matthew Tamsett
/nova/prod/data/keepup/samples/ 
13 60 Matthew Tamsett
</pre>
14 1 Gavin Davies
15 1 Gavin Davies
16 60 Matthew Tamsett
Data and MC files are currently provided in two forms: datasets for the *first analysis* (*FA*) and *keep-up* datasets. First analysis dataset are those designed to be used in the first analysis and as such are processed in stable, tested versions of the software including the latest calibration and state-of-the-art reconstruction and PID algorithms. Keep-up datasets, on the other hand, contain data processed as it comes off of the detectors in as close to real time as possible. These datasets are designed to offer users an early look at data and, as such, may not always include the most up to date calibration or reconstruction. All keep-up datasets include "keepup" in their dataset names.
17 1 Gavin Davies
18 60 Matthew Tamsett
First analysis production proceeds in a stepwise manner starting with simulation and progress through reconstruction to PID. At each stage the files from the previous stage are processed to produce a final dataset of simulation, reconstruction or PID files. At the same time a set of the next stage of files are also produced for validation use. At the current time production have *mostly completed the simulation and reconstruction stages* and are producing *PID validation files*.
19 1 Gavin Davies
20 71 Matthew Tamsett
As of January 2015 production will produce two datasets for each data, dataset. One of these will have the good runs list requirement applied (dq.isgoodrun = true) and will be suffixed with "goodruns" and a second which doesn't include this.
21 71 Matthew Tamsett
22 60 Matthew Tamsett
h1. Contents
23 59 Christopher Backhouse
24 60 Matthew Tamsett
{{toc}}
25 1 Gavin Davies
26 60 Matthew Tamsett
h1. Data
27 58 Gavin Davies
28 60 Matthew Tamsett
First pass data processing for the first analysis is currently complete through the reconstruction stage, with PID validation files being produced now. The input dataset to these samples is the official first-analysis run lists generated by Ryan Patterson and stored in this directory: 
29 58 Gavin Davies
30 60 Matthew Tamsett
<pre>
31 60 Matthew Tamsett
  /nova/app/users/rbpatter/runlists/
32 60 Matthew Tamsett
</pre>
33 1 Gavin Davies
34 1 Gavin Davies
h2. FD
35 1 Gavin Davies
36 71 Matthew Tamsett
| *Data tier* | *Cosmic trigger* | *Cosmic trigger + GRL* | *NuMI trigger* |
37 71 Matthew Tamsett
| *artdaq* | "prod_artdaq_fd_cosmic_fa_runlist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_fd_cosmic_fa_runlist%20%29 | "prod_artdaq_fd_cosmic_fa_runlist_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_fd_cosmic_fa_runlist_goodruns%20%29 | "prod_artdaq_fd_numi_fa_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_fd_numi_fa_goodruns%20%29 |
38 71 Matthew Tamsett
| *pclist* | "prod_pclist_S14-09-29_fd_cosmic_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_S14-09-29_fd_cosmic_keepup%20%29 | | n/a |
39 71 Matthew Tamsett
| *pcliststop* | "prod_pcliststop_S14-09-29_fd_cosmic_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pcliststop_S14-09-29_fd_cosmic_keepup%20%29 | | n/a |
40 71 Matthew Tamsett
| *timecal* | "prod_timecal_S14-09-29_fd_cosmic_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_timecal_S14-09-29_fd_cosmic_keepup%20%29 | | n/a |
41 71 Matthew Tamsett
| *reco*   | "prod_reco_FA14-10-28_fd_cosmic_fa_runlist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_cosmic_fa_runlist%20%29 | "prod_reco_FA14-10-28_fd_cosmic_fa_runlist_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_cosmic_fa_runlist_goodruns%20%29 | "prod_reco_FA14-10-28_fd_numi_fa_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_numi_fa_goodruns%20%29 |
42 76 Matthew Tamsett
| *reco - keep up - post shutdown*   | | | "prod_reco_S15-01-12_fd_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_S15-01-12_fd_numi_keepup%20%29 |
43 76 Matthew Tamsett
| *PID - for validation*   | "prod_pid_FA14-10-28_fd_cosmic_fa_runlist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_cosmic_fa_runlist%20%29 | "prod_pid_FA14-10-28_fd_cosmic_fa_runlist_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_cosmic_fa_runlist_goodruns%20%29 | "prod_pid_FA14-10-28_fd_numi_fa_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_numi_fa_goodruns%20%29 |
44 76 Matthew Tamsett
| *CAF - for validation*   | "prod_caf_FA14-12-16_fd_cosmic_fa_runlist":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-12-16_fd_cosmic_fa_runlist%20%29 | "prod_caf_FA14-12-16_fd_cosmic_fa_runlist_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-12-16_fd_cosmic_fa_runlist_goodruns%20%29 | "prod_caf_FA15-01-16_fd_numi_fa_runlist_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-16_fd_numi_fa_runlist_goodruns%20%29 |
45 76 Matthew Tamsett
| *CAF - keep up - post shutdown*   | | | "prod_caf_S15-01-12_fd_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_S15-01-12_fd_numi_keepup%20%29 |
46 1 Gavin Davies
47 71 Matthew Tamsett
48 1 Gavin Davies
h2. ND
49 1 Gavin Davies
50 72 Gavin Davies
| *Data tier* | *Cosmic trigger* | *DD activity trigger* | *DD tricell trigger* | *NuMI trigger* |
51 1 Gavin Davies
| *artdaq* | | | | "prod_artdaq_FA14-10-03_nd_numi_fullgain_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_FA14-10-03_nd_numi_fullgain_goodruns%20%29 |
52 73 Matthew Tamsett
| *pclist* | "prod_pclist_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup%20%29 | "prod_pclist_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup%20%29 | "prod_pclist_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup%20%29 | n/a |
53 1 Gavin Davies
| *pcliststop* | "prod_pcliststop_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pcliststop_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup%20%29 | "prod_pcliststop_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pcliststop_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup%20%29 | "prod_pcliststop_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pcliststop_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup%20%29 | n/a |
54 1 Gavin Davies
| *timecal* | "prod_timecal_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_timecal_S14-09-29_nd_cosmic_raw2root_FA14-10-03_keepup%20%29 | "prod_timecal_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_timecal_S14-09-29_nd_DDActivity1_raw2root_FA14-10-03_keepup%20%29 | "prod_timecal_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_timecal_S14-09-29_nd_DDCalMu_raw2root_FA14-10-03_keepup%20%29 | n/a |
55 71 Matthew Tamsett
| *reco* | | | | "prod_reco_FA14-12-29_nd_numi_fullgain_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-12-29_nd_numi_fullgain_goodruns%20%29 |
56 76 Matthew Tamsett
| *reco - keep up - preshutdown* | | | | "prod_reco_S14-09-29_neardet_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_S14-09-29_neardet_numi_keepup%20%29 |
57 76 Matthew Tamsett
| *reco - keep up - postshutdown* | | | | "prod_reco_S15-01-12_nd_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_S15-01-12_nd_numi_keepup%20%29 |
58 1 Gavin Davies
| *pid - validation* | | | | "prod_pid_FA14-12-29_nd_numi_fullgain_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-12-29_nd_numi_fullgain_goodruns%20%29 |
59 1 Gavin Davies
| *caf - validation* | | | | "prod_caf_FA14-12-29_nd_numi_fullgain_goodruns":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-12-29_nd_numi_fullgain_goodruns%20%29 |
60 76 Matthew Tamsett
| *caf - keep up - preshutdown* | | | | "prod_caf_S14-09-29_neardet_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_S14-09-29_neardet_numi_keepup%20%29 |
61 76 Matthew Tamsett
| *caf - keep up - postshutdown* | | | | "prod_caf_S15-01-12_nd_numi_keepup":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_S15-01-12_nd_numi_keepup%20%29 |
62 73 Matthew Tamsett
63 1 Gavin Davies
64 71 Matthew Tamsett
In datasets are provided for the *NuMI*, with and without the GRL and before and after the shutdown, these follow the pattern:
65 71 Matthew Tamsett
66 71 Matthew Tamsett
<pre>
67 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain
68 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain_goodruns
69 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain_preshutdown
70 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain_preshutdown_goodruns
71 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain_postshutdown
72 71 Matthew Tamsett
prod_artdaq_FA14-10-03_nd_numi_fullgain_posthutdown_goodruns
73 71 Matthew Tamsett
</pre>
74 71 Matthew Tamsett
75 71 Matthew Tamsett
and are available for the artdaq, reco, pid - validation and caf - validation samples.
76 71 Matthew Tamsett
77 60 Matthew Tamsett
h1. Monte Carlo
78 54 Gavin Davies
79 60 Matthew Tamsett
As with data, first pass MC processing for the first analysis is currently complete through the reconstruction stage, with PID validation produced. The only exception to this are the RHC files which are still being simulated.
80 1 Gavin Davies
81 1 Gavin Davies
In order to facilitate both first analysis studies and future sensitivity studies two types of MC have been produced. Firstly, those with real detector like configurations. These files are simulated with run numbers (and hence the di-block and active channels configurations of the matching data runs)  and a PoT weighting which replicates that in the first analysis data datasets. Secondly, those with "ideal" 14-DB configurations to be used in future sensitivity studies.
82 1 Gavin Davies
83 1 Gavin Davies
h2. FD & ND MC - Real detector-like conditions 
84 1 Gavin Davies
85 64 Matthew Tamsett
| *FarDet* | *FHC swap*                                 | *FHC nonswap*                             | *FHC tau*                             | *Cosmics*                       |
86 1 Gavin Davies
| *artdaq* | "prod_daq_FA14-10-03_fd_genie_fhc_fluxswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_fluxswap%20%29  | "prod_daq_FA14-10-03_fd_genie_fhc_nonswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_nonswap%20%29  |   "prod_daq_FA14-10-03_fd_genie_fhc_tau":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_tau%20%29  | "prod_daq_FA14-10-03_fd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_cry_all%20%29  |
87 1 Gavin Davies
| *reco*   | "prod_reco_FA14-10-28_fd_genie_fhc_fluxswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_fluxswap%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_nonswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_nonswap%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_tau":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_tau%20%29 | "prod_reco_FA14-10-28_fd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_cry_all%20%29 |
88 74 Gavin Davies
| *pid*    | "prod_pid_FA15-01-12_fd_genie_fhc_fluxswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_fluxswap%20%29  | "prod_pid_FA15-01-12_fd_genie_fhc_nonswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_nonswap%20%29  | "prod_pid_FA15-01-12_fd_genie_fhc_tau":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_tau%20%29  | "prod_pid_FA15-01-12_fd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_cry_all%20%29  |
89 74 Gavin Davies
| *caf*    | "prod_caf_FA15-01-12_fd_genie_fhc_fluxswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_fluxswap%20%29  | "prod_caf_FA15-01-12_fd_genie_fhc_nonswap":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_nonswap%20%29  | "prod_caf_FA15-01-12_fd_genie_fhc_tau":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_tau%20%29  | "prod_caf_FA15-01-12_fd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_cry_all%20%29  |
90 1 Gavin Davies
91 80 Matthew Tamsett
| *NearDet preshutdown* | *FHC nonswap*                             |
92 80 Matthew Tamsett
| *artdaq* | "prod_artdaq_FA14-10-03_nd_genie_nonswap_preshutdown_downsampled":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_FA14-10-03_nd_genie_nonswap_preshutdown_downsampled%20%29 |
93 80 Matthew Tamsett
| *reco*   |  "prod_reco_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled%20%29 |
94 80 Matthew Tamsett
| *pid - validation*    | "prod_pid_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled%20%29 |
95 80 Matthew Tamsett
| *caf - validation*    | "prod_caf_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-12-29_nd_genie_nonswap_preshutdown_downsampled%20%29 |
96 64 Matthew Tamsett
97 71 Matthew Tamsett
| *NearDet postshutdown* | *FHC nonswap* |
98 79 Dominick Rocco
| *artdaq* | "prod_artdaq_FA14-10-03_nd_genie_nonswap_postshutdown":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=defname%3Aprod_artdaq_FA14-10-03_nd_genie_nonswap_postshutdown |
99 79 Dominick Rocco
| *reco*   |  "prod_reco_FA14-12-29_nd_genie_nonswap_postshutdown":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=defname%3Aprod_reco_FA14-12-29_nd_genie_nonswap_postshutdown|
100 79 Dominick Rocco
| *pid - validation*    | "prod_pid_FA14-12-29_nd_genie_nonswap_postshutdown":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=defname%3Aprod_pid_FA14-12-29_nd_genie_nonswap_postshutdown |
101 79 Dominick Rocco
| *caf - validation*    | "prod_caf_FA14-12-29_nd_genie_nonswap_postshutdown":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=defname%3Aprod_caf_FA14-12-29_nd_genie_nonswap_postshutdown |
102 71 Matthew Tamsett
103 41 Dominick Rocco
h2. FD MC - Ideal conditions (14db) 
104 63 Matthew Tamsett
105 72 Gavin Davies
| *FarDet* | *FHC swap*                                 | *FHC nonswap*                             | *FHC tau*                             |
106 72 Gavin Davies
| *artdaq* | "prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_14db%20%29  | "prod_daq_FA14-10-03_fd_genie_fhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_nonswap_14db%20%29  | "prod_daq_FA14-10-03_fd_genie_fhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_tau_14db%20%29  |
107 72 Gavin Davies
| *reco*   | "prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_14db%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_nonswap_14db%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_tau_14db%20%29 |
108 72 Gavin Davies
| *pid - validation*    | "prod_pid_FA14-10-28_fd_genie_fhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_fhc_fluxswap_14db%20%29  | "prod_pid_FA14-10-28_fd_genie_fhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_fhc_nonswap_14db%20%29  | "prod_pid_FA14-10-28_fd_genie_fhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_fhc_tau_14db%20%29  |
109 60 Matthew Tamsett
| *caf - validation*    | "prod_caf_FA14-10-28_fd_genie_fhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_fhc_fluxswap_14db%20%29  | "prod_caf_FA14-10-28_fd_genie_fhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_fhc_nonswap_14db%20%29  | "prod_caf_FA14-10-28_fd_genie_fhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_fhc_tau_14db%20%29  |
110 69 Gavin Davies
111 1 Gavin Davies
| *FarDet continued* | *RHC swap*                                 | *RHC nonswap*                             | *RHC tau*                             |
112 69 Gavin Davies
| *artdaq* | "prod_daq_FA14-10-03_fd_genie_rhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_rhc_fluxswap_14db%20%29  |  "prod_daq_FA14-10-03_fd_genie_rhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_rhc_nonswap_14db%20%29 |  "prod_daq_FA14-10-03_fd_genie_rhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_rhc_tau_14db%20%29 |
113 69 Gavin Davies
| *reco*   | "prod_reco_FA14-10-28_fd_genie_rhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_rhc_fluxswap_14db%20%29| "prod_reco_FA14-10-28_fd_genie_rhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_rhc_nonswap_14db%20%29 | "prod_reco_FA14-10-28_fd_genie_rhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_rhc_tau_14db%20%29 |
114 1 Gavin Davies
| *pid*    | "prod_pid_FA14-10-28_fd_genie_rhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_rhc_fluxswap_14db%20%29  | "prod_pid_FA14-10-28_fd_genie_rhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_rhc_nonswap_14db%20%29  | "prod_pid_FA14-10-28_fd_genie_rhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_genie_rhc_tau_14db%20%29  |
115 1 Gavin Davies
| *caf*    | "prod_caf_FA14-10-28_fd_genie_rhc_fluxswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_rhc_fluxswap_14db%20%29  | "prod_caf_FA14-10-28_fd_genie_rhc_nonswap_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_rhc_nonswap_14db%20%29  | "prod_caf_FA14-10-28_fd_genie_rhc_tau_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_genie_rhc_tau_14db%20%29  |
116 1 Gavin Davies
117 1 Gavin Davies
| *FarDet continued* | *Cosmics*                          |
118 1 Gavin Davies
| *artdaq*           | "prod_artdaq_FA14-10-03_fd_cry_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_artdaq_FA14-10-03_fd_cry_14db%20%29 |
119 1 Gavin Davies
| *reco*             | "prod_reco_FA14-10-28_fd_cry_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_cry_14db%20%29 |
120 1 Gavin Davies
| *pid*              | "prod_pid_FA14-10-28_fd_cry_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA14-10-28_fd_cry_14db%20%29 |
121 1 Gavin Davies
| *caf*              | "prod_caf_FA14-10-28_fd_cry_14db":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA14-10-28_fd_cry_14db%20%29 |
122 80 Matthew Tamsett
123 80 Matthew Tamsett
h2. ND ideal conditions
124 80 Matthew Tamsett
125 80 Matthew Tamsett
| *NearDet* | *Cosmics*                       |
126 80 Matthew Tamsett
| *artdaq*  | "prod_daq_FA14-10-03_nd_cry_all":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_nd_cry_all%20%29 |
127 80 Matthew Tamsett
| *pclist* |  "prod_pclist_S14-09-29_nd_cry":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pclist_S14-09-29_nd_cry%20%29 |
128 80 Matthew Tamsett
| *pcliststop* | "prod_pcliststop_S14-09-29_nd_cry":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pcliststop_S14-09-29_nd_cry%20%29 |
129 80 Matthew Tamsett
| *timecal* | "prod_timecal_S14-09-29_nd_cry":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_timecal_S14-09-29_nd_cry%20%29 |
130 80 Matthew Tamsett
131 70 Satish Desai
132 60 Matthew Tamsett
133 41 Dominick Rocco
h1. Supporting sample MC
134 60 Matthew Tamsett
135 41 Dominick Rocco
In addition to the core samples discussed above, some dedicated samples have been produced to study particular effects.
136 60 Matthew Tamsett
137 41 Dominick Rocco
h2. FD MC - Real detector-like conditions, geojittered.
138 60 Matthew Tamsett
139 72 Gavin Davies
| *FarDet* | *FHC swap*                                 | *FHC nonswap*                             | *FHC tau*                             |
140 72 Gavin Davies
| *artdaq* | "prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_geojittered%20%29  | "prod_daq_FA14-10-03_fd_genie_fhc_nonswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_nonswap_geojittered%20%29  | "prod_daq_FA14-10-03_fd_genie_fhc_tau_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_daq_FA14-10-03_fd_genie_fhc_tau_geojittered%20%29  |
141 64 Matthew Tamsett
| *reco*   | "prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_geojittered%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_nonswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_nonswap_geojittered%20%29 | "prod_reco_FA14-10-28_fd_genie_fhc_tau_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_reco_FA14-10-28_fd_genie_fhc_tau_geojittered%20%29 |
142 74 Gavin Davies
| *pid*    | "prod_pid_FA15-01-12_fd_genie_fhc_fluxswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_fluxswap_geojittered%20%29  | "prod_pid_FA15-01-12_fd_genie_fhc_nonswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_nonswap_geojittered%20%29  | "prod_pid_FA15-01-12_fd_genie_fhc_tau_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_pid_FA15-01-12_fd_genie_fhc_tau_geojittered%20%29  |
143 74 Gavin Davies
| *caf*    | "prod_caf_FA15-01-12_fd_genie_fhc_fluxswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_fluxswap_geojittered%20%29  | "prod_caf_FA15-01-12_fd_genie_fhc_nonswap_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_nonswap_geojittered%20%29  | "prod_caf_FA15-01-12_fd_genie_fhc_tau_geojittered":http://samweb.fnal.gov:8480/sam/nova/api/files/list?format=html&dims=%20%28%20defname%3A%20prod_caf_FA15-01-12_fd_genie_fhc_tau_geojittered%20%29  |
144 41 Dominick Rocco
145 60 Matthew Tamsett
146 41 Dominick Rocco
h1.  Processing notes
147 60 Matthew Tamsett
148 41 Dominick Rocco
This section attempts to briefly summarise issues that users should be aware of when using the above datasets. Full details on the releases used can be found on the [[NOvA-ART:History_of_Tagged_Releases]] page.
149 60 Matthew Tamsett
150 43 Matthew Tamsett
h2. FA14-10-03 Data raw2root
151 60 Matthew Tamsett
152 53 Gavin Davies
This version of the software includes the ND geometry version used in the FA MC simulation.
153 60 Matthew Tamsett
154 53 Gavin Davies
*No known issues*.
155 60 Matthew Tamsett
156 53 Gavin Davies
h2. FA14-10-03 MC Simulation
157 60 Matthew Tamsett
158 53 Gavin Davies
This is the official simulation for the first analysis datasets.
159 60 Matthew Tamsett
160 53 Gavin Davies
*No known issues*.
161 60 Matthew Tamsett
162 53 Gavin Davies
h2. FA14-10-28 FD Reconstruction
163 60 Matthew Tamsett
164 47 Dominick Rocco
This is the official reconstruction for the first analysis datasets. It was processed with v04 FD calibrations.
165 60 Matthew Tamsett
166 15 Dominick Rocco
*No known issues*.
167 60 Matthew Tamsett
168 15 Dominick Rocco
h2. FA14-10-28 FD PID and CAF 
169 60 Matthew Tamsett
170 15 Dominick Rocco
These are the PID and CAF validation samples. They are designed so that the physics groups can tune PIDs before the official first analysis production.
171 60 Matthew Tamsett
172 15 Dominick Rocco
*No known issues*.
173 1 Gavin Davies
174 61 Matthew Tamsett
The CAFs can be found here:
175 61 Matthew Tamsett
176 61 Matthew Tamsett
<pre>
177 61 Matthew Tamsett
/nova/prod/mc/FA14-10-28/genie/fd/caf/
178 61 Matthew Tamsett
/nova/prod/mc/FA14-10-28/cry/fd/caf/
179 61 Matthew Tamsett
</pre>
180 1 Gavin Davies
181 1 Gavin Davies
The real-detector configurations CAFs live in sub-directories with numbers 000129-000170, and the "ideal" MC in sub-directory 010000. Note the real detector configurations genie folders contain both the baseline MC and the geojittered MC, so anyone getting files from this location should be sure to require the "genie_fhc" string be contained in their CAF file name if they want to study the baseline sample and similarly that "geojittered" be there if they want to study that sample. 
182 65 Matthew Tamsett
183 15 Dominick Rocco
h2. FA14-11-25 ND reconstruction, PID and CAF 
184 65 Matthew Tamsett
185 1 Gavin Davies
This is the release used for *version 4 of the ND rapid turn around*. These files pick up a few notable changes with respect to those files processed in v3 (FA14-11-11, see [[NOvA-SAM:LegacyDatasets#FA14-11-11-ND-reconstruction-PID-and-CAF]] for details).  First, we now have a new set of subrun-by-subrun channel masks.  Second, an updated absolute calibration has been applied in response to discrepancies observed by the analysis groups.  An error involving infinite calibrated energies in the muon catcher has also been resolved.  This round of processing is meant to include post-shutdown data.  That effort, however, is currently waiting on the good runs list being updated by the DQ group.  Once that effort is complete, it will be an easy task to add the post-shutdown data.  Users will be notified when the processing is completed.  
186 1 Gavin Davies
187 1 Gavin Davies
Only a subsample of all MC events were processed through reconstruction, PID and CAF (~7M / 34M events) at the moment.
188 1 Gavin Davies
189 1 Gavin Davies
*Known issues*:
190 1 Gavin Davies
 
191 1 Gavin Davies
 * It has been reported that the cosmic rejection variables are not filled correctly in files processed in S14-11-25. As reconstruction wasn't frozen until the 27th, this bug likely affects these files.
192 1 Gavin Davies
193 1 Gavin Davies
The CAFs can be found here:
194 61 Matthew Tamsett
195 65 Matthew Tamsett
<pre>
196 60 Matthew Tamsett
MC -- /nova/prod/mc/FA14-11-25/genie/nd/caf/
197 68 Matthew Tamsett
Data -- /nova/prod/data/FA14-11-25/numi/nd/caf/
198 68 Matthew Tamsett
</pre>
199 71 Matthew Tamsett
200 71 Matthew Tamsett
h2. FA14-12-16 FD data CAF
201 71 Matthew Tamsett
202 71 Matthew Tamsett
No known issues. 
203 71 Matthew Tamsett
204 71 Matthew Tamsett
h2. FA14-12-29 ND rapid-turn around v5
205 71 Matthew Tamsett
206 71 Matthew Tamsett
No known issues.
207 71 Matthew Tamsett
208 60 Matthew Tamsett
209 47 Dominick Rocco
h2. S14-09-29 Data keep-up & MC calibration
210 60 Matthew Tamsett
211 15 Dominick Rocco
FD & ND calibration files are currently being produced for the FD cosmic stream as well as the ND DD activity, DD cal mu (tri-cell) and cosmic streams. These samples are constantly topped up using cron jobs. These files were used to produce the v05 ND calibration uses in the latest ND reconstruction files.
212 60 Matthew Tamsett
213 15 Dominick Rocco
*Known issues*:
214 60 Matthew Tamsett
215 15 Dominick Rocco
 * The ND data files have been reconstructed with an old ND geometry.
216 60 Matthew Tamsett
217 15 Dominick Rocco
h2. S14-09-29 Keep-up reconstruction 
218 60 Matthew Tamsett
219 15 Dominick Rocco
Most of the same caveats apply as did for the S14-09-09 reco sample detailed on the [[NOvA-SAM:LegacyDatasets]] page, but will be repeated below for completeness.  The big change in the S14-09-29 is the addition of new information to facilitate basic neutrino searches.  There is a new sel.containment branch in the CAFs and the numu CosRej has been included for the FD stream.  
220 60 Matthew Tamsett
221 60 Matthew Tamsett
Some additional notes:
222 60 Matthew Tamsett
 
223 60 Matthew Tamsett
 * The initial target for this processing is all of the data before the shutdown and after the end of the neutrino hunt.  Unless problems are found, back-processing will extend the sample to before the neutrino hunt. 
224 60 Matthew Tamsett
 * Future reco keep-up will proceed in the near future in a modern release and will include post-shutdown (October 2014) data.
225 60 Matthew Tamsett
 * The FD reco keep-up is *blinded*.
226 60 Matthew Tamsett
 * FD calibration constants are currently only available through diblock 7, but averaged constants are used beyond that point.  
227 60 Matthew Tamsett
 * These datasets are very large. Using SAM projects which include the entire dataset is discouraged.  For assistance in breaking up the sample, reference the [[Sam Web Cookbook]] or email nova_sam@fnal.gov.
228 60 Matthew Tamsett
 * Near detector reconstruction is currently lacking calibration, which will affect the output of any module depends on it.  The most notable example is FuzzyKVertex, but the MichelE filters could also be affected.  Slicer, CosmicTrack and KalmanTrack produce output independent of reconstruction.  
229 15 Dominick Rocco
 * Near detector reconstruction is also currently lacking channel masks and data quality information.  
230 60 Matthew Tamsett
231 15 Dominick Rocco
The CAFs can be found here:
232 1 Gavin Davies
233 1 Gavin Davies
<pre>
234 1 Gavin Davies
/nova/prod/data/keepup/S14-09-29/numi/fd/000XXX/XXXYY/ 
235 1 Gavin Davies
/nova/prod/data/keepup/S14-09-09/numi/nd/000XXX/YYYYY/ 
236 76 Matthew Tamsett
</pre>
237 76 Matthew Tamsett
238 76 Matthew Tamsett
where XXX are the first three digits of the run number and YY are the last two.
239 76 Matthew Tamsett
240 76 Matthew Tamsett
h2. S15-01-12 Keep-up reconstruction 
241 76 Matthew Tamsett
242 78 Matthew Tamsett
This is the first pass at post-shutdown reconstruction. As with all keep-up reco, the FD reco keep-up is *blinded* and *does not include PID*, further *no good-runs list has been used*.
243 76 Matthew Tamsett
244 76 Matthew Tamsett
*No known issues*.
245 76 Matthew Tamsett
246 76 Matthew Tamsett
The CAFs can be found here:
247 76 Matthew Tamsett
248 76 Matthew Tamsett
<pre>
249 76 Matthew Tamsett
/nova/prod/data/keepup/S15-01-12/numi/fd/000XXX/XXXYY/ 
250 76 Matthew Tamsett
/nova/prod/data/keepup/S15-01-12/numi/nd/000XXX/YYYYY/ 
251 1 Gavin Davies
</pre>
252 66 Satish Desai
253 1 Gavin Davies
where XXX are the first three digits of the run number and YY are the last two.