Project

General

Profile

Official Datasets » History » Version 57

Gavin Davies, 11/15/2014 03:15 PM
add 14db ideal conditions MC datasets for daq/reco/pid/caf stages

1 11 Dominick Rocco
h1. Official Datasets
2 11 Dominick Rocco
3 44 Dominick Rocco
4 1 Gavin Davies
The vast majority of NOvA's data and MC reside within a data handling system called Sequential Access Metadata (SAM).  The files are stored on a tape-backed disk array called dCache.  Locations for the files are tracked using a database with a web interface, accessible through the @samweb@ command line client utility.  The [[NOvA-SAM:SAM_web_cookbook]] provides an introduction to using the system.  
5 45 Dominick Rocco
6 45 Dominick Rocco
As a throwback to the old days, some of the data is still stored on bluearc.  CAFs are a notable group of files of which the most recent are almost always stored on bluearc.  There are also small, selected samples of ART files available on bluearc in <some location, documented where?>
7 45 Dominick Rocco
8 45 Dominick Rocco
{{toc}}
9 1 Gavin Davies
10 12 Gavin Davies
h1. Monte Carlo
11 1 Gavin Davies
12 54 Gavin Davies
h1. Latest datasets for detector-like conditions MC
13 54 Gavin Davies
14 54 Gavin Davies
| *FarDet* | *FHC swap*                                 | *FHC nonswap*                             | *RHC swap* | *RHC nonswap* | *FHC tau*                             | *Cosmics*                       |
15 54 Gavin Davies
| *artdaq* | prod_daq_FA14-10-03_fd_genie_fhc_fluxswap  | prod_daq_FA14-10-03_fd_genie_fhc_nonswap  |            |               | prod_daq_FA14-10-03_fd_genie_fhc_tau  | prod_daq_FA14-10-03_fd_cry_all  |
16 56 Satish Desai
| *reco*   | prod_reco_FA14-10-28_fd_genie_fhc_fluxswap | prod_reco_FA14-10-28_fd_genie_fhc_nonswap |            |               | prod_reco_FA14-10-28_fd_genie_fhc_tau | prod_reco_FA14-10-28_fd_cry_all |
17 54 Gavin Davies
| *pid*    | prod_pid_FA14-10-28_fd_genie_fhc_fluxswap  | prod_pid_FA14-10-28_fd_genie_fhc_nonswap  |            |               | prod_pid_FA14-10-28_fd_genie_fhc_tau  | prod_pid_FA14-10-28_fd_cry_all  |
18 54 Gavin Davies
| *caf*    | prod_caf_FA14-10-28_fd_genie_fhc_fluxswap  | prod_caf_FA14-10-28_fd_genie_fhc_nonswap  |            |               | prod_caf_FA14-10-28_fd_genie_fhc_tau  | prod_caf_FA14-10-28_fd_cry_all  |
19 54 Gavin Davies
20 57 Gavin Davies
h1. Latest datasets for ideal conditions (14db) MC
21 57 Gavin Davies
22 57 Gavin Davies
| *FarDet* | *FHC swap*                                 | *FHC nonswap*                             | *RHC swap* | *RHC nonswap* | *FHC tau*                             | *Cosmics*                       |
23 57 Gavin Davies
| *artdaq* | prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_14db  | prod_daq_FA14-10-03_fd_genie_fhc_nonswap_14db  |            |               | prod_daq_FA14-10-03_fd_genie_fhc_tau_14db  |   |
24 57 Gavin Davies
| *reco*   | prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_14db | prod_reco_FA14-10-28_fd_genie_fhc_nonswap_14db |            |               | prod_reco_FA14-10-28_fd_genie_fhc_tau_14db |  |
25 57 Gavin Davies
| *pid*    | prod_pid_FA14-10-28_fd_genie_fhc_fluxswap_14db  | prod_pid_FA14-10-28_fd_genie_fhc_nonswap_14db  |            |               | prod_pid_FA14-10-28_fd_genie_fhc_tau_14db  |   |
26 57 Gavin Davies
| *caf*    | prod_caf_FA14-10-28_fd_genie_fhc_fluxswap_14db  | prod_caf_FA14-10-28_fd_genie_fhc_nonswap_14db  |            |               | prod_caf_FA14-10-28_fd_genie_fhc_tau_14db  |  |
27 57 Gavin Davies
28 40 Christopher Backhouse
h1. Latest datasets for regular configurations
29 40 Christopher Backhouse
30 40 Christopher Backhouse
| *FarDet* | *FHC swap*                               | *FHC nonswap*                              | *RHC swap*                              | *RHC nonswap*                              | *Cosmics*                  |
31 40 Christopher Backhouse
| *artdaq* | prod_artdaq_S14-03-25_fd_genie_fhc_swap  | prod_artdaq_S14-03-25_fd_genie_fhc_nonswap | prod_artdaq_S14-03-25_fd_genie_rhc_swap | prod_artdaq_S14-03-25_fd_genie_rhc_nonswap | prodartdaq_S14-02-05_fdcry |
32 40 Christopher Backhouse
| *reco*   | prodreco_S14-03-25_FDGENIE_fhc_swap      | prodreco_S14-03-25_FDGENIE_fhc_nonswap     | prodreco_S14-03-25_FDGENIE_rhc_swap     | prodreco_S14-03-25_FDGENIE_rhc_nonswap     | prodreco_S14-03-25_FDCRY   |
33 40 Christopher Backhouse
| *pid*    | prod_pid_S14-05-12_fd_genie_neutrino2014 |
34 40 Christopher Backhouse
| *caf*    | prod_caf_S14-07-11_fd_genie_neutrino2014 |
35 40 Christopher Backhouse
36 40 Christopher Backhouse
37 40 Christopher Backhouse
| *NearDet* | *FHC*                            | *RHC* | *Cosmics*                  |
38 40 Christopher Backhouse
| *artdaq*  | prodartdaq_S14-05-12_ndgenie_fix | -     | prodartdaq_S14-05-12_ndcry |
39 40 Christopher Backhouse
| *reco*    | prodreco_S14-08-01_ndgenie_fix   | -     | -                          |
40 40 Christopher Backhouse
| *pid*     | prodpid_S14-08-19_ndgenie_fix    | -     | -                          |
41 40 Christopher Backhouse
| *caf*     | prodcaf_S14-08-19_ndgenie_fix    | -     | -                          |
42 40 Christopher Backhouse
43 18 Gavin Davies
h1. GENIE
44 1 Gavin Davies
45 17 Gavin Davies
h2. ARTDAQ
46 17 Gavin Davies
47 4 Gavin Davies
h3. FD
48 4 Gavin Davies
49 54 Gavin Davies
Real detector-like conditions MC
50 54 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_fluxswap*
51 54 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_nonswap*
52 54 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_tau*
53 54 Gavin Davies
54 57 Gavin Davies
Ideal conditions 14db MC
55 57 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_fluxswap_14db*
56 57 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_nonswap_14db*
57 57 Gavin Davies
*prod_daq_FA14-10-03_fd_genie_fhc_tau_14db*
58 57 Gavin Davies
59 54 Gavin Davies
Nominal
60 35 Gavin Davies
FHC SWAP/NONSWAP
61 35 Gavin Davies
*prod_artdaq_S14-03-25_fd_genie_fhc_swap* (709)
62 35 Gavin Davies
*prod_artdaq_S14-03-25_fd_genie_fhc_nonswap* (709)
63 1 Gavin Davies
64 35 Gavin Davies
RHC SWAP/NONSWAP
65 35 Gavin Davies
*prod_artdaq_S14-03-25_fd_genie_rhc_swap* (709)
66 35 Gavin Davies
*prod_artdaq_S14-03-25_fd_genie_rhc_nonswap* (709)
67 35 Gavin Davies
68 4 Gavin Davies
h3. ND
69 13 Gavin Davies
70 23 Gavin Davies
Fixed version of overlay files
71 26 Gavin Davies
*prodartdaq_S14-05-12_ndgenie_fix* (4359)
72 23 Gavin Davies
73 23 Gavin Davies
-*prodartdaq_S14-05-12_ndgenie* (20087)-
74 4 Gavin Davies
75 24 Gavin Davies
h3. Special
76 24 Gavin Davies
77 36 Gavin Davies
Nominal geometry
78 36 Gavin Davies
*prod_artdaq_S14-08-19_fd_genie_fhc_nonswap* (34)
79 36 Gavin Davies
*prod_artdaq_S14-08-19_fd_genie_fhc_swap* (34)
80 36 Gavin Davies
81 24 Gavin Davies
Jittered geometry
82 39 Matthew Tamsett
*prod_artdaq_S14-08-19_fd_fhc_nonswap_geojittered* (37)
83 39 Matthew Tamsett
*prod_artdaq_S14-08-19_fd_fhc_swap_geojittered* (39)
84 24 Gavin Davies
85 24 Gavin Davies
CRY+GENIE
86 39 Matthew Tamsett
*prod_artdaq_S14-08-19_fd_fhc_nonswap_crygenie* (510)
87 39 Matthew Tamsett
*prod_artdaq_S14-08-19_fd_fhc_swap_crygenie* (511)
88 24 Gavin Davies
89 17 Gavin Davies
h2. RECO
90 8 Gavin Davies
91 1 Gavin Davies
Official production datasets containing all reconstructed files
92 1 Gavin Davies
93 1 Gavin Davies
h3. FD 
94 1 Gavin Davies
95 54 Gavin Davies
Real detector-like conditions MC
96 54 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_fluxswap*
97 54 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_nonswap*
98 54 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_tau*
99 54 Gavin Davies
100 57 Gavin Davies
Ideal conditions 14db MC
101 57 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_fluxswap_14db*
102 57 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_nonswap_14db*
103 57 Gavin Davies
*prod_reco_FA14-10-28_fd_genie_fhc_tau_14db*
104 57 Gavin Davies
105 54 Gavin Davies
Nominal
106 1 Gavin Davies
ALL FD GENIE RECO:          
107 1 Gavin Davies
*prodreco_S14-03-25_FDGENIE* (2807 files)
108 1 Gavin Davies
109 35 Gavin Davies
FHC SWAP/NONSWAP: 
110 35 Gavin Davies
*prodreco_S14-03-25_FDGENIE_fhc_swap* (703)
111 1 Gavin Davies
*prodreco_S14-03-25_FDGENIE_fhc_nonswap* (703)
112 1 Gavin Davies
113 35 Gavin Davies
RHC SWAP/NONSWAP
114 1 Gavin Davies
*prodreco_S14-03-25_FDGENIE_rhc_swap* (698)
115 35 Gavin Davies
*prodreco_S14-03-25_FDGENIE_rhc_nonswap* (703)
116 4 Gavin Davies
117 5 Gavin Davies
h3. ND
118 4 Gavin Davies
119 1 Gavin Davies
ND (FHC) GENIE (rock+detector) OVERLAY RECO
120 25 Gavin Davies
*prodreco_S14-08-01_ndgenie_fix* (4295)
121 1 Gavin Davies
122 24 Gavin Davies
-*prodreco_S14-07-03_ndgenie* (20057 and increasing)-
123 1 Gavin Davies
124 1 Gavin Davies
h3. Special
125 1 Gavin Davies
126 36 Gavin Davies
Nominal geometry
127 36 Gavin Davies
*prod_reco_S14-08-01_fd_genie_fhc_nonswap* (34)
128 36 Gavin Davies
*prod_reco_S14-08-01_fd_genie_fhc_swap* (34)
129 1 Gavin Davies
130 36 Gavin Davies
Jittered geometry
131 36 Gavin Davies
Simulated with jitter and reco+ with standard geom
132 38 Matthew Tamsett
*prod_reco_S14-08-01_fd_fhc_nonswap_geojittered_v5* (37)
133 38 Matthew Tamsett
*prod_reco_S14-08-01_fd_fhc_swap_geojittered_v5* (39)
134 36 Gavin Davies
135 36 Gavin Davies
All tiers with geojittered geometry
136 36 Gavin Davies
-*prod_reco_S14-08-01_fd_fhc_nonswap_geojittered_v1* (34)-
137 36 Gavin Davies
-*prod_reco_S14-08-01_fd_fhc_swap_geojittered_v1* (34)-
138 36 Gavin Davies
139 24 Gavin Davies
CRY+GENIE
140 30 Gavin Davies
*prod_reco_S14-08-01_fd_fhc_nonswap_crygenie* (510)
141 30 Gavin Davies
*prod_reco_S14-08-01_fd_fhc_swap_crygenie* (511)
142 16 Gavin Davies
143 53 Gavin Davies
NO MASK
144 53 Gavin Davies
These files have bad channels turned off.
145 53 Gavin Davies
*prod_reco_S14-10-15_genie_nonswap_nomask*
146 53 Gavin Davies
Note, these files have the POT counting error that propagates from the generation stage so correctly scaling POT is required, i.e. divide by 2 totalpot.
147 53 Gavin Davies
148 53 Gavin Davies
149 17 Gavin Davies
h2. PID
150 34 Gavin Davies
151 32 Satish Desai
h3. FD
152 32 Satish Desai
153 54 Gavin Davies
Real detector-like conditions MC
154 54 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_fluxswap*
155 54 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_nonswap*
156 54 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_tau*
157 54 Gavin Davies
158 57 Gavin Davies
Ideal conditions 14db MC
159 57 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_fluxswap_14db*
160 57 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_nonswap_14db*
161 57 Gavin Davies
*prod_pid_FA14-10-28_fd_genie_fhc_tau_14db*
162 57 Gavin Davies
163 54 Gavin Davies
OLD
164 16 Gavin Davies
*prod_pid_S14-05-12_fd_genie_neutrino2014*
165 16 Gavin Davies
166 1 Gavin Davies
h3. ND
167 16 Gavin Davies
168 1 Gavin Davies
ND (FHC) GENIE (rock+detector) OVERLAY reco
169 1 Gavin Davies
170 1 Gavin Davies
*prodpid_S14-08-19_ndgenie_fix* (4072)
171 1 Gavin Davies
172 30 Gavin Davies
-*prodpid_S14-08-01_ndgenie* (2715, will grow)-
173 1 Gavin Davies
174 1 Gavin Davies
h3. Special
175 1 Gavin Davies
176 36 Gavin Davies
Nominal
177 36 Gavin Davies
*prod_pid_S14-08-19_fd_genie_fhc_nonswap* (34)
178 36 Gavin Davies
*prod_pid_S14-08-19_fd_genie_fhc_swap* (34)
179 36 Gavin Davies
180 1 Gavin Davies
Jittered geometry
181 36 Gavin Davies
Simulated with jitter and reco+ with standard geom
182 36 Gavin Davies
*prod_pid_S14-08-19_fd_fhc_nonswap_geojittered_v5* (37)
183 36 Gavin Davies
*prod_pid_S14-08-19_fd_fhc_swap_geojittered_v5* (39)
184 30 Gavin Davies
185 36 Gavin Davies
All tiers with geojittered geometry
186 36 Gavin Davies
-*prod_pid_S14-08-19_fd_fhc_nonswap_geojittered_v1* (34)-
187 36 Gavin Davies
-*prod_pid_S14-08-19_fd_fhc_swap_geojittered_v1* (34)-
188 36 Gavin Davies
189 19 Gavin Davies
CRY+GENIE
190 1 Gavin Davies
*prod_pid_S14-08-19_fd_fhc_nonswap_crygenie* (486)
191 33 Satish Desai
*prod_pid_S14-08-19_fd_fhc_swap_crygenie* (492)
192 34 Gavin Davies
193 53 Gavin Davies
NO MASK
194 53 Gavin Davies
These files have bad channels turned off.
195 53 Gavin Davies
*prod_pid_S14-10-28_nd_genie_nonswap_nomask*
196 53 Gavin Davies
Note, these files have the POT counting error that propagates from the generation stage so correctly scaling POT is required, i.e. divide by 2 totalpot.
197 53 Gavin Davies
The module "nuecosrej" is empty in the PID files. It is filled at the CAF stage for this set however.
198 53 Gavin Davies
199 33 Satish Desai
h2. CAF
200 19 Gavin Davies
201 33 Satish Desai
h3. FD
202 29 Gavin Davies
203 54 Gavin Davies
Real detector-like conditions MC
204 54 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_fluxswap*
205 54 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_nonswap*
206 54 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_tau*
207 54 Gavin Davies
208 54 Gavin Davies
Bluearc: /nova/prod/mc/FA14-10-28/genie/fd/caf/000XXX/XXXYY/ 
209 54 Gavin Davies
where XXX are the first three digits of the run number and YY are the last two.
210 54 Gavin Davies
You'll need to use appropriate wildcard flags.
211 57 Gavin Davies
212 57 Gavin Davies
Ideal conditions 14db MC
213 57 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_fluxswap_14db*
214 57 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_nonswap_14db*
215 57 Gavin Davies
*prod_caf_FA14-10-28_fd_genie_fhc_tau_14db*
216 57 Gavin Davies
217 57 Gavin Davies
Bluearc: /nova/prod/mc/FA14-10-28/genie/fd/caf/010000/100000X
218 57 Gavin Davies
where X is an integer from 0-5.
219 54 Gavin Davies
220 54 Gavin Davies
OLD
221 29 Gavin Davies
*prod_caf_S14-07-11_fd_genie_neutrino2014*
222 29 Gavin Davies
223 28 Gavin Davies
h3. ND
224 19 Gavin Davies
225 28 Gavin Davies
SAM: *prodcaf_S14-08-19_ndgenie_fix* (4072)
226 1 Gavin Davies
227 1 Gavin Davies
Bluearc: */nova/prod/mc/S14-08-19/genie/nd/caf* (4072)
228 1 Gavin Davies
229 1 Gavin Davies
First pass post-pid:
230 1 Gavin Davies
-*/nova/prod/mc/S14-08-15/genie/nd/caf/* (2441 files)-
231 30 Gavin Davies
(find . -name "neardet*")
232 30 Gavin Davies
CAFMaker Floating point exception for all failures.
233 1 Gavin Davies
234 1 Gavin Davies
Caveat: Two significant issues have been found in this sample in the last day or two.  An old version of PhotonTransport was used in the overlay production, making the light levels low by a factor of two, and also the rock overlay scheme re-used rock events at a much higher rate than intended.  Both of these issues are fixed and replacements for these ND MC CAFs will tentatively be ready in a week we hope. We'll keep you posted. 
235 1 Gavin Davies
The current CAFs should be useful for code development and certain physics studies in the meantime.
236 1 Gavin Davies
237 1 Gavin Davies
h3. Special
238 1 Gavin Davies
239 36 Gavin Davies
Nominal
240 37 Matthew Tamsett
*prod_caf_S14-08-19_fd_genie_fhc_nonswap* (34)
241 37 Matthew Tamsett
*prod_caf_S14-08-19_fd_genie_fhc_swap* (34)
242 36 Gavin Davies
243 1 Gavin Davies
Jittered geometry
244 36 Gavin Davies
Simulated with jitter and reco+ with standard geom
245 36 Gavin Davies
*prod_caf_S14-08-19_fd_fhc_nonswap_geojittered_v5* (37)
246 36 Gavin Davies
*prod_caf_S14-08-19_fd_fhc_swap_geojittered_v5* (39)
247 31 Gavin Davies
248 36 Gavin Davies
All tiers with geojittered geometry
249 36 Gavin Davies
-*prod_caf_S14-08-19_fd_fhc_nonswap_geojittered_v1* (32)-
250 36 Gavin Davies
-*prod_caf_S14-08-19_fd_fhc_swap_geojittered_v1* (33)-
251 36 Gavin Davies
252 30 Gavin Davies
CRY+GENIE
253 30 Gavin Davies
*prod_caf_S14-08-19_fd_fhc_nonswap_crygenie* (484)
254 30 Gavin Davies
*prod_caf_S14-08-19_fd_fhc_swap_crygenie* (490)
255 30 Gavin Davies
256 30 Gavin Davies
Bluearc:
257 19 Gavin Davies
*/nova/prod/mc/S14-08-19/genie/fd/caf/* 
258 36 Gavin Davies
(need to grab *fhc_nonswap* or *fhc_swap* plus either *crygenie* or *geojittered*v5*)
259 1 Gavin Davies
260 53 Gavin Davies
NO MASK
261 53 Gavin Davies
These files have bad channels turned off.
262 53 Gavin Davies
*/nova/ana/nu_e_ana/S14-10-28/nd_nomask/mc*
263 53 Gavin Davies
Note, these files have the POT counting error that propagates from the generation stage so correctly scaling POT is required, i.e. divide by 2 totalpot.
264 53 Gavin Davies
These CAFs have "nuecosrej" filled.
265 53 Gavin Davies
266 17 Gavin Davies
h1. CRY
267 17 Gavin Davies
268 6 Gavin Davies
h2. ARTDAQ
269 1 Gavin Davies
270 54 Gavin Davies
Real detector-like conditions MC
271 54 Gavin Davies
*prod_daq_FA14-10-03_fd_cry_all*
272 54 Gavin Davies
273 54 Gavin Davies
Nominal
274 9 Gavin Davies
ND CRY COSMICS:
275 9 Gavin Davies
*prodartdaq_S14-05-12_ndcry*
276 6 Gavin Davies
277 1 Gavin Davies
FD CRY COSMICS:
278 17 Gavin Davies
*prodartdaq_S14-02-05_fdcry*
279 1 Gavin Davies
280 2 Gavin Davies
h2. RECO
281 1 Gavin Davies
282 54 Gavin Davies
Real detector-like conditions MC
283 54 Gavin Davies
*prod_reco_FA14-10-28_fd_cry_all*
284 54 Gavin Davies
285 54 Gavin Davies
Nominal
286 2 Gavin Davies
FD CRY COSMICS: 
287 17 Gavin Davies
*prodreco_S14-03-25_FDCRY* (10069)
288 3 Gavin Davies
289 2 Gavin Davies
h2. CALIBRATION (pchitlists)
290 2 Gavin Davies
291 2 Gavin Davies
ND CRY COSMICS:
292 2 Gavin Davies
*prodcalib_S14-05-12_ndcry_pclist*
293 2 Gavin Davies
*prodcalib_S14-05-12_ndcry_pcliststop*
294 2 Gavin Davies
295 1 Gavin Davies
FD CRY COSMICS:
296 11 Dominick Rocco
*prodcalib_S14-03-05_FDCry_pclist*
297 12 Gavin Davies
*prodcalib_S14-03-05_FDCry_pcliststop*
298 54 Gavin Davies
299 54 Gavin Davies
h2. PID
300 54 Gavin Davies
301 54 Gavin Davies
Real detector-like conditions MC
302 54 Gavin Davies
*prod_pid_FA14-10-28_fd_cry_all*
303 54 Gavin Davies
304 54 Gavin Davies
h2. CAF
305 54 Gavin Davies
306 54 Gavin Davies
Real detector-like conditions MC
307 54 Gavin Davies
*prod_caf_FA14-10-28_fd_cry_all*
308 54 Gavin Davies
309 54 Gavin Davies
Bluearc: /nova/prod/mc/FA14-10-28/cry/fd/caf/000XXX/XXXYY/ 
310 54 Gavin Davies
where XXX are the first three digits of the run number and YY are the last two.
311 54 Gavin Davies
You'll need to use appropriate wildcard flags.
312 1 Gavin Davies
313 15 Dominick Rocco
h1. DATA
314 1 Gavin Davies
315 50 Dominick Rocco
h2.  First-Analysis -- First Pass FD Processing (FA14-10-28)
316 50 Dominick Rocco
317 50 Dominick Rocco
In advance of the final calibration intended for first-analysis, a first pass with old calibration constants will be produced.  The input to these samples is the official first-analysis run lists generated by Ryan Patterson and stored in this directory: 
318 50 Dominick Rocco
<pre>
319 50 Dominick Rocco
  /nova/app/users/rbpatter/runlists/
320 50 Dominick Rocco
</pre>
321 50 Dominick Rocco
322 50 Dominick Rocco
Those lists have been adapted into very specific datasets.  
323 52 Dominick Rocco
|*Data Tier*|*Cosmic Trigger*| *NuMI Trigger*| 
324 52 Dominick Rocco
|artdaq|@prod_artdaq_fd_cosmic_fa_goodruns@|@prod_artdaq_fd_numi_fa_goodruns @|
325 52 Dominick Rocco
|reco|@prod_reco_FA14-10-28_fd_cosmic_fa_goodruns@|@prod_reco_FA14-10-28_fd_numi_fa_goodruns@|
326 50 Dominick Rocco
327 46 Dominick Rocco
h2. S14-09-29 Keep-up Reconstruction 
328 46 Dominick Rocco
329 46 Dominick Rocco
Most of the same caveats apply as did for the S14-09-09 sample, but will be repeated below for completeness.  The big change in the S14-09-29 is the addition of new information to facilitate basic neutrino searches.  There is a new sel.containment branch in the CAFs and the numu CosRej has been included for the FD stream.  
330 46 Dominick Rocco
331 46 Dominick Rocco
Keep-up reconstuction provides reconstructed art root files and CAFs.  For quick testing the last three files from each of the below datasets is stored on bluearc and updated nightly. They can be found here:
332 46 Dominick Rocco
333 46 Dominick Rocco
/nova/prod/data/keepup/samples/ 
334 46 Dominick Rocco
335 49 Dominick Rocco
h3. ND with MC Calibration 
336 1 Gavin Davies
337 49 Dominick Rocco
In lieu of official ND calibration, the old MC calibration can be applied.  The output of any module which relies on calibration should be taken with a heavy grain of salt, but these files will contain non-zero values for calibrated energies rather than zeroes.  
338 49 Dominick Rocco
339 49 Dominick Rocco
The database has now been populated with bad channel masks, which have been applied in this round of processing.  
340 49 Dominick Rocco
341 49 Dominick Rocco
The following SAM datasets are available.   
342 49 Dominick Rocco
Reco art rootfiles: _prod_reco_S14-09-29_neardet_numi_keepup_mccalib_
343 49 Dominick Rocco
CAF:  _prod_caf_S14-09-29_neardet_numi_keepup_mccalib_
344 49 Dominick Rocco
345 49 Dominick Rocco
The CAFs can also be found on bluearc:
346 49 Dominick Rocco
/nova/prod/data/keepup/S14-09-29/numi/nd/000XXX/XXXYY/ 
347 49 Dominick Rocco
where XXX are the first three digits of the run number and YY are the last two.  Note, the MC calibrated files have "mccalib" in their name.  The standard ones do not.  
348 49 Dominick Rocco
349 49 Dominick Rocco
As of October 16, there is an FTS issue keeping these files from being transferred to bluearc.  The CD experts have been notified and are working on a solution.  
350 49 Dominick Rocco
351 49 Dominick Rocco
h3.  ND (no calibration)
352 49 Dominick Rocco
353 46 Dominick Rocco
Near detector reconstruction is currently lacking calibration, which will affect the output of any module depends on it.  The most notable example is FuzzyKVertex, but the MichelE filters could also be affected.  Slicer, CosmicTrack and KalmanTrack produce output independent of reconstruction.  
354 46 Dominick Rocco
355 46 Dominick Rocco
Near detector reconstruction is also currently lacking channel masks and data quality information.  
356 46 Dominick Rocco
357 46 Dominick Rocco
The following SAM datasets are available.   
358 46 Dominick Rocco
Reco art rootfiles: _prod_reco_S14-09-29_neardet_numi_keepup_
359 46 Dominick Rocco
CAF:  _prod_caf_S14-09-29_neardet_numi_keepup_
360 1 Gavin Davies
361 49 Dominick Rocco
362 1 Gavin Davies
The CAFs can also be found on bluearc:
363 48 Dominick Rocco
/nova/prod/data/keepup/S14-09-29/numi/nd/000XXX/XXXYY/ 
364 49 Dominick Rocco
where XXX are the first three digits of the run number and YY are the last two.  Note, the MC calibrated files mentioned above have "mccalib" in their name.  The standard ones do not.  
365 46 Dominick Rocco
366 46 Dominick Rocco
h3.  FD 
367 46 Dominick Rocco
368 46 Dominick Rocco
The initial target for this processing is all of the data before the shutdown and after the end of the neutrino hunt.  Unless problems are found, back-processing will extend the sample to before the neutrino hunt.  
369 48 Dominick Rocco
370 46 Dominick Rocco
FD calibration constants are currently only available through diblock 7, but averaged constants are used beyond that point.  
371 46 Dominick Rocco
372 46 Dominick Rocco
*NB: The datasets listed below are very large.*  Using SAM projects which include the entire dataset is discouraged.  For assistance in breaking up the sample, reference the [[Sam Web Cookbook]] or email nova_sam@fnal.gov.
373 46 Dominick Rocco
374 46 Dominick Rocco
The following SAM datasets are available: 
375 46 Dominick Rocco
Reco art rootfiles: _prod_reco_S14-09-29_fardet_numi_keepup_ 
376 46 Dominick Rocco
CAF, blinded information removed:  _prod_caf_S14-09-29_fardet_numi_keepup_
377 1 Gavin Davies
378 46 Dominick Rocco
The CAFs can be found here:
379 46 Dominick Rocco
/nova/prod/data/keepup/S14-09-29/numi/fd/000XXX/XXXYY/ 
380 49 Dominick Rocco
where XXX are the first three digits of the run number and YY are the last two.  
381 46 Dominick Rocco
382 46 Dominick Rocco
h2. S14-09-09 Keep-up Reconstruction 
383 41 Dominick Rocco
384 43 Matthew Tamsett
Keep-up reconstuction provides reconstructed art root files and CAFs.  For quick testing the last three files from each of the below datasets is stored on bluearc and updated nightly. They can be found here:
385 1 Gavin Davies
386 43 Matthew Tamsett
/nova/prod/data/keepup/samples/ 
387 43 Matthew Tamsett
388 41 Dominick Rocco
h3.  ND 
389 41 Dominick Rocco
390 41 Dominick Rocco
Near detector reconstruction is currently lacking calibration, which will affect the output of any module depends on it.  The most notable example is FuzzyKVertex, but the MichelE filters could also be affected.  Slicer, CosmicTrack and KalmanTrack produce output independent of reconstruction.  
391 41 Dominick Rocco
392 41 Dominick Rocco
Near detector reconstruction is also currently lacking channel masks and data quality information.  
393 41 Dominick Rocco
394 41 Dominick Rocco
The following SAM datasets are available.   
395 41 Dominick Rocco
Reco art rootfiles: _prod_reco_keepup_S14-09-09_neardet_numi_preshutdown_fullgain_cold_
396 41 Dominick Rocco
CAF:  _prod_caf_keepup_S14-09-09_neardet_numi_preshutdown_fullgain_cold_
397 41 Dominick Rocco
398 41 Dominick Rocco
The CAFs can also be found on bluearc:
399 41 Dominick Rocco
/nova/prod/data/keepup/S14-09-09/numi/nd/000XXX/YYYYY/ 
400 41 Dominick Rocco
where XXX are the first three digits of the run number and YYYYY is the complete run number.  
401 41 Dominick Rocco
402 41 Dominick Rocco
h3.  FD 
403 41 Dominick Rocco
404 41 Dominick Rocco
Far detector keep-up processing is still underway.  When this effort is complete, the goal is to have processed all of the data after the cease of the neutrino hunt, but before the shutdown.  Since the processing is still underway, these datasets will grow.
405 41 Dominick Rocco
406 41 Dominick Rocco
FD calibration constants are currently only available through diblock 7, but averaged constants are used beyond that point.  
407 41 Dominick Rocco
408 41 Dominick Rocco
The following SAM datasets are available: 
409 41 Dominick Rocco
Reco art rootfiles: _prod_reco_keepup_S14-09-09_fardet_numi_preshutdown_posthunt_ 
410 41 Dominick Rocco
CAF, blinded information removed:  _prod_caf_keepup_S14-09-09_fardet_numi_preshutdown_posthunt_
411 1 Gavin Davies
412 41 Dominick Rocco
The CAFs can be found here:
413 41 Dominick Rocco
/nova/prod/data/keepup/S14-09-09/numi/fd/000XXX/YYYYY/ 
414 43 Matthew Tamsett
where XXX are the first three digits of the run number and YYYYY is the complete run number.
415 53 Gavin Davies
416 53 Gavin Davies
417 53 Gavin Davies
h2. Special
418 53 Gavin Davies
419 53 Gavin Davies
h3. No Mask ND
420 53 Gavin Davies
421 53 Gavin Davies
ND datasets with bad channel masks turned off. Runs included from the full gain, pre-shutdown era (runs 10377 -> 10407)
422 53 Gavin Davies
423 53 Gavin Davies
RECO: *prod_reco_S14-10-15_neardet_numi_fullgain_cold_preshutdown_mccalib-nomask*
424 53 Gavin Davies
PID:  *prod_pid_S14-10-28_neardet_numi_fullgain_cold_preshutdown_mccalib-nomask*
425 53 Gavin Davies
CAF:  /nova/ana/nu_e_ana/S14-10-28/nd_nomask/data (Bluearc location only)
426 53 Gavin Davies
Only the CAF files contain filled "nuecosrej" variables.
427 41 Dominick Rocco
428 47 Dominick Rocco
h2.  FD Cosmic Trigger, Summer 2014, Post-Neutrino
429 1 Gavin Davies
430 15 Dominick Rocco
h3. Datasets
431 1 Gavin Davies
432 15 Dominick Rocco
The datasets for these files can be found in SAM.  For PID, there is also a small sample living on Bluearc.  For CAFs, the entire sample lives on Bluearc.  All of the official datasets specify the run range for the the 4db and 7db configurations based on the Stable Partition Milestones wiki page.  The datasets also require subruns (files) to pass the data quality cuts.   
433 1 Gavin Davies
434 15 Dominick Rocco
The SAM datasets and Bluearc locations are as follows:
435 15 Dominick Rocco
436 15 Dominick Rocco
PID - 4db: 
437 1 Gavin Davies
prod_pid_S14-05-12_fardet_cosmics_p1_4db_good
438 15 Dominick Rocco
/nova/prod/data/S14-05-12/fd/pid/   (Run numbers less than 14702)
439 15 Dominick Rocco
440 15 Dominick Rocco
PID - 7db
441 15 Dominick Rocco
prod_pid_S14-05-12_fardet_cosmics_p1_7db_good
442 15 Dominick Rocco
/nova/prod/data/S14-05-12/fd/pid/ (Run numbers greater than or equal to 14702)
443 15 Dominick Rocco
444 15 Dominick Rocco
CAF - 4db: 
445 15 Dominick Rocco
prod_caf_S14-07-11_fardet_cosmics_p1_4db_good
446 47 Dominick Rocco
/nova/prod/data/S14-07-11/cosmic/fd/caf/  (Run numbers less than 14702, excluding runs 14563-14594 which were 9 diblock stress tests.)
447 15 Dominick Rocco
448 15 Dominick Rocco
CAF - 7db
449 15 Dominick Rocco
prod_caf_S14-07-11_fardet_cosmics_p1_7db_good
450 15 Dominick Rocco
/nova/prod/data/S14-07-11/cosmic/fd/caf/ (Run numbers greater than or equal to 14702)
451 15 Dominick Rocco
452 15 Dominick Rocco
The underlying reco and pidpart(sans LEM) datasets exist only in SAM:
453 15 Dominick Rocco
pidpart - 4db:
454 15 Dominick Rocco
prod_pidpart_S14-05-12_fardet_cosmics_p1_4db_good
455 15 Dominick Rocco
456 15 Dominick Rocco
pidpart - 7db:
457 15 Dominick Rocco
prod_pidpart_S14-05-08_fardet_cosmics_p1_7db_good
458 15 Dominick Rocco
459 15 Dominick Rocco
reco - 4db:
460 15 Dominick Rocco
prod_reco_S14-05-08_fardet_cosmics_p1_4db_good
461 15 Dominick Rocco
462 15 Dominick Rocco
reco - 7db:
463 15 Dominick Rocco
prod_reco_S14-05-08_fardet_cosmics_p1_7db_good
464 15 Dominick Rocco
465 15 Dominick Rocco
h3. Comments on Livetime
466 15 Dominick Rocco
467 15 Dominick Rocco
In terms of livetime, the 4db sample is much larger than the 7db sample.  The 4db PID sample is roughly ten times larger than that used for Neutrino 2014.  The four diblock CAF sample is currently twice the size of the Neutrino sample, although it's growing.  The CAFMaking process has been tedious due to CAFMaker crashes*, and the sample will never be as large as the PID sample as a result.  The 7db sample, in terms of livetime, is roughly the same size as the Neutrino 2014 sample.  However, that detector configuration has 75 percent more detector mass.  The same caveat applies to the size of the CAF sample.  
468 15 Dominick Rocco
469 15 Dominick Rocco
h4.  Processing history
470 15 Dominick Rocco
471 15 Dominick Rocco
The processing history for these files is as follows:
472 15 Dominick Rocco
Reco:  S14-05-08 - Notably, this tag resolves the KalmanTrack association bug 
473 1 Gavin Davies
PID: S14-05-12 - Loose nue preselection 
474 39 Matthew Tamsett
CAF: S14-07-11 - Uses LiveGeometry, but gets the CAF metadata right.