Wiki » History » Version 64

Version 63 (Lu Ren, 10/19/2018 10:42 AM) → Version 64/127 (Lu Ren, 10/19/2018 10:43 AM)

h1. MicroBooNE Production

h2. {{collapse(Request a Collaboration wide production sample)

* Figure out what you need for the sample and fill out production request form "[click me]":

* You might need to present your sample request at Analysis Tool Meeting and/or Data Management Meeting

* The production is ONLY effective with physics conveners' approval.

* The fcl files for the production are prepared by user/group who requests the sample and they are also responsible for making sure the production workflow has been tested and is correct.

* The user/group are also responsible for validating the sample produced by production team that reaches their expectations.


h2. Coordination for a large data sample pre-staging

MicroBooNE requires the pre-staging of dataset larger than "dev" sample (4000 files or 10 TB file size) to be coordinated with production team.

* Before making pre-staging request, check pre-staging dataset status table if your requested sample has already been pre-staged. How long the pre-staging last for the dataset depends on various factors. A rule of thumb is if the dataset has been pre-staged within a couple of weeks, then most likely you don't have to pre-stage it.

* Check the dataset information with samweb commands ("learn more about SAM": ("More SAM commands":
$ samweb list-files --summary "defname:Your_Dataset_Name"

Here is an example:
samweb list-files --summary "defname:prodgenie_bnb_intrinsic_nue_cosmic_uboone_mcc8.7_reco2"
File count:    12287 (file numbers)
Total size:    34235530158207 (file size in byte)
Event count:    614350 (available events)

h2. {{collapse(Recommended Data and MC Samples for MCC8 analysis (as of Aug. 20th 2018))

* "Good Run Selected Data for 2018":

* "MCC8.7 Central Value MC":
* "MCC8.8 NuMI MC":
* "Detector variation samples": ("data format": and "description":

h2. Status of Dataset Prestaging

|_. Dataset |_. # of files|_. Size|_. Status|_. Requested by |
prod_reco_optfilter_extbnb_v12_mcc8_gooodruns_v5s|||single photon group (Mark, Kathryn)|
|prod_numi_ext_data_run1_reco2_mcc8_v1_goodruns |16209|56TB|Colton|
prodgenie_bnb_nu_cosmic_uboone_MCC8Tune3_mcc8.7_reco2|||Marco| prodgenie_bnb_nu_cosmic_uboone_MCC8Tune3_mcc8.7_reco2||||Marco|

h2. Data and MC samples

Find full list of samples at "MicroBooNE at Work":

h2. List of Production Requests

Find actual list of production requests to the team at "List of Production Requests":

h2. "MCC9 Production":

h2. For production team members

[[Meetings and Minutes]]
[[Offline Production Shift]]
[[Detector variation sample production]]
"Data Blinding":
"Pubs Era Production Status":

h2. For users

*Before submitting large number of jobs to the grid, first make sure you have tested your workflow and it is correct.* Also make sure your resources request setup follows the uB's Grid Best Practices
* "Herb's Grid Best Practice":
* "Kirby and Wei's Best Practice":
* "Herb's real life example of how to use recursive dataset":
* You can also check [[Matt's Summary of grid best practice]]

Please "contact the production team", if you
* Want to prestage and process a dataset with more than 4000 files or size > 10 TB
* Cancel large number (>100) of jobs on the grid
* Have questions about running jobs on the grid

h2. Collection of legacy wiki pages

"Predefined Datasets Dec. 2015":