Project

General

Profile

Wiki » History » Version 92

« Previous - Version 92/127 (diff) - Next » - Current version
Lu Ren, 02/19/2019 02:41 PM


MicroBooNE Production

Coordination for a large data sample pre-staging

MicroBooNE requires the pre-staging of dataset larger than "dev" sample (4000 files or 10 TB file size) to be coordinated with production team.

  • Before making pre-staging request, check pre-staging dataset status table if your requested sample has already been pre-staged. How long the pre-staging last for the dataset depends on various factors. A rule of thumb is if the dataset has been pre-staged within a couple of weeks, then most likely you don't have to pre-stage it.
  • Check the dataset information with samweb commands (learn more about SAM) (More SAM commands)
    $ samweb list-files --summary "defname:Your_Dataset_Name" 
    

    Here is an example:

      samweb list-files --summary "defname:prodgenie_bnb_intrinsic_nue_cosmic_uboone_mcc8.7_reco2" 
       File count:    12287           (file numbers)
       Total size:    34235530158207     (file size in byte)
       Event count:    614350       (available events)
    

Recommended Data and MC Samples for MCC8 analysis (as of Aug. 20th 2018)

Data: MC:

Status of Dataset Prestaging

Dataset # of files Size Requested by Date Status
prod_reco_optfilter_extbnb_v12_mcc8_gooodruns_v4s
prod_reco_optfilter_extbnb_v12_mcc8_gooodruns_v5s
single photon group (Mark, Kathryn) expire
prod_numi_ext_data_run1_reco2_mcc8_v1_goodruns 16209 56TB Colton expire
prodgenie_bnb_nu_cosmic_uboone_mcc8.7_reco2
prodgenie_bnb_nu_cosmic_uboone_MCC8Tune3_mcc8.7_reco2
Marco expire
prod_numi_data_run1_reco2_mcc8_v1_no_anumi_goodruns 25748 ~62TB Nicolo 11/30/2018 expire
prod_numi_ext_data_run1_reco2_mcc8_v1_goodruns 16209 ~51TB Nicolo 11/30/2018 expire
prodgenie_numi_cosmic_uboone_mcc8.8_detsim_mec_reco2 7322 ~37TB Nicolo 11/30/2018 expire
prodgenie_bnb_intrinsic_nue_cosmic_uboone_mcc8.7_reco2 12210 ~31TB Nicolo 11/30/2018 expire
mcc8.17 detector variation samples 4k*19 ~ Kirby 12/27/2018 expire
NuMI EXT 2.5TB cbarnes4 1/7/2019 expire
NuMI chill2 1/7/2019 expire
prod_bnbhsnc0_swizzle_crt_inclusive_mixv5v6_R3_r13696to15500 15k 6.5 TB Salvatore Davide Porzio 01/16/2019 active
prod_bnbhsnc0_swizzle_crt_inclusive_mixv5v6_R3_r15500to17055 14k 3.7 TB Salvatore Davide Porzio 01/16/2019 active
prod_exthsnc0_swizzle_crt_inclusive_v6_R3_r13696to15500 12k 1.5 TB Salvatore Davide Porzio 01/16/2019 active
prod_exthsnc0_swizzle_crt_inclusive_v6_R3_r15500to17055 14k 2.0 TB Salvatore Davide Porzio 01/16/2019 active
prodgenie_bnb_nu_cosmic_uboone_mcc8.7_reco2_dev 4k 10 TB Afroditi Papadopoulou 01/16/2019 active
prod_reco_optfilter_bnb_v14_unblind_mcc8 4.1k 3.23 TB Adam Lister 01/17/2019 active
prod_reco_optfilter_extbnb_ccinclusive_v15_mcc8 10.6k 0.35 TB Adam Lister 01/17/2019 active
prodgenie_bnb_nu_cosmic_uboone_mcc8.7_reco2 43.5k 110 TB Adam Lister 01/17/2019 active
prod_numi_ext_data_run1_reco2_mcc8_v1 cbarnes4 1/24/2019
prodgenie_numi_cosmic_uboone_mcc8.8_detsim_mec_reco2 cbarnes4 1/24/2019
prod_reco_optfilter_bnb_ccinclusive_v14_mcc8 14.3k 0.5 TB Varuna Meddage 01/25/2019 active
prodgenie_numi_cosmic_uboone_mcc8.8_detsim_mec_reco2 7.3k 37 TB Nicolò Foppiani 02/02/2019 active
Various datasets for SPECalibration ~25k in total Sharankova, Ralitsa 02/19/2019 active
mcc8 run 1 extbnb amogan,alister1 02/19/2019 active

Data and MC samples

Find full list of samples at MicroBooNE at Work

List of Production Requests

Find actual list of production requests to the team at List of Production Requests

For production team members

How-to's and useful information
Meetings and Minutes
Offline Production Shift
MCC9 Production
Detector variation sample production
Optical Filter Production
Archiving Inactive Users
Data Blinding
Pubs Era Production Status

For users

Before submitting large number of jobs to the grid, first make sure you have tested your workflow and it is correct. Also make sure your resources request setup follows the uB's Grid Best Practices Please contact the production team, if you
  • Want to prestage and process a dataset with more than 4000 files or size > 10 TB
  • Cancel large number (>100) of jobs on the grid
  • Have questions about running jobs on the grid

Collection of legacy wiki pages

Predefined Datasets Dec. 2015