Blinding of MicroBooNE Data¶
Here is the current BNB data blinding guidelines:
- Any file containing data triggered on BNB beam window outside of the Good Run List defined for Neutrino2016 (~5E19 P.O.T.) has been blinded
- blinding has been applied to all binary files, swizzled, reco, and anatree files which contain BNB events (bnb and bnb_unbiased).
- remember that since Physics*.ubdaq binary files contain both BNB and other events, those files are blinded after a 48 hour grace period.
- files have been made unreadable at the file system level, except for the user uboonepro.
- files can still be listed through ls commands and listed through SAM.
- having blinded files in your SAM dataset definitions will cause problems (see below)
- The GRL from Neutrino2016 and dataset are unblinded. These datasets may be found:
- Reco Files http://samweb.fnal.gov:8480/sam/uboone/api/definitions/id/4770860?format=html
- anatree Files http://samweb.fnal.gov:8480/sam/uboone/api/definitions/id/4771009?format=html
- All Calibration, BeamOff, Laser, etc files should be readable at the binary, swizzled, reco, anatree level
Additionally, there is now a SAM metadata value stored that will let you remove any blinded files from your SAM dataset definitions. At the end of your data sets, you can add a qualifier that will exclude files from being included in the datasets. For example:
<uboonegpvm01.fnal.gov> source /cvmfs/uboone.opensciencegrid.org/products/setup_uboone.sh <uboonegpvm01.fnal.gov>>setup uboonecode v06_16_00 -q e10:prof <uboonegpvm01.fnal.gov>>samweb list-files "file_name Physics%.root and run_number = 8700 minus (ub_blinding.blind true)"
Note that this lists all files (swizzled and reco) accessible (includes BNB ext (not BNB or BNB unbiased), NuMI, NuMI ext, NuMI unbiased, etc).
Remember you can use the "minus" qualifier for excluding files beyond just blinded files:
samweb list-files "file_name ana%.root minus (ischildof: file_name Physics%.root)"
to exclude any anatree file derived from a Physics run reco file.
Please note that since files are no longer readable, if you submit a job that tries to read a "blinded" file with project.py, you will end up getting an ifdh transfer error and timeout. In the past, the error response has been:
error: globus_ftp_client: the server responded with an error 550 Permission denied^M program: globus-url-copy -rst-retries 1 -gridftp2 -nodcau -restart -stall-timeout 14400 -dp gsiftp://stkendca45a.fnal.gov/pnfs/fnal.gov/usr/uboone/data/uboone\ /raw/online/assembler/v6_00_00/00/00/44/11/PhysicsRun-2016_1_7_23_25_34-0004411-01202.ubdaq file:////storage/local/data1/condor/execute/dir_1201088/glide_MqVJY\ Z/execute/dir_2973998/no_xfer/working_dir.ypRJOJdsN3/work/PhysicsRun-2016_1_7_23_25_34-0004411-01202.ubdaq exited status 1 delaying 6 ... retrying...
If you see such problems, and believe that the files you are trying to access should be accessible, please contact the Data Management group. In the future, project.py will throw an exception if analyzers attempt to process a blinded file.