Blinding of MicroBooNE Data¶
This article describes the 2019 update of MicroBooNE blinding policy and practice.
The blinding procedure described in this article generally follows the procedure proposed in
DocDB 27735, presented at the Oct. 19, 2019 Analysis tools meeting.
Links to older (Kirby-era) blinding procedures articles:
MicroBooNE data are blinded according to the following principles.
- Binary TPC physics data are blind.
- BNB trigger swizzled TPC data are blind. This includes the following swizzler trigger streams (depending on swizzler version).
- Certain runs have been declared unblind. Currently, there are three such sets of runs.
- Run 1 5e19 POT (sam dataset run1_5e19_unblind_runs).
- Run 3 1e19 POT (sam dataset run3_1e19_unblind_runs).
- Runs 1-3 rolling unblind data (sam dataset rolling_unblind_runs).
- Descendants of blind files are blind, with some exceptions.
- Most plain root and histogram files are not blind.
- There is no blanket exception for analysis tree, cell tree, pndr, larlite, or larcv files. These files are blind if their parent is blind.
- Reco2_lite files produced during MCC9 reco1 processing are unblind. These are to be used for evaluating data quality.
- Filtered data sets and data streams that have been approved for unblinding are not blind. Some data sets and data streams that are approved for unblinding are as follows.
- MCC8 CC inclusive filter.
- MCC8 pi0 filter.
- MCC9 coarse CC inclusive filter.
- MCC9 CC inclusive reco2 data stream.
- MCC9 pi0 reco2 data stream.
- MCC9 NC pi0 reco2 data stream.
- MCC9 bad beam reco2 data stream.
- MCC9 antibdt reco2 data stream (selected runs).
- MCC9 single track filtered data.
There are two sam parameters that are involved with blinding.
When files are first declared to sam and stored in enstore, both parameters are undefined and the files are world-readable. When a file is evaluated for blinding by a blinding script, parameter
ub_blinding.processed is set to "true," and parameter
ub_blinding.blind is set to "true" or "false," depending on the blinding decision. The file mode is also set to match parameter
ub_blinding.blind whenever that parameter is updated.
Analyzers can query unblind files in a dataset that is subject to blinding using a sam query such as the following.
$ samweb list-files "defname: some-definition and ub_blinding.blind false"
Accessing Blinded Data¶
Blinded files have a permission mode such that they are readable only by user
$ ls -l /pnfs/uboone/data/uboone/raw/online/assembler/v6_00_00/00/00/34/20/PhysicsRun-2015_10_15_17_12_27-0003420-00003.ubdaq -rw------- 1 uboonepro uboonepro 1784087088 Oct 15 2015 /pnfs/uboone/data/uboone/raw/online/assembler/v6_00_00/00/00/34/20/PhysicsRun-2015_10_15_17_12_27-0003420-00003.ubdaq
If an unprivileged analyzer tries to read a blinded file, the result will be some kind of read permission failure.
To access blinded files interactively, you need to log in to account
uboonepro. To access blinded files in batch jobs, you need to authenticate your job submission using a grid proxy that has role "Production," or submit using
jobsub_submit option "
--role Production" (in
project.py terms use element
<role>Production</role>). If you invoke
jobsub_submit while logged in as
uboonepro and use the managed proxy for authentication, you automatically have role Production.
Analyzers do not have access to role Production by default. Production role needs to be granted to users within the VO. If you think you need Production role to access blinded data for an approved purpose, submit this request to MicroBooNE management. Granting Production role requires a Fermilab Service Desk ticket.
Offline blinding and unblinding of files (that is everything except ubdaq files) is handled by the following cron script.
This script is invoked by a uboonepro's crontab on uboonegpvm01.
$ crontab -l | grep blind # Update blinding. 13 4 * * * cd /uboone/app/home/uboonepro/cron;./cron_blind_data.sh > cron_blind_data.out 2> cron_blind_data.err
Internally, the above cron script invokes the following six scripts.
This script blinds bnb swizzled data.
This script unblinds previously blinded data in unblind runs.
This script blinds children of blinded files.
These scripts unblind filtered datasets and data streams.