Matthew, Jonathan D., Craig, Alex, Kanika, Gareth, Chris, Satish, Susan, Nick, Dominick, Jeny, Paola, Joseph, Ryan.

News (Matthew)

Welcome back - thanks to Kanika and Dominick and all the others who worked over Christmas.

  • Calibration v6 - All done and ready - thanks to a software fix from Kanika. * Bad channels tag (v6) - did it happen? - Matthew will follow up with Jon to determine what went on here. * Build machine upgrade - Kanika proposes that we upgrade the build machine from slf5 to slf6. We'd then use a gpvm machine to build the slf5 build. Jon D. will submit a service desk ticket to get this going. There was some discussion about what fraction of offsite machines are slf5. Paola can determine this. There was then some discussion about whether we could get dedicated sites (OSU, SMU, Harvard etc.) to upgrade. Matthew and Craig will follow up on this.

First analysis production status (all)

  • FD data: Cosmic stream is all done pre-shutdown. NuMI stream is done through LEM but no PID files have appeared (Dominick was waiting on the all clear from Chris). Dominick will submit these jobs now.

We want to add post-shutdown FD data to this dataset. There is about 3M numi spills (about half the size of the pre-shutdown dataset) and about 52M cosmic spills (about the same size as the pre-shutdown cosmic sample and about 3 times what we processed) of data on tape now. The consensus was that if bad channels for this dataset is ready then we can start to process the NuMI data ASAP and can later think about including the cosmic stream.

  • FD beam MC 14 DB: Is this complete? - it looks complete. Matthew will email Gavin to confirm.
  • FD cosmics real runs - extra events ready for PID (note this will take about a week to LEM) - Matthew will contact Gavin to get this going. The 14DB data ready for reco, Satish will start this.
  • ND beam data & MC: both pre and post-shutdown data available through rapid turn around v5 - see below.

Keep up

PC-list keep up is currently not running due to an outstanding question about file names. Paola has a proposed solution but will re-circulate the issue to ensure that the correct solution is implemented. Reco keep-up needs to be re-started with an appropriate release.

Rapid turn around v5 (Dominick)

All done and ready. Some questions about this dataset were:

  • MC definitions missing horn current in name? - this fine for now. * How come 50% of the post-shutdown runs fail DQ? - caused by maximum good run in the GRL. * Are the stats sufficient? For nu_mu yes, for nu_e no, need about 40M events (which is 1e21 PoT@ 2.5e13 PoT per spill). That said we have 2*15M events now, and will no doubt generate another lot with an expanded runs list in the future which should enable us to hit our target.

File renaming (Jeny)

  • Some artdaq files which were renamed have progeny. These need to be update to reflect their new parents name. She corrected all the ND files, but missed some FD files. Script ready to run. * A second issue brought up by Joao involves very old files. Matthew will reply to the email to Joao/Jeny about that issue.

Offsite testing (Paola)


Paola has jobs that work nicely and report results from a large number of sites. Not all of these sites work at the moment and she will continue to work to debug the remaining sites.


  • Matthew to compile list of meta-data to be changed. * There was an email thread about us setting up jobsub tools/client on worker nodes. The issue is that 1. we should not be doing this, and 2. that we should not setup both at once. Gavin and Chris will work on not setting these up on grid nodes by hiding their instantiation behind if statements. This is not something we want to do for a large number of tools as we don't want things that work locally to not work on the grid. The potential conflict between jobsub tools and client hasn't seemed to cause any issues for us yet, so we'll just plan on phasing out tools in the long run. * It was pointed out that there is a version of FD reco keep up sitting on blue arc. One that is not intended for widespread use. We should make sure people aren't using it!