Craig, Gareth, Jonathan, Susan, Nick, Dominick, Paola, Ryan, Gavin, Jeny, Michael, Nitin, Barnali, Matthew.

News (Matthew)

  • New FHC v5 flus files are ready.
  • First analysis test production has started. Although the status of this wasn't made entirely clear at the meeting as Nate was on night shift. The first validation datasets have appeared (minus ND mix files) and Gareth will now run MC checkout over them and report tomorrow at the 3pm meeting.
  • The impact of flux changes on the sim validation is expected to be small, so the plan is to just go ahead and validate on these, maybe for next time re-run when v05 files are in.
  • Test track IDs isn't included in the default MCCheckout job yet. Gareth and Susan will work on this.

Back-porting S14-09-09 fixes & related fallout (all)

  • Jonathan back ported the fix (provided by Ryan) to RunHistory in S14-09-09.
  • Matthew re-ran the production tests and all looked well.
  • It was pointed out that the PChits keep-up log files are very large and that the error messages look suspicious like the same issue as seen in S14-09-09. Matthew will iterate with Paola and Jeny to figure out what is happening.

SLF6 on CVFMS (Kanika)

Kanika sent and email describing this problem:

“Currently we only have the slf5 build of novasoft available on cvmfs. If a job lands on an slf6 cvmfs node, (assuming that the setup is done correctly), the slf5 version of novasoft will be setup. This works and causes no problems, but do we still want to make the slf6 build available through cvmfs?

The advantage is that soon enough, fermilab machines will all be updated to slf6, and then we would have to maintain both slf5 and slf6 versions of novasoft to be able to use all cvmfs nodes available. This is because slf6 build doesn't (or at least didn't in the past) work on slf5 nodes. So we might as well start testing now.

I don't see any disadvantage to this, but others might. Thoughts?”

Jonathan pointed out that both versions are built and pushed to the servers but it was unclear why this didn't solve the issue described here. He will iterate with Kanika to implement here suggestion.

Simulation production update (Nate)

Nate was absent so this update couldn't happen. Matthew caught up with Nate the next day and the status was: Validation production is going well and the datasets are ready for MCCheckout. The only problem was that an issue with the meta-data module is preventing ND mix files from being declared to SAM.

Action Item: Matthew will iterate with Adam to find a resolution.

PC-hits keep-up update (Paola)

  • Paola activated her first cron job on Thursday using the old S14-08-08 release for FD keep-up. This appears to have processed all available runs. This means that we're ready for the next stage.

Action Item: Matthew needs to clarify with Satish about which release the ND fcl is included in and also to find out if there was a physics choice behind the run boundary.

MR brem update (Nitin)


  • Failure rate reduced for ~70% to very low.

Action Item: Matthew will work with Nitin to plan out the next stages of this work.


  • Dominick wants to rerun keep-up with new fixes. * It was pointed out that it is conceptually nicer to make "S" snapshots at the same time as "FA" tag. Then to run keep-up and other such processes in "S" releases. * It was decided to start freezing first ana tonight. The simulation packages as list on this page will now be frozen in the FA branch. * At the start of October FNAL will make the switch from SLF5 to 6. We need to be aware of this and make the appropriate changes. * Paola found a bug concerning the "not" operand in SAM queries. This will be fixed by Robert I. We need to understand if this has impacted anything.