SamWiseMinutes1 23 14

Forgot to take attendance: Nate, Gavin, Andrew, Craig, Chris, Adam, Eric, Jan, Mark, and others...

Action Items
*Fill in the matrix!
*Nate: review output from failed reconstructions jobs.
*Nate: LEMSUM files for Chris B are now a priority.
*Can we take a short cut to get Chris files? Filter + LEMSUM without PID to.
*Gavin: test on 1000 Raw2Root -- move onto the major reprocessing.
*Eric will submit test jobs.
*Gavin: Make sure fcl files in CVMFS are okay
*Adam: fix metadata module. tried to commit a fix for the metadata module – fix broke metadata. Fails during the linking stage.
*Eric: Run reconstruction with properly configured fcl.
*Mengel: Add –f option to JobSub SAM wrapper.
*Jan: update novasoft setup script for database reader password
*Dominick: Add metadata to CAF file


  • Resubmitted jobs for MC, last time it had bad metadata (too many colons) * 512 jobs submitted – 512 log files back, but not files (96 successes?)
  • two problems, left an important option off (reason for 96 success not producing files)
  • failures due to CVMFS failures (7000/10000 fermilab nodes could not see CVMFS)
  • Jan reports recent issues due to OASIS server (Andrew mentions that there is an OSG ticket) * Renaim Cronjob is working * Resubmitted the 512 and 396 were in SAM. * Reconstruction jobs submitted on the 396 (still a large failure rate)
    >* 52 back this morning
    >* SAM monitoring for the project claims 271 seen, 139 busy, 70 finished, 62 errors.
    >*Are the failures CVMFS?
  • Reconstruction step may be up and running.


  • Ran reco on Adams Root dataset. Failed on each event (for known reason, fcl configuration), but ran fine. * Would like feature to send extra files long with the job (Mengel — JobSub has this but can't do it in the current wrapper) * CVMFS novasoft setup needs to point to the new db password.


  • Ready to hit the go button on Raw2Root
    > (Andrew suggests submitting 1000 jobs to see how it goes) * If this works well move to the cronjob * Already running several thousand jobs for processing.


  • Rolled back development – will update once metadata module is fixed. * New DocDB password.