SamWiseMinutes1 27 14¶
Here are my notes from our meeting:
Attendance: Chris, Dominick, Matt, Nate, Andrew, Eric, Gavin, Adam, others…
- Nate: Get the LEMSUM test files for Chris.
- Gavin: Make sure fcl files in CVMFS are okay.
- Jan: update novasoft setup script for database reader password (CVMFS?).
- Nate/Gavin: coordinate on plan to complete PID migration to SAM.
- Gavin/Nate/Adam: not a huge rush yet for the metadata in CAF files (Dominick needs examples)
- Art, Mark, Andrew : everything done by novaana – would like all of the ownership changed to novaprod.
- All: nova.standard -- needs to be set by true for all production .
- Adam: follow up on errors in keep-up Raw-to-root jobs.
- Jan: Follow up on volatile DCACHE.
Action Items from last time (with updates):
*Fill in the matrix! Nate filled in some
*Nate: review output from failed reconstructions jobs.
Just CVMFS flakiness -
*Nate: LEMSUM files for Chris B are now a priority.
Can we take a short cut to get Chris files? Filter + LEMSUM without PID to.
Nate has been working on that – but some problems
Requires two rounds of processing. PID part files, but with no metadata.
Metadata module was missing from the output string
PID part files are in SAM
LEMSUM files crashed with error -
There are multiple outputs
Nate knows what to do from
*Gavin: test on 1000 Raw2Root -- move onto the major reprocessing.
Ran 1000 test jobs – 15 minutes.
Started catch up 11496-today – 50,000 Numi files
Adam submitted – still running.
Nate: since 2am this morning,
A few files http status code 409, file already exists.
A few more with some kind of corruption.
~600 nodes – fighting with Gavin
A few cross checks – Newer member
Already processed ~30,000 files
*Eric still mining log files from offsite tests. Raw2root, Numi and cosmic for FD. Many Oasis failures (mostly Nebraska).
*Gavin: Make sure fcl files in CVMFS are okay
Pre-selection one was fine. After modified to handle metadata
PID files need update to reference correct location CVMFS.
*Adam: fix metadata module. tried to commit a fix for the metadata module – fix broke metadata. Fails during the linking stage.
*Eric: Run reconstruction with properly configured fcl.
On Friday, setup file hadn't been updated with database password location.
- Mengel: Add –f option to JobSub SAM wrapper.
*Jan: update novasoft setup script for database reader password
*Dominick: Add metadata to CAF file
Some notes from reports:
- There is some concern that the database interface can't keep up.
- PID part submission – forgot to rename properly. If metadata definition at the command line gets mixed up.
--> Will start an email thread on this.
- Readiness matrix -
- definition editor interface : ?
- Any new file type needs a metadata tool - CAF for example.
- everything done by novaana – would like all of the ownership changed to novaprod.
- Do we need a non production metadata ?
- nova.standard -- needs to be set by true for all production .
- had some laptop problems – now he is up and running again.
Jan: asked about how to use the volatile DCACHE option while pinning ?
-- Andrew: has discussed this with
- Ticket: how much space to request:
-- SRM interface to dcache – not been done recently.
-- Robert Illingsworth.