Project

General

Profile

SamWiseMinutes1 30 14

Eric, Jan, Michael, Andrew, Craig, Dominick, Chris, Nate, Gavin, Nick

-  Kanika will take over the software maintenance!  Thanks to her!

Some notes:

Result of the Monday simulation discussion?
- Craig: sorry I missed it, but that meeting usually isn't on my radar… Maybe it should be.
- Nate was there:
--- Jim was receptive at convener's meeting – Alex was not sure it should be detector simulation responsibility.

MetaData module changes all characters to the same case.
- Will add the base release (with correct case) to the metadata.

Job statistics discussion:
- We need to improve the way that we gather statistics:
-- Grid monitoring
-- SAM File handeling
-- Mark --> Jon – improve database data logging

- Chris B — we need to make sure we keep our log files. Is there a way to stick them in SAM.

Database issues: only 200-300 jobs can run before we hit the limit.
-- Jon/Igor were working on this.
--- Queue is only 200 deep.
--- If queue is rejected job gets rejected and job should try again.

-- Simple fix can mask the problem – artificial random staggering is already there.  
--- In crease the staggering.

This weeks action items:

- Use issue tracker.
- Adam: recovery dataset for Raw2root? 
-  Gavin:  New LEM part files that pass filter.
- Move Chris B's files to the FTS.
-  Next steps:  Merging the LEM files back into the LEM part file.  
-- Nate is concerned about this step.
-- Gavin/Chris/Nate/Andrew -- flesh out the solution.
- Nate/Gavin:  coordinate on plan to complete PID migration to SAM.  
- Start moving toward early MC production steps next?  Priority?  FD cosmics to help with calibration might be highest priority.

Action items from last time:

- Nate: Get the LEMSUM test files for Chris.
-- Done. Files were empty, but Chris could still get started with them…
-- Nate took a look at the metadata.

- Gavin: Make sure fcl files in CVMFS are okay.
-- to avoid hard-coded file path: PID libraries are not shipped with the library.
-- Not quite yet – PID part – make new environmental variable PID library path.
-- fcl can't read environmental variable.

- Jan: update novasoft setup script for database reader password (CVMFS?).
-- Done. Oasis okay.
-- New ifdhc..ifdhart.. Major update. Some issues with the ArtSamWrap
- philosophy of how this works with CVMFS needs work.
-- NOvA CFMFS – two setup scripts for setting up cvmfs (now include distro_base)
-- Now setup scripts will work everywhere.

- Nate/Gavin: coordinate on plan to complete PID migration to SAM.
-- No progress yet.

- Gavin/Nate/Adam: not a huge rush yet for the metadata in CAF files.
- Dominick needs examples
-- Progress — won't be free until middle of next week — really wants some examples

- Art, Mark, Andrew : everything done by novaana – would like all of the ownership changed to novaprod.
-- Progress: Question from Andrew: What is all? FTS – give prod permissions to write not ana.
-- changing things on disk. New drop boxes for each analysis pass. Organizational structure to file things away.
-- some things in dcache have old permissions.
- All:  nova.standard -- needs to be set by true for all production .

- Adam: follow up on errors in keep-up Raw-to-root jobs.
- A few corrupted files according to metadata. Failed during copy?
-.1 – 1 percent failure — still unknown?

- Jan: Follow up on volatile DCACHE.
-- Followed up — waiting on SCD?