SamWiseMinutes3 3 14¶
SamWiseMinutes3 3 14¶
Gavin, Nate, Eric, Andrew, Jan, Chris, Dominick, (sorry if I missed you) …
Dominick: PC Hits
- metadata module update to get CAF going
- Meet with some of the new people to get them started with daq2rawdigit
(Siva Prasad-Kasetti, Kuldeep Kaur-Mann, Pavan Poot Pandey and possibly Nitin Yadev.)
Adam and Chris:
- follow up on multiple run sections in a single grid job
- Waiting to here back regarding ND cosmics on AMAZON cloud.
— A lot of MC files coming back corrupt. Many "file open" errors.
— GENIE — will use new jobsub to stress test the system.
— Nice spreadsheet summary of a run.
—> thinking about how to generalize this.
— Test Prague cluster could be available to us soon. Expert at Fermilab ready to finalize this.
—> Just needs to be exercised.
- Nate issues (got disconnected for a minute here - this may not be clear of complete):
— Been working on overlays.
— Empty files with overlays ? Suggest change to overlay strategy.
— Need some rock files to test the overlays.
— Suggest starting over with GENIE files.
— Add this on to a "post script” for NUMI files
- PC Hits (Dominick) ->
—> Much debugging for calibration tests last week
—> NOvA production proxy doesn’t like to copy files using IFDH_fetch
—> CD fixed – on DCACHE novapro role was getting mapped to an old disk.
—> Now it should be working
—> In order to use grid resources, studied number of files per job.
—> Multiple jobs using jobsub or artsamwrap
—> Will abandon this for now to get the jobs going…
- DDT files?
— We have three new people.
— Andrew would like to keep all data processing as a central job of the production group.
- Exercise database —
— testing NUMI processing.
— Able to run 18,000 NUMI files in 1/2 day.
— 1500 job concurrently seems stable. Hard to get enough slots to find the limits of the system.
— Good new.
- Multiple sections for job.
— Clearing the metadata is one option to run multiple jobs in single art job – Need adam.
— Art SAM wrap would run multiple art jobs in a grid job– there were commits?
—> Follow up this week… With Adam.. And ART
— Can we get a database warning when things become critical?
—> There is a webpage that lists the database connections.
— Raw2Root —> Siva ? 3 new people.
— LEM merging is working
—> ART issue got fixed that will clean up hackiness with this code.
—> Need new ART code.
— CAF: still waiting on Adam to fix up the metadata module.