Project

General

Profile

09/04/14

Notes taken by M.Tamsett & C.Group. Meeting docdb: http://nova-docdb.fnal.gov:8080/cgi-bin/DisplayMeeting?sessionid=2441

Attendance

Robert I. Micheal, Eric, Paola, Nitin, Nate, Satish, Kanika, Chris, Gavin, Craig, Dominik, Ryan, Jeny.

First analysis sample planning (Matthew)

See docdb 11953, this contains a summary of our first analysis needs. It has a few caveats mentioned in the slides, and it was noticed during the meeting that the FD genie MC estimates are wrong as swap & non-swap files will have differing preselection efficiencies. It was also noted that any unbalance in DB times due to overheads only being accounted for in the first tier that uses them, could be offset with a null tier which caches things. It was also made clear that we don’t have a good idea of the size of the ND cosmic MC needed and that we should also process the ND tricell/activity trigger stream. It was decided that we shouldn’t process the full FD data cosmics sample as this isn’t being used for background estimates this time, only for data/MC comparisons. It was also decided that producing FD genie files in a 1:1 ratio is likely ok.

This talk didn’t address other potential bottle necks in the system, but it was noted that FTSs are design to handle up to 1 TB/h each.

RHC ND was pointed out to not be vital, we could even submit these later in the year. There was also a question over whether the ND is even in the right place in the GDML. This is a problem that would affect both the FHC and RHC samples.

The data estimates in this talk were likely too high due to possible file duplication in the query (should specify a release) and the constraint that we should restrict ourselves to 4DB only data which may reduce the number of files from 96k to ~60k.

It was pointed out that SAM has been demonstrated to work with projects of 40k files, so maybe 60k is ok.

Action Item: Matthew will report on the CAF file size when his results are in.
Action Item: Is a 15 minute cache on the DB too small? Dominick will iterate with DB experts.

Raw2root status, including a discussion of the now fixed duplicate file problem (Paola, Jeny)

File duplication problem resolved and duplicated files removed.

Getting PC hits back up and running (Satish, Paola, Jeny)

Children of duplicate parents have been removed by Paola and Robert. It is ok for Satish to start back up again.

Action Item: Satish will restart PC hit creation.

Progress and plans towards reco keep up (Dominick, Jeny)

Dominick and Jeny worked together on this yesterday and are on the same page with regards to how this processing differs from RAW2ROOT. Dominick has committed everything we need, so we’re just waiting on a tag & a submission script. The plan is that Dominick will start this and then Jeny will automate it. The final output will be reco files and two sets of CAFs blinded & unblinded.

Action Item: Dominick will work with Gavin & Nate to sort out the FTS/dropbox configuration.

Update on MRCC implementation (Kanika)

Adam & Kanika have created the relevant MRXX data tiers. The FTS configs are also committed. Kanika she has had some problems submitting jobs as nova pro and is working on these. The plans for near term production are to process the ND overlay files that Gavin recently made.

Update on geo-jittered and cry/genie files (Nate, Gavin)

Nate had to resubmit the central value jobs due to job sub issues. These are resolved now. The Cry/GENIE sample is complete – just waiting for base files. The geo-jittered reco+ files did not specify the right gdml file in each step so we will need to remake these. The final files will be identified by version number from the wrong one. It was noted that some people had complained that the files are too small.

Metadata issues (Adam et al.)

Action Item: Matthew will email Adam to enquire about this.

Discussion of Simulation tag (all)

  • There was some discussion of the validation procedure. Apparently our model should be that SIM run the production scripts to validate them.
  • Kanika will tag before new art (v11), so SIM can use this to validate. At the moment she isn’t in a position to be able to build offline with this version as DAQ and DDT must be built first. Bryan has reported some problem with this due to an externals version skew, so this will take until next week.
  • Any production jobs should now run out of first ana branch to test this.

AOB

  • Getting process ID into the SAM files - feature request to art, Dominick will do this.
  • Request for R.Illingworth to test postgress db. Dominick will do this too.
  • Eric reported that site ping job wasn’t going off site. Apparently the fermilab OSG grid certificate had expired a month ago! This was fixed and we’re back to running offsite.
    this.