Project

General

Profile

Microboone2015-03-27

  • Jobs
    • List of types of production job types
    • How launched
    • Success criterea
  • Workflow
    • Info in request to approvers
  • Metrics
    • what reports/metrics would you want from system?
  • Monitoring
    • What do they want to see?

Some discussion of Pubs...

Other Notes

Went over document at: http://www.nevis.columbia.edu/~kazuhiro/pubs_doc/PUBS.pdf

Their system is called pubs and have been using this system for only two weeks.

They have some files in SAM, but don't have locations for them because they are in pnfs/scratch.

Q: Are their stages (there are 5 of them) separate projects?
A: One project can handle the full chain or one project per stage.

Book keeping is being done with pubs.

How do you audit success?
-project.py does checks on output files
-exit code status
-verifies checksum

To run a project:
Create a python script and configuration file and register them.
This becomes the project.

Any one generic project can have several instances; each instance having its own configuration file. 1 project can have 45 instances.

How they do reprocessing with sequence numbers is a limitation.

Perhaps have run_number and sub_run_number as inputs for the configuration file?

If it is a project with multiple stages, let's say 7 in this case, you can do:
Passed Passed Failed Idle Idle Idle Idle

Have the system pull tasks off the queue and use jobsub to submit the job. (Robert)

A sub_run is never more than one file.