DBDesign2015-06-16¶

Officially have fermicloud045, and a "poms" account, empty repository here in the Redmine project.

Need to have references to Campaign level tasks. Task maps to 1 jobsub_submit command.

([Person/Experimenter]>----------<[Experiment]------------------+
           |                          |                         |
             --------<[Campain]>------                          |
                          |                                     |
                          A          /--+                        |
                       [Tasks       ]---+ waits-for              |
                       [order       ]                            |
                       [in_datset   ]------------------[Task_Definition    ]
                       [out_dims    ]                  [launch_cmd_template]
                       [completion_threshold]          [param_dict         ]
                       [param_dict  ]                  [in_files_per_job?  ]
                       [            ]                  [name               ]                       
                            |                          [out_files_per_job? ]
                            |
                       [Jobs     ]
                       [Status   ]
                       [cpu_type ]
                       [node_name]
                       [host_site]

                     [Services     ]
                     [status       ]
                     [update_time  ]
                     [host_site    ]                     

swhite will request a poms_dev database.

Other Notes¶

A campaign can have multiple (ordered) batch level tasks attached to it.

A task should have a dataset name as input. It will then generate an output dataset or dimension string.

How many output files are in SAM? If Current == Expected, then the task is complete.

A job is an individual process.

There can be N jobs attached to a task.

Jobsub will submit the task, which will then submit N number of jobs.

A task is one jobsub submit command.

A task will have a task definition.

We are starting with tasks for now. Campaigns for later.

There can be 3 or 4 montecarlo task types (definitions), so task types (definitions) will be very specific.

The task_definition table will contain:
--launch script
--json of parameters

Jobs Table: State, kind of node they are on, cpu type, facility name, and node name.

We need a completion threshold for task dependencies because we don't want to wait around.

Services Table: SAM, dCache.