Project

General

Profile

Adding Tasks and Jobs

Adding/finding the Campaign

In order to add a Task, it has to be attached to a TaskDefinition and a Campaign. For a
given category of job, we should only need to do this part once.

So for now, we have to make sure we have those in the admin screen;
so go to
http://fermicloud045.fnal.gov/poms/admin_screen
(or if you're using your own instance for development, go to /poms/admin_screen there...)

Click on TaskDefinition; note the numerc id of the one you want, or make a new one.
Note that you currently need to enter dates of the form 2005-05-09T00:00 to fill in
date fields, and your numeric experimenter-id for creator and updator.

Thhen go back to the /poms/admin_screen, and click on Campaign and look for a Campaign
that has the task definition, etc. you want, and note it's id (or make one).

Addding the actual task

Okay, now with a task defninition and campaign id in hand, we can add a task.
You can either GET or POST data, i.e with curl with the following data:

field sample value description
campaign_id 1 key from Campaign table
task_definition_id 20 key from TaskDefinition table
input_dataset foo job input dataset if known possibly from jobsub --dataset= option, or SAM_DATASET env var, etc
output_dataset foo job output dataset if knowm
creator 4 experimenter id from Experimenter table
created 2015-07-31T00:05 Current date, usually
status new for new ones(?)
updater 4 experimenter id agai
updated 2015-07-31T00:05 now again
task_order 0 usually 0 for now

For example:

curl "http://fermicloud045.fnal.gov:8080/poms/update_generic?classname=Task&campaign_id=1&task_definition_id=20&input_dataset=foo&output_dataset=foo&creator=4&created=2015-07-31T00:05&status=new&updater=4&updated=2015-07-31T00:05&task_order=0" 

As ouptput from that web call, you should get back:

"Updated Task n."

where n is the new task id. We should now pass that task-id into the job when we run so that we
can track it. The current recommendation for that is to:

export POMS_TASK_ID=_n_

in the environment, and then pass into jobsub_submit by including options:

-e POMS_TASK_ID --lines=+poms_task_id=$POMS_TASK_ID
task_definition_id=20&input_dataset=foo&output_dataset=foo&creator=4&created=2015-07-31T00:0
so we can get that info into the job.

Once the task is started we should be able to update it much more briefly, giving just the task_id and the
thing we want to modify:

curl "http://fermicloud045:8080/poms/update_generic?classname=Task&task_id=_n_&status=running&&updated=2015-07-31T00:05" 

Adding job info

Also, with a task_id in hand, we should be able to add Jobs to the task.

Once again we POST or GET to put the data in:

field sample description
task_id 7 id from task creation, earlier
node_name fnpc3014.fnal.gov hostname
cpu_type i686 cpu type from condor
host_site full jobid from jobsub
status good what list should this be?
updated 2015-07-01T00:00 timestamp
curl "http://fermicloud045:8080/poms/update_generic?classname=Job&task_id=7&node_name=fnpc3014.fnal.gov&cpu_type=i686&host_site=12345@fifebatch1.fnla.gov&status=good&updated=2015-07-01T00:00" 

Now we've added the Job -- but updating it later is harder; we will get status from jobsub_q etc. which
gives us the host_site info back; but we need the returned id to update it. We can track that in the job watching code,
or we need a call to update jobs by host_site...

The curl call in this case will returm:

Updated Job 5.

So we can currently update the job by the id it returns above:

curl "http://fermicloud045.fnal.gov:8080/poms/update_generic?classname=Job&job_id=5&status=better" 

but at the moment we have to remember the job_id of 5 part.