Project

General

Profile

September 19th

Date:
09/19/2017

Attendees:
Anna M, Brian Y., Marc M.,Robert I. Brandon W., Vladimir, Yuyi, Margherita

Agenda:
- Plans for next release v2_3_0 (Going over the list of tasks still open and off any release)

Discussion:

1. Plans for next release v2_3_0

ANNA:
Going over the list of tasks still open and off any release and see if we can include in the next one.

12612: Revised documentation.
ANNA:
Work in progress

11392: failed column in spreadsheet.
MARC:
Drop, unless user ask for it. Rejected.

12195: test button.
ANNA:
This is something that was also discussed at time of OPOS.
Check that nothings is missing and everything is working.
Discussed the possibility for POMS to make a quick test on a campaign -base to test configuration and workflow.
For the experiment would be
important to know that nothing is missing in their configuration before submitting
the whole campaign. This would improve the submission efficiency.
MARC:
We could launch a test job. Or submit a small run but it wouldn't be immediate
because there are many parameters involved.
ANNA:
We have project.py which we could use for some testing.
MARC:
To test we need user's info on how to submit the job.
ANNA:
The parameters should be already in the info in the campaign layers.
MARC:
It will need a DB change to store more parameters override.
ANNA:
OK let's talk offline.

12617: POMS as a valid alternative of PUBS for MicroBooNE.
ANNA:
Need to meet with Mike Kirby for a follow up.

13336: Email Notifications.
ANNA:
It would be useful to have email sent when campaign is done , but then need to agree
on what is the "Campaign-completion". Maybe email when jobs are done (first notification) and then when files are located (second notification).
MARC:
There might be some code already written by Michael Geith for email..
ANNA:
We could prepare the infrastructure, have configuration parameters so the exp will decide
who to send email to, like production group.. and what...

13337: Discussion about event splitter.

ANNA: It's difficult, main issue is to be able to decide the "smallest" unit, event? file?
and it can depend on the experiment.. submit 10 jobs or 10 events?.. THis is something to discuss with the exp.
ATLAS has something in place for this, PANDA , and the smallest unit is the event.
ROBERT:
Suggest not to put too much effort into this yet. There is some ongoing work
on event handling and data management..
POMS probably will not need to know at an "event level", more likely will be at file level.
MARC:
The only thing POMS care is dataset that could be split.
ROBERT:
Data set is simply a chunk of data that can be subdivided.
YUYI:
Example, in CMS that are blogs and files.
ROBERT:
Different ways of slicing dataset.

13923: Error when specifying campaign_definition via python interface
MARC:
Solved. Closed

14373: Register_poms_campaign error
MARC: Closed.

14383: Key error in update_job on JobFiles
MARC: Fixed, can be closed.

14422: Support data transfer campaign layers
ANNA:
Under discussion.
MARC:
It will be for our next big release for Dune.
ANNA:
We should wait ti Dune knows exactly what they need.
The idea would be to be able to run at both sites and be able to merge.
MARC:
Data staging needs to be specified.
We could start an implementation where we do staging.

14385: Consider e-mail reports to experiments if they ask for.
ANNA:
It's related to email report..
MARC:
Related to front page with display of dCache.
Be able to hold jobs for the exp.
ANNA:
If persistent dCache is 99% full we need to put hold on submission.
But we should have a mechanism to put output under scratch area.
IN OPOS we did this for operator to check. POMS is smart enoughnto check automatically.

14841: Revised custom plots.

14894: Already done, we can close it.

14896: Discussed with Steve, have a supervisor for each exp, provided by them

15069: Need to check with Steve.

15207: Done, closed.

15938: Archiving Tasks, Jobs and possibly job logs
ANNA: It should really go in next release.
MARC: Possibly archiving jobs logs.
ANNA: Discuss to have exp to have logs under DCache.
We need to make clear with the exp that if they need
more than 2 weeks worth they need to tell us where to store.
MARC: We will need to have then ability to "un-archive".

16667: Check the questions marks
ANNA: Important, about documentation.
we already talked about this before: have a brief information
as a tooltip, then click and go to Wiki page in another tab.
Also having a brief description when the mouse goes on the question mark.

Action Items:
Please take a look at all the tasks and update status.