Project

General

Profile

Feature #21761

POMS support users' own executables?

Added by Yuyi Guo 8 months ago. Updated 8 months ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
-
Start date:
01/23/2019
Due date:
% Done:

0%

Estimated time:
Scope:
Internal
Experiment:
-
Stakeholders:
Duration:

Description

An analysis user ask if POMS can use users’ own modification of analysis modules that differ from those official modules in cvmfs? The is of the imporant feature of "project.py". They want to have it in POMS world.

Below is his words:
"One critical prjoect.py feature used in these workflows is the “local tarball” feature. I want to check with you if that feature is preserved in the new tool. If not, this might be a higher priority item as most of analysis users use this feature.

This feature tars up their code directory, which contains users’ own modification of analysis modules and differs from those official modules in cvmfs. Project.py takes a single “<local>>Path</local>” in the xml file and takes care of making the tarball on submission node, unwinding them on worker node, and setting up corresponding local larsoft environment."

History

#1 Updated by Marc Mengel 8 months ago

The tarball feature is supported by jobsub... so you would just add to the [submit] section:

[submit]
...
tar_file_name = tardir:/path/from/local/tag

in the job it is visible as $INPUT_TAR_FILE.

We may need to add a prescript= to do whatever other setup of that area that the larbatch stuff does.

#2 Updated by Marc Mengel 8 months ago

Looks like the relevant bits of condor_lar.sh are at:

https://cdcvs.fnal.gov/redmine/projects/larbatch/repository/revisions/develop/entry/scripts/condor_lar.sh#L977

he hacks up the setup script to a setup.local, and sources it and then runs "mrbslp".

Note that he fetches the tarball himself, rather than using the jobsub feature to get it, thus dodging the efforts they're making to make that more efficient...



Also available in: Atom PDF