Project

General

Profile

Feature #24826

Add a mechanism to run custom scripts at the end of the Glidein execution

Added by Marco Mambelli 5 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Category:
-
Target version:
Start date:
08/19/2020
Due date:
% Done:

0%

Estimated time:
Stakeholders:

CMS

Duration:

Description

Glideins allow implicit and explicit <files> files to be downloaded and executed if desired.
All these scripts are
  • before the condor invocation
  • periodic script run also periodically while condor is running

There is a need to have some scripts (at least one) run also after condor completes to perform cleanup operations
This could use a mechanism similar to GLIDEIN_LAST_SCRIPT, e.g. adding the attribute GLIDEIN_CLEANUP_SCRIPT, and have a variable point to a script that is singled out to run after the condor invocation instead of before it like all other setup scripts.

This would allow a cleaner implementation of mechanisms like cvmfsexec using mountrepo/umountrepo.

NOTES:
  • the script should still be downloaded at the beginning (same checksum and download mechanisms)
  • the script could invoke other scripts. This could be a standard mechanism advertised in the instructions and used also by operators or something internal available only for GWMS developers. E.g. another attribute w/ an ordered list of scripts to run? Or GLIDEIN_LAST_SCRIPT being a list instead of one name?
  • state clear error conditins: if the script is missing (or the download fails) will the glidein fail? or run w/o executing it? It there is an error what should be the glidein execution status? How can problems be reported? Standard error_gen mechanism?

Related issues

Related to GlideinWMS - Feature #21885: Support to run test and periodic scripts within SingularityResolved02/11/2019

History

#1 Updated by Marco Mambelli 4 months ago

  • Stakeholders updated (diff)
  • Assignee set to Marco Mascheroni
  • Description updated (diff)

#2 Updated by Marco Mambelli 3 months ago

  • Related to Feature #21885: Support to run test and periodic scripts within Singularity added

#3 Updated by Marco Mambelli 2 months ago

  • Target version changed from v3_7_1 to v3_7_2
  • Assignee changed from Marco Mascheroni to Marco Mambelli

A first mechanism version has been implemented in v37/24826_v1
This uses an attribute, GLIDEIN_CLEANUP_SCRIPT, to identify the file and there can be a single cleanup script.

&lt;td&gt; &lt;b&gt;GLIDEIN_CLEANUP_SCRIPT&lt;/b&gt; &lt;/td&gt;
&lt;TD&gt; String &lt;/TD&gt;
&lt;TD&gt;
&lt;P&gt;Used to run a custom file at cleanup time (right before quitting the Glidein)
instead of the default setup time. The custom file must be specified in the file list as usual.
The value is the file name, as specified in the file list.
NOTE: parameter and job_publish should be both True for this attribute.
&lt;b&gt;Must be a parameter.&lt;/b&gt;&lt;/P&gt;
&lt;/TD&gt;

The next iteration will make something more general

#4 Updated by Marco Mambelli 2 months ago

Here attached is v1b, the implementation by Marco Mascheroni of the same mechanism that was done in parallel.
It is incomplete.

#5 Updated by Marco Mambelli about 2 months ago

  • Assignee changed from Marco Mambelli to Bruno Coimbra
  • Status changed from New to Feedback

#6 Updated by Bruno Coimbra about 2 months ago

  • Assignee changed from Bruno Coimbra to Marco Mambelli

All changes look good to go.

#7 Updated by Marco Mambelli about 2 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF