Project

General

Profile

Support #24637

Feature #24546: Add the ability to use CVMFS on sites that do not provide local installations

Integrate CVMFS feature with GlideinWMS code base

Added by Namratha Urs 2 months ago. Updated about 1 month ago.

Status:
Work in progress
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
07/17/2020
Due date:
% Done:

80%

Estimated time:
Stakeholders:
Duration:

Description

Document all updates with respect to the integration of CVMFS feature with GlideinWMS code base (including running all required tests).

History

#1 Updated by Namratha Urs about 2 months ago

  • % Done changed from 0 to 30
  • Status changed from New to Work in progress
Started work on the CVMFS feature integration with GlideinWMS this week:
  • Gained a fair understanding of the integration process and how the configurations work internally
  • Reviewed the glidein_startup.sh script to understand what happens during startup of glideins
  • Experimented with <file> and <attr> settings to understand how the configurations in the factory xml map to the glidein configuration file
  • Completed configuration for the shipping of the executable script and the tarball with the glidein
  • Successful in incorporating parts of the prototype code into the executable script that performs system checks and logs system information during glidein startup
  • Continuing to work on integrating the remaining of the prototype code with GWMS
Issue(s) encountered:
  • (RESOLVED) Tarball containing cvmfs utilities was not being unpacked in the glidein's main directory
    • Defined sequence in the factory config xml had the executable script first followed by the tarball (file named file_list.xxx.lst under the glidein main directory had the exact same sequence)
      • The contents of the tarball do not get untarred until the executable script completed, thereby making the untar location inaccessible while the script was in execution
    • Reordered the sequence under <file> parameter to have the tarball first followed by the executable script that uses the tarball (xml snippet below)
...
<files>
      <file absfname="/etc/gwms-factory/cvmfs_gwms/cvmfs_utils.tar.gz" executable="False" wrapper="False" untar="True" const="True" relfname="cvmfs_utils.tar.gz"  comment="Tarball containing utilities for CVMFS." />
      <file absfname="/etc/gwms-factory/cvmfs_gwms/cvmfs_test.sh" executable="True" wrapper="False" untar="False" const="True" relfname="cvmfs_test.sh"  comment="Wrapper script used for CVMFS detection/setup." />
</files>
...

#2 Updated by Namratha Urs about 2 months ago

  • % Done changed from 30 to 50

Code refactoring in progress for feedback received during the code review meeting and 1:1 discussions. Intermittent code changes will be pushed to the refactor-code branch on the following github repository - https://github.com/namrathaurs/gwms-cvmfs/tree/refactor-code

#3 Updated by Namratha Urs about 1 month ago

  • % Done changed from 50 to 80
  • Completed testing of the CVMFS-related scripts (including the tarball) with the glideins.
  • Working on the last step of the integration process, i.e. to include the script and the tarball to the default list of uploads.
    • Having trouble locating the cgWParamDict.py script to make the modifications.


Also available in: Atom PDF