GlideinWMS Project information¶
Quick links
- For Stakeholder information, see StakeholderInfo
- For New Member Orientation, see NewMemberOrientation
- For Meetings schedule and notes, see Meetings (Weekly_Meeting_Notes)
- Table of contents
- GlideinWMS Project information
- Gaining access to the glideinWMS code
- Continuous Integration (CI) using Jenkins
- Making Changes to the code
- Making a Glideinwms Release
- Creating the Glideinwms RPMs
- Installing the Glideinwms RPMs
- Testing GlideinWMS
- Glideinwms developers documents
- Glideinwms On Cloud
- Containers with Glideinwms
- Glideinwms Accounting Improvements
- Removing the big binaries from the tree
- Cleaning Remote Branches
Gaining access to the glideinWMS code¶
To gain write access to the code, you need -
- A valid Fermilab Kerberos principal * A valid services account * Developer level access to the glideinwms project in redmine.fnal.gov
Once you have the access you can checkout the code using
git clone ssh://p-glideinwms@cdcvs.fnal.gov/cvs/projects/glideinwms
NOTE If your authentication is failing (Permission denied ...) even if you have a valid Kerberos ticket, make sure that SSL is configured to use Kerberos.
E.g. in ~/.ssh/config you have:
GSSAPIAuthentication yes GSSAPIDelegateCredentials yes
Continuous Integration (CI) using Jenkins¶
Fermilab-wide CI runs also GlideinWMS tests and is available here: http://dbweb6.fnal.gov:8080/GlideinwmsCI/app/view_builds/index
Click on the cells to get the result details and access the log files for pylint, unit_tests and futurize_tests for a specific build. The top row are the latest tests for each platform (sl6/sl7)
The remaining of this section refers to the GlideinWMS managed Jenkins.
NOTE: GlideinWMS related Jenkins projects are parameterized. Do not change the project configuration in the Jenkins unless you know what you are doing. Doing so may break automated tasks.
Getting Access to the Jenkins CI¶
- Open a service desk ticket directed to the Build services group.
- Use your services account to get a FNAL CI-Logon certificate.
- Upload your certificate into your browser to access the CI service.
Setup Your Account to Trigger Builds/Tests Using CURL¶
- This step is optional but lets you trigger Jenkins jobs remotely
- Login into: https://buildmaster.fnal.gov
- Generate API Token for remote builds/tests using CURL
Pylint & PEP8 Validation Tests¶
Jenkins Project: https://buildmaster.fnal.gov/job/GlideinWMS-Pylint
Daily Automated Tests¶
- Daily tests are run from GlideinWMS GIT project account as a crontab. p-glideinwms@cdcvs.fnal.gov
Running Build/Test Jobs Manually¶
Using Jenkins Webpage
- Login into: https://buildmaster.fnal.gov
- Go the project you want to run
- Click "Build With Parameters" and input the parameters you want to build with. For example, GIT branch(es), mail address to notify on completion, etc
Using CURL
# Sample Command # Project Auth Token: This is available from the project configuration page if you have access to the Jenkins curl -X POST "https://<username>:<API Token>@buildmaster.fnal.gov/job/GlideinWMS-Pylint/buildWithParameters?token=<Project Auth Token>&GWMS_BRANCH=<branches>&GWMS_MAILTO=<email address to notify>"
Running manual tests locally¶
After cloning the Git repo you can test manually BATS, shellcheck, pylint/pep8, unit tests, and futurize (for the Python2 branches). Futurize uses ./glideinwms/build/jenkins/run_futurize.sh
, all other tests use ./glideinwms/build/jenkins/runtest.sh
. Use "script -h" to know how to run each script.
Most tests require only Python, a limited number uses libraries only on Linux (e.g. libcrypto, htcondor), so running on a fermicloud VM is recommended.
The new runner allows both to check your current files and to clone the repo in a temporary directory and run the tests on the desired branches.
Supposing that you are in the working directory and cloned the repo into ./glideinwms, available scripts are:
./glideinwms/build/jenkins/runtest.sh -i pylint -a ./glideinwms/build/jenkins/runtest.sh -i pyunittests -a ./glideinwms/build/jenkins/run_futurize.sh
Making Changes to the code¶
Supported versions¶
At the moment, 4/10/19, support for GWMS v2.x instances has been discontinued for a while.
v2 code has been removed (hopefully completely). Remaining v2 only code (if any) should be further removed, making sure that this breaks nothing.
GlideinWMS 3.4.x and 3.5.x code should work w/ Python 2.7 and 2.6
Code Branching Guidelines¶
To Branch or Not To Branch¶
You should always create a development branch for your work. Creating a branch may not be needed for very minor change to docs. Here are some of the guidelines to make this decision.
Create a Branch when ...
- Adding new feature(s) to the code * There is a possibility of incompatibility * Multiple developers are working on the feature(s) in a collaborative manner
Branching and Merging Tips¶
- Create the branch with the branch name that can be easily associated with you or the ticket. Preferred branch naming convention is v3N/xxxx where v3N => this ticket is associated with glideinwms v3.N.x and xxxx => ticket number * Make your changes, test them * Only when you are convinced that your changes are tested and working, assign the ticket to another team member for feedback * If feedback is ok only then merge them back to the respective version branch and/or head. * Remember to update the tags.txt file describing the feature/bug fix * Test the changes after doing the merge to make sure you do not break anything or create incompatibility.
Handling pull requests from collaborators¶
- Changes should be given to you in form of a github pull request with the remote branch name
# STEP 1: Checkout the changes to redmine project repo git checkout <redminerepo_branch_to_fork_from> git checkout -b <redminerepo_patch_branch> git pull https://github.com/<github_patch_provider_username>/glideinWMS.git <github_patched_branch> # STEP 2: TEST THE CHANGES # STEP 3: Merge the changes and push to redmine git checkout <redminerepo_branch_to_merge_into> git merge --no-ff <redminerepo_patch_branch> git push -u origin <redminerepo_patch_branch>
Code Formatting Guidelines¶
Formatting the Code¶
- Use standard libraries as much as possible
- Try avoiding the deprecated API's
- Never use tabs for indentation. Use spaces for indentation.
- Standard python code has 4 spaces for indentation, so use it.
- When in doubt always check with
http://www.python.org/dev/peps/pep-0008/
http://www.python.org/dev/peps/pep-0257 - Use the alias of a type when available (ie. "float" over "types.FloatType".
- To check if a variable is of a certain type, use isinstance(variable, type).
- Don't raise a message directory, pass a message to another function (ie. raise ValueError("My message")
- "has_key()" has been depreciated and is not available in Python v3. Use "in" instead: key_var in dictionary_var.
- Use absolute imports: from future import absolute_import
- Use the print function because is available also in Python 3: from future import print_function ... print(...)
- For more visit Code Formatting Best Practices - NOTE: these best practices are checked by the Futurize tests and are enforced in the code!
Tips in a Nutshell¶
- Use 4 spaces per indentation level.Never mix tabs and spaces.
- Maximum Line Length should be preferably set to 79 or 80 characters
- Separate top-level function and class definitions with two blank lines.
- Method definitions inside a class are separated by a single blank line.
- Extra blank lines may be used (sparingly) to separate groups of related functions.
- Code in the core Python distribution should aways use the ASCII or Latin-1 encoding (a.k.a. ISO-8859-1). For Python 3.0 and beyond, UTF-8 is preferred over Latin-1
- Imports should usually be on separate lines rather than comma-separated.
- Comments that contradict the code are worse than no comments.
- Comments should be complete sentences.
- Block comments generally apply to some (or all) code that follows them, and are indented to the same level as that code.
- Use inline comments sparingly.
- Follow sections in link above for Documentation Strings, Version Bookkeeping, Naming Conventions
- Avoid extraneous whitespace in the following situations:
Bytes and (Unicode) Strings¶
This applies to Python3, but some preventive changes should be applied also to Python2.Types:
- byte-like objects (aka bytes strings): bytes, bytearray
- (Unicode) strings: str
- AnyStr (used in type annotations): byte-like objects and Unicode strings. Sometimes w/ encoding specifier AnyStrASCII, meaning that only the characters in that encoding are accepted
var_bytes = "b'qwert'" var_str = "qwer" # The following are all true var_bytes != var_str var_bytes == var_str.encode("ASCII") var_bytes.decide("ASCII") == var_strOpen the files as binary ( wb/rb ) unless they are really only text. Using binary files:
- helps w/ compatibility w/ python2
- is easier to operate on system files modification
- there are no changes/additions in the end-of-line character
- utf-8 is the default in Python3 (unless the locale chooses something else)
- utf-8 is also the default encoding used in M2Crypto
- ASCII if there are only ASCII characters (like in HEX and base64 encoded content) use this
- latin-1 is a 8 bit (0-256 char) encoding that leaves bytes content invariant. Useful to maintain compatibility w/ python2, to write system files w/ binary or binary-like content (interacting w/ components that do not support Unicode and encodings)
- Cryptographic and encoding libraries often do accept bytes and (ASCII) strings as input but return always bytes
- str() means that you are printing a message with the representation of your object (type and content) for users to read
- repr() means that you are writing the object into a string/stream meant to be executed w/ eval()
Neither of those converts the content of a bytes string (letters "ABC") into a Unicode string (same letters "ABC"). You have to use decode for that and pick an encoding.
If you see "b''" around all or part of the content of one bytes or str variable, probably str() or %s were misused somewhere to concatenate or convert a bytes string.
Other Python 3 conversion tips/pitfalls¶
Thehash()
function is inconsistent across processes in Python 3.
- In Python 2 you can rely on it returning the same output, always
- In Python 3 it uses a different seed for each process, so the hashing of the same input will yield different results, you need to build your own
hash()
if you want something consistent across processes and runs (see [#25291] for more)
Code Documentation Guidelines¶
Try to document your code so we can easily generate API documentation using epydoc
Refer to http://epydoc.sourceforge.net/epytext.html for EpyText? documentation
Epydoc and Sphinx support now multiple documentation formats
(e.g. via the napoleon extension).
Write docstrings in the Google format which is more readable, reasonably compact
and still can be included in the API documentation. Here an example and bel:
def fetch_bigtable_rows(big_table, keys, other_silly_variable=None):
"""Fetches rows from a Bigtable.
Retrieves rows pertaining to the given keys from the Table instance
represented by big_table. Silly things may happen if
other_silly_variable is not None.
Args:
big_table (Bigtable): An open Bigtable Table instance.
keys (list): A sequence of strings representing the key of each table row
to fetch.
other_silly_variable (str): Another optional variable, that has a much
longer name than the other args, and which does nothing.
Returns:
dict: A dict mapping keys to the corresponding table row data
fetched. Each row is represented as a tuple of strings. For
example:
{'Serak': ('Rigel VII', 'Preparer'),
'Zim': ('Irk', 'Invader'),
'Lrrr': ('Omicron Persei 8', 'Emperor')}
If a key from the keys argument is missing from the dictionary,
then that row was not found in the table.
Raises:
IOError: An error occurred accessing the bigtable.Table object.
"""
...
Here an longer example.
Type annotation is preferred, especially for Python 3. Anyway types can be added also to the docstring, e.g.
arg1 (int): Description of arg1
Coding tips and lessons learned¶
Above is a section about best practices for writing and formatting python code.
Check CodingTips for best practices and lessons learned about coding in general: how to approach a bug fix or the addition of a new feature in GlideinWMS
Making a Glideinwms Release¶
Releases are products for the GlideinWMS users (Factory and Frontend operators). These are:- GlideinWMS releases (full releases via RPM and tar ball, identified by a unique by a Major.Minor.Patch version number)
- GlideinWMS patches (drop-in replacements or patch files for existing production releases)
Current WEB Area¶
Contains documentation and tar ball releases. It is hosted in Fermilab's central web server, accessible via fnalu or mountable by other systems:/web/sites/glideinwms.fnal.gov/htdocs/
Instructions for Releasing¶
GlideinWMS releases are managed by gwms_release_manager package that is in the source tree under build/ReleaseManager.
(For Release manager project information check https://cdcvs.fnal.gov/redmine/projects/gwms-release-manager )
gwms_release_manager automates the following tasks for you:- Creates the required checksum files in glideinwms/etc directory
- Strips off the non-essential files and creates 3 release tarballs (factory, frontend and full glideinWMS release).
- Anything you want to be automated during the release process should go in the release manager
Making a GlideinWMS Release¶
The steps below can be divided in 6 parts:- Changing the files, including tags a spec files and merging in the release branch Git (mostly by hand, using process_tags)
- Producing the tar ball release using the release manager release.py and tagging the release in the Git repository
- Updating the documentation
- Uploading to OSG the source tarball using the release manager osg-release.sh
- Building the RPM using OSG tools (requires a RHEL based Linux host)
- Communicating the release
- Choose the correct release branch (master, branch_v...)
- Make the changes and test them. E.g. make sure that the branches of the resolved tickets in Redmine have been merged to the release branch
- Update the tags file with the release notes and release version (
glideinwms/doc/tags.txt
) - Update the documentation pages
glideinwms/doc/download.html
(including possible links to the release tarballs) - Update the tags in
glideinwms/doc/tags.yaml
and useprocess_tags
to updateglideinwms/doc/tags.txt
andglideinwms/doc/history.html
# edit the tags in YAML format, append at the end even if the order is not important vi doc/tags.yaml python3 build/py3tools/process_tags yaml2txt doc/tags.yaml doc/tags.txt cat doc/tags_old.txt >> doc/tags.txt # now you can check tags.txt less doc/tags.txt # change the history file python3 build/py3tools/process_tags yaml2history doc/tags.yaml mv doc/history.html.new.html doc/history.html
- Create the checksum files and release tarballs for your release. Run the command below giving the release version, --source-dir=<path to glideinwms code> and --release-dir=<path to create release related files>. Release tarballs will be created in <path to create release related files>/v3_6_2/
# For example for release v3_6_2, off master, run the command below. MAKE SURE TO USE ABSOLUTE PATHS for source and release (or relative to where you are when running the script) /tmp/release/glideinwms/build/ReleaseManager/release.py \ --release-version=v3_6_2 \ --source-dir=/tmp/release/glideinwms \ --release-dir=/tmp/release/distro
- You can check that the changes are in the expected files with
git diff
- Edit the spec file to add the release notes
vi build/packaging/rpm/glideinwms.spec
- Commit and push the changes to git including the changes to checksum files in etc directory:
git commit ... git push ...
- Tag (annotated) the release in git and push the tag:
git tag ... git push --tags ...
- NOTE that both the branch and the tag are pushed to the FNAL repository (to build the RPM the script will clone the FNAL repo)
- Copy the release tarballs to the web area (Central Web Server Hosting Project Space)
Final Releases: # Check https://cd-docdb.fnal.gov:440/cgi-bin/ShowDocument?docid=5374 for Central Web Server Hosting # There are different ways to access the folder (fnalu, NFS, mounts, ...) scp files into fnalu.fnal.gov:/web/sites/glideinwms.fnal.gov/htdocs/ Release Candidates: In 'rc' sub-directory under the above directory (e.g. fnalu.fnal.gov:/web/sites/glideinwms.fnal.gov/htdocs/rc/ )
- Check out the doc subdirectory in the web area (Optional for RC). Alternatively, you can create a git archive and copy it to the html directory:
git archive --remote=ssh://p-glideinwms@cdcvs.fnal.gov/cvs/projects/glideinwms v3_9_0 doc > /tmp/glideinwms-doc.v3_now.tar scp /tmp/glideinwms-doc.v3_now.tar fnalu.fnal.gov: # And on fnalu (or where the web directory is accessible): mkdir /web/sites/glideinwms.fnal.gov/htdocs/tmp pushd /web/sites/glideinwms.fnal.gov/htdocs/tmp tar xf ~/glideinwms-doc.v3_now.tar mv doc ../doc.v3_9_0
- Create the required symlinks to the latest docs directory. (Optional for RC)
- Build the API documentation and copy it to the Web area (Optional for RC)
# E.g. From the source directory: tox pushd doc/api tar -czf gwms-api-doc.tgz html scp gwms-api-doc.tgz fnalu.fnal.gov: # From fnalu.fnal.gov cd /web/sites/glideinwms.fnal.gov/htdocs tar xzf ~/gwms-api-doc.tgz mv html api.vX_Y_Z
- Upload the source file and create OSG RPMs (step 3 and 4 from the summary - see the next section for instructions)
- Announce the release to following mailing lists: glideinwms-stakeholders, cms-dct-wms@fnal.gov, glideinwms-support@fnal.gov
- If you made any changes to the release manager to accommodate the release, commit those changes.
- On the Redmine Project Page (only for full releases, not release candidates):
- Close all the issues related to the release. If any issues not resolved/closed assign them to future release.
- Close the Version and set the appropriate release date. (Settings -> Version -> Edit the version)
- Create a new version as required
- Update the Custom queries to use next release version
Making a GlideinWMS Patch¶
Consider making a release instead. If a patch is made (solve a problem limited ticket or for a specific installation), follow these step:- Solve the problem
- Test the patch starting from a clean release
- Test for compatibility with the std release
- Release the patch (file set or patch file) attaching them to the ticket or adding them in the Files section of this wiki
- The ticket note or a README file should provide clear installation instructions (which files to substitute, how to apply the patch)
- NEVER send files and patches via email!
Creating the Glideinwms RPMs¶
GlideinWMS RPMs are created and distributed using the OSG tools.
These steps are in this separate section because they require access to OSG resources and the installation of the OSG tools.
(it is actually possible to build RPMs w/o OSG tools but OSG is required to distribute them and it is easier to use directly the OSG build system)
Official OSG Release Site¶
Always verify official OSG release site for latest information
- https://opensciencegrid.github.io/technology/software/development-process/
- https://opensciencegrid.github.io/technology/software/koji-workflow/
- https://opensciencegrid.org/technology/software/rpm-development-guide/
- https://opensciencegrid.github.io/technology/release/release-policy/
- OSG version support policy: https://opensciencegrid.github.io/technology/policy/release-series/
- https://opensciencegrid.github.io/technology/release/acceptance-testing/
- https://www.opensciencegrid.org/bin/view/SoftwareTeam/RPMDevelopmentGuide
- https://www.opensciencegrid.org/bin/view/SoftwareTeam/OSGBuildTools
- https://www.opensciencegrid.org/bin/view/SoftwareTeam/ReleasePolicy
Getting Access to the Build System¶
Access needed (contact: cat@cs.wisc.edu or osg-software@opensciencegrid.org):- moria.cs.wisc.edu (was library.cs.wisc.edu): Or have a OSG software team upload for you
- svn access to vdt.cs.wisc.edu
- koji access (email osg-software)
Creating glideinwms RPM¶
Create a Tar glideinwms source and upload it to OSG area¶
# TAG=<glideinwms release tag to checkout> # USERNAME=<username on moria.cs.wisc.edu to scp files for OSG builds> # Example: glideinwms/build/ReleaseManager/osg-release.sh v3_4_1 parag glideinwms/build/ReleaseManager/osg-release.sh $TAG $USERNAME
Build Glideinwms RPMS¶
NOTE: The previous steps can be executed on any platform with python and bash. The following part has to be executed on a RHEL (SL/CentOS) machine. Mock and the OSG release tools are available only that platform.
Then checkout OSG svn:
svn co https://vdt.cs.wisc.edu/svn/native/
Or if you want just glideinwms:
https://vdt.cs.wisc.edu/svn/native/redhat/trunk/glideinwms/
In the above directory (redhat/trunk/glideinwms) you will find osg/glideinwms.spec.
This is an rpm spec file that you will use to build glideinwms
NOTE: trunk is used for OSG (and GWMS) production releases. Other OSG versions have separate branches, e.g. GWMS development uses OSG upcoming: https://vdt.cs.wisc.edu/svn/native/redhat/branches/upcoming/glideinwms/
osg/glideinwms.spec ------------------ Update the release version and NVRs. NVR for final releases start with non zero int. Example: v3.2.12-1 NVR for RC will always start with 0 with following format: v3.2.12-0.1.rc1 Update any patches and set the version history changelog at the bottom of the upstream/developer.tarball.source ----------------------------- Update the location of tarball in upstream/developer.tarball.source and commit it.
Once you have a working build box (see https://opensciencegrid.github.io/technology/software/osg-build-tools/)
you can use osg-build:
[root@client ~]$ yum --enablerepo=osg-development install osg-build
For example, running in the glideinwms directory locally:
osg-build mock .
Once the above works, you can build to koji and to the repositories:
osg-build koji --scratch .
This will do a scratch build that will build everything in a test environment, but will not push it into any repositories.
You will be able to see the progress at https://koji.opensciencegrid.org/koji/
Once this works, you can run a real koji build:
osg-build koji .
This will build to osg-development. You will eventually need to promote to osg-testing (v3.2.X) or osg-contrib(v3.3.X).
Promoting and Releasing the Glideinwms RPMS¶
NOTE: The following steps are only for final releases, not for Release Candidates.
Final Releases promotion must be approved and coordinated by OSG. OSG release procedures are documented at https://opensciencegrid.github.io/technology/release/cut-sw-release/.
Any promotion must be approved and coordinated with OSG, Brain Lin (formerly Tim Cartwright). There are acceptance tests to pass and approvals from release coordinators. Sometime OSG personnel will do the promotion. Sometimes changes are requested. GlideinWMS developers must follow-up and follow instructions until the software is released.
Use the following commands only with OSG permission. Package promotion:
osg-promote glideinwms
If it is not the main OSG release version, the version number has to be specified:
osg-promote -r 3.3-testing glideinwms
(or you can run the koji commands manually):
koji tag-pkg el6-osg-testing glideinwms-3.2.22-2.osg.el6 koji tag-pkg el6-osg-contrib glideinwms-3.3.3-0pre2.osg.el6
Installing the Glideinwms RPMs¶
A short version is below. For complete information, see:
https://opensciencegrid.org/operations/services/install-gwms-factory/
https://opensciencegrid.github.io/docs/other/install-gwms-frontend/
Frontend¶
For v3.6+, use $REPO=osg-development
For v3.7+, use $REPO=osg-upcoming
yum --enablerepo=$REPO install glideinwms-vofrontend vi /etc/gwms-frontend/frontend.xml vi /etc/httpd/conf/httpd.conf [if you want to customize http to a different port] rm /etc/condor/config.d/00personal_condor.config # present only on some versions of HTCondor vi /etc/condor/certs/condor_mapfile service httpd start service condor start [put a valid proxy into /tmp/vo_proxy] service gwms-frontend upgrade service gwms-frontend start
Factory¶
For v3.6+, use $REPO=osg-development
For v3.7+, use $REPO=osg-upcoming
yum --enablerepo=$REPO install glideinwms-vofrontend vi /etc/gwms-factory/glideinWMS.xml rm /etc/condor/config.d/00personal_condor.config # present only on some versions of HTCondor vi /etc/condor/certs/condor_mapfile vi /etc/httpd/conf/httpd.conf [if you want to customize http to a different port] cd /var/lib/gwms-factory/condor [and then download condor tarball(s)] service httpd start service condor start service gwms-factory upgrade service gwms-factory start
Testing GlideinWMS¶
Manual and automated CI testing¶
There are CI tests (unit tests and linting for Python and Shell) in the build/ci directory.
These are run nightly by the Fermilab CI infrastructure (CI project, Jenkins) which sends an email summary to the developers.
They are run also in TravisCI (using the file .travis.yml in the code) and by GitHub actions after pull requests and push events (using the files in .github/ , see GitHubActions).
To run them manually use runtest.sh. Use runtest.sh -h for instructions. Your code should pass at least the pylint and all unit tests to be eligible to be merged.
Testing for releases¶
Refer GlideinwmsTesting for a checklist and testing best practices.
See below for a testing infrastructure (CEs, reverse proxy) on Fermiclod that can be used to perform the tests.
Testing on Fermicloud for development¶
See the previous section to install the Frontend and Factory RPMs and the next section for the CEs and other hosts you can use for tests.
After installing the Factory or Frontend RPMs you can link a git repository to your installation so you can test and commit changes (like in gwms-dev-frontend and gwms-dev-factory). This requires a few steps that are listed in GlideinwmsGitAndRPM.
Docker is fairly straightforward to set up on fermicloud, see DockerOnFermicloud.
Why are we mentioning docker on this page? We need are moving to SciToken authentication, which means running a CredMon to wrangle them. This process is poorly documented, I managed to get a simple example from CredMons github working, see CredMonScitokensDocker.
GlideinwmsTestInfrastructure¶
Fermicloud hosts also the following infrastructure that can help your testing:- An all-in-one Compute Element (Globus and HTCondor CE with local grid-mapfile, HTCondor slots) on fermicloud378.fnal.gov
- OSG-34, sl7, htcondor-ce 3.4.0-1
- configured to authenticate with GSI and local grid-mapfile.
- An all-in-one Compute Element (Globus and HTCondor CE with local grid-mapfile, HTCondor slots) on fermicloud127.fnal.gov
- OSG-35, sl7, htcondor-ce 4.4.1-1
- configured to authenticate with SCITOKENS issued by wlcg.cloud.cnaf.infn.it or jobsub.fnal.gov
- also configured to authenticate with GSI and local grid-mapfile.
- An all-in-one Compute Element (Globus and HTCondor CE with local grid-mapfile, HTCondor slots) on fermicloud025.fnal.gov: great for small tests, makes it easy to manage new DNs
- A small computing cluster (HTCondor CE using GUMS and 4 worker nodes): worker nodes are bigger (2 cores and 8GB RAM), ideal to test partitionable slots and glidein policies
- CE is fermicloud121.fnal.gov
- Worker nodes are fermicloud111, fermicloud081, fermicloud313, fermicloud314
- gwms-dev-web, A reverse web proxy for Factory and Frontend so that you can send glideins outside Fermilab (e.g. on Amazon AWS or Google Engine)
- gwms-dev-factory, a Factory
- gwms-def-fronted, a Frontend
For more information related to GlideinWMS and all the machines available for development and testing, please refer to GlideinwmsTestInfrastructure
For machines shared across the team, i.e. machines listed in the GlideinwmsTestInfrastructure, please abide by the directives listed in the operation policy to avoid stepping over the work of fellow developers; this applies also to the machine owner/main maintainer.
For machines shared on a personal level, i.e. the owner gave you access, please follow whatever the owner suggested.
Glideinwms developers documents¶
I'm collecting here documents about the inner workings of the GlideinWMS components.Some documents are included in the public documentation on the Web:
- Factory design: http://glideinwms.fnal.gov/doc.prd/factory/design.html
- Factory internals http://glideinwms.fnal.gov/doc.prd/factory/design_factory.html
- Entry internals http://glideinwms.fnal.gov/doc.prd/factory/design_entry.html
- Glidein internals http://glideinwms.fnal.gov/doc.prd/factory/design_glidein.html
- Frontend design http://glideinwms.fnal.gov/doc.prd/frontend/design.html
- Frontend internals http://glideinwms.fnal.gov/doc.prd/frontend/design_frontend.html
- Communication between Frontend and Factory (classads) http://glideinwms.fnal.gov/doc.prd/factory/design_data_exchange.html
Glideinwms On Cloud¶
Refer GlideinwmsOnCloud
Containers with Glideinwms¶
Refer to GlideinwmsSingularity for how to publish an image on OSG, to use it on GlideinWMS
Glideinwms Accounting Improvements¶
Refer Accounting
Removing the big binaries from the tree¶
At some point, we should freeze the repository and remove the two big (100 MB) binaries from the tree -- it will speed up git clone by a factor of 10.
for x in `git branch -r | grep -v '\->' | grep origin | sed -e 's|origin/||g'`; do git co $x; done git filter-branch --index-filter 'git rm --cached --ignore-unmatch extensions/virtualmachines/additional_packages/EucaKernelModules/euca-kernel-modules-2.6.28-11* HEAD' --prune-empty --tag-name-filter cat -- --all git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d git reflog expire --expire=now --all git gc --prune=now git gc --aggressive --prune=now
Now we have a local copy that is clean.
We can then remove the old repository or force push to it.
git push newrepo --all git push newrepo --all --tags
Cleaning Remote Branches¶
# Step 1: Delete local branch git branch --delete <branch_name> # Step 2: Delete remote branch git push origin --delete <branch_name> # Step 3: Every should prune their repo to cleanup references to deleted branches git remote prune origin
Shell script for safe delete that ignores unmerged branches¶
#!/bin/sh branches="<space separated list of branches to delete>" failed="" remote_failed="" for branch in $branches do echo "DELETING LOCAL: $branch" git branch --delete $branch if [ $? -eq 0 ]; then echo "DELETING REMOTE: $branch" git push origin --delete $branch if [ $? -ne 0 ]; then echo "FAILED REMOTE DELETE: $branch" remote_failed="$remote_failed $branch" fi else echo "FAILED LOCAL DELETE: $branch" failed="$failed $branch" fi done echo "FAILED DELETE: $failed" echo "FAILED REMOTE DELETE: $remote_failed"