Project

General

Profile

New Project Member Orientation

Access to code and ticket system, documentation pointers

Access to OSG build system

To be able to build the GlideinWMS RPMs, the following has to be requested to the OSG (Wisconsin) team:
  • give access to library.cs.wisc.edu (username/password, ssh access): it hosts sources for the packages in /p/vdt/public/html/upstream/glideinwms/
  • give access to the koji server (certificate): used for building
  • access to Jira is only for one team member, liaison
  • give commit privileges to the subversion server: used to store spec files and source location files
OSG RPM documentation is on github.io:

Ticket, documentation and coding procedures

The project is managed in Fermilab's Redmine (this system).
In a normal workflow, the tickets go through the following steps:
  1. Creation: all project members should capture issues or feature requests and create tickets. Stakeholders should be added to the ticket whenever possible. The creator should make sure that a ticket is either assigned (to a developer) and scheduled for a release or brought to the attention of the project lead, e.g. during the weekly meeting
  2. The assigned developer becomes the owner of the ticket until it is resolved or re-assigned. Tasks include:
    • Change the status to Assigned or "Work in progress"
    • Do the troubleshooting to understand the exact problem
    • Create a new branch for the ticket code
    • Devise and develop a solution
    • Test that the solution solves the bug or implements the requested feature. She/he can contact the requestors if clarifications/verifications are necessary
    • Write unit tests
    • Document the change if users need to be informed (e.g.: it is a new feature)
    • Add notes to the tickets, especially if the implementation choices and the location of the code are not trivial
    • Assign the ticket to another developer for Feedback (change status to "feedback" and owner) after the changes have been fully tested
    • Follow up with feedback provider and take care of feedback remediation
    • Merge and resolve when appropriate (see below)
  3. The feedback provider should provide feedback (via email, in Redmine, in person) communicating clearly if actions are needed or it is OK to merge. The owner should go back to the developer
  4. If significant actions are needed the ticket should go back to development (and feedback asked afterward). If there are no remediations or are trivial, then the developer can go to the resolution step after the remediations have been taken
  5. The developer merges the code to the main branch (master or the release branch) and marks the ticket as resolved
  6. The release manager (whoever builds the final release, not RC) will make sure that all the tickets in the release are resolved and marks them as closed

The main wiki page, https://cdcvs.fnal.gov/redmine/projects/glideinwms/wiki , has coding standards suggestions and instructions on how to use the repository.

Documentation is also a responsibility of the team members: feel free to contribute end fix errors. GlideinWMS users documentation is in the doc/ subdirectory of the Git repository. Developers documentation is in this wiki: there is a help page ; to include images you can attach them to the page, if you have documents you should add them to the "Documents" section, so they can be linked with "document:TITLE" and updates will be easier.

Here a template for a first ticket to assign to a new project member (employee or intern)

Title: Complete orientation & training for the GlideinWMS project
Description:

You have to complete the following steps in order to get you familiar with GlideinWMS and the tools used in the collaboration:

# Get a FNAL ID number and username (ID), Fermilab Kerberos and services accounts (Windows account is not required)
# Receive a laptop w/ account (FNAL ID) and a property pass (requires 1) 
# Request Fermicloud access (service now ticket, requires services account) 
# Get and check Redmine access as GWMS developers (requires services account) 
# Get an x509 certificate (via CILogon and/or kx509?)
# Check that you have been added to the glideinWMS mailing list to receive the updates (glideinwms@fnal.gov)
# Request to be added to Fermilab computing SLACK (fnal.slack.com) using your Fermilab email, and the #glideinWMS channel
# Create a github.com account and communicate your username to the GlideinWMS team
# Read or attend to the following orientation presentations:
** Basic concepts introduction: https://cdcvs.fnal.gov/redmine/attachments/download/54764/Basic%20Concepts.pdf
** Introductory presentation (GWMS): https://cdcvs.fnal.gov/redmine/attachments/download/54767/GlideinWMS.pdf
** Security in distributed computing: https://cdcvs.fnal.gov/redmine/attachments/download/54768/SecuritybasicsforGWMS.pdf
** Fermicloud hands-on: https://cdcvs.fnal.gov/redmine/attachments/download/54766/Fermicloud.pdf
# Follow the following shell tutorial (if needed):  https://swcarpentry.github.io/shell-novice/ ("the source code is on GitHub":https://github.com/swcarpentry/shell-novice/ )
# Follow the following Git tutorial (if needed):  https://swcarpentry.github.io/git-novice/
# Follow the following introduction to Python (if needed). Lesson 1, 2, 3 and exercises: https://sites.google.com/view/targetpython/
# Ask for git, vi and shell cheat sheets. Practice basic shell commands (navigation, file operations) and create/edit a file with vi (if needed)

Here a few optional references to learn more about some topics: 
* Shell/bash
** Classic: http://www.tldp.org/LDP/abs/html/
** Break it down: http://explainshell.com
** Great guide: http://wiki.bash-hackers.org/
** Good to know: https://mywiki.wooledge.org/BashPitfalls

(this list can also be printed and provided)

Here a template for a second ticket to assign to a new project member (employee or intern)

Title: Install GlideinWMS framework on Fermicloud
Description:

Install GlideinWMS framework on Fermicloud using OSG RPMs.

1. You will need one host for the Factory (including the condor Factory collector and schedd) and one for the Frontend (including the condor User collector/negotioator and a schedd).
Take note of the 2 host addresses (hostname) you will need them for the configuration
Take note also of the certificates DNs. Certificates are in /etc/grid-security/host*

To make new VMs and administer them use fcluigpvm01:
ssh fcluigpvm01.fnal.gov

This script will list your hosts (ID and hostname needed to ssh):
~marcom/bin/myhosts

Then login as root on your VMs:
ssh root@YOURVM.fnal.gov
And follow the install instructions

2. The factory installation instructions are in the OSG documentation. Follow them to install it:
https://opensciencegrid.org/operations/services/install-gwms-factory/

3. The frontend installation instructions are in the OSG documentation. Follow them to install it :
https://opensciencegrid.github.io/docs/other/install-gwms-frontend/

Continue as much as you can and provide also feedback about the instructions