Project

General

Profile

Snapshots, Releases, and Feature Branches

Two types of tags are used in novasoft to identify stable versions of the code: snapshots and releases.

Snapshots

Snapshots refer to a tag made based on the HEAD version of the trunk of the code at a specific point in time with names formatted S<year>-<month>-<day>. For example, S15-09-28 is a snapshot of the latest state of the code on September 9th, 2015. Snapshots are made approximately every two weeks. They provide a way to use nearly the latest version of the code as a base release without worrying about your base release changing nightly as occurs with the development base release. These are not intended for large-scale production or long-term archival purposes.

Snapshots cannot have bug fixes applied to them. Either a new snapshot is required or a release starting from the same point as the snapshot must be made.

At a technical level, snapshots are tagged versions of the trunk of the repository produced with an svn copy from trunk/ to the tags/SYY-MM-DD directory.

Development

There is a new option - development on cvmfs. Currently, we have development on cvmfs along with bluearc (the ordinary location). Every day nightly snapshot that was built on Jenkins is automatically downloaded on cvmfs. Each snapshot is available during 7 days after it’s build date. The names for the nightly snapshots have a format NYY-MM-DD (i.e N18-02-16 for 16 Feb 2018).

Just modify you setup_nova function in .bashrc if you’d like to use the development from cvmfs:

source /cvmfs/nova.opensciencegrid.org/novasoft/slf6/novasoft/setup/setup_nova.sh "$@" 

Putting the development on cvmfs makes it available for grid nodes. If you’d like to run development release on grid use -r development option. The development will be converted into the submission day snapshot.

Releases

Releases refer to a suite of related tags which begin based on the latest version of develop, but may be fixed as needed over time and have names formatted R<year>-<month>-<day>-<purpose>.<patch>. For example, R16-10-01-sim.b would refer to the second patch (b) of a release was based on the state of develop on October 1st, 2016, was intended for Monte Carlo production (sim).

Successive patches in a release should only contain bug fixes and should not introduce physics-incompatible changes. It should always be safe to use the latest patch when a release is used. If the code before a commit could still need to be used, then that commit is not a bug fix and should instead be put in a new release. If a new release is needed based on the same version of develop, it should be given a new <purpose>, i.e. R16-10-01-birkssim.a.

At a technical level, a release is a branch from the trunk produced with an svn copy from trunk/ to branches/RYY-MM-DD-XXX-br, and each patch is an svn copy from the branch to tags/RYY-MM-DD-XXX.a.

Feature Branches

If a disruptive series of changes to the code base are planned, it is best to try to isolate those changes in a feature branch until they are fully tested and can be introduced into the trunk. While only release managers can create tags, anyone can create a branch. Since they are not releases, they cannot be set up as base releases, but should instead be set up against development since they should stay in-sync with changes in the trunk as much as possible.

Procedures

Making a change to an existing release branch

If a release is found to have a problem or otherwise needs to be updated, it can be. First create a test release with the latest patch version as base release.

setup_nova -r R16-10-01-sim.a        # NB patch level, no "br" 
newrel -t R16-10-01-sim.a mytestrel  # NB patch level, no "br" 

Then, add the packages that require fixing from the release branch.

addpkg_svn -b Calibration R16-10-01-sim-br  # NB "br", no patch level

Make any necessary changes here on the branch, not directly on the trunk. (Oh no, I've already committed to the trunk, what do I do?) Test that the code compiles and runs as expected. Since you have set your base release to be the latest tag in this release, you are testing precisely the version of the code that will become the next patch in the release, whether or not other changes have been committed to the trunk. Once you are happy with the changes, commit the changes using

svn_branch_commit -m "<message>" 

This is a script we have written which will commit the changes both to this branch and onto the trunk. To see what's happening under the hood in this script, see here (In special cases, like a very old release, you can just commit as usual to commit only to the release branch, but this is generally discouraged.)

Once all the fixes have been put in, a new patch version of the release can be made just like the first one was, incrementing the final letter.

Creating and using a feature branch

If a disruptive series of changes to the code base are planned, it is best to try to isolate those changes in a feature branch until they are fully tested and can be introduced into the trunk. While only release managers can create tags, anyone can create a branch.

Make the branch

Obviously the branch name is arbitrary, but I think a signifier like "feature" will help distinguish it from the release branches. While only the release managers can create tags (copy into ^/tags) anyone has permission to create branches. You will copy the whole repository, but don't worry about it. Copies in svn are cheap.

svn copy svn+ssh://p-novaart@cdcvs.fnal.gov/cvs/projects/novaart/pkgs.svn/trunk  \
         svn+ssh://p-novaart@cdcvs.fnal.gov/cvs/projects/novaart/pkgs.svn/branches/feature_<myfeature> \
         -m "Creating a feature branch for..." 

Introduce your new features onto the branch

Feel free to use multiple separate commits here. Since you cannot set up the branch as a base release, set up the base release against develop and then add packages from the branch:

newrel -t development <myfeature_test_release>
cd <myfeature_test_release>
addpkg_svn -b <package> feature_<myfeature>

Bring the feature branch up-to-date with the trunk

SVN stores merge info so the commits to merge are known automatically. The initial svn up is harmless and prevents occasional mixed revision errors. This can be repeated as needed if the branch lives a while, though long-lived feature branches are not recommended. You can bring a package on your feauture branch up-to-date with the trunk via

cd <myfeature_test_release>/<package>
svn up
svn merge ^/trunk/<package>
svn commit -m "Bringing <package> on feature branch up-to-date with the trunk." 

Don't worry - this will not commit anything to the trunk.

Reintegrate the feature branch back into the trunk

Once the final merge from the trunk has been done and the code has been tested, it can be reintegrated into the trunk. This must be done from the trunk, not the branch working area. Once a branch has been reintegrated, it is no longer usable and should be deleted.

<setup test release with package from head>
svn up
svn merge --reintegrate ^/branches/feature_<myfeature>/<package>
# Check to make sure the merge went smoothly. 
svn commit -m "Reintegrating feature branch <myfeature> into the trunk." 
svn delete ^/branches/feature_<myfeature> \
             -m "Remove feature_<myfeature> after reintegration into the trunk." 

Additional information

SVN branch & merge documentation: http://svnbook.red-bean.com/en/1.7/svn.branchmerge.html