Project

General

Profile

Feature #18859

Adding support for checking out svn branches

Added by Pengfei Ding over 1 year ago. Updated over 1 year ago.

Status:
Feedback
Priority:
Normal
Assignee:
Target version:
-
Start date:
01/31/2018
Due date:
% Done:

50%

Estimated time:
Duration:

Description

At the moment, the ci test checks out packages from the http://cdcvs.fnal.gov/subversion/novaart.pkgs.svn/tags/ directory for a given svn tag.

We want to use the CI test for a new branch, so it would be great if the ci test can check out packages from http://cdcvs.fnal.gov/subversion/novaart.pkgs.svn/branches directory too.

The desired feature would end up like:

In addition to 'trigger --revisions "novaart.pkgs.svn@S16-03-04' which runs ci tests on the tag "S16-03-14", we want a 'trigger --revisions-branch "novaart.pkgs.svn@S16-03-04"'. It would also be great if the current "--revisions" can be aliased to "--revisions-tag".

History

#1 Updated by Pengfei Ding over 1 year ago

Note, when using branches, the "upadte-release" command should add "--branch" option.
Examples can be found at

https://cdcvs.fnal.gov/redmine/projects/novaart/wiki/Installing_a_local_copy_of_NOvASoft_and_the_external_products#3-releaseND-Development-or-releaseND_Test

#2 Updated by Vito Di Benedetto over 1 year ago

  • Status changed from New to Work in progress

#3 Updated by Vito Di Benedetto over 1 year ago

  • % Done changed from 0 to 50

The checkout CI phase in nova_ci workflow "NOvA_CI_SRT" has been updated with support to test novaart.pkgs.svn branches.

To do that you need to use the trigger "--revisions" option.
A basic example to trigger a NOvA CI build to test the branch "S16-03-14" is

trigger --revisions "novaart.pkgs.svn@S16-03-04"

The current implementation assumes that whenever you uses the "--revisions" option, you are requiring to test a branch.
This excludes the possibility to test tags. But I think the code used to create a tag has already been tested previously.

Please, trigger some test NOvA CI build to check if this implementation works as expected.

#4 Updated by Pengfei Ding over 1 year ago

Thanks Vito. I've triggered the test #4329 with
trigger --build-delay 0 --ci-tests "ci_raw2root_fd_t00_regression_test_novasoft" --revisions "novaart.pkgs.svn@art2root6-integration-trunk"
Triggering CI build

It does check out the novaart pacakges from a branch.

https://buildmaster.fnal.gov/job/nova_ci/4329/

Build #4322 was the one triggered with the same command, but before the "branch" feature was added. So looking at the console output, there were no novaart packages checked out.

https://buildmaster.fnal.gov/job/nova_ci/4322/label_exp=SLF6,label_exp2=swarm/console

#5 Updated by Pengfei Ding over 1 year ago

I took it back. I still saw it was trying to checkout from the tags subdirectory.

svn: URL 'http://cdcvs.fnal.gov/subversion/novaart.pkgs.svn/tags/art2root6-integration-trunk/SRT_NOVA' doesn't exist

Was looking at the build #4322, instead of #4329.

#6 Updated by Vito Di Benedetto over 1 year ago

I think there is still something to update in the "build" CI phase.
Maybe the "source setup/setup_nova.sh" needs a different set of options for a branch?

#7 Updated by Pengfei Ding over 1 year ago

Yes. I think It is trying to do a srt setup and set the SRT_PUBLIC_CONTEXT to /cvmfs/nova.opensciencegrid.org/novasoft/slf6/novasoft/releases/$BRANCH_NAME

For tagged release, we always have one release with the same tag name under /cvmfs/nova.opensciencegrid.org/novasoft/slf6/novasoft/releases/ ; but we do not have that there for branches.

If the srt_setup was successful, it should have SRT_PUBLIC_CONTEXT set and find the novasoft_build from:

${SRT_PUBLIC_CONTEXT}/SRT_NOVA/scripts/novasoft_build

#8 Updated by Pengfei Ding over 1 year ago

We need to create a second "build" step, which, instead of calling novasoft_build from ${SRT_PUBLIC_CONTEXT}/SRT_NOVA/scripts/, call it from

/cvmfs/nova.opensciencegrid.org/novasoft/slf6/novasoft/releases/development/SRT_NOVA/scripts/novasoft_build

#9 Updated by Vito Di Benedetto over 1 year ago

Currently for the build setup step the CI runs the command below:

source ${WORKSPACE}/setup/setup_nova.sh -r ${moduleVers:-development} -b "${NOVA_QUALS}" -6 "${WORKSPACE}" -e "/cvmfs/nova.opensciencegrid.org/externals:/grid/fermiapp/products/common/db:/grid/fermiapp/products/nova/externals"

where, in the case of the "art2root6-integration-trunk" branch, "${moduleVers:-development}" is expanded to "art2root6-integration-trunk" and ${NOVA_QUALS} is expanded to "maxopt"

Then for the actual build, the CI runs the command:

${SRT_PUBLIC_CONTEXT}/SRT_NOVA/scripts/novasoft_build -rel ${moduleVers:-development} -p ${N} -failonerror

where, again, ${moduleVers:-development} in this case is expanded to "art2root6-integration-trunk"

Other than replace, as you suggested, "${SRT_PUBLIC_CONTEXT}/SRT_NOVA/scripts/novasoft_build" with "/cvmfs/nova.opensciencegrid.org/novasoft/slf6/novasoft/releases/development/SRT_NOVA/scripts/novasoft_build", is there anything else to be updated?

I'm running the novaart build procedure interactively to make sure that all the steps are working properly.

#10 Updated by Vito Di Benedetto over 1 year ago

  • Status changed from Work in progress to Feedback
  • Target version set to v1_6_0_RC

In order to get the "build" CI phase to work, Pengfei pointed out that it is needed to create an empty base release with the required branch name and push to cvmfs.

#11 Updated by Vito Di Benedetto over 1 year ago

  • Target version changed from v1_6_0_RC to v1_6_0

#12 Updated by Vito Di Benedetto over 1 year ago

  • Assignee changed from Vito Di Benedetto to Pengfei Ding
  • Target version deleted (v1_6_0)

Assigned the ticket to Pengfei for his part of the work to create the empty base release with the required branch name and push it to cvmfs.



Also available in: Atom PDF