Project

General

Profile

Example of creating and testing new base release

This is an example of creating, building and testing DAQ base release R09_00_00.

Generally, making a new DAQ base release consists of the following steps:
  1. Gather info from developers of recently updated DAQ packages about which version should be used in the new base release;
  2. checkout the correct versions of all packages, and tag them with the new base release tag, and document the versions in the release notes in Redmine;
  3. install and build the new base release;
  4. deploy and test the new base release.
  5. update "ups" package.

This example shows each of the above steps happened when making DAQ base release R09_00_00.

Gather info about package versions.

1. NovaGlobalTrigger in R08_01_00 plus revision 1.35 for GTDataTrig.cpp are tagged as R09_00_00.

2. The following packages with tag R08_01_00 are re-tagged with R09_00_00:

NovaSpillServer, DAQDataFormats, BufferNodeEVB, NovaDataLogger, DDTManager

3. The following packages (all other DAQ packages) head versions are tagged as R09_00_00:

benchmarks, DAQApplicationManager, DAQChannelMap, DAQMessages, DAQNetworkUtils, DAQOperationsTools, DAQQualityCheck, DAQSimulationManager, DatabaseUtils, DCMApplication, DCMBootLoader, dcm_kernel_module, DCM_ProgUtils, DispatcherClientExampleApp, ErrorHandler, EventBuilderClient, EventDispatcher, EventDispatcher_CommandSet, EventDispatcher_Server, EventDispatcher_Server_FMWK, EventDump, EventMemoryViewer, ExternalPackageTest, linux_kernel_dcmtdu, MetaDataTools, NovaDAQCheckout, NovaDAQConfiguration, NovaDAQConventions, NovaDaqDcs, NovaDAQMonitor, NovaDAQMonitorClient, NovaDAQTemplate, NovaDAQUtilities, NovaDatabase, NovaFileTransferSystem, NovaGlobalTrigger, NovaMessageLogger, NovaResourceManager, NovaRunControl, NovaRunControlClient, NovaSNEWSInterface, NovaSuperNova, NovaTimingUtilities, PackageVersion, PedestalDataRunner, PowerUtilities, RawFileParser, ShmMilliBlock, ShmRdWr, SHM_Utilities, SRT_NOVADAQ, SoftRelTools, TDUControl, TDUControlClient, tdu_kernel_module, TDUUtilities, TDUWeb, Trace, TriggerScalars.

Tag packages with R09_00_00 tag.

The desired versions of each package should be tagged with the name of the new base release. In practice, this usually means
  • Tagging all packages in the current test release, followed by
  • Tagging all packages in the current base release. Since CVS will not (by default) move a tag, so the versions of packages in the test release will keep the new tag.
  • Tag, but only after appropriate testing, any other package versions desired by developers in consultation with the DAQ group head. Note, you will likely need the -F cvs tag option on this step to force the tag to move. More details can be found here

The paragraph below is referred from CVS Tips

For tagging a small number of packages:
  1. check that all changes are checked in. If changes are not checked in, they need to be committed:
    • Go to the directory of the package if new files need to be added.
       cvs add fileName 
    • This schedules the new files to be committed, then commit using
       cvs commit file 
    • Use -d novacvs@cdcvs.fnal.gov:/cvs/nova between cvs and add/commit if you don’t have write access to the repository.
  2. apply tag
    • Since code is checked out into novadaq with the anonymous (pserver) CVSROOT, you'll need to specify the writeable CVS root (novacvs@cdcvs.fnal.gov:/cvs/nova) on the command line when tagging
    • Additionally, you'll need to kinit as someone with write permission for the repository
    • In the directory above the package, cvs -d <Writable CVS Root> tag <TAG NAME> <PACKAGE_NAME>
  • If you are tagging several packages (for example, in a test release), you can loop over directories to make it a little faster:
    • create a string variable with the package names
      pkgs="pkg1 pkg2 pkg3 ... " 
    • loop over them
      for pkg in $pkgs; do 
      cvs -d <Writable CVS Root> tag <TAG NAME> $pkg
      done
      

Install and build base release R09_00_00.

Please look at for more information in Adding a base release to an existing set of online releases

  1. setup_online : as novadaq, setup the existing base release
  2. cd $SRT_DIST/releases
  3. newrel -e -d R09_00_00 : makes the new release. The "-d" signifies "development," but the newrel function only seems to distinguish -t (test) from all other types.
  4. cd R09_00_00
  5. addpkg -h -M onlsetup setup : yes, you are using the initial release to build this one.
  6. cd setup
  7. ls *R08_01_00* : You need to look at the previous release.
    1. You will see three files named as
      nova-online-packages-R08_01_00 nova-ups-products-R08_01_00 nova-ups-products-R08_01_00-LinuxPPC
    2. To avoid error and confliction copy them into R09_00_00.
    3. cp nova-online-packages-R08_01_00 nova-online-packages-R09_00_00 : change your package versions in the new file as you have described in Gather info about package version.
    4. cp nova-ups-products-R08_01_00 nova-ups-products-R09_00_00
    5. cp nova-ups-products-R08_01_00-LinuxPPC nova-ups-products-R09_00_00-LinuxPPC
  8. Then you need commit in to the CVS.
    1. cvs add *R09_00_00*
    2. cvs commit -m "making new base release R09_00_00"
    3. cd ..
    4. cp -r $SRT_PUBLIC_CONTEXT/SoftRelTools .
    5. cd include
    6. ln -s ../SoftRelTools/include SoftRelTools
    7. cd ..
  9. addpkg -h SRT_NOVADAQ
  10. gmake SRT_NOVADAQ.all
  11. gmake SoftRelTools.all : Order matters, so we pick up the NOvA versions of certain SRT utilities.
  12. Log out and Log in as a novadaq
  13. source /nova/novadaq/setup/setup_novadaq.sh -r R09_00_00
  14. update-release --rel=R09_00_00 : Check out packages you specified in step 7. If you forget the --rel=option, you risk messing up the original release.
    After checking out the package, make sure you checked out the correct version of source code. Here is an example command.
    $ cd /nova/novadaq/releases/R09_00_00/DAQDataFormats; cvs status GNUmakefile
    You should see a line like this:
    Sticky Tag:        R09_00_00 (revision: 1.6)
Also look at Building a base release
  • You can now proceed to compile the release according to instructions.
    1. log into the novadaq account
    2. it is best to not run the 'setup_online' command (or 'source /home/novadaq/DAQOperationsTools/novadaq_setup.sh'). Since this command sets up a particular base release and the testRelForOperations test release, this can confuse the build that you're trying to do.
    3. run
      '/home/novadaq/.cron/update_base_release_all --no-update <releaseName>'
      • this command does not update any packages in the release, thanks to the "--no-update" option (no "cvs update ..." is run)
      • the command runs the primary builds for all four variants in parallel. It "tail"s the logfile from the x86 build initially, so you can see how things are working. After the x86 build is finished, it "tail"s the logfiles from the other three builds, so you will see the last 200 lines of those logfiles, as well.
      • once the primary four builds are done, the PPC builds of the NovaDaqDcs package are run. As before, you will see a "tail" of the logfile as the builds run.
      • the logfiles from this command are stored in /home/novadaq/.cron/log/<hostname>. (To find the latest logfiles, it's easiest to simply run the latest command after you've moved to this directory.)

Deploy and test base release R09_00_00.

Before testing the new base release, you will need to create a test release using the new base release. Referred from Creating a brand new test release

  1. source /nova/novadaq/setup/setup_novadaq.sh -r R09_00_00 for a brand new base release
  2. cd /home/novadaq; newrel -t R09_00_00 testRel_FD09_00_00 : makes the new test release. The "-t" signifies "test,"
  3. cd /home/novadaq/testRel_FD09_00_00
  4. srt_setup -a - make this your SRT_PRIVATE_CONTEXT
  5. addpkg -h SRT_NOVADAQ - add this package to make SRT work optimally.
  6. build4 SRT_NOVADAQ

Now you are ready to test the new base release.

  1. stop the run in FD/ND, whichever detector you are going to test it. Here we tested it on FD.
  2. edit /nova/config/FarDet/daq-operations.cfg for the entire detector, or /nova/config/FarDet/PartitionX/daq-operations.cfg a specific partition X, to use the base release and test release.
  3. start the run.

Additional things to be done for making novadaq ups product (to be used by DDT release)

  1. checkout the head version of package "ups";
  2. modify the bootstrap script with the new base release tag;
  3. tag the ups package with the new base release tag;
  4. tag Online/pkgs/CMakeLists.txt with R09_00_00.