Project

General

Profile

History

Wiki ยป

The Multi Repository Build System proposal

Use cases

  • A code developer might need to simultaneously work on both experiment specific code and a supporting part of the LarSoft package.
  • End user who just develops experiment specific code. This person should only ever need to use a tagged release of LarSoft.
  • A code developer who is working on some basic piece (for instance, nutools), may need to modify upstream packages to use new APIs.

Build System Requirements

  • The build system must make it easy to get the necessary code.
  • In order to avoid inconsistent builds, the build system needs to make sure all parts of the code chain are present.
  • The build system must also make it easy to checkout and build all of LarSoft.
  • We presume that LarSoft itself will be a small number of packages (perhaps 5-10).

Multi Repository Build System

  • The Multi Repository Build System (mrb) will be closely modeled on the work Adam Lyon has done for g-2.
  • The build system will impose a directory structure
  • Builds for different qualifier sets will require the user to start a new shell (new login)
  • The build system will check for missing parts of the build chain.
  • The build system will use cmake
  • Cmake allows parallel building, which can be very fast.
  • However, a build machine needs multiple cores to take full advantage of parallel builds.
  • Cmake is also very good at sorting out build dependencies.
  • mrb uses cetbuildtools
  • Since LarSoft uses art, it is sensible to use the cmake helper macros already defined in art and cetbuildtools.
  • If additional macros are needed, they will be added to cetbuildtools.
  • Note that a macro to run root cint was added when nutools migrated to a cmake build.

Work Flow

  • login and setup the experiment specific environment
  • This step will make mrb available and define any necessary experiment specific instructions.
  • mrb newDev
  • create a new development working directory when appropriate
  • mrb checkout <svn|git> package
  • mrb checkout recognizes svn and git
  • mrb checkout can, in principal, be taught to recognize any code management system
  • in general, mrb checkout git XXX, etc. for as many packages as necessary
  • special case "mrb checkout larsoft_suite" will get all of LArSoft
  • source mrb setEnv
  • mrb build [-j20]
  • -j20 would do a parallel build with 20 machine cores
  • Please note that this is a general workflow, not a set of instructions

Command Proposal

  • mrb newDev <working directory>
  • initialize a new working directory structure
  • mrb checkout <git|svn> <repository>
  • initialize the build directory structure if necessary
  • checkout the requested code
  • update the master CMakeLists.txt
  • repositories might not all be in Fermi Redmine
  • mrb setEnv
  • looks in the checked out packages for required dependencies (a single file in each package)
  • makes sure that any dependencies that are part of the build chain are checked out
  • if the build chain is missing a dependency, stop with an ERROR
  • setup required ups products that are not part of the build chain
  • create separate build subdirectory for each qualifier combination (e.g. debug, prof)
  • creates a master CMakeLists.txt file to control the build
  • mrb build
  • does the actual build
  • options control parallel building, etc.
  • mrb install
  • generally used by a Software Coordinator when building a tagged release
  • mrb zapBuild
  • cleans up the build directory
  • could just as easily be called mrb clean
  • mrb updateDepsCM
  • update $MRB_SOURCE/CMakeLists.txt to use all products in $MRB_SOURCE
  • use this if you have checked out a package by hand
  • mrb newProduct
  • create an empty skeleton for a new product
  • update the master CMakeLists.txt