Icaruscode CMakefile patching project


After our meeting on 2020-02-24, an important effort is to add patches to package recipes to perform
the cetbuildtools -> cetmodules conversion, so that we can build current releases.

This is currently done for the "art" suite on the fnal_art spack recipe repository, on the
feature/cetmodules_patches branch. We need to complete it for the remaining modules needed
to build icaruscode.


Making the spack instance

If you would like to duplicate my test setup, using the "unified" spack/ups layout, you can:

[If you've been using UPS packages, do an unsetup_all to clear your envronment; in particular the "curl" that is setup with the UPS jobsub_client confuses spack to no end...]

  • Run
      source /cvmfs/
      spack load spack_infrastructure@1.2 os=fe
      make_spack -u /some/directory
  • OR if you don't have /cvmfs available:
    mkdir -p $dest/spack-infrastructure/$ver
    cd $dest
    git clone spack-infrastructure/$ver/NULL/
    (cd spack-infrastructure/$ver/NULL && git checkout $ver)
    make_spack -u $dest

Note the whining about "remote REF refers to nonexistent Head" from git is unfortunately

  • source /some/directory/ to setup your new spack area
  • cd $SPACK_ROOT/var/spack/repos/fnal_art && git checkout feature/cetmodules_patches

Installing gcc

Then get the compiler you want:

    spack buildcache install "gcc@8.2.0 arch=x86_64" 

if it can't find that you may be able to:
    spack buildcache list --allarch --long gcc

This should give you (eventually) a listing like:
-- linux-scientific7-x86_64 / gcc@4.8.5 -------------------------
4s22i2p gcc@8.2.0

-- linux-scientificfermi6-x86_64 / gcc@4.4.7 --------------------
wwsk6a3 gcc@8.2.0

with a hash value for each platform; pick the one for your O.S. and install it
by specifying just the hash:

    spack buildcache install /4s22i2p

Building gcc if needed

If all else fails, you can have spack build gcc, but this is slow...

    mv $SPACK_ROOT/etc/spack/sc*/packages.yaml $SPACK_ROOT/etc/spack/sc*/packages.yaml.hide 
    spack install gcc@8.2.0 arch=x86_64
    mv $SPACK_ROOT/etc/spack/sc*/packages.yaml.hide $SPACK_ROOT/etc/spack/sc*/packages.yaml

and go get some coffee... and maybe lunch...

Actually using the gcc we installed

Now even though we installed gcc with spack, we strangely enough still have to tell it to use it, to wit:

    spack compiler find --scope=site
    spack load gcc@8.2.0
    spack compiler find --scope=site

The first compiler find picks up the system compiler ( on SLF7). The second compiler find picks up . Spack defaults to the last compiler added if one is not specified.

Also it turns out at the moment we need special flags to build with 8.2.0 and cmake... so we need
to edit $SPACK_ROOT/etc/spack/compilers.yaml and find the stanza for , and change:

  flags: {}

    cppflags: -Wno-deprecated -Wno-deprecated-declarations

Building art, larsoft, icaruscode

To build a copy of art, you should now be able to:

spack -k install --no-checksum `cat art_spec`

with the file art_spec

Similarly, to build larsoft, use larsoft_spec

and for icaruscode the full spec (still missing some _data packages?) icaruscode_spec

Packages needing recipes and possibly patches

These are all completed.

The following were on the list originally, but have recipes under slightly different names:

  • pycurl -- actually py-pycurl in spack
  • guideline_sl -- actually cppgsl in spack
  • pythia -- actually pythia6 in spack

Packages just needing cetmodules conversion

The following packages (may) need cetmodules conversion patches. They are listed here in
reverse dependency order. If people working on it would put their initials after the
package in question, we can spread out the work to get the list done quickly.

These all have patches now. These patches could provide a first stab at cetmodules conversion.


libxml2 vs python

A brief note on the libxml2 evil lurking in the package dependencies:

  • doing python+libxml2 says to use a gettext (for internationalization) built with libxml2 support, which we don't need.
  • doing libxml2+python says to build the python bindings for libxml2, which we do need.
  • doing both sends spack into infinite recursion...

We now have the packages.yaml claiming to have gettext without libxml2 (regardless whether it does) to avoid this when using the system gettext.

Notes on how the builds work internally

spack tries multiple ways to get the packages; first

  • it checks the buildcache for that exact version; then
  • it checks the source cache for a cached copy of the source files, then
  • it tries to clone/checkout from source code control, if there is a version entry, then
  • it tries the tarball download url adjusted for this version

If it wasn't the buildcache,then it tries to build it. All of this downloading and building is done in $TMPDIR/$USER/spack-stage , /tmp/$USER/spack-stage by default (although you can switch that in the spack config.yaml as well...)

When a build fails, it leaves the bits in /tmp, which makes it a convenient place to fine-tune your build, especially when it checked the code out of git to do the build; you

  • cd /${TMPDIR:-/tmp}/$USER/spack-stage/spack-stage-package-version-hash/
  • can hack up the directory
  • source the spack-build-env.txt
  • cd spack-build
  • copy and paste the cmake command out of the spack-build-out.txt
    just "git diff" to get a patch file, which you can pop in to the recipe directory, and add a patch('patchfile') to the recipe

Converting from cetbuildtools to cetmodules

In many cases this should be just be applying the changes to the various CMakeLists.txt , and cmakeConfig directories (if any) from the feature/spack-MVP1a branch. Or we can try to do it automatically, or by hand by making the following kinds of changes:

  • make sure cmake_minimum_required(x) is x>=3.9
  • make sure the project() line is there and has VERSION and LANGUAGES e.g. project(ifdh_art VERSION 2.10.02 LANGUAGES CXX )
  • change find_ups_product(pkg vers)
    • for most things just make it find_package(pkg) (except tbb becomes upper case)
    • for non-cmake/cetmodules packages (i.e. libwda, etc.) make it cet_find_simple_package(pkg) -- and if the package is libxxx drop the 'lib'.
    • for ifdhc, cet_find_simple_package( ifdhc INCPATH_SUFFIXES inc INCPATH_VAR IFDHC_INC ) because it has include files in an "inc" directory...
    • for BOOST, find_package( Boost REQUIRED COMPONENTS filesystem system random ...)
  • change find_library() calls that have PATHS ENV XXX_LIB NO_DEFAULT_PATH and remove the whole PATHS... part.
  • any mention of ${XXX_FQ_DIR} or ${XXX_DIR} should become ${yyy_DIR} where yyy is the find_package name (i.e. lower case)
  • any _cet_check...() calls should be deleted
  • ${Boost_XXX_LIBRARY} becomes Boost::xxx (lowercase xxx from uppercase xxx) and note that the xxx has to be in the COMPONENTS list of find_package(Boost...) above.
  • Similarly ROOT_HIST goes to ROOT::Hist (but if so do we need to include COMPONENTS on find_package(root) lines?)
  • ${TBB} -> TBB:tbb
  • any include(UseCPack) stuff can be dropped
  • any add_directory(ups) lines can be dropped

Then take all of those changes, roll them up into a patch file with git diff > patchfile, and make sure we have
a patch('patchfile') line in the recipe, and the patchfile added to the recipe repository next to it.