Project

General

Profile

Integration of SpackSpack Dev into dunetpc build

What is Spack?

What are the issues?

  • The builds takes too long. The MVP built these packages on my disk and took about 8 hours to do so on my not-so-new desktop. It put larsim in
    a separate directory.
 art             cfitsio    dk2nugenie    g4photonevaporation  genie            intel-tbb     lhapdf      messagefacility  postgresql     root
 art-root-io     clhep      fftw          g4pii                gl2ps            larcore       libarchive  mysql            py-numpy       sqlite
 boost           cmake      fhicl-cpp     g4radioactivedecay   gmake            larcorealg    libiconv    netlib-lapack    py-pybind11    unuran
 canvas          cppgsl     ftgl          g4realsurface        gmp              larcoreobj    libuv       nettle           py-setuptools  vdt
 canvas-root-io  cppunit    g4abla        g4saiddata           gsl              lardata       libwda      ninja            pythia6        xerces-c
 catch           critic     g4emlow       g4tendl              hep-concurrency  lardataalg    libxml2     nucondb          python         xrootd
 cetlib          cry        g4ensdfstate  gallery              ifbeam           lardataobj    log4cpp     nusimdata        range-v3
 cetlib-except   davix      g4ndl         geant4               ifdh-art         larevt        lzo         nutools          rhash
 cetmodules      dk2nudata  g4neutronxs   geant4-data          ifdhc            larsoft-data  marley      pdfsets          rngstreams
  • Can users develop just a subset? Say just dunetpc, or maybe more relevant for some people, protoduneana? Or do they have to compile and install larsoft, art, geant, root, gcc and all the rest in their own disk space? This step is slow
    and poses a barrier to developing software if we unleash it on our entire collaboration.
  • How do users distribute their private code to the grid? Without LD_LIBRARY_PATH, some editing of rpaths needs to be done in order to relocate code. Maybe that's what spack buildcache install package-name/hash
    does when it installs in CVMFS? What's the equivalent for user tarballs?
  • How does Spack interface with containers?
  • DUNE wants both build and packaging where the latter includes distribution of libraries, executables, etc. E.g what cvmfs and scisoft provide now. A developer should be able to locate or install all the packages on which his or her package depends and then check out and build code using those. The one time I tried out spack, I was required to build many low level packages in addition to the one of interest. This is not what we want.