FNAL Spack / SpackDev Project

This project documents our efforts to develop a replacement for the current UPS-based package management, development, build, and release ecosystem at Fermilab currently used by art-based and other experiments.


Our presentation (SpackForHEP.pdf) and subsequent paper (SpackForHEP-paper.pdf) from CHEP2018 in Sofia, Bulgaria describe the project, its motivations, and status as of early July, 2018. In short:

  • Our existing system is based on a home-grown application (UPS) that was first released in 1990. In addition to the difficulty of maintaining a software application for so long, through personnel and platform / OS changes, requirements have changed and new solutions need to be considered.
  • Spack is an existing package management system developed at LLNL by HPC system administrators to meet the needs of the HPC community. While not meeting every requirement we have out of the box, it is a community-supported system with a wide contributor base and a demonstrated openness to enhancements that widen its applicability beyond its original goals.
  • In addition to Spack itself, the ecosystem will consist of other products and "glue" to meet the full range of requirements, including:
    • SpackDev: a companion to Spack to facilitate the simultaneous development of multiple Spack-based packages.
    • cetmodules: a replacement for the UPS-based cetbuildtools system for CMake-based building of experiment software.
    • SciSoft will evolve to become the source of Spack-based binary and source-code packages in the same way that it is for UPS-based software currently.

The MVP ("1a"—LArSoft Edition)

The Minimum Viable Product aims to satisfy enough of the requirements that interested experimenters can begin to explore the system and see how it will operate:

  • Software stack for SL7 / GCC (C++17) only (optimized for profiling).
  • Keep track of issues along the way but achieve the narrow goal first.
  • “Pinnacle” of the software stack: the art suite LArSoft distribution.
  • Everything built “our way” to maximize realism & compatibility for experiments.
  • Use system-available packages where possible via packages.yaml.
  • First demonstration of cetmodules.
  • NOT “release”-oriented.
  • NOT a solution to every problem.
  • NOT a guarantee that every remaining problem can be solved.
  • NOT a collection of every piece of software every experiment will need.

The original MVP was released 2018-08-31 and covered only the art suite (hence the strikethrough above). SpackDev was formulated as a standalone application using Spack as an external utility, which led to long initialization times. MVP1a, released 2019-06-20, covers the whole LArSoft offline distribution, including art suite 3.02.04 and features an overhaul of SpackDev, now reconfigured to operated as an external Spack command (`spack dev init` vs `spackdev init`).

Trying out the MVP.

Full instructions can be found at export:spack_glue|MVP/README.pdf@MVP1a ("the MVP"). If anything is missing or unclear, please let us know (contact information below).

Further information

The following documents and presentations may provide helpful information on progress and planning:

Contact us

The FNAL Spack / SpackDev team currently consists of:

  • Jim Amundson (emeritus).
  • Vito Di Benedetto.
  • Lynn Garren.
  • Patrick Gartung.
  • Lisa Goodenough.
  • Chris Green.
  • Marc Mengel.
  • Steve White.

We can be reached at spackdev-team <at> fnal <dot> gov.