Project

General

Profile

Support #23688

Requesting new UPS packages CVODES, Stan-math, Stan (and new version of Eigen)

Added by Jeremy Wolcott 7 months ago. Updated 16 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
-
Start date:
12/03/2019
Due date:
% Done:

100%

Estimated time:
Spent time:
Scope:
Internal
Experiment:
DUNE
SSI Package:
Co-Assignees:
Duration:

Description

Hi,

I'd like to request new UPS products for a few packages to be used with DUNE.

In addition, I'd like to request version 3.3.5 of Eigen (which is already distributed in larsoft, but the newest version of which is v3_3_4a) be built.

I've built these all myself for trial UPS packages that we've been using within NOvA, so I can help with building if needed, though my packages were manually assembled and I think SciSoft uses a more automated system. (The versions given below lag behind the most recent available for all the packages, but these versions are known to work for the desired application; we can investigate upgrades later.) I also wrote UPS table files by hand for the NOvA versions, which I'd be happy to offer if needed, but again, I believe SciSoft has a more automated way of generating them.
I offer descriptions of the packages, the locations of source code, and thoughts on how to build them below.

Let me know if I can be of any more help.

Thanks!

-Jeremy


The new packages are:

  • Sundials CVODES, version 3.2.1.
    CVODES is part of the Sundials package, but we don't need the other bits. When I built it for NOvA, I therefore disabled the other components (see the CMake invocation below) and named the resulting UPS package 'cvodes'.
    Sundials builds with CMake.
    Source code: https://computation.llnl.gov/projects/sundials/download/sundials-3.2.1.tar.gz
    Sample cmake invocation:
    cmake -DCMAKE_INSTALL_PREFIX=/path/to/install/dir   -DCMAKE_INSTALL_LIBDIR=slf6.x86_64.e17.prof/lib -DCMAKE_BUILD_TYPE=Release -DEXAMPLES_INSTALL=Off -DBUILD_ARKODE=Off -DBUILD_CVODE=Off -DBUILD_CVODES=On -DBUILD_IDA=Off -DBUILD_IDAS=Off -DBUILD_KINSOL=Off /path/to/sundials-3.2.1
    

    UPS dependencies: none (as configured)
    Proposed UPS package name: cvodes
  • Stan-math, version 2.18.0.
    Stan-math is entirely headers (apart from some self-tests). Stan-math depends on Eigen, CVODES, Boost, and tbb, and the UPS package should reflect this. It ships with a 'lib' directory in the source package which contains all of these (plus a few more needed to build Boost), but I think it's cleaner to obtain those dependencies via UPS, which is why I requested CVODES above. (UPS packages for Eigen, Boost, and tbb already exist.) Though Stan-math is itself header-only, and thus doesn't need any flavor or compiler qualifiers, it depends on CVODES, which does; I couldn't figure a way to make the UPS dependency chain work without attaching CVODES' flavor & qualifiers to Stan-math as well. Maybe you have a better way of doing this.
    Source code: https://github.com/stan-dev/math/archive/v2.18.0.tar.gz
    UPS dependencies: tbb, boost, eigen, cvodes
    Proposed UPS package name: stan_math
  • Stan, version 2.18.0.
    Like Stan-math, Stan is entirely headers (apart from some self-tests) and has no installer. Stan depends on Stan-math, and the UPS package should reflect this. Like with Stan-math, the Stan bundle ships with a 'lib' dir containing Stan-math in it, but I think it's cleaner to connect the dependencies via UPS rather than bundling it into a subdirectory. Also, in my scheme Stan inherits Stan-math's qualifiers and flavor; again, you may have a better way of doing this.
    Source code: https://github.com/stan-dev/stan/archive/v2.18.0.tar.gz
    UPS dependencies: stan_math
    Proposed UPS package name: stan

History

#1 Updated by Kyle Knoepfel 7 months ago

  • Assignee set to Lynn Garren
  • Status changed from New to Feedback

Packaging a part of an overall system can get complicated. Due to complexity reasons, our preference is to provide a UPS product called "sundials" with only CVODES enabled until requests come in for other solvers.

We are happy for you to provide whatever assistance you can in producing this product. We will provide the buildshims templates for you to fill in with the specific build instructions.

Regarding Stan, please confirm that what you need does not depend on R.

#2 Updated by Jeremy Wolcott 7 months ago

Hi,

Packaging a part of an overall system can get complicated. Due to complexity reasons, our preference is to provide a UPS product called "sundials" with only CVODES enabled until requests come in for other solvers.

Ok. No problem.

We are happy for you to provide whatever assistance you can in producing this product. We will provide the buildshims templates for you to fill in with the specific build instructions.

Ok. I'll see what I can do -- I'm not familiar with the system but with some minimal help I can probably make it happen.

Regarding Stan, please confirm that what you need does not depend on R.

No, no required dependency. (There's an optional R interface, but it's not required unless some of the tests are built. I typically don't build any of the tests myself since they're more targeted at Stan developers -- I just deploy the headers, which only have the dependencies I mentioned above.)

#3 Updated by Kyle Knoepfel 7 months ago

  • Status changed from Feedback to Assigned

#4 Updated by Jeremy Wolcott 6 months ago

Hi,

Any progress here to report? It sounded like I was going to be asked for some contributions but haven't seen anything yet.

Thanks!

#5 Updated by Lynn Garren 2 months ago

  • Assignee changed from Lynn Garren to Jeremy Wolcott
  • Co-Assignees Lynn Garren added

The following ssi build repositories are now available:
ssh:///cvs/projects/build-framework-stan-ssi-build
ssh:///cvs/projects/build-framework-stan_math-ssi-build
ssh:///cvs/projects/build-framework-sundials-ssi-build

#6 Updated by Jeremy Wolcott 2 months ago

I've filled these repositories with bootstrap and build scripts as well as UPS table files. I've tested them locally and they seem to build and set up correctly. What's the next step?

#7 Updated by Lynn Garren 2 months ago

Thanks. We'll take a look and let you know if we see any problems.

#8 Updated by Jeremy Wolcott about 2 months ago

Hi,

Anything to report? I'd love to have these available, and I'm ready to push on anything that needs fixing.

-Jeremy

#9 Updated by Lynn Garren about 2 months ago

Sorry Jeremy, there is a lot that needs fixing. I'm working on it.

#10 Updated by Jeremy Wolcott about 2 months ago

Ok! If I can help at all, let me know. Otherwise I'm happy to stay out of your way.

#11 Updated by Jeremy Wolcott about 1 month ago

Hi again,

Are there any roadblocks I can help with? This is beginning to get in the way of physics work in DUNE.

-Jeremy

#12 Updated by Lynn Garren about 1 month ago

Sorry, it's just time. Almost ready, but got sidetracked by other work.

#13 Updated by Lynn Garren 25 days ago

  • % Done changed from 0 to 100
  • Assignee changed from Jeremy Wolcott to Lynn Garren
  • Status changed from Assigned to Resolved
The following products are now available on SciSoft and larsoft cvmfs:
  • stan v2_18_0
  • stan_math v2_18_0
  • sundials v3_2_1

#14 Updated by Jeremy Wolcott 24 days ago

Fantastic!! Thank you so much for your help!

A new version of Stan with some important new features has appeared in the intervening time since I opened this ticket, so I will likely be requesting a build of that in the not-so-distant future, but (a) hopefully that will be straightforward now that the infrastructure is in place, and (b) this is enough for now.

I'll give it a spin hopefully in the next couple of days. Thanks again!

#15 Updated by Jeremy Wolcott 18 days ago

Just one quick followup: I noticed that there's a debug e17 build, but not a prof one:

 $ ups list -aK+ stan | grep e17
"stan" "v2_18_0" "Linux64bit+2.6-2.12" "debug:e17" "" 

Was that an accident?

#16 Updated by Jeremy Wolcott 18 days ago

Also, I discovered that at least in the e19:prof variant, $STAN_INC isn't pointed at the right directory:

$ echo $STAN_INC
/cvmfs/larsoft.opensciencegrid.org/products/stan/v2_18_0/src/
$ ls $STAN_INC
ls: cannot access /cvmfs/larsoft.opensciencegrid.org/products/stan/v2_18_0/src/: No such file or directory
$ echo $STAN_DIR
/cvmfs/larsoft.opensciencegrid.org/products/stan/v2_18_0
$ ls $STAN_DIR
include  source  ups

Looks like the directory structure in the package as installed on CVMFS has an 'include/' directory inserted in. I made a commit to the build-framework-stan-ssi-build package which I think will fix it.

#17 Updated by Lynn Garren 18 days ago

Jeremy, I had to make a number of changes to align the stan build with conventions. One was adding an include directory and putting the headers there. Unfortunately, I failed to adjust the definition of $STAN_INC in the table file. I will do that and provide stan v2_18_0a.

Note that you cannot unwind the stan source code tarball under $STAN_DIR. That is not supported, so I put it in source. $STAN_INC should point to $STAN_DIR/include.

Please check your $PRODUCTS path. Here is what is available in the larsoft cvmfs:

"stan" "v2_18_0" "Darwin64bit+18" "c7:prof" "" 
"stan" "v2_18_0" "Linux64bit+3.10-2.17" "debug:e19" "" 
"stan" "v2_18_0" "Linux64bit+3.10-2.17" "e19:prof" "" 
"stan" "v2_18_0" "Linux64bit+3.10-2.17" "c7:debug" "" 
"stan" "v2_18_0" "Linux64bit+2.6-2.12" "e19:prof" "" 
"stan" "v2_18_0" "Linux64bit+2.6-2.12" "debug:e19" "" 
"stan" "v2_18_0" "Linux64bit+3.10-2.17" "c7:prof" "" 
"stan" "v2_18_0" "Darwin64bit+18" "c7:debug" "" 
"stan" "v2_18_0" "Linux64bit+2.6-2.12" "c7:debug" "" 
"stan" "v2_18_0" "Linux64bit+2.6-2.12" "c7:prof" "" 

#18 Updated by Jeremy Wolcott 17 days ago

Thanks Lynn. I thought I had filtered out the extra versions of the `stan` package from my local UPS area where I'd done testing, but it appears I hadn't caught them all, and I got excited when I saw an e17 version, which would mean I wouldn't have to change my testing setup as much. But indeed, I only see e19s when I'm more careful about my $PRODUCTS.

When I explicitly override $STAN_INC to the right location, my tests work, so I'm happy to report that v2_18_0a of stan should be enough.

Thanks again!

#19 Updated by Lynn Garren 17 days ago

stan v2_18_0a is now on cvmfs larsoft. I hope there are no more surprises.

#20 Updated by Jeremy Wolcott 17 days ago

Seems to be working! I think we are all set. Thank you once again!

#21 Updated by Lynn Garren 16 days ago

  • Status changed from Resolved to Closed

Great! Closing this issue.



Also available in: Atom PDF