Cross-platform installs into CVMFS

Note: the third option is now available, the cross product installs are now workining for installing MacOS from Linux hosts thanks to patches from Patrick Gartung. So this is of historical interest only.

Basically three options:

  1. install into a local filetree named /cvmfs and rsync
  2. install into a local filetree named /cvxxx and convert
  3. fix spack to do actual cross platform installs from cache

We'll discuss these more below, but in the short term, I think the first option will be used to get
spack areas setup on /cvmfs. The second option sounds like a good bridge option until the third
is available.

Local filetree named /cvmfs

This is straightforward, but requires a lot of disk space. If you install into a locally mounted filesystem called /cvmfs all the paths will be right, so rsync-ing that into the /cvmfs area will get correctly installed binaries, etc. But you'll have to have all the dependencies also installed locally. Note that the local /cmmfs filetree only needs the dependencies for the things you are actually building; not the entirety of the real /cvmfs area; then you can rsync just the /opt areas you built, and reindex the central area to include those new packages.

install into a local filetree named /cvxxx and convert

This is an interesting permutation. If you make a spack area under /cvxxx (which can chain off of
the actual /cvmfs one for dependencies) you can install into it, tar up the resulting /cvxxx/.../opt
subtrees, filter the tar image to replace the /cvxxx/ with /cvmfs/, and unpack it on the cvmfs master
node. This has the advantage of saving the space and time of installing the dependencies. There is a
slight risk that the generated code would have some instance of the string '/cvxxx/' that was not actually a pathname; but matching against a longer string( i.e. replacing '/cvxxx/' or what have you) will generally mitigate that risk. (that would be 5 long ints in a row that would have to match to get an accidental replacement, or several specific doubles in a row: (1.671523e+184, 2.956719e-85, 3.292184e+180)

cross platform install changes

Must fix spack to do cross platform intalls, and make sure there's an install_name_path
available for Linux(?)...

Note current command-line-platform branch seems incomplete, as it doesn't appear to change package module usage in for example -- have to use MacOS tools for MacOS packages, regardless what platform you are on at the moment.
Looks to me like one would have to pass platform info down the chain from through etc. down into the routines in to use the right platform info for the package we're installing, rather than for the platform we're on...

Also need a flag/config option to not install from source, but only from binary cache./ 1501 or so.