Project

General

Profile

DES DM Stack Installation

This is a collection of notes from the initial installation of the DES DM software stack (version 6.3.2) at OSU.

Instructions from Brian and Marko with some feeback from OSU

SV team:
    Here are some notes from Marko Slyz on how to
    get the various 6.3.2 stack DESM codes and prereqs to compile
    on a stand alone Linux box running 'version 6' of redhat or
    scientific Linux.

    There is an older 'version 5' of Linux which we have also
    succeeded in getting to compile.

    SL5 and SL6 versions of Linux will be the 'supported' platforms,
    at least initially, and we will try and help you with
    compilation issues.

======================================================================

Contents:
    Building the DESDM 6.3.2 Stack Under EL6
    Building Other Prerequisites  (optional)
    Building the DESDM Home Code

Prerequisites:
    You want to pick a top level directory, let's call this DES
    Once you have everything installed this directory will contain three subdirectories:
           DES_HOME          for the DES codes such as mkbiascor
           stack             for the DES DM 6.3.2 stack (e.g. swarp, scamp)
           desdm             for the development code (wrappers)

Building the DESDM 6.3.2 Stacks Under EL6

  Note 1: These instructions are for the 6.3.2 stack.

  Note 2: Seem to need at least two cores to do the build, since
  otherwise the build code to handle the threading may get confused.

     1. Setup eups:
          mkdir stack
          export DES_BASE='/n/des/desdm632   - but use your top level directory instead'
          export EUPS_PKGROOT=http://deslogin.cosmology.illinois.edu/desdmpkgs
          cd stack
          export DES_PREREQ=${DES_BASE}/stack
          curl -o desinstall.sh http://deslogin.cosmology.illinois.edu/desdmpkgs/desinstall.sh
          bash ./desinstall.sh eups
          source ${DES_PREREQ}/eups/1.1.1/bin/setups.sh

      1.5 Some might find it convenient to combine some of these statements in a shell script since they will 
          have to be executed frequently.

#!/usr/bin/env bash
export SHELL='/bin/bash'
export DES_BASE='/n/des/desdm632   - but use your top level directory instead'
export EUPS_PKGROOT=http://deslogin.cosmology.illinois.edu/desdmpkgs
export DES_PREREQ=${DES_BASE}/stack
source ${DES_PREREQ}/eups/1.1.1/bin/setups.sh 

      2. Install extra packages if needed:
          yum install gcc-c++ zlib zlib-devel gcc-gfortranlibXft-devel libXpm libXpm-devel \
                libXext-devel readline readline-devel libXft-devel gcc make

         Depending on your local setup this command might require root privileges

      2.5 Cache credentials for svn:
           mv ~/.subversion ~/.subversion__original
           # Now there should be no .subversion directory present.

           # Do a random checkout just to cache the credentials:
           export DES_SVN=https://desweb.cosmology.illinois.edu/svn/desdm/devel
           svn co $DES_SVN/terapix/trunk terapix-trunk/psfex/eye/skymaker/psfex/src/fits

           # Accept the self-signed certificate permanently.
           # The fingerprint on mine is
           #       54:F1:74:56:BD:D3:AA:65:D8:BE:E8:BC:35:4B:BD:06:B6:D9:E7:A1
           # Enter password, and say "yes" when it asks about storing password
           # unencrypted on disk. Another alternative is a keyring:
           #   http://svnbook.red-bean.com/en/1.6/svn.serverconfig.netmodel.html#svn.serverconfig.netmodel.credcache

      3. Start the build going until it produces an error:
           time eups distrib --noclean --install desdmstack 6.3.2

         The --noclean switch here will leave the source code in place.

      4. Several packages have known issues. These will cause the build script to stop (crash)
         nrecipes is the first of these stops that you will encounter
         Copy in a build script for nrecipes that has the -std=c89 switch.
         This switch should keep the system's fmin() function from being
         declared, which was conflicting with the nrecipes fmin() funtion.

          cp nrecipes-2.0.build  stack/EupsBuildDir/Linux64/nrecipes-2.0/
               # see attachment for this fixed nrecipes-2.0.build

          Now build nrecipes manually:
             cd stack/EupsBuildDir/Linux64/nrecipes-2.0/
             bash ./nrecipes-2.0.build

          And "declare" it so that eups knows about it:
             time eups distrib --noclean --nobuild --install nrecipes 2.0

      5. Try to build the rest of the stack until you run into the next problem:
            time eups distrib  --noclean --install desdmstack 6.3.2

         Here is a generic description of the procedures:
         During the build of some of the terrapix modules you might run into build and compile errors
         Most often this is caused by differences in system configuration. To fix this error go to the
         source directory of the product in question. Typically this can be found in
            stack/EupsBuildDir/Linux64/product-version/product-version/product
         For example, if you encounter an error building the psfex module you have to cd to
            stack/EupsBuildDir/Linux64/psfex-3.10.1/psfex-3.10.1/psfex/

         cd to that directory and run
             autoreconf --force --install
         If your system doesn't have the autoreconf command you can get it from here:
             http://stackoverflow.com/questions/3096989/libtool-version-mismatch-error

         then check if your fix worked and try to build this module. 
         Go to the module directory (stack/EupsBuildDir/Linux64/product-version) and run (for the psfex example)
             time eups distrib --noclean --nobuild --install psfex 3.10.1
         Then go back to your top level directory and try again to build the complete DM stack
             time eups distrib --noclean --install desdmstack 6.3.2

         You might have to follow this procedure for these products
           swarp
           scamp
           psfex
           sextractor

      5.1  Fix for psfex (this is an application of the generic procedure described above. 
           You do this only once per product)
             cd stack/EupsBuildDir/Linux64/psfex-3.10.1/psfex-3.10.1/psfex/
             autoreconf --force --install
             cd ../..
             time eups distrib --noclean --nobuild --install psfex 3.10.1
           Or can probably just build the desdm stack.
             time eups distrib --noclean --install desdmstack 6.3.2
           (this worked for me and I could skip the previous step)

      5.2  Do the same thing for the sextractor:
             time eups distrib --noclean --install desdmstack 6.3.2
             cd stack/EupsBuildDir/Linux64/sextractor-2.15.2/sextractor-2.15.2/sextractor/
             autoreconf --force --install
             cd ../..
             bash ./sextractor-2.15.2.build
             time eups distrib --noclean --install sextractor 2.15.2
           Or can probably just build the desdm stack.
             time eups distrib --noclean --install desdmstack 6.3.2
           (this worked for me and I could skip the previous step)

      7. Finish the build:
            time eups distrib --noclean --install desdmstack 6.3.2

----------------------------------------------------------------------
Building Other Prerequisites.
These steps are optional.
At OSU we used our existing Python 2.7.3 installation that already included numpy etc.

1. Install Python-2.7.x:
    # Do this in the same directory that has stack, and that will have DESDM home.
    # The page for the current (8/7/12) Python is http://www.python.org/getit/releases/2.7.3/
    # Going mostly from memory for these commands, so they are somewhat approximate.
    wget http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
    ./configure --prefix=`pwd`
    make
    make install

    # Make sure this is the Python that will be used:
    source setup_runB.sh .  # set PATH to point to it
    python --version

    # May have to do something like
         chmod -R o+r Python-2.7.3
         find Python-2.7.3 -executable -exec chmod o+x {} \;
    # to share with other users, but don't know the
    # reason for the restrictive permissions.

2.  Building numpy:
    wget http://downloads.sourceforge.net/project/numpy/NumPy/1.6.2/numpy-1.6.2.zip?
r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fnumpy%2Ffiles%2FNumPy%2F1.6.2%2F&ts=1343793172&use_mirror=voxel
    export PYTHONPATH=
    # untar
    cd /cloud/login/mslyz/codes_try5b/numpy-1.6.2
    python setup.py build --fcompiler=gnu95
    python setup.py install

    or just type
       pip install numpy

3. Building pyfits:
    # Following instructions at http://www.stsci.edu/institute/software_hardware/pyfits/Download :
    wget http://pypi.python.org/packages/source/p/pyfits/pyfits-3.0.8.tar.gz
    export PYTHONPATH=${DES_BASE}/Python-2.7.3/lib/python2.7/site-packages
    tar xf pyfits-3.0.8.tar.gz
    cd ~/codes_try5b/pyfits-3.0.8/
    python setup.py install

    or just type
       pip install pyfits

  # When done there should be numpy and pyfits directories in
  #   /cloud/login/mslyz/codes_try5b/Python-2.7.3/lib/python2.7

----------------------------------------------------------------------

Building the DESDM Home Code

These programs are written specifically for DES. They
link against the libraries in the stack, so the stack
has to be installed first.

1. Check out the code: 
     cd $DES_BASE         # go to your top level directory
     mkdir -p desdm/devel/ImageProc
     cd desdm/devel/ImageProc
     svn checkout https://desweb.cosmology.illinois.edu/svn/desdm/devel/ImageProc/trunk

2. Fix Makefile in
     desdm/devel/ImageProc/trunk/src/Makefile
   to just compile the needed programs:
        imcorrect DECam_crosstalk fwhm runSExtractor fitscombine
         mkbiascor mkflatcor mkillumcor mksupersky

3. Run
     source setup_build.sh .

4. Do the actual compilation:
      cd desdm/devel/ImageProc/trunk/
      make
      make install

5. Check out the Wrappers and Integration code. This
   is all scripts, and doesn't need building.

    cd desdm/devel
    svn co https://desweb.cosmology.illinois.edu/svn/desdm/devel/Wrappers
    svn co https://desweb.cosmology.illinois.edu/svn/desdm/devel/IntegrationUtils

================

The replacement files and scripts you need for the stack and des code installation are available from this page. Please note that there might be newer versions of these files available directly from DES DM.