Build a non-OFED mvapich and use it.

These instructions assume that you have an SLF6 machine and you are using the RedHat-suppplied InfiniBand support. For instructions how to install this, see the RedHat-provided instructions.

  1. Install the development RPMs for InfiniBand support.
    yum -y install libibverbs-devel libibumad-devel libibcommon-devel compat-dapl-devel \
                   libibmad-devel librdmacm-devel libibcm-devel infiniband-diags-devel opensm-devel \
                   ibutils-devel ibacm-devel dapl-devel
  2. Download the latest source of mvapich2 from these build instructions have only been tested with 2.0.1, but should work with any later version.
  3. Untar the source and change directory into the top level of the source.
  4. If you are building a version older than 2.1 (note: 2.1rc3 is older than 2.1), you must apply the attached patch file with:
    patch --verbose -p0 < <patch-file>
  5. Ensure you are building with the native compiler for consistency reasons (which gcc).
  6. Configure with (e.g.):
    ./configure --prefix=/usr/local/mvapich2-2.0.1 --enable-fast=O3,ndebug --enable-f77 --enable-fc \
                --disable-cxx --enable-romio --enable-versioning --enable-threads=runtime --enable-registration-cache \
                --enable-rsh --enable-shared --enable-static --enable-yield=sched_yield --enable-rdma-cm --with-pm=hydra
    (change the prefix if desired, but retaining version information is strongly recommended).
  7. Build with make -j<num>
  8. Install with make install (as super-user if required for installation destination).
  9. Use product-stub from a recent version of cetpkgsupport, e.g.:
    product-stub mvapich2 v2_0_1 /usr/local/mvapich2-2.0.1 <product-topdir>
  10. Change artdaq/ups/product_deps to use the desired version, set up an ib variant, build and test.