Project

General

Profile

Setting up dseb8

Reinstall SLF6

  • Partitions:
  • /
  • swap
  • /local
  • /data1
  • /data2
  • Basic server install with no additional customization
  • Disable network boot in BIOS boot order
  • Disable NetworkManager, iptables, ip6tables, firewall, and selinux
  • chkconfig NetworkManager off
  • chkconfig ip6tables off
  • chkconfig iptables of
  • reboot

edit /etc/yum.conf

proxy=http://192.168.1.1:3128/
# This is to avoid problems with the IB drivers from OFED.org.
exclude=compat-dapl,compat-dapl-devel,dapl,dapl-debuginfo,dapl-devel,dapl-devel-static,dapl-utils,ibacm,ibsim,ibsim-debuginfo,ibutils,infiniband-diags,infinipath-psm,infinipath-psm-devel,kernel-ib,kernel-ib-devel,libcxgb3,libcxgb3-debuginfo,libcxgb3-devel,libcxgb4,libcxgb4-debuginfo,libcxgb4-devel,libibcm,libibcm-debuginfo,libibcm-devel,libibmad,libibmad-debuginfo,libibmad-devel,libibmad-static,libibumad,libibumad-debuginfo,libibumad-devel,libibumad-static,libibverbs,libibverbs-debuginfo,libibverbs-devel,libibverbs-devel-static,libibverbs-utils,libipathverbs,libipathverbs-debuginfo,libipathverbs-devel,libmlx4,libmlx4-debuginfo,libmlx4-devel,libmthca,libmthca-static,libmthca-debuginfo,libmthca-devel-static,libnes,libnes-debuginfo,libnes-devel-static,librdmacm,librdmacm-debuginfo,librdmacm-devel,librdmacm-utils,libsdp,libsdp-debuginfo,libsdp-devel,mpi-selector,mpitests_mvapich2_gcc,mpitests_mvapich_gcc,mpitests_openmpi_gcc,mstflint,mvapich2_gcc,mvapich_gcc,ofed-docs,ofed-scripts,openmpi_gcc,opensm,opensm-debuginfo,opensm-devel,opensm-libs,opensm-static,perftest,qperf,qperf-debuginfo,rds-devel,rds-tools,sdpnetstat,srptools

install rsh

  • yum install rsh and rsh-server
  • chkconfig rsh on
  • chkconfig rlogin on
  • chkconfig rexec on
  • remove /etc/hosts.allow and /etc/hosts.deny
  • add lines for rexec, rlogin, and rsh to top of /etc/securetty
  • reboot

user logins

  • edit /etc/idmapd.conf
  • add Domain = private.net near the top
  • restart idmapd or reboot
  • if you don't do this, all uids and gids will be squashed to nobody
  • do this on dsfr6, dseb8, and ds50wsp
  • make sure /home is empty
  • mkdir /products
  • mount /home, /products, and /usr/local from dsfr6
  • use cut and paste to copy UID's and GID's from dsfr6
  • mount ds50ws home directory as /home-ds50ws

host definitions

/etc/hosts:

192.168.1.1 ds50wsp.private.net ds50wsp
192.168.1.6 dsfr6.private.net dsfr6
192.168.1.8 dseb8.private.net dseb8
#IB
192.168.10.6 ibdsfr6.private.net ibdsfr6
192.168.10.8 ibdseb8.private.net ibdseb8

/etc/hosts.equiv:

ds50wsp
dsfr6
dseb8

check and edit:
  • /etc/sysconfig/network-scripts/ifcfg-eth0
  • /etc/sysconfig/network-scripts/ifcfg-eth1
  • /etc/sysconfig/network-scripts/ifcfg-ib0

working environment

Required rpms
Install OFED drivers and mvapich

  • add a line to /etc/security/limits.conf
  • @daq - rtprio 5
  • this enables real time priorities for everyone in the daq group
  • As Ron found on 08-Jul-2013, it may also be necessary to change the UsePAM setting in /etc/ssh/sshd_config from no to yes to allow SSH logins to change the realtime priority (as well as rsh logins)
  • install A3818 driver (already built)
  • cp -p /usr/local/CAEN/A3818Drv-1.3/src/a3818.ko /lib/modules/2.6.32-279.19.1.el6.x86_64/extra/
  • cp -p /home/garren/85-a3818.rules /etc/udev/rules.d
  • /sbin/depmod -a
  • check with "/sbin/modprobe a3818" (no such device)

ssh tunnelling

  • needed for both totalview and git
  • get a local copy of /usr/local/cet-chg (Chris' scripts)
  • make sure they are edited to reference ds50wsp
  • edit /etc/ssh/ssh_config
    Host 192.168.* ds50wsp dseb* dsfr*
            GSSAPIAuthentication no
            GSSAPIDelegateCredentials no
            PasswordAuthentication no
            PubKeyAuthentication yes
            ForwardAgent yes
            ForwardX11Trusted yes
            ForwardX11 yes
    
  • cd /etc/init.d; ln -s /usr/local/cet-chg/unix-admin/tv-ssh-tunnel
  • chkconfig --add tv-ssh-tunnel
  • edit /etc/hosts
  • add flmvm1.fnal.gov flmvm1 to the end of the 127.0.0.1 line
  • generate a root public key for ds50wsp