Project

General

Profile

Support #18107

minos-dcs03-nd SLF5 nugen fallback

Added by Arthur Kreymer almost 2 years ago. Updated over 1 year ago.

Status:
Work in progress
Priority:
Normal
Start date:
11/02/2017
Due date:
12/31/2017
% Done:

60%

Estimated time:
5.00 h
Duration: 60

Description

Building offline nugen code requires an SLF5 system.
We should try using minos-dcs03-nd, the spare Minos DCS system.

History

#1 Updated by Arthur Kreymer almost 2 years ago

We presently use minos-slf5 to build Nugen. This goes away 2018 Jan 12.

The spare DCS system minos-dcs03-nd is running the same SLF 5.11,
and has plenty of capacity ( 8 cores, 16 GB memory )

Testing this would require :
  • get permission from Minerva and Neutrino Div.
  • create a minsoft local account, with usage logging
  • mount /grid/fermiapp read/write
  • install additonal software as needed.
We should have a usage agreement , something like
  • log all usage
  • use no local files, just /grid/fermiapp
  • stop minsoft activity if minos-dcs03-nd becomes the DCS production system.

#2 Updated by Arthur Kreymer almost 2 years ago

I have updated an existing Minos SLF5 document to outline
conditions for use of minos-dcs03-nd as an SLF fallback.

https://cdcvs.fnal.gov/redmine/projects/software/wiki/Mnslf5

If approved by Minerva we should give this a try soon,
so that minos-slf5 can be shut down.
Please note approval in this Issue.

#3 Updated by Arthur Kreymer over 1 year ago

  • Status changed from New to Work in progress
  • % Done changed from 0 to 10

Based on email responses so far indicating approval,
I have submitted

RITM0633247 11/30 minos-dcs-03 /grid/fermiapp and minsoft

Please mount /grid/fermiapp read-write on minos-dcs03-nd
and establish a minsoft account with home /grid/fermiapp/minos/home/minsoft

This for SLF5 offline builds as documented in
https://cdcvs.fnal.gov/redmine/issues/18107

Subject to Minerva approval, as documented in
https://cdcvs.fnal.gov/redmine/projects/software/wiki/Mnslf5

This will require support from Networking for the DAQ firewalls,
and NCS for the /grid/fermiapp export.

We need to demonstrate that this works by the end of 2017,
so that that minosgpvm01 (minos-slf5) can be shut down.

#4 Updated by Arthur Kreymer over 1 year ago

RITM0633247

2017-12-05 12:06:00 CST - Bonnie King (Additional comments)

To reiterate my points at the MINERvA meeting this morning,
SLAM no longer has the infrastructure to support SLF5.
We can make no support commitments at all for this machine.

Please note that if it fails, we can't reinstall it with SLF5,
and it is not in configuration management.
 
I have:
 
Transferred Primary Admin to Nur,
and added Donatella and Howard as authorized admins.
MINERvA will be responsible for keeping security exemptions up to date
and primary/authorized admins will receive any security notices.
 
I created the minsoft user with /grid/fermiapp/minos/home/minsoft as home
and set up autofs to mount /grid/fermiapp.
 
Please check it out, thanks!
__________________________________________________________________

Thanks !

I have logged in and verified write access to /grid/fermiapp/minos

ssh
ssh minsoft@minos-dcs03-nd
cat log/logins.log
Tue Dec 5 15:38:32 CST 2017 kreymer
Wed Dec 6 08:00:01 CST 2017 kreymer minos-dcs03-nd
Wed Dec 6 08:09:57 CST 2017 kreymer minos-dcs03-nd

We now need to verify that the formaer minos-slf5 users
can login and build code on minos-dcs03-nd.

I have added to minos@minos-gateway-nd .k5login



and removed some obsolete entries










#5 Updated by Arthur Kreymer over 1 year ago

  • % Done changed from 10 to 50

To log into the minos-dcs03-nd system :

ssh

ssh minsoft@minos-dcs03-nd

Be careful,
the session will have write access to all the minsoft files in /grid/fermiapp/minos.

I see that rchen2 got access Dec 8 :

cat log/logins.log
Tue Dec 5 15:38:32 CST 2017 kreymer
Wed Dec 6 08:00:01 CST 2017 kreymer minos-dcs03-nd
Wed Dec 6 08:09:57 CST 2017 kreymer minos-dcs03-nd
Fri Dec 8 07:57:08 CST 2017 rchen2 minos-dcs03-nd
Mon Dec 11 20:50:11 CST 2017 kreymer minos-dcs03-nd

#6 Updated by Rui Chen over 1 year ago

The first test is to see if I can source the minos setup scripts. I think it failed due to the ups errors.

source /grid/fermiapp/minos/minossoft/setup/setup_minossoft_FNALU.sh -r R3.05 -O
setup_minossoft_FNALU forcing 64bit build
setup_minossoft_FNALU thinks R3.05 (305) doesn't have a --64bit build, force --32bit
setup_minossoft_FNALU_parser forcing flavor: Linux+2.6-2.12
ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ups)
/grid/fermiapp/minos/products/db/../etc/setups.sh: line 24: .: filename argument required
.: usage: . filename [arguments]
ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ups)
ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ups)
could not find a gcc version for release "R3.05" on
ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ups)
ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by ups)
-bash: .: filename argument required
.: usage: . filename [arguments]
/grid/fermiapp/minos/products/prd/ups/v5_1_4/Linux-2-6-2-12/bin/ups: /lib/libc.so.6: version `GLIBC_2.7' not found (required by /grid/fermiapp/minos/products/prd/ups/v5_1_4/Linux-2-6-2-12/bin/ups)
-bash: .: filename argument required
.: usage: . filename [arguments]
/usr/bin/which: no loon in (/grid/fermiapp/minos/products/prd/ups/v5_1_4/Linux-2-6-2-12/bin:/usr/krb5/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin)
adjust_tsql.sh ENV_TSQL_USER="reader" based on EXPERIMENT env ():GRID_GROUP env ():GROUP env ():id -g -n (e875)

Maybe we need to modify the ups setup for dcs03 machine?

#7 Updated by Robert Hatcher over 1 year ago

  • % Done changed from 50 to 60

Try:

ssh minos@minos-gateway-nd.fnal.gov
ssh minsoft@minos-dcs03-nd
source /grid/fermiapp/minos/minossoft/setup/setup_minossoft_FNALU.sh -r R3.05 -O -H Linux+2.6-2.5 --32bit

Two things were broken. First the fix for setting the UPS_OVERRIDE for when it lands on the freaky grid nodes wasn't working for the the now "weird" SLF5 case, so we'll explicitly force it from the command line (with the -H flag). Secondly, when setting ups the "minos_32bit_libs" product for "slf5" vs. "slf6" it was keying off the node name ... back when our only SLF5 machine was minos-slf5. I've made that a bit more flexible.

Hopefully this will resolve the problem of setting up MINOS SLF5 builds ... [sigh]



Also available in: Atom PDF