Project

General

Profile

Bug #5115

Trouble using MRB on remote installation

Added by William Seligman about 6 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Install / build / code mgmt
Target version:
Start date:
12/19/2013
Due date:
% Done:

0%

Estimated time:
Occurs In:
Experiment:
MicroBooNE
Co-Assignees:
Duration:

Description

I'm trying to run the LArSoft git+mrb combination at Nevis, starting with the non-experiment-specific distribution. The OS is Scientific Linux 5 (not SLF5).

The setup goes fairly smoothly. Downloaded and installed software using instructions on http://oink.fnal.gov/distro/larsoft/larsoft-v0_02_01.html. (Note: edited downloadLArSoft-v0_02_01.sh to change "tar xf" to "tar xvf" throughout; the long pause while unpacking files without any hint of progress is disturbing.)

After that:

prod_db=/a/data/westside/seligman/larsoft
source /a/data/westside/seligman/larsoft/setup
setup git
  1. I get this error:
  2. ERROR: Product 'git' (with qualifiers ''), has no current chain (or may not exist)
  3. Since I have git-1.8.2 installed on this machine anyway, I ignore this message.
    setup gitflow
    setup mrb
    export MRB_PROJECT=larsoft
    cd /a/share/westside/seligman/microboone/gitDev
    mrb newDev -v v0_02_01 -q e4:debug
    source /a/share/westside/seligman/microboone/gitDev/localProducts_larsoft_v0_02_01_e4_debug/setup
    cd srcs
    mrb gitCheckout larsoft_suite
    cd ../build
    source mrb setEnv

I get the error:

basename: missing operand
Try `basename --help' for more information.
source:.: no such file or directory: /a/home/tanya/setEnv.sh

History

#1 Updated by William Seligman about 6 years ago

I should have mentioned: I'm using zsh. (And must continue using zsh, given the other scripts I use in my environment.)

#2 Updated by Erica Snider about 6 years ago

  • Status changed from New to Assigned
  • Assignee set to Lynn Garren

Hi Bill,
It appears that the lines dealing with git have been commented out of the download script, but I'm not sure why. We clearly either need to fix the script or the documentation, so I'll follow up on that once I understand why those lines were commented out.

As for the basename error, I think the problem is zsh. This is not something that I recall discussing, but mrb expects to be sourced from bash. Since you are sourcing it from zsh, a bash-specific variable that is being used to test whether mrb is being sourced is not defined, hence the basename error.

I'll think about possible solutions, but will probably need to wait until Lynn gets back from vacation to see what we can do. In the mean time, all I can suggest is that you perform the build phase, starting from the 'source mrb setEnv', in bash, and then go back to zsh. The larsoft product setup appears to work from zsh (although I didn't try running anything after the setup), so this somewhat awkward zig-zag work-around should allow you to continue working.

Let me know if this will work.

Rick

#3 Updated by William Seligman about 6 years ago

  • Assignee deleted (Lynn Garren)

Can confirm: the lines work in bash. This is a zsh implementation bug.

#4 Updated by William Seligman about 6 years ago

I did a bit of homework. The fix in mrb is to replace all occurrences of

${BASH_SOURCE[0]}
with
${BASH_SOURCE:-$_}

Hopefully this will save Lynn a few minutes of work.

#5 Updated by Erica Snider about 6 years ago

That's great! Thanks for the tip.

#6 Updated by William Seligman about 6 years ago

It's getting complicated. The correct syntax is

${BASH_SOURCE:-${(%):-%N}}

And instead of

thisDirAB=$(cd ${BASH_SOURCE[0]%/*} && echo $PWD/${BASH_SOURCE[0]##*/} )
it's easier to use
scriptName=${BASH_SOURCE:-${(%):-%N}}
thisDirAB=$(cd `dirname ${scriptName}` && echo $PWD/`basename ${scriptName}` )

But that's still not sufficient, because setEnv.sh also has problems in zsh.

I'll continue to work on this when I can. It may be that all this shell scripting virtuosity is a bit of overkill for what mrb is trying to do; much of it could be replaced with

which mrb
if you didn't want to allow for the possibility of script aliasing and linking and whatnot.

#7 Updated by William Seligman about 6 years ago

To fix setEnv.sh to work under both bash and zsh, I had to replace all tests of the form

"string1" == "string2" 

with
"string1" = "string2" 

This got me through setEnv.sh, but I'm balked by setEnv in the sources directory:

The working build directory is /a/share/westside/seligman/microboone/gitDev/build
The source code directory is /a/data/westside/seligman/larsoft
----------- check this block for errors -----------------------
/a/share/westside/seligman/microboone/gitDev/srcs/setEnv:1: no such file or directory: /a/data/westside/seligman/larsoft/mrb/v0_05_01/bin/setup_products /a/data/westside/seligman/larsoft /a/share/westside/seligman/microboone/gitDev/build debug e4 
source:shift: shift count must be <= $#
source:.: not enough arguments
/a/share/westside/seligman/microboone/gitDev/srcs/setEnv:1: no such file or directory: /a/data/westside/seligman/larsoft/mrb/v0_05_01/bin/define_local /a/data/westside/seligman/larsoft /a/share/westside/seligman/microboone/gitDev/build debug e4 
source:shift: shift count must be <= $#
source:.: not enough arguments
----------------------------------------------------------------

In other words, setEnv (a shell-agnostic scripts of the type beloved by ups) picks up the wrong source code directory in zsh; it should be /a/share/westside/seligman/microboone/gitDev/srcs for my installation.

So I give up on zsh for now, and move strictly to bash. I'm interested in building larexamples by itself; that's the instructional package whose instructions I'd have to revise with the git+mrb move. Working only in bash, I can get everything to work, but with a modification to your quick-start guide: a simple

setup larsoft v0_02_01 -qdebug:e4

won't work. In order for "mrb install" to work, I have to do:

setup larsoft v0_02_01 -qdebug:e4
setup larcore v0_02_01 -qdebug:e4
setup lardata v0_02_01 -qdebug:e4
setup larevt v0_02_01 -qdebug:e4

Probably there's something in the guide I missed, but I can't figure it out.

#8 Updated by Lynn Garren about 6 years ago

  • Assignee set to Lynn Garren

Well, this is interesting. I never expected anyone to still be using zsh instead of bash. For my edification, would you explain why you need to use zsh? Thanks! (And thanks for all the investigative work you've already done.)

#9 Updated by Lynn Garren about 6 years ago

  • % Done changed from 0 to 10

There is a new download script which has the following properties:

USAGE: downloadLArSoft-v0_02_01.sh [-v] <product_dir> <e4> <debug|opt|prof>
use -v for verbose tar output

Also, it will check the version of git and use upd to install git if git is older than 1.8. If it finds a version of git that is at least 1.8, it will not attempt to install git.

http://oink.fnal.gov/distro/larsoft/downloadLArSoft-v0_02_01.sh

#10 Updated by William Seligman about 6 years ago

  • Assignee deleted (Lynn Garren)
  • % Done changed from 10 to 0

CERN uses zsh, so we use it.

#11 Updated by Lynn Garren about 6 years ago

  • Assignee set to Lynn Garren

#12 Updated by William Seligman about 6 years ago

  • Occurs In v0_02_02 added
  • Occurs In deleted (v0_02_01)

I'm sure you have more to worry about than remote installations right now, but I'd like to follow-up on these problems for when you have the time.

As previously noted, I can get everything to work in the bash shell, with one curious exception: You modified the download script to checkout git if the version 1.8 wasn't already installed. On mys SL5 systems, the installed version of git is 1.8.2; on SL6 it's 1.7.2. (I have no idea why!) So when downloadLArSoft runs, it does the upd install of git.

Side note: This did not work until I put a link: "ln -sf /usr/bin/perl /usr/local/bin/perl"; FNAL's upd script expects perl to be in /usr/local/bin for some reason.

The problem is that when using the upd git, I get a library error:

$ git branch
git: error while loading shared libraries: libcrypto.so.6: wrong ELF class: ELFCLASS32

I hunted a bit. Even though this library is in git/v1_8_0_1/Linux64bit-2/lib/libcrypto.so.6, it appears that it's the 32-bit version!

If I do "unsetup git", things seem to work.

#13 Updated by William Seligman about 6 years ago

With further tests, I found out that "unsetup git" on SL6 did not work. On SL6, the default version of git is 1.7, which does not support the --set-upstream-to option.

My workaround: I just compiled the git 1.8 src RPMs for SL6 myself and set it up the Nevis RPM repository. From this point forward, any SL6 systems at Nevis will automatically get that version of git.

(Why does SL5 have a later version of git than SL6? Answer: Redhat added git to RHEL6, so that's the version of git picked up by SL6. Redhat didn't have git in RHEL5, so the EPEL repositories included it there, and they've maintained it to the latest version in their EL5 repository, but not in their EL6 repository. Ah, the joys of distributing Redhat!)

#14 Updated by Lynn Garren about 6 years ago

  • Status changed from Assigned to Resolved

I'm going to close this issue. I understand the problem with the git in upd and have an open question to the maintainer about that. As far as I am concerned, it is to your advantage to install a modern git on your own system if you can. We will provide a more robust release of git for those without that option. I suspect this means that we have to provide the ability to build git from source, the same as any other package we require.

#15 Updated by Lynn Garren over 5 years ago

  • Status changed from Resolved to Closed

Addendum: we are now building git v1_8_5_3 from source.



Also available in: Atom PDF