Project

General

Profile

Bug #18696

Setup errors for v06_62_00 on d16

Added by David Adams almost 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Start date:
01/08/2018
Due date:
% Done:

0%

Estimated time:
Duration:

Description

I am getting lots of errors when I set up to build the head of dunetpc on my mac:

Running mrbsetenv
The working build directory is /Users/davidadams/dune/dev/dudev01/workdir/build_d16.x86_64
The source code directory is /Users/davidadams/dune/dev/dudev01/workdir/srcs
----------- check this block for errors -----------------------
INFO: no optional setup of caffe v1_0e -q +e14:+prof
ERROR: Action parsing failed on "unsetuprequired(dunepdsprce v0_0_3 -q +e14:+gen:+prof)" 
ERROR: Found no match for product 'dunepdsprce'
ERROR: Action parsing failed on "setupRequired( dunepdsprce v0_0_3 -q +e14:+gen:+prof )" 
ERROR: setup -B dune_raw_data v1_10_00 -q +nu:+e14:+s56:+prof failed
ERROR: For more information, type "ups depend dune_raw_data v1_10_00 -q +nu:+e14:+s56:+prof" 
                             or "ups list -aK+ dune_raw_data v1_10_00" 
ERROR: setup of required products has failed

History

#1 Updated by Lynn Garren almost 2 years ago

At this time, caffe is only available for SLF. If someone added it to ups/product_deps it should be made optional.

#2 Updated by David Adams almost 2 years ago

I guess I need to install the new product dunepdsprce. But when I look in SciSoft, I see two choices with avx and avx2:

http://scisoft.fnal.gov/scisoft/packages/dunepdsprce/v0_0_3/dunepdsprce-0_0_3-Darwin64bit+16-avx-e14-prof.tar.bz2
http://scisoft.fnal.gov/scisoft/packages/dunepdsprce/v0_0_3/dunepdsprce-0_0_3-Darwin64bit+16-avx2-e14-prof.tar.bz2

Which of these should I install?

#3 Updated by David Adams almost 2 years ago

I suppose dunepsdprce and caffe are two distinct issues. Or do I misunderstand? For the latter, I get an info message indicating the product was not installed. Is this the expected and intended behavior?

da

#4 Updated by David Adams almost 2 years ago

I tried making dune_raw_data optional in the dunetpc product_deps and now get these errors:

----------- check this block for errors -----------------------
INFO: no optional setup of caffe v1_0e -q +e14:+prof
ERROR: Action parsing failed on "unsetuprequired(dunepdsprce v0_0_3 -q +e14:+gen:+prof)" 
ERROR: Found no match for product 'dunepdsprce'
ERROR: Action parsing failed on "setupRequired( dunepdsprce v0_0_3 -q +e14:+gen:+prof )" 
INFO: no optional setup of dune_raw_data v1_10_00 -q +nu:+e14:+s56:+prof
----------------------------------------------------------------

It is still complaining about dunepdsprce. I thought that dependence came from dune_raw_data and should now go away.

#5 Updated by Thomas Junk almost 2 years ago

Yes, dunepdsprce is a different issue from caffe. There should be three choices: gen, avx, and avx2, for mac. avx2 uses special vector instructions that are available on Haswell processors and later. You can check the capability of your mac with

sysctl -a | grep avx

The "gen" version doesn't use these instructions and should run on any x86-64 mac. JJ Russell had wanted to write a version that just uses AVX instead of AVX2 instructions but didn't get around to it, so the "avx" version is the same as "gen" (for now).

All three versions should be in CVMFS, if you have that set up on your mac. I did, but found CVMFS to be slow on mac.

#6 Updated by Thomas Junk almost 2 years ago

There is code in the develop head of dunetpc that depends on dune_raw_data, so your builds will fail if you don't set it up. The version of dune_raw_data it depends on is v1_10_00, which should set up dunepdsprce v0_0_3. dune_raw_data may not be available in scisoft yet (I believe you requested its new directory). dunepdsprce is in scisoft. Tingjun volunteered on Friday to upload dune_raw_data v1_10_0 to scisoft, unless you already did it.

There's another version of dune_raw_data waiting in devleop -- v1_11_0, which loads dunepdsprce v0_0_4. To be included in this week's release!

The Jenkins build of dunetpc builds dune_raw_data and the tarballs should be installed in CVMFS and scisoft and /grid/fermiapp for all of these. I can take care of dunepdsprce.

#7 Updated by David Adams almost 2 years ago

I have dune_raw_data. I believe this was installed in SciSoft with our last dunetpc build if it was not already present.

I understand better why there are multiple versions but it is still not clear whether I should install both (eventually three) or only the one appropriate to my machine. And then how do ups and dunetpc select which to use?

I see the following on my machine:

mac> sysctl -a | grep avx
hw.optional.avx2_0: 1
hw.optional.avx1_0: 1

I suppose this means I have avx2.

#8 Updated by Lynn Garren almost 2 years ago

I suggest that the appropriate version of dunepdsprce and dune_raw_data be added to the dune distribution manifests.

#9 Updated by Lynn Garren almost 2 years ago

Also, if you look at the original error, you will need the "gen" qualified release of dunepdsprce.

#10 Updated by Thomas Junk almost 2 years ago

Yes, "gen" is listed in the product deps so it should be installed always. The gen and avx2 versions are binary compatible, so one can unsetup dunepdcsrpce and setup the avx2 version. But unless you're unpacking protodune raw data you won't need the speed difference anyway, so might as well just have "gen".

#11 Updated by David Adams almost 2 years ago

I see now the gen version is also present in SciSoft and suppose that is all I need. I will take that and see if it resolves the problem.

I agree that the manifest should list dune_raw_data and dunepdsprce (gen version). I hope that some magic that went into the dunetpc build scripts will handle that automatically. If not, I may be asking for help when we make our next build later this week.

#12 Updated by David Adams almost 2 years ago

Installing the gen version of dunepdsprce in my UPS area does seem to have resolved the problem:

Running mrbsetenv
The working build directory is /Users/davidadams/dune/dev/dudev01/workdir/build_d16.x86_64
The source code directory is /Users/davidadams/dune/dev/dudev01/workdir/srcs
----------- check this block for errors -----------------------
INFO: no optional setup of caffe v1_0e -q +e14:+prof
----------------------------------------------------------------

I will wait and make sure the build succeeds before closing this ticket.

#13 Updated by David Adams almost 2 years ago

  • Status changed from New to Closed

All is fine. Apparently the INFO message is intended.



Also available in: Atom PDF