Support #20563
build a product stack with hep_hpc
Description
NOvA would like a product stack that includes hep_hpc. The version of hdf5 needs to match the version used by caffe.
History
#1 Updated by Lynn Garren almost 2 years ago
- Status changed from New to Assigned
- Assignee set to Lynn Garren
Per Chris Green, we need hep_hpc v0_08_07. We probably also want to include the hep_hpc_toolkit bundle so that h5py is available.
#2 Updated by Lynn Garren almost 2 years ago
Also, check and make sure this bundle includes the latest castxml.
#3 Updated by Lynn Garren almost 2 years ago
caffe v1_0k has been made available. It is built with hdf5 v1_10_2a, protobuf v3_5_2, and lapack v3_8_0a.
#4 Updated by Lynn Garren almost 2 years ago
Issues with NOvA code being unable to activate compression on HDF5 output datasets were identified, and Brandon has implemented the correct scheme in his code. Steven will propagate those changes to the modern code generation system within NOvA.
It was also noticed that NOvA is not activating the recommended HDF5 error handling mode. This should be done once per job:
#include "hep_hpc/hdf5/errorHandling.hpp"
...
ErrorController::setErrorHandler(ErrorMode::EXCEPTION);
Errors were identified in hep_hpc which had contributed to the difficulty of tracing the issue:
- Faulty logic in an assertion in hep_hpc::hdf5::setColumnProperties(), called by make_column().
- Logic in hep_hpc::hdf5::PropertyList::isClass(), which returned true if the underlying call to H5Pequal() failed and the recommended error handling setting (see above) was not in place. It now returns false in this case.
Armor was applied to hep_hpc/Utilities/detail/compiler_macros.hpp to mitigate compiler notes in compilation units which also include (usually indirectly) cetlib/compiler_macros.h. Similar armor has been applied to cetlib, but as a workaround one should ensure that hep_hpc headers are included after any art framework headers.
There are new versions of hep_hpc (v0_08_09) and hep_hpc_toolkit (v00_08_09). There have been no changes to other dependencies.
hep_hpc_toolkit v00_08_09 is now available on SciSoft
#5 Updated by Lynn Garren almost 2 years ago
- Status changed from Assigned to Resolved
- % Done changed from 0 to 100
The nu v2_21_01b distribution is now available for e15 on SLF6 and SLF7. Let us know if you need other builds as well.
This distribution includes caffe v1_0k, hep_hpc_toolkit v00_08_09, castxml v0_00_00_f20180122, nutools v2_22_01, and geant4 v4_10_3_p01d.
#6 Updated by Lynn Garren over 1 year ago
- Status changed from Resolved to Closed