Project

General

Profile

Necessary Maintenance #19539

Restore tensorflow-dependent code disabled for support to Clang

Added by Gianluca Petrillo over 1 year ago. Updated over 1 year ago.

Status:
Assigned
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
04/02/2018
Due date:
% Done:

0%

Estimated time:
Experiment:
-
Duration:

Description

The code using tensorflow library in larsoft has been disabled, as documented in the notes of this ticket.
After that library is successfully integrated in all the supported platforms, this disabled code needs to be reenabled.

Associated revisions

Revision 90ee5325 (diff)
Added by Gianluca Petrillo over 1 year ago

Disabled tensorflow-dependent code as in issue #19539

History

#1 Updated by Gianluca Petrillo over 1 year ago

The following code has been disabled from LArReco only when tensorflow is not set up:

#2 Updated by Lynn Garren over 1 year ago

  • Status changed from New to Accepted

The tensorflow interface was meant to be modular. Everything that depends on it should be put into a single place where it can be disabled easily.

#3 Updated by Lynn Garren over 1 year ago

  • Status changed from Accepted to Feedback
  • Assignee set to Christoph Alt

Christoph, is there someone from DUNE who is responsible for the Tensorflow interface?

#4 Updated by Christoph Alt over 1 year ago

Hi,

I just had a look at this together with Leigh.

Assuming that the library larreco_RecoAlg_ImagePatternAlgs_PointIdAlg is not of much use without the source code larreco/RecoAlg/ImagePatternAlgs/PointIdAlg/PointIdAlg.cxx, we could do the following:

1. Create a new folder larreco/RecoAlg/ImagePatternAlgs/Tensorflow/

2. Move folders (libraries) larreco/RecoAlg/ImagePatternAlgs/TF/ and larreco/RecoAlg/ImagePatternAlgs/PointIdAlg/ to this folder.

3. Move the five modules depending on the PointIdAlg library to larreco/RecoAlg/ImagePatternAlgs/Tensorflow/Modules/

Tensorflow could be disabled by removing this new folder from the CMakeList. Does this solve the problem?

Only one module in dunetpc depends on larreco_RecoAlg_ImagePatternAlgs_PointIdAlg: https://cdcvs.fnal.gov/redmine/projects/dunetpc/repository/revisions/develop/entry/dune/CVN/adcutils/CMakeLists.txt

Does code from other experiments depend on either larreco_RecoAlg_ImagePatternAlgs_PointIdAlg or larreco_RecoAlg_ImagePatternAlgs_TF ?

#5 Updated by Lynn Garren over 1 year ago

Thank you! This is exactly the sort of solution we hoped for. I see that lariatsoft apparently has their own copy of PointIdAlg??? lariatsoft also links against larreco_RecoAlg_ImagePatternAlgs_PointIdAlg. Is this something you will be able to investigate?

Please use this structure in the CMakeLists.txt file:

if( DEFINED ENV{TENSORFLOW_DIR} )
  add_subdirectory(xxx)
endif ()

#6 Updated by Leigh Whitehead over 1 year ago

For LArIAT, I think the work was done by Daniel Smith in collaboration with Robert. I have added him to the watch list now.

#7 Updated by Christoph Alt over 1 year ago

Okay, I will make a feature branch for larreco and dunetpc with the changes described above.

@Daniel: once this is done, could you update the linker for larreco_RecoAlg_ImagePatternAlgs_PointIdAlg in LArIAT? I'll let you know what the new linker looks like as soon as I figured this out.

#8 Updated by Daniel Smith over 1 year ago

Hello all. I'd be happy to make any changes for links / CMake files in LArIAT. LArIAT doesn't use its own copy of PointIDAlg, it uses the version in LArReco. There have been discussions of getting our own copy for the sake of simplicity, but, for the time being, we would need to update to match the changes made in LArReco.

#9 Updated by Christoph Alt over 1 year ago

Here is the feature branch for larreco: https://cdcvs.fnal.gov/redmine/projects/larreco/repository?utf8=%E2%9C%93&rev=feature%2Fchalt_Tensorflow

All tensorflow code is now in the folder "Tensorflow", which can disabled by the following lines in larreco/RecoAlg/ImagePatternAlgs/CMakeLists.txt:

if( DEFINED ENV{TENSORFLOW_DIR} )
  add_subdirectory(Tensorflow)
endif ()

@Daniel: In dunetpc,

- I disabled the folder dune/CVN/adcutils/ in the same way since it contains a module depending on the PointIdAlg library.
- I updated the PointIdAlg library linker inside this folder. New linker: larreco_RecoAlg_ImagePatternAlgs_Tensorflow_PointIdAlg
- Also, modules depending on this library have to update the include of PointIdAlg.h as follows: #include "larreco/RecoAlg/ImagePatternAlgs/Tensorflow/PointIdAlg/PointIdAlg.h"

I guess you need to do the same for LArIAT.

feature branch for dunetpc: https://cdcvs.fnal.gov/redmine/projects/dunetpc/repository?utf8=%E2%9C%93&rev=feature%2Fchalt_Tensorflow

I hope this does the trick.

Christoph

#10 Updated by Lynn Garren over 1 year ago

  • Status changed from Feedback to Assigned

Unfortunately, this is not yet ready to integrate into a larsoft release. The problem is that pointidalg.fcl is used by larreco/TrackFinder/trackfinderalgorithms.fcl. trackfinderalgorithms.fcl is, as expected, used in a number of places.

However, there is now a lariatsoft feature/team_issue19539 branch.

#11 Updated by Christoph Alt over 1 year ago

Oops!

I did split up the pointidalg.fcl (and created dataprovider.fcl outside of the Tensorflow directory), but I forgot to change the include in trackfinderalgorithms.fcl. I did this now and pushed the changes to my feature branch.



Also available in: Atom PDF