We talked through a plan to integrate Mike Wang's performance improvements back into the mu2e tracking code and make the code available for further performance studies.
Chris Green, Rob Kutschke, Mike Wang, and Jim K were present.
- Back-port exploratory work (organization, structure)
- Address maintenance issue (updates from Mike and Dave to code copies)
- Continued explorations (Xeon Phi, Multi-core)
- Further work (pattern recognition and fitting, clustering)
- Start with one module - the "make stereo hits" (MSH here)
- use fork / join model for development with two repositories
- some refactoring will need to be done in parts of mu2e code
- pragmas for parallel processing will exist in mu2e
- use TBB as much as possible
- have overall review in the fall - may be able to combine into one repository then
- rdtrack repository is available in redmine (git) for the tracking exploratory work
- art-hough model of working with TBB and offload is working and is good
- art module for MSH exists (issues with build, some offload going with TBB and OpenMP, separation might not be clean enough between mu2e and new repos)
- old detector models on still in the current mu2e code and need to be cleaned up eventually
- build issues resolved (minimized local mu2e code)
- ready for review (includes code walk-through of MSH, candidate for inclusion in main mu2e repo)
- include reorganization suggestions for data structures, geometry interface, and other utility functions
Brief summary of our discussion: The reason for some of this complexity is that icc is not yet ready to handle the c++11 code that we have in art and a new other packages. Many of the externals (ROOT, etc.) compile with it, but it cannot do everything. The intel compiler allows for compatibility with gcc and we have it hooked up to gcc 4.8.x. The current plan is to use mu2e in its current state as an external product, along with art and the externals. There is a special configuration of TBB that allows code to be used on the Xeon Phi (using icc) and on a standard multi-core processor (using gcc). The exploratory repository discussed above will contain the MSH module built using icc and the cetbuildtools. It will access code in mu2e and the module can be used directly in mu2e fhicl scripts. When results are shown to be the same as the existing MSH code and the code is looked at, it will be merged into the mu2e repository.
Note that the code that is offloaded cannot access the gcc-compiled libraries directly. The algorithms running in this mode will need to have access to information copied out of the geometry and conditions services.
Key to success with forked repo model: quick turn-around on repo fork / join of MSH in mu2e main and exploratory areas.
See attached whiteboard diagram.