Old LArSoftWiki » History » Version 57
Erica Snider, 12/08/2013 07:26 PM
1 | 54 | Erica Snider | If you are looking for the legacy cvs-based LArSoft site that was previously hosted on this site, note that *all content has been moved to "LArSoft cvs (legacy site) ":https://cdcvs.fnal.gov/redmine/projects/larsoft-cvs/wiki.* |
---|---|---|---|
2 | 54 | Erica Snider | If you are looking for the legacy svn-based LArSoft site, "go here":https://cdcvs.fnal.gov/redmine/projects/larsoftsvn/wiki. |
3 | 1 | Brian Rebel | --------------------------------------------------------------- |
4 | 1 | Brian Rebel | |
5 | 54 | Erica Snider | {{>toc}} |
6 | 1 | Brian Rebel | |
7 | 1 | Brian Rebel | This is the beta LArSoft redmine project and the future home of the LArSoft redmine project. |
8 | 29 | Erica Snider | |
9 | 54 | Erica Snider | h1. LArSoftWiki (beta) |
10 | 1 | Brian Rebel | |
11 | 54 | Erica Snider | *This page is in beta and content is still under construction.* Will go live when the migration to git/cmake is completed. |
12 | 31 | Erica Snider | |
13 | 1 | Brian Rebel | |
14 | 54 | Erica Snider | The LArSoft software is designed to work for all planned and running liquid argon experiments at Fermilab. It is written in C++ and built on the "ROOT":http://root.cern.ch data analysis software, and the "art":https://cdcvs.fnal.gov/projects/art/wiki analysis framework supported by the Fermilab Scientific Computing Division for intensity frontier experiments. |
15 | 1 | Brian Rebel | |
16 | 1 | Brian Rebel | To join the LArSoft mailing list, please follow these "instructions":http://listserv.fnal.gov/users.asp#subscribe%20to%20list using the list name LARSOFT. |
17 | 1 | Brian Rebel | |
18 | 1 | Brian Rebel | |
19 | 54 | Erica Snider | h1. Preliminaries |
20 | 35 | Erica Snider | |
21 | 54 | Erica Snider | * Fermilab computing accounts: see the instructions on the [[larsoftsvn:Fermilab Computing Access]] page if you need an account. |
22 | 54 | Erica Snider | * See [[larsoftsvn:Load Balanced Access to General Purpose VMs]] for information on accessing experiment VMs at Fermilab (e.g., lbnegpvm0X, uboonegpvm0X, etc., collectively known as GPCF.). |
23 | 54 | Erica Snider | * See the [[LArSoftWiki#documentation|documentation]] section for information on working with/developing LArSoft code, and using git, git flow, mrb and other tools you will need. |
24 | 54 | Erica Snider | * The procedures and tools for building and developing software with this system are _completely different_ from those of the previous system based on svn repositories and the SoftRelTools build system. See the [[LArSoftWiki#documentation|documentation]] section for details. |
25 | 1 | Brian Rebel | |
26 | 35 | Erica Snider | h2. Where to find the software |
27 | 1 | Brian Rebel | |
28 | 54 | Erica Snider | In a standard LArSoft installation, the software is accessed via a set of re-locatable ups products, each of which corresponds to the code within a git repository. Each product and the associated repository contain LArSoft software components (i.e., SoftRelTool "packages") that are within the same layer of functionality. The products and repository read-write urls are the following (read-only access is available via http://cdcvs.fnal.gov/projects/<repo_name>): |
29 | 35 | Erica Snider | |
30 | 35 | Erica Snider | |*Product |
31 | 35 | Erica Snider | repository |
32 | 35 | Erica Snider | name* | *repository url (all in Redmine)* | lxr link |
33 | 35 | Erica Snider | (not yet avail) | Redmine browser | |
34 | 49 | Erica Snider | |larcore| ssh://p-larcore@cdcvs.fnal.gov/cvs/projects/larcore | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larcore/repository | |
35 | 49 | Erica Snider | |lardata| ssh://p-lardata@cdcvs.fnal.gov/cvs/projects/lardata | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/lardata/repository | |
36 | 49 | Erica Snider | |larevt | ssh://p-larevt@cdcvs.fnal.gov/cvs/projects/larevt | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larevt/repository | |
37 | 49 | Erica Snider | |larsim | ssh://p-larsim@cdcvs.fnal.gov/cvs/projects/larsim | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larsim/repository | |
38 | 1 | Brian Rebel | |larreco| ssh://p-larreco@cdcvs.fnal.gov/cvs/projects/larreco | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larreco/repository | |
39 | 49 | Erica Snider | |larana | ssh://p-larana@cdcvs.fnal.gov/cvs/projects/larana | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larana/repository | |
40 | 49 | Erica Snider | |lareventdisplay| ssh://p-lareventdisplay@cdcvs.fnal.gov/cvs/projects/lareventdisplay | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/lareventdisplay/repository | |
41 | 1 | Brian Rebel | |larexamples| ssh://p-larexamples@cdcvs.fnal.gov/cvs/projects/larexamples | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/larexamples/repository | |
42 | 49 | Erica Snider | |larpandora| ssh://p-larpandora@cdcvs.fnal.gov/cvs/projects/larpandora | -- | "redmine:https://cdcvs.fnal.gov/redmine/projects/larpandora/repository | |
43 | 49 | Erica Snider | |larsoft | ssh://p-larsoft@cdcvs.fnal.gov/cvs/projects/larsoft |
44 | 49 | Erica Snider | (A product used to set up the others with a single command. Contains the "setup" package.) | -- | |
45 | 49 | Erica Snider | "redmine":https://cdcvs.fnal.gov/redmine/projects/larsoft/repository | |
46 | 1 | Brian Rebel | |
47 | 36 | Erica Snider | (The SoftRelTools-based packages in each product/repository can be found "here":https://cdcvs.fnal.gov/redmine/projects/larsoft-alpha/wiki/LarSoft_Product_List along with the approximate order of dependency.) |
48 | 1 | Brian Rebel | |
49 | 55 | Erica Snider | Although you can use @git clone <repository url>@ to download a copy of each repository to your local area, additional steps are needed to use, build or develop the software. See the [[LArSoftWiki#documentation|documentation]] section below for details. |
50 | 35 | Erica Snider | |
51 | 50 | Erica Snider | h3. Re-factoring of experiment-specific software |
52 | 50 | Erica Snider | |
53 | 55 | Erica Snider | Most experiment-specific code, fcl and gdml files have been removed from the @larcore@, @lardata@, @larevt@ and @larsim@ repositories and moved into beta versions of experiment-specific git repositories. (One exception to this statement is that the @DetId_t@ enum type, which effectively maps detector name strings to integers, remains in core LArSoft, pending migration to a new, string-based mechanism for storing and using this information. Other exceptions include fcl files that were not identified during the initial migration.) The beta versions of experiment-specific repositories are the following: |
54 | 50 | Erica Snider | |
55 | 50 | Erica Snider | |*Product |
56 | 50 | Erica Snider | repository |
57 | 50 | Erica Snider | name* | *repository url (all in Redmine)* | lxr link |
58 | 50 | Erica Snider | (not yet avail) | Redmine browser | |
59 | 50 | Erica Snider | |lbnecode| ssh://p-lbnecode@cdcvs.fnal.gov/cvs/projects/lbnecode | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/lbnecode/repository | |
60 | 50 | Erica Snider | |uboonecode| ssh://p-uboonecode@cdcvs.fnal.gov/cvs/projects/lardata | -- | "redmine":https://cdcvs.fnal.gov/redmine/projects/uboonecode/repository | |
61 | 50 | Erica Snider | |
62 | 50 | Erica Snider | |
63 | 50 | Erica Snider | |
64 | 35 | Erica Snider | |
65 | 1 | Brian Rebel | |
66 | 45 | Erica Snider | h1. Release notes |
67 | 1 | Brian Rebel | |
68 | 1 | Brian Rebel | | *Release* | *Date* | *Purpose* | *Changes / notes* | *Full release notes* | |
69 | 42 | Erica Snider | | v1.00.00| Jan 2014 | First production |
70 | 42 | Erica Snider | release | Replica of final svn-based release. Future LArSoft development |
71 | 42 | Erica Snider | proceeds from this release. | xxx | |
72 | 55 | Erica Snider | | v0.0x.yy| 12/10/2013| "beta" public release | Beta suitable for general user testing | N/A | |
73 | 52 | Erica Snider | | v0.01.01| 12/02/2013| "beta" limited release | Beta suitable for expert testing | N/A | |
74 | 35 | Erica Snider | | v0.00.09| 11/25/2013| "beta" pre-release | Second full release under new system. First full re-factoring |
75 | 35 | Erica Snider | of experiment-specific and core LArSoft code in the larcore, |
76 | 35 | Erica Snider | lardata, larevt, and larsim products. Preparation for expert |
77 | 35 | Erica Snider | user testing of beta release.| N/A| |
78 | 42 | Erica Snider | | v0.00.04| 9/15/2013| "alpha" release | First release of git/cmake/ups-based LArSoft products |
79 | 42 | Erica Snider | Used for mrb, configuration and re-factoring |
80 | 42 | Erica Snider | development and testing | N/A | |
81 | 33 | Erica Snider | |
82 | 1 | Brian Rebel | h1. Documentation |
83 | 32 | Erica Snider | |
84 | 48 | Erica Snider | * [[ LArSoft ]] |
85 | 1 | Brian Rebel | |
86 | 57 | Erica Snider | * [[Quick-start guide to using and develop LArSoft code]] |
87 | 1 | Brian Rebel | |
88 | 1 | Brian Rebel | * [[ Detailed documentation on using and developing LArSoft code ]] |
89 | 52 | Erica Snider | |
90 | 53 | Erica Snider | * *[[ Special instructions for beta testers ]]* |
91 | 1 | Brian Rebel | |
92 | 43 | Erica Snider | |
93 | 44 | Erica Snider | h2. Overview of the user and developer environment |
94 | 43 | Erica Snider | |
95 | 51 | Erica Snider | LArSoft releases are distributed via "re-locatable" ups products. People interested in using the core LArSoft software, but who have no need to modify or develop that software can in principle simply perform the appropriate ups @setup <product> <version> -q <qualifier>@ commands, then build their code against those products by reference to the corresponding $<PRODUCT_NAME>_INC and $<PRODUCT_NAME>_LIB environment variables. In addition to the individual products, there is a "@larsoft@" product that can be used to set up all other products with a single command: |
96 | 36 | Erica Snider | <pre> |
97 | 40 | Erica Snider | setup larsoft <version> -q <qualifiers> |
98 | 36 | Erica Snider | </pre> |
99 | 36 | Erica Snider | The versions and qualifiers available can be obtained by using the following command: |
100 | 36 | Erica Snider | <pre> |
101 | 36 | Erica Snider | ups list -aK+ larsoft |
102 | 1 | Brian Rebel | </pre> |
103 | 43 | Erica Snider | The qualifiers will be one of the following: "debug:e2", "prof:e2", "opt:e2", where: |
104 | 1 | Brian Rebel | |
105 | 36 | Erica Snider | * debug = debugging symbols available |
106 | 43 | Erica Snider | * opt = compiler optimizations enabled, no debug symbols |
107 | 36 | Erica Snider | * prof = compiler optimizations enabled, profiling code generated |
108 | 43 | Erica Snider | * e2 = built with gcc 4.7.1 and -std=c++0x |
109 | 36 | Erica Snider | |
110 | 43 | Erica Snider | In general, only @debug@ and @prof@ versions will be provided, since there is no run-time performance penalty for code that has the profiling code present. |
111 | 41 | Erica Snider | |
112 | 43 | Erica Snider | The recommended way to work with LArSoft is to use *@mrb@*, the multi-repository build tool, to check out and build your own code. This build system is based upon *@cmake@* and a toolkit of @cmake@ macros in the *@cetbuildtools@* product -- the same set of tools that are used to build the @art@ framework that underpins LArSoft. @mrb@ operates above @cmake@ and drives the build procedure across (possibly) multiple repositories within one's working area. Using @mrb@ ensures the integrity and structure of the working. |
113 | 38 | Erica Snider | |
114 | 1 | Brian Rebel | One's working directory within this system has the following structure: a source code sub-directory where all development takes place; a build sub-directory where all build activities take place; and a local products area, where all successfully built software is installed, and from which it can be run. |
115 | 43 | Erica Snider | |
116 | 51 | Erica Snider | All software packages built and installed by @mrb@ / @cmake@ / @cetbuildtools@ are in the form of a re-locatable ups products. (A "re-locatable" ups product has a simplified structure compared to that of legacy ups products, and does not require that it be "declared" to a ups database. These features simplify distribution, installation and maintenance of re-locatable products.) @mrb@ provides a simple product template that includes two files that must be modified by the user: a top-level @CMakeLists.txt@ file, that specifies which sub-directories will be included in the build along with any global configuration needed to build the product; and a @product_deps@ file that specifies the dependencies for the package. The product template can otherwise be customized for an experiment. Although this scheme may sound cumbersome and arcane, it is essentially no different than following the package structure conventions imposed by SRT, using a GNUmakefile to configure the build, and a global release setup script to configure the global environment. With @cmake@, the GNUmakefiles are replaced by @CMakeLists.txt@ files, while the "global environment" for the product is managed locally with the @product_deps@ and top-level CMakeLists.txt files for the product. @mrb@ then manages all the details of utilizing ups to configure the build environment, driving the build, and packaging and installing the ups product in the local products area. |
117 | 1 | Brian Rebel | |
118 | 43 | Erica Snider | |
119 | 45 | Erica Snider | h3. The LArSoft developer environment |
120 | 41 | Erica Snider | |
121 | 46 | Erica Snider | As previously mentioned, all LArSoft code is archived in a set of repositories based on the "git version control system":http://git-scm.com/. There are numerous resources on the web on how to use git, starting with the authoritative "Pro Git Book":http://git-scm.com/book and "git reference manual":http://git-scm.com/docs provided on the official git website. A search on "git documentation" yields many more. |
122 | 1 | Brian Rebel | |
123 | 46 | Erica Snider | The LArSoft project has adopted the git branching model described at "nvie.com":http://nvie.com/posts/a-successful-git-branching-model to assist with managing the development workflow and maintaining a stable development environment. Within this framework, the git repositories have the following branch structure: |
124 | 41 | Erica Snider | * A "main" branch that will have only tagged releases. Used only by the software manager. |
125 | 41 | Erica Snider | * A "develop" branch that will have the working head of the repository. Used by all developers. |
126 | 45 | Erica Snider | * One or more "release" branches for the integration of specific tagged releases. Used or authorized only by the software manager. |
127 | 45 | Erica Snider | * An arbitrary set of "feature" branches on which all on-going work takes place. In most cases, these branches will be in local repositories, although publishing them to the reference repository is useful in many cases. Developers can create as many feature branches as needed. |
128 | 45 | Erica Snider | * One or more "hotfix" branches that is used to develop patches to tagged releases. Used or authorized only by the software manager. |
129 | 1 | Brian Rebel | |
130 | 45 | Erica Snider | The "git flow":https://github.com/nvie/gitflow tool is a set of git extension that provide high-level operations for working within this branching model. We recommend that developers utilize @git flow@ when developing LArSoft code. Developers who choose not to use @git flow@ should nonetheless adhere to the branching model. |
131 | 41 | Erica Snider | |
132 | 41 | Erica Snider | |
133 | 41 | Erica Snider | |
134 | 41 | Erica Snider | |
135 | 41 | Erica Snider | |
136 | 45 | Erica Snider | h2. Links to the tools used in working with the software |
137 | 32 | Erica Snider | |
138 | 32 | Erica Snider | * git |
139 | 32 | Erica Snider | * git flow |
140 | 32 | Erica Snider | * mrb : the multi-repository build tool |
141 | 32 | Erica Snider | * Re-locatable ups |
142 | 32 | Erica Snider | * cmake |
143 | 33 | Erica Snider | |
144 | 33 | Erica Snider | |
145 | 45 | Erica Snider | h1. How-to's |
146 | 33 | Erica Snider | |
147 | 33 | Erica Snider | |
148 | 33 | Erica Snider | h2. Advanced technical how-to's |
149 | 33 | Erica Snider | |
150 | 33 | Erica Snider | h2. Release procedures |
151 | 33 | Erica Snider | |
152 | 1 | Brian Rebel | h1. Walk-through exercises |
153 | 33 | Erica Snider | |
154 | 51 | Erica Snider | h1. LArSoft code management and distribution procedures |
155 | 34 | Erica Snider | |
156 | 34 | Erica Snider | h1. Working areas |
157 | 34 | Erica Snider | |
158 | 34 | Erica Snider | [[Beta re-factoring]] |