Project

General

Profile

Product Mixing » History » Version 4

Christopher Green, 10/26/2011 11:19 PM

1 1 Christopher Green
h1. Product Mixing
2 1 Christopher Green
3 1 Christopher Green
h2. Overview.
4 1 Christopher Green
5 1 Christopher Green
*Product Mixing* is our label for the ability to take a feed from a source of secondary events, and combine the products (usually collections) from multiple events into a single product or collection to be placed in the primary event. The motivation is of course obvious: background (eg cosmic) events or pile-up.
6 1 Christopher Green
7 1 Christopher Green
This feature is implemented by providing a module template, @MixFilter@ as a skeleton to handle the low-level interaction with the primary event and secondary files, including the collection of multiple products from different secondary events and the registration of outgoing products. The user must provide a "detail" class implementation which has some mandatory and some optional members in order to register necessary products at the right time and mix the prepared groups of incoming products. The @MixFilter@ will take care of reading the secondary data, either sequentially or randomly and cycling through multiple secondary input files as necessary.
8 1 Christopher Green
9 1 Christopher Green
In order to make the mixing task easier, we have provided several auxiliary utilities which will be described later.
10 1 Christopher Green
11 1 Christopher Green
In summary, the MixFilter module will do the following tasks:
12 1 Christopher Green
13 1 Christopher Green
# Construct a @MixHelper@ object for use by the user's detail class, hereafter referred to as @Detail@ (class) or @detail@ (object).
14 1 Christopher Green
# Construct @detail@, passing the helper and the module's parameter set. At this point, @detail@ should register any mix operations (one per product to be mixed) using @MixHelper::declareMixOp<>()@, and any non-mixing products to go in to the event (e.g. bookkeeping objects) using @MixHelper::produces<>()@.
15 1 Christopher Green
# For each primary event:
16 1 Christopher Green
## Call the optional @Detail::startEvent()@ function if it exists so @detail@ can position itself for the coming event.
17 1 Christopher Green
## Call @Detail::nSecondaries()@ to decide how many secondary events to read.
18 1 Christopher Green
## Decide which secondary events are to be read.
19 1 Christopher Green
## Call the optional @Detail::processEventIDs()@ function in case the Detail class is interested in the sequence of event IDs (e.g. for bookkeeping purposes).
20 1 Christopher Green
## Call the registered mixing operation for each product to be mixed, providing the secondary data to be mixed and putting the resultant mixed product into the primary event.
21 1 Christopher Green
## Call the optional @Detail::finalizeEvent()@ function to tidy up and possibly put any bookkeeping products (registered with @MixHelper::produces<>()@) into the primary event.
22 1 Christopher Green
23 1 Christopher Green
The @MixFilter@ module will take care of opening new secondary files as necessary, and will additionally call an optional function @Detail::eventsToSkip()@ as each file is opened (in sequential mode only) if it is desired to have a particular offset into each file to insure against coherent backgrounds in events.
24 1 Christopher Green
25 2 Christopher Green
h2. Mix operations: what are they and what do they need to do?
26 2 Christopher Green
27 3 Christopher Green
A mix operation is a function provided by the user which returns a @bool@ and expects three arguments:
28 3 Christopher Green
29 3 Christopher Green
* @vector<PROD const *> const &@ containing the secondary products to be mixed;
30 3 Christopher Green
* @PROD &@, the output product to be filled by the mix function; and
31 3 Christopher Green
* @PtrRemapper const &@, a helper for use remapping @Ptr@ and friends (see below).
32 3 Christopher Green
33 3 Christopher Green
It may be a free function, a member function of @Detail@ or some other user class, or a function object ("functor") whose @operator()@ has the correct signature. It is supposed to use the information contained in the secondary products to fill the output product, using the @PtrRemapper@ if necessary, and return a @bool@ indicating whether the output product should be placed in the event.
34 3 Christopher Green
35 3 Christopher Green
All mix operations must be declared _at construction time_ by @detail@, using the @MixHelper@ provided as a constructor argument.
36 3 Christopher Green
37 1 Christopher Green
h2. Mandatory interface for @Detail@.
38 1 Christopher Green
39 3 Christopher Green
h3. Constructor.
40 3 Christopher Green
41 4 Christopher Green
<pre><code class="cplusplus">Detail(fhicl::ParameterSet const & ps, art::MixHelper & helper);</code></pre>The constructor should use the provided parameter set to extract any configuration; and the helper to declare mix operations and any bookkeeping products with @declareMixOp<>()@ and @produces<>();
42 3 Christopher Green
43 3 Christopher Green
h3. @nSecondaries()@.
44 3 Christopher Green
45 4 Christopher Green
<pre><code class="cplusplus">size_t nSecondaries();</code></pre>Tell @MixFilter@ how many secondaries to read for merging into this primary. If required, a random number engine may be obtained from the @RandomNumberGenerator@ service.
46 3 Christopher Green
47 1 Christopher Green
h2. Optional interface for @Detail@.
48 3 Christopher Green
49 3 Christopher Green
h3. @startEvent()@.
50 1 Christopher Green
51 4 Christopher Green
<pre><code class="cplusplus">void startEvent(art::Event const & evt);</code></pre>Do any per-event initialization of @detail@ here.
52 1 Christopher Green
53 4 Christopher Green
54 1 Christopher Green
h3. @eventsToSkip()@.
55 3 Christopher Green
56 4 Christopher Green
<pre><code class="cplusplus">size_t eventsToSkip();</code></pre>If provided and the @MixFilter@ is in sequential secondary mode, this function will be called at the beginning of every file to ascertain how many events to skip before pulling products to merge.
57 3 Christopher Green
58 4 Christopher Green
59 3 Christopher Green
h3. @processEventIDs()@.
60 1 Christopher Green
61 4 Christopher Green
<pre><code class="cplusplus">void processEventIDs(art::EventIDSequence const & seq);</code></pre>@seq@ is an ordered sequence of @art::EventID@. Use as you will.
62 1 Christopher Green
63 4 Christopher Green
64 3 Christopher Green
h3. @finalizeEvent()@.
65 3 Christopher Green
66 4 Christopher Green
<pre><code class="cplusplus">void finalizeEvent(art::Event & evt);</code></pre>Write any non-merge products (i.e. those you declared with @MixHelper::produces<>()@) to the event with @evt.put<>()@.
67 1 Christopher Green
68 4 Christopher Green
69 1 Christopher Green
h2. Helper utilities.
70 1 Christopher Green
71 1 Christopher Green
h3. @MixHelper@.
72 4 Christopher Green
73 4 Christopher Green
h4. @produces<>()@.
74 4 Christopher Green
75 4 Christopher Green
Analogous to producer or filter @produces<>()@ calls: declare products to be put into the event which are _not_ mix products.<pre><code class="cplusplus">template <class PROD>
76 4 Christopher Green
void produces(std::string const & instanceName = std::string());
77 4 Christopher Green
78 4 Christopher Green
template <class PROD, BranchType B>
79 4 Christopher Green
void produces(std::string const & instanceName = std::string());</code></pre>
80 4 Christopher Green
81 4 Christopher Green
h4. @DeclareMixOp<>()@.
82 4 Christopher Green
83 4 Christopher Green
These function templates should be used to declare a mix operation.
84 4 Christopher Green
85 4 Christopher Green
Such an operation may be specified by providing:
86 4 Christopher Green
87 4 Christopher Green
# an @InputTag@ specifying which secondary products should be mixed;
88 4 Christopher Green
# an optional instance label for the mixed product (defaulting to the instance label of the incoming product if unspecified); and
89 4 Christopher Green
# a callable mixer such as:<pre><code class="cplusplus">bool mixfunc(std::vector<PROD const *> const &,
90 4 Christopher Green
              PROD &,
91 4 Christopher Green
              PtrRemapper const &);</code></pre>As the user may prefer, the mixer may take the form of:
92 4 Christopher Green
** an arbitrarily-named free function, or
93 4 Christopher Green
** a function object whose @operator()@ must have the correct type (see @mixfunc@ above), or
94 4 Christopher Green
** an arbitrarily-named member function of any class. In each case, the mixer must have the same type (i.e., the same return type and the same parameter types) illustrated by @mixfunc@ above. The return value of the mix function is taken to indicate whether the product should be placed in the event.
95 4 Christopher Green
   For free functions, function objects, and pre-bound member functions, the product type template argument need not be specified as it can be deduced from the signature of the provided function.
96 4 Christopher Green
   If the provided function is a member function it may be provided bound to the object upon which it is to be called by the user (in which case it is treated as a free function by the registration method) or by specifying the member function followed by the object to which it should be bound (in which case the bind will be done for the user). In this latter case the template argument specifying the product type need *not* be specified usually as it may be deduced from the signature of the provided function. If one specifies an overload set however (e.g. in the case where a class has several @mix()@ member functions, each with arguments indicating a different product) then the template argument must be specified in order to constrain the overload set to a single function.
97 4 Christopher Green
# An optional boolean, "outputProduct," defaulting to, "true." A false value for this parameter indicates that the mix product will *never* be put into the event and should therefore not be declared. If the mix operation so declared ever returns true an exception will be thrown.
98 4 Christopher Green
99 4 Christopher Green
@declareMixOp<>()@ may be called with any of the following argument combinations:
100 4 Christopher Green
101 4 Christopher Green
# Provide an @InputTag@ and a mixer that is a free function or function object (or pre-bound member function).
102 4 Christopher Green
# Provide an @InputTag@, an output instance label, and a mixer that is a free function or function object (or pre-bound member function).
103 4 Christopher Green
# Provide an @InputTag@, a mixer that is a non-@const@ member function (of any class), and an object to which that member function should be bound.
104 4 Christopher Green
# Provide an @InputTag@, an output instance label, a mixer that is a non-@const@ member function (of any class), and an object to which that member function should be bound.
105 4 Christopher Green
# As 3, but with a @const@ member function.
106 4 Christopher Green
# As 4, but with a @const@ member function.
107 4 Christopher Green
108 4 Christopher Green
The full set of templates are listed below for completeness, numbered per the list above:<pre><code class="cplusplus">// 1.
109 4 Christopher Green
template <typename PROD>
110 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
111 4 Christopher Green
                  std::function < bool (std::vector<PROD const *> const &,
112 4 Christopher Green
                                        PROD &,
113 4 Christopher Green
                                        PtrRemapper const &)
114 4 Christopher Green
                  > mixFunc,
115 4 Christopher Green
                  bool outputProduct = true);
116 4 Christopher Green
117 4 Christopher Green
// 2.
118 4 Christopher Green
template <typename PROD>
119 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
120 4 Christopher Green
                  std::string const & outputInstanceLabel,
121 4 Christopher Green
                  std::function < bool (std::vector<PROD const *> const &,
122 4 Christopher Green
                                        PROD &,
123 4 Christopher Green
                                        PtrRemapper const &)
124 4 Christopher Green
                  > mixFunc,
125 4 Christopher Green
                  bool outputProduct = true);
126 4 Christopher Green
127 4 Christopher Green
// 3.
128 4 Christopher Green
template <typename PROD, typename T>
129 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
130 4 Christopher Green
                  bool (T::*mixfunc)(std::vector<PROD const *> const &,
131 4 Christopher Green
                                     PROD &,
132 4 Christopher Green
                                     PtrRemapper const &),
133 4 Christopher Green
                  T & t,
134 4 Christopher Green
                  bool outputProduct = true);
135 4 Christopher Green
136 4 Christopher Green
// 4.
137 4 Christopher Green
template <typename PROD, typename T>
138 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
139 4 Christopher Green
                  std::string const & outputInstanceLabel,
140 4 Christopher Green
                  bool (T::*mixfunc)(std::vector<PROD const *> const &,
141 4 Christopher Green
                                     PROD &,
142 4 Christopher Green
                                     PtrRemapper const &),
143 4 Christopher Green
                  T & t,
144 4 Christopher Green
                  bool outputProduct = true);
145 4 Christopher Green
146 4 Christopher Green
// 5.
147 4 Christopher Green
template <typename PROD, typename T>
148 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
149 4 Christopher Green
                  bool (T::*mixfunc)(std::vector<PROD const *> const &,
150 4 Christopher Green
                                     PROD &,
151 4 Christopher Green
                                     PtrRemapper const &) const,
152 4 Christopher Green
                  T const & t,
153 4 Christopher Green
                  bool outputProduct = true);
154 4 Christopher Green
155 4 Christopher Green
// 6.
156 4 Christopher Green
template <typename PROD, typename T>
157 4 Christopher Green
void declareMixOp(InputTag const & inputTag,
158 4 Christopher Green
                  std::string const & outputInstanceLabel,
159 4 Christopher Green
                  bool (T::*mixfunc)(std::vector<PROD const *> const &,
160 4 Christopher Green
                                     PROD &,
161 4 Christopher Green
                                     PtrRemapper const &) const,
162 4 Christopher Green
                  T const & t,
163 4 Christopher Green
                  bool outputProduct = true);
164 4 Christopher Green
</code></pre>
165 1 Christopher Green
166 1 Christopher Green
h3. @CollectionUtilities.h@: @concatContainers()@ and @flattenCollections()@.
167 1 Christopher Green
168 1 Christopher Green
h3. @PtrRemapper@.