Writing your own input sources¶
Sometimes, the RootInput
and EmptyEvent
input sources may not be sufficient to meet experimental requirements. Possible use cases include:
- You are reading raw binary files singly (e.g. from the DAQ) or receiving the data via some other method (streaming, for example).
- You are generating the data based on input parameters or you need to read multiple files simultaneously.
The source:art/Framework/IO/Sources/Source.h input source template is able to meet the requirements of these use cases. The header documentation therein (and also in source:art/Framework/IO/Sources/SourceTraits.h) lays out the basic requirements and constraints on experimental code, but below you can find more in-depth documentation explaining how everything fits together, and more particularly how to fulfill the requirements of the use cases enumerated above.
Source template overview.¶
Unlike a normal producer or analyzer module, here the experimental developer is not expected to write a module. Instead they write a "detail" class which satisfies the requirements for use as a template argument to the Source
template. So, for an imagined HighSpeedMuons experiment, one might make a file HSMRawInput_source.cc
:
#include "art/Framework/Core/InputSourceMacros.h"
#include "art/Framework/IO/Sources/Source.h"
#include "art/Framework/IO/Sources/SourceTraits.h"
namespace hsm {
// Forward declaration of detail class.
class HSMRawInputDetail;
}
////////////////////////////////////////////////////////////////////////
// Any declarations of specializations of Source traits (see
// art/Framework/IO/Sources/SourceTraits.h) should go here.
//
// ...
//
////////////////////////////////////////////////////////////////////////
// Definition of detail class.
class hsm::HSMRawInputDetail {
...
};
// Optional typedef.
namespace hsm {
using HSMRawInputSource = art::Source<HSMRawInputDetail>;
}
// Define the input source to the art system.
DEFINE_ART_INPUT_SOURCE(hsm::HSMRawInputSource)
The source template "steers" the art framework, and asks an instance of the detail class for the information it needs at the appropriate points in the framework job.
Under the most basic configuration, the Source template implements the first use case of those listed above. Defined (and documented) in source:art/Framework/IO/Sources/SourceTraits.h are two "traits" templates which can be specialized by the experimental developer in order to allow the Source template to satisfy the other two (and more).
Your detail class T must supply the following non-static functions:
- A constructor with the following signature:
TheT(fhicl::ParamterSet const &, art::ProductRegistryHelper &, art::SourceHelper const &)
art::ProductRegistryHelper
providestemplate <typename P, art::BranchType B> reconsistutes(...)
necessary for declaring the data products your source will produce. Theart::SourceHelper
provides functions for making principals (art::RunPrincipal
,art::SubRunPrincipal
andart::EventPrincipal
), which are the lower-level entities behind the transactional objects with which you may be more familiar (art::Run
,art::SubRun
andart::Event
).
This class must provide a validvoid readFile(std::string const & filename, art::FileBlock*& fb);
art::FileBlock
when called even if thefilename
input parameter is ignore (for generators, say). If something is wrong andreadFile(...)
cannot do so, then it must throw.
This function is expected to provide principals to the art framework when appropriate.bool readNext(art::RunPrincipal const* const inR, art::SubRunPrincipal const* const inSR, art::RunPrincipal*& outR, art::SubRunPrincipal*& outSR, art::EventPrincipal*& outE);
inR
andinSR
are provided and are non-null when appropriate (and should be used when invokingart::SourceHelper
functions to create new principals lower down the hierarchy).outR
andoutSR
should only be non-null if they are new: do not simply returninR
andinSR
.outE
should always be provided unless the function returns false, indicating no data.
Change as of art
2.06.00: Whenever an input file is closed, the inR
and inSR
pointers to readNext
are set to null, requiring users to set outR
and outSR
at the first call to readNext
after the next input file is opened.
This function must exist, and will be called by the framework at appropriate times, including at job end.void closeCurrentFile();
Use case studies.¶
You are reading raw binary files (e.g. from the DAQ) or receiving the data via some other method (streaming, for example).¶
If you are reading files singly (the "normal" case), then the framework is reponsible for obtaining the files listed in the source.fileNames
parameter (for instance from SAM via the appropriate services if configured), and passing each filename to the detail class' readFile(...)
for (presumably) opening, and creation of the required art::FileBlock
object. If you do not wish the framework to use its file services to handle the files (for instance the contents of fileNames
are actually URIs), then you must specialize the art::Source_wantFileServices
template at the appropriate place in your (_e.g.) HSMRawInput_source.cc
file, viz:
namespace art {
template <>
struct Source_wantFileServices {
static constexpr bool value = false;
}
}
This will cause the framework to pass the URIs to your readFile(...)
function verbatim, one at a time in the original order.
If your readNext(...)
function should ever return false
, then the framework will call closeCurrentFile()
, obtain the next file from source.fileNames
if there is one (and file services are in use), and then readFile(...)
will be invoked before resuming calls to readNext(...)
. If there are no remaining elements in source.fileNames
, the framework will end the job.
You are generating the data based on input parameters or you need to read multiple files simultaneously.¶
In this case, you will need to specialize the art::Source_Generator
template, e.g.:viz:
namespace art {
template <>
struct Source_Generator {
static constexpr bool value = true;
}
}
The art::Source_wantFileServices
template will behave as required (deactivating use of art's file services), because the default implementation of the template defines art::Source_WantFileServices<T>::value = !art::Source_Generator<T>::value
, and need not be specialized therefore.
For generators, the framework does not care about the source.fileNames
parameter and you are free to use it (or not) as you wish. The framework still calls the detail class' readFile(...)
and closeCurrentFile(...)
functions as appropriate, but takes its cue on when to do this from the return value of the readNext(...)
function when called. If the function returns false
, indicating a lack of data and therefore no new principals, then what happens next depends on whether the class has a function bool hasMoreData()
or bool hasMoreData() const
. If so, then a result of true
upon calling hasMoreData()
will cause the framework to call closeCurrentFile(...)
, followed by readFile(...)
for a new art::FileBlock
before resuming calls to readNext(...)
. A result of false
from hasMoreData()
is treated the same way as if the function did not exist, and the framework ends the job.
General notes.¶
- The implementor of
readNext(...)
is entirely responsible for providing consistent principals, including creation of required auxiliary objects where appropriate, consistent run, subrun and event numbers and time stamps.
- With regard to time stamps, while the framework defines a syntax (a single unsigned 64-bit number per canvas:source:canvas/Persistency/Provenance/Timestamp.h), it does not define a semantic: the implementor of the input source is responsible for this. See also source:art/Framework/Core/EmptyEventTimestampPlugin.h: if you have an implementation of same that is appropriate, your input source should use it directly.