Configuration validation and fhiclcpp
types¶
The purpose of introducing fhiclcpp
types is to enable:
- a way to validate a user-provided, run-time configuration against a supported configuration,
- a way to provide a description of a supported configuration for a given module, service, or other plugin library,
- an improvement over using the
'pset.get<T>("key",optional_default)'
pattern.
The fhiclcpp
type system was designed so that users would not need to interact directly with a fhicl::ParameterSet
object, but can do so if they desire.
Table of contents¶
- Introduction
- Examples
- fhiclcpp types in detail
- Auxiliary classes
- Conditional configuration
- System details
Introduction¶
fhiclcpp
types vs. FHiCL categories¶
The mapping between the fhiclcpp
and FHiCL types is shown in this table:
fhiclcpp type |
FHiCL category | FHiCL example |
Each of the fhiclcpp types below reside in the fhicl namespace. |
||
Atom<T> OptionalAtom<T> |
atom | parameter : value |
Sequence<T> OptionalSequence<T> |
sequence |
|
Sequence<T, std::size_t> OptionalSequence<T, std::size_t> |
|
|
Tuple<T...> OptionalTuple<T...> |
|
|
Table<T> Table<T, KeysToIgnore> OptionalTable<T> |
table |
|
Since the FHiCL sequence type is very flexible, three fhiclcpp
types have been introduced to improve the semantics of the C++ code. Formally speaking, the types Sequence<T>
and Sequence<T, std::size_t>
map to the same FHiCL sequence category. However, Sequence<T,std::size_t>
is a bounded sequence, which is explained below. The Optional*
parameters above are described here.
Necessary header files¶
#include "fhiclcpp/types/Atom.h" // fhicl::Atom<T>
#include "fhiclcpp/types/DelegatedParameter.h" // fhicl::DelegatedParameter
#include "fhiclcpp/types/Sequence.h" // fhicl::Sequence<T> & fhicl::Sequence<T,size_t>
#include "fhiclcpp/types/Tuple.h" // fhicl::Tuple<T...>
#include "fhiclcpp/types/TupleAs.h" // fhicl::TupleAs<T(U...)>
#include "fhiclcpp/types/Table.h" // fhicl::Table<T>
#include "fhiclcpp/types/TableFragment.h" // fhicl::TableFragment<T>
#include "fhiclcpp/types/OptionalAtom.h" // fhicl::OptionalAtom<T>
#include "fhiclcpp/types/OptionalDelegatedParameter.h" // fhicl::OptionalDelegatedParameter
#include "fhiclcpp/types/OptionalSequence.h" // fhicl::OptionalSequence<T> & fhicl::OptionalSequence<T,size_t>
#include "fhiclcpp/types/OptionalTuple.h" // fhicl::OptionalTuple<T...>
#include "fhiclcpp/types/OptionalTupleAs.h" // fhicl::OptionalTupleAs<T(U...)>
#include "fhiclcpp/types/OptionalTable.h" // fhicl::OptionalTable<T>
// Headers that can be included
// - 'fhicl::Name', 'fhicl::Comment', and
// 'fhicl::use_if' and 'fhicl::use_unless' are already
// provided by each of the headers above
#include "fhiclcpp/types/Name.h"
#include "fhiclcpp/types/Comment.h"
#include "fhiclcpp/types/ConfigPredicate.h"
Usage pattern¶
The order in which the system is used is the following:
fhiclcpp
parameter declarations:fhiclcpp
parameters are declaratedParameterSet
validation:fhicl::ParameterSet
is validated against thefhiclcpp
parameter collection- Value filling: Values are filled for
fhiclcpp
parameters from theParameterSet
- Value retrieval: Values are usable in C++ code
Steps 1 and 4 are exposed to the user. Steps 2 and 3 can be exposed to the user, in principle, but we do not encourage the user to do them unless there is a good reason (please consult artists@fnal.gov for guidance).
For art users, please consult here. Users outside of art should consult the link below.
1. fhiclcpp
parameter declaration¶
A fhiclcpp
parameter declaration follows the pattern:
fhiclcpp_type < arg_type(s) > name { Name("name"), [Optional arguments] };
where fhiclcpp_type
is either Atom
, Sequence
, Tuple
, or Table
.
arg_type(s)
¶
The allowed set of arg_type(s)
values depends on the specific value of fhiclcpp_type
(and described in detail <HERE>). In general, almost any C++ type that is currently supported in a ParameterSet::get
call is allowed. The std
containers, however, are not allowed for any fhiclcpp
type:
std::array
std::pair
std::tuple
std::vector
Name("name")
¶
All fhiclcpp
types must be initialized with a fhicl::Name
rvalue as its first argument. The "name"
string corresponds to the sequence of characters corresponding to the parameter name in a FHiCL configuration:
name : "some value for name"
In general, we suggest the same character sequence for the variable name 'name'
as the string supplied in the fhicl::Name
. For example,
Atom<int> debugValue { Name("threshold") }; // discouraged
is discouraged as the mismatch in
debugValue
vs. threshold
is likely to cause confusion when parameter value retrievals are performed (see <HERE>).
Optional arguments¶
All optional arguments must be specified after the fhicl::Name
argument. Both options below can be specified with the same fhiclcpp
parameter declaration.
fhicl::Comment
Each of the types may include a fhicl::Comment
rvalue argument, whose string will be printed whenever the description of the configuration is requested:
Atom<int> param {
Name("param"), // 'Name' is NOT optional
Comment("A description") // 'Comment' is optional
};
std::function<bool()>
- conditional configuration predicate
In some cases, it is desirable to introduce components of a configuration only if the value of a previously-supplied parameter satisfies a given condition (i.e a predicate). This is done by the user providing an std::function
object whose function type receives no arguments, and whose return type is bool
. Details as to how a user can enable this feature are given here.
Parameter value default
All fhiclcpp
types can receive a default parameter value except for fhicl::Table
objects. If a FHiCL configuration does not provide the parameter value, then the default value will be used. For example,
Atom<int> param { Name("param"), 4 };
will yield the same behavior as auto param = pset.get<int>("param", 4);
whenever the value of "parm"
is retrieved. A significant difference, however, is that all parameter values are loaded prior to retrieval with the fhiclcpp
-typed system.
2. Configuration validation¶
The validation feature is designed to catch- missing parameters as determined by the declared
fhiclcpp
parameters in the C++ code - extra parameters in a user's configuration file that are not supported by any corresponding declared
fhiclcpp
parameters
For fhiclcpp
parameters that receive a default value, a corresponding FHiCL parameter that is absent in a configuration file is not an error. Any misspelled FHiCL parameters -- i.e. those that do not match the string supplied in the fhicl::Name
argument of the fhiclcpp
parameter constructor -- are classified as extra parameters and are thus caught.
Consider the following allowed configuration:
struct Config {
Atom<std::string> name { Name("name"), "Gustav" };
};
Table<Config> pset { Name("pset") };
An attempt is made to provide a configuration to the file that uses this configuration. However the file FHiCL file is misconfigured:
# config.fcl
pset : {
nam : Mahler # attempt to override default value of "Gustav"
flag : false
}
The parameter nam
is misspelled with respect to the allowed configuration name "name"
, and flag
is not supported. When the configuration validation step is performed, an exception is thrown and the following error message is printed to STDOUT:
Unsupported parameters: + flag [ ./config.fcl:3 ] + nam [ ./config.fcl:2 ]
The characters in brackets denote the location of the unsupported parameter [ file-name:line-number
]. The listing order of the unsupported parameters is based on the std::string
ordering operator, which is lexicographical.
3. Parameter value filling¶
If the validation step (2) is successful -- i.e. the fhicl::ParameterSet
object conforms to the supported configuration as declared in the C++ code -- then the values of the parameters are filled using the specified values in the configuration file, or the default values as specified in the C++ code if the corresponding FHiCL parameter has not been provided. After this step, the fhiclcpp
parameter now has a value corresponding to the user-provided ones.
The parameter-value-filling step is done internally, the details of which are not necessary to understand for the user.
4. Parameter retrieval¶
Standard parameters¶
For a standard fhiclcpp
parameter declared as:
Atom<int> param { Name("param"), 4 };
the integer value of the parameter can be retrieved using the function-call syntax '()'
:auto param_value = param(); // 'param_value' is int
assuming the supplied value in the FHiCL configuration can be decoded to the C++ type int
.
Optional parameters¶
For an optional fhiclcpp
parameter declared as:
OptionalAtom<int> optParam { Name("optParam") };
the integer value of the parameter can be retrieved if present in a configuration file by passing a variable by reference using the function-call syntax '()'
:int j{};
if ( optParam(j) ) {
// 'j' can now be used
}
If 'optParam'
is present with a non-nil value in a configuration file, the value of optParam(j)
will return true
, and j
will be filled with the configuration-set value. Otherwise, j
retains its initial value ('0' in this case). Note that optional parameters can receive no default value.
General structure¶
The fhiclcpp
-typed system has been designed in such a way that the parameter declaration and retrieval syntaxes are similar. For example, the FHiCL configuration:
# FHiCL parameter declarations
settings : {
verbosity : true
}
# FHiCL value retrieval
verbosity : @local::settings.verbosity
would have a fhiclcpp
representation of:// fhiclcpp parameter declarations
struct Settings{
Atom<bool> verbosity { Name("verbosity") };
};
Table<Settings> settings { Name("settings") };
// fhiclcpp value retrieval
auto verbosity = settings().verbosity();
Details are here regarding specific return values for the different fhiclcpp
types.
Configuration description¶
Consider the following fhiclcpp
parameter configuration:
struct G4Settings {
Atom<double> energyCutoff { Name("energyCutoff") };
Atom<string> physicsList { Name("physicsList"), "QGSP_BERT" };
};
struct Config {
Atom<bool> flag { Name("flag"), false };
Atom<int> threshold { Name("threshold"), Comment("ADC count threshold") };
OptionalTable<G4Settings> g4Settings { Name("g4Settings") };
Sequence<string,2u> particles { Name("particles") };
};
Table<Config> config { Name("config") };
Formation¶
The description of the allowed FHiCL configuration (as shown above) is formed based on the order of initialization of the class members. In addition to registering the expected key, the expected type and any user-provided defaults are also registered with the fhiclcpp
system. The below table shows the order of initialization and the corresponding FHiCL keys and types for each fhiclcpp
parameter.
Order | C++ variable | Registered FHiCL key | Registered FHiCL type |
1 | config |
"config" |
table |
2 | flag |
"config.flag" |
atom |
3 | threshold |
"config.threshold" |
atom |
4 | g4Settings |
"config.g4Settings" |
table |
5 | energyCutoff |
"config.g4Settings.energyCutoff" |
atom |
6 | physicsList |
"config.g4Settings.physicsList" |
atom |
7 | particles |
"config.particles" |
sequence |
8 | particles[*] |
"config.particles[*]" |
atom |
9 | atom |
The asterisk '*'
means that the order of initialization of the individual Sequence
elements is unknown and therefore the key assignment needs to be handled specially.
Print-out¶
For users that have access to the 'config' object, a print-out of the allowed description can be produced using:
Table<Config> config { Name("config") };
config.print_allowed_configuration(std::cout); // Any 'std::ostream' object can be provided
The printed description is sent to STDOUT:
config: { flag: false # default ## ADC count threshold threshold: <int> # g4Settings: { # # energyCutoff: <double> # # physicsList: "QGSP_BERT" # default # } particles: [ <string>, <string> ] }
The pound signs ('#'
) preceding the g4Settings
table indicate that it is an optional table, which the user is not required to specify in the configuration file.
For art users, printing out the allowed configuration is described here.
Non-art
users¶
For users outside of art, it is possible to use the fhiclcpp
types system. The following pattern is recommended:
namespace {
struct MyConfig {
// fhiclcpp parameter declarations ...
};
fhicl::Table<MyConfig>
retrieveConfiguration( fhicl::ParameterSet const & pset )
{
std::set<std::string> ignorable_keys {}; // keys that should be ignored
// by the validation system (can be empty)
fhicl::Table<MyConfig> const result { pset, ignorable_keys }; // performs validation and value setting
return result;
}
} // anon. namespace
void some_func( fhicl::ParameterSet const & pset )
{
auto config = retrieveConfiguration( pset );
// 'config' now usable
}