have daqinterface support the specification of cores on which the various artdaq processes can run
For SBN, it seems highly likely that we will need to avoid running artdaq processes on core 0.
We need a way to tell daqinterface this.
As we discussed in the group meeting, it is desirable that no-specification be allowed (i.e backward compatibility maintained).
The specification should allow for something compatible (or similar to) the specification of the taskset -c option.
Note: the case where there are more processes than available cores should be covered.
#1 Updated by John Freeman almost 2 years ago
- % Done changed from 0 to 50
- Assignee set to John Freeman
- Status changed from New to Work in progress
With commit ca6aad7767ae8504b7bf1b23f56093ed16895fa7 on branch feature/issue22258_specify_cores, when DAQInterface manages processes in "direct" mode you can instruct it to launch each artdaq process prefaced with "
taskset --cpu-list <cpulist>". The way to do this is just to edit $DAQINTERFACE_SETTINGS by adding this line:
This will allow an experiment to ensure that artdaq processes don't have an affinity for core 0, and it also means that by default taskset doesn't play a role in determining an artdaq process's core affinity - a user must explicitly set this in the settings file. However, I'm a bit confused by this line from the previous comment:
"Note: the case where there are more processes than available cores should be covered"
Apologies if this was discussed in the Monday meeting and I forgot, but could this be elaborated on? Is the idea that if there are more processes than cores DAQInterface should ignore the taskset_cpu_list option and make sure that each process has equal affinity for all cores, or...?
#2 Updated by John Freeman almost 2 years ago
- % Done changed from 50 to 100
- Status changed from Work in progress to Resolved
I consider this issue resolved. At the now-head of feature/issue22258_specify_cores, f4e8d7180e9e6bc1efdb0f54713c1e2cec6eb93d, you can tell DAQInterface to launch all artdaq processes with taskset by providing the argument to
taskset --cpu-list by adding a line like this to $DAQINTERFACE_SETTINGS:
where <list> takes an allowed format acceptable to the --cpu-list argument, e.g.
You can also fine tune the list of allowed processors for individual processes. For non-boardreader processes, this is done in the boot file, e.g. like so:
EventBuilder host: mu2edaq05 EventBuilder allowed_processors: 0-15 EventBuilder label: EventBuilder1
where here we want to run an eventbuilder on mu2edaq05, but we only want it to use 16 of its 48 available processors.
For a boardreader process, you want to add a fifth value to a given boardreader's entry. E.g.:
component01 localhost -1 1 0-15
where we're saying we want boardreader component01 to run on the same host as DAQInterface, with the "-1" telling it to calculate the port on the fly, the "1" telling it to put the process in subsystem 1, and the 0-15 telling it that we want it to use just the first 16 processors on the host.