Feature #22258
have daqinterface support the specification of cores on which the various artdaq processes can run
100%
Description
For SBN, it seems highly likely that we will need to avoid running artdaq processes on core 0.
We need a way to tell daqinterface this.
As we discussed in the group meeting, it is desirable that no-specification be allowed (i.e backward compatibility maintained).
The specification should allow for something compatible (or similar to) the specification of the taskset -c option.
Note: the case where there are more processes than available cores should be covered.
History
#1 Updated by John Freeman almost 2 years ago
- % Done changed from 0 to 50
- Assignee set to John Freeman
- Status changed from New to Work in progress
With commit ca6aad7767ae8504b7bf1b23f56093ed16895fa7 on branch feature/issue22258_specify_cores, when DAQInterface manages processes in "direct" mode you can instruct it to launch each artdaq process prefaced with "taskset --cpu-list <cpulist>
". The way to do this is just to edit $DAQINTERFACE_SETTINGS by adding this line:
taskset_cpu_list: <cpulist>
This will allow an experiment to ensure that artdaq processes don't have an affinity for core 0, and it also means that by default taskset doesn't play a role in determining an artdaq process's core affinity - a user must explicitly set this in the settings file. However, I'm a bit confused by this line from the previous comment:
"Note: the case where there are more processes than available cores should be covered"
Apologies if this was discussed in the Monday meeting and I forgot, but could this be elaborated on? Is the idea that if there are more processes than cores DAQInterface should ignore the taskset_cpu_list option and make sure that each process has equal affinity for all cores, or...?
#2 Updated by John Freeman almost 2 years ago
- % Done changed from 50 to 100
- Status changed from Work in progress to Resolved
I consider this issue resolved. At the now-head of feature/issue22258_specify_cores, f4e8d7180e9e6bc1efdb0f54713c1e2cec6eb93d, you can tell DAQInterface to launch all artdaq processes with taskset by providing the argument to taskset --cpu-list
by adding a line like this to $DAQINTERFACE_SETTINGS:
allowed_processors: <list>
where <list> takes an allowed format acceptable to the --cpu-list argument, e.g.
0-15
You can also fine tune the list of allowed processors for individual processes. For non-boardreader processes, this is done in the boot file, e.g. like so:
EventBuilder host: mu2edaq05 EventBuilder allowed_processors: 0-15 EventBuilder label: EventBuilder1
where here we want to run an eventbuilder on mu2edaq05, but we only want it to use 16 of its 48 available processors.
For a boardreader process, you want to add a fifth value to a given boardreader's entry. E.g.:
component01 localhost -1 1 0-15
where we're saying we want boardreader component01 to run on the same host as DAQInterface, with the "-1" telling it to calculate the port on the fly, the "1" telling it to put the process in subsystem 1, and the 0-15 telling it that we want it to use just the first 16 processors on the host.
#3 Updated by Gennadiy Lukhanin over 1 year ago
- Status changed from Resolved to Reviewed