- Table of contents
- 4/13/2015 Notes
Pengfei, Zukai, Jan, Leon and Martin (taking notes)
- BNEVB does not get data any longer, so it decides to stop the run.
- fetchProcessInfo.sh gets slowed down when DDT hogs all of the processing, so it will turn red in the DAQApplicationManager.
- CSET gives the ability to run processes on shielded CPU sets. Pengfei thinks that it might be that the shield prevents the processes from being migrated off a CPU set.
- Leon asks why we cannot just have the standard system scheduler take care of the process management instead of manually managing it with these CPU sets.
- Of course, the ideal solution would be to have more buffer nodes to reduce the load.
- Leon suggests that we just wait the amount of time that it takes for all of DDT to be spun up. This is something that we can time in the control room.
- One concern that I have is that by waiting very long, we will have all DDT and DDS processes on one CPU which could be an issue.
- Until we have a chance for a more thorough discussion of this with Andrew in the room, let's keep things running as they are.
Restarting DDT Issues¶
- When some processes got stuck, Leon tried to restart the DDT processes.
- Restarting the DDT processes somehow crashes the DAQ run.
- Perhaps this was related to the situations where DDT was crowding out DDS since it was only tried on these hosts.
- When there is some downtime, we could test stopping/starting DDT to make sure there is not other issue.
- We do not need to worry as per Eric and Ron.
- Raw Data is looking at the shared memory, but this memory can get read/written out faster than we poll it.
- If we lose data, there will be many red flags/errors that will be issued.
- Eric requested a simple activity trigger for NDOS.
- I will get that set up for him.
- We will add a 10 min discussion on the CPU Sets and how we should move forward.
- It would be particularly nice for everyone to understand why we need them instead of simply using the system scheduler.