DAQInterface should be able to resurrect an artdaq process
In light of the fact that DUNE is intended to have very low deadtime, if an artdaq process DAQInterface controls dies or enters the error state while running, DAQInterface should be able to relaunch a process of the same type and put it through transitions to bring it into the running state. Even if it's not currently the case that a resurrected process will seamlessly integrate into a running system, this DAQInterface feature will prove valuable when it comes to modifying artdaq processes so that they will support this functionality in the future.
JCF: Issue #23760: if a process dies DAQInterface will relaunch it, though it won't send it transitions
JCF: Issue #23760: if a process enters an Error state, clean it up, but let the heartbeat function decide what to do with it
JCF: Issue #23760: if a process is found to have died, then after relaunch, send it the init and start transitions
JCF: Issue #23760: no longer automatically ending things if there's a generic exception throw when querying process status
JCF: Issue #23760: sending shepherding transitions as soon as formerly-deceased process is alive, rather than after arbitrary 10-second wait
JCF: Issue #23760: have shepherding occur not just when process dies, but also when it enters the error state
#1 Updated by John Freeman 8 months ago
- % Done changed from 0 to 100
- Status changed from New to Resolved
Resolved with commit e2fe23ca71def385a2974305853f967209fd3f06 at the head of feature/23760_resurrect_processes. To test this, you want to add the line
to the $DAQINTERFACE_SETTINGS file and use $DAQINTERFACE_PROCESS_MANAGEMENT_METHOD == "direct" before launching DAQInterface. Then if a process dies or goes into an Error state when DAQInterface is in the running state, it will relaunch the process and attempt to send it an "init" transition followed by a "start" transition. Note, of course, that there's zero guarantee that datataking will proceed smoothly if this happens, since there's currently no requirement that artdaq processes support this behavior. This feature will prove useful for testing artdaq process behavior when they're modified to support continuous running, however.