Directory scanner doesn't clean up properly after errors
If the directory scanner hits an unexpected exception it doesn't reset properly for the next pass. This can lead to errors similar to
File "/grid/fermiapp/products/common/prd/FileTransferService/v4_2_1/NULL/python/fts/scandir.py", line 113, in __call__ self.task.resume() File "/grid/fermiapp/products/common/prd/twisted/v14_0_2/Linux64bit-2/lib/python/twisted/internet/task.py", line 427, in resume raise NotPaused() twisted.internet.task.NotPaused:
If this happens the scanner is no longer running, so new files are no longer discovered.
#2 Updated by Robert Illingworth about 4 years ago
- Status changed from New to Assigned
There are at least two problems:
There is a call to glob which isn't protected by an exception handler and nor is it dispatched to the thread pool. So it could fail or block everything else.
If there is an unhandled exception anywhere in the scanner the workqueue never gets paused and the scan ended deferred is never fired. This means the scanner fails on the next run.