Use elasticsearch condor events for better job data
We should use elasticsearch to fetch job data as well as (and possibly instead of)
using condor_q output. We should then get actual final data for the jobs and not
have any without cpu_time, etc. once they're completed.
Michal G left us a nice webservice/elasticsearch.py interface file that should let
us build a new agent for this.
#3 Updated by Joe Boyd almost 4 years ago
reporter agent script collecting job data from condor
event logs (from elasticsearch or directly from condor).
-- actual termination (so we can turn off 3 strikes code in condor_q watcher)
-- final memory/cpu usage
-- job exit code (vs user executable we have now)
#5 Updated by Joe Boyd almost 4 years ago
We are currently ingesting these attributes into the fifebatch-jobs table:
We are NOT ingesting these from jobs:
Will have to get those to have full functionality.