Project

General

Profile

Scripts

Last modified: {{last_modified}}

watchDataDisk.sh

A bash script that watches all known data disks for

  • Excessively large files (every sleep seconds)
    • Warnings are reported to msglogger on affected partition when file size exceeds warnSize for files updated in previous activeTime minutes
    • Errors are reported to msglogger on affected partition when file size exceeds errSize for files updated in previous activeTime minutes
  • Changes of state of data taking for each partition (every sleep seconds)
    • Info message reported to msglogger on affected partition
    • State change determined by absence/presence of file update in previous activeTime minutes
  • Status of data taking for each partition (every heartbeatSleepCycles cycles for the above two)
    • Info message reported to partition 0 msglogger for each partition
    • Long-term goal is to have MsgAnalyzer feed these messages to a DAQ status web page

Other notes:

  • If DDS server is not detected for a partition, messages are logged to stdout.
  • Directories to search are specified as diskList. This is set depending on which NOVADAQ_ENVIRONMENT is detected.
    • This means the script needs to run on a node that has all datadisk areas mounted. In practice, this means *-master
    • Directory specifications include hardcode of detector name, e.g., /data1/NDOS, to not prejudice future possible directory naming conventions.
  • The script runs in an indefinite loop
  • When the script starts, it sets up all partitions, and captures the OPSL_URI for each in an array for efficient switching of message destinations

To Do

  • As of 2014-04-15, only NDOS and FarDet data paths (diskList) are defined. Need to add NearDet
  • Add error handling for non-existence of directories
  • As of 2014-04-15, determining partition from data is done with EventDump |head 1000| ... more processing...
    • Need to investigate whether this is too much disk/CPU load. Replace with database queries?