Cron jobs for creating JSON metadata files on the np04 cluster¶
For reference, here are my elog posts on this subject: [to see older elog posts, use the Advanced Search]
07-Dec-2017 I have installed a cron job on np04-srv-014 that will periodically check if new raw data files have appeared in /nfs/sw/data, and if so, create companion JSON metadata files in the same directory. The future intention of these metadata files is to serve as indicators that the raw data files are ready to be copied downstream. These metadata files should not affect the running of the DAQ, and they won't actually initiate any sort of file copy until additional steps are taken to install the data-copying tools (not sure when that might happen). This script is simply being tested now. At the moment, the contents of the metadata files are fairly basic; they just have a few parameters to describe the associated raw data file. The naming and contents of the metadata files may change in the future. The script that is being run by the cron job is /nfs/home/np04daq/.cron/createMetadataFile.sh. The crontab file that is used to register the cron job is /nfs/home/np04daq/.cron/np04-14.crontab. The script creates log file entries in np04-srv-014:/log/metadataFileCreator/createMDFile.log. There are tunable parameters at the top of the createMetadataFile.sh script that can be changed to modify its behavior. Please save a backup copy if you change the script. To un-register the cron job, you can use 'crontab -r' (run from the np04daq account on np04-srv-014).
Sunday, 27-Jan-2018 Subject: crontab created on np04-srv-001 to create metadata files for raw data files To do this, I copied the createMetadataFile.sh script to createMetadataFile01A.sh in /nfs/home/np04daq/.cron and modified the new "01A" copy to look at the /data0 disk (and create lock and log files with "01A" in their name). I then created the /nfs/home/np04daq/.cron/np04_01.crontab file and used it to create the cronjob ('crontab np04_01.crontab). Sunday, 28-Jan-2018 I also created a directory for log files: /log/metadataFileCreator