Create a script to prune and purge the web-area
HEPCloud, Factory Ops
The web-area is the staging area for the files downloaded by the glideins.
Its size can grow considerably, especially if there are many entries and if there are many reconfig/update operations.
Deleting files from it at random may cause inconsistencies that cause errors when the Factory is restarted.
This happens especially because of signature files and download lists, that may end up referencing files that have been deleted.
A script for cleanup operations would be a useful tool. It should allow:
- purging of old files no more used by glideins or referenced by newer download files
- pruning files for entries that are no more in use
All of the above without removing files that are still referenced by files that are kept
This would have been useful in a recent cleanup of the HEPCloud Factory