There are certain types of administrative tasks that we (Nova) need to be able to do within the SAM environment. These are detailed below.
Adding a User to be Admin¶
To add a user to have administrator access to the NOvA samweb instance:
- Login to samweb.fnal.gov as user "sam"
- Go to the sam_web product directory
- In that directory there is a nova.conf files
- Edit the nova.conf and add the users to the "admin_users" list.
You will have to restart the samweb server to have the changed propagate.
To do this send a HUP to the process:
kill -HUP <pid of samweb> apachectrl graceful
Deleting a file from SAM¶
If a file is declared in SAM but needs to be completely expunged, that can be done
$ curl -k --cert /tmp/x509up_u$(id -u) -X DELETE https://samweb.fnal.gov:8483/sam/nova/api/files/name/<file name>
Alternatively you can use the retire file command. This doesn't remove it from the database, but it does hide it (you have to do this for files that have been included in snapshots as they can't be completely deleted)
Retire a file from SAM¶
When a file has been included in snapshots but you know you need to retire it then you must retire the file and remove it from it's /pnfs and/or bluearc location.
The File Transfer Service has a retentive memory of its transfers. Actually it is sam_cp that has the memory, not FTS itself. sam_cp writes a log file listing all the transfers it has successfully
done and consults that log before before trying a transfer. The log lives in /var/tmp/sam_cp_novapro.log on each SAM VM instance.
The safest thing to do is to stop the FTS, move the existing log files to a backup area (in case you want to consult them later), retire the files from SAM and remove the files from their physical location.
A restart of the FTS will then allow a file with the same name to be transferred and added to the DB.
The latest version of sam_cp (v9_0_6) will rotate that log every couple of hours.
So you can stop FTS, explicitly setup sam_cp v9_0_6 and restart FTS. That should result in the log getting cleared periodically.
If you are in a hurry to get the files out, you can delete the log file before restarting FTS, as described above.
If you have a list of files to retire then you can use a python script that I put together to achieve this.
Currently it's in my devs area (probably will get a more permanent novasoft home:
% svn export svn+ssh://firstname.lastname@example.org/cvs/projects/novaart-devs/trunk/users/gsdavies/scripts/retire-remove-files.py
Python script that, given a list of files in text format as first argument, will:
1/ loop over the list
2/ check file exists in SAM
3/ get file location
4/ retire file
5/ remove file from location(s)
% setup sam_web_client v1_5
Then to take a snapshot, do
% kx509 % samweb -e nova take-snapshot <dataset>
This can take a while as sam builds a list of all the files satisfying
the query. You can then use the snapshot id to make another frozen
% samweb create-definition <frozen dataset name> "snapshot_id = <id>"
You can also only take segments of the snapshot using snapshot_file_number.