Archives of Former Nova Users¶
Procedure¶
When users leave Nova, if they have a significant quantity of files on BlueArc, their areas will be archived to tape using sam_archive_directory_image
.
First, request that the permissions be setup for archiving. Ask someone with admin privileges on the Nova machines (CS liaison, scientific server support) to run the following:
chmod -R g+rwX /path/to/user/directory
Log in as novapro and execute the following. It can take a while so a screen session is suggested.
# Log into the appropriate machine ssh novapro@novasamgpvm01.fnal.gov # This process takes a long time, I suggest doing # it in a screen session screen # or screen -x if a session has already been started # Setup the environment setup_nova setup_fnal_security # Do the archiving mkdir -p /pnfs/nova/archives/users/<user>/<app or ana> sam_archive_directory_image -v -4 -e nova -d /pnfs/nova/archives/users/<user>/<app or ana> -s <source> # Confirm copy was successful and image is in sam ls -ltr /pnfs/nova/archives/users/<user>/<app or ana>/ samweb list-files "archive.source \"<source>\"" samweb get-metadata <archive image> # If it checks out, remove the source directory and add info for retrieval rm -rf <source>/* echo 'samweb list-files "archive.source \"<source>\""' > <source>/ARCHIVE
How to access archived files:¶
This is a specific example, but should be instructive.
First, locate the archive file.
$ samweb list-files "archive.source \"/nova/app/users/betan009\"" image-d6577cd9-ba81-4849-96f7-3377dd22fe46.tgz $ samweb locate-file image-d6577cd9-ba81-4849-96f7-3377dd22fe46.tgz enstore:/pnfs/nova/archives/users/betan009/app(8@vpl194)
Next, check if the archive is cached or only on tape:
$ cache_state.py image-d6577cd9-ba81-4849-96f7-3377dd22fe46.tgz Cached: 0 Tape: 1
If the file is already cached, you can skip the next step.
To cache the file from tape, do the following:
$ samweb create-definition temp-prestage-betan009 "archive.source \"/nova/app/users/betan009\"" $ samweb run-project --defname=temp-prestage-betan009 $ samweb delete-definition temp-prestage-betan009
When the command completes (can take some time) you can continue.
It will be fairly slow to copy over and untar the whole thing, but there is a trick to access specific files. Do as follows:
$ tar -tzf /pnfs/nova/archives/users/betan009/app/image-d6577cd9-ba81-4849-96f7-3377dd22fe46.tgz | grep "<the file you want>" $ tar -xzf pnfs/nova/archives/users/betan009/app/image-d6577cd9-ba81-4849-96f7-3377dd22fe46.tgz /path/to/the/file/you/want
Areas that have been archived:¶
to/nova/Ana/users/cerretan
/pnfs/nova/archives/2015-May/cerretan
to182G /nova/app/users/denis
/pnfs/nova/archives/users/denis/app
to178G /nova/app/users/timdkut
/pnfs/nova/archives/uers/timdkut/app
to176G /nova/app/users/betan009
/pnfs/nova/archives/uers/beta009/app
to200G /nova/app/users/blinehan
/pnfs/nova/archives/users/blinehan/app
Areas that need to be archived:¶
128G /nova/app/users/rocco
113G /nova/app/users/xbbu
108G /nova/app/users/waldron
113G /nova/app/users/kkelly12
/nova/app/users/tamsett
/nova/app/users/raddatz
/nova/app/users/dmitryr
/nova/app/users/rtoner
/nova/app/users/avgavril
/nova/app/users/mercurio
/nova/app/users/kravtsov