Project

General

Profile

Disk Quotas for MicroBooNE

Active Monitoring Links

Disk Volume Usage: https://fifemon.fnal.gov/monitor/dashboard/db/experiment-overview?panelId=10&fullscreen&var-experiment=uboone

Persistent dCache Usage
/uboone/app usage by user
/uboone/data usage by user

SAM4Users tutorial: https://microboone-docdb.fnal.gov:441/cgi-bin/ShowDocument?docid=6896

Understanding Storage volumes at Fermilab: https://redmine.fnal.gov/redmine/projects/fife/wiki/Understanding_storage_volumes

Disk Quotas

MicroBooNE has access to a number of hard disk storage elements and is controlling users access to some of those volumes through quotas. Full details of the system are available here . The volumes that get the largest amount of use are as follows:

Volume Name File System Quota Designated Usage Notes
/uboone/app BlueArc NFS 100 GB code development, no data mounted read only on GPGrid, no mount on OSG
/uboone/data BlueArc NFS 1500 GB user personal data files no mount on GPGrid/OSG, slow access
/uboone/data BlueArc NFS 10000 GB for group accts official production data groups should utilize areas /uboone/data/<groupid>/ to store their production files
/pnfs/uboone/scratch dCache NFS no quota temp user personal data files no mount on GPGrid/OSG, fast access, files are not permanent, files are immutable
/pnfs/uboone/persistent dCache NFS 1500 GB user personal data files no mount on GPGrid/OSG, fast access, files are persistent, files are immutable
/pnfs/uboone/persistent dCache NFS 10000 GB for group accts official production data no mount on GPGrid/OSG, fast access, files are persistent, files are immutable
/pnfs/uboone/resilient dCache NFS no quota for storing files that may need to be accessed by many grid nodes simultaneously no mount on GPGrid/OSG, fast access, files are persistent, files are immutable

Special Notes on dCache Scratch volume

The /pnfs/uboone/scratch volume is a dCache pool that is rather large, but it is not infinite. If a new file is written to that /pnfs/uboone/scratch and there is no available space, the Least Recently Accessed (LRA) file will be removed without warning. Please understand that your files may be removed without warning if the /pnfs/uboone/scratch volume becomes full. You can monitor the age and lifetime of files in dCache public scratch here . You should actively use this volume for the output of grid jobs, but know that if the files are critical, you should find another location for permanent storage. You should also be aware that files in dCache are immutable, meaning that they cannot be modified after they have been written to the volume.

Special Notes on dCache Persistent volume

The /pnfs/uboone/persistent volume is a dCache pool that is not tape backed, but does not have a removal policy. You should also be aware that files in dCache are immutable, meaning that they cannot be modified after they have been written to the volume. This means that a file placed in that area will never be deleted except by the owner or a MicroBooNE admin. But there is also a limit to the total size of the volume and so users should limit their usage to the same usage on BlueArc. This means that users have a limit of 1500 GB, and physics groups have a limit of 10000 GB. These are not hard limits so can be exceeding, but if you abuse the system you will be publicly shamed in front of your peers and colleagues.

Special Notes on dCache Resilient volume

Data /pnfs/uboone/resilient is replicated 20 times. This allows data from the resilient pool to be served to many consumers at a much higher rate than is the case for standard dCache data pools. It is intended for files, such as release tarballs, that may need to be accessed by many batch nodes at once. There is no specific suggested quota for the resilient pool. Space in the resilient pool is very expensive, so people are encouraged to be careful about using excessive amounts of resilient space, and should clean up files in this area. The resilient pool should not be used for long term storage of data.

Proposed Deletion Policies for Data in Nonvolatile Disk Areas

Data owned by users who have left MicroBooNE

This policy applies to files owned by former MicroBooNE collaborators on /uboone/app, /uboone/data, and /pnfs/uboone/persistent. Files owned by former MicroBooNE collaborators may be archived and deleted without the consent of the owner, subject to the following conditions.

  • Files become subject to archival and deletion six months after a collaborator is no longer on the MicroBooNE masthead.
  • Approval by MicroBooNE spokepersons is required before involuntary archival and deletion of any files owned by former MicroBooNE collaborators.
  • Files are never deleted outright, but are archived according to the procedure in this wiki article: Archiving_and_Deleting_User_App_Areas. Archived files are accessible as tarballs in the tape-backed area of dCache /pnfs/uboone/archive/sam_managed_users/uboonepro/image.
  • Reasonable attempts will be made to contact other stakeholders before data is archived and deleted, including owner (if reachable), owner's PI, physics conveners, etc. Stakeholders will be given the opportunity to argue for temporary retention of files that are actively being used.

Automatic Deletion of Files in Resilient Pool

In order to limit the amount of data in the resilient pool, we are considering whether to implement an automatic deletion policy for resilient files.