NOvA has access to a variety of different "disks" on which to store files. They each have different properties and different recommended uses as described below. Important things to note:

  • There is no "perfect" disk that you can use for any kind of file in any circumstance.
  • There is no area of unlimited disk where you can put files and have them last forever. Be judicious about what you keep.

/pnfs/nova/scratch/users/<your name>

This area does not have a limited size, but files also have a limited lifetime. Typically about 30 days since last access. This is where all direct grid output should go. While the disk can be accessed via NFS (e.g. like a regular disk), this is not performant and only recommended for small scale work. The xrootd protocol is more reliable, and can be used easily with the pnfs2xrootd command.

root `pnfs2xrootd /pnfs/nova/scratch/users/<user>/myjob/job0.root`
hadd /nova/ana/users/<user>/combined.root `pnfs2xrootd /pnfs/nova/scratch/users/<user>/myjob/*.root`

Note that xrootd requires some special authentication (called a "proxy") which the setup_fnal_security script will set up for you. If you receive authentication errors using xrootd, try to run the security setup first.

/pnfs/nova/persistent/users/<your name>

This area is limited in size, but files will never be removed automatically. As of now there is no way to apply formal quotas on this disk area, so we ask that everyone please try to use this space responsibly. Files you intend to keep can be placed here.

/nova/app/users/<your name>
/nova/ana/users/<your name>

This is a much more "normal" disk system than dCache, and will give the best performance when using files interactively. However, as such it does not scale well so it is not accessible from the grid. app is higher performance, but smaller, so it is intended more for code than for data files.

/pnfs/nova/users/<your name>

This area will allow you to archive files long-term to tape. However, please use this option with care. Files on tape are difficult to work with if they are not in SAM, so we generally recommend against using this area except through a sam4users dataset. (Be sure to copy your files to the tape area before creating the sam4users dataset however.) More details here:

Also note that files in this tape area are subject to caching; if you have files which you can list in this area but which appear to be unavailable when you attempt to use them, this is why; they will need to be restaged into the cache before they can be used. See Tape and Cache for more explanation, including how to check the cache status and how to re-cache files.