Computing Data Storage in Theoretical Physics

Linux desktop system

Home directories

Files in one's home directory are stored on an NFS server. Quota is 75GB. This can be extended by request, but space on the server is limited for the time being. This space is mounted at /home/$USER and /network/home/theory/$USER and is backed up daily.

Client hard disks

Files for local processing are be stored at /local/home/$USER and /scratch on machines with an extra disk (/scratch/$USER is preferred, please make this directory yourself).
These are not backed up except by request.

Interactive systems

These are the same as desktops, but the /local and /scratch partitions are larger (several TB in some cases). The space available is listed on log in.

Cluster systems

The Theory cluster systems have tens of TB of commonly-mounted disk space. This space is intended for the (often very large) data sets that calculations either use or produce; it is not intended for the long-term storage of data.

Apple desktop system

Information about how to connect to the Theory File Server can be found here

Home directories

Prior to 10.13, network storage was mounted directly. In 10.13 and on, files are copied to the local disk and synced to network storage and backed up with Time Machine on the central Time Machine server.

Home directories on the Apple desktop system are stored centrally on disk arrays connected to a dedicated file server. These disk arrays are backed up according to the backup schedule. The home directories themselves are made available (via the file server) on every client of the Apple system, under the directory:

Hard Disk

/home/username

If you had a different username on the Apple desktop system prior to 2010 or so then your previous username is kept in you home directory path on the Apple file server. This may change in future.

Home directory space is currently not limited by quota, however we may enable quotas in the future depending on disk space. It is also limited on 10.13 and up by the amount of space free on the local system.

Because this data is backed-up, and backing-up data uses resources, we ask you to ensure that any large amounts of data are academically necessary. What is considered large, and what is considered academically necessary, is of course a subjective question, but some examples might clarify matters:

  • If the data is easily re-obtainable from elsewhere (e.g. astronomical data-sets), we ask that you not store it in your home directory, but in your /Data partition, from where it will not be unnecessarily backed-up. Such files are certainly academic, but it isn't necessary to store them in home directories.
  • Music and videos, etc. should not be stored in home directories, because they are both 'large' and not academic. Please store such files in your /Data partition
  • Archives of personal documents, although not academic, are not considered 'large' and so may be stored in home directories. Please consider encrypting private files before entrusting them to our system.

Client hard disks

On 10.13 and up, there is only one partition, so care needs to be taken about how much of the disk is used. When overfilled, first the system will slow down as swap files cannot be created, then the computer will stop working altogether.

Client hard disks on the Apple desktop system prior to 10.13 are divided into two partitions: System HD which contains system files and applications; and Scratch (/Data on newer systems) which is a free-for-all, not-backed-up data partition. Scratch partitions are usually large (hundreds GB) and there are very many of them (one on every client).

Scratch partitions are primarily intended for (a) processes which need very rapid I/O response that the network-mounted home directories cannot provide, and (b) for local storage of large data-sets that do not need to be backed up. Do note, that disks can easily crash, and scratch-partitions can be easily wiped by support-staff without notice (though we try not to...), and if you have any concern at all about the safety of your data, you should probably put it in your home directory, or if it is too large, speak to support for advice.

However, since we have copious quantities of scratch space, there is no particular problem if you wish to keep large collections of music or video on the scratch space of your desktop machine (provided you respect copyright laws, etc).

Scratch space can be found on each client machine at:

/Volumes/Scratch or /Data

Groups shared space

The Apple system has a large, shared backed-up collaboration space called Groups, It can be accessed either via AFP or SMB and is shared from the Theoretical Physics file server in the Groups mount point. To access it from an Apple system in Finder click Go > Connect to Server from the menu bar at the top of the screen and type the following:

This space is particularly useful for files that need to be accessed by multiple users, since these users will typically want to access the files from different machines.

Categories: Services | Theory