Computing Data Storage in Theoretical Physics

Apple desktop system

Information about how to connect to the Theory File Server can be found here

Home directories

Home directories on the Apple desktop system are stored centrally on disk arrays connected to a dedicated file server. These disk arrays are backed up according to the backup schedule. The home directories themselves are made available (via the file server) on every client of the Apple system, under the directory:

Hard Disk

/home/username

If you had a different username on the previous Tiger/Leopard Apple desktop system then your previous username is kept in you home directory path.

Home directory space is currently not limited by quota, however we may enable quotas in the future depending on disk space.

Because this data is backed-up, and backing-up data uses resources, we ask you to ensure that any large amounts of data are academically necessary. What is considered large, and what is considered academically necessary, is of course a subjective question, but some examples might clarify matters:

  • If the data is easily re-obtainable from elsewhere (e.g. astronomical data-sets), we ask that you not store it in your home directory, but in your Scratch partition, from where it will not be unnecessarily backed-up. Such files are certainly academic, but it isn't necessary to store them in home directories.
  • Music and videos, etc. should not be stored in home directories, because they are both 'large' and not academic. Please store such files in your Scratch partition
  • Archives of personal documents, although not academic, are not considered 'large' and so may be stored in home directories. Please consider encrypting private files before entrusting them to our system.

Client hard disks

Client hard disks on the Apple desktop system are divided into two partitions: System HD which contains system files and applications; and Scratch which is a free-for-all, not-backed-up data partition. Scratch partitions are usually large (hundreds GB) and there are very many of them (one on every client).

Scratch partitions are primarily intended for (a) processes which need very rapid I/O response that the network-mounted home directories cannot provide, and (b) for local storage of large data-sets that do not need to be backed up. Do note, that disks can easily crash, and scratch-partitions can be easily wiped by support-staff without notice (though we try not to...), and if you have any concern at all about the safety of your data, you should probably put it in your home directory, or if it is too large, speak to support for advice.

However, since we have copious quantities of scratch space, there is no particular problem if you wish to keep large collections of music or video on the scratch space of your desktop machine (provided you respect copyright laws, etc).

Scratch space can be found on each client machine at:

/Volumes/Scratch

Groups shared space

The Apple system has a large, shared backed-up collaboration space called Groups, It can be accessed either via AFP or SMB and is shared from the Theoretical Physics file server in the Groups mount point. To access it from an Apple system in Finder click Go > Connect to Server from the menu bar at the top of the screen and type the following:

This space is particularly useful for files that need to be accessed by multiple users, since these users will typically want to access the files from different machines.

Cluster systems

The Theory cluster systems have tens of TB of commonly-mounted disk space. This space is intended for the (often very large) data sets that calculations either use or produce; it is not intended for the long-term storage of data.

Categories: Services | Theory