What is your home directory

Your home directory is a place for you to store all your files and data related to your research. On the clusters, this home directory is shared between all the compute/execution systems and is created when you log in for the first time.

You home directory may be in one of three file systems depending on when your account was created. Newly created accounts will have their home directory in the Ceph file system (/mnt/ceph). Historically, some home directories may be in /mnt/lfs2 (Lustre).

Ceph

New users’ (From Dec 2017 on) home directories will be in the new distributed file system Ceph (/mnt/ceph/). Regardless of where your home directory is located, every user has a /mnt/ceph/username directory, and large amounts of new data should be stored here - as this will be the long-term supported file system.

Lustre

The other distributed file system available is Lustre (/mnt/lfs2/ ). Our Lustre file system uses 15k RPM SAS drives in RAID 6 or 7.2k RPM drives in RAID 10 with Infiniband networking to make it very high performance. The price for all this speed is, well, price. These high speed disks are expensive, so the capacity of the Lustre system is limited, currently about 200TB. The Lustre files system is also called scratch because it is not backed up by default. The servers that compose Lustre all use RAID, and the danger of losing data due to disk failures is low. However, there is no default protection against accidental file deletions or whole server failures. There is an opt-in backup system in place. To mark a directory for backup, create a .backup file in that directory. The easiest way is simply to:

cd /mnt/lfs2/<username>/my_important_files/
touch .backup

The backup script will then know to rsync the contents of that directory. Please note that subdirectories are not backed up. Do not mark data otherwise stored in Ceph, or easily reproduced (such as software), for backup.

Important Note: The Lustre file system is the only distributed file system mounted by all the cluster nodes in the standard partitions. In general, to run analyses on the cluster (not standalones) your scripts and data files should reside in Lustre, and you should submit the job from there. We are working to get Ceph mounted on all the cluster nodes. You can keep your data in Ceph and run your job with the -C "ceph" command line option to request nodes that have the /mnt/ceph directory. For example :

sbatch -C "ceph" myjob.slurm

Summary of RCDS HPC Storage Systems

SystemRedundancySpeedBackup
Ceph
/mnt/ceph/username
GreatestModerateData stored in triplicate, regular backups stored offsite
Lustre
/mnt/lfs2/username
Withstands disk failuresModerateOpt-in backup
Local Scratch
/mnt/local-scratch/
NoneFastestNone

Each home directory contains configuration files associated with specific programs. Some of these include:

.bashrc

This is your configuration file for the bash shell which is read when a user requests an interactive shell. You can change this to add aliases, functions, change your shell prompt, load modules, and run specific programs when you start an interactive shell.

.bash_history

This file is updated by bash upon logout and contains a history of all the commands you have run during your system. Even if you are a big privacy advocate and do not want the admins to see what you are doing, I recommend keeping the file intact, as it is a useful tool for the admins to help debug any issues you may be having.

.emacs

This is your emacs configurations file. Edit this file to change your emacs configuration.

.vimrc

This is your Vi IMproved configuration file. Edit this file to change the behavior of vi/vim.