File Storage¶
Determine Disk Space Quotas¶
Use the following command,
$ showquota
results in output similar to:
Filesystem MB used quota files fquota
/home 3105 5000 44537 0
/work 133 0 33849 4000000
/project 259782 300000 1676424 4000000
See system specific quotas here
Notes on Number of Files in a Directory¶
All users should limit the number of files that they store in a single directory to < 1000. Large numbers of files stored within a single directory can severely degrade performance, negatively impacting the experience of all individuals using that file system.
Home Directory¶
10 GB
Files can be stored on /home permanently, making it an ideal place for your source code and executables. In the meanwhile, it is not a good idea to use /home for batch job I/O.
Work (Scratch) Directory¶
Please note that the scratch space should only be used to store output files during job execution, and by no means for long term storage. Emergency purge without advanced notice may be executed when the usage of disk approaches its full capacity.
For all HPC clusters, no disk space quota is enforced on /work (/scratch) but we do enforce a 60-90 days purging policy, which means that any files that have not been accessed for the last 60-90 days will be permanently deleted. Also, a quota for the total number of files is 4 million.
Project Directory¶
There is also a project volume. To obtain a directory on this volume, an allocation request needs to be made. Allocations on this volume are for a limited time, usually 12 months. This volume uses quotas and is not automatically purged. An allocation of 100 GB can be obtained easily by any user, but greater allocations require justification and higher level of approval. Any allocation over 1 TB requires approval from the allocations committee. Since this is a limited resource, approval is also based on availability.
Disk/Storage troubleshooting¶
Often times, full disk partitions are the cause of many weird problems. Sometimes the error messages do not indicate an unwritable or unreadable disk, but the possibility should be investigated anyway.
Checking the Filesystem¶
Using the df command, a user may get a global view of the file system. df provides information such as the raw device name, mount point, available disk space, current usage, etc. For example,
dfx -hT
Filesystem Type Size Used Avail Use% Mounted on
tmpfs tmpfs 126G 4.0K 126G 1% /dev/shm
/dev/sda2 ext4 864G 12G 808G 2% /
homeserver:/home nfs 12T 2.3T 9.1T 20% /home
cluster26lustre:/cluster lustre 1.8P 1.3P 585T 68% /ddnA
The du program will calculate the amount of disk space being used in the specified directory. This is useful if a user needs to find some offending directory or file. It is often the case that a user exceeds his quota because of a small number of large files or directories.
Potential Issues¶
The Partition in Question is Full¶
Ensure that the directory that you are trying to write in is not full. Contact us if are any directories are full
Temporary directories are cleaned up automatically on job end.
/tmp is Full¶
A lot of programs use this partition to dump files, but very few clean up after themselves. Ensure that this directory is not full.
/var is Full¶
A lot of system programs use this partition to store data, but very few use the space efficiently or clean up after themselves.
Are directories backed up?¶
Generally a user is responsible for their own backups. See our storage policy