File Management Guidelines

This section provides information on how file system use should be guided by and can be impacted by policies implemented at LC.

Backup Policy Summary

Because some file systems contain valuable, frequently used information whose loss would be very disruptive, full backups are made once a month, and incremental backups occur every night except Saturday and Sunday. Other heavily used file systems are simply too large to allow practical backup copies to be made, or they are mounted on machines for which no appropriate backup software is commercially available. For example, the Lustre parallel file systems with their multiple terabytes of capacity are not backed up, and there is minimal redundancy in their underlying storage servers. (This means that one hardware failure can make many distributed files unavailable.)

If you have files on these systems, it is your responsibility to make storage copies of all crucial files in case you need to restore them on your own after a disk failure. To easily store very large archives, consider using HTAR, LC's highly efficient software tool designed for this specific task. For more details about using storage, consult EZSTORAGE. For details about using HTAR, consult the HTAR Reference Manual.

This table summarizes the backup status for each major file system on the LC production machines. (Those not listed are not protected by backup.) Files from the four most recent backups of your common home directory (i.e., /g/gnn) are retrievable from your .snapshot directory. The .snapshot subdirectory is not reported by running ls, as the other dot-named children of your home directory are, but you can change directories (cd) into it to list and copy its files.

File System

Backup Status

Automatic Backup
(all machines)
No Backup
/g/gnn X  
/usr/local X  
/usr/gapps X  
/usr/tmp   X
/var/tmp   X
/p/lustre* (Lustre)   X
/usr/workspace/ws*   X

File Purge Policy

File systems at or near their capacity often show degraded performance, higher I/O error rates, or sometimes complete service failure. To make service more predictable and reliable, LC has historically destroyed ("purged") files intentionally on at-risk file systems. LC has moved away from this model in favor of a quota system that limits file system usage. Presently LC does not purge any of its file systems.

Search Paths

When you try to execute a program by typing its (simple) file name, the UNIX shell searches through the file structure looking for a file with that name to execute. The order in which it searches directories is specified by your search path, a colon-delimited ordered list of directories stored in the environment variable PATH (all uppercase). If you use a program's absolute path name, the shell ignores your search path. You can reveal your current search path on any LC machine by executing echo $PATH.