Technical Bulletin #523: Lustre Parallel File System Purge Policy Update
NOTE The purge policy outlined herein supersedes the purge policy in Technical Bulletin Nos. 351, 411 and 492. This updated purge policy will go into effect on all of the /p/lscratch* Lustre file systems on the Open and Secure Computing Facilities (OCF and SCF) beginning May 10, 2018.
Livermore Computing (LC) recognizes that our users' work can be seriously impacted when a file system fills to capacity or near capacity, and users may experience degraded performance and/or receive I/O errors when writing to the file system. To minimize the potential disruption of service, a Lustre parallel file system purge policy was implemented in August 2006 and is updated in this Technical Bulletin.
- Purging of files on the /p/lscratch* file systems will take place on an on-going, as needed basis. Purges are typically triggered when a file system reaches 80% of capacity.
- In an attempt to prevent a file system purge, as a file system approaches 80% utilization the LC Hotline will contact the top users of the file system and request that they remove files in an attempt to keep utilization below 80%.
- If after contacting the top users utilization still reaches 80% a purge will be initiated. Files that have not been accessed for 60 days are eligible to be deleted.
- Intentional subversion of the purge policies of any temporary LC file system (e.g., the use of the UNIX command touch) is strictly forbidden. Users observed intentionally subverting these policies will have their responsible manager contacted and may have their computing privileges suspended until assurances are provided to LC by the user and their responsible manager that the behavior will not continue.
Once files are purged, there is no possibility of recovering them. Lustre file systems are scratch file systems and are not backed up, so any critical files should always be backed up to another medium, such as archival storage.