Technical Bulletin #491: lscratch Data Movement Recommendations
Livermore Computing (LC) will be retiring a number of older, ldiskfs-based Lustre file systems. The current retirement schedule is as follows:
March 4 lscratch2 retires
April 15 lscratch4 retires
June 2 lscratch5 retires
LC asks that you consider the following prior to transferring your files:
Leave any files you no longer need. There is no need to delete unwanted files.
Use HTAR on CSLIC to move critical files and those requiring long-term storage to SCF archival storage.
Juno, Graph, Muir, and Inca users should transfer files to lscratch7 or the new lscratch6 that will become available in mid-March.
All transfers should only be initiated on CSLIC, which mounts all Lustre file systems and has excellent network bandwidth.
LC recommends using the rsync command or the Synchronize Directory operation in Hopper for copying files from one Lustre file system to another. Using these methods will preserve timestamps, groups, and permissions of the copied files, with an additional advantage being that if the initial transfer fails or is interrupted, the command can be reissued without re-copying data that has already been transferred.
For example, to migrate data from /p/lscratch2/kelly/mydata to /p/lscratch7/kelly/mydata, use either the rsync command or Hopper as shown below.