LC Hotline: 2-4531

From offsite: (925) 422-4531

 

Hours

Monday–Friday
8am–12pm, 1–4:45pm
B453 R1103 | Q-clearance area

 

Technical #494: OCF CZ and SCF Lustre File System Expansion

Published on: 
September 16, 2014

Livermore Computing (LC) recently expanded the capacity and network bandwidth of three of its Lustre file systems—lscratchd, lscratch6, and lscratch7.

In the Open Computing Facility (OCF) Collaboration Zone (CZ), the /p/lscratchd file system capacity was expanded to 5.7 PB, and network bandwidth is now 90 GB/s. In the Secure Computing Facility (SCF), both /p/lscratch6 and /p/lscratch7 capacities were expanded to 5.7 PB, and network bandwidth is now 90 GB/s. 

The following tables summarize the current capacity and network bandwidth of each Lustre file system and maximum Lustre bandwidth between each file system and computing system.

OCF File System Bandwidth (GB/s) Capacity (PB)
lscratchrzb 18 1.2
lscratchd 90 5.7
lscratche 90 5.7
lscratchf 36 2.4
lscratchv 106 6.7
SCF File System Bandwidth (GB/s) Capacity (PB)
lscratch1 850 53
lscratch3 90 5.7
lscratch6 90 5.7
lscratch7 90 5.7
OCF Maximum Lustre Bandwidths (GB/s)
OCF System (CZ) lscratchd lscratche lscratchf lscratchv
Ansel 12 12 - -
Aztec .125 .125 - -
Cab 48 48 - -
Catalys - - 36 -
Edge 10 10 - 10
Herd 1.25 1.25 - -
OSLIC 3 3 3 3
Sierra 48 48 - -
Vulcan - - - 108
OCF System (RZ) lscratchrzb
RZCereal 10
RZMerl 12
RZGPU 12
RZSLIC 3
RZuSeq 16
RZZeus 10
SCF Maximum Lustre Bandwidths (GB/s)
SCF System lscratch1 lscratch3 lscratch6 lscratch7
CSLIC 3 3 3 3
Graph - - 15 15
Inca - - .125 .125
Juno - - 40 40
Max 48 - - -
Muir - - 40 40
Sequioa 850 - - -
Zin - 90 - -

Maximum bandwidth limits are determined by either file system disk size, network/Lustre router speed, or nodes (i.e., single node job scheduling). 

Users are reminded of the following:

  • When moving files between Lustre file systems, always log in to the SLIC system on that network (OSLIC, RZSLIC or CSLIC).
  • MyLC has a portlet called “scratch file system usage” that shows current Lustre file system usage. Just type mylc in the address bar of any web browser to access MyLC.
  • Lustre file systems are not backed up. Users should copy important files to archival storage. Files older than 60 days are subject to purge without notice.
  • Subverting the purge policy by touching all of your files will result in a loss of compute privileges.
  • Performance degrades as a file system fills up. Deleting or archiving files not in use will keep the file system performing well. 

PDF of TB494 for download and distribution.