Job Limits

Each LC platform is a shared resource. Users are expected to adhere to the following usage policies to ensure that the resources can be effectively and productively used by everyone. You can view the policies on a system itself by running:

news job.lim.MACHINENAME

Web version of Magma Job Limits

There are 5 generally available machines on the SCF. Please consider your needs to decide which machine is best for your flavor of work.

  • Agate—a 48 node machine for up to 36 core jobs (no interconnect)
  • Mica—a small machine for debugging (8hr job limit)
  • Jadeita—a machine for small jobs
  • Magma—a machine for small to medium jobs
  • Jade—a machine for large jobs (4 jobs/user limit)

Magma is now in General Availability mode and all jobs are scheduled per node in two pools. Remember you can run in the guests bank to get additional resources if there are a lot of empty nodes.

  • pdebug—20 nodes (1920 cores), interactive use only.
  • pbatch—732 nodes (70272 cores), batch use only.
Pools               Max nodes/job       Max runtime
---------------------------------------------------
pdebug                    4(*)            1 hour
pbatch                   64(**)          36 hours
---------------------------------------------------

(*) pdebug is for debugging, visualization, and other interactive work.

  •    Jobs are core scheduled
  •    There is a limit of 384 cores per user per bank
  •    Do not use pdebug for production work or chain jobs
  •    Using more than these limits can result in job removal without notice.

(**) pbatch is for small to large jobs.

  •    Jobs are node scheduled
  •    There is a limit of 150 nodes per user per bank at any one time
  •    There is a limit of 20 jobs per user per bank at any one time

Hardware

  • Intel Cascade 2.3GHz cores
  • Intel Omni-Path Interconnect
  • 96 cores/node
  • 256GB memory/node
Zone
SCF
Vendor
Penguin
User-Available Nodes
Login Nodes*
8
Batch Nodes
732
Debug Nodes
20
Total Nodes
772
CPUs
CPU Architecture
Intel Cascade Lake AP
Cores/Node
96
Total Cores
74,112
Memory Total (GB)
296,448
CPU Memory/Node (GB)
384
Peak Performance
Peak TFLOPS (CPUs)
5,313.0
Clock Speed (GHz)
2.3
Peak single CPU memory bandwidth (GB/s)
282
OS
TOSS 4
Interconnect
Cornelis Networks Omni-Path
Parallel job type
multiple nodes per job
Recommended location for parallel file space
Program
ASC
Class
CTS-1
Password Authentication
OTP
Year Commissioned
2020
Compilers

See Compilers page

Documentation