Job Limits

Each LC platform is a shared resource. Users are expected to adhere to the following usage policies to ensure that the resources can be effectively and productively used by everyone. You can view the policies on a system itself by running either:

joblimits
news job.lim.MACHINENAME

Web Version of Tioga Job Limits

Tioga is an CORAL2 early access system. There are 2 login nodes and 24 compute nodes in a pdebug partition. The compute nodes have 64 AMD EPYC cores/node, 4 AMD gfx90a gpus/node, and 512 GB memory per node.

Jobs are scheduled with FLUX. There are two general use scheduling pools:

  • pdebug - 8 nodes
  • pci - 4 nodes

Scheduling

Pools           Max nodes/job          Max runtime 
--------------------------------------------------- 
pdebug                  1                12 hours 
pci                     4                 2 hours 
---------------------------------------------------

Do NOT run computationally intensive work on the login nodes. There are a limited number of login nodes which are meant primarily for editing files and launching jobs. A majority of the time when a login node is laggy, it is because a user has started up a compile on that login node.

Pdebug is intended for debugging, visualization, and other inherently interactive work. Do not use pdebug to run batch jobs.  Do not chain jobs to run one after the other. Individuals who misuse the pdebug queue in this or any similar manner may be denied access to running jobs in the pdebug queue.

Pci is intended for CI/CD testing jobs originating from GitLab pipelines. It is not for production or debugging work. Do not use pci to run batch jobs. Individuals who misuse the pci queue in this or any similar manner may be denied access to running jobs in the pci queue. Technical limits are in place to restrict the total number of nodes that a given user may use across all jobs within pci. Large parameterized test suites may therefore submit many jobs without interrupting other users.

Documentation

Contact

Please call or send email to the LC Hotline if you have questions. LC Hotline | phone: 925-422-4531 | email: lc-hotline@llnl.gov
Zone
CZ
Vendor
HPE Cray
User-Available Nodes
Login Nodes*
2
Batch Nodes
0
Debug Nodes
30
Total Nodes
32
CPUs
CPU Architecture
AMD Trento
Cores/Node
64
Total Cores
2,048
GPUs
GPU Architecture
AMD MI-250X
Total GPUs
128
GPUs per compute node
8
GPU global memory (GiB)
128.00
Memory Total (GiB)
12,288
CPU Memory/Node (GiB)
512
Peak Performance
Peak PFLOPS (CPUs)
0.064
Peak PFLOPs (GPUs)
5.760
Peak PFLOPS (CPUs+GPUs)
5.824
Clock Speed (GHz)
2.0
OS
TOSS 4
Interconnect
HPE Slingshot 11
Scheduling Policy (main batch queue)
node-scheduled
Scheduler
Flux
Program
ASC, M&IC
Class
ATS-4/EA, CORAL-2
Year Commissioned
2022
Compilers

See Compilers page
Documentation

El Capitan Early Access Systems Documentation on LC Confluence