NOTE The system software on these systems is pre-GA. Although a Non-Disclosure Agreement is not required for access to these systems, LC requests that results from any El Capitan system (Tuolumne, RZAdams, El Capitan, Tioga, RZVernal, and Tenaya) including manuscripts, reports, presentations, and data releases are submitted for review to LC before public dissemination. To submit results for review please email a draft to elcapitan-coe-issues@llnl.gov.

Introduction

This users guide is intended for users of Livermore Computing's El Capitan systems. It includes a quickstart section on gaining access, familiarizing yourself with El Capitan hardware, choosing a compiler, and setting up your environment. The topic of running jobs is detailed, including using Flux (or flux_wrappers for slurm commands), as well as walkthrough documentation for C++ and Fortran code examples. Tips on how to get help and how to stay up to date on system changes follow.

More in-depth information for each topic is provided, and can be found in the pages linked in the left-hand menu.

Level/Prerequisites: Intended for those who are new to the El Capitan environment. A basic understanding of command line use is required. Parallel programming in C or Fortran may be needed. Familiarity with MPI and OpenMP is desirable. The material covered by EC3501 - Introduction to Livermore Computing Resources would also be useful.

Major updates and changes to this documentation will be announced via email to users of the El Capitan, Tuolumne, and RZAdams systems.

Background

The CORAL 2 contract between Livermore Computing and HPE ushers in a new generation of powerful HPC computing systems. El Capitan, at a projected performance of over 2 exaflops, rightfully gets the most attention, but there are several other systems included.

The systems follow a "Yosemite" naming theme and consist of large, long-term production systems as well as early-access (EAS) and test systems. Amazingly, the three EAS systems are each powerful enough to be one of the 250 most powerful systems on the Top500 list even though they have merely one rack of compute blades!

Full production systems

  • El Capitan: Will permanently reside in the SCF once it is fully accepted and may be the most powerful computer system in the world when it enters production.
  • Tuolumne (short name tuo): Will reside in the CZ and should assume the title of the second most powerful computer in LC.
  • RZAdams: The flag bearer CORAL2 system for the RZ.

Early Access Systems (EASs) 

As part of the CORAL 2 contract, several Early Access Systems (EAS) have been sited. Full documentation for these systems can be found in the El Cap EA Systems confluence space (requires login). These systems are still available, including:

  • Tioga: EAS system residing in the CZ
  • RZVernal: EAS system for the RZ
  • Tenaya: EAS system for the SCF
  • Hetchy: an early test/development system

File Systems

Along with the compute clusters, HPE is also providing three storage clusters, named after rivers flowing out of Yosemite or the western Sierras. They will run on Lustre 2.15 to start.  Communication is over the new Slingshot interconnect, providing the fastest Lustre bandwidth to date in LC. Merced will also be the largest file system LC has ever had.

  • Merced (/p/lustre4): The large Lustre file system cluster that is paired with El Capitan.
  • Yuba (/p/lustre5): The Lustre file system cluster that will accompany Tuolumne in the CZ.
  • Kern: A small, early-access Lustre file system cluster paired with Hetchy.