Archival Storage

July 7, 2017.
On Wednesday, July 12, the Livermore Computing Data Storage Group will release a new version of HTAR on all open (OCF) and secure (SCF) production machines. HTAR—a utility that combines a flexible file-bundling tool with fast parallel access to high performance storage—allows users to store and selectively retrieve very large sets of files efficiently. This update to HTAR allows specifying entries to exclude during creation of new archive files.

This page lists available online tutorials related to parallel programming and using LC's HPC systems.

Useful introductory information on LC's software environment is presented in the Software and Development Environment section of the Introduction to Livermore Computing Resources, or the Linux Clusters Overview for system-specific information.

Here are a number of the more popular tools you'll find on LC systems for moving data between different destinations.

The High Performance Storage System (HPSS) is available on all OCF (CZ/RZ) and SCF production systems. Each user has a storage account with passwordless access from LC systems. The current storage quotas can be found by searching for "yearly growth quota" and picking the most recent Technical Bulletin.  The EZSTORAGE manual provides an overview of tools for storing and archiving files.

Trans is a utility from the Cray machines that has been ported to run on the IBM machines. It knows about several forms of files on older machines and how to convert them for use on current machines.