LC Hotline: 2-4531

From offsite: (925) 422-4531

 

Hours

Monday–Friday
8am–12pm, 1–4:45pm
B453 R1103 | Q-clearance area

 

Technical Bulletin #436: HSI—Hierarchical Storage Interface

The Livermore Computing (LC) Data Storage Group (DSG) recently released the utility program called HSI (Hierarchical Storage Interface) on all open (OCF) and secure (SCF) production machines. HSI is a powerful interface for users of the High Performance Storage System (HPSS). It is intended to provide a familiar UNIX-style environment for working within the HPSS environment, while automatically taking advantage of the power of HPSS (e.g., for high speed parallel file transfers) without requiring any special user interaction, where possible.

How to Use HSI

On any LC production system machine simply run:

hsi

Upon startup, HSI will first indicate that it is authenticating your identity, then a line with your user name, uid, an HSI version number, and build date is returned, followed by a question mark prompt.

Features

HSI has many features, including some that are unique among LC HPSS user interfaces:

  • Familiar UNIX-style command interface, with commands such as cd, chgrp, chmod, chown, du, find, ls, etc.
  • Recursive option is available for most commands; including the ability to copy an entire directory tree to or from HPSS with a single, simple command.
  • Interactive “history” and command-line editing.
  • Interactive, batch, or “one-liner” execution modes.
  • Ability to distinguish whether files are resident on HPSS disk cache or on tape (i.e., purged from disk).
  • Ability to automatically retrieve files in tape-order, improving performance for stage/get operations.
  • Ability to change the Class of Service of an HPSS file (typically useful when mission critical data require dual-copy). Ability to interactively pipe data into or out of HPSS, using filters such as “TAR.”
  • Conditional put and get operations, including ability to update based on file time stamps.
  • Automatically uses HPSS parallel I/O features for file transfer operations.
  • Command aliases and abbreviations.

HSI Examples

To put a file with the same name locally as in storage:

? put filename

To put a file in storage with a different name than the local name:

? put localname : storagename

To get a file from storage with a different name than the local nameL

? get storagename : localname

To list a file and determine if it is on disk or tape:

? ls -U filename
-rw-r----- 1 user group 140 6859 TAPE 1073741824 Aug 16 2005 filename

More information is available with the -X option instead of -U. The two lines for storage levels 0 and 1 are most informative. It is possible to observe a large file being staged from tape and to monitor its progress.

? ls -lX filename
-rw-r-----  1  user  group  140  6950 TAPE  1073741824  Aug 16 2005  filename
Storage   VV   Stripe  
Level   Count  Width  Bytes at Level
----------------------------------------------------------------------------
0 (disk)   0       1                                 (no data at this level)
1 (tape)   1       1  1073741824
. . . 

To display the number of bytes contained in all files, and recursively, directories within a specified directory or file name:

? du –k
144414335       /users/u39/username/
-----------------------
144414335       total 1024-byte blocks, 501 Files (147,880,278,678 bytes) 

To conditionally save a file to HPSS only if the local file time stamp is newer than the HPSS file:

hsi put –u filename

To specify a very large number of input files yet avoid an HSI command line too long for the shell to accept:

hsi < input.file

More Information

For details on HSI’s capabilities and usage, see the HSI man page or visit the HSI Web site at http://www.mgleicher.us/index.html/hsi

A brief syntax summary is available in the HSI man pages or by using the command 

hsi -? 

PDF of TB436 for download distribution.

LLNL-WEB-733249