Here are a number of the more popular tools you'll find on LC systems for moving data between different destinations.

Tool Description When to Use How to Learn More
dcp CLI tools developed at LLNL for moving data between two Lustre filesystems on the same host. Uses the MPI protocol and is part of a suite of tools called mpiFileUtils. When you need to move a large amount of data from one Lustre file system to another.
  • module load mpifileutils
  • dcp --help
  • mpiFileUtils (formerly PFSTools)
ftp Standard Linux/Unix CLI tool for moving data between systems and between LC systems and HPSS. Neither the data nor authentication are encrypted. The ftp protocol is one of the older internet protocols and is sometimes used for transferring non-sensitive data between sites. The version on LC systems does parallel transfers to or from HPSS and between LC systems. When moving a small number of large files (> 256 MB) into or out of HPSS, or between systems.
hopper A GUI tool developed at LLNL and available on all LC systems that provides an "umbrella" interface to all of the tools listed here, plus more. Whenever you prefer to use a GUI for managing your data.
hsi CLI tool provided with HPSS, the Hierarchical Storage Interface (HSI) communicates with HPSS via a UNIX shell-style interface that makes it easy to transfer files and manipulate files and directories using familiar UNIX-style commands. HSI supports recursion for most commands, as well as csh-style support for wildcard patterns and interactive command line and history mechanisms. When you need advanced features or you want to use one of the standard HPSS tools for interacting with your archived data.
htar CLI tool provided with HPSS for moving data between LC clusters and HPSS. Similar to GNU 'tar' in how it works, except the archive file lives in HPSS instead of on local disk. This is the fastest way to get data into HPSS, especially when working with small or medium sized files. When copying a large number of small to medium files from an LC system into HPSS, especially if you want to later retrieve those files efficiently.
nft Scriptable CLI tool developed at LLNL for moving data between LC systems and HPSS. Provides an interface to the Endeavor server, which offers persistent transfers that will continue to try even if hosts, file systems, or the storage system goes down. When you want to "set and forget" a transfer into or out of HPSS.
rsync Standard Linux/Unix CLI tool that copies files either to or from a remote host, or locally on the current host. Most useful for synchronizing two directories by comparing size, attributes, and or contents of the source and sink files before deciding if it needs to be copied. When synchronizing directory trees between different hosts or different filesystems.
  • man rsync
sftp / scp Standard Linux/Unix CLI tools for securely moving data between systems. Based on the SSH protocol, so data and authentications are encrypted. When moving data between a desktop and LC system, or between LC systems, and you want it to be encrypted in flight.
  • man sftp
  • man scp