HTAR Limitations and Restrictions

The current version of HTAR has the following known limitations or usage restrictions.

I/O

You can redirect any HTAR output into a file (with >) for separate postprocessing (see the Retrieving Files section for one helpful application of this), but HTAR normally does not read from or write to UNIX pipes (standard input, standard output). Two HTAR control options, however, let you enable the use of pipes if you need them:

  • Read From Standard Input. Use HTAR's -L inputfile option with a hyphen as the input file (that is, -L -) to read a list of files from standard input instead of from the usual execute-line filelist. The "Too Many Names" discussion later in this section shows how to apply this technique to solve a practical problem when creating archives with very many input files
  • Write To Standard Output. Use HTAR's -O option to write a file extracted with the -x option to standard output. Thus,
 htar -xf abc.tar -O def

 extracts file DEF from archive ABC.TAR in your storage home directory and displays it at your terminal, while

 htar -xf abc.tar -O def | wc

 instead reports DEF's line, word, and character count. Because HTAR does not separate files in the output stream, this usually is useful only when extracting a single file.

Metacharacters

HTAR leaves all processing of metacharacters (filters or wild cards, such as *) to the shell. This means that when you create an HTAR archive you can use * to select from among your local files to store, but when you retrieve specific files from within an already stored archive you cannot use * to select from among the stored files to retrieve. See the Retrieving Files example for details on this limitation and a few suggested ways to work around it. Another side effect of this approach to metacharacters is that C shell (csh) users must type the three-character string -\? (instead of -?) to display HTAR's help message.

Updates

No options exist to update (replace), remove (delete), or append to individual files that HTAR has already archived. You must replace (create again) an entire archive to alter the member files within it.

Name Length

To comply with POSIX 1003.1 standards regarding TAR-file input names, the longest input file name of the form prefix/name that HTAR can accept has 154 characters in the prefix and 99 characters in the name. Link names likewise cannot exceed 99 characters.

File Size

The maximum size of a single member file within an HTAR archive is 68 GB. HTAR's site-imposed maximum size for an archive file is about 10.7 TB, and local disk space (when using -E) or storage space might externally limit an archive's size. Users can specify a maximum number of member files per archive with HTAR's -M option.

Passwordless FTP

Because HTAR (unlike FTP) does not support user dialog with a server and has no password-passing option, you can only manipulate HTAR archives on machines with preauthenticated (passwordless) FTP notrun the PFTP client.

Too Many Names

For users who make HTAR archives containing thousands of files, limitation is of the UNIX shell rather than of the HTAR program itself. One would normally select multiple files for archiving by using a UNIX "ambiguous file reference," a partial file name adjacent to one or more shell metacharacters (or "wild card" filters, such as the asterisk). Your current shell automatically expands the metacharacter(s) to generate a (long) alphabetical list of matching file names, which it inserts into the execute line as if you had typed them. Thus,

htar -cf test.tar a*

might become equivalent to a command line with dozens of a-named files on the end. Each shell has a maximum length for execute lines, however, and if your specified metacharacter filter matches thousands of file names, HTAR's execute line may grow too long for the shell to accept, which would prevent building your intended many-file archive.

The most effective, least resource-intensive way to work around the problem of having a (virtual) HTAR execute line too long for the shell to handle is to plan ahead and keep (or generate) in a single directory all and only the files that you want to archive. HTAR processes directory names recursively by default, so, if you specify only the relevant directory name on HTAR's execute line, HTAR will (internally) archive every file within the directory without any filter-induced length problems. For example,

htar -cf test.tar projdir

will successfully archive any number of files within the "projdir" directory (and use no shell-mediated file-name generation to do it).

The UNIX find utility is designed to produce lists of files (that meet specified criteria) to feed into other programs for further processing and so offers a second way for HTAR to archive very large numbers of files without having a very long execute line. Indirection is required for success, however. Enable find to pipe standard input directly into HTAR by invoking HTAR's -L option with a hyphen (-) argument instead of a file name. The correct sequence is:

find . -name 'a*' -print | htar -cf test.tar -L -

The find -name option generates the list of matching names internally without expanding find's execute line, and the use of the metacharacter * in the execute line does not pose the same too-long problem as it did originally in HTAR's execute line because the surrounding quotes shelter the filter from shell processing. If you need to keep the list of input names (for verification or audit purposes, for example), you could break this single line into two equivalent steps mediated by a helper file (here called "alist") that you preserve.

find . -name 'a*' -print > alist
htar -cf test.tar -L alist