Added a "structured copy" option that preserves the structure of the directory tree containing the files to be copied.
Added a "Connect to Prod Host" menu item to the "Connect" menu to conveniently connect to clusters and specific login nodes.
Introduced a new, minimal transfer status display to the bottom of the directory windows. Now one can see at a glance how many active, failed, and successful jobs have been launched.
When run on LC production hosts, Hopper now automatically adopts your other jobs running in the same LC zone (CZ, RZ, SCF). This job information is available both in the minimal and the full transfer status displays.
Directory window operations are now concurrent. Operations that previously tied up the entire user interface now tie up just a single directory window.
Significantly enhanced the file transfer choice algorithm, which reduces the number of two-hop transfers when working with multiple zones and/or multiple users.
Made detailed file transfer window more memory efficient.
Decreased memory footprint of "Show Clipboard" dialog.
Decreased memory footprint of audit dialog.
New automatic directory refresh capability.
Added a ping of prospective Hopper server hosts for availability before attempting to connect.
Can now make HSI connections to LANL and Sandia HPSS storage systems (hpss.lanl.gov and smss.sandia.gov) on the SCF.
Modified server timeout behavior so that an attached client does not prevent server termination after a prescribed amount of server inactivity.
Fixed issue preventing TAR file updates from completing successfully.
Chopper: added aggregated copy and multi-volume copy commands.
Added ability to filter file transfer logs in order to view error messages only.
Fixed issue preventing NFT file transfers from being aborted properly.
Fixed memory leak that could crash long-running Hopper servers.
Significantly enhanced support for Enclave (multi-zone and multi-user) operations.
Improved speed of status database when operating on large number of files.
Added more per-user disk usage views. View usage by directory, by users, or for a specific user per directory.
Added ability to convert format of text files.
Improved performance of client/server communications.
Reduced password prompts for desktop operations involving RZstage.
Reduced password prompts for access to RZFIS.
Enhanced disk usage help information.
Reduced amount of memory used for job descriptions.
Added preference for disabling use of POD.
Added ability to view disk usage (DU) for multiple users, such as for a shared project directory.
Fixed problem that could result in excessive memory use when creating an HTAR archive.
Fixed DU error that could result in items not being counted during a scan.
Fixed DU error that affected scans run on production machines if the scans were started from the root of any LC shared directory.
Fixed Windows-specific DU error that affected scans that were started from the Windows root directory.
Removed retired hosts and added new hosts to site file.
Added Multi-Volume Archive help information.
Fix for error that occurs when transferring files to/from some non-LC FTP servers.
Added workarounds to the file transfer logic in order to accommodate an FTP client that hardwires user names when connecting to storage.
Fixed problems with file transfer logic that occurred when initiating transfers from the EN between CZ and RZ/IZ.
Fixed race conditions between threads that could occur when parsing directory list output.
Can now handle an unexpected date/time format in the directory listing returned by a Windows-based FTP server.
Fixed an array-out-of-bounds exception caused by an update to the directory window in detailed view made by a thread other than the event-dispatch thread.
Improved management of jobs in order to correctly handle multiple user names per user.
Added multi-volume archive operation; a convenient way to transfer large directories to storage.
Better usage of existing host connections. Previously, Hopper required separate connections for the directory window and for each simultaneous transfer involving that host. Now, a single connection can be used for both a directory window and a single transfer as long as they don't both need to use the connection simultaneously. This can result in fewer password prompts and better interactivity.
Many usability improvements related to HPC Enclave, including fewer (and better defined) password prompts, particularly when connecting to RZFIS and RZstage from desktop machines.
Transfers with HSI now make use of HSI's multi-file parallelism.
It now takes significantly less memory to internally represent directory listings.
Better file locking code for the file-transfer status database.
Removed limit on size of search results.
Numerous bug fixes, optimizations, and general improvements.
[Mac] Removed dependency on PPC version of JavaApplicationStub that prevented start-up on OS X 10.7.
Fixed issue where output from some operations could leave directory window in non-functioning state.
Disabled the use of SSH ControlMaster for Hopper SSH-related connections that do not need it.
Changed site file so CZ machines are not treated as production machines within the RZ .
Support for RZstage host, including ControlMaster connections via JSFTP to minimize authentications.
Added retry logic when some status database operations fail due to file system glitches.
No longer allow NFT transfers when source and sink user names are not the same.
Enhancements in support of HPC Enclave (CZ/RZ).
New Multi-Volume copy operation automatically breaks very large transfers into manageable TAR/HTAR archives.
New history editor function allows users to remove/arrange various cached values.
Converted message exchange from XML to serialized objects.
Allow aggregated copy operation in more cases (e.g., machine to machine).
New autosave preferences, configuration information, history, etc.
Now use built-in splash screen for faster startup.
Improved memory usage for directory parsing.
Improved error and status reporting for aggregated copy, sync, and TAR/HTAR.
Removed inverse video for all directory views.
"Show Clipboard" dialog now resizes correctly.
No longer encounter errors when operating on many search hits.
Fixed bug in Connect to Remote that could inappropriately disable Connect button.
User name now restricted to login name when connecting with HPSS, HSI, NFT.
Updated to latest HPSS shared library.
Various improvements and bug fixes in handling of TAR and HTAR files with unusual contents.
Fixed bug causing non-optimal selection of server host. Result to the user could be unexpected password prompt.
Extensive improvements to job handling, including ability to monitor jobs from multiple clients, nearly instantaneous job adoption, and the ability to easily resubmit previous jobs. This is accessible by clicking on the "More Details" button in the Transfer Manager dialog.
"Copy Here (with options)" added to drop menu. This contains some of the less frequently used copy operations.
Added simple synchronization option with copy (via "Copy with Options").
Added ability to transfer remote TAR and HTAR files to a desktop and automatically expand them.
Added copy-with-rename operation (via "Copy with Options").
Added duplicate operation (via the right mouse menu).
Added checksum option to the synchronize operation.
Optimized handling of diagnostics messages between client and server.
Added built-in viewer capability to the Content Mode's preview pane. Now can preview text and image files directly.
Added real text preview to text icons in Content Mode.
Improved robustness of Java FTP authentication.
Improved Endeavor job handling so that jobs are queued immediately.
Can now reconnect to Endeavor server after network interruptions or server restarts.
Improved memory use when viewing contents of TAR or HTAR file. Now can open arbitrarily sized archives.
Added "Copy if needed" option to copy only if the source file was changed or added.
Fixed problem causing bad password to be re-used in certain circumstances.
DuManager no longer kills server if can't create /nfs/tmp2 subdirectory.
Fixed display problem with Terminal Manager on Mac.
Fixed handling of asterisks in file names with FTP file system.
Fixed problems handling HTAR files with bad indices.
Fixed problem on Windows when using SCP with the preserve timestamp option.
Opening HTAR when storage is down no longer throws an exception.
Now display proper size for files >4 GB on 32-bit systems.
Searching by permission or by date no longer throws exception.
Reordering columns in transfer manager dialog no longer throws exception.
Fixed problem preventing aborting of some aggregated copies.
Connection problems between client and server no longer cause server to exit.
Aggregated copy no longer throws exception if source files are deleted during process.
Clicking on search hit with 1000s of lines of context no longer hangs.
Displaying du info for empty dir in tar/htar archive no longer throws exception.
Fixed sync issue with multiple "cache" or "hidden files" toolbar buttons.
Added toolbar editor functionality. Now you can personalize the toolbar by choosing the button operators as well as their locations. Several new operations are now available via the toolbar.
Added two new buttons to the default toolbar: "search" and "new window".
Added new copy option called "Copy with htar expansion", available when transferring from storage to a production host. This acts like a normal copy except any HTAR archives will automatically be expanded in place. If you want to completely expand an HTAR archive, just drag the HTAR file from storage and choose "Copy with htar expansion". This will cause the contents of the file to be expanded in the directory on which you dropped it.
Added support for "Give" and "Take" on LC production hosts. To give files to another user, highlight the file(s) and use the "Give Files..." option from the right-mouse menu. To take files another user has given you, bring up the right-mouse menu when over a directory icon or white space in the Hopper directory window and choose "Take Files...".
Added "sort by suffix" option in Hopper's tabular view.
Now can open arbitrarily large HTAR archive.
Now have ability to manually edit a "launch" command just prior to launching it on selected items.
Increased max number of unique directories displayed in the disk-usage bar-chart view.
When viewing a remote file, now limit the file size retrieved to a local temp directory.
When using the backup feature of synchronize, backup dirs are no longer created if no files are backed up.
Fixed problem when connecting to Mac from Windows caused by changes in OS X 10.6 SSH behavior.
Fixed problem caused by changes to the TAR utility under Mac OS X 10.6.
Added Disk Usage view. Click "pie chart" icon in toolbar to open the disk usage pane.
Various minor bug fixes.
Windows: Fixed tar and copy related problems dealing with drive-level folders.
Windows: Now support direct SFTP connections and transfers.
Fixed issue caused by putting top-level directory in clipboard.
Now check for existence before creating an htar/tar/jar/zip archive.
Now display more error output when a user-defined Launch Command fails to run.
Sped up searches in built-in viewer.
Fixed problem dealing with embedded spaces in directory names with FTP connections on Mac OS X 10.6.
Now remember column configurations between sessions when in detailed (tree) view mode. This includes column sequence and widths.
Added Aggregate Copy feature for efficiently storing huge directory trees.
Added sort-by-type option for content view.
Added timestamp-preserving copies to storage.
Removed file size limit for built-in viewer. Now file size is limited by temporary disk space available for index file.
Default for emptying Hopper storage trash can is now 30 days. Items older than 30 days old will be removed the next time Hopper is run.
Now handle case where directory from which Hopper was started is deleted.
Now prevent Endeavor connections to non-suitable hosts.
Now check for missing DISPLAY variable in Hopper start-up script.
Now display proper file count when transferring files with HSI.
Now can connect to remote servers that use old /bin/sh.
Fixed problem causing remote tar file creation on UNIX hosts to fail in some circumstances.
Fixed issues related to making FTP transfer of top-level folder on Windows.
Fixed rare problem causing active search and transfer jobs to hang after adoption.
Now remember column configurations between sessions when in detailed (tree) view mode. This includes column sequence and widths.
Fixed problem causing some two-hop transfers to fail if a storage proxy host was being used.
Now can connect with ssh to MacOS X 10.6 system.
Simple recursive copy no longer terminates if it encounters a bad file.
Worked around problem with IBM java causing spurious end-of-stream errors.
Adopting huge jobs no longer hangs up.
No longer get spurious second abort request following an initial abort.
Fixed array out-of-bounds problem when dealing with single character paths in Windows.
Fixed issue from desktop that would report "copy operation not supported" for some 3rd party transfers.
New directory window view, "Content View". Files are represented with icons based on their type (source files, binary, PDF, etc.). Image files are represented with thumbnails.
Added automatic retry logic to FileSync. If there are failures (with either rsync or built-in) detected during a FileSync, then the entire synchronization will be repeated in an attempt to pick up anything that was missed due to the failures.
Added code to FileSync to prevent a user from performing simultaneous identical synchronizations.
A number of changes were made to HpssFileSystem and HpssThread to improve behavior when Hopper is unexpectedly disconnected from HPSS.
Sync operation no longer creates zero-length files when underlying transfers fail in unusual way.
Fixed problem causing rare, spurious NPE on Hopper start-up.
Fixed various problems that might cause an exception when working with the Hopper help package.
Starting two clients simultaneously no longer generates exceptions during subsequent operations.
Exiting client while an abort operation is underway no longer generates an exception.
Closing confirmation dialogs with the window manager, rather than clicking "yes" or "no", no longer generates an NPE.
Now check that user has account before launching server on Slic node.
Improved renewal of Kerberos credentials.
Improved output from running built-in sync in dry-run mode.
Fixed bug causing search function in the Help package to throw an exception.
Fixed OOM during multiple built-in file syncs of large directories.
Fixed exception caused by SFTP returning null link target.
Fixed rare OverlappingFileLockException problem when adopting jobs.
Fixed memory leak in client.
Now print site file being used in client debug file.
Now detect and recover from deletion of sshctlr executable.
Improved error message when trying to overwrite a dir with a file by the same name (UNIX).
Reworked logic in directory parser in ways that minimize memory usage.
Fixed retry logic for FTP connections to servers that don't accept "ls -al".
Aborting Unix operations now occurs more promptly.
Fixed bug causing some Lustre dir listings to hang if another file system was down.
Made Endeavor disconnect improvements.
Added ability to specify port number in chopper connection.
Fixed bug causing built-in viewer to fail when file-size preference was set to infinity.
Added preference for emptying Hopper's HPSS trashcan (File->Preferences->General->HPSS).
Now better handle unexpected Endeavor (NFT) disconnections.
Sync operations with htar archives as the source now behave properly.
Add line number option to viewer invocation in Search window. (Now can start up editor on exact line of a search hit.)
Now renew kerberos credentials automatically.
Now handle symlinks properly on Mac Leopard (10.5) systems.
Built-in viewer now recognizes more types of images, particularly newer jpeg formats.
Fixed bug affecting some long-running indexed tar archive operations.
Now treat .war and .ear files as jar files.
Improved searches involving dates on Mac.
Added built-in file comparison package for comparing and merging text files; user preference controls whether built-in or external comparison tool is used. Initiate comparison by dropping one file on top of another.
Added complete Help package, accessible via both Help menu and Help icons on various dialogs.
Added search capability to built-in viewer.
Set timestamps immediately with sync when possible.
Change default so dot (hidden) files not shown.
Made preferences pane non-modal.
Now include user name in password prompt dialog in more cases.
Now included FTP-related transfer rate information in diagnostics.
Improved default file viewers for Windows.
Improved diagnostics for HPSS-related errors.
Added HTAR extraction queue.
Added HTAR creation queue.
Added HTAR timeout to avoid indefinite hangs when HTAR fails.
Simplified connection process for HPSS and FIS from desktops.
Simplified Hopper trashcan naming conventions.
Extended trashcan to work with HPSS regardless of connection type.
Improved recovery from unexpected HPSS downtimes.
Fixed out of memory error from adopting a huge job.
Fixed problem preventing aborting of adopted jobs.
Added "check for updates" and "provide feedback" entries to Help menu.
Added more feedback during multi-hop transfers.
Added feedback to "Adopting Jobs" dialog.
Fixed numerous problems with remote tar creation.
Now quickly abort two-hop transfers.
Now promptly kill HTAR process when create operation is aborted.
Now can search for a file name that is a number.
Now properly mark search context lines that contain a colon.
Now handle new 'h' identifier in latest GNU tar listings.
No longer exit if directory in which server was started is deleted.
New code dependencies now require at least Java 5 to run. Java 6 is preferred. Check version with "java -version".
Fixed bug that prevented sync operations to ftp-connected hosts (e.g. between a desktop and LC storage) from functioning properly. This did not affect syncs between LC production hosts and storage.
Now optimize transfers between production hosts and FIS.
Now retry when experiencing HPSS connection problems.
Now recover from unusual Endeavor disconnections.
Significant expansion of synchronize capabilities, including new options to ignore existing sink files (i.e., only transfer missing files), to only consider files that are present on both the source and sink, or to only use the size when comparing files (i.e., ignore timestamp).
Added size-only comparison for sync operation (i.e., ignore timestamp).
Added sync option for only transferring missing files.
Now ignore seconds in sync comparison if connection protocol doesn't support it.
Added YY/MM/DD to job debug info.
Added support for preserve-timestamp option to underlying copy mechanism.
Added info icon buttons to job status dialogs.
Now handle zero-length searchRuleSets, historyRev1 and hostmarks files.
Now include date with start/end times in transfer monitor.
Now can show actual seconds in long list if underlying protocol supports it.
Improved performance of directory parsing logic.
Fixed directory parsing issue on Mac Panther systems.
Improved error messages caused by unexpected SFTP output.
Improved robustness of terminal interaction code.
COS now always set for large files written to storage from desktop.
FTP and HSI child processes now always terminated if transfer aborted.
Now can access search hits even when adopting job from another host.
Syncs from storage to local host no longer transfer more than necessary.
Server no longer exits if user prematurely terminates client.
Server no longer exits if sync job aborted immediately.
Server no longer exits if aborting an SFTP transfer.
Now can use SFTP on Fedora Core systems.
Bad SFTP connections no longer use CPU time.
Strange SFTP progress messages no longer generate warnings.
Various other bug fixes.
Now support direct-from-tape htar extractions when reading small number of files from large archive. (Significantly faster.)
Now support transparent use of storage proxy host such as oslic and cslic.
"No locks available" NFS bug no longer causes permanent hang; server now recovers once locks are freed.
Now support hiding of dot files when viewing archives.
Improved diagnostic messages in log files.
Minor changes to chmod GUI to reduce confusion.
Numerous HSI reporting improvements.
No longer have to scroll menu to see all search criteria options.
Streamlined display of transfer information in client to remove memory issues.
Various other bug fixes.
Completely revamped and simplified search interface, with new ability to graphically construct search queries. File searches now support date and age options, in addition to name, permissions, owner, group, size, type, and content. Content searches now support optional lines of context.
Streamlined job adoption process to use significantly less time and memory.
Sped-up and simplified all SSH-based logic as well as invocation of remote commands.
Added full support for HSI utility, including connections to storage and transfers to/from storage.
Resolved server out-of-memory errors caused by huge transfer and search jobs.
Resolved client out-of-memory errors caused by huge transfer and search jobs.
Restored ability to scroll through search hits using up/down arrow.
Server now exits cleanly when a thread encounters a fatal error.
Improved detection and handling of server failures in client. Now, if server dies all existing jobs will exit cleanly and new server will be started.
Added auto-cleanup of Hopper-created log files, tmp directories, etc.
No longer can copy a directory onto itself or descendant.
Removed occasional extra password prompt when running rsync on SCF.
Added trashcan for deletes in HPSS from HSI connections.
Added COS setting for streaming tar files created in HPSS.
Improved reporting to client of server errors.
Fixed problem causing rsync between production and non-production system to fail.
Fixed rare but recurring "no locks available" error in server.
Fixed HTAR extraction problem when archive was created using "./" notation.
Various other bug fixes.
Better handling of dirs with spaces during archive extraction (cmdwrapper).
Verify existence of status db lockfile before using.
Improved handling of archives containing odd characters in name.
Fixed problem with using ' <' operator in "find" mode.
Now print unexpected statusdb server errors to log file.
Optimized server choice when adopting jobs.
Various bug fixes in: indexed tar archive creation, ftp handling of unusual characters, Endeavor server choice.
Revamped architecture based on client/server model that (1) allows client to disconnect while server continues to perform operations and (2) significantly reduces the number of two-hop transfers involving production hosts.
Added "favorites" capability for choosing favorite connections and directories. With the associated autoconnect option users can do things such as automatically open a window to a certain directory in storage when Hopper starts up.
New "smart" mode for automatically selecting transfer technology. This mode, the new default, will be optimized for both speed and minimization of authentications based on knowledge of the source and sink host.
Added ability to operate on search hits. Users can drag and drop search hit items or choose operations from a pop-up menu including copying to clipboard, deleting, changing permissions, viewing, etc.
New ways to perform data aggregation during a transfer: users can create a tar or indexed tar file on a remote host from local data, even when running on desktop systems. This aggregating style of transfer is up to 10 times faster than copying files individually with FTP.
Support for comparing any two files, even if on different hosts. Initial version supports use of external diff tools, such as xdiff, on Unix hosts; future version will also have a built-in diff tool for use on any platform.
Added pre-authentication logic to guarantee that a job has all necessary information before allowing Hopper client to exit.
Autoscrolling enabled when extending selections in directory windows.
Comprehensive audit capability.
Simplified process for getting into 'rename' mode. Now can select file and hit RETURN.
Added "clear cache" shortcut to the Connect menu.
Added logic to validate a cached connection before re-using it.
Improved and simplified preference layout and internal representation.
Added client-side logging.
Added support in the hopper wrapper script for executing arbitrary versions of Hopper using the -v flag.
Added support for <,>,<=, and >= operators when examining file size in the "find" option of the search operation.
Improved error message when trying to perform a GNU tar operation on a machine that does not have GNU tar.
Added symlink support for FTP connections where client and server both support it.
Added support for following symlinks when creating tar file.
Added support for HSI utility.
Added ability to change permissions and group through jftp connections (e.g., on Windows).
Added support for dual-copy class of service for writes to HPSS.
Reduced memory use when handling large transfer and search progress messages.
Fixed synchronize bug causing incorrect modification time for files on Windows.
Fixed synchronize bug causing created directories to have wrong mod time.
Fixed bug causing xdiff to be launched without appdefaults file (Java bug).
Modified LDAP connection code to retry in case of slow ssl handshake.
Fixed bug causing Hopper window to become unusable if underlying working directory was deleted.
Fixed bug preventing zip archive extraction on Windows.
Fixed failover issues with HPSS connections.
Force garbage collection when clicking on memory usage icon.
Sped up job adoption from remote server.
Revamped server start-up logic to use command-line ssh if available.
Sped up htar extractions.
No longer select a directory when expanding it in tree view mode.
Allow users to choose transfer method for synchronize operation.
Prettified copy choice menu.
Worked around ftp problem caused by filenames with leading dashes.
Fixed hang caused by transferring zero-length file with scp.
Fixed bug causing server to not terminate after completing last job.
Fixed bug causing error messages during deletion from zip archive.
Fixed bug causing synchronize-with-backup option to sometimes fail on Windows.
Fixed problem when aborting tar file operations.
Fixed problem caused when using a timed-out Endeavor connection.
Fixed jftp exceptions.
All client actions are now logged in the audit window.
Error status now cleared when status button clicked.
Significantly sped up GUI updates when reporting large amounts of information (e.g., lots of paths).
Defined copy-choice sequences via the Site file to allow easy tweaking during deployment.
Changed file name escape method for FTP connections to deal better with unusual characters.
Fixed bug preventing searches of HPSS to properly detect tar and htar files.
Fixed bug causing timeout when using FTP non-passive mode to connect to a host that requires passive mode.
Fixed bug in the "locate" option of the search operation that caused file content searches to have incomplete labels if only one match was found.
Fixed bug that could cause incorrect file count to be reported during synchronize operation.
Fixed bug when using rsync on remote hosts with --include or --exclude option.
Fixed bug causing NPE if running htar at a site without htar support.
Fixed bug in EndeavorFileSystem preventing creation of relative symlinks.
Fixed bug in FtpFileSystem triggered by unusual filenames.