Job Status and Reporting

NFT Job Numbers and Classes

Using Job Numbers

Executing NFT begins a session (namely, session 0), and every NFT request (or job) in that session has a unique integer job number, starting with 1 and increasing sequentially. The NFT server remembers your past jobs for up to 4 days depending on NFT traffic (unless you flush their records with CLR or a session has more than 499 job statuses), and during this interval it will not reuse the numbers of remembered jobs. So frequent NFT users may find job numbers incremented by 1 from their most recent job rather than starting at 1.

You can use each job's unique number to get information on its status in case it does not complete immediately (or at all). For example, to request the status of the job numbered 13, use NFT's RPT option:

User: rpt 13
Rtne: 13.0 error. Cannot clobber existing sink
           /g/g0/abc/test4

See the Job Reporting with RPT section for details on other ways to request status reports.

For the special situation where you have jobs in multiple NFT sessions, with multiple job-numbering sequences, see the NFT Sessions section.


Job Class Hierarchy

NFT classifies all jobs into eight states (or job classes), shown here in their nested hierarchy:

Class

Member Jobs

all

all jobs that NFT remembers, regardless of state *

  incomplete

jobs waiting to run or not finished running

    held

jobs in the scheduling queue

    active

jobs currently running

  complete

successfully or unsuccessfully completed jobs

    okay

successfully completed jobs

    error

unsuccessfully completed jobs

    aborted

jobs terminated by the user

* If you use multiple NFT "sessions," a special practice described in the NFT Sessions section below, then ALL includes only jobs in the currently selected session, NOT in all the sessions that you have created.

You can use NFT's RPT option to request status information for a whole class of jobs as well as for a specific job number (see the Job Reporting with RPT section). The NFT server remembers which class your jobs are in for about 4 days, but if you submit many jobs and want to monitor only the newest and most interesting ones, you can flush or "clear" NFT job-status records (by job number or for a whole class of jobs) using the CLR option.


Job Reporting with RPT

You can at any time review the status of your NFT requests or jobs by using the RPT (report) option. Two related options let you clear NFT status records no longer of interest (CLR) or abort jobs found to be still incomplete (ABT). The NFT STATUS option reports on your environment-variable settings, not on your file transfer jobs.

Scope

RPT by default reports the status of your most recent job (in the currently selected NFT session). But you can specify as RPT's argument a specific job number (n) or a range of numbers (n-m, using a hyphen, not a comma, as separator) to get reports on just the jobs that you select. The Using Job Numbers section above gives an example.

In addition, RPT (with CLR and ABT) understands eight classes of NFT jobs. Using a one-letter code for each job class, you can request status information on all your jobs that belong to the class you select:

Job Class

RPT

ABT

CLR

all

-a

-a

-a

incomplete

-i

-i

 

held

-h

-h

 

active

-x

-x

 

complete

-c

 

-c

okay

-o

 

-o

error

-e

 

-e

aborted

-k

 

-k

Note that while you can report (RPT) on any class of job, you can only abort (ABT) incomplete jobs (held or active) and you can only clear (CLR) the status records of completed jobs (okay, error, or aborted). See the Format and Examples section below.

Besides the job number(s) or job class(es) you select, three other factors are relevant to the scope of RPT status reports:

  1. Record Persistence. The NFT server remembers your jobs, their numbers, and their status for up to 4 days before purging its records. So frequent NFT users will get reports on all members of a job class throughout this time range, not just on the jobs started with their currently running NFT client. If this is a problem, use CLR to overtly delete the old(er) records that are no longer of interest.
  1. Sessions. Most NFT uses have only one NFT session, and RPT reports on the jobs (job class members) in that session. If you start multiple sessions, RPT reports only on the jobs in your currently selected session, ignoring all your jobs in other sessions. This can give you more control of your status reports or lead to confusion, depending on your awareness of the sessions you have started. For details on the effect of multiple sessions, see the NFT Sessions section.
  1. Verbosity. The NFT VERBOSE option lets you specify which state changes NFT reports interactively as it runs (e.g., when jobs begin as well as when they end). VERBOSE does not, however, affect which jobs (or job class members) NFT includes in RPT status reports, nor the amount of detail provided on each job's status line. Thus, VERBOSE does not change RPT's scope at all, although it does change general NFT dialog.

Format and Examples

Most RPT status reports consist of one line per job reported, with the format

jnumFrnum. status info

Example:

12.0. done. ~/nfttest/test4

where

jnum

is the unique integer number NFT assigned to this job.

F

is a flag that indicates either a primary job (.) or a secondary job (/). A primary, user-submitted job such as GET * may generate many secondary jobs, one for each file retrieved.

rnum

is NFT's retry count, which always starts at 0.

status

is a standard status-reporting term (begin, done, error, start, accept).

info

characterizes your specific job, for example, by giving the path name of the file retrieved or of the directory listed.

To illustrate a typical RPT job-status report on a class of jobs, assume that you have started an NFT client and issued these commands:

cd nfttest
dir
get test4
clobber
get test4

Then, if you request a status report on all of these jobs by using

rpt -a

NFT's response might look like this:

11.0. done. ~/nfttest
12.0. done. ~/nfttest
13.0. error. Cannot clobber existing sink
         /g/g0/abc/test4
14.0. done. /g/g0/abc/test4

Note that your CLOBBER command, which does not involve any information transfer between machines, is not treated as a numbered job by NFT and so is omitted from RPT's status report. Invoking RPT's -v ("verbose") option inserts on each reported line the command (action) used as well as the usual status information. For example:

14.0. done. get /g/g0/abc/test4

Diagnostic Verbosity

NFT jobs pass through several states from submittal to completion, and you can control how finely NFT reports on these changes of state by using its VERBOSE option. By default, NFT passes along transfer statistics from the FTP daemon that actually moves files at NFT's request, as well as sending error and abort diagnostic messages if a job completes unsuccessfully. But there are other state changes, too, and you can request messages about any or all of them by using the appropriate argument for VERBOSE. (VERBOSE does not change the scope of jobs covered in status reports from RPT, which has its own verbose -v option, nor the environment-variable setting reports from STATUS.)

Each possible state change for an NFT job corresponds to one bit in a (32-bit) mask that VERBOSE sets. You request diagnostic messages about a state change by setting its bit in the mask, and you set each bit by using the decimal value shown in the table below. To request a combination of reports, ADD the corresponding decimal values and use the sum as the argument for VERBOSE (for example, the default combination of diagnostic messages corresponds to the sum 4+8+64=76).

State Change

Decimal Value

Diagnostic Meaning

Begin

1

Client has submitted job

Done

2

Job has completed successfully

Error *

4

Job has failed (unsuccessfully completed)

Abort *

8

Job was killed by user

Accepted

16

Job was received by server

Reserved

--

--

Transfer stats *

64

FTP transfer amount and rate

Start

128

Server has started job execution

Progress errors

256

Immediately reports in-progress errors in secondary jobs

Reserved

--

--

* Default verbosity (combination 76)

To see how changing the VERBOSE value changes the grain size of state-change reports during NFT dialogs, compare this default-value exchange (VERBOSE 76)

User: get test4
Rtne: 14.0. 95 bytes received in 1.3 seconds (0.1 Kbytes/s)
      from ~/test4 to /tmp/jfk/test4
      14.0. 1 entry copied ~/test4
      nft>

with this maximum-value exchange for the same job (VERBOSE 479):

User: get test4
Rtne: 14.0. accept.
      14.0. begin ~/test4
      14.0. start ~/test4
      14.0. 95 bytes received in 1.3 seconds (0.1 Kbytes/s)
      from ~/test4 to /tmp/jfk/test4
      14.0. 1 entry copied ~/test4
      14.0. done. /tmp/jfk/test4
      nft>

NFT Sessions

Grouping Jobs By Session

Whenever you start an NFT client, you start some NFT "session" with a unique (integer) session identifier. (This is true even when you use quoted execute-line commands or input from a file, although the absence of a prompt or echo can hide NFT's use of a session here.) By default your session is 0. The NFT server associates the current session identifier with every job you submit during that session as part of its persistent job tracking and execution. As a result, sessions provide another dimension along which you can group jobs, independent of their job class, for monitoring and analyzing their status. An NFT session is a logical set of jobs sharing the same session number. It is not the same as an NFT client (one client can use several sessions), nor a sequence of jobs (some session jobs may be asynchronous and mixed with other-session jobs), nor a time block (two sessions may overlap in time).

The prime reasons for invoking multiple sessions are:

  • To subdivide the jobs in a class (all complete jobs, all error jobs, etc.) between sessions for separate status monitoring where you need such detail or extra control.
  • To allow separate RPT reports, not intermixed, on two (or more) sets of secondary jobs. Thus running a large GET * request in one session and a large PUT * request in another session would let you generate separate RPT reports on the many subordinate file transfers of each without interference with the other.
  • To let you recover and inspect the RPT status reports on all the jobs in each session (perhaps including jobs from several classes) independently of one another, even after your NFT client ends.

One Client, Multiple Sessions

Techniques. You can start a new session at any time while running NFT by typing

session n

where n is an integer from 1 to 99 inclusive. This command closes your former NFT session (session 0 is the default) and opens a new one, in which the new session number n is associated with all your subsequent NFT commands. NFT numbers the jobs in each session with an increasing sequence of integers that ignores all other sessions (so, e.g., each session may have an unrelated job numbered 13).

You can reopen a former session by using its session number in this same command (e.g., SESSION 0 reopens that session and closes session 1 if issued while you are in session 1). Reopening a session lets you check on its jobs with RPT or start more jobs associated with it.

NFT gives no overt confirmation when you close one session and open another (just the usual prompt for input). And RPT status reports do not reveal which session they cover. So to discover which session is now open (or to confirm a requested change of session) use NFT's STATUS command, which reports the current session number along with other NFT environment settings.

Because NFT remembers your session numbers for up to 4 days and because picking a session number already in use reopens that session rather than creates a new one, you may sometimes want NFT to open a new session by automatically picking a number for it that is guaranteed to be unused. To guarantee a fresh session number, use

session new

to which NFT responds with the message

New Session: n

where n is the next unused (by you) session number available. As before, your previous session is now closed and the nth session is now open.

Effects. NFT sessions group together jobs, not log messages or environment variable settings.

RPT scope

RPT reports on the status of only those NFT jobs associated with the session open when you issue the RPT command. To report on jobs in other sessions, you must first reopen the session of interest, then type RPT. RPT does not accept session-number arguments and its reports do not state which session number they cover (use STATUS). Remember also that you may have several jobs with the same job number, one in each of several independent sessions.

LOG scope

If you start recording your NFT interactions with LOG logfile, then logfile will continue to collect messages in an unbroken sequence even if you open new sessions or reopen old ones. You can change log files if you wish, but changing sessions does NOT automatically change or close log files, and you cannot permanently assign separate log files to separate sessions. Sessions divide NFT jobs while log files divide NFT messages.

environment-variable scope

Environment-variable settings (e.g., SYNC/ASYNC or NOCLOBBER/CLOBBER) persist independently of NFT session changes. You cannot permanently assign some settings to one session and different settings to another. Thus you cannot simply declare one NOCLOBBER GET session and another CLOBBER PUT session, for example, because each time you set that variable all (incomplete) jobs in all sessions will be affected. Only by using separate NFT clients (not multiple sessions with one client) can you have two sets of environment-variable settings for two sets of jobs at once. See the next section for details.


Multiple Clients, Multiple Sessions

You can run several NFT clients at once (each in its own window, for example), but to do so you must compensate for the way NFT manages sessions when the same user runs more than one client.

 

Client 0

Client 1

Default

session 0

"steals" session 0

To regain, you type

session 0

session 1

By default, your first NFT client (e.g., client 0) opens session 0. Executing a second NFT client (e.g., client 1) automatically transfers that open session to the second client. One result of this "session theft" affects output messages, all of which are diverted from the first to the second NFT client. Another result affects subsequent input (commands) to the first client. Attempts to execute more commands (other than SESSION) yield a fatal error, terminating the NFT client:

***Panic. Another application instance has opened this session.

You can overcome this session theft and use both NFT clients with the help of NFT's SESSION command, used either interactively or in batch runs:

Interactively. Issue SESSION 1 to the second client (client 1) to open a new, independent session for it. Then issue SESSION 0 to the first client (client 0) to reopen or recover the lost session. Now both NFT clients will separately accept and process input, keep logs, and report job status with RPT. They will also maintain different NFT environment variable settings (e.g., NOCLOBBER for one, CLOBBER for the other) if you wish. In fact, only by using such separate NFT clients, each associated with a different NFT session, can you have two sets of environment-variable settings for two sets of NFT jobs at once.

Batch Runs. The above technique is not very practical if you want to execute several batch scripts at the same time, each involving NFT, yet keep their NFT transactions separately recorded. Instead, invoke SESSION NEW when you first run NFT in each script, for example,

nft "session new;log nftlog;status;clobber;put abc;quit"

This requests (and reports to your log file) a previously unused session number, so that your file-transfer records end up segregated in that uniquely identified session. You can then reopen that same session (with the SESSION n command) to accumulate records for every subsequent NFT execution in the same script, or repeatedly use SESSION NEW for each NFT run.

Multiple clients, each with their own session, are usually used to divide NFT job streams so they can be processed or recorded differently. But, if this is not your goal, you can pass NFT sessions back and forth repeatedly between clients at any time simply by typing SESSION n to cause the current client to open and feed its jobs into the nth session. NFT numbers jobs sequentially by session, NOT by client, so exchanging sessions between clients in this way will allow both clients to contribute jobs, with one sequence of job numbers, to one accumulating status record (per session) that RPT will report. See the Grouping Jobs by Session section for more details on how NFT sessions and job-status reports interact.