There are several ways to obtain an allocation on an LC system:
- Talk to your computer coordinator to access an existing bank
- Request an LDRD Allocation, if applicable
- Purchase an allocation (contact Greg Tomaschke)
How do allocations work?
LC manages and tracks your use of computer resources when your job is scheduled by Slurm or LSF. Charge (aka bank) accounts are established for every project and assigned a target share of the machine. Every job a user submits must specify a charge account or a default will be assigned. If the user is not a member of any charge accounts, the job will be rejected. The computing resource usage (typically processor minutes) will be charged to the associated account. The ratio of this usage to the charge account's shares is used in calculating job priority on LC production clusters via a fair-tree priority scheme.
In fair-tree priority, banks are organized in a hierarchical tree. LC creates parent banks for the various programs that have contributed to the purchase of a cluster with shares that are proportional to the contributions of those programs. The individual programs then use various schemes to define child banks for different projects under those parent banks and add users to those child banks. Usage from jobs submitted by a given user with a given bank is added to the usage of that bank and all of its parents. The relative priority of two queued jobs is then determined by comparing the usage and shares at the highest level of the tree that the banks differ. So, for example, if two users each submit a job using the same bank, the job from the user who has run less will be higher priority. If two users each submit a job using different banks that have the same parent, the job using a bank that has less usage relative to its shares will run first. Since the bank includes the usage of all users of that bank, even if a user has never run before, they may get a lower priority than a user who has run a lot if they're using a bank that has seen heavy usage from other users. This continues up the tree. So, the relative priorities of jobs submitted using a bank associated with an LDRD project (under M&IC) and a job submitted using a bank associated with a project under ASC will depend on the total usage of all users of all M&IC banks and the total usage of all users of all ASC banks. Slurm's implementation of this is described in more detail at https://slurm.schedmd.com/fair_tree.html. Flux and LSF use similar systems for calculating job priority.
The other major factor in how usage and allocations affect job priority is usage decay. While we want to ensure fair access to LC resources, we also want to ensure that LC resources maintain high utilization. Usage in the fair-tree priority calculations is weighted by age with weights that follow an exponential decay with a one week half-life. Thus, usage from a job that finished one week ago is weighted half as much as usage for a job that just finished, usage from a two week old job is weighted one quarter, etc. Under this system of fair-tree with usage decay, your charge account is never completely 'spent' and you don't need to worry too much about current usage excessively penalizing you if you need to run more in the future. A given bank can be overserviced, when its usage is a larger fraction of the total system usage than its shares, but this just leads to lower priority for jobs submitted to that bank. Over time, that usage will decay and the priority for jobs submitted using that bank will go up.
While the fair-tree priority is the dominant factor in job priority, other factors such as job age and quality of service can also affect job priority. See Multifactor Job Priority for more information, or our comprehensive batch system tutorial.
What if I need a larger allocation or use of an entire machine?
You should request a DAT (dedicated application time):