With a recent upgrade to GitLab runner software, we are now able to support Flux directly for CI pipelines running on Flux-enabled clusters. This bulletin describes the changes that will occur when this feature is enabled.
On 6/1/2023, the meaning of the "batch" tag will change from today's meaning of "Slurm wrappers" to "Flux executor" on Flux-enabled clusters. If you are currently selecting runners based solely on the "batch" tag on a Flux-enabled cluster, your jobs will break since the GitLab runner will switch from using Slurm wrappers to using Flux directly. If you are using the slurm tag, your jobs will continue to run as they previously did with no intervention required on your part.
As of this writing, Flux-enabled clusters are: corona, tenaya, tioga, rzalastor, rzvernal
GitLab Runner Tags and their Meanings on Flux-Enabled Clusters
|
Tag |
Current Meaning |
New Meaning (as of 6/1/2023) |
|---|---|---|
|
batch |
Slurm wrappers |
Flux executor |
|
batch-secondary |
n/a |
Slurm wrappers or slurm executor, if cluster is running both Flux and Slurm |
|
flux |
n/a |
Flux executor |
|
slurm |
Slurm executor or slurm wrappers |
Slurm executor or slurm wrappers |
|
shell |
Shell on a login node |
Shell on a login node |
NOTE The "batch" tag will continue to mean "Slurm executor" on clusters that only support Slurm, and "LSF executor" on clusters that only support LSF.
Selection of Runners (as of 6/1/2023)
|
Desired Effect |
Tag Selection |
|---|---|
|
Run on a cluster's primary scheduler |
batch |
|
Run on a cluster's secondary scheduler |
batch-secondary |
|
Run on flux specifically |
flux |
|
Run on slurm (or wrappers) specifically |
slurm |
|
Run on LSF specifically |
lsf |
|
Run on a login node |
shell |