This is a simple example to build an Almalinux 8 image and inject LLNL CA certificate. After building, the container will be pushed to the Gitlab registry for this repository.
1. Build
Note that we are using the SLURM batch method to build the container in this example.
SLURM parameters
The build job runs in the batch queue using the Gitlab batch executor. The following segment of the CI yaml specifies which arguments should be passed to SLURM when allocating the job:
variables: LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -p pdebug --userns --exclusive"
The key SLURM parameters are:
- --nodes=1: The build must be done with a single-node allocation
- --userns: This indicates that SLURM should populate the user namespace mappings for your user
- --exclusive: The build must be the only job running on the batch node. On CPU-shared systems, such as oslic and borax, this prevents other jobs from starting on the node.
Specifying the executor
We use tags to specify how and where the CI job should run. In this case, we're running on quartz using the batch executor. The shell executor is also available, but will run on a login node rather than allocating a compute node and will not use the SLURM parameters we set above.
build: stage: build tags: - quartz - batch
2. Container Registry
We are using the Gitlab registry to store the final image. The $CI_* variables are automatically populated to point to the registry for this repository.
podman push "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" podman logout $CI_REGISTRY
3. Working with the image
Once the image is stored in the registry, you'll be able to pull it and run containers on LC systems.
4. Pulling the image
First, you'll need to generate a Gitlab PAT (Personal Access Token) here: https://lc.llnl.gov/gitlab/-/profile/personal_access_tokens
The PAT you create should minimally include the read_registry scope. When prompted for your password when authenticating to the registry, the PAT should be used as your password.
5. Using the Image in Another CI Pipeline
Method 1: Pulling the Image in the CI Pipeline
To use the built container image in another CI pipeline, you can pull the image from GitLab's registry in your GitLab CI YAML file. Here's an example showing how to run a container from the pulled image:
variables: LLNL_SLURM_SCHEDULER_PARAMETERS: "--nodes=1 -p pdebug --userns --exclusive" build: stage: build tags: - quartz - batch script: - /collab/usr/gapps/lcweg/containers/scripts/enable-podman.sh - echo "${CI_REGISTRY_PASSWORD}" | podman login -u="$CI_REGISTRY_USER" --password-stdin $CI_REGISTRY - podman pull "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" - podman run -d --name my_deployed_app "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG" - podman logout $CI_REGISTRY
Method 2: Referencing the Image in Another Dockerfile
Once the container image is pushed to the GitLab Container Registry, the image can be used as the base image in another project's Dockerfile. For exmaple, your Dockerfile might start out with:
FROM: czregistry.llnl.gov:5050/weg/examples/podman-ci-example:latest
Any necessary modifications to the image can be added in the Dockerfile. Then, the image can be rebuild using the build instructions detailed earlier, resulting in an updated version of the container image.
Podman
Configure podman for LC
Podman won't work out-of-the-box on LC systems and requires a few configuration tweaks. Since these settings may change over time, we have a script that automatically generates and installs an appropriate set of podman configuration files for you.
Run the podman configuration script:
/collab/usr/gapps/lcweg/containers/scripts/enable-podman.sh
Pulling the image with podman
podman login -u <your username> czregistry.llnl.gov:5050 podman pull czregistry.llnl.gov:5050/weg/examples/podman-ci-example:main
A note about subuid errors
You may see the following errors when using podman:
ERRO[0000] cannot find UID/GID for user green77: No subuid ranges found for user "green77" in /etc/subuid - check rootless mode in man pages. WARN[0000] using rootless single mapping into the namespace. This might break some images. Check /etc/subuid and /etc/subgid for adding sub*ids
When running a container, these messages are typically benign and may be ignored unless other failures occur.
Singularity
This will pull the image and convert it to a Singularity SIF file.
singularity remote login -u <your username> docker://czregistry.llnl.gov:5050 singularity build almalinux8.sif docker://czregistry.llnl.gov:5050/weg/examples/podman-ci-example:main