1. Building from a Dockerfile with Podman
Our recommendation is that, if you want to build a container from scratch (e.g. using a Dockerfile), build that container using podman build. To do so, you'll want to
- A. Build your container with podman.
- B. Save the container to a docker `.tar` archive.
This guide covers building an image by hand, if you're interested in automating your build and persisting the image in LC's Gitlab container registry, check out this guide.
In greater detail:
A. Building with podman
If you haven't already you'll first have to configure your environment by running the enable-podmam utility script. On LC systems, it's available by running enable-podman from a terminal on the system you want to build on. The script writes a configuration file to your home directory, so you'll only need to run it once per enclave you use. However, if you run into issues, try re-running the script. We sometimes update the script to work around new problems that arise.
It's a best-practice to do builds on an allocated compute node so that your build doesn't interfere with other users sharing the login nodes. Once you have an allocation, you can build the container with podman build, specifying the Dockerfile with -f and the tag of the container created with -t. Using the Dockerfile Dockerfile.ubuntu and tag myimage, we'd run
podman build -f Dockerfile.example -t myimage
To try this yourself, you can grab the contents of Dockerfile.example.
After podman build ..., run podman images to see your container image.
If you're running each step interactively, the whole build might look something like this:
janeh@pascal43:~$ . /collab/usr/gapps/lcweg/containers/scripts/enable-podman.sh janeh@pascal43:~$ podman build -f Dockerfile.example -t myimage STEP 1/3: FROM almalinux:8 Resolved "almalinux" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf) Trying to pull docker.io/library/almalinux:8... Getting image source signatures Copying blob 2609da11fd88 done | Copying config f1218159f1 done | Writing manifest to image destination STEP 2/3: RUN dnf update -y ... --> bad7ec9d60d8 STEP 3/3: RUN dnf install -y python3.11 ... COMMIT myimage --> 18ad68021579 Successfully tagged localhost/myimage:latest 18ad680215797a5853c04e6f621049cd29453057b76421547c243d9629847039 janeh@pascal43:~$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/myimage latest 18ad68021579 About a minute ago 436 MB docker.io/library/almalinux 8 f1218159f16a 5 weeks ago 196 MB
B. Saving the container to a docker archive
By default, the container files created on a compute node will be deleted when the allocation ends. After building and before the allocation ends, you need to explicitly save the container! For example, immediately after building the container in the above, if we terminate the allocation and start a new allocation, we see no remaining images with podman images:
janeh@pascal83:~$ salloc -N 1 -t 1 salloc: Pending job allocation 1535430 salloc: job 1535430 queued and waiting for resources salloc: job 1535430 has been allocated resources salloc: Granted job allocation 1535430 salloc: Waiting for resource configuration salloc: Nodes pascal129 are ready for job janeh@pascal129:~$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE janeh@pascal129:~$
Instead, use the syntax
podman save TAG_NAME > OUTPUT_FILENAME
to save the container after building. For a container with tag myimage, we might run
podman save myimage > myimage.tar
after building, as below:
janeh@pascal7:~$ podman build -f Dockerfile.example -t myimage ... janeh@pascal7:~$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/myimage latest 0ef15bc4f175 10 seconds ago 436 MB docker.io/library/almalinux 8 f1218159f16a 5 weeks ago 196 MB janeh@pascal7:~$ podman save myimage -o myimage.tar Copying blob a5d3eecefb86 done | Copying blob 6c6bcdeaf3b8 done | Copying blob 57ec06e597cb done | Copying config 18ad680215 done | Writing manifest to image destination
This creates the file myimage.tar. Once you've saved your image, you can run it later with "podman run docker-archive:./myimage.tar ..." The "docker-archive:" portion of the image name tells podman that the image you want to run is stored in a tarfile at the given path in the filesystem.
You may also load the image back into the image store using the "podman load" command:
janeh@pascal23:~$ podman load -i myimage.tar Getting image source signatures Copying blob 57ec06e597cb done | Copying blob a5d3eecefb86 done | Copying blob 6c6bcdeaf3b8 done | Copying config 18ad680215 done | Writing manifest to image destination Loaded image: localhost/myimage:latest janeh@pascal23:~$ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/myimage latest 0ef15bc4f175 20 minutes ago 436 MB
2. Special Considerations
A. Debian/Ubuntu containers and setgroups errors
You'll need to specify some additional arguments when building from an image based off of a Debian-like distribution, such as Ubuntu.
podman build --userns-uid-map=0:0:1 --userns-uid-map=1:1:1999 --userns-uid-map=65534:2000:2 ...You'll see error messages that look like this without those arguments:
STEP 3/6: RUN apt-get update && apt-get install -y curl E: setgroups 65534 failed - setgroups (22: Invalid argument) E: setegid 65534 failed - setegid (22: Invalid argument) Reading package lists... E: setgroups 65534 failed - setgroups (22: Invalid argument) E: setegid 65534 failed - setegid (22: Invalid argument) E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (22: Invalid argument) E: Method gave invalid 400 URI Failure message: Failed to setgroups - setgroups (22: Invalid argument) E: Method http has died unexpectedly! E: Sub-process http returned an error code (112) Error: building at STEP "RUN apt-get update && apt-get install -y curl": while running runtime: exit status 100