Docker BuildKit

2020-04-16 2 min read martin

In the previous blog post of the building docker images series we had a look at Docker image build pipelines. We revealed typical problems and looked at how to overcome some by utilizing multistage builds.

In part 4 of this series we work with an addition to the Docker tooling which is BuildKit.

BuildKit

BuildKit was introduced with Docker 17.05 and has been an experimental feature for a while. It still has to be enabled explicitely but adds vast performance improvements as it optimizes uploading the Docker context and is able to process instructions in parallel.

Pluggable frontends add new features provided as Docker image. Those features include cache mounting, SSH secret mounting, SSH connection forwarding and more.

For those eager to get in-depth knowledege, look at BuildKit’s repository at GitHub.

To enable BuildKit we have a few options. I personally run my Docker builds with BuildKit all the time. So I have my Docker daemon configured to run on steroids while building an image.

/etc/docker/daemon.json

{ "features": { "buildkit": true } }

Alternatively it is enabled using the environment variable DOCKER_BUILDKIT.

export DOCKER_BUILDKIT=1
docker image build ...
# or
DOCKER_BUILDKIT=1 docker image build ...

Docker BuildKit also adds a new CLI command that has BuildKit turned on all the time. Though this new CLI feature has to be enabled by another variable DOCKER_CLI_EXPERIMENTAL.

export DOCKER_CLI_EXPERIMENTAL=enabled
docker buildx build -f Dockerfile .

Now let’s first look at a fairly simple Dockerfile from the hellowhale project I forked on GitHub.

FROM nginx:latest
COPY wrapper.sh /
COPY html /usr/share/nginx/html
CMD ["./wrapper.sh"]

After cloning the repository we first build it without BuildKit.

export DOCKER_BUILDKIT=0
docker image build -t martinahrer/hellowhale .

Build log

Sending build context to Docker daemon  643.1kB
...
real    0m20.460s
user    0m0.123s
sys     0m0.081s

Then we execute it with BuildKit enabled this time.

export DOCKER_BUILDKIT=1
docker image build -t martinahrer/hellowhale .

We are going to notice that it executed a bit faster. Speed improvements may vary depending on the complexity of the Dockerfile. But most noticeable are the log lines stating the transfer of Docker context. Earlier we have seen that around 640 kB were uploaded from the client to the Docker engine. This time with BuildKit enabled we see that multiple junks are transferred with the total size much lower than before. BuildKit optimizes by analyzing the Dockerfile and then only requesting the required content from the client.

Build log

 => => transferring dockerfile: 134B
 => => transferring context: 2B
 => => transferring context: 278.14kB
...
real    0m18.689s
user    0m0.137s
sys     0m0.080s