Skip to content
Platform Docs

Docker Build Optimization with BuildKit

BuildKit is the default Docker build engine and unlocks a set of powerful optimization features. This guide covers the most useful options available on the platform, all configurable via the DOCKER_BUILD_ARGS CI variable.

The two main goals are:

  • Smaller images — exclude unnecessary files and layers from the final image
  • Faster builds — reuse cached layers and avoid re-running unchanged steps

Some features below require the labs channel frontend syntax. Add this as the first line of your Dockerfile when needed:

# syntax=docker/dockerfile:1-labs

Build Optimizations

Using .dockerignore file

Using a .dockerignore file allows decreasing the size of the context, and then speed up the build.
The syntax is similar to the .gitignore file.

Using a positive filtering can help simplify the content of the file, and avoid maintaining the file when new content is added to the project.

As an example, for a node project, we can add only a few directories and files:

*
!src
!package*.json
!.npmrc
!tsconfig.json

Copying files using a filter

Using the option --parents, COPY preseves the directory structure while copying only the required files.
This option allows to reduce the files in the layer, thus allowing to have a better hit rate for the cached layer.

As example, the following command will copy in the container only the package.json and .npmrc files contained in a repository while preserving the structure.

COPY --parents ./**/package.json ./**/.npmrc ./

Using the option --exclude, can help improving the filtering.
As example:

COPY --exclude=./**/node_modules --parents ./**/package.json ./**/.npmrc ./

Using another context

This optimization leverages Named contexts. Named contexts allow to import files from another directory, or with another pattern using .dockerignore file.

The following command will do the same thing as the previous point i.e. copy in the container only the package.json and .npmrc files contained in a repository while preserving the structure.

COPY --from=package_json ./app ./

The package_json context needs to be passed to the component using the DOCKER_BUILD_ARGS variable and the build-context argument.

variables:
  DOCKER_BUILD_ARGS: "--build-context package_json=.. "

The package_json context is reduced using the following .dockerignore file.

# Using this file as a context to allow to put only package.json files in the container.
*
!**/.npmrc
!**/package.json
**/node_modules/**/package.json

Using Cache Mounts

Cache mounts allows caching directories, that will not be part of the resulting images, but will be used as a cache for the following builds.

The following example leverage the --mount option to reuse the /root/.npm/ the directory across builds:

RUN --mount=type=cache,target=/root/.npm npm install

Other Optimizations

Ordering Layers

Ordering layers correctly ensures that the most stable steps (e.g. dependency installation) are cached and only invalidated when necessary. Read this documentation.

Warning: The ARG instruction invalidates the cache for all subsequent RUN commands when its value changes. Declare ARG just above the first instruction that uses it to minimise cache invalidation. See documentation

Security — Secrets with BuildKit

The --secret option of build allows to mount external resources (files or environment variables) as file or environment variables of a RUN command.

  • Mounting an environment variable as an environment variable:
RUN --mount=type=secret,id=npm-package-registry-token,env=NPM_PACKAGE_REGISTRY_TOKEN \
    npm ci

With .gitlab-ci.yml

variables:
  DOCKER_BUILD_ARGS: "--secret id=NPM_PACKAGE_REGISTRY_TOKEN"

or

variables:
  DOCKER_BUILD_ARGS: "--secret type=env,id=NPM_PACKAGE_REGISTRY_TOKEN,env=NPM_PACKAGE_REGISTRY_TOKEN"
  • Mounting an file variable as a file:
RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
  aws s3 cp s3://... ...

With .gitlab-ci.yml

variables:
  DOCKER_BUILD_ARGS: "--secret type=file,id=aws,src=~/.aws/credentials"

More information can be found there

Local Setup — BuildX and BuildKit with Colima

brew install docker-buildx buildkit

Then the following command should work:

% docker buildx version
github.com/docker/buildx v0.23.0 Homebrew

If the plugin is not found, check that it is linked in you ~/.docker/cli-plugins directory along the compose plugin.

% ls -l ~/.docker/cli-plugins
total 0
lrwxr-xr-x  1 david.clement  staff  31 May  7 15:32 docker-buildx -> /opt/homebrew/bin/docker-buildx
lrwxr-xr-x  1 david.clement  staff  32 May  7 15:32 docker-compose -> /opt/homebrew/bin/docker-compose

You can create the link using the following command

mkdir -p  ~/.docker/cli-plugins && ln -s $(which docker-buildx) ~/.docker/cli-plugins/docker-buildx

You can also add compose as a plugin:

mkdir -p  ~/.docker/cli-plugins && ln -s $(which docker-compose) ~/.docker/cli-plugins/docker-compose

And test it:

% docker compose version
Docker Compose version 2.23.3

Running with BuildKit

To be able to run with some options, BuildX may require a dedicated context.

To create a context named local-builder, create the associated Docker context:

docker context create local-builder

Create the BuildX context and use it:

docker buildx create local-builder --driver docker-container --use