Command Palette

Search for a command to run...

Hectal
Lab 0.2·SecretsCRITICAL

Deleted in the Dockerfile, alive in the image

The Dockerfile copies .env to build the app, then runs rm .env. Anyone who can pull the image can read every value in it.

Weakness
CWE-538 · Insertion of sensitive information into externally-accessible file
Target
the shoplite:bad container image
image layersdocker historyextracting layerstrivy image --scanners secretBuildKit secret mounts

Run these techniques only against the lab you own. Using them on systems you don't have permission to test is illegal.

Lab setup

  1. 01Build the vulnerable image

    Two classic mistakes in one Dockerfile: a secret passed as a build ARG, and a secret file copied in and deleted later in a separate step.

    sec-lab/Dockerfile.badwhole filedocker
    FROM node:22-alpine
WORKDIR /app
ARG NPM_TOKEN
COPY .env package.json ./
RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc && npm install && rm .npmrc
RUN rm .env
COPY server.js ./
CMD ["node", "server.js"]
    terminal
    $ printf 'AWS_ACCESS_KEY_ID=AKIA2E7QXKZ3MNB4LP5W\nDB_PASSWORD=hunter2-prod\n' > .env
    echo '{"name":"lab","version":"1.0.0"}' > package.json && echo 'console.log("hi")' > server.js
    docker build -f Dockerfile.bad --build-arg NPM_TOKEN=npm_Fz8Kq2Lr5Tx9Wb3Nc6Vd1Hj4Mp7Ys0Ga2Qe -t shoplite:bad .
    ── output ──
    => => naming to docker.io/library/shoplite:bad

The threat

  1. 01Read the build args from the image history

    Every layer records the command that created it, and build ARGs used in RUN are included in that record. No special tools needed.

    terminal
    $ docker history --no-trunc shoplite:bad | grep -o 'NPM_TOKEN=[^ ]*'
    ── output ──
    NPM_TOKEN=npm_Fz8Kq2Lr5Tx9Wb3Nc6Vd1Hj4Mp7Ys0Ga2Qe

  2. 02Recover the 'deleted' .env from its layer

    An image is a stack of tar archives. rm .env in a later layer only adds a 'whiteout' marker hiding the file from the final filesystem; the layer that added it still contains the bytes. Export the image and look inside the layers.

    terminal
    $ mkdir img && docker save shoplite:bad | tar -x -C img
    for layer in img/blobs/sha256/*; do tar -tf "$layer" 2>/dev/null | grep -q '^app/.env$' && tar -xOf "$layer" app/.env; done
    ── output ──
    AWS_ACCESS_KEY_ID=AKIA2E7QXKZ3MNB4LP5W
    DB_PASSWORD=hunter2-prod

What's at risk

  • The npm token (publish rights to your packages: a supply-chain attack vector) and every secret in .env.
  • Every copy of the image: every registry it was pushed to, every node that pulled it, every CI cache.

Detect

  1. 01Scan images for secrets in every layer

    Trivy's secret scanner walks all layers, not just the final filesystem, and reports which Dockerfile instruction added each finding. Run it in CI after every image build and fail the job on findings.

    terminal
    $ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:0.63.0 image --scanners secret shoplite:bad
    ── output ──
    shoplite:bad (secrets)
     
    /app/.env (secrets)
    Total: 1 (CRITICAL: 1)
     
    CRITICAL: AWS (aws-access-key-id)
    ══════════════════════════════════════════
    AWS Access Key ID
    ──────────────────────────────────────────
    /app/.env:1 (added by 'COPY .env package.json ./ # buildkit')
    ──────────────────────────────────────────
    1 [ AWS_ACCESS_KEY_ID=********************
    ──────────────────────────────────────────
    Build-arg values in history aren't files, so check them with a policy or by reviewing `docker history` in CI, or just never pass secrets as ARGs.

Defend

  1. 01Secret mounts: available during one RUN, never stored in a layer

    BuildKit's RUN --mount=type=secret mounts the secret as a file only for that single instruction. It's never written into the image, never in history, never in the cache. .dockerignore keeps .env out of the build context entirely, so even an accidental COPY . . can't include it.

    Vulnerable

    Dockerfilewhole filedocker
    ARG NPM_TOKEN
COPY .env package.json ./
RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > .npmrc \
    && npm install && rm .npmrc
RUN rm .env

    Hardened

    Dockerfilewhole filedocker
    # syntax=docker/dockerfile:1
COPY package.json package-lock.json ./
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \
    npm ci --omit=dev
# .env is never copied: runtime config comes from the
# orchestrator (ECS secrets, Kubernetes Secrets, etc.)

# .dockerignore
# .env
# .git
# *.pem
    terminal
    $ docker build --secret id=npmrc,src=$HOME/.npmrc -t shoplite:good .
    ── output ──
    => => naming to docker.io/library/shoplite:good

  2. 02Rotate what the bad image exposed

    Revoke the npm token and change every value that was in .env. Delete the bad image tags from every registry, but assume copies exist.

Verify

  1. 01The same attacks against the new image find nothing

    terminal
    $ docker history --no-trunc shoplite:good | grep -c NPM_TOKEN
    docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:0.63.0 image --scanners secret -q shoplite:good
    ── output ──
    0
    (no findings)

The concepts

Layers are additive

Each Dockerfile instruction creates a layer: a tar of the files it added or changed. Deleting a file in a later layer hides it from the running container but doesn't remove it from the image. The same applies to files created then deleted in DIFFERENT RUN steps. Within a single RUN, create-use-delete never persists, but secret mounts are the cleaner answer.

Build-time vs run-time secrets

BUILD-time secrets (a private registry token, a git SSH key) are needed only while building: use --mount=type=secret or --mount=type=ssh. RUN-time secrets (database passwords, API keys) must never be in the image at all; inject them when the container starts from the orchestrator's secret store (ECS secrets, Kubernetes Secrets or CSI drivers, Vault).

Your turn

01

Name a tool that lets you browse an image layer by layer and see which files each one added.

02

Is ENV API_KEY=... in a Dockerfile any safer than ARG?

Interview questions

01

How do you use a secret during docker build without leaking it into the image?

02

Why doesn't RUN rm secret.txt remove a secret from an image?

0/4 · 0%