Command Palette

Search for a command to run...

Hectal
Lab 0.1·SecretsCRITICAL

The key that was 'already deleted'

A developer committed an AWS key, noticed, and deleted it in the next commit. The key is still one git log -p away for anyone who clones the repo.

Weakness
CWE-798 · Use of hard-coded credentials
Target
the sec-lab git repository
searching git historygitleakspre-commit hooksgit filter-reporotation before cleanup

Run these techniques only against the lab you own. Using them on systems you don't have permission to test is illegal.

Lab setup

  1. 01Create the lab repo and make the mistake

    A key that looks real (it matches AWS's key format) but isn't. First commit: config with the key. Second commit: 'oops, remove key'. This is exactly what happens in real repos, usually minutes before a push.

    terminal
    $ mkdir sec-lab && cd sec-lab && git init -q
    cat > config.js <<'EOF'
    module.exports = {
    AWS_ACCESS_KEY_ID: "AKIA2E7QXKZ3MNB4LP5W",
    AWS_SECRET_ACCESS_KEY: "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",
    region: "ap-south-1",
    };
    EOF
    git add config.js && git commit -qm "add s3 upload config"
    sed -i '/AWS_/d' config.js && git commit -qam "remove keys, use env vars instead"
    git log --oneline
    ── output ──
    b41e2c7 remove keys, use env vars instead
    9f2c1ab add s3 upload config

The threat

  1. 01Clone, then search every commit ever made

    The current files are clean, but git stores every version of every file forever. git log -p prints every diff in history; -S finds commits that added or removed a string. Attackers run exactly this, automated across millions of public repos, often within minutes of a push.

    terminal
    $ git log -p --all -S AKIA | grep -E 'AKIA|SECRET'
    ── output ──
    - AWS_ACCESS_KEY_ID: "AKIA2E7QXKZ3MNB4LP5W",
    - AWS_SECRET_ACCESS_KEY: "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",
    + AWS_ACCESS_KEY_ID: "AKIA2E7QXKZ3MNB4LP5W",
    + AWS_SECRET_ACCESS_KEY: "q9Tz4Vb1Xw8Lm2Rk7Pd5Hs3Jn6Yc0Fg4Ua8Ei2Wo",

  2. 02Use it (in real life — don't: these keys are fake)

    With a real key, the next command would be aws sts get-caller-identity to learn whose key it is, then enumerating what it can reach. Keys leaked to public GitHub are typically tried by automated scanners within minutes; Lab 0.4 shows how to notice when that happens.

What's at risk

  • Whatever permissions the key's IAM user has, for as long as the key stays active.
  • Deleting the file in a later commit changed nothing: every clone, fork, and CI cache still has the old commit.

Detect

  1. 01gitleaks: scan the whole history

    gitleaks matches hundreds of credential formats (AWS, GitHub, Stripe, Slack, private keys...) across every commit. The git subcommand scans history; dir scans a plain folder.

    terminal
    $ docker run --rm -v "$PWD:/repo" ghcr.io/gitleaks/gitleaks:v8.24.0 git /repo
    ── output ──
    ○
    │╲
    │ ○
    ○ ░
    ░ gitleaks
     
    Finding: AWS_ACCESS_KEY_ID: "AKIA2E7QXKZ3MNB4LP5W",
    Secret: AKIA2E7QXKZ3MNB4LP5W
    RuleID: aws-access-token
    Entropy: 3.921928
    File: config.js
    Line: 2
    Commit: 9f2c1ab
    Author: Priya
    Date: 2026-09-26T10:02:11Z
    Fingerprint: 9f2c1ab:config.js:aws-access-token:2
     
    INF 2 commits scanned.
    WRN leaks found: 1

  2. 02Where detection should happen, earliest first

    1) A pre-commit hook on the developer's machine, before the secret ever exists in a commit. 2) GitHub push protection (secret scanning), which blocks the push. 3) gitleaks in CI on every pull request. 4) Periodic full-history scans of every repo. Each later stage is a backstop for the earlier ones, not a replacement.

Defend

  1. 011. Rotate FIRST — cleanup second

    The moment a secret touches a remote, assume it's compromised. Deactivate and replace the key in the provider before touching git. Rewriting history doesn't un-leak anything that was already cloned, forked, cached, or scraped.

    terminal
    $ aws iam update-access-key --user-name s3-uploader --access-key-id AKIA2E7QXKZ3MNB4LP5W --status Inactive
    aws iam delete-access-key --user-name s3-uploader --access-key-id AKIA2E7QXKZ3MNB4LP5W
    ── output ──
    (no output — key deactivated, then deleted)

  2. 022. Stop needing the key at all

    The deepest fix: no long-lived keys in the app. The AWS SDK resolves credentials from the environment: an ECS task role or EC2 instance profile in AWS, OIDC in CI (Terraform course, Mission 7.1), and SSO on laptops. Nothing to commit, so nothing to leak.

    Vulnerable

    config.jswhole filejs
    module.exports = {
  AWS_ACCESS_KEY_ID: "AKIA...",
  AWS_SECRET_ACCESS_KEY: "q9Tz...",
  region: "ap-south-1",
};

    Hardened

    s3.jswhole filejs
    const { S3Client } = require("@aws-sdk/client-s3");

// No keys: the SDK's default credential chain finds the
// ECS task role / instance profile / SSO session automatically.
module.exports = new S3Client({ region: process.env.AWS_REGION });

  3. 033. Block the next one at commit time

    The pre-commit framework runs gitleaks against staged changes before each commit. Commit the config so every developer gets it with pre-commit install, and run the same check in CI for anyone who skips the hook.

    Hardened

    .pre-commit-config.yamlwhole fileyaml
    repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.0
    hooks:
      - id: gitleaks

  4. 044. Then clean the history

    git filter-repo --replace-text rewrites every commit, replacing matched strings. Everyone must re-clone afterwards, and forks and caches keep the old history, which is why rotation came first. On GitHub, also contact support to purge cached views of the old commits.

    terminal
    $ printf 'AKIA2E7QXKZ3MNB4LP5W==>REDACTED\nq9Tz4Vb1Xw8Lm2Rk7Pd5Hs3Jn6Yc0Fg4Ua8Ei2Wo==>REDACTED\n' > /tmp/replace.txt
    git filter-repo --replace-text /tmp/replace.txt --force
    ── output ──
    Parsed 2 commits
    New history written in 0.05 seconds; now repacking/cleaning...
    Completely finished after 0.21 seconds.

Verify

  1. 01Re-run the attack and the scanner

    terminal
    $ git log -p --all -S AKIA | grep -c AKIA
    docker run --rm -v "$PWD:/repo" ghcr.io/gitleaks/gitleaks:v8.24.0 git /repo 2>&1 | tail -1
    ── output ──
    0
    INF no leaks found

  2. 02Try to commit a new key — the hook refuses

    terminal
    $ pip install pre-commit && pre-commit install
    echo 'const k = "AKIA2E7QXKZ3MNB4LP5W";' > leak.js && git add leak.js && git commit -m test
    ── output ──
    Detect hardcoded secrets.................................................Failed
    - hook id: gitleaks
    - exit code: 1
     
    Finding: const k = "AKIA2E7QXKZ3MNB4LP5W";
    RuleID: aws-access-token
    WRN leaks found: 1

The concepts

Git never forgets

A commit is an immutable snapshot; later commits add new snapshots but never change old ones. 'Removing' a secret in a new commit leaves it in history, visible to anyone with repo access, in every clone and fork, and in CI caches. Only rewriting history removes it from YOUR copy, and never from copies others already have.

The leaked-secret playbook

1) Revoke or rotate immediately. 2) Check the provider's audit logs for use of the leaked credential since it was exposed (Lab 0.4). 3) Remove it from code and replace it with a runtime mechanism (roles, a secrets manager). 4) Clean history where it matters. 5) Add prevention (hooks, push protection, CI scanning). The order matters: rotation first, because everything after it takes time.

Your turn

01

gitleaks flags a string in a test fixture that is definitely not a real secret. How do you stop it failing CI without disabling the rule?

02

Why is scanning only the latest commit in CI not enough?

Interview questions

01

A developer pushed an API key to GitHub and removed it five minutes later. What do you do?

02

How do you prevent secrets from being committed in the first place?

0/4 · 0%