GitHub Actions to Kubernetes without a secret

A pipeline that deploys to Kubernetes needs a credential, and it usually lives in a repository secret. With kubelatch, a GitHub Actions job gets its own instead, in exchange for a token GitHub signs for that run. It lasts an hour by default; nothing is stored in GitHub and nothing needs rotating. It only works with GitHub Actions, and the runners have to reach your kubelatch over https.

The secret in every repository

The usual way to let a pipeline deploy is a kubeconfig saved as a repository secret: an administrator’s, or a ServiceAccount token made by hand for the job. It works, and it’s a common way to start.

It also leaves a long-lived credential in every repository that deploys. Every pipeline that holds a copy looks the same to the cluster, so the audit can’t tell them apart. And rotating it means finding every copy and changing them all together.

The exchange

GitHub Actions can issue an OIDC id_token, signed by GitHub for one run, with the repository, the branch, the run_id and who triggered the job inside it. The workflow asks GitHub for one, with kubelatch’s address as its audience, and sends it to kubelatch.

kubelatch checks it strictly: the signature against GitHub’s public keys, the exact issuer, an audience equal to its own address, the token’s times, and single use, so a captured token can’t be exchanged twice. Then it looks for a trust rule, which an administrator created to match your repository to a bot, optionally for one branch, one tag or one GitHub environment only. Rules compare numeric ids, not names, because repositories and organizations get renamed and names get recycled.

If a rule matches, kubelatch answers once with that bot’s credential and its kubeconfig. The credential lasts 1 hour by default, enough for a job; an administrator can change that with KUBELATCH_CI_TOKEN_TTL, never beyond a bot’s maximum. A job that runs longer exchanges a new token. There’s nothing to rotate, because every run gets its own (what is verified).

The workflow

This is the example from the docs, the one that ships in the kubelatch repository, without its header comments. It goes in .github/workflows/kubelatch.yml:

name: kubectl via kubelatch

on:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
  id-token: write   # required: lets the job request the OIDC token
  contents: read

env:
  KUBELATCH_URL: https://kubelatch.example.com   # exactly KUBELATCH_BASE_URL: it is the token's audience

jobs:
  pods:
    runs-on: ubuntu-latest
    # environment: prod   # uncomment if the trust rule sets an environment
    steps:
      - name: Request the OIDC token for kubelatch
        run: |
          ID_TOKEN=$(curl -sSf -G \
            -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
            --data-urlencode "audience=$KUBELATCH_URL" \
            "$ACTIONS_ID_TOKEN_REQUEST_URL" | jq -r .value)
          echo "::add-mask::$ID_TOKEN"
          echo "ID_TOKEN=$ID_TOKEN" >> "$GITHUB_ENV"

      - name: Exchange it for a kubelatch credential
        run: |
          RESPONSE=$(curl -sS -w '\n%{http_code}' -H 'Content-Type: application/json' \
            -d "$(jq -n --arg token "$ID_TOKEN" '{token:$token}')" "$KUBELATCH_URL/v1/ci/github-actions/token")
          STATUS=$(printf '%s' "$RESPONSE" | tail -n1)
          BODY=$(printf '%s' "$RESPONSE" | sed '$d')
          if [ "$STATUS" != "201" ]; then
            echo "kubelatch answered $STATUS: $(printf '%s' "$BODY" | jq -r '.error // .')" >&2
            exit 1
          fi
          # The token and kubeconfig are shown exactly once: keep them out of the log.
          printf '%s' "$BODY" | jq -r .token | xargs -I{} echo "::add-mask::{}"
          # Only this job's user may read it; never ~/.kube/config.
          (umask 077 && printf '%s' "$BODY" | jq -r .kubeconfig > "$RUNNER_TEMP/kubeconfig")
          chmod 600 "$RUNNER_TEMP/kubeconfig"
          echo "KUBECONFIG=$RUNNER_TEMP/kubeconfig" >> "$GITHUB_ENV"
          echo "credential expires at $(printf '%s' "$BODY" | jq -r .expires_at)"

      - name: kubectl through kubelatch
        run: |
          kubectl auth whoami          # bot:<name> with the kubelatch:* groups of its grants
          kubectl get pods -n default

Three things matter. id-token: write lets the job ask for the token; without it there’s none. KUBELATCH_URL has to match the server’s KUBELATCH_BASE_URL letter for letter, because it’s the audience kubelatch requires. And the kubeconfig goes to $RUNNER_TEMP, never to the runner’s ~/.kube/config, with the token masked in the log. If the rule sets an environment, uncomment the environment: line and set its name. An administrator can also hand you a version filled in for your rule: right after it’s created, kubelatch shows them a Workflow snippet with your URL, the trigger for its branch or tag, and its environment. Change the last step to whatever the job has to do on the cluster.

What the audit shows

The job’s kubectl auth whoami prints bot:<name>, and that’s who the cluster sees. Every request the workflow sends is a row in kubelatch’s audit log, with that bot as its subject.

The credential behind those rows says where it came from. It’s listed in Credentials as github-actions, marked “CI token · run 1234 · octocat”, and its note carries the repository, the branch, the run and who triggered it: owner/repo@refs/heads/main · run <id> · <actor>. In Audit, the Control plane tab records its issue as a credential.issue with via: github-actions, and a failed exchange as a ci.exchange.failure. In CI, each rule’s Last exchange says when a workflow last used it.

So from a request you get to the credential, and from the credential to the repository, the branch, the run and whoever triggered it (check that it works).

What it does not cover

The exchange is for GitHub Actions, on GitHub or on GitHub Enterprise Server. Other CI systems, and tools like Argo CD, use a bot with a credential issued by hand: a secret again, kept in that tool’s secret manager, but with an owner, an expiry and a record. A bot’s credential lasts 90 days at most by default, and Credentials lists the nearest expiry first, amber with 7 days or less left and red under 24 hours (users and bots).

The runners also have to reach kubelatch over https with a certificate they trust, so a kubelatch on a laptop or behind a private network won’t work with GitHub-hosted runners. And a rule set to Any workflow, which pins no ref, also applies to workflows from a pull request on a branch of the same repository: pin a branch or a tag for anything that isn’t read-only (how the rule is chosen).

Try it

To see kubelatch itself, the local tutorial takes about 15 minutes on your machine. For a pipeline you need a kubelatch your runners can reach: an administrator creates a bot, grants it permissions on the cluster and creates a trust rule (CI: trust rules), and you add the workflow (credentials for GitHub Actions). Free lets you have as many bots and CI rules as you like; its limits are 2 clusters, 5 people and 24 hours of audit (pricing). kubelatch is made by one person, in Spain: if something here doesn’t match what you see, write to me.