diff --git a/.agents/skills/pr-fixup/SKILL.md b/.agents/skills/pr-fixup/SKILL.md index d5cd705d33..d2e17e1600 100644 --- a/.agents/skills/pr-fixup/SKILL.md +++ b/.agents/skills/pr-fixup/SKILL.md @@ -1,6 +1,6 @@ --- name: pr-fixup -description: Wait for CI checks and automated reviews (CodeRabbit, Greptile, Claude) on a PR, fix failures and address comments, then push. +description: Wait for CI checks and automated reviews (CodeRabbit, Greptile, Claude, cubic) on a PR, fix failures and address comments, then push. --- # PR Fixup @@ -28,7 +28,7 @@ Create these tasks immediately (use your task/todo tracking tool if available): 1. **Gather PR state** — Fetch checks, comments, and review status 2. **Wait for CI checks** — Poll until all checks resolve -3. **Wait for automated reviews** — Poll for CodeRabbit and Greptile +3. **Wait for automated reviews** — Poll for CodeRabbit, Greptile, Claude, and cubic 4. **Fix failing CI checks** — Read logs, fix issues, run E2E tests locally if needed 5. **Triage review comments** — Classify each comment as valid, already addressed, nitpick, or wrong 6. **Address each comment** — Fix or reply with reasoning @@ -75,12 +75,13 @@ Mark task 2 as completed. Mark task 3 as in_progress. -Check if CodeRabbit, Greptile, and Claude have posted or are generating reviews. +Check if CodeRabbit, Greptile, Claude, and cubic have posted or are generating reviews. **Bot usernames** (`gh pr view --json comments` uses GraphQL and returns `author.login` **without** the `[bot]` suffix; `gh api /.../reviews` and `/.../issues//comments` use REST and return `user.login` **with** the suffix — filters below use whichever form the invoked endpoint returns): - CodeRabbit: `coderabbitai[bot]` - Greptile: `greptile-apps[bot]` - Claude: `claude[bot]` on same-repo PRs (posts a real review with inline comments via the Claude GitHub App), or `github-actions[bot]` on fork PRs (posts findings as issue comments via `GITHUB_TOKEN`; identify by body markers — tracker starts with `**Claude finished `, findings comment starts with `## Code Review`). +- cubic (cubic.dev): `cubic-dev-ai[bot]` (posts reviews via the GitHub review API, similar to Greptile; has its own `cubic · AI code reviewer` check). **CodeRabbit — stop waiting if:** - A comment contains `` — rate-limited, won't review. @@ -94,6 +95,10 @@ Check if CodeRabbit, Greptile, and Claude have posted or are generating reviews. - An issue comment from `github-actions[bot]` whose body starts with `**Claude finished ` or `## Code Review` exists (fork PR, `GITHUB_TOKEN` fallback path — findings are issue comments, no GitHub Review object). - The `claude-review` check in `gh pr checks` has completed (regardless of conclusion). +**cubic — stop waiting if any of these holds:** +- A review from `cubic-dev-ai[bot]` exists (posts via the GitHub review API). +- The `cubic · AI code reviewer` check in `gh pr checks` has completed (regardless of conclusion). + **Keep polling if:** - A bot hasn't commented yet AND `gh pr checks` shows its check is still `pending`. @@ -107,6 +112,8 @@ gh api repos/:owner/:repo/pulls//reviews --jq '.[] | select(.user.login gh api repos/:owner/:repo/pulls//reviews --jq '.[] | select(.user.login == "claude[bot]") | {user: .user.login, state: .state}' # Claude — fork PRs: issue comments from github-actions[bot] (match by body marker) gh pr view --json comments --jq '.comments[] | select(.author.login == "github-actions" and ((.body | startswith("**Claude finished ")) or (.body | startswith("## Code Review")))) | {author: .author.login, body_start: (.body[0:80])}' +# cubic posts reviews (with inline review comments) +gh api repos/:owner/:repo/pulls//reviews --jq '.[] | select(.user.login == "cubic-dev-ai[bot]") | {user: .user.login, state: .state}' ``` Mark task 3 as completed. @@ -145,7 +152,7 @@ Mark task 4 as completed. Mark task 5 as in_progress. -Fetch all review comments — human reviewers, CodeRabbit, Greptile, and Claude: +Fetch all review comments — human reviewers, CodeRabbit, Greptile, Claude, and cubic: ```bash gh pr view --json reviews,comments diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 084be46dbc..5ae7580652 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -430,9 +430,112 @@ jobs: cache-from: type=gha cache-to: type=gha,mode=max + docker-universal: + name: Build & push universal container image + needs: [prepare, docker] + if: ${{ !inputs.dry_run }} + runs-on: ubuntu-latest + permissions: + contents: read + packages: write # push to GHCR + # Build flow: push to a per-run staging tag, gate on size against the + # staging tag, then `buildx imagetools create` to promote the same + # manifest to the user-facing tags. The promote step is metadata-only - + # no rebuild, no new bytes - so this costs us one extra registry call + # in exchange for never publishing an oversized image as :universal. + # + # Tradeoff: staging tags accumulate in GHCR over time (one per release + # run). Auto-cleanup isn't safe here - `imagetools create` produces a + # byte-identical manifest list when retagging a single source, so the + # staging tag and the promoted user-facing tags share the same package + # version in GHCR. Deleting the version to drop the staging tag would + # also drop :universal / :vX.Y.Z-universal. A separate cleanup workflow + # filtering for versions whose ONLY tags match `universal-staging-*` + # would be the correct way to address this if it becomes a problem. + env: + STAGING_IMAGE: ghcr.io/kdlbs/kandev:universal-staging-${{ needs.prepare.outputs.tag }}-${{ github.run_id }} + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ needs.prepare.outputs.tag }} + + - uses: docker/setup-buildx-action@v3 + + - name: Login to GHCR + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + # Build universal on top of the vanilla image we just pushed. Passing + # the exact `:vX.Y.Z` tag (not `:latest`) keeps the pair coherent: + # universal-of-vX.Y.Z always rests on vanilla-vX.Y.Z, even if a later + # release moves `:latest`. The tag pushed here is the staging tag; + # final tags are applied below only if the size gate passes. + - name: Build and push (staging tag) + uses: docker/build-push-action@v6 + with: + context: . + file: Dockerfile.universal + push: true + platforms: linux/amd64,linux/arm64 + build-args: | + BASE_IMAGE=ghcr.io/kdlbs/kandev:${{ needs.prepare.outputs.tag }} + tags: ${{ env.STAGING_IMAGE }} + cache-from: type=gha,scope=universal + cache-to: type=gha,mode=max,scope=universal + + # Universal size gate. Inspects the staging tag - the image is in + # GHCR but under a per-run tag no user pulls. Failing the gate aborts + # before promotion, so the user-facing :universal / :vX.Y.Z-universal + # tags never move to an oversized image. Bump MAX_COMPRESSED_BYTES + # deliberately when adding scope, with a one-sentence justification + # in the commit msg. + - name: Enforce size budget + env: + MAX_COMPRESSED_BYTES: "2684354560" # 2.5 GiB + run: | + set -euo pipefail + # Inspect the multi-arch manifest and sum compressed layer sizes + # for linux/amd64. amd64 is the bigger of the two arches in + # practice, so gating on it gives us a single-number signal. + docker buildx imagetools inspect "$STAGING_IMAGE" --raw \ + > /tmp/index.json + AMD64_DIGEST="$(jq -r ' + .manifests[] + | select(.platform.architecture == "amd64" and .platform.os == "linux") + | .digest + ' /tmp/index.json)" + docker buildx imagetools inspect "ghcr.io/kdlbs/kandev@${AMD64_DIGEST}" --raw \ + > /tmp/manifest.json + TOTAL="$(jq '[.layers[].size] | add' /tmp/manifest.json)" + echo "Universal image (amd64) compressed size: $TOTAL bytes (max: $MAX_COMPRESSED_BYTES)" + if [ "$TOTAL" -gt "$MAX_COMPRESSED_BYTES" ]; then + echo "::error::Universal image exceeds compressed-size budget. Staging tag $STAGING_IMAGE will not be promoted. See docs/images.md inclusion policy." >&2 + exit 1 + fi + + # Promote: `buildx imagetools create` retargets an existing manifest + # list to new tag(s) without rebuilding or re-uploading layers. Multi- + # arch is preserved. Only runs if the size gate passed (default step + # ordering); if any earlier step failed, the user-facing tags stay at + # whatever they pointed to before this run. + - name: Promote staging to final tags + run: | + set -euo pipefail + docker buildx imagetools create \ + --tag "ghcr.io/kdlbs/kandev:${{ needs.prepare.outputs.version }}-universal" \ + --tag "ghcr.io/kdlbs/kandev:${{ needs.prepare.outputs.tag }}-universal" \ + --tag "ghcr.io/kdlbs/kandev:universal" \ + "$STAGING_IMAGE" + publish-release: name: Publish GitHub release - needs: [prepare, build-bundles] + # docker-universal is included so its size gate (and any build failure) + # blocks the release. Without this, publish-release runs in parallel with + # docker-universal and an oversized universal image still ships a release. + needs: [prepare, build-bundles, docker-universal] if: ${{ !inputs.dry_run }} runs-on: ubuntu-latest permissions: diff --git a/.github/workflows/universal-rebuild.yml b/.github/workflows/universal-rebuild.yml new file mode 100644 index 0000000000..685ce074a1 --- /dev/null +++ b/.github/workflows/universal-rebuild.yml @@ -0,0 +1,115 @@ +name: universal-rebuild + +# Weekly rebuild of the universal image on top of the current :latest +# vanilla image, so apt security updates flow into universal even when +# kandev itself hasn't been re-released. +# +# - Pushes a moving `:universal-weekly-YYYYMMDD` tag plus refreshes +# `:universal` (the "latest universal" alias). +# - Does NOT push `:vX.Y.Z-universal`. Those are immutable per release; +# only the floating `:universal` and the dated weekly tags move here. +# +# Build flow mirrors release.yml's docker-universal: push to a per-run +# staging tag, gate on size against the staging tag, then promote the +# manifest to the user-facing tags via `buildx imagetools create`. The +# promote step is metadata-only - no rebuild, no extra layer upload - so +# a failed size gate aborts before `:universal` ever points at an +# oversized image (the previous week's tag remains live). +# +# Tradeoff: staging tags (one per weekly run) accumulate in GHCR. See the +# comment in release.yml's docker-universal job for why per-run cleanup +# isn't safe (shared manifest digest with promoted tags). A separate +# cleanup workflow filtering for `universal-staging-*` versions whose +# only tag matches is the right escape hatch if accumulation becomes a +# problem; ~52 stale tags/year is acceptable in the meantime. +# +# Manual trigger is also wired up so a maintainer can force a rebuild +# after a known CVE without waiting for the cron. + +on: + schedule: + # Mondays 06:00 UTC. Picks up the previous week's Debian security + # patches without colliding with most weekday release work. + - cron: "0 6 * * 1" + workflow_dispatch: {} + +permissions: + contents: read + +jobs: + rebuild: + name: Rebuild universal on top of :latest + runs-on: ubuntu-latest + permissions: + contents: read + packages: write + env: + STAGING_IMAGE: ghcr.io/kdlbs/kandev:universal-staging-weekly-${{ github.run_id }} + steps: + - uses: actions/checkout@v4 + + - name: Compute date tag + id: date + run: echo "tag=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT" + + - uses: docker/setup-buildx-action@v3 + + - name: Login to GHCR + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + # Rebuild against :latest deliberately - the whole point of the + # weekly is to pick up any base-image moves. Use --pull so the + # current :latest is freshly fetched, not whatever's in BuildKit's + # cache from a previous run. + - name: Build and push (staging tag) + uses: docker/build-push-action@v6 + with: + context: . + file: Dockerfile.universal + push: true + pull: true + platforms: linux/amd64,linux/arm64 + build-args: | + BASE_IMAGE=ghcr.io/kdlbs/kandev:latest + tags: ${{ env.STAGING_IMAGE }} + cache-from: type=gha,scope=universal + cache-to: type=gha,mode=max,scope=universal + + # Same size gate as release.yml's docker-universal job. A base-image + # update that adds significant weight (e.g. a new Debian dep pulled by + # apt security updates) is caught here before :universal is promoted. + # Keep MAX in sync with release.yml. + - name: Enforce size budget + env: + MAX_COMPRESSED_BYTES: "2684354560" # 2.5 GiB + run: | + set -euo pipefail + docker buildx imagetools inspect "$STAGING_IMAGE" --raw \ + > /tmp/index.json + AMD64_DIGEST="$(jq -r ' + .manifests[] + | select(.platform.architecture == "amd64" and .platform.os == "linux") + | .digest + ' /tmp/index.json)" + docker buildx imagetools inspect "ghcr.io/kdlbs/kandev@${AMD64_DIGEST}" --raw \ + > /tmp/manifest.json + TOTAL="$(jq '[.layers[].size] | add' /tmp/manifest.json)" + echo "Universal image (amd64) compressed size: $TOTAL bytes (max: $MAX_COMPRESSED_BYTES)" + if [ "$TOTAL" -gt "$MAX_COMPRESSED_BYTES" ]; then + echo "::error::Universal weekly rebuild exceeds compressed-size budget. Staging tag $STAGING_IMAGE will not be promoted; previous :universal remains live. See docs/images.md inclusion policy." >&2 + exit 1 + fi + + # Promote: metadata-only retag from staging to the user-facing tags. + # If any earlier step fails, the previous :universal stays in place. + - name: Promote staging to final tags + run: | + set -euo pipefail + docker buildx imagetools create \ + --tag "ghcr.io/kdlbs/kandev:universal" \ + --tag "ghcr.io/kdlbs/kandev:universal-weekly-${{ steps.date.outputs.tag }}" \ + "$STAGING_IMAGE" diff --git a/Dockerfile.universal b/Dockerfile.universal new file mode 100644 index 0000000000..352b3745a6 --- /dev/null +++ b/Dockerfile.universal @@ -0,0 +1,206 @@ +# Kandev Universal Dockerfile +# +# Extends the vanilla kandev image with language toolchains and developer +# tools so agents can build/test code in the most common stacks without +# extra setup. Built and published alongside vanilla in the release +# workflow. Inclusion policy: see docs/images.md. +# +# Build: +# docker build -f Dockerfile.universal -t kandev:latest-universal . +# +# Run (drop-in replacement for the vanilla image): +# docker run -p 38429:38429 -v kandev-data:/data ghcr.io/kdlbs/kandev:universal +# +# The BASE_IMAGE arg lets the release workflow point at a freshly built +# vanilla image inside the same run (so universal-of-X.Y.Z is built on top +# of vanilla-X.Y.Z, not on whatever :latest happens to be). + +ARG BASE_IMAGE=ghcr.io/kdlbs/kandev:latest +FROM ${BASE_IMAGE} + +USER root + +# Pinned versions. Bump these deliberately; CI's size gate will catch growth. +# +# The Playwright Chromium apt deps below are hand-derived against Playwright +# 1.48.0. When that target moves, the apt-get install line may need to +# add/drop packages - check the upstream list at +# https://github.com/microsoft/playwright/blob/v/packages/playwright-core/src/server/registry/nativeDeps.ts +# or run `npx playwright@ install-deps --dry-run` to diff. (Tracked +# only via this comment, not an ARG, since the apt package names are hardcoded +# below and a build-arg override would silently disagree with what's installed.) +ARG GO_VERSION=1.26.0 +ARG GOLANGCI_LINT_VERSION=1.62.0 +ARG PNPM_VERSION=9.15.9 + +# Single RUN to minimize image layers and clean apt lists in the same layer. +# Tools grouped by purpose; comments explain the why for each cluster so +# future maintainers can prune deliberately rather than guess. +RUN set -eux; \ + apt-get update; \ + apt-get install -y --no-install-recommends \ + # Native compile (CGO, pip wheels with C extensions, native npm modules) + build-essential \ + pkg-config \ + python3-dev \ + libssl-dev \ + # Useful CLI tools agents reach for. Small, broadly applicable. + ripgrep \ + fd-find \ + jq \ + # Playwright Chromium runtime deps. Browsers themselves are NOT + # preinstalled - users run `pnpm exec playwright install chromium` + # once and the download persists at ~/.cache/ms-playwright on the PV. + # This list mirrors `npx playwright@1.48.0 install-deps chromium`. + libnss3 \ + libnspr4 \ + libatk1.0-0 \ + libatk-bridge2.0-0 \ + libcups2 \ + libdrm2 \ + libxkbcommon0 \ + libxcomposite1 \ + libxdamage1 \ + libxfixes3 \ + libxrandr2 \ + libgbm1 \ + libpango-1.0-0 \ + libcairo2 \ + libasound2 \ + fonts-liberation \ + fonts-noto-color-emoji \ + ; \ + rm -rf /var/lib/apt/lists/* + +# Go toolchain - tarball install to /usr/local/go (the standard layout the +# go.dev installer creates). Add to PATH below. +# +# Integrity: go.dev publishes a per-tarball `.sha256` file (a bare hex hash, +# no filename), fetched alongside the tarball and verified before extraction. +# This is TOFU on first fetch; if the go.dev CDN itself is compromised both +# files are bad. Strongest available option without bumping the hash inline +# every Go release. +RUN set -eux; \ + arch="$(dpkg --print-architecture)"; \ + case "$arch" in \ + amd64) goarch=amd64 ;; \ + arm64) goarch=arm64 ;; \ + *) echo "unsupported arch: $arch" >&2; exit 1 ;; \ + esac; \ + curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-${goarch}.tar.gz" -o /tmp/go.tar.gz; \ + curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-${goarch}.tar.gz.sha256" -o /tmp/go.tar.gz.sha256; \ + echo "$(cat /tmp/go.tar.gz.sha256) /tmp/go.tar.gz" | sha256sum -c -; \ + tar -C /usr/local -xzf /tmp/go.tar.gz; \ + rm /tmp/go.tar.gz /tmp/go.tar.gz.sha256 + +# golangci-lint - prebuilt binary, no compilation needed. +# Integrity: each release publishes a single `*-checksums.txt` file with +# ` ` lines for every artifact. Grep for our tarball, +# pipe to `sha256sum -c -`. Download into /tmp and cd there so the relative +# filename in the checksum line resolves. +RUN set -eux; \ + arch="$(dpkg --print-architecture)"; \ + case "$arch" in \ + amd64) lintarch=amd64 ;; \ + arm64) lintarch=arm64 ;; \ + *) echo "unsupported arch: $arch" >&2; exit 1 ;; \ + esac; \ + tarball="golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${lintarch}.tar.gz"; \ + base="https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_LINT_VERSION}"; \ + curl -fsSL "${base}/${tarball}" -o "/tmp/${tarball}"; \ + curl -fsSL "${base}/golangci-lint-${GOLANGCI_LINT_VERSION}-checksums.txt" -o /tmp/golangci-lint-checksums.txt; \ + (cd /tmp && grep " ${tarball}$" golangci-lint-checksums.txt | sha256sum -c -); \ + tar -xz -C /tmp -f "/tmp/${tarball}"; \ + mv "/tmp/golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${lintarch}/golangci-lint" /usr/local/bin/; \ + rm -rf "/tmp/${tarball}" "/tmp/golangci-lint-${GOLANGCI_LINT_VERSION}-linux-${lintarch}" /tmp/golangci-lint-checksums.txt + +# Rust toolchain via rustup. RUSTUP_HOME/CARGO_HOME live in /usr/local so the +# install is system-wide and read-only for non-root users at runtime - +# matching the rest of the image. Users who need writable per-user state +# (e.g. `cargo install `, `rustup toolchain install nightly`) should +# override CARGO_HOME and RUSTUP_HOME to a path on the PV, e.g.: +# CARGO_HOME=~/.cargo RUSTUP_HOME=~/.rustup cargo install +# +# `--profile minimal` keeps the install lean (rustc + cargo + rust-std); we +# then explicitly add clippy and rustfmt, which agents working on Rust +# projects almost always reach for. `--profile default` would also work but +# pulls in rust-docs which we don't need. +# +# Integrity: download rustup-init for our arch and verify its SHA-256 +# against the upstream .sha256 sidecar before executing - matches the +# pattern used for Go/golangci-lint/yq above, rather than `curl | sh`. +ENV RUSTUP_HOME=/usr/local/rustup \ + CARGO_HOME=/usr/local/cargo +RUN set -eux; \ + arch="$(dpkg --print-architecture)"; \ + case "$arch" in \ + amd64) triplet=x86_64-unknown-linux-gnu ;; \ + arm64) triplet=aarch64-unknown-linux-gnu ;; \ + *) echo "unsupported arch: $arch" >&2; exit 1 ;; \ + esac; \ + base="https://static.rust-lang.org/rustup/dist/${triplet}"; \ + curl -fsSL "${base}/rustup-init" -o /tmp/rustup-init; \ + curl -fsSL "${base}/rustup-init.sha256" -o /tmp/rustup-init.sha256; \ + (cd /tmp && sha256sum -c rustup-init.sha256); \ + chmod +x /tmp/rustup-init; \ + /tmp/rustup-init -y --default-toolchain stable --no-modify-path --profile minimal; \ + /usr/local/cargo/bin/rustup component add --toolchain stable clippy rustfmt; \ + rm /tmp/rustup-init /tmp/rustup-init.sha256; \ + chmod -R a+rX "$RUSTUP_HOME" "$CARGO_HOME" + +# pnpm via corepack - install system-wide so users don't need +# `corepack enable` themselves. Drops shims into /usr/local/bin. +# +# No separate sha256 step: corepack downloads from registry.npmjs.org and +# verifies the tarball against the registry's `integrity` field (SHA-512) +# before activating. The other tools above pin upstream `.sha256` sidecars +# because they're served from CDNs that don't expose package integrity at +# the API level; npm does. Decision is deliberate, not an oversight. +RUN set -eux; \ + corepack enable; \ + corepack prepare "pnpm@${PNPM_VERSION}" --activate + +# fd is installed as `fdfind` on Debian to avoid clashing with the legacy +# fd package - symlink so agents calling plain `fd` still work. +RUN ln -s /usr/bin/fdfind /usr/local/bin/fd + +# yq is not in Debian stable; install the prebuilt binary from upstream +# (Mike Farah's yq, the Go implementation - the one most scripts assume). +# +# Integrity: mikefarah/yq publishes a `checksums` file with one row per +# artifact and multiple hash columns, plus a `checksums_hashes_order` file +# that lists which hash algorithm corresponds to each column. We compute the +# SHA-256 column index dynamically (defensive against upstream reordering) +# and verify the binary against that column before installing. +ARG YQ_VERSION=4.44.3 +RUN set -eux; \ + arch="$(dpkg --print-architecture)"; \ + target="yq_linux_${arch}"; \ + base="https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}"; \ + curl -fsSL "${base}/checksums" -o /tmp/yq-checksums; \ + curl -fsSL "${base}/checksums_hashes_order" -o /tmp/yq-hashes-order; \ + sha256_col="$(awk '/^SHA-256$/ {print NR + 1; exit}' /tmp/yq-hashes-order)"; \ + if [ -z "$sha256_col" ]; then echo "SHA-256 not in upstream checksums_hashes_order" >&2; exit 1; fi; \ + expected="$(awk -v t="$target" -v c="$sha256_col" '$1==t {print $c}' /tmp/yq-checksums)"; \ + if [ -z "$expected" ]; then echo "no checksum row for $target" >&2; exit 1; fi; \ + curl -fsSL "${base}/${target}" -o /tmp/yq; \ + echo "${expected} /tmp/yq" | sha256sum -c -; \ + mv /tmp/yq /usr/local/bin/yq; \ + chmod +x /usr/local/bin/yq; \ + rm /tmp/yq-checksums /tmp/yq-hashes-order + +# Append Go's toolchain bin and cargo's bin to whatever PATH the base image +# set. Appending rather than hardcoding the full list keeps the agent CLI +# dir (/data/.npm-global/bin) first - as the base image already arranges - +# and stays correct if the base image's PATH ever changes. +# +# Note: $GOPATH/bin (defaults to ~/go/bin, i.e. /data/home/go/bin in this +# image) is NOT added here - it's user-specific and lives on the PV. Users +# who `go install` tools and want them on PATH should add it to their shell +# profile (e.g. `export PATH=$PATH:$(go env GOPATH)/bin`). +ENV PATH=$PATH:/usr/local/go/bin:/usr/local/cargo/bin + +# Drop back to the unprivileged user. Matches the vanilla image's runtime +# user; the docker-entrypoint.sh in the base image handles root → kandev +# privilege drop via gosu for users who run the container as root. +USER kandev diff --git a/docs/docker.md b/docs/docker.md index 13b3b582fc..38f55aeaba 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -22,6 +22,16 @@ docker pull ghcr.io/kdlbs/kandev:latest docker pull ghcr.io/kdlbs/kandev:0.9.0 ``` +### Choosing your image: vanilla vs. universal + +Two flavors are published. The default vanilla image is smallest and bundles npm-installable agent CLIs only. The `:universal` image (~1.4 GB) adds language toolchains (Go, Rust, build-essential), linters, and Playwright Chromium system libs - pick this if your agents work on Go/Rust/Python projects or drive headless browsers. + +```bash +docker pull ghcr.io/kdlbs/kandev:universal +``` + +See [`images.md`](./images.md) for the full comparison, inclusion policy, and recipes for deriving your own image. + ## Building from Source From the repository root: diff --git a/docs/images.md b/docs/images.md new file mode 100644 index 0000000000..8c6c2f856c --- /dev/null +++ b/docs/images.md @@ -0,0 +1,146 @@ +# Choosing a Kandev Image + +Kandev publishes two container image flavors to GitHub Container Registry. Both are functionally identical kandev - same backend, same web UI, same persistence model. They differ only in what *else* is preinstalled in the image. + +| Tag | Size (compressed) | Contents | Pick when… | +|-----------------------------------------|-------------------|--------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------| +| `ghcr.io/kdlbs/kandev:X.Y.Z` | ~600 MB | kandev + Node 24 + npm + git + gh + python3 + pipx | You only need npm-installable agent CLIs (claude-code, codex, …) and want the smallest possible footprint. | +| `ghcr.io/kdlbs/kandev:X.Y.Z-universal` | ~1.4 GB | vanilla **+** language toolchains, build tools, Playwright Chromium deps | Your agents work on Go / Rust / Python projects, run native test suites, or drive headless browsers. | + +`:latest` aliases the vanilla image; `:universal` aliases the latest universal. **Tag pinning is strongly recommended in production** - `:latest` moves and we don't promise it stays the same forever. + +## What's in `:universal` (and what's not) + +**Language toolchains:** Go (latest stable), Rust (rustup, stable default toolchain), pnpm (preinstalled via corepack so `pnpm` is on `$PATH` for everyone). + +**Build essentials:** `build-essential` (gcc, g++, make, libc-dev), `pkg-config`, `python3-dev`, `libssl-dev`. These cover CGO compilation, native Python pip wheels, and native Node modules. + +**Linters and developer CLIs:** `golangci-lint`, `ripgrep`, `fd`, `jq`, `yq`. + +**Playwright Chromium system libraries** (browsers themselves are *not* preinstalled - see below). + +**Not included:** + +- **JDKs / .NET / Mono** - out of scope in v1; if these matter to your workflow, derive your own image (see *Customizing your image* below). +- **Playwright browsers** (`chromium`, `firefox`, `webkit`). Run `pnpm exec playwright install chromium` (or whichever browsers you need) once after the container starts. Downloads land at `~/.cache/ms-playwright`, which lives on the PV under `HOME=/data/home` and survives pod restarts. This keeps universal under 1.5 GB; it adds a one-time setup command per agent that wants browsers. +- **Database servers** (Postgres, MySQL, Redis). Use a sidecar or external service. +- **Kandev's own test dependencies beyond what's listed above.** Universal *is* enough to run kandev's backend Go tests, but the Playwright e2e suite needs the user to `pnpm install` and run `playwright install` first, same as on a fresh dev machine. + +## Inclusion policy + +We add a tool to universal when **all four** of these hold: + +1. It's commonly needed by agent-driven dev workflows (Go/Rust/Python/Node project work). +2. It's reasonably available in apt or a well-maintained prebuilt binary. +3. It adds less than 200 MB to the installed image. +4. It doesn't carry a license that conflicts with redistributing the image. + +We **decline** additions that fail any of these. The escape hatch is always *derive your own image* - see below. + +## Switching between flavors + +There is no migration. Both images mount `/data` the same way and use the same database schema. To switch a running deployment: + +**Docker:** +```bash +docker pull ghcr.io/kdlbs/kandev:X.Y.Z-universal +docker stop kandev && docker rm kandev +docker run -p 38429:38429 -v kandev-data:/data ghcr.io/kdlbs/kandev:X.Y.Z-universal +``` + +**Kubernetes:** +```bash +kubectl set image deployment/kandev kandev=ghcr.io/kdlbs/kandev:X.Y.Z-universal +``` + +All your data (database, worktrees, npm globals, agent auth state) carries over because it lives on the PV, not in the image. + +## Customizing your image + +Universal is opinionated. If you need something it doesn't include - a JDK, a private apt repository, a specific Node version, your company's CA cert - **derive your own image** from whichever base fits. + +The pattern: a tiny Dockerfile that does `FROM ghcr.io/kdlbs/kandev:X.Y.Z` (or `…:X.Y.Z-universal`), drops back to root, installs what you need, drops back to `kandev`. Build it, push it to a registry you control, point your deployment at it. + +### Recipe: add a JDK on top of universal + +```dockerfile +FROM ghcr.io/kdlbs/kandev:0.45.0-universal + +USER root +RUN apt-get update && apt-get install -y --no-install-recommends \ + openjdk-21-jdk-headless \ + && rm -rf /var/lib/apt/lists/* +USER kandev +``` + +```bash +docker build -t my-registry.example.com/kandev:0.45.0-jdk . +docker push my-registry.example.com/kandev:0.45.0-jdk +kubectl set image deployment/kandev kandev=my-registry.example.com/kandev:0.45.0-jdk +``` + +### Recipe: lightweight extras on top of vanilla + +If universal is too big for you but you need a couple of specific tools: + +```dockerfile +FROM ghcr.io/kdlbs/kandev:0.45.0 + +USER root +RUN apt-get update && apt-get install -y --no-install-recommends \ + make \ + rsync \ + && rm -rf /var/lib/apt/lists/* +USER kandev +``` + +### Recipe: bake a corporate CA cert + +```dockerfile +FROM ghcr.io/kdlbs/kandev:0.45.0-universal + +USER root +COPY corporate-ca.crt /usr/local/share/ca-certificates/ +RUN update-ca-certificates +USER kandev +``` + +### Recipe: preinstall a specific agent CLI version + +The Settings → Agents *Install* button calls `npm install -g` at runtime. If you want a fixed version baked into the image (so every container starts with it), do it at build time - and install **outside** `/data`, otherwise the named volume mounted at `/data` will shadow your install on every container after the first one is created: + +```dockerfile +FROM ghcr.io/kdlbs/kandev:0.45.0 + +USER root +# Install to /usr/local (NOT /data/.npm-global, which is on the named volume +# and only seeded on first volume creation). /usr/local/bin is already on +# PATH after /data/.npm-global/bin, so a user-installed runtime version +# would still win if anyone adds one later. +RUN npm install -g --prefix /usr/local @anthropic-ai/claude-code@1.2.3 +USER kandev +``` + +Note: kandev users will see the agent as already installed on the agents page. Baking it in means the version is tied to your image rather than to user choice - if a user runtime-installs a different version, that one wins (because the runtime path `/data/.npm-global/bin` precedes `/usr/local/bin` on `$PATH`). + +## What if I need something that doesn't fit any of these patterns? + +Two reasonable paths: + +1. **Open an issue** suggesting an addition to universal. Cite (a) the workflow that needs it, (b) the installed size, (c) whether it's apt-available or needs a prebuilt binary. We'll evaluate against the inclusion policy. +2. **Just maintain your own derived image.** This is what most teams end up doing for org-specific needs (corporate CAs, internal tools, JDK pinning). It's a 10-line Dockerfile and a 2-line CI step. + +## A note on building locally + +The repository also contains the source Dockerfiles (`Dockerfile` and `Dockerfile.universal`). You can build either locally: + +```bash +# Vanilla +docker build -t kandev:dev . + +# Universal (depends on a vanilla base - build vanilla first if you want a +# fully-from-source universal, or let BASE_IMAGE default to the published one) +docker build -f Dockerfile.universal --build-arg BASE_IMAGE=kandev:dev -t kandev:dev-universal . +``` + +In CI we always build vanilla first and pass its tag as `BASE_IMAGE` to the universal build, so a single release run produces a matched `vX.Y.Z` + `vX.Y.Z-universal` pair. diff --git a/docs/k8s.md b/docs/k8s.md index 2b927b0384..4ee88039a5 100644 --- a/docs/k8s.md +++ b/docs/k8s.md @@ -38,6 +38,16 @@ Or reference it in your K8s deployment: image: ghcr.io/kdlbs/kandev:latest ``` +### Choosing your image: vanilla vs. universal + +Kandev publishes two flavors: the default vanilla image (smallest, npm-installable agent CLIs only) and a `:universal` image (~1.4 GB) that adds language toolchains (Go, Rust, build-essential), linters, and Playwright Chromium system libs - useful when your agents work on Go/Rust/Python projects or drive headless browsers. + +```yaml +image: ghcr.io/kdlbs/kandev:universal +``` + +See [`images.md`](./images.md) for the full comparison, inclusion policy, and recipes for deriving your own image when you need something neither flavor includes. + ## Deploying to Kubernetes ### Quick Start @@ -80,6 +90,8 @@ No extra configuration is needed. The frontend automatically uses `window.locati The kandev image ships with `git`, `gh` (GitHub CLI), `node`, and `npm`, but **does not bundle the coding-agent CLIs** (`claude-code`, `codex`, `auggie`, etc.) — agent choice is per-user, and bundling all of them would bloat the image significantly. +> Looking to add tools *beyond* agent CLIs - language toolchains, build tools, internal CLIs? See [`images.md`](./images.md) for the universal-image option and recipes for deriving your own image. + To install an agent inside the running pod, open **Settings → Agents** in the UI and click **Install** on the agent card under "Available to Install". The backend runs the agent's hard-coded install script (`npm install -g `) and rescans on success. The image sets `NPM_CONFIG_PREFIX=/data/.npm-global` so user-installed npm globals land on the PV and **survive pod restarts and image upgrades**. The same persistence applies if you `kubectl exec` and install manually: