dreampiler/kandev
1
0
Fork 0
Code Issues 43 Pull Requests Actions 6 Packages Projects Releases Wiki Activity

chore: add provider-agnostic bootstrap (#1377)

Browse Source
This commit is contained in:
Carlos Florêncio
2026-06-14 22:05:49 +01:00
committed by GitHub
parent 828385be2a
commit d6cf4a1acc
6 changed files with 273 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Makefile
+10
View File
@@ -62,6 +62,8 @@ help:
@echo "Kandev - AI Agent Kanban Board"
@echo ""
@echo "Development Commands:"
@echo " bootstrap Install mise tools, workspace deps, and git hooks"
@echo " bootstrap-e2e Bootstrap plus Playwright browser/system deps"
@echo " dev Run backend + web via CLI (auto ports)"
@echo " dev-prod-db Run dev mode against the production db at ~/.kandev"
@echo " dev-backend Run backend in development mode (port 38429)"
@@ -129,6 +131,14 @@ help:
# Development
#
.PHONY: bootstrap
bootstrap:
@scripts/bootstrap-dev-env
.PHONY: bootstrap-e2e
bootstrap-e2e:
@scripts/bootstrap-dev-env --with-e2e
# Build the linux/amd64 agentctl on every host except Linux/x86_64 (where the
# host build IS already linux/amd64). Computed up here, OS-conditionally, so
# the recipe doesn't shell out to `uname` — that fails on Windows under cmd
README.md
+14 -8
View File
@@ -188,11 +188,20 @@ apps/
### Prerequisites
- Go 1.26+
- Node.js 18+
- pnpm
- [mise](https://mise.jdx.dev/) for the pinned toolchain in `mise.toml` (`make bootstrap` installs it when missing)
- Go 1.26, Node.js 24, pnpm 9.15.9 (installed by `make bootstrap`)
- Docker (optional)
### Environment Setup
```bash
# Install mise tools, workspace dependencies, and git hooks
make bootstrap
# Also install Playwright Chromium browser/deps for E2E
make bootstrap-e2e
```
### Running Dev Servers
```bash
@@ -216,11 +225,8 @@ make fmt # Format all code
### Pre-commit Hooks
```bash
# Install pre-commit (https://pre-commit.com/#install)
pipx install pre-commit
# Install git hooks
pre-commit install
# Installed and wired by make bootstrap
make doctor
```
</details>
docs/remote-cloud-environment.md
+16 -7
View File
@@ -4,11 +4,8 @@ Setup notes and caveats for developing Kandev in ephemeral cloud VMs (Cursor Clo
## Runtime requirements
- **Go 1.26** — install to `/usr/local/go` and ensure `PATH` includes `/usr/local/go/bin`.
- **Node.js 24** — use `nvm install 24 && nvm use 24`.
- **pnpm 9.15.9** — matches `packageManager` in `apps/package.json`. Install with `npm install -g pnpm@9.15.9`.
- **golangci-lint v2** — required for `make lint-backend`. Install with `go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@latest`.
- **gcc** — required for CGO (SQLite FTS5). Pre-installed on most Ubuntu-based cloud VMs.
- **mise** — `mise.toml` at the repo root pins the developer toolchain (Go 1.26, Node.js 24, pnpm 9.15.9, golangci-lint 2.9.0, go-licenses 1.6.0, pre-commit, etc.).
- **gcc / sqlite headers** — required for CGO (SQLite FTS5). `scripts/bootstrap-dev-env` installs these on common Linux/macOS package managers when it has permission.
- **Azure Repos PR creation (optional)** — only needed when testing `worktree.create_pr` against Azure remotes outside the published Docker image:
- [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) (`az`) on `PATH`
- DevOps extension: `az extension add --name azure-devops`
@@ -16,6 +13,18 @@ Setup notes and caveats for developing Kandev in ephemeral cloud VMs (Cursor Clo
- The `ghcr.io/kdlbs/kandev` image ships `gh`, `az`, and the `azure-devops` extension preinstalled (see root `Dockerfile`).
- Host routing table: `apps/backend/internal/agentctl/AGENTS.md`.
Provider prepare scripts should do auth + clone first, then run the shared bootstrap from the repo root:
```bash
scripts/bootstrap-dev-env
```
For E2E-capable environments:
```bash
scripts/bootstrap-dev-env --with-e2e
```
## Generated files before typecheck
Before running `make typecheck`, you must generate two files that are gitignored:
@@ -45,7 +54,7 @@ All documented in the root `Makefile`:
These apply to Cursor Cloud, Codex, and similar Firecracker-backed sandboxes:
- **golangci-lint PATH**: `go install` places the binary in `~/go/bin/`. Ensure `PATH` includes it (the update script appends to `~/.bashrc`, but in-session shells may need `export PATH="$PATH:$HOME/go/bin"`).
- **tool PATH**: `scripts/bootstrap-dev-env` activates mise for its own process. New shells should run `eval "$(mise activate bash)"` or use a shell profile that activates mise so `node`, `pnpm`, `go`, `golangci-lint`, and `pre-commit` resolve from the pinned toolchain.
- **First page load**: `make dev` compiles pages on first visit via Turbopack — expect ~25 s for the initial load.
- **CLI test flake**: `src/ports.test.ts > isPortInUse > returns false within the timeout when the host black-holes packets` fails because `192.0.2.1` behaves differently inside Firecracker. This is a known environment-specific flake, not a code issue.
- **Playwright browser installation**: `playwright install chromium` hangs during zip extraction (Node.js io_uring incompatibility with the Firecracker kernel). Workaround: `scripts/install-playwright-browsers.sh` runs the install with a timeout, then falls back to extracting the already-downloaded zips with `unzip`. The update script calls this automatically. System deps can be installed via `playwright install-deps chromium`.
- **Playwright browser installation**: `playwright install chromium` hangs during zip extraction (Node.js io_uring incompatibility with the Firecracker kernel). Workaround: `scripts/install-playwright-browsers.sh` runs the install with a timeout, then falls back to extracting the already-downloaded zips with `unzip`. `scripts/bootstrap-dev-env --with-e2e` calls this automatically. System deps can be installed via `playwright install-deps chromium`.
mise.toml
+15
View File
@@ -0,0 +1,15 @@
[tools]
node = "24"
pnpm = "9.15.9"
go = "1.26.0"
jq = "1.8.1"
ripgrep = "15.1.1"
uv = "0.11.21"
golangci-lint = "2.9.0"
git-cliff = "2.13.1"
"go:github.com/google/go-licenses" = { version = "v1.6.0", depends = "go" }
"pipx:pre-commit" = { version = "4.6.0", depends = "uv" }
[env]
NODE_OPTIONS = "--dns-result-order=ipv4first"
_.path = ["./apps/node_modules/.bin"]
scripts/bootstrap-dev-env
Executable
+190
View File
@@ -0,0 +1,190 @@
#!/usr/bin/env bash
# bootstrap-dev-env - provider-agnostic Kandev development environment setup.
#
# Cloud executor prepare scripts should handle provider concerns first:
# authentication, clone/checkout, workspace placement, and agentctl startup.
# After cloning, call this script from the repo root to install the Kandev
# toolchain and dependencies consistently across Sprites, Daytona, Modal,
# Codespaces, local VMs, and similar environments.
#
# Usage:
# scripts/bootstrap-dev-env [options]
#
# Options:
# --with-e2e Install Playwright browser/system deps for E2E
# --without-hooks Skip pre-commit hook wiring
# --without-node-modules Skip pnpm install
# --without-os-packages Skip best-effort OS package installation
# --without-mise-install Require mise to already be installed
# --provider NAME Record caller/provider name in logs only
# -h, --help Show this help
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
WITH_E2E=0
WITH_HOOKS=1
WITH_NODE_MODULES=1
WITH_OS_PACKAGES=1
WITH_MISE_INSTALL=1
PROVIDER="generic"
usage() {
awk 'NR > 1 { if ($0 !~ /^#/) exit; sub(/^# ?/, ""); print }' "$0"
exit "${1:-2}"
}
while [[ $# -gt 0 ]]; do
case "$1" in
--with-e2e) WITH_E2E=1; shift ;;
--without-hooks) WITH_HOOKS=0; shift ;;
--without-node-modules) WITH_NODE_MODULES=0; shift ;;
--without-os-packages) WITH_OS_PACKAGES=0; shift ;;
--without-mise-install) WITH_MISE_INSTALL=0; shift ;;
--provider) PROVIDER="${2:?--provider requires a value}"; shift 2 ;;
-h|--help) usage 0 ;;
*) echo "bootstrap-dev-env: unknown argument '$1'" >&2; usage 2 ;;
esac
done
log() { printf '[bootstrap:%s] %s\n' "$PROVIDER" "$*" >&2; }
warn() { printf '[bootstrap:%s] warning: %s\n' "$PROVIDER" "$*" >&2; }
die() { printf '[bootstrap:%s] error: %s\n' "$PROVIDER" "$*" >&2; exit 1; }
run_privileged() {
if [[ "${EUID:-$(id -u)}" -eq 0 ]]; then
"$@"
elif command -v sudo >/dev/null 2>&1; then
sudo "$@"
else
return 127
fi
}
install_os_packages() {
[[ "$WITH_OS_PACKAGES" -eq 1 ]] || { log "skipping OS packages"; return 0; }
if command -v apt-get >/dev/null 2>&1; then
log "installing OS packages with apt-get"
if ! run_privileged apt-get update; then
warn "apt-get requires root/sudo; skipping OS package installation"
return 0
fi
run_privileged env DEBIAN_FRONTEND=noninteractive apt-get install -y \
bash build-essential ca-certificates curl file git gzip lsof make openssh-client \
pipx pkg-config python3 python3-pip rsync sqlite3 tar unzip xz-utils zip \
libsqlite3-dev || warn "apt-get package installation failed; continuing because packages may already be present"
return 0
fi
if command -v dnf >/dev/null 2>&1; then
log "installing OS packages with dnf"
run_privileged dnf install -y \
bash ca-certificates curl file gcc gcc-c++ git gzip lsof make openssh-clients \
pipx pkgconf-pkg-config python3 python3-pip rsync sqlite sqlite-devel tar unzip xz zip \
|| warn "dnf package installation failed; continuing because packages may already be present"
return 0
fi
if command -v apk >/dev/null 2>&1; then
log "installing OS packages with apk"
run_privileged apk add --no-cache \
bash build-base ca-certificates curl file git gzip lsof make openssh-client \
pipx pkgconf python3 py3-pip rsync sqlite sqlite-dev tar unzip xz zip \
|| warn "apk package installation failed; continuing because packages may already be present"
return 0
fi
if command -v brew >/dev/null 2>&1; then
log "installing OS packages with Homebrew"
brew install bash ca-certificates curl file git lsof make openssh pipx pkg-config \
python rsync sqlite unzip xz zip \
|| warn "Homebrew package installation failed; continuing because packages may already be present"
return 0
fi
warn "no supported package manager found; install build tools, sqlite headers, unzip, pipx, and OpenSSH manually if needed"
}
install_mise() {
if command -v mise >/dev/null 2>&1; then
log "mise already installed: $(mise --version 2>/dev/null || true)"
persist_mise_activation
return 0
fi
[[ "$WITH_MISE_INSTALL" -eq 1 ]] || die "mise is not installed and --without-mise-install was passed"
command -v curl >/dev/null 2>&1 || die "curl is required to install mise"
log "installing mise"
tmp_mise_script="$(mktemp)"
trap 'rm -f "$tmp_mise_script"' EXIT
curl -fsSL https://mise.run -o "$tmp_mise_script"
sh "$tmp_mise_script"
export PATH="$HOME/.local/bin:$PATH"
command -v mise >/dev/null 2>&1 || die "mise install finished but mise is not on PATH"
persist_mise_activation
}
persist_mise_activation() {
[[ -n "${HOME:-}" ]] || return 0
local bashrc="$HOME/.bashrc"
touch "$bashrc"
grep -Fqx 'export PATH="$HOME/.local/bin:$PATH"' "$bashrc" 2>/dev/null \
|| printf '\nexport PATH="$HOME/.local/bin:$PATH"\n' >> "$bashrc"
grep -Fqx 'eval "$(mise activate bash)"' "$bashrc" 2>/dev/null \
|| printf 'eval "$(mise activate bash)"\n' >> "$bashrc"
}
activate_mise() {
export PATH="$HOME/.local/bin:$PATH"
cd "$REPO_ROOT"
[[ -f "$REPO_ROOT/mise.toml" ]] || die "missing mise.toml at repo root"
log "trusting mise.toml"
mise trust "$REPO_ROOT/mise.toml" >/dev/null 2>&1 || mise trust >/dev/null
log "installing mise tools"
mise install
log "activating mise environment for bootstrap"
eval "$(mise activate bash)"
}
install_node_modules() {
[[ "$WITH_NODE_MODULES" -eq 1 ]] || { log "skipping pnpm install"; return 0; }
log "installing pnpm workspace dependencies"
cd "$REPO_ROOT/apps"
pnpm install --frozen-lockfile
}
install_hooks() {
[[ "$WITH_HOOKS" -eq 1 ]] || { log "skipping git hook setup"; return 0; }
log "wiring pre-commit hooks"
cd "$REPO_ROOT"
make doctor
}
install_e2e() {
[[ "$WITH_E2E" -eq 1 ]] || { log "skipping Playwright E2E browser setup"; return 0; }
log "installing Playwright Chromium browser/deps"
"$REPO_ROOT/scripts/install-playwright-browsers.sh"
}
main() {
cd "$REPO_ROOT"
log "bootstrapping Kandev development environment at $REPO_ROOT"
install_os_packages
install_mise
activate_mise
install_node_modules
install_hooks
install_e2e
log "bootstrap complete"
}
main
scripts/install-playwright-browsers.sh
+28 -5
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# install-playwright-browsers.sh — install Playwright chromium browser
# install-playwright-browsers.sh — install Playwright chromium browser artifacts
# with a workaround for Firecracker VMs where Node.js io_uring-based
# zip extraction hangs indefinitely.
#
@@ -14,8 +14,30 @@ TIMEOUT_SECS=90
cd "$(dirname "$0")/../apps"
# Check if browsers are already fully installed
if pnpm --filter @kandev/web exec playwright install chromium --dry-run 2>&1 | grep -q "already installed"; then
PW_ARGS=(chromium chromium-headless-shell)
missing=1
found=0
if dry_run="$(pnpm --filter @kandev/web exec playwright install --dry-run "${PW_ARGS[@]}" 2>&1)"; then
missing=0
while IFS= read -r loc; do
[ -n "$loc" ] || continue
found=1
if [ ! -f "$loc/INSTALLATION_COMPLETE" ]; then
missing=1
break
fi
done < <(printf '%s\n' "$dry_run" | awk -F'Install location:[[:space:]]*' 'NF > 1 {print $2}' | sort -u)
if [ "$found" -eq 0 ]; then
missing=1
fi
else
echo "[playwright] dry-run failed; continuing with browser installation"
fi
if [ "$missing" -eq 0 ]; then
echo "[playwright] browsers already installed"
exit 0
fi
@@ -25,7 +47,7 @@ pnpm --filter @kandev/web exec playwright install-deps chromium 2>/dev/null || t
# Attempt normal install with timeout
rm -rf "$PW_CACHE/__dirlock"
if timeout "$TIMEOUT_SECS" pnpm --filter @kandev/web exec playwright install chromium 2>&1; then
if timeout "$TIMEOUT_SECS" pnpm --filter @kandev/web exec playwright install "${PW_ARGS[@]}" 2>&1; then
echo "[playwright] install completed normally"
exit 0
fi
@@ -42,7 +64,8 @@ for comp_dir in "$PW_CACHE"/chromium-* "$PW_CACHE"/chromium_headless_shell-* "$P
[ -f "$comp_dir/INSTALLATION_COMPLETE" ] && continue
comp_name=$(basename "$comp_dir" | sed 's/-[0-9]*$//')
zipfile=$(find /tmp -name "playwright-download-${comp_name}-*" -type f 2>/dev/null | sort | tail -1)
comp_name_alt=${comp_name//_/-}
zipfile=$(find /tmp \( -name "playwright-download-${comp_name}-*" -o -name "playwright-download-${comp_name_alt}-*" \) -type f 2>/dev/null | sort | tail -1)
if [ -z "$zipfile" ]; then
echo "[playwright] no zip found for $comp_name — skipping"