Files
kandev/apps/backend/internal/github/deployment_app_manifest.go
T
João Salavisa 37b56ba11c feat(automations): reliable cron scheduling and webhook-driven push/CI triggers (#1972)
* feat(automations): reliable cron scheduling and webhook-driven push/CI triggers

Automations were "wounded": cron schedules mishandled timezones and some
never fired, and the GitHub push/CI trigger types were coming-soon stubs.

Cron (fix):
- The scheduler reduced every cron expression to a fixed interval derived
  only from a */N step in the minute/hour field, so any pinned schedule
  (0 9 * * *, 30 14 * * *, weekday ranges, day-of-month) never fired, and
  the configured timezone was parsed but never used.
- Replace the interval approximation with robfig/cron/v3: shouldFire now
  computes the true next fire time in the configured timezone (DST-aware,
  defaulting to UTC for host-independence), anchored to last-evaluated or
  creation time. Dead interval-parsing helpers and shorthand constants
  removed.

GitHub push + CI triggers (feat):
- Implement push and CI-failure triggers end to end via the existing
  per-workspace GitHub App webhook plumbing (#1810): the HMAC-verified
  webhook service now parses push and completed check_run events, resolves
  the installation to its workspaces, and publishes normalized bus events.
- A new automation subscriber matches enabled github_push / github_ci
  triggers by workspace + repo + branch/conclusion/check-name filters and
  fires them (deduped by commit SHA / check-run id). The old poll-based
  stub evaluators are removed; the PR poll path is unchanged.
- Subscribe the App manifest to push/check_run and enable both trigger
  types in the registry (CI defaults to the failure conclusion).

Docs:
- Update docs/public/feature-status.md and automation-and-mcp.md so the
  cron and push/CI rows reflect shipped reality.

Tests: regression tests pin the previously-never-firing cron shapes,
timezone/DST behavior, and host-independence; backend tests cover the
webhook parse->publish and subscriber match/fire/dedup flows for both
push and CI-failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): populate push.message and ci.url placeholders in fired trigger data

Address PR review blockers: the default push and CI automation prompts
reference {{push.message}} and {{ci.url}}, which the interpolator maps to
data["message"] / data["html_url"] — but the webhook subscriber never
populated those fields, so both always rendered empty.

- Parse head_commit.message (push) and check_run.html_url (CI) in the
  GitHub webhook service and carry them on the published bus payloads.
- Include "message" / "html_url" in the fired trigger data so the
  placeholders resolve; extend the webhook and subscriber tests to assert
  both flow through end to end.

Non-blocking review cleanups:
- Replace the misleading reuse of matchesAuthors for CI conclusions/check
  names with a clearly-named matchesFilterValue helper.
- Drop dead context.Background() line and its unused import from the cron
  regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): scope webhook triggers to active installs and per-branch dedup

Address CodeRabbit review on the push/CI webhook triggers:

- Security: resolveWorkspaceIDs now only returns connections whose status is
  active, so a suspended or revoked GitHub App installation can no longer fire
  push/CI automations.
- Correctness: include the branch in the push dedup key
  (push:owner/name@branch@sha) so the same commit pushed to two matching
  branches fires once per branch instead of being suppressed.
- Correctness: add an optional head-branch filter to github_ci triggers and
  gate CI matching on it, matching the push trigger's branch-scoping.
- Stability: make the webhook subscriber's Start retryable — a subscription
  failure rolls back partial subscriptions and leaves the subscriber
  un-started so a later Start re-attempts, instead of permanently disabling
  push/CI processing.
- Tests: cover multi-connection workspace resolution (dedup + inactive
  filtering), per-branch push dedup, and CI branch filtering.
- Docs: note the CI branch filter and per-branch push dedup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(automations): cover webhook subscriber Start retry-after-failure path

Add TestGitHubWebhookSubscriberStartRetryableAfterFailure with a minimal
flaky bus whose second Subscribe fails, pinning the retry contract: a
subscription failure rolls back the partial subscription and leaves the
subscriber un-started, and a later Start re-attempts and succeeds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 10:24:57 +01:00

352 lines
11 KiB
Go

package github
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"net"
"net/netip"
"net/url"
"regexp"
"strings"
"time"
"github.com/google/uuid"
)
const (
DeploymentAppManifestRevision = 1
deploymentAppManifestFlowTTL = time.Hour
deploymentAppNameMaxLength = 34
)
var (
ErrDeploymentAppManifestOwnerInvalid = errors.New("GitHub App owner is invalid")
ErrAppRegistrationIDInvalid = errors.New("GitHub App registration ID is invalid")
ErrAppRegistrationVisibilityInvalid = errors.New("GitHub App visibility is invalid")
ErrDeploymentAppManifestStateUnavailable = errors.New(
"GitHub App manifest state is expired, consumed, or invalid",
)
ErrPublicGitHubBaseURLInvalid = errors.New(
"public GitHub base URL must be an HTTPS origin without credentials, path, query, or fragment",
)
ErrPublicGitHubBaseURLNotGlobal = errors.New(
"public GitHub base URL must resolve only to globally routable addresses",
)
ErrPublicGitHubBaseURLUnresolvable = errors.New(
"public GitHub base URL could not be resolved",
)
githubOwnerLoginPattern = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?$`)
globallyRoutableSpecialAddressRanges = mustParseAddressPrefixes(
"192.0.0.9/32",
"192.0.0.10/32",
"2001:1::1/128",
"2001:1::2/128",
"2001:1::3/128",
"2001:3::/32",
"2001:4:112::/48",
"2001:20::/28",
"2001:30::/28",
)
nonGlobalAddressRanges = mustParseAddressPrefixes(
"0.0.0.0/8",
"10.0.0.0/8",
"100.64.0.0/10",
"127.0.0.0/8",
"169.254.0.0/16",
"172.16.0.0/12",
"192.0.0.0/24",
"192.0.2.0/24",
"192.88.99.0/24",
"192.168.0.0/16",
"198.18.0.0/15",
"198.51.100.0/24",
"203.0.113.0/24",
"224.0.0.0/4",
"240.0.0.0/4",
"::/128",
"::1/128",
"64:ff9b:1::/48",
"100::/64",
"100:0:0:1::/64",
"2001::/23",
"2001:db8::/32",
"2002::/16",
"3fff::/20",
"5f00::/16",
"fc00::/7",
"fe80::/10",
"fec0::/10",
"ff00::/8",
)
)
type ManifestOwnerType string
const (
ManifestOwnerUser ManifestOwnerType = "user"
ManifestOwnerOrganization ManifestOwnerType = "organization"
)
type DeploymentAppManifestHook struct {
URL string `json:"url"`
Active bool `json:"active"`
}
type DeploymentAppManifest struct {
Name string `json:"name"`
Description string `json:"description"`
URL string `json:"url"`
HookAttributes DeploymentAppManifestHook `json:"hook_attributes"`
RedirectURL string `json:"redirect_url"`
CallbackURLs []string `json:"callback_urls"`
SetupURL string `json:"setup_url"`
Public bool `json:"public"`
DefaultPermissions map[string]string `json:"default_permissions"`
DefaultEvents []string `json:"default_events"`
RequestOAuthOnInstall bool `json:"request_oauth_on_install"`
SetupOnUpdate bool `json:"setup_on_update"`
}
type DeploymentAppManifestSubmission struct {
Revision int
RegistrationURL string
Manifest DeploymentAppManifest
}
type AppRegistrationManifestRequest struct {
RegistrationID string
OwnerType ManifestOwnerType
OwnerLogin string
PublicBaseURL string
Visibility AppRegistrationVisibility
}
func BuildAppRegistrationManifest(
request AppRegistrationManifestRequest,
) (DeploymentAppManifestSubmission, error) {
registrationID, err := uuid.Parse(request.RegistrationID)
if err != nil || registrationID.String() != request.RegistrationID {
return DeploymentAppManifestSubmission{}, ErrAppRegistrationIDInvalid
}
public, err := manifestVisibility(request.Visibility)
if err != nil {
return DeploymentAppManifestSubmission{}, err
}
submission, err := BuildDeploymentAppManifest(
request.OwnerType, request.OwnerLogin, request.PublicBaseURL,
)
if err != nil {
return DeploymentAppManifestSubmission{}, err
}
baseURL := strings.TrimRight(submission.Manifest.URL, "/") +
"/api/v1/github/app/registrations/" + registrationID.String()
submission.Manifest.HookAttributes.URL = baseURL + "/webhook"
submission.Manifest.RedirectURL = baseURL + "/manifest/callback"
submission.Manifest.CallbackURLs = []string{baseURL + "/personal/callback"}
submission.Manifest.SetupURL = baseURL + "/install/callback"
submission.Manifest.Public = public
return submission, nil
}
func manifestVisibility(visibility AppRegistrationVisibility) (bool, error) {
switch visibility {
case "", AppRegistrationVisibilityPrivate:
return false, nil
case AppRegistrationVisibilityPublic:
return true, nil
default:
return false, ErrAppRegistrationVisibilityInvalid
}
}
type PublicGitHubBaseURLResolver interface {
LookupNetIP(context.Context, string, string) ([]netip.Addr, error)
}
func ValidatePublicGitHubBaseURL(
ctx context.Context,
rawURL string,
resolver PublicGitHubBaseURLResolver,
) (string, error) {
parsed, err := parsePublicGitHubBaseURL(rawURL)
if err != nil {
return "", err
}
hostname := strings.ToLower(strings.TrimSuffix(parsed.Hostname(), "."))
if address, parseErr := netip.ParseAddr(hostname); parseErr == nil {
if !isGloballyRoutable(address) {
return "", ErrPublicGitHubBaseURLNotGlobal
}
return canonicalPublicOrigin(parsed, hostname, address), nil
}
if resolver == nil {
resolver = net.DefaultResolver
}
addresses, err := resolver.LookupNetIP(ctx, "ip", hostname)
if err != nil || len(addresses) == 0 {
return "", ErrPublicGitHubBaseURLUnresolvable
}
for _, address := range addresses {
if !isGloballyRoutable(address) {
return "", ErrPublicGitHubBaseURLNotGlobal
}
}
return canonicalPublicOrigin(parsed, hostname, netip.Addr{}), nil
}
func BuildDeploymentAppManifest(
ownerType ManifestOwnerType,
ownerLogin, publicBaseURL string,
) (DeploymentAppManifestSubmission, error) {
if !validManifestOwner(ownerType, ownerLogin) {
return DeploymentAppManifestSubmission{}, ErrDeploymentAppManifestOwnerInvalid
}
baseURL, err := canonicalManifestBaseURL(publicBaseURL)
if err != nil {
return DeploymentAppManifestSubmission{}, err
}
return DeploymentAppManifestSubmission{
Revision: DeploymentAppManifestRevision,
RegistrationURL: manifestRegistrationURL(ownerType, ownerLogin),
Manifest: DeploymentAppManifest{
Name: deploymentAppName(ownerLogin, baseURL),
Description: "Repository automation for Kandev workspaces.",
URL: baseURL,
HookAttributes: DeploymentAppManifestHook{
URL: baseURL + "/api/v1/github/app/webhook", Active: true,
},
RedirectURL: baseURL + "/api/v1/github/app/registration/callback",
CallbackURLs: []string{baseURL + "/api/v1/github/personal-connection/callback"},
SetupURL: baseURL + "/api/v1/github/app/install/callback",
Public: false,
DefaultPermissions: map[string]string{
"actions": "read", "administration": "read", "checks": "read",
"contents": "write", "issues": "write", "members": "read",
"metadata": "read", "pull_requests": "write", "statuses": "read",
"workflows": "write",
},
// push and check_run enable webhook-driven github_push/github_ci automation
// triggers. Deployment caveat: GitHub Apps only deliver events a user has
// explicitly subscribed to at installation time, so existing installations
// created before this manifest change must be re-accepted (reinstalled, or
// updated via the App's GitHub settings page) before push/check_run
// webhooks start arriving for them.
DefaultEvents: []string{
"installation", "installation_repositories", "github_app_authorization",
"push", "check_run",
},
RequestOAuthOnInstall: true,
SetupOnUpdate: false,
},
}, nil
}
func DeploymentAppManifestFlowExpiresAt(createdAt time.Time) time.Time {
return createdAt.UTC().Add(deploymentAppManifestFlowTTL)
}
func ValidateDeploymentAppManifestFlow(expiresAt time.Time, consumedAt *time.Time, now time.Time) error {
if consumedAt != nil || expiresAt.IsZero() || !expiresAt.After(now.UTC()) {
return ErrDeploymentAppManifestStateUnavailable
}
return nil
}
func parsePublicGitHubBaseURL(rawURL string) (*url.URL, error) {
if strings.TrimSpace(rawURL) != rawURL || rawURL == "" {
return nil, ErrPublicGitHubBaseURLInvalid
}
parsed, err := url.Parse(rawURL)
if err != nil || !validPublicOriginShape(parsed) {
return nil, ErrPublicGitHubBaseURLInvalid
}
if parsed.Hostname() == "" || strings.HasSuffix(parsed.Host, ":") {
return nil, ErrPublicGitHubBaseURLInvalid
}
return parsed, nil
}
func validPublicOriginShape(parsed *url.URL) bool {
return parsed.Scheme == "https" && parsed.Host != "" && parsed.User == nil &&
parsed.RawQuery == "" && !parsed.ForceQuery && parsed.Fragment == "" &&
(parsed.Path == "" || parsed.Path == "/") && parsed.RawPath == "" && parsed.Opaque == ""
}
func canonicalManifestBaseURL(rawURL string) (string, error) {
parsed, err := parsePublicGitHubBaseURL(rawURL)
if err != nil {
return "", err
}
hostname := strings.ToLower(strings.TrimSuffix(parsed.Hostname(), "."))
address, _ := netip.ParseAddr(hostname)
return canonicalPublicOrigin(parsed, hostname, address), nil
}
func canonicalPublicOrigin(parsed *url.URL, hostname string, address netip.Addr) string {
host := hostname
if address.IsValid() && address.Is6() {
host = "[" + hostname + "]"
}
if port := parsed.Port(); port != "" && port != "443" {
host = net.JoinHostPort(hostname, port)
}
return "https://" + host
}
func validManifestOwner(ownerType ManifestOwnerType, ownerLogin string) bool {
return (ownerType == ManifestOwnerUser || ownerType == ManifestOwnerOrganization) &&
githubOwnerLoginPattern.MatchString(ownerLogin)
}
func manifestRegistrationURL(ownerType ManifestOwnerType, ownerLogin string) string {
if ownerType == ManifestOwnerOrganization {
return "https://github.com/organizations/" + url.PathEscape(ownerLogin) + "/settings/apps/new"
}
return "https://github.com/settings/apps/new"
}
func deploymentAppName(ownerLogin, baseURL string) string {
digest := sha256.Sum256([]byte(ownerLogin + "\x00" + baseURL))
suffix := hex.EncodeToString(digest[:4])
prefix := "Kandev " + ownerLogin
maxPrefixLength := deploymentAppNameMaxLength - len(suffix) - 1
if len(prefix) > maxPrefixLength {
prefix = strings.TrimRight(prefix[:maxPrefixLength], "-")
}
return fmt.Sprintf("%s-%s", prefix, suffix)
}
func isGloballyRoutable(address netip.Addr) bool {
if !address.IsValid() || address.Zone() != "" {
return false
}
address = address.Unmap()
if !address.IsGlobalUnicast() || address.IsPrivate() || address.IsLoopback() ||
address.IsLinkLocalUnicast() || address.IsLinkLocalMulticast() || address.IsMulticast() ||
address.IsUnspecified() {
return false
}
for _, addressRange := range globallyRoutableSpecialAddressRanges {
if addressRange.Contains(address) {
return true
}
}
for _, addressRange := range nonGlobalAddressRanges {
if addressRange.Contains(address) {
return false
}
}
return true
}
func mustParseAddressPrefixes(values ...string) []netip.Prefix {
prefixes := make([]netip.Prefix, 0, len(values))
for _, value := range values {
prefixes = append(prefixes, netip.MustParsePrefix(value))
}
return prefixes
}