Files
kandev/apps/backend/internal/automation/models.go
T
João Salavisa 37b56ba11c feat(automations): reliable cron scheduling and webhook-driven push/CI triggers (#1972)
* feat(automations): reliable cron scheduling and webhook-driven push/CI triggers

Automations were "wounded": cron schedules mishandled timezones and some
never fired, and the GitHub push/CI trigger types were coming-soon stubs.

Cron (fix):
- The scheduler reduced every cron expression to a fixed interval derived
  only from a */N step in the minute/hour field, so any pinned schedule
  (0 9 * * *, 30 14 * * *, weekday ranges, day-of-month) never fired, and
  the configured timezone was parsed but never used.
- Replace the interval approximation with robfig/cron/v3: shouldFire now
  computes the true next fire time in the configured timezone (DST-aware,
  defaulting to UTC for host-independence), anchored to last-evaluated or
  creation time. Dead interval-parsing helpers and shorthand constants
  removed.

GitHub push + CI triggers (feat):
- Implement push and CI-failure triggers end to end via the existing
  per-workspace GitHub App webhook plumbing (#1810): the HMAC-verified
  webhook service now parses push and completed check_run events, resolves
  the installation to its workspaces, and publishes normalized bus events.
- A new automation subscriber matches enabled github_push / github_ci
  triggers by workspace + repo + branch/conclusion/check-name filters and
  fires them (deduped by commit SHA / check-run id). The old poll-based
  stub evaluators are removed; the PR poll path is unchanged.
- Subscribe the App manifest to push/check_run and enable both trigger
  types in the registry (CI defaults to the failure conclusion).

Docs:
- Update docs/public/feature-status.md and automation-and-mcp.md so the
  cron and push/CI rows reflect shipped reality.

Tests: regression tests pin the previously-never-firing cron shapes,
timezone/DST behavior, and host-independence; backend tests cover the
webhook parse->publish and subscriber match/fire/dedup flows for both
push and CI-failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): populate push.message and ci.url placeholders in fired trigger data

Address PR review blockers: the default push and CI automation prompts
reference {{push.message}} and {{ci.url}}, which the interpolator maps to
data["message"] / data["html_url"] — but the webhook subscriber never
populated those fields, so both always rendered empty.

- Parse head_commit.message (push) and check_run.html_url (CI) in the
  GitHub webhook service and carry them on the published bus payloads.
- Include "message" / "html_url" in the fired trigger data so the
  placeholders resolve; extend the webhook and subscriber tests to assert
  both flow through end to end.

Non-blocking review cleanups:
- Replace the misleading reuse of matchesAuthors for CI conclusions/check
  names with a clearly-named matchesFilterValue helper.
- Drop dead context.Background() line and its unused import from the cron
  regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): scope webhook triggers to active installs and per-branch dedup

Address CodeRabbit review on the push/CI webhook triggers:

- Security: resolveWorkspaceIDs now only returns connections whose status is
  active, so a suspended or revoked GitHub App installation can no longer fire
  push/CI automations.
- Correctness: include the branch in the push dedup key
  (push:owner/name@branch@sha) so the same commit pushed to two matching
  branches fires once per branch instead of being suppressed.
- Correctness: add an optional head-branch filter to github_ci triggers and
  gate CI matching on it, matching the push trigger's branch-scoping.
- Stability: make the webhook subscriber's Start retryable — a subscription
  failure rolls back partial subscriptions and leaves the subscriber
  un-started so a later Start re-attempts, instead of permanently disabling
  push/CI processing.
- Tests: cover multi-connection workspace resolution (dedup + inactive
  filtering), per-branch push dedup, and CI branch filtering.
- Docs: note the CI branch filter and per-branch push dedup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(automations): cover webhook subscriber Start retry-after-failure path

Add TestGitHubWebhookSubscriberStartRetryableAfterFailure with a minimal
flaky bus whose second Subscribe fails, pinning the retry contract: a
subscription failure rolls back the partial subscription and leaves the
subscriber un-started, and a later Start re-attempts and succeeds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 10:24:57 +01:00

258 lines
12 KiB
Go

// Package automation provides a generic automation system for Kandev.
// Automations are named rules with triggers (cron, GitHub events, webhooks)
// that automatically create and optionally start tasks when fired.
package automation
import (
"encoding/json"
"time"
"github.com/kandev/kandev/internal/github"
)
// TriggerType identifies the kind of trigger.
type TriggerType string
const (
TriggerTypeScheduled TriggerType = "scheduled"
TriggerTypeGitHubPR TriggerType = "github_pr"
TriggerTypeGitHubPush TriggerType = "github_push"
TriggerTypeGitHubCI TriggerType = "github_ci"
TriggerTypeWebhook TriggerType = "webhook"
)
const (
defaultBranchMain = "main"
exampleRepositoryOwner = "org/repo"
placeholderRepositoryOwner = "Repository (owner/name)"
triggerCategoryGitHub = "github"
triggerDataSourceKey = "source"
triggerDataSourceManual = "manual"
)
// RunStatus tracks the outcome of a trigger firing.
type RunStatus string
const (
RunStatusTriggered RunStatus = "triggered"
RunStatusTaskCreated RunStatus = "task_created"
RunStatusSucceeded RunStatus = "succeeded"
RunStatusFailed RunStatus = "failed"
RunStatusSkipped RunStatus = "skipped"
// RunStatusArchived is a read-time-derived terminal status: it is never
// written by a trigger-firing code path, only computed by ListRuns when
// a task_created run's generated task has been archived (archived_at
// set) — whether archived through the UI or by the agent itself (e.g.
// via an "archive this task" instruction). The task's outcome may still
// be known (succeeded/failed already recorded); this only overrides the
// still-open task_created status, never a real terminal one.
RunStatusArchived RunStatus = "archived"
// RunStatusCancelled is a read-time-derived terminal status: it is never
// written by a trigger-firing code path, only computed by ListRuns (and
// implied by CountActiveRuns' filter) for a task_created run whose
// generated task either no longer exists (deleted — outcome
// unrecoverable) or whose task's *current* session (task_sessions row
// with is_primary = 1) was explicitly put into the CANCELLED state —
// set only when an agent run was manually stopped (a deliberate
// cancellation, distinct from archiving; the task's own state is not
// touched by a stop and so isn't consulted here). Archived_at takes
// precedence over the cancelled-session check when both apply — see
// listRunsWithTaskState.
RunStatusCancelled RunStatus = "cancelled"
)
// ExecutionMode controls whether an automation firing creates a visible
// kanban task or a fire-and-forget ephemeral run.
type ExecutionMode string
const (
// ExecutionModeTask is the default: a trigger fires, a normal kanban
// task is created, the user can see/track/comment on it.
ExecutionModeTask ExecutionMode = "task"
// ExecutionModeRun is the lightweight path: a trigger fires, an
// ephemeral task is created so the session pipeline still works, but
// the task is hidden from the kanban. Only the AutomationRun row is
// surfaced to the user. Intended for "scheduled prompt that produces
// output the user just reads" use cases.
ExecutionModeRun ExecutionMode = "run"
)
// Valid reports whether the execution mode is a recognised value.
func (m ExecutionMode) Valid() bool {
return m == ExecutionModeTask || m == ExecutionModeRun
}
// Automation is a named rule with triggers, a prompt template, and agent/executor config.
type Automation struct {
ID string `json:"id" db:"id"`
WorkspaceID string `json:"workspace_id" db:"workspace_id"`
Name string `json:"name" db:"name"`
Description string `json:"description" db:"description"`
WorkflowID string `json:"workflow_id" db:"workflow_id"`
WorkflowStepID string `json:"workflow_step_id" db:"workflow_step_id"`
AgentProfileID string `json:"agent_profile_id" db:"agent_profile_id"`
ExecutorProfileID string `json:"executor_profile_id" db:"executor_profile_id"`
// RepositoryID is the explicit repository to use for trigger firings.
// Empty falls back to the workspace's first repository (for scheduled /
// webhook triggers) or the PR's repository (for github_pr triggers, which
// always override this field — see resolveAutomationRepository).
RepositoryID string `json:"repository_id" db:"repository_id"`
Prompt string `json:"prompt" db:"prompt"`
TaskTitleTemplate string `json:"task_title_template" db:"task_title_template"`
ExecutionMode ExecutionMode `json:"execution_mode" db:"execution_mode"`
Enabled bool `json:"enabled" db:"enabled"`
MaxConcurrentRuns int `json:"max_concurrent_runs" db:"max_concurrent_runs"`
WebhookSecret string `json:"-" db:"webhook_secret"`
LastTriggeredAt *time.Time `json:"last_triggered_at,omitempty" db:"last_triggered_at"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UpdatedAt time.Time `json:"updated_at" db:"updated_at"`
// Hydrated separately, not stored in the automations table.
Triggers []AutomationTrigger `json:"triggers" db:"-"`
}
// AutomationTrigger is a single trigger attached to an automation.
type AutomationTrigger struct {
ID string `json:"id" db:"id"`
AutomationID string `json:"automation_id" db:"automation_id"`
Type TriggerType `json:"type" db:"type"`
Config json.RawMessage `json:"config" db:"-"`
ConfigJSON string `json:"-" db:"config"`
Enabled bool `json:"enabled" db:"enabled"`
LastEvaluatedAt *time.Time `json:"last_evaluated_at,omitempty" db:"last_evaluated_at"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UpdatedAt time.Time `json:"updated_at" db:"updated_at"`
}
// AutomationRun records a single trigger firing for audit/observability.
type AutomationRun struct {
ID string `json:"id" db:"id"`
AutomationID string `json:"automation_id" db:"automation_id"`
TriggerID string `json:"trigger_id" db:"trigger_id"`
TriggerType TriggerType `json:"trigger_type" db:"trigger_type"`
TaskID string `json:"task_id,omitempty" db:"task_id"`
Status RunStatus `json:"status" db:"status"`
DedupKey string `json:"dedup_key" db:"dedup_key"`
TriggerData json.RawMessage `json:"trigger_data" db:"-"`
TriggerDataJSON string `json:"-" db:"trigger_data"`
ErrorMessage string `json:"error_message,omitempty" db:"error_message"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
}
// --- Trigger config types ---
// ScheduledTriggerConfig holds configuration for cron-based triggers.
type ScheduledTriggerConfig struct {
CronExpression string `json:"cron_expression"`
Timezone string `json:"timezone,omitempty"`
}
// GitHubPRTriggerConfig filters PR events.
type GitHubPRTriggerConfig struct {
Events []string `json:"events"` // opened, commented, merged, review_requested, closed
Repos []github.RepoFilter `json:"repos"` // empty = all repos
Branches []string `json:"branches,omitempty"` // base branch filter
Authors []string `json:"authors,omitempty"` // PR author filter
Labels []string `json:"labels,omitempty"` // label filter
ExcludeDraft bool `json:"exclude_draft,omitempty"`
}
// GitHubPushTriggerConfig filters push-to-branch events.
type GitHubPushTriggerConfig struct {
Repos []github.RepoFilter `json:"repos"`
Branches []string `json:"branches"` // glob patterns: ["main", "release/*"]
}
// GitHubCITriggerConfig filters CI completion events.
type GitHubCITriggerConfig struct {
Repos []github.RepoFilter `json:"repos"`
Conclusions []string `json:"conclusions"` // success, failure, etc.
CheckNames []string `json:"check_names,omitempty"`
Branches []string `json:"branches,omitempty"` // head-branch glob filter; empty = all
}
// WebhookTriggerConfig holds configuration for webhook triggers.
type WebhookTriggerConfig struct {
FilterExpression string `json:"filter_expression,omitempty"`
}
// --- Request/response DTOs ---
// CreateAutomationRequest is the payload for creating an automation.
type CreateAutomationRequest struct {
WorkspaceID string `json:"workspace_id"`
Name string `json:"name"`
Description string `json:"description"`
WorkflowID string `json:"workflow_id"`
WorkflowStepID string `json:"workflow_step_id"`
AgentProfileID string `json:"agent_profile_id"`
ExecutorProfileID string `json:"executor_profile_id"`
RepositoryID string `json:"repository_id"`
Prompt string `json:"prompt"`
TaskTitleTemplate string `json:"task_title_template"`
ExecutionMode ExecutionMode `json:"execution_mode"`
MaxConcurrentRuns int `json:"max_concurrent_runs"`
Triggers []CreateTriggerSpec `json:"triggers"`
}
// CreateTriggerSpec defines a trigger to add during automation creation.
type CreateTriggerSpec struct {
Type TriggerType `json:"type"`
Config json.RawMessage `json:"config"`
Enabled bool `json:"enabled"`
}
// UpdateAutomationRequest is the payload for updating an automation.
type UpdateAutomationRequest struct {
Name *string `json:"name,omitempty"`
Description *string `json:"description,omitempty"`
WorkflowID *string `json:"workflow_id,omitempty"`
WorkflowStepID *string `json:"workflow_step_id,omitempty"`
AgentProfileID *string `json:"agent_profile_id,omitempty"`
ExecutorProfileID *string `json:"executor_profile_id,omitempty"`
RepositoryID *string `json:"repository_id,omitempty"`
Prompt *string `json:"prompt,omitempty"`
TaskTitleTemplate *string `json:"task_title_template,omitempty"`
ExecutionMode *ExecutionMode `json:"execution_mode,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
MaxConcurrentRuns *int `json:"max_concurrent_runs,omitempty"`
}
// AddTriggerRequest adds a trigger to an existing automation.
type AddTriggerRequest struct {
AutomationID string `json:"automation_id"`
Type TriggerType `json:"type"`
Config json.RawMessage `json:"config"`
Enabled bool `json:"enabled"`
}
// UpdateTriggerRequest updates a trigger's config or enabled state.
type UpdateTriggerRequest struct {
Config *json.RawMessage `json:"config,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
}
// CreateAutomationResponse wraps a newly-created automation and returns the
// generated webhook secret in plaintext exactly once, so the UI can show the
// user a value they can paste into an external system. Subsequent GET / list
// responses keep the secret hidden (WebhookSecret is `json:"-"`) — the
// dedicated reveal endpoint is the way back to it.
type CreateAutomationResponse struct {
*Automation
WebhookSecret string `json:"webhook_secret"`
}
// RevealWebhookSecretResponse returns the webhook secret for an automation.
type RevealWebhookSecretResponse struct {
WebhookSecret string `json:"webhook_secret"`
}
// AutomationTriggeredEvent is published when a trigger fires.
type AutomationTriggeredEvent struct {
AutomationID string `json:"automation_id"`
TriggerID string `json:"trigger_id"`
TriggerType TriggerType `json:"trigger_type"`
TriggerData json.RawMessage `json:"trigger_data"`
DedupKey string `json:"dedup_key"`
}