Files
kandev/apps/backend/internal/automation/scheduler.go
T
João Salavisa 37b56ba11c feat(automations): reliable cron scheduling and webhook-driven push/CI triggers (#1972)
* feat(automations): reliable cron scheduling and webhook-driven push/CI triggers

Automations were "wounded": cron schedules mishandled timezones and some
never fired, and the GitHub push/CI trigger types were coming-soon stubs.

Cron (fix):
- The scheduler reduced every cron expression to a fixed interval derived
  only from a */N step in the minute/hour field, so any pinned schedule
  (0 9 * * *, 30 14 * * *, weekday ranges, day-of-month) never fired, and
  the configured timezone was parsed but never used.
- Replace the interval approximation with robfig/cron/v3: shouldFire now
  computes the true next fire time in the configured timezone (DST-aware,
  defaulting to UTC for host-independence), anchored to last-evaluated or
  creation time. Dead interval-parsing helpers and shorthand constants
  removed.

GitHub push + CI triggers (feat):
- Implement push and CI-failure triggers end to end via the existing
  per-workspace GitHub App webhook plumbing (#1810): the HMAC-verified
  webhook service now parses push and completed check_run events, resolves
  the installation to its workspaces, and publishes normalized bus events.
- A new automation subscriber matches enabled github_push / github_ci
  triggers by workspace + repo + branch/conclusion/check-name filters and
  fires them (deduped by commit SHA / check-run id). The old poll-based
  stub evaluators are removed; the PR poll path is unchanged.
- Subscribe the App manifest to push/check_run and enable both trigger
  types in the registry (CI defaults to the failure conclusion).

Docs:
- Update docs/public/feature-status.md and automation-and-mcp.md so the
  cron and push/CI rows reflect shipped reality.

Tests: regression tests pin the previously-never-firing cron shapes,
timezone/DST behavior, and host-independence; backend tests cover the
webhook parse->publish and subscriber match/fire/dedup flows for both
push and CI-failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): populate push.message and ci.url placeholders in fired trigger data

Address PR review blockers: the default push and CI automation prompts
reference {{push.message}} and {{ci.url}}, which the interpolator maps to
data["message"] / data["html_url"] — but the webhook subscriber never
populated those fields, so both always rendered empty.

- Parse head_commit.message (push) and check_run.html_url (CI) in the
  GitHub webhook service and carry them on the published bus payloads.
- Include "message" / "html_url" in the fired trigger data so the
  placeholders resolve; extend the webhook and subscriber tests to assert
  both flow through end to end.

Non-blocking review cleanups:
- Replace the misleading reuse of matchesAuthors for CI conclusions/check
  names with a clearly-named matchesFilterValue helper.
- Drop dead context.Background() line and its unused import from the cron
  regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): scope webhook triggers to active installs and per-branch dedup

Address CodeRabbit review on the push/CI webhook triggers:

- Security: resolveWorkspaceIDs now only returns connections whose status is
  active, so a suspended or revoked GitHub App installation can no longer fire
  push/CI automations.
- Correctness: include the branch in the push dedup key
  (push:owner/name@branch@sha) so the same commit pushed to two matching
  branches fires once per branch instead of being suppressed.
- Correctness: add an optional head-branch filter to github_ci triggers and
  gate CI matching on it, matching the push trigger's branch-scoping.
- Stability: make the webhook subscriber's Start retryable — a subscription
  failure rolls back partial subscriptions and leaves the subscriber
  un-started so a later Start re-attempts, instead of permanently disabling
  push/CI processing.
- Tests: cover multi-connection workspace resolution (dedup + inactive
  filtering), per-branch push dedup, and CI branch filtering.
- Docs: note the CI branch filter and per-branch push dedup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(automations): cover webhook subscriber Start retry-after-failure path

Add TestGitHubWebhookSubscriberStartRetryableAfterFailure with a minimal
flaky bus whose second Subscribe fails, pinning the retry contract: a
subscription failure rolls back the partial subscription and leaves the
subscriber un-started, and a later Start re-attempts and succeeds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 10:24:57 +01:00

207 lines
6.0 KiB
Go

package automation
import (
"context"
"encoding/json"
"fmt"
"strings"
"sync"
"time"
"github.com/robfig/cron/v3"
"go.uber.org/zap"
"github.com/kandev/kandev/internal/common/logger"
)
const defaultSchedulerCheckInterval = 30 * time.Second
// CronScheduler evaluates scheduled triggers on a timer.
// It checks all enabled scheduled triggers and fires those whose cron expression
// indicates they should run based on time elapsed since last evaluation.
type CronScheduler struct {
svc *Service
logger *logger.Logger
cancel context.CancelFunc
wg sync.WaitGroup
started bool
}
// NewCronScheduler creates a new cron scheduler.
func NewCronScheduler(svc *Service, log *logger.Logger) *CronScheduler {
return &CronScheduler{
svc: svc,
logger: log,
}
}
// Start begins the scheduler loop.
func (cs *CronScheduler) Start(ctx context.Context) {
if cs.started {
return
}
cs.started = true
ctx, cs.cancel = context.WithCancel(ctx)
cs.wg.Add(1)
go cs.loop(ctx)
cs.logger.Info("automation cron scheduler started")
}
// Stop cancels the scheduler and waits for it to finish.
func (cs *CronScheduler) Stop() {
if !cs.started {
return
}
if cs.cancel != nil {
cs.cancel()
}
cs.wg.Wait()
cs.started = false
cs.logger.Info("automation cron scheduler stopped")
}
func (cs *CronScheduler) loop(ctx context.Context) {
defer cs.wg.Done()
// Initial check on startup.
cs.evaluate(ctx)
ticker := time.NewTicker(defaultSchedulerCheckInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
cs.evaluate(ctx)
}
}
}
// evaluate checks all enabled scheduled triggers and fires those that are due.
func (cs *CronScheduler) evaluate(ctx context.Context) {
triggers, err := cs.svc.Store().ListEnabledTriggersByType(ctx, TriggerTypeScheduled)
if err != nil {
cs.logger.Error("failed to list scheduled triggers", zap.Error(err))
return
}
if len(triggers) == 0 {
return
}
now := time.Now()
for i := range triggers {
t := &triggers[i]
if cs.shouldFire(t, now) {
cs.fire(ctx, t, now)
}
}
}
// shouldFire determines if a scheduled trigger is due based on its cron
// expression. It computes the next fire time after the trigger's anchor (the
// last time it was evaluated, or its creation time on first evaluation) in the
// trigger's configured timezone, and reports whether that time has arrived.
//
// This is a true wall-clock schedule, not an interval approximation: pinned
// expressions ("0 9 * * *"), weekday ranges, and day-of-month schedules all
// resolve to concrete instants, and the configured timezone (including its DST
// transitions) is honored. A schedule missed while the scheduler was down
// fires once on the next tick rather than once per missed occurrence, matching
// the per-minute dedup key in fire().
func (cs *CronScheduler) shouldFire(t *AutomationTrigger, now time.Time) bool {
var cfg ScheduledTriggerConfig
if err := json.Unmarshal(t.Config, &cfg); err != nil {
cs.logger.Debug("invalid scheduled trigger config",
zap.String("trigger_id", t.ID), zap.Error(err))
return false
}
if cfg.CronExpression == "" {
return false
}
// Anchor the next-fire computation to the last evaluation, falling back to
// the trigger's creation time so a pinned schedule created mid-day doesn't
// fire immediately — its first fire is the next scheduled occurrence after
// creation.
anchor := t.CreatedAt
if t.LastEvaluatedAt != nil {
anchor = *t.LastEvaluatedAt
}
next, err := nextCronFire(cfg.CronExpression, cfg.Timezone, anchor)
if err != nil {
cs.logger.Debug("unparseable cron expression",
zap.String("trigger_id", t.ID),
zap.String("expression", cfg.CronExpression),
zap.Error(err))
return false
}
return !next.After(now)
}
func (cs *CronScheduler) fire(ctx context.Context, t *AutomationTrigger, now time.Time) {
data, _ := json.Marshal(map[string]string{
triggerDataSourceKey: string(TriggerTypeScheduled),
"timestamp": now.Format(time.RFC3339),
})
dedupKey := fmt.Sprintf("scheduled:%s:%d", t.ID, now.Unix()/60) // Dedup by minute
if err := cs.svc.FireTrigger(ctx, t.AutomationID, t.ID, TriggerTypeScheduled, data, dedupKey); err != nil {
cs.logger.Error("failed to fire scheduled trigger",
zap.String("trigger_id", t.ID),
zap.String("automation_id", t.AutomationID),
zap.Error(err))
}
}
// cronParser accepts the standard 5-field cron syntax plus the named
// descriptors (@hourly, @daily, @weekly, @monthly, @yearly) and @every
// interval shorthand that the automation UI exposes as presets.
var cronParser = cron.NewParser(
cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow | cron.Descriptor,
)
// nextCronFire computes the next time the given cron expression fires strictly
// after `after`, interpreted in the given timezone. An empty timezone means
// UTC, so scheduling is deterministic regardless of the host's local clock.
//
// Wall-clock schedules honor the timezone's DST transitions (a "0 9 * * *"
// daily schedule always fires at 09:00 local, whether that is 13:00 or 14:00
// UTC). The @every interval shorthand is timezone-independent by nature.
func nextCronFire(expr, timezone string, after time.Time) (time.Time, error) {
expr = strings.TrimSpace(expr)
if expr == "" {
return time.Time{}, fmt.Errorf("empty cron expression")
}
// Bind the schedule to its timezone. @every is a pure interval and the
// parser rejects a CRON_TZ prefix on it, so only wall-clock schedules get
// the prefix. An expression that already carries its own TZ/CRON_TZ prefix
// is left untouched.
if !strings.HasPrefix(expr, "@every") && !hasCronTZPrefix(expr) {
tz := timezone
if tz == "" {
tz = "UTC"
}
if _, err := time.LoadLocation(tz); err != nil {
return time.Time{}, fmt.Errorf("invalid timezone %q: %w", tz, err)
}
expr = "CRON_TZ=" + tz + " " + expr
}
schedule, err := cronParser.Parse(expr)
if err != nil {
return time.Time{}, fmt.Errorf("parse cron expression: %w", err)
}
return schedule.Next(after), nil
}
func hasCronTZPrefix(expr string) bool {
return strings.HasPrefix(expr, "TZ=") || strings.HasPrefix(expr, "CRON_TZ=")
}