Files
kandev/apps/backend/internal/automation/github_webhook_subscriber.go
T
João Salavisa 37b56ba11c feat(automations): reliable cron scheduling and webhook-driven push/CI triggers (#1972)
* feat(automations): reliable cron scheduling and webhook-driven push/CI triggers

Automations were "wounded": cron schedules mishandled timezones and some
never fired, and the GitHub push/CI trigger types were coming-soon stubs.

Cron (fix):
- The scheduler reduced every cron expression to a fixed interval derived
  only from a */N step in the minute/hour field, so any pinned schedule
  (0 9 * * *, 30 14 * * *, weekday ranges, day-of-month) never fired, and
  the configured timezone was parsed but never used.
- Replace the interval approximation with robfig/cron/v3: shouldFire now
  computes the true next fire time in the configured timezone (DST-aware,
  defaulting to UTC for host-independence), anchored to last-evaluated or
  creation time. Dead interval-parsing helpers and shorthand constants
  removed.

GitHub push + CI triggers (feat):
- Implement push and CI-failure triggers end to end via the existing
  per-workspace GitHub App webhook plumbing (#1810): the HMAC-verified
  webhook service now parses push and completed check_run events, resolves
  the installation to its workspaces, and publishes normalized bus events.
- A new automation subscriber matches enabled github_push / github_ci
  triggers by workspace + repo + branch/conclusion/check-name filters and
  fires them (deduped by commit SHA / check-run id). The old poll-based
  stub evaluators are removed; the PR poll path is unchanged.
- Subscribe the App manifest to push/check_run and enable both trigger
  types in the registry (CI defaults to the failure conclusion).

Docs:
- Update docs/public/feature-status.md and automation-and-mcp.md so the
  cron and push/CI rows reflect shipped reality.

Tests: regression tests pin the previously-never-firing cron shapes,
timezone/DST behavior, and host-independence; backend tests cover the
webhook parse->publish and subscriber match/fire/dedup flows for both
push and CI-failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): populate push.message and ci.url placeholders in fired trigger data

Address PR review blockers: the default push and CI automation prompts
reference {{push.message}} and {{ci.url}}, which the interpolator maps to
data["message"] / data["html_url"] — but the webhook subscriber never
populated those fields, so both always rendered empty.

- Parse head_commit.message (push) and check_run.html_url (CI) in the
  GitHub webhook service and carry them on the published bus payloads.
- Include "message" / "html_url" in the fired trigger data so the
  placeholders resolve; extend the webhook and subscriber tests to assert
  both flow through end to end.

Non-blocking review cleanups:
- Replace the misleading reuse of matchesAuthors for CI conclusions/check
  names with a clearly-named matchesFilterValue helper.
- Drop dead context.Background() line and its unused import from the cron
  regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(automations): scope webhook triggers to active installs and per-branch dedup

Address CodeRabbit review on the push/CI webhook triggers:

- Security: resolveWorkspaceIDs now only returns connections whose status is
  active, so a suspended or revoked GitHub App installation can no longer fire
  push/CI automations.
- Correctness: include the branch in the push dedup key
  (push:owner/name@branch@sha) so the same commit pushed to two matching
  branches fires once per branch instead of being suppressed.
- Correctness: add an optional head-branch filter to github_ci triggers and
  gate CI matching on it, matching the push trigger's branch-scoping.
- Stability: make the webhook subscriber's Start retryable — a subscription
  failure rolls back partial subscriptions and leaves the subscriber
  un-started so a later Start re-attempts, instead of permanently disabling
  push/CI processing.
- Tests: cover multi-connection workspace resolution (dedup + inactive
  filtering), per-branch push dedup, and CI branch filtering.
- Docs: note the CI branch filter and per-branch push dedup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(automations): cover webhook subscriber Start retry-after-failure path

Add TestGitHubWebhookSubscriberStartRetryableAfterFailure with a minimal
flaky bus whose second Subscribe fails, pinning the retry contract: a
subscription failure rolls back the partial subscription and leaves the
subscriber un-started, and a later Start re-attempts and succeeds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 10:24:57 +01:00

207 lines
7.0 KiB
Go

package automation
import (
"context"
"encoding/json"
"fmt"
"go.uber.org/zap"
"github.com/kandev/kandev/internal/common/logger"
"github.com/kandev/kandev/internal/events"
"github.com/kandev/kandev/internal/events/bus"
"github.com/kandev/kandev/internal/github"
)
// GitHubWebhookSubscriber turns webhook-driven GitHub bus events (push,
// completed check runs) into fired github_push / github_ci automation
// triggers. It owns its subscriptions: Start registers handlers on the bus,
// Stop unsubscribes. The bus delivers to handlers synchronously on the
// publisher's goroutine, so this type spawns no goroutines of its own.
type GitHubWebhookSubscriber struct {
svc *Service
eventBus bus.EventBus
logger *logger.Logger
subs []bus.Subscription
started bool
}
// NewGitHubWebhookSubscriber creates a new subscriber.
func NewGitHubWebhookSubscriber(svc *Service, eventBus bus.EventBus, log *logger.Logger) *GitHubWebhookSubscriber {
return &GitHubWebhookSubscriber{svc: svc, eventBus: eventBus, logger: log}
}
// Start subscribes to the GitHub webhook bus events. Idempotent, and
// retryable: a subscription failure leaves the subscriber un-started (partial
// subscriptions are rolled back) so a later Start re-attempts both, rather than
// permanently disabling push/CI processing.
func (s *GitHubWebhookSubscriber) Start(_ context.Context) {
if s.started || s.eventBus == nil {
return
}
subs, err := s.subscribeAll()
if err != nil {
for _, sub := range subs {
_ = sub.Unsubscribe()
}
s.logger.Error("failed to subscribe automation GitHub webhook events; will retry on next Start",
zap.Error(err))
return
}
s.subs = subs
s.started = true
s.logger.Info("automation GitHub webhook subscriber started")
}
// subscribeAll subscribes to both webhook events, returning any subscriptions
// created so far alongside the first error (for rollback by the caller).
func (s *GitHubWebhookSubscriber) subscribeAll() ([]bus.Subscription, error) {
var subs []bus.Subscription
pushSub, err := s.eventBus.Subscribe(events.GitHubPushReceived, s.handlePush)
if err != nil {
return subs, fmt.Errorf("subscribe %s: %w", events.GitHubPushReceived, err)
}
subs = append(subs, pushSub)
ciSub, err := s.eventBus.Subscribe(events.GitHubCheckRunCompleted, s.handleCheckRun)
if err != nil {
return subs, fmt.Errorf("subscribe %s: %w", events.GitHubCheckRunCompleted, err)
}
return append(subs, ciSub), nil
}
// Stop unsubscribes from the bus. Idempotent.
func (s *GitHubWebhookSubscriber) Stop() {
if !s.started {
return
}
for _, sub := range s.subs {
if sub != nil {
_ = sub.Unsubscribe()
}
}
s.subs = nil
s.started = false
s.logger.Info("automation GitHub webhook subscriber stopped")
}
func (s *GitHubWebhookSubscriber) handlePush(ctx context.Context, event *bus.Event) error {
payload, ok := event.Data.(*github.GitHubPushEventPayload)
if !ok || payload == nil {
return nil
}
triggers, err := s.svc.Store().ListEnabledTriggersByType(ctx, TriggerTypeGitHubPush)
if err != nil {
s.logger.Error("failed to list github_push triggers", zap.Error(err))
return nil
}
for i := range triggers {
s.checkPushTrigger(ctx, &triggers[i], payload)
}
return nil
}
func (s *GitHubWebhookSubscriber) checkPushTrigger(
ctx context.Context, t *AutomationTrigger, payload *github.GitHubPushEventPayload,
) {
var cfg GitHubPushTriggerConfig
if err := json.Unmarshal(t.Config, &cfg); err != nil {
s.logger.Debug("invalid github_push trigger config", zap.String("trigger_id", t.ID), zap.Error(err))
return
}
if !s.triggerMatchesWorkspaceAndRepo(ctx, t, payload.WorkspaceIDs, payload.Owner, payload.Name, cfg.Repos) {
return
}
if !matchesBranches(payload.Branch, cfg.Branches) {
return
}
// The same commit SHA can be pushed to multiple matching branches; key the
// dedup on the branch too so a later branch's push isn't suppressed.
dedupKey := fmt.Sprintf("push:%s/%s@%s@%s", payload.Owner, payload.Name, payload.Branch, payload.SHA)
data, _ := json.Marshal(map[string]interface{}{
"repo": fmt.Sprintf("%s/%s", payload.Owner, payload.Name),
"branch": payload.Branch,
"sha": payload.SHA,
"pusher_login": payload.PusherLogin,
"message": payload.HeadCommitMsg,
})
if err := s.svc.FireTrigger(ctx, t.AutomationID, t.ID, TriggerTypeGitHubPush, data, dedupKey); err != nil {
s.logger.Error("failed to fire push trigger", zap.String("trigger_id", t.ID), zap.Error(err))
}
}
func (s *GitHubWebhookSubscriber) handleCheckRun(ctx context.Context, event *bus.Event) error {
payload, ok := event.Data.(*github.GitHubCheckRunEventPayload)
if !ok || payload == nil {
return nil
}
triggers, err := s.svc.Store().ListEnabledTriggersByType(ctx, TriggerTypeGitHubCI)
if err != nil {
s.logger.Error("failed to list github_ci triggers", zap.Error(err))
return nil
}
for i := range triggers {
s.checkCITrigger(ctx, &triggers[i], payload)
}
return nil
}
func (s *GitHubWebhookSubscriber) checkCITrigger(
ctx context.Context, t *AutomationTrigger, payload *github.GitHubCheckRunEventPayload,
) {
var cfg GitHubCITriggerConfig
if err := json.Unmarshal(t.Config, &cfg); err != nil {
s.logger.Debug("invalid github_ci trigger config", zap.String("trigger_id", t.ID), zap.Error(err))
return
}
if !s.triggerMatchesWorkspaceAndRepo(ctx, t, payload.WorkspaceIDs, payload.Owner, payload.Name, cfg.Repos) {
return
}
if !matchesFilterValue(payload.Conclusion, cfg.Conclusions) ||
!matchesFilterValue(payload.CheckName, cfg.CheckNames) ||
!matchesBranches(payload.Branch, cfg.Branches) {
return
}
dedupKey := fmt.Sprintf("ci:%s/%s#%d", payload.Owner, payload.Name, payload.CheckRunID)
data, _ := json.Marshal(map[string]interface{}{
"repo": fmt.Sprintf("%s/%s", payload.Owner, payload.Name),
"branch": payload.Branch,
"sha": payload.SHA,
"check_name": payload.CheckName,
"conclusion": payload.Conclusion,
"check_run_id": payload.CheckRunID,
"html_url": payload.HTMLURL,
})
if err := s.svc.FireTrigger(ctx, t.AutomationID, t.ID, TriggerTypeGitHubCI, data, dedupKey); err != nil {
s.logger.Error("failed to fire CI trigger", zap.String("trigger_id", t.ID), zap.Error(err))
}
}
// triggerMatchesWorkspaceAndRepo loads the trigger's owning automation and
// checks its workspace is among the event's resolved workspaces, and that
// the event's repo matches the trigger's repo filter (empty repo filter
// never matches — these webhook triggers cannot act without a repo).
func (s *GitHubWebhookSubscriber) triggerMatchesWorkspaceAndRepo(
ctx context.Context, t *AutomationTrigger, workspaceIDs []string, owner, name string, repos []github.RepoFilter,
) bool {
if !matchesRepo(owner, name, repos) {
return false
}
a, err := s.svc.GetAutomation(ctx, t.AutomationID)
if err != nil || a == nil {
s.logger.Debug("automation trigger is missing workspace ownership",
zap.String("trigger_id", t.ID), zap.Error(err))
return false
}
for _, id := range workspaceIDs {
if id == a.WorkspaceID {
return true
}
}
return false
}