Files
kandev/apps/backend/internal/gitlab/poller.go
T
João Salavisa dfaab11959 feat: full GitLab integration — parity with GitHub (#1120)
* feat(backend): full GitLab parity — store, service, poller, WS, presets

Add the watcher/poll/write-action surface on top of the existing GitLab
client + service skeleton:

* 5 new SQLite tables (mr_watches, review_watches, review_mr_tasks,
  issue_watches, issue_watch_tasks, action_presets) + CRUD
* Client + Service: MergeMR, GetProjectMergeMethods, GetProtectedBranch,
  ListUserProjects, SearchProjects, SetMRLabels, SetMRAssignees
* Service: review/issue/MR watch CRUD + Check/Trigger, GetStats,
  CleanupAllReview/IssueTasks (cleanup-policy aware), TriggerMRSync,
  action-presets CRUD with defaults
* Background poller (MR / review / issue loops) wired from main.go
* ~35 GitLab WS action constants + handlers.go registrations
* Mock controller for E2E seeding (MRs, issues, pipelines, discussions,
  approvals, branches)
* eventBus + taskDeleter + taskSessionChecker plumbed through Service
* Event types: GitLabMRFeedback / NewReviewMR / NewIssue / TaskMRUpdated

* feat(web): GitLab frontend parity — types, API, store slice, domain hooks

Extend the frontend GitLab integration on top of the new backend surface:

* HTTP API client expansion (gitlab-api.ts): full CRUD for review/issue/MR
  watches, MR write actions (merge/approve/labels/assignees), MR
  files/commits/feedback, action presets, projects autocomplete, stats
* Store slice expansion: review/issue/MR watches lists, action presets
  by workspace, stats + status with loading flags. Properties prefixed
  with gitlab to avoid collision with GitHub slice
* Domain hooks: use-gitlab-status, use-gitlab-stats,
  use-gitlab-review-watches, use-gitlab-issue-watches,
  use-gitlab-action-presets — mirror github hook shapes
* Type model: ReviewWatch, IssueWatch, MRWatch, ActionPresets, Stats,
  ProjectMergeMethods, MRApproval, Pipeline, MRFeedback, etc.
* Backend controller HTTP routes for watches/presets/write-actions/
  projects/stats so the frontend HTTP API client has endpoints to hit

All passes: backend (build + vet + lint + tests), web (typecheck + lint +
2400 tests).

* fix(gitlab): wire orchestrator event handlers, fix label-filter drop, add tests

Address Claude review blockers on the parity PR:

1. Wire orchestrator subscriptions for GitLab review/issue watch events.
   Add event_handlers_gitlab.go with handleGitLabNewReviewMR /
   handleGitLabNewIssue, dedup reservation handshake, and task-creator
   interfaces (mirrors event_handlers_github.go). main.go now calls
   orchestratorSvc.SetGitLabService so the dedup APIs are usable
2. Fix silent label-filter drop: when a custom_query is set on an issue
   watch, labels were being appended to the unused filter arg. Fold
   labels into customQuery (or default filter) so they actually reach
   GitLab; extracted appendLabelsToQuery helper with explicit handling
   of pre-existing labels= clauses
3. Switch event payloads to pointer types (mirrors GitHub) so
   handlers can type-assert pointer events
4. Backend tests: store_watches_test.go (CRUD + reserve/assign for all
   4 new tables + presets), service_watches_test.go (label-filter
   helper, cleanup policy, project normalisation), action_presets_test.go
5. Frontend slice tests: review/issue watch CRUD round-trips, action
   presets + stats reducer coverage

All green: backend (build+vet+lint+tests), web (typecheck+lint+2403 tests).

* fix(gitlab): address remaining blockers — cleanup wiring, safe defaults, nil guards

Address the second round of Claude review blockers:

1. main.go: wire SetTaskDeleter + SetTaskSessionChecker on services.GitLab
   (mirrors GitHub). Without these, manual cleanup sweeps would always
   error with "task deleter not configured"
2. service_cleanup.go: default cleanup policy on transient DB error is now
   CleanupPolicyNever (preserve tasks) instead of CleanupPolicyAuto
   (silently delete). Genuine "watch was deleted" path also falls under
   the safe-side default; user can manually delete the orphan tasks
3. service_watches.go: add nil guards on requireStore() so the 6 MR-watch
   list/get/delete methods return an errStoreUnavailable error instead
   of panicking when the SQLite store failed to construct at boot
4. use-gitlab-status.ts + use-gitlab-stats.ts: add per-mount attemptedRef
   so an unreachable GitLab doesn't trigger an infinite re-fetch loop —
   the previous useEffect re-ran every render because the failure path
   left status null, satisfying the !status guard

* fix(gitlab): propagate session-check errors, clamp poll interval on update

Address remaining Claude blockers:

* service_cleanup.go: HasUserAuthoredMessage transient errors are now
  preserved (return false → skip delete) instead of silently ignored;
  the alternative was occasionally deleting tasks a user had touched
* service_watches.go: extract clampPollInterval helper applying the same
  bounds (0 → default, <60 → 60) as the create path; both
  applyReviewWatchPatch and applyIssueWatchPatch use it so user-supplied
  zero or tiny values via UpdateXxxWatchRequest no longer hammer GitLab

* fix(gitlab): clear dupl lint, fix preset retry loop + per-workspace watch cache

* service_watches.go: applyReviewWatchPatch / applyIssueWatchPatch get
  nolint:dupl markers — they share shape but per-domain field validation
  lives in the create paths, so deduplicating via generics would obscure
  the contract. Restores backend lint to 0 issues
* use-gitlab-action-presets.ts: per-workspace attemptedRef set so a
  failing preset fetch doesn't retry on every render
* use-gitlab-review-watches.ts + use-gitlab-issue-watches.ts: track
  lastFetchedRef per consumer so a workspace switch triggers a refetch.
  The slice-level loaded flag is shared across instances and can't
  double as a per-workspace cache key

* fix(gitlab): add nil-store guards across review/issue watch + cleanup paths

Round-3 Claude finding: ReviewWatch / IssueWatch / preset / reservation
methods still called s.requireStore().X() directly. If NewStore fails at
boot (table migration error), the service struct's store is nil and every
list/get/update/create/reserve method panics on first request.

Apply the same store-nil → errStoreUnavailable pattern from f95a929 to
every remaining method:
* CreateReviewWatch / CreateIssueWatch
* GetReviewWatch / ListReviewWatches / ListAllReviewWatches /
  UpdateReviewWatch / TriggerReviewWatchAll
* GetIssueWatch / ListIssueWatches / ListAllIssueWatches /
  UpdateIssueWatch / TriggerIssueWatchAll
* ReserveReviewMRTask / AssignReviewMRTaskID / ReleaseReviewMRTask
* ReserveIssueWatchTask / AssignIssueWatchTaskID / ReleaseIssueWatchTask
* lookupReviewPolicy / lookupIssuePolicy (return CleanupPolicyNever)

* fix(gitlab): nil-store guards on CheckXxxWatch + DeleteXxxWatch + cleanup paths

Round-4 Claude finding: 4 remaining panic sites the previous nil-guard
pass missed — these run from the background poller (every 5 min) and the
manual delete/cleanup flows, so a boot-time NewStore failure would
crash the orchestrator process on first tick.

* service_watches.go: add store-nil guards to DeleteReviewWatch /
  DeleteIssueWatch / CheckReviewWatch / CheckIssueWatch
* service_cleanup.go: guard CleanupAllReviewTasks / CleanupAllIssueTasks
  entry points; wrap the trailing DeleteReviewMRTask / DeleteIssueWatchTask
  calls in `if store := s.requireStore(); store != nil` blocks so the
  cleanup succeeds even when the dedup-row delete is unreachable

* refactor(gitlab): split service_watches.go to stay under 800-line revive limit

CI's --new-from-rev lint caught service_watches.go at 808 lines (limit
800). Split into three focused files:

* service_events.go (94 LOC) — publish helpers for MR feedback /
  new review MR / new issue / watch lifecycle events
* service_issue_watches.go (302 LOC) — Issue watch CRUD + Check +
  Trigger + fetch + helpers
* service_reservations.go (59 LOC) — Reserve/Assign/Release dedup
  handles used by the orchestrator event handlers

service_watches.go is now 505 lines and contains MR watch + Review
watch only. No behavioral change.

* fix(gitlab): address inline code-review feedback

Address actionable inline comments from Claude/CodeRabbit/cubic/Greptile:

* appendLabelsToQuery: use url.ParseQuery for exact key match instead
  of strings.Contains; previously false-matched keys like mylabels=
  and silently dropped the watch's labels (test added)
* fetchReviewMRs: drop dead `filter = watch.CustomQuery` assignment —
  SearchMRs's buildMRSearchQuery returns customQuery verbatim and
  ignores filter when customQuery is non-empty
* store.go: add workspace_id indexes on gitlab_review_watches and
  gitlab_issue_watches (the 5-min poller and HTTP list endpoints did
  full table scans)
* mock_client.SearchProjects: switch from == to case-insensitive
  Contains so partial-query autocomplete returns the seeded project,
  matching the doc comment
* Poller: add sync.Mutex to guard `started` field; Start/Stop are
  now safe to call concurrently (go test -race would have flagged
  the previous read/write race)
* Add ErrWatchNotFound sentinel; controller_watches.go maps it to
  HTTP 404 via httpRespondError helper. Update/Trigger handlers
  previously returned 500 for missing watches, hiding the distinction
  from real server faults
* service_events.go: log Publish errors on watch lifecycle events
  instead of swallowing them silently
* handlers.go: reject malformed JSON payloads in list-style WS
  handlers rather than silently falling through to broader
  list/search behavior
* event_handlers_gitlab.go: remove dead gitlabRepoSlug helper +
  blank-identifier suppression

* ci: re-trigger to clear flaky E2E shard 3

* fix(gitlab): address final review round — nil guards, store merge, logs, body validation

- CheckMRWatch, UpdateReviewWatch, UpdateIssueWatch: nil guards for store/req
- handlers: wsNewDiscussionNote rejects empty body
- action_presets: Update reads raw stored row (no default freezing);
  Reset validates workspace_id
- service_search.GetStats: log Warn per sub-call failure before zero
- event_handlers_gitlab: log Release* dedup-row errors with context
- web/store.ts: post-slice merge restores every GitLab sub-state field;
  extract buildStateOverrides helper
- web/default-state.ts: add GitLab fields to defaultState +
  mergeInitialState via mergeGitLabFields helper
- web/gitlab-api.ts: UpdateXxxWatchRequest now Omit workspace_id

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Kandev Agent <agent@kandev.dev>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 22:10:07 +01:00

204 lines
4.3 KiB
Go

package gitlab
import (
"context"
"sync"
"time"
"go.uber.org/zap"
"github.com/kandev/kandev/internal/common/logger"
"github.com/kandev/kandev/internal/events/bus"
)
const (
defaultMRPollInterval = 1 * time.Minute
defaultReviewPollInterval = 5 * time.Minute
defaultIssuePollInterval = 5 * time.Minute
)
// Poller runs background loops for MR monitoring + review/issue queue checking.
type Poller struct {
service *Service
eventBus bus.EventBus
logger *logger.Logger
mu sync.Mutex
cancel context.CancelFunc
wg sync.WaitGroup
started bool
}
// NewPoller creates a new background poller.
func NewPoller(svc *Service, eventBus bus.EventBus, log *logger.Logger) *Poller {
return &Poller{service: svc, eventBus: eventBus, logger: log}
}
// Start kicks off the polling loops. Repeated Start calls are no-ops.
// Safe to call concurrently with Stop.
func (p *Poller) Start(ctx context.Context) {
p.mu.Lock()
defer p.mu.Unlock()
if p.started {
return
}
p.started = true
ctx, p.cancel = context.WithCancel(ctx)
p.wg.Add(3)
go p.mrMonitorLoop(ctx)
go p.reviewWatchLoop(ctx)
go p.issueWatchLoop(ctx)
}
// Stop cancels all loops and waits for them to drain. Safe to call
// concurrently with Start.
func (p *Poller) Stop() {
p.mu.Lock()
if !p.started || p.cancel == nil {
p.mu.Unlock()
return
}
cancel := p.cancel
p.mu.Unlock()
cancel()
p.wg.Wait()
p.mu.Lock()
p.started = false
p.mu.Unlock()
}
// --- MR watcher loop ---
func (p *Poller) mrMonitorLoop(ctx context.Context) {
defer p.wg.Done()
ticker := time.NewTicker(defaultMRPollInterval)
defer ticker.Stop()
// Initial run.
p.runMRMonitor(ctx)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
p.runMRMonitor(ctx)
}
}
}
func (p *Poller) runMRMonitor(ctx context.Context) {
watches, err := p.service.ListActiveMRWatches(ctx)
if err != nil {
p.logger.Warn("gitlab poller: list MR watches", zap.Error(err))
return
}
for _, w := range watches {
if ctx.Err() != nil {
return
}
if _, _, err := p.service.CheckMRWatch(ctx, w); err != nil {
p.logger.Debug("gitlab poller: check MR watch",
zap.String("watch_id", w.ID), zap.Error(err))
}
}
}
// --- Review watcher loop ---
func (p *Poller) reviewWatchLoop(ctx context.Context) {
defer p.wg.Done()
ticker := time.NewTicker(defaultReviewPollInterval)
defer ticker.Stop()
p.runReviewWatch(ctx)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
p.runReviewWatch(ctx)
}
}
}
func (p *Poller) runReviewWatch(ctx context.Context) {
watches, err := p.service.ListAllReviewWatches(ctx)
if err != nil {
p.logger.Warn("gitlab poller: list review watches", zap.Error(err))
return
}
now := time.Now().UTC()
for _, w := range watches {
if ctx.Err() != nil {
return
}
if !w.Enabled || !shouldPollWatch(w.LastPolledAt, w.PollIntervalSeconds, now) {
continue
}
newMRs, err := p.service.CheckReviewWatch(ctx, w)
if err != nil {
p.logger.Debug("gitlab poller: review watch check",
zap.String("watch_id", w.ID), zap.Error(err))
continue
}
for _, mr := range newMRs {
p.service.publishNewReviewMREvent(ctx, w, mr)
}
}
}
// --- Issue watcher loop ---
func (p *Poller) issueWatchLoop(ctx context.Context) {
defer p.wg.Done()
ticker := time.NewTicker(defaultIssuePollInterval)
defer ticker.Stop()
p.runIssueWatch(ctx)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
p.runIssueWatch(ctx)
}
}
}
func (p *Poller) runIssueWatch(ctx context.Context) {
watches, err := p.service.ListAllIssueWatches(ctx)
if err != nil {
p.logger.Warn("gitlab poller: list issue watches", zap.Error(err))
return
}
now := time.Now().UTC()
for _, w := range watches {
if ctx.Err() != nil {
return
}
if !w.Enabled || !shouldPollWatch(w.LastPolledAt, w.PollIntervalSeconds, now) {
continue
}
newIssues, err := p.service.CheckIssueWatch(ctx, w)
if err != nil {
p.logger.Debug("gitlab poller: issue watch check",
zap.String("watch_id", w.ID), zap.Error(err))
continue
}
for _, issue := range newIssues {
p.service.publishNewIssueEvent(ctx, w, issue)
}
}
}
func shouldPollWatch(last *time.Time, intervalSec int, now time.Time) bool {
if last == nil {
return true
}
if intervalSec <= 0 {
intervalSec = defaultWatchPollIntervalSec
}
return now.Sub(*last) >= time.Duration(intervalSec)*time.Second
}