fix: reduce GitHub cleanup and watch polling load (#3866)

This commit is contained in:
Carlos Florêncio
2026-09-23 13:31:12 +01:00
committed by GitHub
parent 16ca1657ce
commit 9944354518
83 changed files with 6567 additions and 348 deletions
+138
View File
@@ -0,0 +1,138 @@
---
created: 2026-09-22
status: complete
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-001
- REQ-INTEGRATIONS-WATCH-CLEANUP-002
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-001
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-002
- REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002
system_design:
- ../../specs/integrations/system-design/watch-task-cleanup.md
- ../../specs/integrations/system-design/github-pr-discovery-health.md
- ../../specs/integrations/system-design/github-workflow-attention.md
legacy_specs: []
---
# Implementation plan: Watch task cleanup
## Overview
Prevent historical tasks from consuming provider quota. Implement GitHub
eligibility first, GitHub batch termination second, and sibling filters third.
Then reduce cleanup reads, bound fallback, add adaptive discovery, and cache
Actions observations. All seven sequential work orders are complete.
## Source documents
- [Requirements](../../specs/integrations/requirements/watch-task-cleanup.md)
- [System design](../../specs/integrations/system-design/watch-task-cleanup.md)
- [Discovery requirements](../../specs/integrations/requirements/github-pr-discovery-health.md)
- [Discovery design](../../specs/integrations/system-design/github-pr-discovery-health.md)
- [Actions requirements](../../specs/integrations/requirements/github-workflow-attention.md)
- [Actions design](../../specs/integrations/system-design/github-workflow-attention.md)
## Confirmed root cause and reproduction
GitHub cleanup list queries read historical deduplication rows without task
eligibility checks. Service gates fetch remote state before engagement checks.
Fetch errors become false eligibility results, so the batch continues.
A temporary in-memory SQLite reproduction executed the actual SQL extracted
from both GitHub review list methods. Both returned task IDs `active`,
`archived`, `deleted`, and an empty reservation. Only the first and last should
be cleanup candidates. No production database or provider API was accessed.
GitLab review and issue queries also lack task eligibility filters. Azure
DevOps review and work-item queries filter watch generation only. Both
providers fetch upstream state before checking task engagement.
The current GitHub feedback helper requests PR data, reviews, comments, checks,
and optional workflow attention. The exact request count differs from the
reported older implementation, but historical-row amplification remains.
## Scope and compatibility
Preserve cleanup policies, lifecycle-prompt protection, empty GitHub
reservations, explicit watch deletion, and reset behavior. Ignore historical
records without changing deduplication. No UI or schema changes are required.
Unarchived Done tasks remain eligible. Archive races after query selection
remain outside the atomicity guarantee.
## Work orders
- [x] [Task 01: Filter GitHub cleanup candidates](task-01-github-candidates.md)
- [x] [Task 02: Stop GitHub cleanup on rate limits](task-02-github-rate-limits.md)
- [x] [Task 03: Filter sibling provider cleanup candidates](task-03-sibling-candidates.md)
- [x] [Task 04: Use minimal PR reads for cleanup](task-04-minimal-cleanup.md)
- [x] [Task 05: Bound per-workspace PR fallback](task-05-bounded-fallback.md)
- [x] [Task 06: Back off idle searching watches](task-06-adaptive-discovery.md)
- [x] [Task 07: Cache Actions observations with bounded expiry](task-07-actions-cache.md)
## Polling and cache values
| Function | Shipped behavior | Bound/details |
| --- | --- | --- |
| PR poller scheduler tick | 1 minute | Fixed scheduler tick |
| Searching, running or idle below 2 hours | 1 minute | Fast discovery |
| Searching, idle 2 to below 24 hours | 15 minutes | Activity restores next-tick eligibility |
| Searching, idle at least 24 hours | 30 minutes | Never permanently stops |
| Known PR background checks | 1 minute | Fixed cadence |
| Review-watch discovery and cleanup | 5 minutes | Eligible tasks only |
| Issue-watch discovery and cleanup | 5 minutes | Eligible tasks only |
| Status/search response TTL | 30 seconds | Outer response cache |
| Explicit feedback response TTL | 8 seconds | Explicit refresh invalidates |
| PR sync freshness window | 30 seconds | Passive status freshness |
| Actions changing, empty, or attention-required | 30 seconds | Run and job observations |
| Actions all complete, no attention required | 5 minutes | Same-SHA reruns appear after expiry |
| Actions runs/jobs capacity | 512 entries each | In-memory bounded caches |
| Actions batch concurrency / total budget | 4 / 5 seconds | Preserved |
| Fallback target budget | 5 per workspace, 10 globally | Rotating per cycle |
Activity restores next-tick eligibility. Explicit refresh bypasses idle and
Actions cache delays but never quota/auth admission. Poll ticks can add up to
1 minute to the interval. Missing activity information selects fast polling.
A completed-SHA external rerun may take up to 6 minutes to appear automatically.
Retain discovery rate-limit retry delays of 1, 2, 4, 8, then 15 minutes when
no provider retry time is available. Existing provider retry deadlines are
capped at 15 minutes by discovery health. Auth/config circuit backoff starts
at 2 minutes, caps at 6 hours, and uses 25 percent jitter. The poller rate-limit
sleep cap remains 10 minutes per wait; it does not clear the exhaustion state.
## Tests
Each work order lists new regression names, existing test locations, and exact
commands. Use store-backed service tests to prove candidate selection reaches
provider-call behavior. Browser tests add no evidence for this backend-only
change. Run each regression before production edits and record its expected
failure. Then run the affected package suites.
## Verification results
- `python3 scripts/list-docs.py validate`: passed, 299 decisions and 1110 specifications.
- `python3 scripts/lint-spec-files.py --all`: passed.
- `python3 scripts/lint-spec-files.test.py`: passed, 36 tests.
- `git diff --check`: passed.
Implementation is complete. Production, permanent test, specification, plan,
and public documentation changes are present in the working tree and remain
uncommitted.
## Risks
- Existing tests often attach fictitious task IDs without task rows. Update
active fixtures rather than weakening eligibility.
- GitHub by-watch queries also support explicit watch deletion. Preserve its
unfiltered inventory through existing task-ID methods.
- A batch can stop only after an error becomes visible. Concurrent requests
within its current feedback fetch may already be running.
- Ignored historical deduplication rows remain stored. This fix bounds polling,
not storage retention or quota consumed by other active work.
## Documentation impact
Internal requirements, designs, and work orders describe the expanded scope.
No controls change. The public integration guide documents delayed idle
discovery and bounded Actions freshness as shipped behavior.
@@ -0,0 +1,103 @@
---
id: "01-github-candidates"
title: "Filter GitHub cleanup candidates"
status: done
wave: 1
depends_on: []
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-001
acceptance_criteria:
- AC-INTEGRATIONS-WATCH-CLEANUP-001.1
- AC-INTEGRATIONS-WATCH-CLEANUP-001.2
- AC-INTEGRATIONS-WATCH-CLEANUP-001.3
- AC-INTEGRATIONS-WATCH-CLEANUP-001.4
- AC-INTEGRATIONS-WATCH-CLEANUP-001.5
system_design:
- ../../specs/integrations/system-design/watch-task-cleanup.md
---
# Task 01: Filter GitHub cleanup candidates
## Summary
Filter GitHub cleanup candidates. Implement with TDD and record the regression results.
## In scope
Add mixed-state regressions in `store_task_cleanup_test.go` and
`service_review_cleanup_test.go`. Name the new tests
`TestCleanupCandidateListsExcludeHistoricalTasks` and
`TestCleanupMergedReviewTasksSkipsHistoricalTasks`.
Cover by-watch, global, orphan, and workspace cleanup for reviews and issues.
Assert zero upstream calls for archived and missing tasks. Cover terminal and
nonterminal empty reservations, all policies, engagement, and lifecycle prompts.
Apply the candidate predicate to all four GitHub list methods. Preserve
explicit watch deletion with unfiltered task-ID queries in `service_reviews.go`
and `service_issues.go`. Add archived-child deletion assertions to existing
watch deletion tests. Assert deduplication remains present through unfiltered
lookups, not filtered cleanup queries.
Update obsolete missing-task cleanup expectations to zero provider calls and
zero deletions. Keep task-not-found race coverage by deleting an initially
eligible task after selection.
## Out of scope
New policies, UI, schema migrations, and unrelated provider changes.
## Acceptance
- Mixed candidate lists and store-backed cleanup satisfy AC 001.1 through 001.4.
- Explicit watch deletion, reset, scope, and deduplication satisfy AC 001.5.
- Regression tests fail before the query changes and pass afterward.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/store.go`
- `apps/backend/internal/github/service_reviews.go`
- `apps/backend/internal/github/service_issues.go`
- `apps/backend/internal/github/store_task_cleanup_test.go`
- `apps/backend/internal/github/service_review_cleanup_test.go`
- `apps/backend/internal/github/service_cleanup_policy_test.go`
- `apps/backend/internal/github/store_watch_reset_test.go`
- `apps/backend/internal/github/store_watch_disable_test.go`
## Dependencies
None.
## Risks
Preserve task inventory outside cleanup. Seed actual active tasks in fixtures.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/watch-task-cleanup.md)
- [Design](../../specs/integrations/system-design/watch-task-cleanup.md)
- Existing source and tests listed above.
## Results
Implemented the task eligibility predicate for all four GitHub cleanup list
queries. Explicit watch deletion keeps its unfiltered task-ID inventory, so
archived and missing task rows are preserved for deduplication while active
tasks and empty reservations remain cleanup candidates. Added mixed-state
store and service regressions, updated active-task fixtures, and retained a
post-selection task-not-found race regression.
Verification: `go test ./internal/github -count=1` passed.
@@ -0,0 +1,111 @@
---
id: "02-github-rate-limits"
title: "Stop GitHub cleanup on rate limits"
status: done
wave: 2
depends_on: ["01-github-candidates"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-002
acceptance_criteria:
- AC-INTEGRATIONS-WATCH-CLEANUP-002.1
- AC-INTEGRATIONS-WATCH-CLEANUP-002.2
- AC-INTEGRATIONS-WATCH-CLEANUP-002.3
- AC-INTEGRATIONS-WATCH-CLEANUP-002.4
system_design:
- ../../specs/integrations/system-design/watch-task-cleanup.md
---
# Task 02: Stop GitHub cleanup on rate limits
## Summary
Stop GitHub cleanup on rate limits. Implement with TDD and record the regression results.
## In scope
Add `TestCleanupBatchStopsOnRateLimit` in
`service_cleanup_policy_test.go`, with review and issue cases. Include an empty
reservation first, partial deletion counts, wrapped 403/429 errors, unrelated
403 errors, authenticated-user errors, context cancellation, and later retries.
Add `TestCleanupPollCycleSkipsRemainingCleanupAfterRateLimit` in
`poller_test.go`. Cover per-watch and final orphan calls for both poll loops.
Propagate errors through the private deletion gates, batch helpers, and public
cleanup methods in `service_cleanup.go`. Use the existing typed classifier,
including normalization of GitHub CLI stderr rate-limit failures at the
`GHClient` boundary. Thread the resolved automation tracker into batch
admission and stop against the resource used by the resolved client (including
GraphQL for CLI-backed PR and issue reads). Include a stub that exhausts the
tracker while returning successful feedback. Preserve scope: do not consult
another workspace's tracker.
Preserve ordinary row-error continuation and completed counts. In `poller.go`,
stop further cleanup calls for the current cycle after a rate-limit error.
Keep scheduled recovery and discovery behavior intact.
## Out of scope
New policies, UI, schema migrations, and unrelated provider changes.
## Acceptance
- Review and issue batches stop on errors and tracker exhaustion.
- Public cleanup methods preserve partial counts and expose the stop error.
- Poller regressions prove remaining cleanup and orphan calls are suppressed.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/service_cleanup.go`
- `apps/backend/internal/github/poller.go`
- `apps/backend/internal/github/service_cleanup_policy_test.go`
- `apps/backend/internal/github/service_review_cleanup_test.go`
- `apps/backend/internal/github/poller_test.go`
- `apps/backend/internal/github/poller_cleanup_log_test.go`
## Dependencies
01-github-candidates.
## Risks
Preserve task inventory outside cleanup. Seed actual active tasks in fixtures.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/watch-task-cleanup.md)
- [Design](../../specs/integrations/system-design/watch-task-cleanup.md)
- Existing source and tests listed above.
## Results
Implemented rate-limit and cancellation stop propagation through review and
issue cleanup batches. Batches preserve completed deletion counts, continue
after ordinary row errors, honor the resolved workspace tracker (Core for REST
clients and GraphQL for GHClient), and return recognized 403/429 errors.
GHClient now normalizes CLI rate-limit
stderr into the typed error path, and cleanup admission checks the resolved
client's resource bucket so GraphQL-backed CLI cleanup does not continue after
its quota is exhausted. Poller cycles stop before later watches and the final
orphan sweep after a stop-worthy cleanup error. Added service and poller
regressions for wrapped errors, tracker exhaustion, CLI-shaped rate limits,
cancellation, partial counts, ordinary 403 continuation, and later retry
recovery.
Cleanup admission also reopens automatically once the tracked resource reset
deadline has passed, even when no intervening provider call refreshed the
tracker.
Verification: `go test ./internal/github -count=1` passed.
@@ -0,0 +1,102 @@
---
id: "03-sibling-candidates"
title: "Filter sibling provider cleanup candidates"
status: done
wave: 3
depends_on: ["02-github-rate-limits"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-001
acceptance_criteria:
- AC-INTEGRATIONS-WATCH-CLEANUP-001.1
- AC-INTEGRATIONS-WATCH-CLEANUP-001.2
- AC-INTEGRATIONS-WATCH-CLEANUP-001.4
- AC-INTEGRATIONS-WATCH-CLEANUP-001.5
system_design:
- ../../specs/integrations/system-design/watch-task-cleanup.md
---
# Task 03: Filter sibling provider cleanup candidates
## Summary
Filter sibling provider cleanup candidates. Implement with TDD and record the regression results.
## In scope
Add `TestCleanupCandidateListsExcludeHistoricalTasks` and
`TestCleanupSkipsHistoricalTasks` to the GitLab and Azure DevOps packages.
Create `gitlab/service_cleanup_test.go` beside the existing
`service_cleanup_reason_test.go` and `service_cleanup_mr_automation_test.go`.
Use their existing store and cleanup test fixtures. Cover active, archived,
missing, and empty-task rows together. Assert no remote status calls for
historical tasks and preserve active-task policies.
Filter GitLab review and issue queries in `store_watches.go`, including global,
workspace, and by-watch methods. Filter Azure DevOps work-item and PR list
queries in `watch_store.go`. Preserve workspace and generation constraints.
GitLab and Azure DevOps continue to skip empty reservations in cleanup.
Do not change sibling rate-limit handling or task deletion adapters.
## Out of scope
New policies, UI, schema migrations, and unrelated provider changes.
## Acceptance
- Both provider packages exclude historical tasks from upstream cleanup calls.
- Active policy behavior and workspace/generation boundaries remain intact.
- Empty reservations and reset inventory preserve existing semantics.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/gitlab ./internal/azuredevops -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/gitlab/store_watches.go`
- `apps/backend/internal/gitlab/store_task_cleanup_test.go`
- `apps/backend/internal/gitlab/service_cleanup_test.go`
- `apps/backend/internal/azuredevops/watch_store.go`
- `apps/backend/internal/azuredevops/watch_store_test.go`
- `apps/backend/internal/azuredevops/watch_cleanup_test.go`
## Dependencies
02-github-rate-limits.
## Risks
Preserve task inventory outside cleanup. Seed actual active tasks in fixtures.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/watch-task-cleanup.md)
- [Design](../../specs/integrations/system-design/watch-task-cleanup.md)
- Existing source and tests listed above.
## Results
Implemented the shared task eligibility predicate for GitLab review and issue
candidate queries, including global, workspace, and by-watch reads. Azure
DevOps work-item and pull-request reads now apply the same archived-or-missing
task filter while preserving watch generation scoping. Empty reservations stay
visible to cleanup, which continues to skip them before provider status calls.
Added mixed-state store regressions and instrumented cleanup tests for both
providers. Active task policies still delete terminal records, while archived
and missing task IDs generate zero provider calls. Updated SQLite fixtures to
provide the central task table and seed active task rows where cleanup reads
task eligibility.
Verification: `go test ./internal/gitlab ./internal/azuredevops -count=1`
passed.
@@ -0,0 +1,92 @@
---
id: "04-minimal-cleanup"
title: "Use minimal PR reads for cleanup"
status: done
wave: 4
depends_on: ["03-sibling-candidates"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-001
acceptance_criteria:
- AC-INTEGRATIONS-WATCH-CLEANUP-001.6
system_design:
- ../../specs/integrations/system-design/watch-task-cleanup.md
---
# Task 04: Use minimal PR reads for cleanup
## Summary
Use minimal PR reads for cleanup. Implement with TDD and preserve workspace identity.
## In scope
Replace cleanup-only GetPRFeedback calls with GetPR and conditional review
reads. Fetch viewer identity only when an approval candidate requires it.
Preserve all cleanup policies and approval matching. Propagate each fetch error
to the batch handling from Task 02. Update existing cleanup stubs and tests.
Add TestCleanupUsesMinimalPRReads to service_review_cleanup_test.go. Assert
zero comments, checks, workflow-run, and job reads. Terminal PRs require only
GetPR. Open PRs retain approval-based cleanup. Include rate limits at each
required fetch and preserve partial counts.
## Out of scope
New UI controls, credential policies, global schedulers, and unrelated providers.
## Acceptance
- Meet every linked acceptance criterion with deterministic regressions.
- Preserve existing policy, scope, and error-handling contracts.
- Record the expected red test result and passing package results.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/service_cleanup.go`
- `apps/backend/internal/github/service_review_cleanup_test.go`
- `apps/backend/internal/github/service_cleanup_policy_test.go`
## Dependencies
03-sibling-candidates.
## Risks
Mocks must implement the smaller read path. Preserve existing review approval semantics.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/watch-task-cleanup.md)
- [Design](../../specs/integrations/system-design/watch-task-cleanup.md)
- Source files and existing adjacent tests listed above. New helper/test files are created as needed.
## Results
Implemented the minimal GitHub cleanup read path. Cleanup now fetches PR state
with `GetPR`, fetches reviews only for open PRs, and requests the authenticated
user only when at least one approval review exists. Cleanup no longer requests
comments, check runs, workflow runs, or workflow jobs through the feedback
helper. Fetch errors use the Task 02 batch handling, including rate-limit stop
and partial-count behavior.
Added deterministic coverage for terminal and open PR reads, approval-based
cleanup, historical-row cleanup, and rate limits. The focused red tests failed
before the production change as expected. Verification passed:
```bash
(cd apps/backend && go test ./internal/github -count=1)
```
@@ -0,0 +1,109 @@
---
id: "05-bounded-fallback"
title: "Bound per-workspace PR fallback"
status: done
wave: 5
depends_on: ["04-minimal-cleanup"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-002
acceptance_criteria:
- AC-INTEGRATIONS-GITHUB-PR-POLLING-002.1
- AC-INTEGRATIONS-GITHUB-PR-POLLING-002.2
- AC-INTEGRATIONS-GITHUB-PR-POLLING-002.3
- AC-INTEGRATIONS-GITHUB-PR-POLLING-002.4
system_design:
- ../../specs/integrations/system-design/github-pr-discovery-health.md
---
# Task 05: Bound per-workspace PR fallback
## Summary
Bound per-workspace PR fallback. Implement with TDD and preserve workspace identity.
## In scope
Replace the all-workspace boolean result from tryBatchedPRWatchCheck with
per-workspace outcomes. Retain auth, rate-limit, and invalid-query suppression.
Publish successful workspace results even when another workspace fails.
Permit at most 5 fallback targets per workspace and 10 per cycle globally.
Rotate both workspaces and targets across cycles. Unsupported GraphQL clients
use the same budget. Stop the affected workspace on auth or rate-limit failure.
Do not mark deferred work as checked or consume admission permits indefinitely.
Add TestPRWatchFallbackBudgetAndFairness to poller_test.go. Cover multiple
workspaces, successful earlier batches, unsupported clients, transient errors,
rate limits during fallback, cancellation, event publication, and fair rotation.
Retain TestCheckPRWatches_BatchedAuthFailureSkipsPerWatchFallback.
## Out of scope
New UI controls, credential policies, global schedulers, and unrelated providers.
## Acceptance
- Meet every linked acceptance criterion with deterministic regressions.
- Preserve existing policy, scope, and error-handling contracts.
- Record the expected red test result and passing package results.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/poller.go`
- `apps/backend/internal/github/poller_test.go`
- `apps/backend/internal/github/poller_circuit_test.go`
- `apps/backend/internal/github/service_pr_watch_batched.go`
## Dependencies
04-minimal-cleanup.
## Risks
A target can cost several HTTP requests. The budget limits targets, not pagination.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/github-pr-discovery-health.md)
- [Design](../../specs/integrations/system-design/github-pr-discovery-health.md)
- Source files and existing adjacent tests listed above. New helper/test files are created as needed.
## Results
Implemented per-workspace PR batch outcomes and bounded REST fallback. A
successful workspace publishes its results immediately and is never replayed
through REST when another workspace fails. Authentication, rate-limit, and
invalid-query outcomes remain deferred; transient failures and clients without
GraphQL support use a rotating round-robin fallback with a five-target
workspace budget and ten-target cycle budget. Fallback stops the affected
workspace on rate-limit or authentication errors and stops the cycle on
cancellation. The passive workspace refresh path applies the same five-target
workspace and ten-target rolling one-minute budgets with fair target rotation;
it classifies batch failures before fallback and runs fallback checks
sequentially so an authentication or rate-limit error leaves no unscheduled
tail of calls for that workspace.
Equivalent workspace targets are deduplicated before workspace and global
budgets are admitted, and one fallback result is applied to every watch in the
target group.
Added deterministic coverage for mixed workspaces, successful event
publication, unsupported and transient clients, target rotation, fallback
budgets, rate-limit stopping, and cancellation. Verification passed:
```bash
(cd apps/backend && go test ./internal/github -count=1)
```
@@ -0,0 +1,153 @@
---
id: "06-adaptive-discovery"
title: "Back off idle searching watches"
status: done
wave: 6
depends_on: ["05-bounded-fallback"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-001
acceptance_criteria:
- AC-INTEGRATIONS-GITHUB-PR-POLLING-001.1
- AC-INTEGRATIONS-GITHUB-PR-POLLING-001.2
- AC-INTEGRATIONS-GITHUB-PR-POLLING-001.3
- AC-INTEGRATIONS-GITHUB-PR-POLLING-001.4
- AC-INTEGRATIONS-GITHUB-PR-POLLING-001.5
system_design:
- ../../specs/integrations/system-design/github-pr-discovery-health.md
---
# Task 06: Back off idle searching watches
## Summary
Back off idle searching watches. Implement with TDD and preserve workspace identity.
## In scope
Add a due-target selector with an injected clock. Keep the scheduler tick at
1 minute. Searching intervals are 1 minute below 2 hours idle or while running,
15 minutes from 2 to below 24 hours, and 30 minutes thereafter.
New watches receive a next-tick attempt. Known PRs remain at 1 minute.
Load task activity in bulk through a narrow adapter. Reuse task-domain query
patterns without treating generic UpdatedAt or provider notifications as activity.
Include session state, human conversation, agent execution, and observed branch
commit/push activity. Missing evidence retains fast discovery. No provider reads
or remote Git scans are allowed for activity classification.
Use the fastest eligible member for shared discovery targets. Preserve due
state across reconciliation and restart using existing persisted activity and
LastCheckedAt. Passive sync must honor scheduling; explicit refresh bypasses
idleness only. Keep quota/auth admission intact.
Add TestSearchingWatchAdaptiveSchedule in a new poller_schedule_test.go.
Cover exact boundaries, restart, activity after a slow check, unknown activity,
all-idle and mixed shared targets, archive/delete, and external PR discovery
without a new commit. Add service-level passive/explicit refresh regressions.
Inspect LoadTaskLastActivity in task/repository/sqlite/task_status_summary.go
as a query pattern, not as an assumed clean activity signal.
Update `docs/public/integrations.md` during implementation to describe the
shipped freshness bounds. Keep the public guide unchanged during planning.
Run these additional checks after that documentation edit:
```bash
node --test scripts/validate-public-docs.test.mjs
node scripts/validate-public-docs.mjs
```
## Out of scope
New UI controls, credential policies, global schedulers, and unrelated providers.
## Acceptance
- Meet every linked acceptance criterion with deterministic regressions.
- Preserve existing policy, scope, and error-handling contracts.
- Record the expected red test result and passing package results.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/poller.go`
- `apps/backend/internal/github/poller_schedule.go`
- `apps/backend/internal/github/poller_schedule_test.go`
- `apps/backend/internal/github/store.go`
- `apps/backend/internal/github/models.go`
- `apps/backend/internal/github/service_pr_watch.go`
- `apps/backend/internal/github/service_pr_watch_batched.go`
- `apps/backend/internal/task/repository/interface.go` (only if the activity adapter needs a new task-domain query)
- `apps/backend/internal/task/repository/sqlite/task_status_summary.go` and adjacent tests
If the task activity query changes, also run:
```bash
(cd apps/backend && go test ./internal/task/repository/sqlite -run "Activity|Watch" -count=1)
```
- `docs/public/integrations.md`
## Dependencies
05-bounded-fallback.
## Risks
Provider writes can contaminate generic timestamps. Passive UI requests can defeat backoff.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/github-pr-discovery-health.md)
- [Design](../../specs/integrations/system-design/github-pr-discovery-health.md)
- Source files and existing adjacent tests listed above. New helper/test files are created as needed.
## Results
Implemented adaptive passive discovery scheduling with a one-minute scheduler
tick. Searching watches check every minute while running or recently active,
then back off to 15 minutes after two hours of inactivity and 30 minutes after
24 hours. A newly created watch uses the newer of its creation time and the
authoritative task activity timestamp, so an old task does not make a fresh
watch enter a slow tier after its first check. Known pull requests remain on
the one-minute cadence, and activity, restart, archive, and deletion behavior
retain due state correctly.
Added a bulk task-activity projection and backend adapter. The projection uses
human conversation, agent execution, session state, and observed branch
activity, while excluding generic task timestamps, lifecycle messages, and
live-monitor snapshots. Shared discovery targets use the fastest eligible
watch, while numbered and searching watches sharing a branch keep separate due
admission. Passive refresh and its bounded per-watch fallback honor due admission;
explicit refresh remains immediate.
Passive workspace reads also use a one-minute per-workspace admission cooldown
when no stale task requires refresh. New or reset watches clear that cooldown,
and stale-task refreshes bypass it.
The task WebSocket sync boundary now carries an explicit refresh bit. Automatic
subscription, reconnect, and retry requests use passive due admission, while a
user refresh bypasses idle admission and still obeys quota and authentication
gates. An explicit refresh supersedes an active passive provider attempt rather
than reusing its result; concurrent explicit refreshes still share provider
work, and discovery retry deadlines remain authoritative. Added deterministic scheduler, repository-query, production-shaped
handler, and passive/explicit refresh regressions. Verification passed:
```bash
(cd apps/backend && go test ./internal/github -count=1)
(cd apps/backend && go test ./internal/task/repository/sqlite -run "Activity|Watch" -count=1)
node --test scripts/validate-public-docs.test.mjs
node scripts/validate-public-docs.mjs
```
@@ -0,0 +1,137 @@
---
id: "07-actions-cache"
title: "Cache Actions observations with bounded expiry"
status: done
wave: 7
depends_on: ["06-adaptive-discovery"]
plan: "plan.md"
requirements:
- REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002
acceptance_criteria:
- AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.1
- AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.2
- AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.3
- AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.4
system_design:
- ../../specs/integrations/system-design/github-workflow-attention.md
---
# Task 07: Cache Actions observations with bounded expiry
## Summary
Cache Actions observations with bounded expiry. Implement with TDD and preserve workspace identity.
## In scope
Share workflow-run and job observations across batch enrichment and service
status/feedback reads. Scope by credential generation and repository; runs also
use head SHA, and jobs use run ID plus attempt. Classify each PR separately.
Use 30 seconds for empty, pending, running, unknown, or attention-required data.
Use 5 minutes only for nonempty completed data without attention requirements.
Keep 512 entries each for runs and jobs, and coalesce concurrent misses. Do not
cache errors or change merge/check eligibility. Preserve the existing 4-worker,
5-second batch enrichment budget.
Explicit refresh invalidates relevant outer and Actions cache entries. Prevent
in-flight old observations from repopulating after invalidation. Head and
credential changes must select fresh entries. Future mutation hooks can reuse
this invalidation seam; no approval or rerun UI is added.
Add TestWorkflowAttentionCacheExpiryAndInvalidation to a new
workflow_attention_cache_test.go. Cover same-SHA reruns, new run attempts,
empty/approval/error results, concurrent callers, credential isolation, different
PRs sharing one SHA, explicit refresh, and bounded eviction. Use fake time.
Retain the existing workflow attention classification suite.
Update `docs/public/integrations.md` during implementation to describe the
shipped freshness bounds. Keep the public guide unchanged during planning.
Run these additional checks after that documentation edit:
```bash
node --test scripts/validate-public-docs.test.mjs
node scripts/validate-public-docs.mjs
```
## Out of scope
New UI controls, credential policies, global schedulers, and unrelated providers.
## Acceptance
- Meet every linked acceptance criterion with deterministic regressions.
- Preserve existing policy, scope, and error-handling contracts.
- Record the expected red test result and passing package results.
## Verification
Run from the repository root.
```bash
(cd apps/backend && go test ./internal/github -count=1)
git diff --check
```
## Files likely touched
- `apps/backend/internal/github/service_pr_watch_batched.go`
- `apps/backend/internal/github/service_pr.go`
- `apps/backend/internal/github/service.go`
- `apps/backend/internal/github/ttl_cache.go`
- `apps/backend/internal/github/workflow_attention.go`
- `apps/backend/internal/github/workflow_attention_cache.go`
- `apps/backend/internal/github/workflow_attention_cache_test.go`
- `apps/backend/internal/github/workflow_attention_test.go`
- `docs/public/integrations.md`
## Dependencies
06-adaptive-discovery.
## Risks
Same-SHA results can change. Never freeze a completed workflow collection indefinitely.
## Parallelism
`sequential`
## Inputs
- [Requirements](../../specs/integrations/requirements/github-workflow-attention.md)
- [Design](../../specs/integrations/system-design/github-workflow-attention.md)
- Source files and existing adjacent tests listed above. New helper/test files are created as needed.
## Results
Implemented bounded Actions observation caches shared by batched workflow
enrichment and service status, feedback, and watch reads. Run entries are
scoped by credential, repository, and head SHA. Job entries are scoped by
credential, repository, run ID, and attempt. Completed nonempty observations
without attention use a five-minute TTL; empty, changing, unknown, and
attention-required observations use 30 seconds. Each cache has 512 entries,
singleflight coalescing, and provider-error omission.
Explicit PR refresh invalidates the affected outer status or feedback entry
and repository Actions entries. Automatic task subscriptions, reconnects, and
retries use passive sync and preserve a completed Actions observation; only a
user refresh carries explicit refresh intent and invalidates the relevant
entries. Per-key epochs prevent an older in-flight read from repopulating an
invalidated cache. Credential changes clear the caches and credential
generations remain part of every workspace cache key. Existing workflow
classification, fork identity, check state, merge eligibility, and the
four-worker, five-second batch budget remain unchanged.
Added `TestWorkflowAttentionCacheExpiryAndInvalidation` and an in-flight
invalidation regression covering same-SHA reruns, attempts, empty and
attention data, provider errors, concurrent callers, credential isolation,
shared SHAs, explicit refresh, and bounded eviction. The focused test failed
before the cache seams existed as expected. Verification passed:
```bash
(cd apps/backend && go test ./internal/github -count=1)
node --test scripts/validate-public-docs.test.mjs
node scripts/validate-public-docs.mjs
```
+24
View File
@@ -109,6 +109,19 @@ The **PR discovery failed** warning is separate from the quota values:
When discovery succeeds, the warning clears and the status revision advances. If the warning remains after a quota refresh, wait for the retry time or verify the selected connection and repository scope.
#### Pull request watch freshness
Background checks for a known pull request run at least once per minute. A
watch that is still searching for a pull request also checks once per minute
while its task is running or has activity within the last two hours. Searching
checks use a 15-minute interval after two hours of inactivity and a 30-minute
interval after 24 hours. The checks continue at the 30-minute interval, so a
pull request opened later on an unchanged branch remains discoverable.
Passive workspace refreshes follow these intervals. An explicit pull request
refresh checks immediately, subject to the workspace's authentication and
GitHub rate limits.
### Automation and personal identity
PAT and CLI connections are human identities. They provide both workspace automation and the fallback identity for **My GitHub** views and user-triggered actions. Settings show this shared identity inside **Workspace GitHub access** instead of repeating it as a separate **My GitHub identity** section.
@@ -341,6 +354,17 @@ For a task with linked GitHub pull requests, open the PR status control above th
If a current pull-request head has a GitHub Actions workflow that requires maintainer approval, the PR status control shows **Awaiting maintainer approval**, even when GitHub reports no checks. Detailed desktop and mobile views show the workflow name, the reason, and a link to GitHub. Approval-only workflow attention is not a failed check, does not start **Auto-fix CI & address comments**, and does not make the pull request ready for **Auto-merge or requeue when ready**. If GitHub does not provide enough evidence, Kandev keeps the workflow state unavailable or marks the last same-head observation as stale instead of claiming approval.
Kandev refreshes GitHub Actions observations for empty, running, changing, or
attention-required results within 30 seconds. Nonempty completed results with
no attention requirement remain cached for up to five minutes. The cache is
Run observations are scoped to the selected credential, repository, and head
SHA. Job observations are scoped to the selected credential, repository, run
ID, and workflow attempt.
An explicit pull request refresh clears the relevant observation immediately.
Reruns on the same SHA appear after the completed-result cache expires or after
an explicit refresh. Provider errors are retried and are not stored as an
empty workflow result.
This is a GitHub-only lifecycle feature. Kandev reuses the existing lightweight task PR poller, which checks watched linked PRs roughly once per minute; it does not add a separate scheduler. Saving enabled options also evaluates the task's current linked PRs without waiting for the next poll.
Auto-fix waits for the pull request checks to settle, then sends one repair round for newly failed checks, review comments, an ordinary merge conflict, or an actionable merge-queue removal. It snapshots each check, comment, conflict, and removal state, so repeated observations do not create duplicate rounds. An authoritative conflict resolution clears its checkpoint without using a round; an unknown mergeability state preserves the checkpoint. Updating an already queued auto-fix message does not use another round, and auto-fix pauses after 10 rounds unless the user disables and re-enables it.
@@ -72,6 +72,21 @@ and provider status. Task and UI systems consume its results.
immediate fallback using the same invalid query or an automatic retry more
often than once per minute for that discovery target.
### REQ-INTEGRATIONS-GITHUB-PR-POLLING-001: Adaptive background discovery
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-001.1:** Searching watches shall use a 1-minute interval while their task runs or has less than 2 hours of inactivity. They shall use 15 minutes from 2 hours to less than 24 hours, then 30 minutes.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-001.2:** Task execution, task conversation activity, or an observed branch commit or push shall restore fast discovery. A new watch shall receive an initial attempt on the next poll tick.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-001.3:** Idle discovery shall not stop permanently. A PR opened externally on an unchanged branch shall remain discoverable. Known PRs shall retain their 1-minute background cadence.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-001.4:** Explicit PR refresh shall bypass the idle schedule but obey existing rate-limit and authentication admission. Passive frontend refreshes shall obey the idle schedule for searching targets. An explicit refresh shall not reuse the result of an active passive provider attempt.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-001.5:** Poll attempts and provider status writes shall not count as task activity. Shared targets shall use the fastest eligible member's schedule. Archived and deleted members shall not accelerate it.
### REQ-INTEGRATIONS-GITHUB-PR-POLLING-002: Bounded batch fallback
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-002.1:** Rate-limit, authentication, and invalid-query batch failures shall not trigger per-watch fallback for the affected workspace. Successful workspace batches shall not be repeated because another workspace failed.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-002.2:** Transient failures shall permit at most 5 fallback target checks per workspace and 10 across one poll cycle. Deferred targets shall retain their state and receive a fair opportunity on later cycles.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-002.3:** A rate-limit or authentication error during fallback shall stop remaining checks for that workspace. Cancellation shall stop the cycle. Existing retry deadlines remain authoritative.
- **AC-INTEGRATIONS-GITHUB-PR-POLLING-002.4:** Clients without GraphQL support shall retain REST support under the same bounded schedule. Deferral shall not imply a successful provider observation or discard pending status events.
## Exclusions
No new credentials, automatic PR creation, inferred PR links from chat, changed
@@ -86,3 +101,7 @@ runtime status, not a permanent incident archive.
- [System design](../system-design/github-pr-discovery-health.md)
- [Implementation package](../../../plans/github-pr-discovery-health/plan.md)
- [Watch reconciliation repair](../../../plans/github-pr-watch-reconciliation/plan.md)
## Polling efficiency package
- [Implementation plan](../../../plans/watch-task-cleanup/plan.md)
@@ -26,11 +26,18 @@ The shared [task summary](../../ui/requirements/pr-task-status-summary.md) owns
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.2:** Approval-only workflows shall not count as failed, passed, or running checks. They shall not trigger CI repair rounds or enable automatic merging.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.3:** Desktop task summaries and the existing phone PR drawer shall show the same localized reason. Detailed surfaces shall link to GitHub.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.4:** When approval and failed checks coexist, Kandev shall show both. Each linked PR shall retain its own status.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.5:** After approval, cancellation, rerun, or a head change, the next successful refresh shall reconcile the reason. Reloads shall retain the latest stored observation.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.5:** After approval, cancellation, rerun, or a head change, the next successful provider refresh shall reconcile the reason. Explicit refresh shall bypass cached Actions observations. Reloads shall retain the latest stored observation.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.6:** Missing checks or unstable mergeability alone shall not imply approval. Unavailable provider evidence shall remain distinct from an observed absence of approval gates.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.7:** When GitHub requires an action but the reason is ambiguous, Kandev shall show "Workflow needs attention" instead of an approval claim.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001.8:** An approval reason shall replace unexplained "unstable" copy in the affected summary. Independent conflicts, reviews, queue state, and actual failures shall remain visible.
### REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002: Bounded Actions caching
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.1:** Equivalent Actions reads shall share cached results and concurrent fetches within one credential scope, repository, and head SHA. Different credentials shall never share results.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.2:** Background observations shall expire after 30 seconds for empty, pending, running, or attention-required results. Nonempty completed results without attention requirements shall expire after 5 minutes, including success and failure.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.3:** A head change, credential change, or explicit refresh shall prevent reuse of the affected stale observation. Same-SHA reruns shall become visible after expiry and the next successful poll.
- **AC-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002.4:** Provider errors shall not become cached absence. Cached observations shall not become merge eligibility evidence. Classification shall remain specific to each PR, including fork identity.
## Out of scope
- Approving, rerunning, cancelling, or merging GitHub workflows or pull requests.
@@ -41,3 +48,5 @@ The shared [task summary](../../ui/requirements/pr-task-status-summary.md) owns
## Implementation plans
- [Workflow attention](../../../plans/github-workflow-attention/plan.md)
- [Polling efficiency](../../../plans/watch-task-cleanup/plan.md)
@@ -0,0 +1,41 @@
---
status: draft
system: integrations
created: 2026-09-22
owners:
- kandev
---
# Watch task cleanup
## Overview
Integration cleanup must preserve provider capacity for active work. Historical
tasks must not generate repeated provider requests. Integrations own this
contract because they select cleanup candidates and request provider state.
## Requirements
### REQ-INTEGRATIONS-WATCH-CLEANUP-001: Cleanup eligibility
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.1:** Every archived or deleted task shall generate zero provider state requests during a cleanup sweep. This applies to GitHub reviews and issues, GitLab reviews and issues, and Azure DevOps reviews and work items.
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.2:** In a collection containing active, archived, deleted, and unassigned records, each existing unarchived task shall remain eligible independently. Eligibility shall not depend on another record's state.
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.3:** GitHub reservations without an assigned task shall remain eligible. Terminal reservations shall be removed under `auto` and `always`, without increasing the deleted-task count. `never` shall preserve them without provider requests.
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.4:** Existing policies shall remain effective for active tasks. `never` shall make no cleanup state requests. `always` shall delete terminal tasks. `auto` shall preserve user-engaged tasks and existing lifecycle-prompt protections.
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.5:** Background and manual cleanup shall use the same task eligibility rules. Watch reset, explicit watch deletion, workspace authorization, and deduplication shall retain their existing behavior.
- **AC-INTEGRATIONS-WATCH-CLEANUP-001.6:** Review cleanup shall request only PR state and, for open PRs, review and viewer identity evidence. It shall not request comments, checks, workflow runs, or workflow jobs.
### REQ-INTEGRATIONS-WATCH-CLEANUP-002: GitHub rate-limit termination
- **AC-INTEGRATIONS-WATCH-CLEANUP-002.1:** After a GitHub cleanup batch receives a recognized rate-limit error, it shall start no further row checks. Completed deletions shall remain counted. Unprocessed tasks and reservations shall remain intact.
- **AC-INTEGRATIONS-WATCH-CLEANUP-002.2:** Rate-limit recognition shall include HTTP 429 and HTTP 403 responses indicating rate limits or abuse detection. An unrelated HTTP 403 shall not be classified as a rate limit.
- **AC-INTEGRATIONS-WATCH-CLEANUP-002.3:** A cleanup sweep shall return its rate-limit error to its caller. That poll cycle shall skip its remaining cleanup calls after this error. Ordinary row errors shall preserve the row and retain existing continuation behavior.
- **AC-INTEGRATIONS-WATCH-CLEANUP-002.4:** A later scheduled sweep shall remain able to retry. Cancellation shall stop remaining batch work. Requests already running within one feedback fetch may finish or be cancelled.
## Exclusions
No cleanup policy, task retention setting, UI control, or provider credential
contract is added. Unarchived tasks on a Done step remain active candidates.
This contract does not guarantee spare quota against unrelated API consumers.
GitLab and Azure DevOps reservation semantics remain unchanged.
@@ -2,6 +2,8 @@
status: draft
system: integrations
requirements:
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-001
- REQ-INTEGRATIONS-GITHUB-PR-POLLING-002
- REQ-INTEGRATIONS-GITHUB-PR-DISCOVERY-001
- REQ-INTEGRATIONS-GITHUB-PR-DISCOVERY-HEALTH-001
created: 2026-09-11
@@ -187,3 +189,86 @@ historical scope and results.
- [Sync coordination](github-task-pr-sync-coordination.md)
- [Authentication ownership ADR](../../../decisions/0047-github-authentication-ownership.md)
- [Implementation package](../../../plans/github-pr-discovery-health/plan.md)
## Adaptive discovery schedule
`REQ-INTEGRATIONS-GITHUB-PR-POLLING-001` maps to this section.
Keep `defaultPRPollInterval` at 1 minute. Select due searching targets before
batch construction. Known numbered watches keep the current cadence.
| Searching task state | Minimum interval |
| --- | --- |
| Running or activity age below 2 hours | 1 minute |
| Activity age from 2 hours to below 24 hours | 15 minutes |
| Activity age at least 24 hours | 30 minutes |
Use injected time and `PRWatch.LastCheckedAt` for due checks. Round execution
to the next 1-minute tick, so actual delay may include one extra tick.
Use watch creation as an activity baseline for the first 2 hours.
A never-checked watch is immediately eligible. Missing activity evidence
fails open to fast polling. No new random delay is required in this package.
Add a narrow task-activity projection, loaded in bulk for eligible watch tasks.
Reuse the bounded query pattern in
`task/repository/sqlite/task_status_summary.go:LoadTaskLastActivity`, but not
its generic task-update source. Preserve that existing UI query unchanged.
Use persisted session execution state and conversation timestamps, plus observed
commit/push activity. Do not use generic task or watch `UpdatedAt`: GitHub sync
can update those fields itself. Exclude automatic provider notifications from
conversation activity. Include human messages and agent execution activity.
Do not scan remote branches or invoke GitHub to calculate idleness.
Record observed branch activity from existing local Git event paths; absence of
such an event is not proof that no remote push occurred.
An activity transition restores next-tick eligibility even after a slow poll.
Persisted task activity and last-check timestamps survive restart. A runtime
branch-activity marker may reset on restart without stopping periodic discovery.
Expire runtime markers after 24 hours and remove them with their watch.
Reconciliation of an unchanged watch must preserve its age and last check.
Group equivalent discovery targets before admission. Any active member selects
the 1-minute schedule for that target; do not duplicate the provider query.
Numbered targets and searching targets remain separate groups even when they
share a repository and branch, because their polling cadences differ.
Passive task/page refresh must use the same admission rule or it defeats the
background savings. Explicit user refresh bypasses only idle admission, never
quota or auth gates. Preserve `PRSyncFreshnessWindow` at 30 seconds for existing
sync consumers and keep the distinction between automatic and explicit reads.
If a passive provider attempt is already active, an explicit refresh starts a
new admitted attempt instead of joining the passive result. The superseded
attempt cannot publish its result, and concurrent explicit refreshes share the
explicit batch singleflight key. A retry deadline from an authentication,
rate-limit, or invalid-query failure remains authoritative.
When a passive workspace read finds no stale task, admit workspace discovery
reads at most once per minute. A newly created or reset watch clears that
cooldown, and stale-task refreshes bypass it.
## Bounded fallback
`REQ-INTEGRATIONS-GITHUB-PR-POLLING-002` maps to this section.
Replace the poller's all-or-nothing boolean batch outcome with per-workspace
outcomes: completed, deferred, fallback-eligible. Publish successful results
without waiting for every workspace to succeed. Preserve existing shared
health admission for rate limits and invalid queries, and auth circuits.
Allow 5 fallback target checks per workspace, capped at 10 per cycle globally.
Apply this limit to unsupported-GraphQL clients too. Rotate workspace and target
selection between cycles so fixed ordering cannot starve later targets.
These are target-check limits, not HTTP-request limits: a check may paginate
or fetch several resources. Stop a workspace on the first authentication,
configuration, rate-limit, or invalid-query error and check cancellation before
each target. Never retry successful batches via REST.
Do not mark deferred targets as successfully checked.
Deduplicate equivalent watches before consuming either budget and apply one
fallback result to every watch in the selected target group.
Retain existing failure schedules: discovery retry starts at 1 minute and caps
at 15 minutes; auth/config circuits start at 2 minutes and cap at 6 hours with
25 percent jitter. Provider reset deadlines and credential changes retain
existing handling. This package adds no independent retry loop.
## Polling efficiency delivery
See [the implementation plan](../../../plans/watch-task-cleanup/plan.md).
Use deterministic clock tests for exact 2-hour and 24-hour boundaries,
restart, mixed groups, passive reads, explicit refresh, and fallback fairness.
@@ -2,6 +2,7 @@
status: draft
system: integrations
requirements:
- REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002
- REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-001
---
@@ -144,3 +145,42 @@ Desktop and mobile Playwright tests verify the existing entry points and the pro
- [Requirements](../requirements/github-workflow-attention.md)
- [Shared task summary design](../../ui/system-design/pr-task-status-summary.md)
- [Implementation plan](../../../plans/github-workflow-attention/plan.md)
## Expiring Actions observations
`REQ-INTEGRATIONS-GITHUB-WORKFLOW-ATTENTION-002` maps to this section.
Share raw workflow-run observations across batched enrichment and service
feedback/status reads. Keep per-PR classification outside the shared cache.
Key entries by credential scope and generation, normalized repository identity,
and head SHA. Reuse the bounded `ttlCache` and its singleflight/invalidation
patterns; add per-entry expiry support if needed. Keep at most 512 run entries
and 512 job entries. Errors are not cached.
| Observation | TTL |
| --- | --- |
| Empty list or any pending/running/unknown status | 30 seconds |
| Any approval/action-required or ambiguous attention result | 30 seconds |
| Nonempty, all completed, no attention requirement | 5 minutes |
A SHA does not freeze the workflow run collection. Reruns and new runs can
arrive for the same SHA. Never use an infinite TTL. Cache job observations by
credential scope, repository, run ID, and attempt, with the same expiry classes.
A run-attempt change must not reuse jobs from an earlier attempt.
Keep existing batched enrichment limits: concurrency 4 and total budget
5 seconds. Preserve the outer status/search cache TTL of 30 seconds and the
feedback TTL of 8 seconds. Long Actions entries only affect workflow attention,
not checks, review status, or auto-merge decisions.
Explicit refresh invalidates the relevant Actions and outer response entries,
then coalesces concurrent refreshes. Generation checks prevent an older
in-flight request from repopulating invalidated data. Passive page polling
uses the cache. Route future approval/rerun mutations through this invalidation
seam; adding those mutations is outside this package.
External reruns on a completed SHA can take up to 5 minutes plus the next
1-minute poll tick to appear. Attention and running results use the short TTL.
Provider errors retain unknown/stale-positive semantics and existing retry
admission. Terminal PRs continue to skip Actions reads altogether.
See [the implementation plan](../../../plans/watch-task-cleanup/plan.md).
@@ -0,0 +1,126 @@
---
status: draft
system: integrations
requirements:
- REQ-INTEGRATIONS-WATCH-CLEANUP-001
- REQ-INTEGRATIONS-WATCH-CLEANUP-002
created: 2026-09-22
owners:
- kandev
---
# Watch task cleanup design
## Ownership and mapping
Integration stores select candidates. Integration services preserve cleanup
policies and call the existing task deletion adapters.
| Requirement | Design section |
| --- | --- |
| REQ-INTEGRATIONS-WATCH-CLEANUP-001 | Candidate selection and compatibility |
| REQ-INTEGRATIONS-WATCH-CLEANUP-002 | GitHub batch termination |
## Candidate selection and compatibility
Cleanup queries use this predicate for each deduplication row `d`:
```sql
d.task_id = '' OR EXISTS (
SELECT 1 FROM tasks t
WHERE t.id = d.task_id AND t.archived_at IS NULL
)
```
Keep watch, workspace, and generation predicates outside this parenthesized
expression. A plain inner join loses reservations. A left join with only an
archive-null check incorrectly includes deleted tasks.
Apply this filter to GitHub `ListReviewPRTasksByWatch`, `ListAllReviewPRTasks`,
`ListIssueWatchTasksByWatch`, and `ListAllIssueWatchTasks` in `store.go`.
Workspace cleanup already uses the all-record queries and watch ownership.
There are no GitHub workspace-specific list methods in this checkout.
`DeleteReviewWatch` and `DeleteIssueWatch` also consume the by-watch methods.
Move their explicit deletion inventory to the existing unfiltered
`ListReviewPRTaskIDsByWatch` and `ListIssueWatchTaskIDsByWatch` methods.
Preserve their authorization, empty-ID handling, and best-effort deletion.
Reset and deduplication lookups remain unfiltered.
GitLab applies the predicate to all, workspace, and by-watch review and issue
queries in `store_watches.go`. Preserve workspace joins, generation values,
and ordering. Its cleanup currently skips empty reservations and retains that
behavior.
Azure DevOps applies the predicate to `ListWorkItemWatchTasks` and
`ListPullRequestWatchTasks` in `watch_store.go`. Preserve watch generation
scoping. Its cleanup also continues to skip empty reservations.
Archived and missing-task records are ignored, not purged by this change.
Retaining them avoids changing deduplication and allows restored tasks to
become eligible again. Existing task-deletion handlers may still remove them.
No schema migration or index is needed: the existence query uses the task key.
Eligibility reflects committed task state at query time. This change does not
make remote requests atomic with a concurrent archive or deletion.
## GitHub batch termination
Extend the private deletion gates to return errors alongside eligibility and
reason. Extend both batch helpers to return a partial count and error.
Use `errors.As` with `GitHubAPIError` and the existing
`isGitHubRateLimitAPIError` classifier in `errors.go`.
Check cancellation before each row. On a rate-limit error, return immediately,
including on the empty-reservation path. Ordinary fetch errors retain existing
per-row failure tracking and continuation. Do not classify every 403 as a
rate limit. Propagate rate-limit errors from `GetAuthenticatedUser` as well.
Per-watch, global, orphan, and workspace cleanup entry points propagate the
partial count and error. Global sweeps stop before the next workspace batch.
In each review or issue poll cycle, a rate-limit error suppresses subsequent
cleanup calls, including its final orphan sweep. Discovery remains subject to
its existing search-bucket rules. This is a cycle-local stop, not a new global
credential circuit or scheduler.
Keep existing client rate tracking and reset behavior. A later poll retries
normally. Emit one batch-level error through existing caller logging instead
of one new failure for each unvisited row. Unvisited rows do not increment
failure counters.
Cleanup admission evaluates the current reset deadline, so an exhausted
resource becomes eligible again after reset even if no provider call refreshes
the tracker in between sweeps.
## Minimal review cleanup lookup
Replace the cleanup gate's `GetPRFeedback` call with `GetPR`. For merged or
closed PRs, apply the existing policy without additional provider reads.
For open PRs, fetch `ListPRReviews` and the authenticated user only when needed
for the existing approval rule. Preserve approval matching and policy behavior.
Never fetch comments, check runs, workflow runs, or workflow jobs for cleanup.
Reuse existing authenticated-user caching. Do not introduce another cleanup TTL.
Propagate rate-limit errors from each required read to the batch stop path.
Thread the resolved automation `RateTracker` into batch admission and honor
core exhaustion before each row. Never substitute another workspace's tracker.
## Test boundaries
Use real store fixtures and instrumented clients. Cover mixed collections,
all cleanup entry points, policy matrices, empty reservations, preserved
deduplication, and explicit watch deletion. Fixtures representing active tasks
must insert actual task rows. Missing-task fixtures must assert zero provider
calls rather than treating an absent task as active.
Rate tests cover wrapped typed errors, ordinary 403 responses, cancellation,
partial counts, an error on the first reservation, tracker exhaustion, and
later-cycle recovery. Poller tests prove the final orphan sweep does not restart
cleanup after a rate-limit stop.
## Related contracts
- [GitHub identity ownership](../../../decisions/0047-github-authentication-ownership.md)
- [PR lifecycle prompt protection](../../../decisions/0051-pr-agent-notifications-extend-task-pr-automation.md)
- [Requirements](../requirements/watch-task-cleanup.md)
Existing authorization and lifecycle protections remain authoritative. This
local candidate filter needs no new architecture decision.