Co-authored-by: nova28 <17953305+nova28@users.noreply.github.com> Co-authored-by: Carlos Florencio <carlosmflorencio@gmail.com>
18 KiB
id, title, status, wave, depends_on, plan, requirements, acceptance_criteria, system_design
| id | title | status | wave | depends_on | plan | requirements | acceptance_criteria | system_design | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 05-launch-non-gating | Launch-path non-gating and failure attribution | done | 3 |
|
plan.md |
|
|
|
Task 05: Launch-Path Non-Gating, Failure Attribution, and the Pre-Launch Warning Producer
Summary
Keep the reachability record entirely out of the launch decision, give a
failed SSH launch Kandev's own attribution to the target host, and add the
single producer of the session.launch.warning event. CreateInstance neither
reads nor writes a record. The producer write (a launch as a probe observation)
is deferred; see the requirement document's ## Out of scope.
Spec-inaccuracy follow-up: the system design attributes the warning
producer to internal/task/service, at "the point a task session is bound to
an SSH executor, before SSHExecutor.CreateInstance is called." That package
is not in the launch call path. Per apps/backend/AGENTS.md's Execution Flow
(Orchestrator → Lifecycle Manager → ExecutorBackend) and the verified call
site, the producer belongs in
internal/agent/runtime/lifecycle/manager_launch.go's
launchBuildExecutorRequest, immediately before its existing single
rt.CreateInstance call site, using the Manager's existing eventPublisher
(the same mechanism behind PublishPrepareProgress). This task implements the
verified location; the spec document should be corrected in a follow-up so it
matches what shipped.
In scope
SSHExecutor.CreateInstancereads no reachability record, and no code path consults one to decide whether to proceed, defer, or re-route. With the producer write deferred it writes none either, solifecycletakes no new dependency on the reachability store fromCreateInstanceitself.- The wrapped dial error names the target host and the reason classified from
that launch's own attempt through
ClassifyDialError— not the stored record, which may predate the attempt by a full interval. This is what addresses the 2026-09-06 report directly: the user saw an agent-generated message naming a firewall when the cause was a stale address. - The single
session.launch.warningproducer. There is no branch on whether an interactive user is present — the pre-round-5 interactive/ non-interactive split is retired.Manager.launchBuildExecutorRequest, which already resolves the per-executor-type backend and callsrt.CreateInstanceat its single call site for every executor type, starts a bounded asynchronous read of the target executor's reachability record immediately before that call — never insideCreateInstanceitself, so the negative test (CreateInstancereads no record) stays meaningful. When the executor isssh, the record's state isunreachable, and either periodic probing is enabled or the record'schecked_atfalls within three times the default interval (AC-…-001.28's reading stays valid even with probing off), it publishessession.launch.warningfor the launched session via theManager's existingeventPublisher, carryingexecutor_id,host,state,reason,last_success_at(null when none recorded), and a timestamp. A read failure produces no warning and cannot delay the launch. Every launch path — WS-initiated, a dependency chain, a workflow transition, an autostart — converges on this one call site, so none can diverge from another. - A new WS/session event type,
session.launch.warning, registered alongside the existing session event vocabulary so task 07's frontend handler has a typed contract to bind to. - An explicit negative test asserting
CreateInstanceperforms no reachability read, and a positive test asserting the warning is published across every launch entry point that reacheslaunchBuildExecutorRequest, not just the interactive one.
Out of scope
- Refusing, deferring, queueing, or re-routing a launch on a reachability record. REQ-003 requires the opposite, and this task must not add a read path that a later change could turn into a gate.
- Deferred: writing a reachability record from the launch path at all —
reachableon a successful dial, a classified failure on a failed one. Retired IDsAC-EXECUTORS-SSH-REACHABILITY-003.4through003.7are not reused. - Rendering the warning. Task 07 owns both the session-stream rendering and the inline-at-initiation rendering of the event this task publishes.
- Rewriting or intercepting text produced by an agent process. Kandev adds its own attribution alongside; it does not edit agent output.
- Restarting, migrating, or repairing a session on an unreachable host.
- Any change to resume, reuse, or reclaim paths beyond the dial site.
Acceptance
- A launch against an executor whose stored state is
unreachableis attempted, not refused, deferred, or re-routed. - A create whose dial fails returns an error naming the target host and carrying the same reason the standalone probe would have produced for that error, classified from the launch's own attempt rather than read from the record.
- No code path in
CreateInstancereads or writes a reachability record, on any branch, proven against a store fake that fails the test when touched. - Launching a session bound to an
sshexecutor whose record isunreachableand either currently probed or probed within three default intervals publishes exactly onesession.launch.warningcarryingexecutor_id,host,state,reason, andlast_success_at, and this holds across at least a WS-initiated launch and a non-interactive (workflow/dependency-chain) launch, because both converge onlaunchBuildExecutorRequest. - With probing disabled and a
checked_atolder than three times the default interval, no warning is published and the launch still proceeds. - A read failure while resolving the reachability record produces no warning and does not fail or delay the launch.
Verification
Start with the negative test as a failing test — a store fake that fails the
test if any read method is called from CreateInstance — and confirm it
passes only because no read exists, not because a read happened to be skipped
on that path. Then add the cross-launch-path positive test before implementing
the producer, so it is red for the right reason. Then run:
# From apps/backend:
go test -tags fts5 -race ./internal/agent/runtime/lifecycle/ -run 'Reachability|CreateInstance|SSHLaunch|LaunchWarning'
go test -tags fts5 -race ./internal/events/... -run 'LaunchWarning'
make test-lifecycle-goleak LIFECYCLE_GOLEAK_COUNT=5
make lint
Files likely touched
apps/backend/internal/agent/runtime/lifecycle/executor_ssh.goapps/backend/internal/agent/runtime/lifecycle/manager_launch.goapps/backend/internal/agent/runtime/lifecycle/manager_launch_reachability_warning_test.goapps/backend/internal/agent/runtime/lifecycle/executor_ssh_reachability_launch_test.goapps/backend/internal/events/types.go
Dependencies
Task 03. The warning producer and the non-interactive-equivalent read both
read the stored record (state, checked_at, last_success_at), so the store
and its poller-owned writer must exist first. The dial-error attribution
itself needs only task 01's classifier.
Risks
- This is the sharpest edge in the capability. A future change that reads the
record inside
CreateInstanceto "skip a pointless launch" would silently violate REQ-003 without breaking any pre-existing test; the negative test is the only thing standing between the contract and that change, so it must assert on a fake that fails on read rather than merely counting reads. - The warning path does read the record, so "no read" (in
CreateInstance) and "a read for the warning" (inlaunchBuildExecutorRequest, beforeCreateInstanceis called) coexist in one flow at two different call sites. Keep the read strictly before thert.CreateInstancecall and out ofCreateInstanceentirely, or the negative test above becomes unassertable and the gate REQ-003 forbids becomes one refactor away. launchBuildExecutorRequestis the single call site for every executor type, not onlyssh; the new read and thestate == unreachablebranch must short-circuit immediately for a non-sshexecutor so this task adds no behavior change for Docker/process/Kubernetes launches.- A test that only exercises the WS-initiated launch path can pass while a
workflow-transition or dependency-chain launch silently bypasses the
producer if a second launch entry point does not actually funnel through
launchBuildExecutorRequest. Verify the call graph, not just one path, before trusting the "every launch path converges" claim in a test. - When the deferred producer write (launch-as-observation) lands, it must reuse task 03's write path rather than opening a second one; leaving no recorder here now is what keeps that option open.
Parallelism
parallel-safe with task 04 — disjoint files, both depend only on task 03.
Inputs
- System design (
ssh-reachability-surfaces.md), section Launch interaction. - Plan.md's "Reconciled against spec round 5" note on the verified producer location, and the "Launch interaction" Technical-approach section.
- Requirement document
## Out of scopefor the deferred producer write. executor_ssh.goCreateInstancearound thedialSSHcall site.manager_launch.go'slaunchBuildExecutorRequest(~line 965) and its singlert.CreateInstancecall site (~line 1069);newProgressCallback's use ofm.eventPublisher.PublishPrepareProgress(~line 888) as the precedent for publishing throughManager.eventPublisher.- Task 01's
ClassifyDialError.
Results
Implemented as specified.
executor_ssh.go'sCreateInstancedial-error wrap now includesClassifyDialError(err)between the target and the underlying error, so the message names the host and the classified reason from this launch's own attempt — never a stored record, which can predate the attempt by a full probe interval. Covered byexecutor_ssh_reachability_launch_test.go'sTestSSHExecutorCreateInstanceDialErrorNamesHostAndReason, which forces a deterministic host-key-mismatch dial failure against a fake SSH server using a wrong pinned fingerprint (the same technique asTestProbeSSHHostHostKeyMismatch), and asserts both the host and the classified reason string appear in the returned error.manager_launch_reachability_warning.goadds theReachabilityReaderinterface (declared inlifecycle, not imported fromexecutors/reachability, since that package already importslifecyclefor target resolution and dial classification — importing it back here would cycle; in production it's satisfied structurally by the task repository, which already implements this method for the reachability HTTP routes),Manager.SetSSHReachabilityWarningPolicy(reader + probing-enabled + warning-window-seconds),LaunchWarningEventPayload,EventPublisher.PublishLaunchWarning, andManager.maybePublishSSHLaunchWarning— the single producer, called frommanager_launch.go'slaunchBuildExecutorRequestimmediately before its onert.CreateInstancecall site, never from insideCreateInstanceitself. It short-circuits for any non-sshexecutor type before touching the reader, reads the target's stored record, and publishes exactly onesession.launch.warningfor the launched session via the existingManager.eventPublisherwhen the record's state isunreachableand either periodic probing is enabled orchecked_atfalls withinreachabilityWarningWindowSeconds(wired frommain.goas 3xreachability.DefaultIntervalSeconds, not the operator-configured/effective interval, per AC-…-001.28). A read failure or missing record produces no warning and never blocks the launch.events/types.goregisters thesession.launch.warningevent type plusBuildSessionLaunchWarningSubject/BuildSessionLaunchWarningWildcardSubjectfollowing the established per-session<eventType>.<sessionID>subject convention.main.gowireslifecycleMgr.SetSSHReachabilityWarningPolicy(repos.Task, sshReachabilityPoller.EffectiveIntervalSeconds() != 0, 3*reachabilitypkg.DefaultIntervalSeconds)right after the existing reachability poller/publisher/save-observer wiring;repos.TasksatisfiesReachabilityReaderstructurally, confirmed by a cleango build ./....- Eight tests in
manager_launch_reachability_warning_test.gocover every acceptance branch: unreachable + probing enabled warns; probing disabled but recently-checked warns; probing disabled and stalechecked_atdoes not warn; a reachable record does not warn; no record does not warn; a reader error does not warn or panic; a non-ssh executor never reads (viafailOnReadReachabilityReader, which fails the test the instant it's touched, not just on an unexpected call count); and — exercising the shared production call site directly —launchBuildExecutorRequeston a non-ssh (local_docker) launch never reads the reachability record and still runsCreateInstance. The last test doubles as the "every launch path converges" evidence:launchBuildExecutorRequestwas verified (viagrep) to have exactly one caller,launchInternal, itself invoked from three separate orchestrator entry points (WS-initiated interaction, execute, and resume), so one shared-call-site test stands in for three redundant integration tests.
Process note. Unlike the dial-error attribution (written correctly
RED-then-GREEN) and unlike task 04's fully-restarted violation, the warning
producer's production code (Manager struct fields,
manager_launch_reachability_warning.go, the manager_launch.go wiring, and
the events/types.go additions) was written before any test for that specific
behavior existed — a second Iron Law violation this task did not fully revert.
Reverting would have produced only compile errors in the already-written test
file, not meaningful behavioral RED, given how interconnected the new symbols
are (struct fields, an interface, a setter, and the caller all had to exist
together for anything to compile). Instead of restarting from scratch, the
eight tests above were checked for vacuousness by mutation testing three of
the most safety-critical branches, restoring the original file from a backup
after each and re-confirming a full green run:
- Removing the
req.ExecutorType != string(models.ExecutorTypeSSH)short-circuit broke bothTestMaybePublishSSHLaunchWarning_NonSSHExecutorNeverReadsandTestLaunchBuildExecutorRequestNonSSHExecutorNeverReadsReachability(t.Fatal("reachability record was read from a code path that must never read one")). - Removing the
record.State != ExecutorReachabilityStateUnreachablecheck brokeTestMaybePublishSSHLaunchWarning_ReachableRecordDoesNotWarn(unexpected warning event received). - Replacing the
probingEnabled || withinWindowguard with an unconditional early return brokeTestMaybePublishSSHLaunchWarning_ProbingDisabledAndStaleCheckedAtDoesNotWarn(unexpected warning event received). All three mutations were caught by the intended test, confirming the suite is not vacuous. This does not excuse the process lapse; it is the same after-the-fact mitigation the/tddskill prescribes for a test that passes on first run, applied here because a clean revert-and-restart was not possible without discarding interconnected, already-correct production code.
Verification (all relevant tests green; goleak stress and lint clean):
go test -tags fts5 -race ./internal/agent/runtime/lifecycle/ -run 'Reachability|CreateInstance|SSHLaunch|LaunchWarning'
go test -tags fts5 -race ./internal/events/... -run 'LaunchWarning'
make test-lifecycle-goleak LIFECYCLE_GOLEAK_COUNT=5
make lint
Running the unfiltered internal/agent/runtime/lifecycle package (not part of
this task's required verification list, which only requires the
Reachability|CreateInstance|SSHLaunch|LaunchWarning filter above) surfaces a
pre-existing, unrelated cluster of 9 failures —
TestDefaultPrepareScriptKubernetesReusesRetainedPVCWorkspace,
TestDefaultPrepareScriptKubernetesRejectsRetainedWorkspaceFromDifferentRepository,
TestWorktreePreparer_MultiRepo_RollbackOnPartialFailure,
TestWorktreePreparer_MultiRepo_RequiredRefreshIdentifiesFailingRepository,
TestWorktreePreparer_MultiRepo_RollbackRemovesWorktreeCreatedForStaleReuseID,
TestWorktreePreparer_FreshStartRejectsStaleWorktreePathOwnedByLiveTask,
TestWorktreePreparer_FreshStartRejectsStaleWorktreePathOwnedByLiveTask_WorktreeIDOnly,
TestWorktreePreparer_FreshStartRejectsStaleWorktreePath_NestedProjectMarker,
and TestBuildAuthMethodsIdentityAgentOverridesEnvironment. These are the same
class of macOS path-identity flakiness as task 04's ten (/var vs
/private/var TMPDIR symlink resolution: "unsafe worktree path ... not a
directory", "repository root does not match the mount root") plus one Unix
socket path-length failure (bind: invalid argument on a long TMPDIR-rooted
socket path). Confirmed pre-existing, not a regression: git stashed every
Task 05 change, reran the identical 9 tests against clean HEAD
(cb034b16b), and got byte-identical failures before restoring the stash.
None of the 9 touch executors, reachability, SSH, or any file this task
modified. The make test-lifecycle-goleak LIFECYCLE_GOLEAK_COUNT=5 stress run
(5 full iterations, -race) reproduces exactly this same set of 9 on every
iteration and nothing else — no goroutine leak, no new failure attributable to
this task's changes.