kandev/docs/specs/executors/requirements/standalone-control-server-single-driver.md

status, system, created, owners

status	system	created	owners
draft	executors	2026-09-02	kandev

Standalone Control-Server Single Driver and Unowned Lifetime Requirements

Overview

Standalone control-server ownership states what a backend must prove before it may adopt a surviving control server. This document states what is true afterwards: that exactly one backend drives the server at a time, and that a server nobody drives stops its instances and exits rather than leaving agents running unattended.

The split is by subject, not by size. The other document owns admission; this one owns tenure. Removing the parent-death binding that agent survival across a backend restart requires is what makes both necessary: without admission a foreign process could be driven, and without tenure an abandoned server would leak.

Terminology

Control server, instance, adoption, unowned period, capability set, recovery inventory, owning backend and attached are used exactly as defined in standalone control-server ownership.

Requirements

REQ-EXECUTORS-CONTROL-OWNERSHIP-002: Exactly one backend drives an adopted control server

Intent: Adoption introduces a second potential driver of a live agent. Two drivers on one instance produce interleaved turns, duplicated completions, and a transcript that describes neither.

User story: As a Kandev operator, I want a superseded backend to lose control the moment another backend adopts, so a restart overlap cannot corrupt a turn.

Acceptance criteria

• AC-EXECUTORS-CONTROL-OWNERSHIP-002.1: When a backend adopts a control server, the system shall replace the ownership credential as part of adoption, and shall reject every subsequent operation presented with the superseded credential once that replacement is confirmed.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.7: Credential replacement shall take effect in two steps: the control server shall continue to accept the superseded credential until the adopting backend confirms, naming the rotation identifier of AC-EXECUTORS-CONTROL-OWNERSHIP-002.6, that it has durably stored the replacement, and shall stop accepting it on that confirmation. Durably storing the replacement shall mean that both the replacement credential is written to the secret store and the control-server record's credential reference is updated to name it, and the confirmation shall be sent only after both writes have durably completed. Their order relative to each other is unconstrained, because a crash between them is recoverable either way: with only the secret written the record still names the superseded credential, which the control server still accepts and AC-EXECUTORS-CONTROL-OWNERSHIP-002.10 replays; with only the record written it names a credential the secret store does not hold, which is the credential-unavailable path of AC-EXECUTORS-CONTROL-OWNERSHIP-001.9. Sending it earlier is the one ordering that loses the server: the control server stops accepting the superseded credential while the record still names it, so the next start presents a credential that authenticates nothing and cannot adopt a server it owns until the unowned period elapses, which is exactly the lockout this two-phase window exists to make impossible. When either write fails the confirmation shall not be sent, and AC-EXECUTORS-CONTROL-OWNERSHIP-002.5 governs the incomplete adoption. An interruption at any point shall therefore leave at least one credential that both sides hold, so a backend that fails between replacing and storing can adopt again rather than being locked out of a server it owns. Within that window the superseded credential shall authenticate only a further adoption attempt or the ownership-shutdown operation of AC-EXECUTORS-CONTROL-OWNERSHIP-002.9; it shall not authenticate any other instance operation and shall not establish a stream, so the window cannot reintroduce the second driver that AC-EXECUTORS-CONTROL-OWNERSHIP-002.2 exists to prevent. When no confirmation arrives, the superseded credential shall remain acceptable on those terms until the unowned period elapses.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.5: When credential replacement does not succeed, adoption shall be incomplete: the system shall report no recovered instances for that control server and shall not issue any further operation to it until adoption is attempted again.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.6: Every credential replacement shall carry a rotation identifier that the control server assigns from a strictly increasing sequence, returns to the adopting backend with the replacement credential, and requires on the confirmation of AC-EXECUTORS-CONTROL-OWNERSHIP-002.7. The credential issued by the highest-numbered rotation the control server has performed shall be the only one that authenticates an instance operation or a stream. A confirmation naming an earlier rotation identifier shall be accepted and shall have no effect, so a delayed or duplicated confirmation can never revoke a credential issued after it. A repeated confirmation of the highest-numbered rotation shall likewise be accepted and have no effect beyond the first, so a retried confirmation is safe. A confirmation naming an identifier the control server has not issued shall be rejected and shall change nothing, because it cannot have come from a backend this server rotated for.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.8: At most two credentials shall be acceptable to a control server at any moment: the replacement issued by its highest-numbered rotation, for every operation, and the single credential that rotation directly superseded, on the adoption-only terms of AC-EXECUTORS-CONTROL-OWNERSHIP-002.7. Every other credential shall cease to authenticate anything, including a replacement issued by an earlier rotation that was never confirmed. Without this an abandoned unconfirmed replacement would stay valid outside any backend's custody, which is the second driver AC-EXECUTORS-CONTROL-OWNERSHIP-002.2 forbids.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.9: The control server shall expose one ownership-shutdown operation that stops every instance it supervises, together with their agent subprocesses, and exits. It shall be authenticated by any credential in the acceptable set of AC-EXECUTORS-CONTROL-OWNERSHIP-002.8, including the superseded credential on its adoption-only terms, and shall require no prior adoption and no credential rotation. Admitting the superseded credential here does not reintroduce the second driver AC-EXECUTORS-CONTROL-OWNERSHIP-002.2 forbids, because the operation only destroys: it cannot drive an instance, observe its transcript, or outlive the call. Without it a backend that holds only a superseded credential can neither adopt nor stop, which would make AC-EXECUTORS-SURVIVAL-005.5 unsatisfiable on exactly the crash path AC-EXECUTORS-CONTROL-OWNERSHIP-002.7 exists to create.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.10: Credential replacement shall be idempotent under retry. When an adoption attempt presents the credential that the highest-numbered rotation directly superseded, and no confirmation has been received for that rotation, the control server shall return that same rotation's identifier and replacement credential again and shall not allocate a new rotation, so no new identifier is drawn from the strictly increasing sequence of AC-EXECUTORS-CONTROL-OWNERSHIP-002.6 and the acceptable set of AC-EXECUTORS-CONTROL-OWNERSHIP-002.8 does not move. It shall allocate a new rotation only when the presented credential is the one that rotation issued. This satisfies AC-EXECUTORS-CONTROL-OWNERSHIP-002.1 rather than excepting it: the attempt still ends with the adopting backend holding the replacement, which is what that criterion requires. Without this a backend that never receives a rotation's response is driven to retry, each retry rotates again, and after the second the credential it still holds falls outside the two-credential set of AC-EXECUTORS-CONTROL-OWNERSHIP-002.8 and it is locked out of a server it owns until the unowned period elapses, which is the outcome the two-phase window of AC-EXECUTORS-CONTROL-OWNERSHIP-002.7 exists to make impossible. A retry is therefore unbounded-safe: any number of attempts presenting the superseded credential converge on the same rotation.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.2: When a backend adopts a control server, the system shall terminate every event and workspace stream established under a superseded credential, so a prior holder cannot keep consuming an instance's events.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.3: When an operation is rejected for a superseded credential, the system shall stop treating the affected executions as its own, shall not retry it, and shall not acquire a fresh credential without adopting again.
• AC-EXECUTORS-CONTROL-OWNERSHIP-002.4: When two backends would otherwise act on the same installation, the system shall continue to refuse the second at startup through the runtime-state ownership contract in port collision and backend ownership safety, and this capability shall not weaken or bypass that refusal.

REQ-EXECUTORS-CONTROL-OWNERSHIP-003: An unowned control server terminates itself

Intent: Removing the parent-death binding must not trade a restart problem for a process leak. An unowned control server keeps agents running and spending tokens with nobody watching.

User story: As a Kandev operator, I want an abandoned control server to shut itself down, so a backend that never returns leaves no agents running unattended.

Acceptance criteria

• AC-EXECUTORS-CONTROL-OWNERSHIP-003.1: When a control server has had no owning backend for the unowned period, the system shall stop every instance it supervises, together with their agent subprocesses, and shall exit.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.2: While a backend holds ownership, the system shall keep that ownership current without operator action and shall not begin an unowned shutdown, whether or not any user is interacting with a session and whether or not any agent is mid-turn. Ownership shall be renewed at an interval strictly shorter than one third of the unowned period, so that after two consecutive failed renewals a third attempt still falls strictly inside the period; a successful adoption shall itself count as a renewal. Expiry shall be judged by the elapsed time since the last successful renewal being greater than or equal to the unowned period. Because the interval is strictly shorter than one third, the third attempt is always due strictly before that test can be reached, so the guarantee does not rest on any tiebreak between a renewal and an expiry test falling due at the same instant.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.8: The control server shall expose one ownership-claim operation that both establishes and renews the claim, authenticated by the credential of AC-EXECUTORS-CONTROL-OWNERSHIP-002.6 and naming no instance. A successful call shall set the last-successful-renewal instant to the control server's own monotonic now. Exactly two other operations shall also set it, and no others: the bootstrap handshake that mints the first credential, and a successful credential rotation under AC-EXECUTORS-CONTROL-OWNERSHIP-002.7, including the idempotent replay of AC-EXECUTORS-CONTROL-OWNERSHIP-002.10, which allocates no new rotation but does prove a live backend is still trying to adopt. Beyond those three, no operation shall renew ownership: neither an instance operation, nor an open stream, nor enumeration, so a backend that is attached but idle is not treated as gone and a backend that is busy but has stopped renewing is not treated as present. The handshake carve-out is what gives a freshly spawned server a claim without a separate call; a server that is spawned but never completes that handshake has no successful renewal and its unowned period runs from its start, so a backend that dies mid-handshake leaks nothing. The rotation carve-out is what makes AC-EXECUTORS-CONTROL-OWNERSHIP-003.2's "a successful adoption shall itself count as a renewal" true of an actual operation rather than of nothing: rotation is the first authenticated operation an adopting backend issues, so without it the adopting backend would inherit whatever remained of the previous owner's period, and a server whose owner died just under that period ago would begin its unowned shutdown moments into recovery and stop the very instances being re-tracked. An adopting backend shall begin issuing the ownership-claim operation on the interval of AC-EXECUTORS-CONTROL-OWNERSHIP-003.2 from the point its rotation succeeds, and shall not wait until re-tracking has finished to do so.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.3: When ownership renewal fails for a reason that does not establish the owner is gone, the system shall not shorten the unowned period below its configured value. The claim shall be judged by the time elapsed since the last successful renewal, measured on the control server's own monotonic clock, and not by the outcome of the most recent attempt nor by any wall-clock value either side supplies.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.4: The unowned period shall be configurable and shall default to ten minutes. While per-instance idle reaping is enabled, the unowned period shall be shorter than the idle timeout, so an unowned control server never has its instances reaped for idleness before the unowned shutdown runs. Because both values are independently configurable, the system shall validate the pair when the control server starts; when idle reaping is enabled and the configured unowned period is not shorter than the idle timeout, it shall reduce the unowned period to half the idle timeout and shall record that adjustment, rather than starting with the ordering inverted.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.6: When the per-instance idle timeout is set to the value that disables idle reaping, the ordering constraint of AC-EXECUTORS-CONTROL-OWNERSHIP-003.4 shall be treated as satisfied and the configured unowned period shall be used unchanged, because there is no reaper for the unowned shutdown to precede. The system shall never derive an unowned period from a disabled idle timeout. Disabling idle reaping is a supported configuration, so treating its sentinel as an ordinary duration would let a valid setting collapse the unowned period and stop every surviving instance at start.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.7: The unowned period in force after any validation or adjustment shall be at least one minute. A configured or derived value below that floor shall be raised to it and the adjustment recorded, so that no configuration produces an unowned period at or near zero, which would expire ownership before an adopting backend could claim it and would leave the renewal interval of AC-EXECUTORS-CONTROL-OWNERSHIP-003.2 with no usable value. The floor shall be applied after the adjustment of AC-EXECUTORS-CONTROL-OWNERSHIP-003.4 and shall take precedence over it: when the idle timeout is short enough that no value satisfies both the floor and that ordering, the system shall use the floor, record that the ordering could not be honoured, and start. The consequence is only that the idle reaper may stop instances before the unowned shutdown runs, which AC-EXECUTORS-CONTROL-OWNERSHIP-003.4 already treats as tolerable, whereas honouring the ordering there would reintroduce the near-zero period this criterion exists to forbid.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.9: The decision to begin an unowned shutdown shall be taken once and shall be irreversible. Once taken, the control server shall refuse every ownership-claim operation and every adoption attempt with a distinct shutting-down outcome, and no claim, adoption or renewal shall cancel a shutdown already begun. A claim that the control server accepts strictly before that decision shall prevent the shutdown, so a backend that renews in time is never stopped. A backend receiving the shutting-down outcome shall treat that control server exactly as AC-EXECUTORS-CONTROL-OWNERSHIP-001.8 treats a recorded endpoint that nothing answers: it shall report no recovered instances, shall issue no further operation to it, and shall start a fresh control server. Without this ordering a backend could adopt a server whose instances are already being stopped and re-track them, publishing sessions whose agents are being killed as it publishes them — the failure AC-EXECUTORS-SURVIVAL-002.15 exists to prevent, arriving by a route that criterion does not cover.
• AC-EXECUTORS-CONTROL-OWNERSHIP-003.5: When an unowned shutdown stops an instance, that instance's recovery-inventory record shall be repaired so it no longer claims a live process, preserving its resume token and worktree identity. That repair shall be performed by the next backend to start, through the existing stale-execution repair path, and not by the control server: by construction no backend is attached at that moment, and the control server has no access to the durable store. The deferral is a consequence of that absence, not a rule about who may repair; when a backend is attached and stops a server itself, it repairs immediately, as AC-EXECUTORS-CONTROL-OWNERSHIP-004.3 requires. Until that repair runs the record may name a process that no longer exists, which is the condition that path already exists to correct.

Related design

• Part 1: lifetime and ownership contracts
• Part 2: control flow, failure and scope
• Part 3: recovery data contracts

Out of scope

• What a backend must prove before adopting, and refusal across an incompatible protocol change, both owned by standalone control-server ownership.
• Which sessions are re-tracked after adoption, owned by agent survival across a backend restart.
• Refusing a second backend on one installation, owned by port collision and backend ownership safety.
• Ownership of agentctl processes on the Docker, SSH, Sprites, Kubernetes, and remote-Docker executors. They adopt this contract in later work.