12 KiB
status, created, owner
| status | created | owner |
|---|---|---|
| draft | 2026-04-28 | cfl |
Subscription Usage Tracking
Why
The Office cost dashboard shows $0 for every agent running on a subscription plan (Claude Max, Codex subscription). This is because subscriptions have no per-token billing — cost_cents is always zero, so budget gauges are flat and cost explorer data is meaningless for those agents.
The problem has three layers:
- Visibility: subscription users have no feedback on how intensively they are using their plan quota. They cannot see whether they are at 5% or 95% of their 5-hour rate window.
- Budget enforcement: the existing
BudgetPolicy/ pre-execution budget check machinery is bypassed entirely for subscription agents because there is no spend to check against. Heavy usage just continues unchecked. - Smart scheduling: the wakeup scheduler launches agents without knowing whether the underlying provider API will immediately return a rate-limit error, wasting the retry budget and introducing latency.
Subscription usage tracking closes all three gaps by fetching utilization data directly from provider APIs, surfacing it in the dashboard alongside an estimated-dollar equivalent derived from token counts, and feeding it into the scheduler so over-quota agents are deferred rather than launched into an immediate failure.
What
Billing type detection on agent profiles
Each AgentProfile gains a billing_type field with two values:
api_key— the agent authenticates with an API key (ANTHROPIC_API_KEY,OPENAI_API_KEY, etc.). Dollar costs are calculated from token counts as today.subscription— the agent authenticates via an OAuth flow or subscription credential (Claude OAuth token in~/.claude/.credentials.json, Codex subscription token in~/.config/codex/auth.json). For these agents, utilization percentage is the primary metric.
Billing type is auto-detected from the credential configuration on the AgentProfile rather than being a user-set field:
- Claude profiles: if
CLAUDE_CODE_OAUTH_TOKENis in the credential set, or~/.claude/.credentials.jsonexists and containsclaudeAiOauth, billing type issubscription. Otherwiseapi_key. - Codex profiles: if
~/.config/codex/auth.jsonexists with a subscription token, billing type issubscription. Otherwiseapi_key. - All other agents:
api_key.
The backend computes billing type at profile read time (not stored in DB) and includes it in the AgentProfile response so the frontend can choose the correct display mode without business logic on the client.
Provider usage client
A new internal/agent/usage package exposes a ProviderUsageClient interface:
type UtilizationWindow struct {
Label string // e.g. "5-hour", "7-day"
UtilizationPct float64 // 0-100
ResetAt time.Time // when this window resets
}
type ProviderUsage struct {
Provider string // "anthropic", "openai"
Windows []UtilizationWindow
FetchedAt time.Time
}
type ProviderUsageClient interface {
FetchUsage(ctx context.Context) (*ProviderUsage, error)
}
Two implementations:
ClaudeUsageClient
- Reads
access_tokenfrom~/.claude/.credentials.json(claudeAiOauth.accessToken). - Calls
GET https://api.anthropic.com/api/oauth/usagewithAuthorization: Bearer <token>. - Parses
five_hour.utilizationandseven_day.utilizationfrom the response body. - If the token is expired (
expiresAtis in the past), refreshes it viaPOST https://platform.claude.com/v1/oauth/tokenbefore calling the usage endpoint, then writes the new token back to the credentials file. - Returns two
UtilizationWindowentries:five_hourandseven_day, withResetAtcomputed from the window start + window duration.
CodexUsageClient
- Reads the bearer token from
~/.config/codex/auth.json. - Calls
GET https://chatgpt.com/backend-api/wham/usagewithAuthorization: Bearer <token>. - Reads
x-codex-primary-used-percentandx-codex-secondary-used-percentresponse headers. - Returns two
UtilizationWindowentries:primaryandsecondary.
Utilization cache
A UsageCache sits in front of both clients:
- Per-agent cache entry keyed on
(provider, credential_hash). - Cache TTL: 5 minutes. Usage data is not real-time — polling more than once per 5 minutes is wasteful and may itself count against quotas.
- Cache is invalidated on demand when a scheduler deferral decision needs fresh data.
- The cache is in-memory only (not persisted to SQLite) — a backend restart causes a single fresh fetch per agent, which is acceptable.
The usage service exposes:
func (s *UsageService) GetUsage(ctx context.Context, profileID string) (*ProviderUsage, error)
func (s *UsageService) IsPotentiallyRateLimited(ctx context.Context, profileID string) (bool, time.Time, error)
// Returns (limited, resetAt, err). resetAt is zero if not limited.
IsPotentiallyRateLimited returns true when any utilization window is ≥ 90%.
Estimated dollar equivalent for subscription agents
For subscription agents, cost_cents from the cost events table is zero (no per-token billing). However, token counts are still recorded. The cost explorer augments subscription agent rows with an estimated equivalent cost computed from token counts × LiteLLM community pricing.
LiteLLM pricing table:
- Fetched from
https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.jsonat startup and cached in memory with a 24-hour refresh interval. - The existing hardcoded
pricingTableininternal/office/costs/pricing.gois replaced by the LiteLLM-sourced table, with the current hardcoded values serving as a fallback if the fetch fails. - Pricing resolution: LiteLLM table first, then hardcoded fallback, then
cost_cents = 0with "pricing unavailable" display.
Estimated cost is displayed as ~$X.XX est. in the UI, visually distinct from actual API costs which show as $X.XX. The distinction matters because estimated costs for subscription users do not reflect real money — only quota consumption.
Cost explorer changes
The /office/workspace/costs page is updated for dual-mode display:
API key agents — display unchanged:
- Dollar spend in
$X.XXformat. - Budget utilization gauge if a
BudgetPolicyexists.
Subscription agents — new display:
- Utilization progress bars per window:
5h window: [████░░░] 62%,7d window: [██░░░░░] 28%. - Window reset time shown on hover: "Resets in 2h 14m".
- Estimated equivalent cost below the gauge:
~$3.40 est.(derived from token counts). - A subtle "subscription" badge on the agent row to distinguish from API key agents.
- Budget policies are still available for subscription agents but operate on estimated-cost thresholds, with a tooltip explaining they are estimates.
The cost overview summary bar gains a new entry: "Subscription quota" showing the highest utilization across all subscription agents. If any subscription agent is ≥80%, the card turns amber; ≥90% turns red.
Smart scheduling: subscription-aware deferral
The scheduler's processWakeup method gains a utilization check step inserted between the existing checkBudget step and the resolveExecutorForWakeup step:
Guard → Checkout → Budget check → Utilization check → Executor resolve → Launch
The utilization check:
- Calls
UsageService.IsPotentiallyRateLimited(ctx, agent.AgentProfileID). - If the agent is a
subscriptionbilling type AND any window is ≥ 90%:- Mark the wakeup
status = "deferred"(new status value) withdeferred_until = resetAt. - Release the task checkout.
- Log activity:
wakeup_utilization_deferredwith{agent, window, utilization_pct, deferred_until}.
- Mark the wakeup
- A new scheduler pass runs every 30 seconds to re-queue deferred wakeups whose
deferred_untilhas passed (sets status back toqueued). - For
api_keyagents, the utilization check is skipped entirely — it only applies to subscription accounts where provider rate limits are enforced per-subscription rather than per-request.
The deferral threshold (90%) is configurable via a workspace setting subscription_deferral_threshold_pct (default 90).
Inbox notification for utilization spikes
When a subscription agent's utilization crosses 80% for any window, an inbox item is created (once per window per crossing, not on every check):
- Title: "Claude subscription quota at 82% (5-hour window)"
- Body: "Agent [name] is near its rate limit. Wakeups will be deferred automatically at 90%. Resets in 3h 20m."
Crossing detection uses a persistent per-agent watermark stored in SQLite to avoid flooding the inbox on repeated checks.
Agent detail page
The agent detail page at /office/agents/[id] Overview tab gains a Quota section (only shown for subscription billing type):
- Progress bar per utilization window with percentage and reset time.
- A note if the agent is currently deferred.
Per-agent billing type in API responses
The AgentInstance response (from /api/v1/office/workspaces/:id/agents) gains a read-only billing_type field ("api_key" | "subscription") and a utilization field (null for api_key agents, ProviderUsage struct for subscription agents). The frontend reads these and selects the correct display mode without any client-side provider detection logic.
Scenarios
-
GIVEN a Claude agent configured with OAuth credentials, WHEN the cost explorer loads agent data, THEN the agent row shows two utilization progress bars (
5hand7d), an estimated cost marked as~$X.XX est., and a "subscription" badge. No dollar spend column is shown. -
GIVEN an agent using an Anthropic API key, WHEN the cost explorer loads, THEN the agent row shows
$X.XXactual spend with a budget gauge if a policy exists. No utilization bars are shown. -
GIVEN a Claude subscription agent whose 5-hour window is at 91% utilization, WHEN the scheduler attempts to process a queued wakeup for that agent, THEN the wakeup is deferred (not failed) until the window resets. An activity log entry records the deferral. When the window resets, the wakeup is re-queued and processed normally.
-
GIVEN a Claude OAuth token that has expired, WHEN the usage client fetches utilization, THEN the token is refreshed via the OAuth refresh endpoint, the new token is written back to
~/.claude/.credentials.json, and the utilization fetch succeeds. The caller is unaware the refresh happened. -
GIVEN a subscription agent crossing 80% utilization on the 7-day window, WHEN the usage cache is updated, THEN an inbox notification is created once. Subsequent checks that still show 80%+ do not create duplicate notifications.
-
GIVEN the LiteLLM pricing fetch fails at startup, WHEN a cost event is recorded, THEN the system falls back to the hardcoded pricing table. No errors are surfaced to users.
Out of scope
- Real-time (sub-second) utilization polling — provider APIs are not real-time and polling aggressively would itself consume quota.
- Utilization tracking for providers without a public usage API (Gemini, Copilot, Amp, etc.) — those agents remain API key billing type with dollar costs only.
- Billing integration or actual charge reconciliation — all costs remain estimates.
- Per-model quota breakdown for Codex — the Codex usage API only returns two aggregate percentages.
- Automatic token refresh for Codex — the Codex auth.json token refresh flow is not publicly documented; if the token is expired the fetch returns an error and the check is skipped (fail-open).