lucas/omp-deck

Watch

Fork 0

Account for usage quota in Forgejo scheduler #238

New issue

Closed

opened 2026-11-09 10:59:42 +01:00 by lucas · 1 comment

lucas commented

2026-11-09 10:59:42 +01:00

Owner

Problem

The Forgejo scheduler needs a way to monitor quotas on model providers and halt or delay triggers when quota would be exceeded. Currently, it blindly starts the turn, which then fails and leaves a non-descriptive error comment.

Ideally, it would also create a temporary comment informing the user of that state. This comment would then be removed, once the scheduler does start the turn.

Credits must not count toward the quota. The regular 5-hour/weekly quota must be below a threshold (e.g. 90%) for the scheduler to run turns.

What exists today

ForgejoDispatcher (apps/server/src/forgejo/dispatcher.ts) is the only scheduler: pump() (:474) picks artifacts, drain(key) (:495) walks that artifact's pending rows oldest-first, gate(row) (:533) returns Ready | "deferred" | null, and runReady/runIntake start the turn. Nothing in the deck reads provider usage, classifies a 429, deletes a Forgejo comment, or holds a trigger for anything but the 15s checkbox settle window and the implementer intake-slot limit.

The data does exist in the SDK, unused: AuthStorage.usage.reports({provider, signal}) returns one UsageReport per stored account, each with limits: UsageLimit[]. For Anthropic those are anthropic:5h, anthropic:7d, anthropic:7d:opus, anthropic:7d:sonnet (all with a recurring scope.windowId) plus anthropic:extra ("Claude Extra Usage", scope.windowId: "extra") — the paid credit bucket that must not count. Reports are cached for 5 minutes (USAGE_REPORT_TTL_MS), so gating per trigger adds no upstream traffic.

Plan

1. Quota reader (`apps/server/src/forgejo/quota.ts`, new)

No equivalent exists; this is the only new file.

Exports:

/** Whether scheduled Forgejo turns may start, given the default model provider's usage. */
export type QuotaVerdict =
	| { kind: "ok" }
	| { kind: "over"; usedPercent: number; thresholdPercent: number; measuredAt: number; resetsAt?: number }
	| { kind: "unreadable"; measuredAt: number };

export const QUOTA_THRESHOLD_VAR = "OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT";
export const DEFAULT_QUOTA_THRESHOLD_PERCENT = 90;

/** Threshold in percent, read live from the environment; out-of-range or unparsable falls back to the default. */
export function quotaThresholdPercent(): number;

export async function readForgejoQuota(): Promise<QuotaVerdict>;

quotaThresholdPercent() parses process.env[QUOTA_THRESHOLD_VAR] with Number.parseInt(…, 10), accepts 1..100, otherwise returns DEFAULT_QUOTA_THRESHOLD_PERCENT. Read per call: applyManagedEnvUpdatesToProcess (apps/server/src/env-store.ts:74-87) writes Settings → Environment edits straight into process.env, so no restart and no caching.

readForgejoQuota() steps, in order:

Resolve the provider whose quota the scheduler will spend. Dispatcher-created sessions pass no model (bridge.createSession({cwd, trigger, placement})), so they run on the configured default model role:
- const registry = await getDeckModelRegistry() (apps/server/src/auth-singleton.ts:26).
- const settings = await buildDeckSessionSettings(getAgentDir()) — exported at apps/server/src/model-config.ts:568; it is the deck's isolated Settings, not the SDK process singleton.
- const { model } = resolveRoleModelFull(settings, "default", registry.getAvailable("all"), undefined) from @oh-my-pi/pi-coding-agent/session/role-models.
- No model → {kind: "ok"} (nothing to measure, never block).
const auth = registry.authStorage. If auth.usage.providerFor(String(model.provider)) is undefined the provider has no usage concept at all → {kind: "ok"}.
const reports = await auth.usage.reports({ provider: String(model.provider) }) inside try/catch. A throw, or reports === null, → {kind: "unreadable", measuredAt: Date.now()}. reports.length === 0 (no stored account reports for that provider) → {kind: "ok"}.
Per report, keep limits with a recurring window and a readable fraction:
report.limits.filter((l) => l.scope.windowId !== undefined && l.scope.windowId !== "extra"), map through resolveUsedFraction(limit) (exported from @oh-my-pi/pi-ai), drop undefined. An account with no such limit is unmeasurable and is skipped. extra is excluded by id, not by an allowlist of 5h/7d, so a future window id (anthropic:7d:haiku) is gated automatically while the credit bucket stays out.
If every account was skipped → {kind: "unreadable", measuredAt: Date.now()} (a report arrived but says nothing usable).
Each account's load is the maximum of its fractions (a full weekly blocks even with a fresh 5h window). The deck's load is the minimum across accounts, because the SDK rotates to whichever account still has headroom — if any account is under the threshold the turn can run.
usedPercent = Math.round(best * 100). usedPercent < thresholdPercent → {kind: "ok"}.
Otherwise {kind: "over", usedPercent, thresholdPercent, measuredAt: Date.now(), resetsAt} where resetsAt is the latest limit.window?.resetsAt among the best account's limits whose own percent is at or above the threshold — all of them must roll over before that account drops under. Omitted when any offending window reports no resetsAt.

2. Threshold env entry (`apps/server/src/env-schema.ts`)

Append one ENV_SCHEMA entry next to the other dispatcher-facing vars, following the OMP_DECK_MAINTENANCE_GATE_* shape:

{
	key: "OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT",
	defaultValue: "90",
	valueType: "int",
	sensitive: false,
	restartRequired: false,
	hotApply: true,
	description:
		"Forgejo scheduler holds triggers while the default model provider's 5-hour/weekly usage is at or above this percentage. Paid extra-usage credits are not counted. 100 holds only a fully exhausted account.",
},

No applyHotUpdates branch: the value is read from process.env at each gate, exactly like OMP_MODEL and PI_NO_TITLE.

3. Persist the notice comment id

New migration apps/server/src/db/migrations/025-forgejo-trigger-quota-notice.sql (024 is the highest today), in the style of the existing files:

-- Id of the temporary "holding this event for usage quota" comment omp-deck
-- posted on the thread while a trigger is quota-deferred. Set when the notice
-- is posted, cleared (and the comment deleted) when the row leaves the hold.
-- NULL for rows that were never held and for watched-branch rows (number 0),
-- which have no thread.
ALTER TABLE forgejo_triggers ADD COLUMN quota_notice_comment_id INTEGER;

In apps/server/src/db/forgejo-triggers.ts:

Row gains quota_notice_comment_id: number | null.
StoredForgejoTrigger gains quotaNoticeCommentId: number | null; toStored maps it. Deliberately not on ForgejoTriggerSummary (@omp-deck/protocol) — it is server-internal and must not widen the wire type.
New setter beside setForgejoTriggerSession (:192):

/** Record (or clear, with null) the temporary quota-hold comment posted for `triggerId`. */
export function setForgejoTriggerQuotaNotice(triggerId: string, commentId: number | null): void

4. Comment post/delete on the Forgejo API (`apps/server/src/forgejo/dispatch-api.ts`)

Change postComment to return the created comment's id: Promise<ApiResult<{ id: number }>>. Forgejo's POST …/issues/{n}/comments returns the comment object; the implementation at :351 already client.jsons it, so only the type parameter and the return mapping change. Update the existing call sites in dispatcher.ts (postNotice :2569, surfaceFailure :2596) and anywhere else grep -n "postComment(" apps/server/src reports — they ignore the payload and keep compiling, but the interface change must be applied to the fake in apps/server/src/forgejo/dispatcher.test.ts.
Add:

/** Delete issue comment `commentId`; a 404 counts as success (already gone). */
deleteComment(owner: string, repo: string, commentId: number, signal?: AbortSignal): Promise<ApiResult<void>>;

implemented as client.json<void>("DELETE", repoApiPath(owner, repo, "issues", "comments", String(commentId)), { signal }), mapping status === 404 to {ok: true, data: undefined}. ForgejoClient.json already treats an empty 204 body as success (client.ts:214).

5. The gate (`apps/server/src/forgejo/dispatcher.ts`)

Dependency and state. Add to ForgejoDispatcherDeps (:220):

/** Current provider usage verdict. Default: quota.ts `readForgejoQuota`. */
quota?: () => Promise<QuotaVerdict>;

Add fields beside deferred/settleTimers (:412-414):

/** Rows deferred by the quota gate; a subset of `deferred`, keyed by row id. */
private readonly quotaDeferred = new Map<string, StoredForgejoTrigger>();
private quotaTimer: ReturnType<typeof setTimeout> | undefined;

and a module constant next to CHECKBOX_SETTLE_MS (:204):

/** Longest a quota hold waits before re-reading usage, even when the provider reports a later reset. */
export const QUOTA_RECHECK_MAX_MS = 30 * 60_000;

Check placement. Insert the quota check in gate() immediately before the intake-capacity block at :784 (if (implementation !== null) {), i.e. after routing, addressed-to-agent detection, write-access, workspace matching and PR-head resolution, and before any intake slot is taken. Rationale, both load-bearing: a row that would be dropped anyway must never post a quota notice, and a quota-held row must not sit on an intake slot.

const quota = await (this.deps.quota ?? readForgejoQuota)();
if (quota.kind !== "ok") {
	await this.holdForQuota(job, quota);
	return "deferred";
}

holdForQuota(job, verdict) (new private method):

If job.row.number === 0 (watched-branch CI, no thread) post nothing.
Else if job.row.quotaNoticeCommentId === null, post the body for verdict.kind via job.api.postComment and, on success, setForgejoTriggerQuotaNotice(job.row.id, posted.data.id). A failed post is a log.warn only — the hold still applies. The notice is posted once and never rewritten while the row stays held, so its body names the measurement time rather than implying live data.
Register the hold and arm the shared timer: this.quotaDeferred.set(row.id, row); this.armQuotaTimer(verdict).

Bodies (exact strings, module constants next to the other notice texts):

const QUOTA_HOLD_NOTICE = (usedPercent: number, thresholdPercent: number, measuredAt: string, resetsAt: string | null) =>
	`omp-deck is holding this event: the model provider's usage was at ${usedPercent}% as of ${measuredAt}, at or above the ${thresholdPercent}% ceiling omp-deck keeps for scheduled work.` +
	(resetsAt ? ` The quota window resets at ${resetsAt}.` : "") +
	` The turn starts automatically once usage drops, and this comment is removed then.`;

const QUOTA_UNREADABLE_NOTICE =
	"omp-deck is holding this event: the model provider's usage quota could not be read, so omp-deck is not starting the turn. " +
	"It retries automatically and removes this comment once the turn starts.";

Timestamps are ISO-8601 UTC (new Date(ms).toISOString()). Neither body names an account, an email, a model, or a raw provider error: the repository may be public.

armQuotaTimer(verdict) (new private method): compute delay = Math.max(1_000, Math.min(verdict.kind === "over" && verdict.resetsAt ? verdict.resetsAt - Date.now() : Number.POSITIVE_INFINITY, QUOTA_RECHECK_MAX_MS)). If this.quotaTimer is already armed, leave it (one shared timer serves every held row; quota is global). On fire: clear quotaTimer, and for every row in quotaDeferred remove it from this.deferred and this.gated, clear quotaDeferred, then this.pump(). The hold is indefinite — a row that is still over quota simply re-defers and re-arms.

dispose() (:450) must clearTimeout(this.quotaTimer) alongside the existing sweep and settle timers.

Releasing the notice. In gateSafely (:513), after gate(row) settles — including the catch path that calls failInternal — delete the notice whenever the result is not "deferred":

private async clearQuotaNotice(row: StoredForgejoTrigger): Promise<void>

reads row.quotaNoticeCommentId, returns when null, calls api.deleteComment(owner, repo, id) (a log.warn on failure, never a trigger failure) and setForgejoTriggerQuotaNotice(row.id, null). One site covers every exit: the turn starting, the row being dropped, and the row failing. drain re-reads rows from listPendingForgejoTriggers() on each iteration, so the row handed to gateSafely always carries the id written by the previous pass.

6. Keep turn-free events moving while an artifact is held

Quota holds can last hours, and pump() skips any artifact that has a deferred row (:480-482), so without this step a closed event queued behind a held comment row would never archive its session, and pushed/status rows would never enqueue their follow-ups. These actions consume no quota: gate() dispatches them at :535-548 to handleLifecycle, handleAbandon, handleBasePush, handleCommitStatus and returns null.

Add a module constant:

/** Actions `gate()` completes without starting a turn, so a quota hold must not block them. */
const TURN_FREE_ACTIONS = new Set(["closed", "reopened", "implementation_abandoned", "pushed", "status"]);

Replace the row-picking logic in drain and hasUngated with one helper so both agree (a disagreement makes pump()'s finally re-pump forever):

/** The next row of artifact `key` that `drain` may take; only turn-free rows while the artifact is quota-held. */
private nextDrainable(key: string): StoredForgejoTrigger | undefined {
	const held = [...this.quotaDeferred.values()].some((r) => artifactKey(r) === key);
	const rows = listPendingForgejoTriggers().filter((r) => artifactKey(r) === key && !this.gated.has(r.id));
	return held ? rows.find((r) => TURN_FREE_ACTIONS.has(r.action)) : rows[0];
}

hasUngated(key) (:470) becomes this.nextDrainable(key) !== undefined.
drain (:495) takes const row = this.nextDrainable(key) instead of its inline find, and on "deferred" continues the loop instead of returning when the row is in quotaDeferred (the hold is already registered by holdForQuota, so the next nextDrainable call restricts itself to turn-free rows); a non-quota deferral (checkbox settle, intake capacity) keeps today's return.
pump (:480) computes blocked from non-quota deferrals only: new Set([...this.deferred.values()].filter((r) => !this.quotaDeferred.has(r.id)).map(artifactKey)), so a newly arrived lifecycle event on a held artifact still gets drained.

Consequence to accept: a closed processed during a hold archives the session the held turn would have used; resolveSession then creates a fresh one when the turn finally runs. That is the same outcome as a close arriving while a turn is queued today.

Not in scope: startImplementer (:2632), the deck-UI implementer start, stays ungated — it is user-initiated, not scheduler-initiated. Mid-turn quota exhaustion also stays as today's generic agent failure comment.

Critical files & anchors

apps/server/src/forgejo/dispatcher.ts — pump :474, drain :495, gateSafely :513, gate :533, intake-capacity block :784 (insertion point), settleCheckboxBurst :820 (the deferral pattern to copy), dispose :450, postNotice :2560.
apps/server/src/db/forgejo-triggers.ts — Row/toStored :32-69, setForgejoTriggerSession :192 (setter shape), listPendingForgejoTriggers :175.
apps/server/src/forgejo/dispatch-api.ts — DispatchApi interface :15-95, postComment impl :351.
apps/server/src/model-config.ts — buildDeckSessionSettings :568, the isolated Settings the quota reader must use.
apps/server/src/env-schema.ts — ENV_SCHEMA entries and validateEnvValue :349.

Verification

Working directory apps/server unless noted.

Unit — quota reader. New apps/server/src/forgejo/quota.test.ts exercising the pure scoring on synthetic UsageReport[] (factor the step 4-8 computation into an exported scoreUsageReports(reports, thresholdPercent, nowMs): QuotaVerdict so it is testable without auth storage):
- one account, anthropic:5h at 0.5 and anthropic:extra at 0.99 → ok (credits excluded);
- one account, anthropic:7d at 0.94, threshold 90 → over with usedPercent: 94 and resetsAt from that limit's window;
- two accounts, one at 0.99 and one at 0.40 → ok (minimum across accounts);
- a report whose only limits are extra → unreadable.
  Run: bun test src/forgejo/quota.test.ts.
Unit — dispatcher hold/release. Extend apps/server/src/forgejo/dispatcher.test.ts using its existing harness (fake api, deps overrides) with deps.quota returning over then ok:
- first pump: the trigger stays pending, the fake API recorded one postComment whose body contains 94% and 90%, and the row's quotaNoticeCommentId is the posted id;
- after flipping deps.quota to ok and calling wake(): the turn runs, the row reaches done, and the fake API recorded a deleteComment for that id;
- a second pending row on the same artifact with action: "closed" is processed while the hold is active (its session is archived) and the held row still has not run.
  Run: bun test src/forgejo/dispatcher.test.ts.
End-to-end, manual. With the deck running and a Forgejo repo connected: set OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT to 1 in Settings → Environment (no restart), then mention the agent in a comment on an issue of that repo. Expected: no session starts, the trigger shows pending in Settings → Forgejo, and the issue gains the hold comment naming 1%. Set the value back to 90 and either wait out the re-check (≤30 min) or post another comment to force a wake(). Expected: the hold comment disappears and the turn runs.
bun run typecheck at the repo root (the postComment signature change must leave no stale callers).

Assumptions & contingencies

Threshold semantics. 100 means "hold only a fully exhausted account"; there is no separate off switch. If an off switch turns out to be needed, add 0 as "gate disabled" in quotaThresholdPercent() rather than a second variable.
Fail-open vs fail-closed. A provider with no usage endpoint, or no resolvable default model, runs ungated; a usage fetch that errors or returns nothing usable holds the trigger. If a /usage outage proves common enough to freeze the scheduler in practice, change the unreadable verdict to ok in one place (readForgejoQuota's catch) rather than adding a bypass in the dispatcher.
resolveRoleModelFull with no configured role. If it returns no model on a deck that clearly has a default model, resolve the provider instead from registry.getAvailable("all")[0] filtered by hasConfiguredAuth, and keep the "no model → ok" fallback.
Head-of-line blocking. Turn-free actions skip the hold (step 6). If that ordering exception causes trouble, the fallback is to drop step 6 entirely — nextDrainable becomes rows[0], pump's blocked goes back to all deferrals — and accept that an artifact is frozen for the duration of the hold.

## Problem The Forgejo scheduler needs a way to monitor quotas on model providers and halt or delay triggers when quota would be exceeded. Currently, it blindly starts the turn, which then fails and leaves a non-descriptive error comment. Ideally, it would also create a temporary comment informing the user of that state. This comment would then be removed, once the scheduler does start the turn. Credits must not count toward the quota. The regular 5-hour/weekly quota must be below a threshold (e.g. 90%) for the scheduler to run turns. ### What exists today `ForgejoDispatcher` (`apps/server/src/forgejo/dispatcher.ts`) is the only scheduler: `pump()` (:474) picks artifacts, `drain(key)` (:495) walks that artifact's pending rows oldest-first, `gate(row)` (:533) returns `Ready | "deferred" | null`, and `runReady`/`runIntake` start the turn. Nothing in the deck reads provider usage, classifies a 429, deletes a Forgejo comment, or holds a trigger for anything but the 15s checkbox settle window and the implementer intake-slot limit. The data does exist in the SDK, unused: `AuthStorage.usage.reports({provider, signal})` returns one `UsageReport` per stored account, each with `limits: UsageLimit[]`. For Anthropic those are `anthropic:5h`, `anthropic:7d`, `anthropic:7d:opus`, `anthropic:7d:sonnet` (all with a recurring `scope.windowId`) plus `anthropic:extra` ("Claude Extra Usage", `scope.windowId: "extra"`) — the paid credit bucket that must not count. Reports are cached for 5 minutes (`USAGE_REPORT_TTL_MS`), so gating per trigger adds no upstream traffic. ## Plan ### 1. Quota reader (`apps/server/src/forgejo/quota.ts`, new) No equivalent exists; this is the only new file. Exports: ```ts /** Whether scheduled Forgejo turns may start, given the default model provider's usage. */ export type QuotaVerdict = | { kind: "ok" } | { kind: "over"; usedPercent: number; thresholdPercent: number; measuredAt: number; resetsAt?: number } | { kind: "unreadable"; measuredAt: number }; export const QUOTA_THRESHOLD_VAR = "OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT"; export const DEFAULT_QUOTA_THRESHOLD_PERCENT = 90; /** Threshold in percent, read live from the environment; out-of-range or unparsable falls back to the default. */ export function quotaThresholdPercent(): number; export async function readForgejoQuota(): Promise<QuotaVerdict>; ``` `quotaThresholdPercent()` parses `process.env[QUOTA_THRESHOLD_VAR]` with `Number.parseInt(…, 10)`, accepts `1..100`, otherwise returns `DEFAULT_QUOTA_THRESHOLD_PERCENT`. Read per call: `applyManagedEnvUpdatesToProcess` (`apps/server/src/env-store.ts:74-87`) writes Settings → Environment edits straight into `process.env`, so no restart and no caching. `readForgejoQuota()` steps, in order: 1. Resolve the provider whose quota the scheduler will spend. Dispatcher-created sessions pass no `model` (`bridge.createSession({cwd, trigger, placement})`), so they run on the configured `default` model role: - `const registry = await getDeckModelRegistry()` (`apps/server/src/auth-singleton.ts:26`). - `const settings = await buildDeckSessionSettings(getAgentDir())` — exported at `apps/server/src/model-config.ts:568`; it is the deck's isolated `Settings`, not the SDK process singleton. - `const { model } = resolveRoleModelFull(settings, "default", registry.getAvailable("all"), undefined)` from `@oh-my-pi/pi-coding-agent/session/role-models`. - No model → `{kind: "ok"}` (nothing to measure, never block). 2. `const auth = registry.authStorage`. If `auth.usage.providerFor(String(model.provider))` is `undefined` the provider has no usage concept at all → `{kind: "ok"}`. 3. `const reports = await auth.usage.reports({ provider: String(model.provider) })` inside `try`/`catch`. A throw, or `reports === null`, → `{kind: "unreadable", measuredAt: Date.now()}`. `reports.length === 0` (no stored account reports for that provider) → `{kind: "ok"}`. 4. Per report, keep limits with a recurring window and a readable fraction: `report.limits.filter((l) => l.scope.windowId !== undefined && l.scope.windowId !== "extra")`, map through `resolveUsedFraction(limit)` (exported from `@oh-my-pi/pi-ai`), drop `undefined`. An account with no such limit is unmeasurable and is skipped. `extra` is excluded by id, not by an allowlist of `5h`/`7d`, so a future window id (`anthropic:7d:haiku`) is gated automatically while the credit bucket stays out. 5. If every account was skipped → `{kind: "unreadable", measuredAt: Date.now()}` (a report arrived but says nothing usable). 6. Each account's load is the **maximum** of its fractions (a full weekly blocks even with a fresh 5h window). The deck's load is the **minimum** across accounts, because the SDK rotates to whichever account still has headroom — if any account is under the threshold the turn can run. 7. `usedPercent = Math.round(best * 100)`. `usedPercent < thresholdPercent` → `{kind: "ok"}`. 8. Otherwise `{kind: "over", usedPercent, thresholdPercent, measuredAt: Date.now(), resetsAt}` where `resetsAt` is the **latest** `limit.window?.resetsAt` among the best account's limits whose own percent is at or above the threshold — all of them must roll over before that account drops under. Omitted when any offending window reports no `resetsAt`. ### 2. Threshold env entry (`apps/server/src/env-schema.ts`) Append one `ENV_SCHEMA` entry next to the other dispatcher-facing vars, following the `OMP_DECK_MAINTENANCE_GATE_*` shape: ```ts { key: "OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT", defaultValue: "90", valueType: "int", sensitive: false, restartRequired: false, hotApply: true, description: "Forgejo scheduler holds triggers while the default model provider's 5-hour/weekly usage is at or above this percentage. Paid extra-usage credits are not counted. 100 holds only a fully exhausted account.", }, ``` No `applyHotUpdates` branch: the value is read from `process.env` at each gate, exactly like `OMP_MODEL` and `PI_NO_TITLE`. ### 3. Persist the notice comment id New migration `apps/server/src/db/migrations/025-forgejo-trigger-quota-notice.sql` (024 is the highest today), in the style of the existing files: ```sql -- Id of the temporary "holding this event for usage quota" comment omp-deck -- posted on the thread while a trigger is quota-deferred. Set when the notice -- is posted, cleared (and the comment deleted) when the row leaves the hold. -- NULL for rows that were never held and for watched-branch rows (number 0), -- which have no thread. ALTER TABLE forgejo_triggers ADD COLUMN quota_notice_comment_id INTEGER; ``` In `apps/server/src/db/forgejo-triggers.ts`: - `Row` gains `quota_notice_comment_id: number | null`. - `StoredForgejoTrigger` gains `quotaNoticeCommentId: number | null`; `toStored` maps it. Deliberately **not** on `ForgejoTriggerSummary` (`@omp-deck/protocol`) — it is server-internal and must not widen the wire type. - New setter beside `setForgejoTriggerSession` (:192): ```ts /** Record (or clear, with null) the temporary quota-hold comment posted for `triggerId`. */ export function setForgejoTriggerQuotaNotice(triggerId: string, commentId: number | null): void ``` ### 4. Comment post/delete on the Forgejo API (`apps/server/src/forgejo/dispatch-api.ts`) - Change `postComment` to return the created comment's id: `Promise<ApiResult<{ id: number }>>`. Forgejo's `POST …/issues/{n}/comments` returns the comment object; the implementation at `:351` already `client.json`s it, so only the type parameter and the return mapping change. Update the existing call sites in `dispatcher.ts` (`postNotice` :2569, `surfaceFailure` :2596) and anywhere else `grep -n "postComment(" apps/server/src` reports — they ignore the payload and keep compiling, but the interface change must be applied to the fake in `apps/server/src/forgejo/dispatcher.test.ts`. - Add: ```ts /** Delete issue comment `commentId`; a 404 counts as success (already gone). */ deleteComment(owner: string, repo: string, commentId: number, signal?: AbortSignal): Promise<ApiResult<void>>; ``` implemented as `client.json<void>("DELETE", repoApiPath(owner, repo, "issues", "comments", String(commentId)), { signal })`, mapping `status === 404` to `{ok: true, data: undefined}`. `ForgejoClient.json` already treats an empty 204 body as success (`client.ts:214`). ### 5. The gate (`apps/server/src/forgejo/dispatcher.ts`) **Dependency and state.** Add to `ForgejoDispatcherDeps` (:220): ```ts /** Current provider usage verdict. Default: quota.ts `readForgejoQuota`. */ quota?: () => Promise<QuotaVerdict>; ``` Add fields beside `deferred`/`settleTimers` (:412-414): ```ts /** Rows deferred by the quota gate; a subset of `deferred`, keyed by row id. */ private readonly quotaDeferred = new Map<string, StoredForgejoTrigger>(); private quotaTimer: ReturnType<typeof setTimeout> | undefined; ``` and a module constant next to `CHECKBOX_SETTLE_MS` (:204): ```ts /** Longest a quota hold waits before re-reading usage, even when the provider reports a later reset. */ export const QUOTA_RECHECK_MAX_MS = 30 * 60_000; ``` **Check placement.** Insert the quota check in `gate()` immediately before the intake-capacity block at `:784` (`if (implementation !== null) {`), i.e. after routing, addressed-to-agent detection, write-access, workspace matching and PR-head resolution, and before any intake slot is taken. Rationale, both load-bearing: a row that would be dropped anyway must never post a quota notice, and a quota-held row must not sit on an intake slot. ```ts const quota = await (this.deps.quota ?? readForgejoQuota)(); if (quota.kind !== "ok") { await this.holdForQuota(job, quota); return "deferred"; } ``` **`holdForQuota(job, verdict)`** (new private method): - If `job.row.number === 0` (watched-branch CI, no thread) post nothing. - Else if `job.row.quotaNoticeCommentId === null`, post the body for `verdict.kind` via `job.api.postComment` and, on success, `setForgejoTriggerQuotaNotice(job.row.id, posted.data.id)`. A failed post is a `log.warn` only — the hold still applies. The notice is posted once and never rewritten while the row stays held, so its body names the measurement time rather than implying live data. - Register the hold and arm the shared timer: `this.quotaDeferred.set(row.id, row); this.armQuotaTimer(verdict)`. Bodies (exact strings, module constants next to the other notice texts): ```ts const QUOTA_HOLD_NOTICE = (usedPercent: number, thresholdPercent: number, measuredAt: string, resetsAt: string | null) => `omp-deck is holding this event: the model provider's usage was at ${usedPercent}% as of ${measuredAt}, at or above the ${thresholdPercent}% ceiling omp-deck keeps for scheduled work.` + (resetsAt ? ` The quota window resets at ${resetsAt}.` : "") + ` The turn starts automatically once usage drops, and this comment is removed then.`; const QUOTA_UNREADABLE_NOTICE = "omp-deck is holding this event: the model provider's usage quota could not be read, so omp-deck is not starting the turn. " + "It retries automatically and removes this comment once the turn starts."; ``` Timestamps are ISO-8601 UTC (`new Date(ms).toISOString()`). Neither body names an account, an email, a model, or a raw provider error: the repository may be public. **`armQuotaTimer(verdict)`** (new private method): compute `delay = Math.max(1_000, Math.min(verdict.kind === "over" && verdict.resetsAt ? verdict.resetsAt - Date.now() : Number.POSITIVE_INFINITY, QUOTA_RECHECK_MAX_MS))`. If `this.quotaTimer` is already armed, leave it (one shared timer serves every held row; quota is global). On fire: clear `quotaTimer`, and for every row in `quotaDeferred` remove it from `this.deferred` and `this.gated`, clear `quotaDeferred`, then `this.pump()`. The hold is indefinite — a row that is still over quota simply re-defers and re-arms. `dispose()` (:450) must `clearTimeout(this.quotaTimer)` alongside the existing sweep and settle timers. **Releasing the notice.** In `gateSafely` (:513), after `gate(row)` settles — including the `catch` path that calls `failInternal` — delete the notice whenever the result is not `"deferred"`: ```ts private async clearQuotaNotice(row: StoredForgejoTrigger): Promise<void> ``` reads `row.quotaNoticeCommentId`, returns when null, calls `api.deleteComment(owner, repo, id)` (a `log.warn` on failure, never a trigger failure) and `setForgejoTriggerQuotaNotice(row.id, null)`. One site covers every exit: the turn starting, the row being dropped, and the row failing. `drain` re-reads rows from `listPendingForgejoTriggers()` on each iteration, so the row handed to `gateSafely` always carries the id written by the previous pass. ### 6. Keep turn-free events moving while an artifact is held Quota holds can last hours, and `pump()` skips any artifact that has a deferred row (`:480-482`), so without this step a `closed` event queued behind a held comment row would never archive its session, and `pushed`/`status` rows would never enqueue their follow-ups. These actions consume no quota: `gate()` dispatches them at `:535-548` to `handleLifecycle`, `handleAbandon`, `handleBasePush`, `handleCommitStatus` and returns `null`. Add a module constant: ```ts /** Actions `gate()` completes without starting a turn, so a quota hold must not block them. */ const TURN_FREE_ACTIONS = new Set(["closed", "reopened", "implementation_abandoned", "pushed", "status"]); ``` Replace the row-picking logic in `drain` and `hasUngated` with one helper so both agree (a disagreement makes `pump()`'s `finally` re-pump forever): ```ts /** The next row of artifact `key` that `drain` may take; only turn-free rows while the artifact is quota-held. */ private nextDrainable(key: string): StoredForgejoTrigger | undefined { const held = [...this.quotaDeferred.values()].some((r) => artifactKey(r) === key); const rows = listPendingForgejoTriggers().filter((r) => artifactKey(r) === key && !this.gated.has(r.id)); return held ? rows.find((r) => TURN_FREE_ACTIONS.has(r.action)) : rows[0]; } ``` - `hasUngated(key)` (:470) becomes `this.nextDrainable(key) !== undefined`. - `drain` (:495) takes `const row = this.nextDrainable(key)` instead of its inline `find`, and on `"deferred"` continues the loop instead of returning **when the row is in `quotaDeferred`** (the hold is already registered by `holdForQuota`, so the next `nextDrainable` call restricts itself to turn-free rows); a non-quota deferral (checkbox settle, intake capacity) keeps today's `return`. - `pump` (:480) computes `blocked` from non-quota deferrals only: `new Set([...this.deferred.values()].filter((r) => !this.quotaDeferred.has(r.id)).map(artifactKey))`, so a newly arrived lifecycle event on a held artifact still gets drained. Consequence to accept: a `closed` processed during a hold archives the session the held turn would have used; `resolveSession` then creates a fresh one when the turn finally runs. That is the same outcome as a close arriving while a turn is queued today. Not in scope: `startImplementer` (:2632), the deck-UI implementer start, stays ungated — it is user-initiated, not scheduler-initiated. Mid-turn quota exhaustion also stays as today's generic `agent` failure comment. ## Critical files & anchors - `apps/server/src/forgejo/dispatcher.ts` — `pump` :474, `drain` :495, `gateSafely` :513, `gate` :533, intake-capacity block :784 (insertion point), `settleCheckboxBurst` :820 (the deferral pattern to copy), `dispose` :450, `postNotice` :2560. - `apps/server/src/db/forgejo-triggers.ts` — `Row`/`toStored` :32-69, `setForgejoTriggerSession` :192 (setter shape), `listPendingForgejoTriggers` :175. - `apps/server/src/forgejo/dispatch-api.ts` — `DispatchApi` interface :15-95, `postComment` impl :351. - `apps/server/src/model-config.ts` — `buildDeckSessionSettings` :568, the isolated `Settings` the quota reader must use. - `apps/server/src/env-schema.ts` — `ENV_SCHEMA` entries and `validateEnvValue` :349. ## Verification Working directory `apps/server` unless noted. 1. **Unit — quota reader.** New `apps/server/src/forgejo/quota.test.ts` exercising the pure scoring on synthetic `UsageReport[]` (factor the step 4-8 computation into an exported `scoreUsageReports(reports, thresholdPercent, nowMs): QuotaVerdict` so it is testable without auth storage): - one account, `anthropic:5h` at 0.5 and `anthropic:extra` at 0.99 → `ok` (credits excluded); - one account, `anthropic:7d` at 0.94, threshold 90 → `over` with `usedPercent: 94` and `resetsAt` from that limit's window; - two accounts, one at 0.99 and one at 0.40 → `ok` (minimum across accounts); - a report whose only limits are `extra` → `unreadable`. Run: `bun test src/forgejo/quota.test.ts`. 2. **Unit — dispatcher hold/release.** Extend `apps/server/src/forgejo/dispatcher.test.ts` using its existing harness (fake `api`, `deps` overrides) with `deps.quota` returning `over` then `ok`: - first pump: the trigger stays `pending`, the fake API recorded one `postComment` whose body contains `94%` and `90%`, and the row's `quotaNoticeCommentId` is the posted id; - after flipping `deps.quota` to `ok` and calling `wake()`: the turn runs, the row reaches `done`, and the fake API recorded a `deleteComment` for that id; - a second pending row on the same artifact with `action: "closed"` is processed while the hold is active (its session is archived) and the held row still has not run. Run: `bun test src/forgejo/dispatcher.test.ts`. 3. **End-to-end, manual.** With the deck running and a Forgejo repo connected: set `OMP_DECK_FORGEJO_QUOTA_MAX_PERCENT` to `1` in Settings → Environment (no restart), then mention the agent in a comment on an issue of that repo. Expected: no session starts, the trigger shows `pending` in Settings → Forgejo, and the issue gains the hold comment naming `1%`. Set the value back to `90` and either wait out the re-check (≤30 min) or post another comment to force a `wake()`. Expected: the hold comment disappears and the turn runs. 4. `bun run typecheck` at the repo root (the `postComment` signature change must leave no stale callers). ## Assumptions & contingencies - **Threshold semantics.** `100` means "hold only a fully exhausted account"; there is no separate off switch. If an off switch turns out to be needed, add `0` as "gate disabled" in `quotaThresholdPercent()` rather than a second variable. - **Fail-open vs fail-closed.** A provider with no usage endpoint, or no resolvable default model, runs ungated; a usage fetch that errors or returns nothing usable holds the trigger. If a `/usage` outage proves common enough to freeze the scheduler in practice, change the `unreadable` verdict to `ok` in one place (`readForgejoQuota`'s catch) rather than adding a bypass in the dispatcher. - **`resolveRoleModelFull` with no configured role.** If it returns no model on a deck that clearly has a default model, resolve the provider instead from `registry.getAvailable("all")[0]` filtered by `hasConfiguredAuth`, and keep the "no model → ok" fallback. - **Head-of-line blocking.** Turn-free actions skip the hold (step 6). If that ordering exception causes trouble, the fallback is to drop step 6 entirely — `nextDrainable` becomes `rows[0]`, `pump`'s `blocked` goes back to all deferrals — and accept that an artifact is frozen for the duration of the hold.

lucas added the

Kind

Enhancement

label

2026-11-09 10:59:42 +01:00

lucas commented

2026-11-09 14:01:33 +01:00

Author

Owner

Also make sure that credits don't count toward the quota. The regular 5-hour/weekly quota must be below a threshold (e.g. 90%) for the scheduler to run turns.