T3 Code: a draft can retry its first send after a failed bootstrap

PR #8226, branch fix/draft-retry-after-bootstrap-failure, head 1c06a8831 (two commits on current upstream/main). Production +175/−12, tests +505/−10, all in apps/server. Open for review, on hold, not merged. First written 2026-08-23, updated 2026-08-26.

The bug

Sending the first message on a new thread creates the thread and starts the turn in one server-side bootstrap. When that bootstrap fails partway (worktree prep on a repo with no commits, a setup script, a dropped connection), the server rolls back with thread.delete. That is a soft delete: the row stays with deletedAt set. The client draft keeps its client-minted thread id, so the retry sends thread.create with the same id. requireThreadAbsent found the tombstone and refused:

Orchestration command invariant failed (thread.create): Thread '…' already exists and cannot be created twice.

Every retry failed the same way until the draft was abandoned. Tracked as pingdotgg/t3code#4647 (closed by #7664, which only covers the case where the server manages to report the deletion to the client) and #5721 (still open). Three earlier server fixes (#4184, #6041, #7608) were closed in favour of orchestrator V2.

The fix

Files: commandInvariants.ts, ProjectionPipeline.ts, ws.ts, OrchestrationEventStore.ts (service + layer), ProjectionPendingApprovals.ts (service + layer, new deleteByThreadId), plus tests.

Verification

CheckResult
vp run typecheck (apps/server)0 errors
vp lint, vp fmt --check on changed filesclean
orchestration/, persistence/, server.test.ts, reaper, startup-reconcile48 files, 442 / 442 pass
server.test.ts bootstrap + drain-gate subset10 / 10, three consecutive runs, no flake

Mutation check. Each fix was reverted to upstream one at a time and its tests re-run.

RevertedOutcome
commandInvariants.ts1 test fails (soft-deleted id rejected again)
ProjectionPipeline.ts2 tests fail (stale rows survive; attachment wiped on replay)
ws.ts1 test fails (thread.create no longer waits for cleanup)

Real engine, throwaway test (not committed). Five create → delete cycles on one id, a sixth create, then turn.start. Shell title "attempt 6", exactly one message, six thread.created events in the log, and a seventh create while live still rejected with "cannot be created twice". Pass.

Codex adversarial review. Four findings against the first draft. Two were real and are fixed: the deletion reactor's check-then-stop race (replaced by the drain gate) and the attachment wipe on replay. Two were scoped out as unreachable from the draft-retry path because the failed incarnation never starts a turn: a stale provider resume cursor and a reused turn-zero checkpoint ref. Both are pre-existing delete leaks worth their own change.

Codex black-box, real engine + SQLite. 8 / 8: delete and recreate, pending approval reset, proposed plan reset, three incarnations, live duplicate still rejected, archived id still rejected, full replay, incremental replay.

Live app, pass 1 (computer use)

Isolated dev server, a git repo with no commits, project added through the UI, draft set to "New worktree".

Step 4. First send. git worktree add fails because the repo has no commits. Server rolls the thread back.
Step 4. First send. git worktree add fails because the repo has no commits. Server rolls the thread back.
Step 5. Retry from the same draft. Same git error, no
Step 5. Retry from the same draft. Same git error, no "already exists" banner.
Step 6. Third retry. Still only the git error.
Step 6. Third retry. Still only the git error.
Step 6. Switched the draft to Current checkout and sent. Thread created, one copy of
Step 6. Switched the draft to Current checkout and sent. Thread created, one copy of "hello", provider replied.
Step 7. Interrupted-bootstrap case: sent, navigated away within ~150 ms, came back. One thread, one message, no duplicate, no banner.
Step 7. Interrupted-bootstrap case: sent, navigated away within ~150 ms, came back. One thread, one message, no duplicate, no banner.

Caveat: the event log for this pass showed each retry used a fresh thread id (53a4 → fe10 → db57 → 02ea → 71d4). That is #7664's client-side rotation, which fires when the server reports the deletion. This pass proves the UI flow, not the same-id server path.

Live app, pass 2 (same id forced)

The client's id rotation was stubbed out for the run (uncommitted, reverted after) to simulate the case where the server's deletion report never reaches the client. Retries then reuse the id and hit the server path under test. The event log for thread c3c55857:

seq 41 thread.created   c3c55857
seq 42 thread.deleted   c3c55857
seq 43 thread.created   c3c55857
seq 44 thread.deleted   c3c55857
… (nine create → delete cycles, same id)
seq 57 thread.created   c3c55857
seq 58 thread.deleted   c3c55857

Server log for the whole session: 0 occurrences of "cannot be created twice", 0 reactor cleanup failures. Final projection for that id: title "retry same id", soft-deleted, 0 message rows. The browser agent's own narration for this pass is unreliable (Chrome's automation lease expired mid-run), so the database is the evidence here.

Not in this change

Review fix and live re-test on the final head (added 2026-08-26)

Macroscope flagged one High on the PR: the drain gate only waited for the deletion reactor's queue, while thread.deleted reaches that queue through an asynchronous subscriber, so a retry landing between publish and enqueue could still be followed by the old cleanup. Fixed in 1c06a8831: the reactor tracks the highest event sequence its subscriber has handed on, and drain first waits for that to reach the engine's latestSequence. The new reactor test fails against the queue-only drain and passes with the barrier. Thread resolved. CI on this head: Check, Test, Test Server 1-3, Release Smoke, Rust, Macroscope Correctness and Conventions all green. 453/453 locally.

Live app re-run on 1c06a8831, driven by a Codex computer-use subagent through the Chrome plugin, with the client's id rotation stubbed out so retries reuse the id (uncommitted, reverted after). Event log for thread a049cbfd:

seq 2  thread.created   a049cbfd
seq 3  thread.deleted   a049cbfd
seq 4  thread.created   a049cbfd
seq 5  thread.deleted   a049cbfd
seq 6  thread.created   a049cbfd
seq 7  thread.deleted   a049cbfd
seq 8  thread.created   a049cbfd
seq 9  thread.deleted   a049cbfd
seq 10 thread.created   a049cbfd
seq 11 thread.message-sent

Five creates, four rollbacks, same id, then live with one user message after switching to Current checkout. Server log: 0 occurrences of "cannot be created twice".

Retry from the same draft on the final head. Only the git worktree error, no invariant banner.
Retry from the same draft on the final head. Only the git worktree error, no invariant banner.
Switched to Current checkout. Thread opens on the same id with one message and a provider reply.
Switched to Current checkout. Thread opens on the same id with one message and a provider reply.

Status

Fixed and verified on the final head. PR #8226 is open for review with green CI and no unresolved threads. It stays unmerged until the release hold lifts.