> ## Documentation Index
> Fetch the complete documentation index at: https://dev.haico.gr/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Conversation branching

> The turn-as-commit model: branching, restore, checkpointer seeding, and the trajectory graph.

# Conversation branching & history graph

This document explains how HAICO's conversation branching, restore, and history-graph features work as currently implemented (issue #55). It is the practical companion to [ADR-0003](/docs/adr/0003-conversation-branching-via-new-thread-and-state-seeding), which records *why* the design was chosen; here we describe *how* the pieces fit together so a contributor can extend or debug them.

## The problem in one paragraph

A co-construction session used to be a straight line: only the latest workspace survived, and there was no way to see how an artifact evolved, fork an alternative direction, or return to an earlier state. The difficulty is that a conversation's state lives in **two independent stores**. The LangGraph checkpointer versions the chat transcript (the `messages` channel) keyed by `thread_id`. The four user-visible surfaces, the document, preferences, todos, and artifacts, are ordinary SQL rows, also keyed by `thread_id`, with no version axis (`documents` is even `unique(thread_id)`). Forking only the transcript would leave two branches sharing one workspace, silently corrupting both. The feature therefore versions the **workspace**, not just the transcript, and the transcript fork is the one axis the existing checkpointer already gives us for free per thread.

## Core concepts

**Turn as commit.** A turn is HAICO's coarse clock: one tick per user message, stored as `conversation_users.turn_index` and incremented just before the agent runs. Every turn already writes an immutable `workspace_snapshots` row keyed by `(thread_id, turn_index)`, capturing the objective, document, preferences, and todos (each with its nesting `depth`) *at the start of that turn*. A turn therefore behaves like a commit: an addressable, immutable point with a parent (the previous turn) and a captured workspace tree.

**Snapshot timing: read this carefully.** Snapshot *N* is captured *before* the agent acts on turn *N*. So the workspace **after** turn *k* is not snapshot *k*; it is snapshot *k+1* (the state the next turn started from, which also absorbs any manual edits the user made in between). When *k* is the latest completed turn there is no snapshot *k+1* yet, and the **live** workspace rows are authoritative. This `snapshot(k+1)`-or-live rule is the single most important invariant in the codebase; it appears in `resolve_restore_state` and in the graph builder, and getting it wrong is the most likely source of off-by-one bugs.

**Branch as a new thread.** A branch is not a column on existing rows; it is a brand-new `thread_id` (`th_<uuid4hex>`, the same format the rest of the app mints) whose workspace is copied as-of the branch point and whose parent link is recorded in a small `conversation_branches` table. Because every workspace table, every tool, the dynamic prompt, and the checkpointer config are all keyed by `thread_id`, a fresh thread flows through the turn loop, the prompt, the tools, and the SSE router **unchanged**. Switching to a branch reuses the existing `onLoadConversation` path verbatim; to the rest of the system, a branch is just another conversation.

**Continuing from a step is a branch.** The Studio's "Continue from here" action never truncates or deletes anything. It previews the chosen step and, the moment the user sends a message there, creates a branch at that node and switches to it. The git analogy is `git switch -c <new> <old-commit>`, never `git reset --hard`. The original conversation stays intact and reachable in the trajectory map.

Putting these together, turns chain linearly within one thread, and branching at turn *k* mints a fresh thread that owns turns *> k* while turns *≤ k* stay with the parent:

```mermaid theme={null}
flowchart LR
    subgraph Parent["Parent thread"]
        direction LR
        T1["turn 1"] --> T2["turn 2"] --> Tk["turn k"] --> T4["turn k+1"]
    end

    subgraph Branch["Branch thread · new thread_id"]
        direction LR
        B1["turn k+1"] --> B2["turn k+2"]
    end

    Tk -->|"branch at k · copy workspace as-of k"| B1
```

## Data model

One new table, `conversation_branches` (migration `0006_conversation_branches.py`, applied automatically at container start by `entrypoint.sh`):

| Column              | Meaning                                                                                                                        |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `thread_id`         | The child (branch) thread. **Unique**, a thread has at most one parent, which makes the lineage a tree and rules out cycles.   |
| `parent_thread_id`  | The thread this one forked from. Indexed for the children lookup.                                                              |
| `branch_point_turn` | The parent turn *k* the fork happened at. The branch owns turns *> k*; turns *≤ k* belong to (and render under) its ancestors. |
| `reason`            | Optional free-text label for the branch (carried by the API for future use; the current UI sends none).                        |
| `user_id`           | Owner; indexed. Guards compensation deletes.                                                                                   |
| `status`            | `pending` while a branch is being created, `active` once committed. Only `active` rows are visible to readers.                 |
| `created_at`        | Used to order sibling branches into lanes in the graph.                                                                        |

No existing table was altered. `ConversationSchema` (returned by `GET /api/conversations`) gains two additive, nullable fields (`parent_thread_id` and `branch_point_turn`) so the history menu can nest branches under their parent. The model column types use the same `BigInteger().with_variant(Integer, "sqlite")` idiom as the rest of the schema, so the in-memory SQLite test database stays compatible (tests build the schema from the ORM via `create_all`, never from migrations).

## Branch creation

`POST /api/conversations/{thread_id}/branch` with body `{"from_turn": k, "reason": "…"}` runs through `app/services/branching.py::create_branch`. The service opens its own sessions through `async_session_factory` (like the query router's snapshot capture) because a request-scoped `get_db` session cannot safely span the checkpointer network call.

**Validation** happens first, in one read session, and maps cleanly onto HTTP codes:

* thread unknown or soft-deleted → `404`; owned by someone else → `403` (the same split the DELETE endpoint uses).
* `from_turn < 0` → `422` (Pydantic `ge=0`); `from_turn = 0` is valid and forks from the empty start (a first-message edit/retry, only off the family root).
* `from_turn > current turn_index` → `404` (that turn does not exist).
* branching a thread that is itself a branch, with `from_turn <= its own branch_point_turn` → `400` (those turns belong to an ancestor, not this thread).
* the workspace as-of `from_turn` cannot be reconstructed because snapshot `from_turn+1` is missing → `409`.

**What gets copied.** `resolve_restore_state` produces the workspace after turn *k* (snapshot *k+1*, or the live rows when *k* is latest). The branch then receives: the resolved document title and content; the objective text (snapshots captured before the objective feature have no value and default to `None`); the preferences (snapshot JSON carries no `position`, so positions are re-assigned by array index; the per-preference `locked` flag *is* carried so a locked constraint stays locked on the branch; snapshots captured before the lock feature have no `locked` key and default to unlocked); the todos with their nesting `depth` (snapshots captured before the nesting feature have no `depth` key and default to 0); and every artifact row with `turn_index <= k` (re-inserted in ascending original-id order so `get_latest_artifact` stays correct). Snapshots are deliberately **never** copied; each embeds a full document body, so copying turns `0..k` would be quadratic. Inherited history for turns `≤ k` is reconstructed lazily by the graph endpoint walking the lineage instead.

**Edited-preview override.** When the branch is forked from a restored preview the user has *edited*, the frontend sends the **displayed** workspace as an optional `workspace` field on the branch request (`BranchWorkspaceOverride`: preferences, todos, objective, document), and `create_branch` seeds from it instead of the snapshot. This is what makes manual edits made in a preview (a toggled todo, a renamed/removed/added preference, a newly-set **lock**, an edited objective or document) carry onto the branch rather than being silently dropped (without it, only the raw snapshot reached the branch, so a lock set in the preview never bound the agent there). `resolve_restore_state` still runs to enforce the lineage/missing-snapshot **409** guard and to supply the per-field fallback for anything the override omits; `position` is not accepted from the client (re-derived from the override's list order) and an empty document body seeds no document. The override is fully optional, so an unedited preview, an explicit retry/edit fork, and any non-preview branch behave exactly as before.

**Turn numbering continues.** The branch's `conversation_users.turn_index` starts at *k*, so its first message becomes turn *k+1* via the existing `increment_turn`. Turn numbers therefore stay aligned across sibling branches, which keeps a future compare feature simple. The `message_count` on a branch is seeded as `2*k`, a display-only approximation, noted as such in the code.

**Message seeding.** The checkpointer is the second store, and `AsyncPostgresSaver` has no `acopy_thread` (the base-class method raises `NotImplementedError`). Instead, a minimal one-node `StateGraph(MessagesState)` is compiled with the same checkpointer; it reads the parent's full message list via `aget_state`, computes the prefix for turn *k* (every message strictly before the *(k+1)*-th `HumanMessage`), and writes it into the new thread with a single `aupdate_state(config, {"messages": prefix}, as_node="__start__")`. No LLM client, tool set, or prompt is constructed for this; the seeder graph only touches the messages channel. This was verified end-to-end against a real Postgres checkpointer by `backend/scripts/spike_branch_seeding.py` (and its pytest twin, `tests/test_branch_seeding_integration.py`, gated behind `RUN_PG_TESTS`) before any plumbing depended on it.

**Three phases and compensation.** There is no cross-store transaction (the checkpointer uses its own connection), so creation is staged and compensated:

1. **TX1** commits the conversation row, a `status='pending'` branch row, and all workspace and artifact copies.
2. **Seed** writes the message prefix into the checkpointer (no SQL session held open).
3. **TX2** flips the branch row to `'active'`.

If seeding or TX2 fails, a compensating cleanup deletes every SQL row created for the new thread and best-effort deletes its checkpointer thread, then the endpoint returns `502`. Because each request mints a fresh `thread_id`, a retry after failure simply creates a new branch; the cleanup guarantees no orphans, so no idempotency key is needed. The pending/active flag is what keeps a half-built branch invisible: both the conversation list and the graph count only `active` rows, so a branch interrupted between TX1 and TX2 never surfaces as a usable conversation.

## The history graph

`GET /api/conversations/{thread_id}/graph` returns the whole conversation family as a DAG, assembled by `build_conversation_graph` purely from reads: no checkpointer fork, no document bodies loaded.

The builder first **walks up** the `conversation_branches` chain (depth-capped at 50) to find the family root. Soft-deleted ancestors are kept on the way up, because a live branch inherits their history and must still render it. It then **walks down** from the root, breadth-first over `active` child branches, skipping soft-deleted descendants (a deleted branch takes its subtree with it, unless that subtree is an ancestor of the thread you asked about).

Each thread contributes one **node** per owned turn (turns after its `branch_point_turn` for a branch, all turns for the root) built from lightweight snapshot metadata (`id`, `turn_index`, `user_message` preview, `captured_at`) plus a per-turn artifact count. Edges come in two kinds: a `turn` edge links consecutive owned nodes within a thread, and a `branch` edge links a parent's branch-point node to a child's first owned node, carrying the child's `reason`. A branch with no owned turns yet (you branched but have not sent a message) contributes no nodes; the frontend draws a stub head from the `threads[]` list instead.

Two more details keep the payload safe and legible. The `is_on_active_path` flag marks the lineage of the *requested* thread: all of its own nodes, plus each ancestor's nodes up to the turn where the lineage forks toward it. And a hard cap of 500 nodes drops the **oldest** nodes across the family first (setting `truncated: true`), after which any edge whose endpoints were dropped is removed so the frontend never sees a dangling reference.

The response shape (`ConversationGraphResponse`) is designed for direct consumption by React Flow: a node id is `"{thread_id}:{turn_index}"` and an edge id is `"{source}->{target}"`, so the frontend mapping is a projection rather than a join.

## Frontend

The studio's right column (`frontend/src/app/app/page.tsx`) now stacks a collapsible **Co-construction Trajectory** panel above the chat. The panel (`components/app/graph/conversation-graph-panel.tsx`) renders the DAG with React Flow (`@xyflow/react`): pan, zoom, and drag over compact commit cards (à la VS Code's Git Graph) connected by arrowed edges. Each card carries a lane dot, its step number, and a mini transcript (the question, a reasoning-step indicator, and the agent's reply) each clamped with an ellipsis. The reply and reasoning count are enriched from the loaded transcript (`turnDetails`) for steps on the displayed path; off-path branch cards show the question only. The numbering is anchored to the turn position and the branch ordinal is **path-wide**: root steps are `1`, `2`, `3`; opening a path from a *past* step is an alternative at the next position, and each new branch of a path takes the next free ordinal in creation order no matter which node it forked from, so branching at step `3` gives `4.1, 5.1`, a later branch at step `2` gives `3.2, 4.2` (never colliding with the first), and an alternative of an alternative nests as `4.1.1`. Opening a path from the very *tip* simply continues the count (`4`); such a linear continuation silently reserves its ordinal, which has two deliberate consequences: visible ordinals may have gaps (the first visible alternative can be `.2`), and if the original path is later continued past the fork, the continuation flips to its reserved ordinal (plain `4` becomes `4.2`, descendants re-root under it) without renumbering any sibling. Equal suffixes imply the same path with disjoint turn ranges, so labels are globally unique. The path currently displayed in the studio is drawn in brand red (dots and arrows alike); everything else is dimmed gray. Layout is deterministic and needs no layout library; `frontend/src/lib/graph.ts::buildFlowGraph` places a node at `x = turn_index * gap` and `y = lane * gap`, assigning each branch its own horizontal lane, computing the labels (`computeThreadSuffixes`), and synthesizing stub head nodes for freshly opened paths that own no turns yet. Keeping this mapping as a pure module (no React imports) lets it be unit-tested without a DOM and counts toward the frontend coverage gate.

Graph data is fetched once at the page level by `frontend/src/hooks/use-conversation-graph.ts` (which guards against out-of-order responses) and shared with both the panel and the conversation. The hook refetches whenever the displayed family changes or a `graphVersion` counter is bumped; the counter ticks after each completed SSE turn and on conversation load.

**Restore now, branch only on send.** Selecting a step (the detail card's **Continue from here**, a double-click on a dot, or the **Saved checkpoint · Click to restore** divider rendered between turns in the chat) immediately restores the studio to that step: the conversation is truncated to that turn and the workspace is shown as it stood then (the page's `loadWorkspaceAsOf` reads `snapshot(k+1)` or the live rows), with the "you are here" marker on the graph following the restored step (`markCurrentId`). No branch is created at this point; the studio simply detaches from any writable thread. The branch is minted lazily by `onSend` calling `ConversationsAPI.branch(...)` only when the user actually sends a message from the restored step, so merely revisiting an earlier point never leaves an empty thread behind. Navigating away (opening another conversation, starting a new one) discards the restored view with nothing created. The preview is fully editable while detached, the workspace handlers apply edits to local state when there is no writable thread, and on send the displayed (edited) workspace is passed as a `workspace` override on the branch request, so those edits are seeded into the new branch (see *What gets copied*); navigating away without sending still discards them, so the Studio guards every node/conversation switch (and **New conversation**) with a confirm dialog when the restored preview has uncommitted edits, compared against a baseline signature captured on entry (objective + prefs + todos, plus the document's `dirtyRef`), so an unedited preview switches without a prompt.

**Conversation ↔ trajectory links.** `activePathGuide(graph)` annotates each turn of the displayed route with its step label and any alternative paths that fork from it. The chat tags each user message with its step number beside the timestamp (`Node: 3.1`) and renders the offshoots as clickable chips that open those existing paths. Clicking a step on the displayed path scrolls the conversation to the matching message and briefly flashes it. The history menu (`components/app/chat-panel.tsx`) nests branch conversations one level under their parent (via `parent_thread_id`), labels them "branched at step N", and strips the legacy "@ turn N" suffix from titles so the list reads cleanly.

## API summary

| Method & path                          | Purpose                                                                                                       |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `POST /api/conversations/{tid}/branch` | Fork at `from_turn` into a new thread; returns `{thread_id, parent_thread_id, branch_point_turn, title}`.     |
| `GET /api/conversations/{tid}/graph`   | The conversation family DAG (nodes, turn/branch edges, thread metadata, `truncated`).                         |
| `GET /api/conversations`               | Existing list; each item now also carries `parent_thread_id` and `branch_point_turn` (null for non-branches). |

## Known limits and deferred work

The current implementation is deliberately the minimum coherent slice of the larger issue. A few constraints and non-goals are worth stating plainly:

* **No merge.** Three-way merging a reasoning transcript is causally meaningless and is a permanent non-goal. The feature offers branch and restore, not merge.
* **No compare view yet.** The graph payload is shaped so a turn-aligned structural diff of two nodes can be added later, but the diff endpoint and UI are deferred.
* **No decision map yet.** Soft-deleting rejected preferences and todos (so discarded options render in the graph) is a planned follow-up; today's deletes are still hard deletes.
* **Pending orphans.** A crash in the narrow TX1-to-TX2 window leaves an invisible `pending` branch row with a copied workspace. It is correctly hidden from the list and the graph, but no janitor yet reclaims its rows; a periodic sweep of stale `pending` branches is a reasonable future addition.
* **Quadratic-copy trap.** If you extend the copy step, never copy `workspace_snapshots`; each row embeds the full document body. Copy live or single-snapshot state only.

## Where the code lives

* `backend/app/services/branching.py`: prefix rule, restore-state resolution, checkpointer seeding, 3-phase creation with compensation, and graph assembly.
* `backend/app/db/repositories/branch_repository.py`: CRUD over `conversation_branches`.
* `backend/app/db/repositories/workspace_repository.py`: `restore_workspace`, `copy_artifacts`, `list_snapshot_meta`, `count_artifacts_by_turn`, and the compensation delete.
* `backend/app/routers/conversations.py`: the two new endpoints plus the lineage-annotated list.
* `backend/alembic/versions/0006_conversation_branches.py`: the schema migration.
* `frontend/src/lib/graph.ts`, `frontend/src/hooks/use-conversation-graph.ts`, `frontend/src/components/app/graph/`: the panel, layout, and data hook.
* `backend/scripts/spike_branch_seeding.py`: the standalone checkpointer-seeding spike.