Skip to main content
HAI-Co² is, among other things, a data-collection platform for studying human-AI co-construction. Every session produces two complementary records:
  1. Execution traces of how the agent thought: the LLM calls, prompts, completions, token usage, tool calls, and timings, captured through OpenTelemetry and viewable in Phoenix (and optionally LangSmith).
  2. A first-party structured dataset in the application’s own Postgres database: the full message history, a per-turn snapshot of the entire workspace, the conversation trajectory and its branches, the agent’s reasoning trace, and every artifact produced.
Together these let you reconstruct, replay, and analyse exactly what happened on any turn of any conversation. This page covers both streams and the consent and privacy model around them.

​
Two recording streams

​
Agent tracing (Phoenix & LangSmith)

A single module, backend/app/services/observability.py, is the entry point. init_observability() runs once at application startup (from the FastAPI lifespan in main.py, before any agent is built) and activates backends based on the observability.provider flag, which takes one of none, langsmith, phoenix, or both. Both backends auto-instrument LangChain and LangGraph globally, so there are no per-call callbacks to maintain:
  • Phoenix is registered through phoenix.otel.register(..., auto_instrument=True), which patches LangChain via OpenInference and exports spans over OTLP.
  • LangSmith is activated by setting LangChain’s LANGCHAIN_* environment variables; LangChain then traces itself.
If a backend’s packages are missing or it is disabled, that backend is skipped and the app runs unaffected.

​
What gets captured

Because instrumentation sits at the LangChain/LangGraph layer, a trace records the agent’s actual execution:
  • every LLM call: the rendered prompt, the completion, the model, token usage, and latency;
  • every tool call: arguments and return values, including the (content, artifact) results;
  • LangGraph node transitions and the model-node retry wrapper;
  • errors and retries along the way.
Spans are tagged for analysis. get_session_context(thread_id) wraps a turn so every span carries session.id = thread_id, which groups all of a conversation’s spans into one Session in Phoenix. get_trace_context(...) layers on optional metadata keys and string tags. Separately, the @workspace_tool decorator logs each tool’s action_and_reasoning sentence, the same plain-language line the user sees in the Internal Reasoning trace. From the admin dashboard, a Trace button next to any conversation opens that thread’s Phoenix session. The backend resolves the link lazily: GET /api/admin/threads/{thread_id}/trace looks up the session through the Phoenix REST API (authenticated with the System PHOENIX_API_KEY), builds the public URL from phoenix.web_base_url, and caches it on conversation_users.phoenix_url so later requests are a single DB read. The link is null until Phoenix has a session for the thread, and the whole feature is disabled when phoenix.web_base_url is empty. The same resolved link is embedded in the GitHub issue that the “create issue from report” action files (see User feedback).

​
Configuration

Tracing is configured per environment in deployment/<env>/backend/config.*.yaml, with secrets supplied through the environment. Secrets (the LangSmith API key and the Phoenix System API key) never live in YAML. The default everywhere is provider: phoenix with langsmith.enabled: false, so tracing goes only to the self-hosted Phoenix and no session content is sent to LangSmith (an external SaaS). This keeps prompts and completions inside the deployment’s own infrastructure, which is the appropriate default for a GDPR research deployment handling personal data. To also send traces to LangSmith, set provider: both, langsmith.enabled: true, and provide LANGCHAIN_API_KEY. To turn tracing off entirely, set observability.provider: none.

​
Phoenix deployment

Phoenix runs as its own container (arizephoenix/phoenix) in each environment’s docker-compose.yml, alongside Postgres, the backend, and the frontend. It serves its UI on port 6006 and its OTLP collector on 4317; the backend exports to http://phoenix:4317 over the Compose network. Traces persist to a SQLite database in a named volume, and the default retention policy is 0 days, meaning traces are never purged. In dev and production it is exposed behind the reverse proxy at /phoenix (PHOENIX_HOST_ROOT_PATH=/phoenix), so it is reachable at https://dev.haico.gr/phoenix and https://haico.gr/phoenix; locally it is bound to 127.0.0.1:6006.
Two Phoenix gotchas, both covered in Deployment. PHOENIX_SECRET must be at least 32 characters or the container crash-loops. And with Phoenix auth enabled, its OTLP collector is protected too, so the backend needs a Phoenix System API key in PHOENIX_API_KEY or trace exports fail with UNAUTHENTICATED.

​
First-party data collection (the research dataset)

This is the part that makes HAI-Co² a study instrument rather than just a traced app: independently of Phoenix, the backend records a complete, self-contained, replayable account of every co-construction in its own database. The schema is documented in full in Database schema; the table below is the data-collection view of it. Two of these deserve emphasis, because together they make a turn fully reconstructable:
  • Workspace snapshots are the backbone. Snapshot N is written at the start of turn N, capturing the triggering user message plus the entire workspace the agent is about to act on (document, objective, preferences, todos), each embedded as self-contained JSON. That is enough to re-render the exact context the agent saw on any past turn, which is also what powers restore and branching. The timing rule (snapshot k+1, or the live rows, gives the state after turn k) is detailed in Database schema.
  • The LangGraph checkpointer stores the verbatim message history (every Human, AI, and Tool message, with serialised prompts, completions, and tool I/O), so the conversation can be replayed message by message.
Because every workspace table is scoped by thread_id and joined to a user through conversation_users(thread_id, user_id), the whole dataset is attributable per user and per conversation for analysis. HAI-Co² records research data, so it carries an explicit consent marker. The users.recording_consent_at timestamp is NULL until the user consents; POST /api/auth/consent sets it once (idempotently) and logs a recording_consent_granted activity event. The field is returned on the user object so the frontend can gate the recording-consent flow. What is stored about a user: username and email, auth_provider and (for Google sign-in) google_id, a bcrypt password hash (null for OAuth-only accounts), the consent timestamp, and an ip_address recorded on each activity event (login, register, query, consent). Generic error messages on registration avoid confirming whether an email already exists.
Retention. There is no automatic purge in the application: workspace snapshots and LangGraph checkpoints persist indefinitely, deleting a conversation is a soft delete (it sets deleted_at and preserves the workspace data), and Phoenix’s trace retention is set to never purge. Removing data is therefore a deliberate administrative action. Plan retention according to your study’s data-governance and ethics requirements.
  • Database schema: every table, snapshots, and how restore and branching reconstruct a turn.
  • Conversation branching: how the trajectory tree is built and forked.
  • Artifacts: the typed outputs recorded in the artifacts table.
  • Deployment: the Phoenix service, its secrets, and the reverse-proxy route.