After removing a Linear connection, connecting again could skip signing in
and save Linear as an unconnected custom tool: a client cached before the
removal still held the old tokens, and a late token write re-created a
connection for them. A token write for a named connection no longer creates
one, removing or disconnecting a connection drops its cached clients, and a
sign-in server is never saved without its connection.
The connection setup endpoint takes sync.config, validated like an
upload's config, and passes it to the ingest task so the synced source
gets the chosen chunking and retrieval settings. An invalid config is
refused before the idempotency key is claimed.
The Linear connector now syncs as well as giving agents its tools, from
one connection. Linear's MCP server is its own OAuth issuer, so its
tokens are read through the same MCP tools the agents use (list_issues,
get_issue, list_comments, list_documents) rather than Linear's GraphQL
API, and no OAuth app has to be registered.
A source picks teams and projects, with comments (on by default) and the
projects' documents. Each issue becomes one document with its state,
assignee, priority, labels, description and comments, filed under its
team and citing its Linear URL. Each sync reads up to 500 issues and 100
documents again. /api/connections/<id>/linear lists the teams and
projects to pick from. Sources sync on their schedule with the owner's
connection, and pause when the sign-in needs reconnecting.
A GitHub connection's MCP tool can now point at GitHub's full endpoint
(/mcp/) instead of the read-only one when its owner opts in, at setup
(allow_writes) or later (PUT /api/connections/<id>/writes), which re-reads
the actions and keeps the choices for those on both endpoints. Actions from
the write endpoint are writes unless GitHub marks them read-only, so they
default to asking first.
Admins can forbid it per connector (allow_writes in Admin > Connectors,
kept in app_metadata). Then the option is refused, a refresh goes back to
read-only, and at run time the tool only ever calls the read-only endpoint
and write calls are denied with a reason.
Connections get an account_name (migration 0039) that the owner sets with
PATCH /api/connections/<id>; the account label stays the account's
identity, so signing in again still finds it. When someone has more than
one account of a service, its tools are listed as "Telegram · <account>"
and the model sees each account's actions under the account's name
(telegram_send_message_alerts_bot) with the account in the description,
instead of _1 and _2. Actions shared by different services are named
after the service. Tools a user renamed keep their name.
A Telegram connection can now hold an optional default chat ID. When it
is set, both Telegram actions send there, the model is no longer asked
for a chat, and a chat it names anyway is ignored. In member mode each
member's own connection supplies their own chat. The same bot with a
different chat is a separate connection.
PUT /api/connections/<id>/tools/<tool_id>/parameters takes an action and
a map of parameter to value (always use it) or null (let the model
decide), checked against the action's schema; only the tool's owner can
call it. Connection tools now list each action's parameters and whether
they are fixed.
update_tool_actions now checks the submitted actions against the tool's
stored ones: no new actions or parameters, fixed values must fit their
parameter's type. A team editor can still switch actions on and off but
gets 403 for changing a fixed value.
The admin connectors API lists the settings that add Sign in with GitHub
and whether they are complete, next to the required settings (none for
GitHub, which works with tokens alone).
One GitHub connection feeds both a Knowledge source and an agent tool.
Users connect with a personal access token, checked against GitHub and
named after the account, or, when an admin registers a GitHub App
(GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, GITHUB_APP_SLUG), with Sign in
with GitHub. App tokens expire after eight hours and are refreshed before
each sync or tool call; tokens with no expiry are never treated as expired.
The catalog gains an optional second sign-in method (oauth_settings,
exposed as sign_in_methods) without changing any other connector.
Sync lists the repositories the connection can read (the token's own, or
the App installations') and ingests one with the connection's token. The
tool is GitHub's read-only MCP server (api.githubcopilot.com/mcp/readonly):
setup discovers its actions and creates it bound to the connection, and
the executor sends the connection's token only to that server.
GITHUB_ACCESS_TOKEN belongs to the server, but any user could ingest any
repository it can see, private ones included. It is now used only for
public repositories; the loader checks the repository's visibility first
and asks for a GitHub connection otherwise.
The loader also takes a token from the connection a source syncs from.
Merging a connection's keys into a plain repository URL no longer fails
on json.loads, manual Sync now passes the source's connection, and a
token GitHub rejects pauses the connection's sources for reconnect.
mcp 2.x reads code, state and the RFC 9207 iss off the callback handler's
result; the handler still returned a (code, state) tuple, so every MCP
sign-in failed after the user approved it. The callback route now keeps the
issuer too, since servers that advertise it (Linear) refuse a sign-in
without it.
The client follows an MCP sign-in by its task's events, but the test route
kept only success, requires_oauth and auth_url, so the wizard and the custom
MCP modal reported a failed sign-in before the popup could load.
- One control for members' own MCP servers: the custom MCP server row's
switch (it sets both the policy and the instance switch); the separate
top-level switch is gone.
- Connectors that only sync content show "No tools" instead of a
sharing policy, and the policies read "The sharer decides per share",
"Always the sharer's account", "Always each person's own account".
- A connector that needs server settings can't be switched on until they
exist; a tooltip says why.
- On phones the table becomes a list ("On · 2 connections · …") and each
connector's controls open in a sheet.
- Add Source hands a connection tile to the connect wizard: an existing
account goes straight to choosing what to sync, otherwise it connects
one. The legacy per-service forms stay only for installs without
connectors, so S3 no longer has two different forms.
- Add Tool lists services from the connector catalog first, MCP presets
included, each with its connection state; a connected one opens its
drawer, the rest start the wizard. Built-in tools follow.
- Service tools carry the connector's name everywhere ("Telegram", not
"Telegram Bot"), and the composer groups custom tools (API tool, MCP
servers) as Custom, like the agent builder.
- OpenAPI opens the API tool editor as an unsaved draft; the tool is
created on its first save, so exploring the menu leaves nothing behind.
Notion, Linear, Atlassian, Sentry, Asana and Stripe no longer open the
MCP server form. The wizard shows one "Sign in to Notion" button: the
pop-up opens inside the click (so browsers allow it), follows the
worker's mcp.oauth events to the provider, and the tool is saved on
success with its actions on the done step. Reconnecting a preset uses
the same flow, as do the chat bar and the health toast.
Saving an OAuth MCP server without a new handshake now uses the stored
sign-in instead of failing, and the custom MCP form keeps scopes and
timeout under Show advanced.
An agent called with its API key (widget, API) runs as its owner, and
nobody can approve an action there, so a write set to Always allow ran
on the owner's account for anyone holding the key. A caller is external
when the request carries the key and is not signed in as the owner (an
owner previewing their agent keeps full access).
For external callers, write actions on connection-backed tools are
refused unless listed in the agent config's new api_write_allowlist
(tool_id:action), and a missing connection is refused instead of pausing
on a Connect card the widget cannot show. The flag and list survive a
paused-and-resumed stream. Owners pick the allowed actions under Access
details; a team editor's update keeps the stored list.
A connector's enabled switch is now optional: unset means on when the
connector has the server settings it needs, so Google Drive, SharePoint
and Confluence start off until their OAuth settings are present, and an
admin's explicit switch always wins. Changing only the sharing mode no
longer switches a connector on.
Members only see connectors they can use. The catalog, Add Source tiles
and Add Tool leave out anything turned off or still needing setup, except
a connector a member already has a connection to, which stays listed as
turned off so it can be managed or removed.
- Creating a tool from an existing connection validates its config too;
the connection supplies the secrets the request leaves out.
- When the connector policies cannot be read, the MCP test and save
routes answer 503 instead of contacting the server.
- Editing a connection-backed tool validates the config before writing
new credentials, so a rejected edit no longer rotates the connection's
key and resumes its sources.
- An MCP edit keeps its previous connection only when the caller owns it
and it signs in the way the tool now does.
Covers creating and editing tools whose key lives on a connection, the
MCP action refresh, and the auth, ownership and error branches of the
connections API. Editing a tool's key now also recovers a connection
whose stored credentials can no longer be decrypted.
Editing an MCP server's URL kept the previous connection when no new
one could be stored (default encryption key), so its key would reach the
new server. The save route now keeps a previous connection only for the
same server, and the executor refuses an MCP tool whose connection was
stored for another server.
- An MCP connection's tokens only go to its own server, and a
connection_id can no longer come from a client: the MCP test/save
routes drop it and the tool executor uses only the resolved one.
- API-key connections are reused only when they hold the same
credentials. Two keys that share a hint or a label get separate
connections ("…abcd (2)"), at runtime and in the 0038 backfill.
- Reconnecting a connection whose secrets cannot be decrypted no longer
flags it from a second transaction while holding its row lock.
- Connection setup validates the sync request before claiming its
Idempotency-Key, so a corrected retry is queued.
- A request that names no connection resolves to none without opening
a database connection; fixes two tests that reached a real database.
Six vendor-run MCP servers (Notion, Linear, Atlassian, Sentry, Asana,
Stripe) join the catalog as presets from docsgpt/connectors/presets/
mcp.yaml; existing connections to those servers show under them.
connector_policies lets an admin turn a connector off or force whose
account every share uses, and app_metadata's
connectors.allow_custom_mcp turns custom MCP servers off. Both are
enforced on the server: new connections, OAuth sign-ins, MCP test and
save, and the tools of a disabled connector stop resolving.
GET/PUT /api/admin/connectors reads and changes them.
A shared tool's owner picks owner or member credentials with
PUT /api/connections/tools/<id>/credential-mode, and a member running
the owner's account always confirms write actions.
Tool-call events and retrieved chunks name the connector they came
from (key and display name, never the account), and tool calls keep
those fields when the conversation is reloaded.
Migration 0038 moves every stored secret (OAuth tokens, MCP OAuth
tokens and client registrations, API keys) into the connection's
encrypted envelope, links API-key tools to one connection per distinct
credential, allows several accounts per provider, and adds
credential_mode to sources and tools. OAuth MCP tools keep resolving
each member's own token, as they did before.
docsgpt.connectors.service is now the only reader of OAuth tokens:
get_valid_token_info refreshes under a row lock and persists rotated
refresh tokens, and a revoked grant flags the connection, pauses its
sources and notifies the owner. Loaders build from a connection
(BaseConnectorLoader.from_connection), so scheduled sync covers Drive,
SharePoint and Confluence sources with no browser. S3 and Reddit keys
stay on the connection instead of in remote_data.
New endpoints: POST /api/connections, /setup, /reconnect,
/picker-token, /claim, DELETE /api/connections/<id>, per-action
permissions and MCP refresh-tools. Upload, file listing, sync and
validate-session take a connection_id; session tokens keep working for
this release. The tool executor reads credentials from the resolved
connection (owner or member mode) and pauses on a Connect card when a
connection needs signing in. docsgpt connectors reencrypt rewrites
stored credentials after a key rotation.
connector_sessions already holds one row per signed-in account or MCP
server. Migration 0038 names each row (connector_key, auth_kind,
display_name, account_label) and lets sources and tools point at the
row they use through connection_id, backfilling the links it can prove.
docsgpt/connectors/catalog.py declares every service DocsGPT connects
to, with the server settings it needs and what a connection provides.
New endpoints: GET /api/connectors/catalog, GET /api/connections,
GET /api/connections/<id> and POST /api/connections/<id>/disconnect.
/api/available_tools now groups tools into built_in, service and
custom, and source listings carry connectionId.
chunks is a total per request, split across the attached sources, so an
agent with two sources and the default of 2 got a single chunk from each,
and its answers changed with whichever chunk won. 6 gives three per
source for about 4-5k more input tokens per retrieval turn.
Every literal default moves from 2 to 6: the request default, the
retrievers, the internal search tool, workflow agent nodes, scheduled and
headless runs, agent create/update/import, the source retrieval config and
the frontend forms. Existing agents and sources keep what they store; a
source saved with chunks=2 now counts as configured at 2, which is pinned
by a test.
Headless runs also read chunks=0 as unset (`or 2`), so an agent with
retrieval switched off retrieved anyway on scheduled runs; 0 now stays 0.
A turn whose agent raised wrote no user_logs row, so it only surfaced as
the agent's system error row. Every finished turn now writes its chat row,
at level error with the error when it failed, and linked to its trace; the
system row for the same traced activity is no longer listed twice.
The OTel replay and the trace INSERT ran in the stream's finally, so a slow
database held the SSE connection open after the last event. The trace is
still frozen when the stream ends, but written on a small writer pool.
A failed trace-summary lookup now leaves the Logs page intact without
chips. Tool-call counts include only calls that ran, not their paused,
denied or skipped records. A local guardrail that fires unchanged on every
streamed segment is recorded once, so it cannot use up the span cap.
build_agent no longer takes request_id from the request body: it becomes
the primary LLM's usage request id, and quotas count distinct request ids,
so a client could make every call count as one. Requests refused after
setup started (unauthorized, over quota, resume conflict, setup error) now
write their trace, marked error, through an after-request hook; streaming
routes hand the trace to complete_stream instead.
Tool results, denial comments and tool exception text now reach a span only
as a capture-gated preview; span.error is a fixed message, since it is
stored and exported regardless of content settings. A turn whose stream
yields an error event is recorded as failed. Search traces are listed by a
query copied into the small summary column, so the Logs timeline never
reads the spans JSONB. Adds GraphRAG span tests.
The trace lifecycle moves into a decorator so complete_stream keeps its
real parameters for introspection. GET /api/traces is readable with the
analytics:read scope, like the Logs endpoint it complements.
GET /api/traces returns a Logs row's stored traces, scoped to the caller or
to an agent they own. get_user_logs rows now carry the ids that find their
trace and a merged summary (duration, LLM and tool calls, tokens, retrieval
time) from one batched lookup per page; searches, MCP searches and graph
builds, which have no log row of their own, are listed from their traces.
run_agent_headless records each unattended run under its endpoint; the
scheduler passes its run id and the webhook worker its task id so Logs rows
can find their trace, while the LLM's own request id stays untouched for
quota counts. /api/search and MCP search_docs record their retrieval, and a
graph build records every extraction call under one step.
StreamProcessor starts the trace and mints the request id before the
agent is built, so pre-fetch retrieval and compression are inside it and
side-channel LLM calls share the id. complete_stream activates it in the
SSE pump thread, binds the message and conversation, and writes it once
however the stream ends: paused, failed, abandoned or superseded (dropped).
user_logs rows now carry request_id and message_id.
One row per execution holding its span tree as JSONB, linked to Logs rows
by request, message, activity and workflow-run ids. message_id cascades so
deleting or truncating a conversation drops its traces; a daily beat task
enforces TRACES_RETENTION_DAYS.
`float("inf") > 0` is true, so a stored `"inf"` was preserved as if it
were a real count and reached the UI as "∞". Require a finite positive
value, which also covers `nan` explicitly rather than by accident.
Chunk cards in the source viewer read `metadata.token_count` and print a
bare "-" whenever it is absent. Ingestion records the count, but chunks
indexed before it was written -- and any path that rebuilds a chunk's
metadata from scratch -- reach the UI without it, so the whole file shows
"-" with no way to recover the number short of a re-ingest.
Fill the key in on the way out when it is missing or unusable (empty,
non-numeric, zero or negative), counting only the page being returned so
the cost is bounded by `per_page`. A count that is already recorded is
left untouched, including a numeric string, since stores round-trip
metadata types differently.
The fallback counts in cl100k rather than the embedding model's
tokenizer: loading that tokenizer can reach for a Hugging Face download,
which does not belong in a request path.
Correctness
- /api/remote never recorded source.created, so URL, GitHub and connector
sources had a source.deleted with no matching creation. All three creation
paths now go through one _audit_source_created helper.
- The prompt-cache rate divided cached tokens by a whole bucket's prompt
tokens. A bucket is a day and mixes calls whose provider reports a cache
breakdown with calls whose provider does not, so filtering buckets in the
client could not separate them and the rate was understated by however much
traffic ran on a non-reporting provider. The denominator is now computed in
SQL over the reporting rows.
- The outcome pill matched values nothing writes. Guardrails emit triggered /
not_evaluated and the device feed emits dispatched; the map had blocked /
denied / allowed, so a guardrail that fired rendered neutral grey -- the one
signal the merged feed exists to surface. Fixtures were seeding the
fictional values, so the tests passed on it too.
- Stream duration_ms timed the consumer. stream_token_usage is a generator,
so start-to-exhaustion includes the agent loop's tool handling and the SSE
client's pace; a slow browser recorded ~30s for a sub-second call. It now
accumulates only the time spent inside next().
Safety
- Activity filters failed open: an unknown facet or unparseable timestamp was
dropped, and no filter means every row, so a typo widened an audit view and
on the export streamed the full history. Both are now a 400.
- The search term was interpolated into an ILIKE pattern, so "100%" matched
everything and "q1_report" matched more than it should. Escaped.
- 0034 set actor_id NOT NULL with no default. A previous-release process
inserting mid-rollout would raise, and in admin/routes.py that insert shares
the request transaction, so a role grant beside it would roll back too.
Noise and dead code
- The per-user panel is a security panel: data-plane events file under the
actor, so an active account's routine deletes pushed a denied login out of
the 20-row window. It now excludes them; the Activity tab shows everything.
- device_audit_log had no created_at-leading index, so the merged feed
sequentially scanned that branch every page (migration 0036).
- conversation.deleted_all no longer records when nothing was deleted, and
agent.updated no longer records an empty field list.
- Dropped by_model from /admin/usage (no consumer; an extra aggregate per page
load), the duplicate filter surface on AuthEventsRepository that nothing
called, and the unreachable FLOW_LABELS.schedule entry.
- Type hints on record_event's conn and the remaining unannotated helpers.
arc53-machine