27 Commits
Author SHA1 Message Date
arc53-machine 3cab8af753 Keep the removed-connection note server-owned
Only removing a connection notes it on a kept tool. A config save through
update_tool or update_tool_config carries the stored note and ignores the
client's copy, so it can neither fake nor clear it; creating a tool, an
MCP server save and an agent import start without one. The note is now
written even for a connection with no catalog key, and a kept tool its
owner gave credentials of its own since runs again.
2026-09-29 18:12:54 +01:00
arc53-machine 5c8b97b609 Gate outside writes on the owner's stored credentials, not only connections
API-key, widget and public-link callers were held to the write allowlist
only on connected accounts, so they could still write through an API
tool or a signed-in MCP server that carries the owner's credentials. Any
write on credentials the caller doesn't hold is now refused unless the
owner allowlisted it, and a scheduled run for such a caller counts as not
holding any. The tool list names these writes per tool, so the allowlist
can offer them, and its copy now names every route it covers.
2026-09-29 15:41:34 +01:00
arc53-machine 4f5cd68771 Keep fixed values, tool type and connected servers out of editors' tool saves
/api/update_tool now checks submitted actions against the stored ones the
same way /api/update_tool_actions does: nothing can be added, and only the
tool's owner can change a fixed value. The tool type (name) can no longer
be changed after creation by anyone.

For API tools, changing who fills an existing header, query or body
parameter, its value, or clearing a stored value is now owner-only on both
/api/update_tool and /api/update_tool_config. Before, an editor could hand
a stored secret query value to the model and read it back in the chat.
The chat now masks every value that came from the stored action rather
than from the model.

A connection-backed MCP tool can no longer be moved to another server or
sign-in method through /api/update_tool or /api/update_tool_config (400,
pointing to /api/mcp_server/save), and a new key saved through
/api/update_tool_config goes to its connection, owner only.
2026-09-29 14:11:36 +01:00
arc53-machine a1789d2ab4 Merge main (roles and access revamp) into connectors
Main's access model (team editors and viewers, edit_credentials, owner-only
OAuth servers, per-user tool preferences, resource sponsors) now applies to
connection-backed tools and sources. Our migrations are renumbered to
0040_connections and 0041_connection_account_name, after main's
0038_resource_access_settings and 0039_resource_sponsors.

Where the two sides met: MCP tools save and load their connections as the
tool owner, editors may add a key (a new connection on the owner's account)
but never rewrite an existing connection's secret, fixed values stay
owner-only, and a member's own connection is used in member mode.
2026-09-29 14:02:26 +01:00
arc53-machine 56ababae71 Stop a removed MCP connection from saving its server as a custom tool
After removing a Linear connection, connecting again could skip signing in
and save Linear as an unconnected custom tool: a client cached before the
removal still held the old tokens, and a late token write re-created a
connection for them. A token write for a named connection no longer creates
one, removing or disconnecting a connection drops its cached clients, and a
sign-in server is never saved without its connection.
2026-09-29 13:13:27 +01:00
Pavel 258c3fb352 Mcp fix 2026-09-29 16:06:05 +04:00
Pavel 68c1e12b6f MCP rollback for connectors 2026-09-29 14:48:18 +04:00
arc53-machine bdcb88866a Let people name accounts and use the names to tell accounts apart
Connections get an account_name (migration 0039) that the owner sets with
PATCH /api/connections/<id>; the account label stays the account's
identity, so signing in again still finds it. When someone has more than
one account of a service, its tools are listed as "Telegram · <account>"
and the model sees each account's actions under the account's name
(telegram_send_message_alerts_bot) with the account in the description,
instead of _1 and _2. Actions shared by different services are named
after the service. Tools a user renamed keep their name.
2026-09-29 10:41:54 +01:00
arc53-machine 54bea5a1d8 Keep fixed values when an MCP server's tools are refreshed or re-saved
Parameters that still exist after re-reading the server keep the value the
user fixed; parameters the server dropped go with the old schema.
2026-09-29 10:41:20 +01:00
arc53-machine f6bdd853d5 Validate tool action updates and keep fixed values owner-only
update_tool_actions now checks the submitted actions against the tool's
stored ones: no new actions or parameters, fixed values must fit their
parameter's type. A team editor can still switch actions on and off but
gets 403 for changing a fixed value.
2026-09-29 10:41:13 +01:00
Pavel 98afa5d93c Roles audit and revamp 2026-09-29 10:42:39 +04:00
arc53-machine 0d42916d67 Hand the MCP SDK an authorization result with the issuer
mcp 2.x reads code, state and the RFC 9207 iss off the callback handler's
result; the handler still returned a (code, state) tuple, so every MCP
sign-in failed after the user approved it. The callback route now keeps the
issuer too, since servers that advertise it (Linear) refuse a sign-in
without it.
2026-09-28 23:32:21 +01:00
arc53-machine 46928cba3d Return the OAuth task id from the MCP test route
The client follows an MCP sign-in by its task's events, but the test route
kept only success, requires_oauth and auth_url, so the wizard and the custom
MCP modal reported a failed sign-in before the popup could load.
2026-09-28 23:16:42 +01:00
arc53-machine 23229181f1 One connect flow and one name from every entry point
- Add Source hands a connection tile to the connect wizard: an existing
  account goes straight to choosing what to sync, otherwise it connects
  one. The legacy per-service forms stay only for installs without
  connectors, so S3 no longer has two different forms.
- Add Tool lists services from the connector catalog first, MCP presets
  included, each with its connection state; a connected one opens its
  drawer, the rest start the wizard. Built-in tools follow.
- Service tools carry the connector's name everywhere ("Telegram", not
  "Telegram Bot"), and the composer groups custom tools (API tool, MCP
  servers) as Custom, like the agent builder.
- OpenAPI opens the API tool editor as an unsaved draft; the tool is
  created on its first save, so exploring the menu leaves nothing behind.
2026-09-28 21:49:01 +01:00
arc53-machine bfcc4a9773 Sign in to MCP presets from the connect wizard
Notion, Linear, Atlassian, Sentry, Asana and Stripe no longer open the
MCP server form. The wizard shows one "Sign in to Notion" button: the
pop-up opens inside the click (so browsers allow it), follows the
worker's mcp.oauth events to the provider, and the tool is saved on
success with its actions on the done step. Reconnecting a preset uses
the same flow, as do the chat bar and the health toast.

Saving an OAuth MCP server without a new handshake now uses the stored
sign-in instead of failing, and the custom MCP form keeps scopes and
timeout under Show advanced.
2026-09-28 21:42:50 +01:00
arc53-machine a564dea401 Hide connectors that need admin setup; they start turned off
A connector's enabled switch is now optional: unset means on when the
connector has the server settings it needs, so Google Drive, SharePoint
and Confluence start off until their OAuth settings are present, and an
admin's explicit switch always wins. Changing only the sharing mode no
longer switches a connector on.

Members only see connectors they can use. The catalog, Add Source tiles
and Add Tool leave out anything turned off or still needing setup, except
a connector a member already has a connection to, which stays listed as
turned off so it can be managed or removed.
2026-09-28 20:16:40 +01:00
arc53-machine 324d54ea67 Validate configs on the existing-connection path; MCP policy fails closed
- Creating a tool from an existing connection validates its config too;
  the connection supplies the secrets the request leaves out.
- When the connector policies cannot be read, the MCP test and save
  routes answer 503 instead of contacting the server.
2026-09-28 19:47:28 +01:00
arc53-machine 473e8ec15c Validate tool edits before touching the connection; scope kept MCP connections
- Editing a connection-backed tool validates the config before writing
  new credentials, so a rejected edit no longer rotates the connection's
  key and resumes its sources.
- An MCP edit keeps its previous connection only when the caller owns it
  and it signs in the way the tool now does.
2026-09-28 19:31:02 +01:00
arc53-machine 4d533d24d6 Test connection-backed tools, MCP refresh and API failure paths
Covers creating and editing tools whose key lives on a connection, the
MCP action refresh, and the auth, ownership and error branches of the
connections API. Editing a tool's key now also recovers a connection
whose stored credentials can no longer be decrypted.
2026-09-28 19:11:54 +01:00
arc53-machine fff3e7c904 Never send an MCP connection's secret to a different server
Editing an MCP server's URL kept the previous connection when no new
one could be stored (default encryption key), so its key would reach the
new server. The save route now keeps a previous connection only for the
same server, and the executor refuses an MCP tool whose connection was
stored for another server.
2026-09-28 19:11:54 +01:00
arc53-machine 6b4220d401 Address review: MCP token scope, API-key identity, reconnect lock, setup retries
- An MCP connection's tokens only go to its own server, and a
  connection_id can no longer come from a client: the MCP test/save
  routes drop it and the tool executor uses only the resolved one.
- API-key connections are reused only when they hold the same
  credentials. Two keys that share a hint or a label get separate
  connections ("…abcd (2)"), at runtime and in the 0038 backfill.
- Reconnecting a connection whose secrets cannot be decrypted no longer
  flags it from a second transaction while holding its row lock.
- Connection setup validates the sync request before claiming its
  Idempotency-Key, so a corrected retry is queued.
- A request that names no connection resolves to none without opening
  a database connection; fixes two tests that reached a real database.
2026-09-28 18:45:27 +01:00
arc53-machine b692eaf7ad Add MCP presets, admin connector policies and connector attribution
Six vendor-run MCP servers (Notion, Linear, Atlassian, Sentry, Asana,
Stripe) join the catalog as presets from docsgpt/connectors/presets/
mcp.yaml; existing connections to those servers show under them.

connector_policies lets an admin turn a connector off or force whose
account every share uses, and app_metadata's
connectors.allow_custom_mcp turns custom MCP servers off. Both are
enforced on the server: new connections, OAuth sign-ins, MCP test and
save, and the tools of a disabled connector stop resolving.
GET/PUT /api/admin/connectors reads and changes them.

A shared tool's owner picks owner or member credentials with
PUT /api/connections/tools/<id>/credential-mode, and a member running
the owner's account always confirms write actions.

Tool-call events and retrieved chunks name the connector they came
from (key and display name, never the account), and tool calls keep
those fields when the conversation is reloaded.
2026-09-28 17:30:53 +01:00
arc53-machine 3511004f50 Connections own their credentials
Migration 0038 moves every stored secret (OAuth tokens, MCP OAuth
tokens and client registrations, API keys) into the connection's
encrypted envelope, links API-key tools to one connection per distinct
credential, allows several accounts per provider, and adds
credential_mode to sources and tools. OAuth MCP tools keep resolving
each member's own token, as they did before.

docsgpt.connectors.service is now the only reader of OAuth tokens:
get_valid_token_info refreshes under a row lock and persists rotated
refresh tokens, and a revoked grant flags the connection, pauses its
sources and notifies the owner. Loaders build from a connection
(BaseConnectorLoader.from_connection), so scheduled sync covers Drive,
SharePoint and Confluence sources with no browser. S3 and Reddit keys
stay on the connection instead of in remote_data.

New endpoints: POST /api/connections, /setup, /reconnect,
/picker-token, /claim, DELETE /api/connections/<id>, per-action
permissions and MCP refresh-tools. Upload, file listing, sync and
validate-session take a connection_id; session tokens keep working for
this release. The tool executor reads credentials from the resolved
connection (owner or member mode) and pauses on a Connect card when a
connection needs signing in. docsgpt connectors reencrypt rewrites
stored credentials after a key rotation.
2026-09-28 17:03:09 +01:00
arc53-machine 5321a94388 Add the connector catalog and read-only connections API
connector_sessions already holds one row per signed-in account or MCP
server. Migration 0038 names each row (connector_key, auth_kind,
display_name, account_label) and lets sources and tools point at the
row they use through connection_id, backfilling the links it can prove.

docsgpt/connectors/catalog.py declares every service DocsGPT connects
to, with the server settings it needs and what a connection provides.
New endpoints: GET /api/connectors/catalog, GET /api/connections,
GET /api/connections/<id> and POST /api/connections/<id>/disconnect.
/api/available_tools now groups tools into built_in, service and
custom, and source listings carry connectionId.
2026-09-28 16:30:48 +01:00
arc53-machine 934ae1afb3 fix(pat): close message replay to restricted tokens, filter builtin tools, reject non-object bodies
Message tail and the ASGI reconnect stream cannot tie a message to an
allowlist, so any token with a resource filter is refused there. The tools
listing now applies the allowlist to default and builtin rows as well. Token
creation answers 400 instead of 500 for a JSON body that is not an object.
2026-09-19 23:54:35 +01:00
arc53-machine 82ec6cba0e feat(pat): enforce scopes and resource restrictions, deny by default
A central rule table maps each route and method to the scope a token needs;
a route that is not listed cannot be called with a token, and a test fails
when a registered route is left unclassified. Token management, admin, team
management, sign-in, device pairing and OAuth handshakes are never token
reachable, and a token never carries the admin role.

A token restricted to specific agents, sources, prompts, tools or workflows
is held to its allowlist: ids are checked wherever a route carries them,
listings are filtered, creation is refused, and routes whose rows cannot be
tied to the allowlist are closed. Agent import checks the resolved target.
2026-09-19 23:39:58 +01:00
Alex 574f96341e refactor: rename the application package to docsgpt
The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.

Kept for one release:
- A top-level application package whose meta-path finder resolves
  application.x.y to the already-imported docsgpt.x.y object, so old imports
  and entry points (celery -A application.app.celery,
  uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
  docsgpt.* task on start-up, so messages queued by the previous release still
  run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
  the previous release wrote are left unread instead of firing twice.

The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
2026-09-07 10:20:43 +01:00