Commit Graph
5705 Commits
Author SHA1 Message Date
arc53-machine 212fa4f8dc Set a tool's permissions by group, with actions in words
What a tool looks up and what it does are each set at once to Allow, Ask
first or Off; long lists fold behind Customize, where each action shows its
name in words and its description. A note says what Ask first means over
an agent's API. The panel follows a refreshed tool's actions.
2026-09-29 00:54:07 +01:00
arc53-machine 5c2a45fb74 Show connections that need signing in again where their tools are picked
The chat and agent tool pickers mark a tool whose connection expired and
offer Reconnect, which redoes the sign-in in place (or opens the connector
page for a custom server). The connector page names the account without
claiming it is connected and says what went wrong in plain words, keeping
the provider's message on hover.
2026-09-29 00:54:07 +01:00
arc53-machine 97e37a7e37 Manage a connected tool on its connector page
A tool from a connection shows the catalog's description and a Manage
connection button instead of its own switch, and its menu no longer opens
the raw MCP server form to reconnect. It says when it is off or needs
signing in again.
2026-09-29 00:54:07 +01:00
arc53-machine cc7b07817f Strings for connected tools, sign-in prompts, grouped permissions and Knowledge
Adds the copy the next changes use and finishes the Sources to Knowledge
wording in connector flows: reconnect toasts, the disconnect and remove
confirmations, the wizard's done step and the Knowledge picker. Permission
choices read Allow / Ask first / Off everywhere.
2026-09-29 00:54:07 +01:00
arc53-machine 8cf46f2e57 Give a different account its own connection when reconnecting 2026-09-29 00:43:48 +01:00
arc53-machine cdc5084e02 Refuse a disabled connector before falling back to a legacy tool secret 2026-09-29 00:41:57 +01:00
arc53-machine a260ca44f1 Check a tool's connection before writing its permissions 2026-09-29 00:40:33 +01:00
arc53-machine 33f29cb408 Authenticate MCP token requests one way under Basic auth
With client_secret_basic the MCP SDK sets the Basic header and still sends
client_id in the body; Linear refuses that as two authentication methods.
Token and refresh requests now drop the body client_id when the Basic
header is set, as RFC 6749 describes.
2026-09-28 23:43:49 +01:00
arc53-machine 0d42916d67 Hand the MCP SDK an authorization result with the issuer
mcp 2.x reads code, state and the RFC 9207 iss off the callback handler's
result; the handler still returned a (code, state) tuple, so every MCP
sign-in failed after the user approved it. The callback route now keeps the
issuer too, since servers that advertise it (Linear) refuse a sign-in
without it.
2026-09-28 23:32:21 +01:00
arc53-machine cdaaaaf427 Prefer the configured MCP OAuth callback over the page's origin
A page opened on a plain-HTTP address registered its own origin as the
callback, which servers such as Linear refuse. MCP_OAUTH_REDIRECT_URI now
wins when set; the page's value is the fallback.
2026-09-28 23:20:44 +01:00
arc53-machine 46928cba3d Return the OAuth task id from the MCP test route
The client follows an MCP sign-in by its task's events, but the test route
kept only success, requires_oauth and auth_url, so the wizard and the custom
MCP modal reported a failed sign-in before the popup could load.
2026-09-28 23:16:42 +01:00
arc53-machine a9390ec39e Fix MCP OAuth for servers that publish their endpoint as the resource
Linear and Sentry publish https://host/mcp as their protected resource.
DocsGPTOAuth gave the MCP SDK only the server's origin, so the SDK's
resource check (RFC 8707) failed before sign-in with "Protected resource
https://mcp.linear.app/mcp does not match expected
https://mcp.linear.app". The provider now gets the full MCP URL; servers
that publish their origin still pass, and tokens stay keyed by origin.
2026-09-28 22:57:54 +01:00
arc53-machine 79fd2ac6cd A connector's page is the one home for its account, knowledge and tools
- Tools from a connection turn on and off on the connector's page (a
  real switch, not a badge), next to their action permissions.
- Knowledge synced from a connection lists its sync state there, with
  Sync now (connector sources and S3/Reddit sources each through their
  own endpoint).
- On the Tools and Knowledge pages, anything from a connection has
  "Manage connection" in its menu, which opens that page; the tool
  editor with its second credentials form is no longer offered for them.
- A link to a part (Jira & Confluence) opens its parent's page.
2026-09-28 22:38:49 +01:00
arc53-machine 3b1288097d One vocabulary: Knowledge, tools, and connectors in plain words
- Sources (the collection) are called Knowledge: the settings page (now
  /settings/knowledge; /settings/sources redirects), the composer and
  agent pickers, Add knowledge, and "syncs into Knowledge" on connectors.
  Citations keep "sources". All seven locales.
- Connector cards describe what they do ("Syncs into Knowledge", "Looks
  things up", "Takes actions") and drop publisher labels ("Built in",
  "Preset"). Filters read "Files & storage" and "Docs & wikis" so
  "knowledge" means one thing.
- Custom tools are tools: "Add custom tool". The agent picker no longer
  splits built-in tools into "Built-in" and "Default".
- Confluence is one card: syncing pages into Knowledge and the Jira &
  Confluence MCP actions (part_of on the preset) share a page. The part
  shows on its own when Confluence sync is not set up.
2026-09-28 22:31:26 +01:00
arc53-machine 897b244d8d Admin phone list: say On/Off once, setup state in the badge 2026-09-28 22:00:23 +01:00
arc53-machine 6cbd6091d4 Point people to connecting their data
- The empty chat screen swaps its last demo card for "Connect your
  data" while nothing is connected, naming services this install can
  connect.
- A warning dot on Settings and on Settings > Connectors marks a
  connection that needs signing in again; unlike the toast it stays until
  the connection is fixed. Connections load once at start for it.
- Category pills on the Connectors page follow the composer's capability
  filter, so none leads to an empty page.
2026-09-28 21:55:45 +01:00
arc53-machine df70869dc7 Simplify Admin > Connectors
- One control for members' own MCP servers: the custom MCP server row's
  switch (it sets both the policy and the instance switch); the separate
  top-level switch is gone.
- Connectors that only sync content show "No tools" instead of a
  sharing policy, and the policies read "The sharer decides per share",
  "Always the sharer's account", "Always each person's own account".
- A connector that needs server settings can't be switched on until they
  exist; a tooltip says why.
- On phones the table becomes a list ("On · 2 connections · …") and each
  connector's controls open in a sheet.
2026-09-28 21:52:18 +01:00
arc53-machine 23229181f1 One connect flow and one name from every entry point
- Add Source hands a connection tile to the connect wizard: an existing
  account goes straight to choosing what to sync, otherwise it connects
  one. The legacy per-service forms stay only for installs without
  connectors, so S3 no longer has two different forms.
- Add Tool lists services from the connector catalog first, MCP presets
  included, each with its connection state; a connected one opens its
  drawer, the rest start the wizard. Built-in tools follow.
- Service tools carry the connector's name everywhere ("Telegram", not
  "Telegram Bot"), and the composer groups custom tools (API tool, MCP
  servers) as Custom, like the agent builder.
- OpenAPI opens the API tool editor as an unsaved draft; the tool is
  created on its first save, so exploring the menu leaves nothing behind.
2026-09-28 21:49:01 +01:00
arc53-machine bfcc4a9773 Sign in to MCP presets from the connect wizard
Notion, Linear, Atlassian, Sentry, Asana and Stripe no longer open the
MCP server form. The wizard shows one "Sign in to Notion" button: the
pop-up opens inside the click (so browsers allow it), follows the
worker's mcp.oauth events to the provider, and the tool is saved on
success with its actions on the done step. Reconnecting a preset uses
the same flow, as do the chat bar and the health toast.

Saving an OAuth MCP server without a new handshake now uses the stored
sign-in instead of failing, and the custom MCP form keeps scopes and
timeout under Show advanced.
2026-09-28 21:42:50 +01:00
arc53-machine f6c89503d8 Reconnect in place from the chat bar and the health toast
- The Connect card's payload carries connection_id for the caller's own
  connection (never an owner's), so "Reconnect" in chat signs in again
  right there instead of sending the user to Settings. The bar offers
  Continue as soon as the connection works again, also after leaving and
  coming back to the chat.
- The reconnect event carries source and tool counts, and the toast says
  what stopped: "3 sources are paused", "so agents can use it again",
  or both. Reconnect signs in again in place and the toast closes itself
  once the connection works; MCP servers still reconnect from their
  drawer.
2026-09-28 21:37:19 +01:00
arc53-machine a23ace3491 Refuse writes on the owner's accounts from API-key callers unless allowed
An agent called with its API key (widget, API) runs as its owner, and
nobody can approve an action there, so a write set to Always allow ran
on the owner's account for anyone holding the key. A caller is external
when the request carries the key and is not signed in as the owner (an
owner previewing their agent keeps full access).

For external callers, write actions on connection-backed tools are
refused unless listed in the agent config's new api_write_allowlist
(tool_id:action), and a missing connection is refused instead of pausing
on a Connect card the widget cannot show. The flag and list survive a
paused-and-resumed stream. Owners pick the allowed actions under Access
details; a team editor's update keeps the stored list.
2026-09-28 21:33:41 +01:00
arc53-machine 7c6154749a Address review: model metadata, serialized admin saves, member account choice
- models.py: connector_policies.enabled is the nullable column; the
  earlier edit had loosened user_custom_models.enabled instead. A test
  pins both against their migrations.
- Admin > Connectors saves one change at a time, so an older snapshot
  can no longer overwrite a newer one.
- In member mode, a member with several connected accounts of the same
  service uses the one they used (or connected) most recently, not the
  oldest.
2026-09-28 20:59:36 +01:00
arc53-machine ca8fb94777 Bring the connectors UI in line with DESIGN.md
From a design review against frontend/DESIGN.md:
- Drawer panels use the subtle surface; ToolPermissions takes the
  surface it sits on (outline in the wizard, subtle in the drawer).
- The OAuth sign-in is the wizard's footer action with a Cancel
  (useConnectorAuth), and sign-in buttons show the service's own glyph
  instead of a hard-coded Drive logo. Add Tool's links moved into its
  ModalActions footer.
- Failed saves, disconnects and connector creates say so in a toast;
  the admin page reports save errors as a toast too.
- Capability badges are neutral; status hues are left for states.
- Row titles use SectionHeader actions, setup settings a DescriptionList,
  empty lists EmptyState, retry and row buttons their pill roles.
- The drawer names its close button and describes itself with
  SheetDescription; counts are formatted for the app language.
- One ConnectorSetupNotice for the drawer and Upload; warning Alerts use
  TriangleAlert; its link keeps the Alert colour (Approved exceptions).
- Tools and Sources tiles show the same "via <service>" meta line; the
  paused badge explains itself in a Tooltip.
- On phones the category filter is a Select, empty categories are not
  offered, and a composer's capability filter says so with Show all.
- The admin page uses PageToolbar and SectionHeader descriptions, and is
  English throughout.
2026-09-28 20:33:34 +01:00
arc53-machine a564dea401 Hide connectors that need admin setup; they start turned off
A connector's enabled switch is now optional: unset means on when the
connector has the server settings it needs, so Google Drive, SharePoint
and Confluence start off until their OAuth settings are present, and an
admin's explicit switch always wins. Changing only the sharing mode no
longer switches a connector on.

Members only see connectors they can use. The catalog, Add Source tiles
and Add Tool leave out anything turned off or still needing setup, except
a connector a member already has a connection to, which stays listed as
turned off so it can be managed or removed.
2026-09-28 20:16:40 +01:00
arc53-machine 324d54ea67 Validate configs on the existing-connection path; MCP policy fails closed
- Creating a tool from an existing connection validates its config too;
  the connection supplies the secrets the request leaves out.
- When the connector policies cannot be read, the MCP test and save
  routes answer 503 instead of contacting the server.
2026-09-28 19:47:28 +01:00
arc53-machine 473e8ec15c Validate tool edits before touching the connection; scope kept MCP connections
- Editing a connection-backed tool validates the config before writing
  new credentials, so a rejected edit no longer rotates the connection's
  key and resumes its sources.
- An MCP edit keeps its previous connection only when the caller owns it
  and it signs in the way the tool now does.
2026-09-28 19:31:02 +01:00
arc53-machine 4d533d24d6 Test connection-backed tools, MCP refresh and API failure paths
Covers creating and editing tools whose key lives on a connection, the
MCP action refresh, and the auth, ownership and error branches of the
connections API. Editing a tool's key now also recovers a connection
whose stored credentials can no longer be decrypted.
2026-09-28 19:11:54 +01:00
arc53-machine fff3e7c904 Never send an MCP connection's secret to a different server
Editing an MCP server's URL kept the previous connection when no new
one could be stored (default encryption key), so its key would reach the
new server. The save route now keeps a previous connection only for the
same server, and the executor refuses an MCP tool whose connection was
stored for another server.
2026-09-28 19:11:54 +01:00
arc53-machine 269aa13bba Address review: tool group order in the agent builder, saved S3/Reddit account
- Connection groups sit between the default and custom tools; a second
  sort that did not know connection labels moved them to the top.
- Upload picks the first saved account when connections load after the
  modal opened on S3 or Reddit, without overriding the user's choice.
2026-09-28 18:45:27 +01:00
arc53-machine 6b4220d401 Address review: MCP token scope, API-key identity, reconnect lock, setup retries
- An MCP connection's tokens only go to its own server, and a
  connection_id can no longer come from a client: the MCP test/save
  routes drop it and the tool executor uses only the resolved one.
- API-key connections are reused only when they hold the same
  credentials. Two keys that share a hint or a label get separate
  connections ("…abcd (2)"), at runtime and in the 0038 backfill.
- Reconnecting a connection whose secrets cannot be decrypted no longer
  flags it from a second transaction while holding its row lock.
- Connection setup validates the sync request before claiming its
  Idempotency-Key, so a corrected retry is queued.
- A request that names no connection resolves to none without opening
  a database connection; fixes two tests that reached a real database.
2026-09-28 18:45:27 +01:00
arc53-machine 3e11442a83 Merge remote-tracking branch 'origin/main' into connectors
# Conflicts:
#	frontend/DESIGN.md
#	frontend/src/locale/de.json
#	frontend/src/locale/en.json
#	frontend/src/locale/es.json
#	frontend/src/locale/jp.json
#	frontend/src/locale/ru.json
#	frontend/src/locale/zh-TW.json
#	frontend/src/locale/zh.json
#	frontend/src/upload/Upload.tsx
2026-09-28 18:07:12 +01:00
arc53-machine 8f00118afc Document connectors and the upgrade to encrypted connections
- Guides > Connectors: using connections, sharing modes, attribution,
  and admin setup (encryption key and rotation, redirect URIs, Google
  Drive, SharePoint, Confluence, S3, MCP presets).
- Integration pages register CONNECTOR_REDIRECT_BASE_URI as-is; the
  ?provider= suffix never matched what the backend sends.
  VITE_GOOGLE_CLIENT_ID is optional.
- Upgrading: set ENCRYPTION_SECRET_KEY before migrating multi-user installs.
2026-09-28 17:59:36 +01:00
arc53-machine d9f823d601 Stop syncing sources of a connector an admin turned off
Scheduled syncs checked the connection's status but not the admin
policy, so a disabled connector kept pulling data. Connector sources now
return "disabled" and S3/Reddit syncs get no credentials until the
connector is turned back on.
2026-09-28 17:59:36 +01:00
arc53-machine 2411118048 Show which service did what, and let owners pick whose account a share uses
- Tool chips and the streaming status name the service: "Searched
  Notion", "Used Linear: create issue". Citations from a connector
  source read "From Google Drive".
- The share dialog for a connection-backed tool asks whose account team
  members use. Owner mode warns which account they act as and needs an
  explicit OK when the tool can write; an admin-forced mode is locked.
- Admin > Connectors: enable or disable each connector, force a sharing
  mode, allow custom MCP servers, and a setup guide per OAuth connector
  with its redirect URI and the server settings still missing. Warns
  when credentials use the default encryption key.
- The composer and the agent builder group tools by connection, and the
  Sources and Tools popovers link to Connect more.
- MCP presets open the server form titled after the service.
2026-09-28 17:55:56 +01:00
arc53-machine cefb117277 Keep tool keys when rolling back 0038
Downgrade removes API-key connections and assumed every tool still had
its v1 copy. Tools added through the wizard after the upgrade never had
one, and a reconnect can change the key after the backfill, so write the
connection's current credentials back to each linked tool first.
2026-09-28 17:55:56 +01:00
arc53-machine b692eaf7ad Add MCP presets, admin connector policies and connector attribution
Six vendor-run MCP servers (Notion, Linear, Atlassian, Sentry, Asana,
Stripe) join the catalog as presets from docsgpt/connectors/presets/
mcp.yaml; existing connections to those servers show under them.

connector_policies lets an admin turn a connector off or force whose
account every share uses, and app_metadata's
connectors.allow_custom_mcp turns custom MCP servers off. Both are
enforced on the server: new connections, OAuth sign-ins, MCP test and
save, and the tools of a disabled connector stop resolving.
GET/PUT /api/admin/connectors reads and changes them.

A shared tool's owner picks owner or member credentials with
PUT /api/connections/tools/<id>/credential-mode, and a member running
the owner's account always confirms write actions.

Tool-call events and retrieved chunks name the connector they came
from (key and display name, never the account), and tool calls keep
those fields when the conversation is reloaded.
2026-09-28 17:30:53 +01:00
arc53-machine 6e2ee43944 Connect services from one wizard, with no tokens in the browser
Every entry point (the Connectors page and drawer, Add Source, Add
Tool, the chat) opens one connect wizard: sign in or paste the
credentials, choose what to sync (content services only, skippable),
then a summary with Try it in chat. Tool services create their tools on
sign-in with write actions needing approval, and each action's
permission (Always allow, Needs approval, Off) can be changed in the
wizard or the drawer. The drawer also reconnects, syncs more content,
refreshes an MCP server's tools and removes a connection, keeping or
deleting what it feeds.

The file pickers work from a connection id and fetch a short-lived
picker token when they need one; no session token is kept in
localStorage any more, and any left by an older build is claimed once
and removed. S3 and Reddit reuse the keys stored on a connection.

A tool whose account needs signing in pauses on a Connect card in the
chat; after connecting, Continue resumes the call. A connection whose
sign-in stops working raises a Reconnect toast. The Connectors page is
hidden when the backend does not report connectors_enabled.
2026-09-28 17:21:44 +01:00
arc53-machine 3511004f50 Connections own their credentials
Migration 0038 moves every stored secret (OAuth tokens, MCP OAuth
tokens and client registrations, API keys) into the connection's
encrypted envelope, links API-key tools to one connection per distinct
credential, allows several accounts per provider, and adds
credential_mode to sources and tools. OAuth MCP tools keep resolving
each member's own token, as they did before.

docsgpt.connectors.service is now the only reader of OAuth tokens:
get_valid_token_info refreshes under a row lock and persists rotated
refresh tokens, and a revoked grant flags the connection, pauses its
sources and notifies the owner. Loaders build from a connection
(BaseConnectorLoader.from_connection), so scheduled sync covers Drive,
SharePoint and Confluence sources with no browser. S3 and Reddit keys
stay on the connection instead of in remote_data.

New endpoints: POST /api/connections, /setup, /reconnect,
/picker-token, /claim, DELETE /api/connections/<id>, per-action
permissions and MCP refresh-tools. Upload, file listing, sync and
validate-session take a connection_id; session tokens keep working for
this release. The tool executor reads credentials from the resolved
connection (owner or member mode) and pauses on a Connect card when a
connection needs signing in. docsgpt connectors reencrypt rewrites
stored credentials after a key rotation.
2026-09-28 17:03:09 +01:00
arc53-machine 667d4bbab0 Encrypt connection credentials with an owner-bound AES-GCM envelope
Adds a v2 credential envelope next to the v1 tool-secret helpers:
AES-256-GCM, a master key derived once per process from
ENCRYPTION_SECRET_KEY, and a per-record key from HKDF over the owner's
id, which is also the associated data, so a blob moved onto another
user's row does not decrypt. The envelope names its key, so
ENCRYPTION_SECRET_KEY_PREVIOUS keeps old rows readable during a
rotation.

Log redaction now also covers token_info, tokens and client_info, and
the API warns at startup when the public default key is in use.
2026-09-28 17:03:02 +01:00
arc53-machine 45c4d928c4 Group Add Source and Add Tool by where things come from
Add Source splits into Upload and web, and From a connection. Connector
tiles are always shown and say whether an account is connected or the
server still needs setup, instead of disappearing when a VITE_ client
id is unset; without the Google Picker key, Drive falls back to the
server-side file browser. The name follows the first picked item and
the button reads Add source.

Add Tool splits into Built in and From a service. The MCP and API tool
cards move to Connectors > Add custom connector.
2026-09-28 16:30:56 +01:00
arc53-machine aad43c2fd0 Add the Connectors page
Settings > Workspace gets a Connectors page between Sources and Tools:
a searchable catalog filtered by category, one card per service with
its capabilities (Sync, Read, Write) and state (Connect, Connected,
Reconnect, Needs admin setup, Disabled by admin, Custom). A card opens
a drawer listing the accounts, the sources each syncs and the tools
each provides, with Reconnect and Disconnect.

Tools and Sources show which connection an item comes from and when
its sync is paused. Brand icons now draw in currentColor so they
follow the theme, and all new copy is translated in every locale.
2026-09-28 16:30:56 +01:00
arc53-machine 5321a94388 Add the connector catalog and read-only connections API
connector_sessions already holds one row per signed-in account or MCP
server. Migration 0038 names each row (connector_key, auth_kind,
display_name, account_label) and lets sources and tools point at the
row they use through connection_id, backfilling the links it can prove.

docsgpt/connectors/catalog.py declares every service DocsGPT connects
to, with the server settings it needs and what a connection provides.
New endpoints: GET /api/connectors/catalog, GET /api/connections,
GET /api/connections/<id> and POST /api/connections/<id>/disconnect.
/api/available_tools now groups tools into built_in, service and
custom, and source listings carry connectionId.
2026-09-28 16:30:48 +01:00
Pavel ac7bbd811f Merge pull request #2842 from arc53/Agent-menu-UI
Agent menu UI
2026-09-28 17:20:26 +02:00
Pavel b12f5b6c81 Chunk nav fixed 2026-09-28 19:02:50 +04:00
Pavel f4331cd3a3 rabbit fixes 2026-09-28 18:40:49 +04:00
Alex 335f2af70c Merge pull request #2841 from arc53-machine/default-chunks-6
Default retrieval to 6 chunks instead of 2
2026-09-28 15:19:02 +01:00
arc53-machine 56aebc882c Cover the chunks fallbacks on agent update and YAML import
A blank chunks on update and an unparsable one on import both fall back
to the default.
2026-09-28 14:55:43 +01:00
arc53-machine c40f06e10b Say that chunks can be exceeded when sources outnumber it
Every attached source gets at least one chunk, so eight sources at
chunks: 6 return eight.
2026-09-28 14:45:06 +01:00
Pavel cdbc4695ea main sync 2026-09-28 17:41:29 +04:00
Pavel 706a0cb2b2 Big source revamp 2026-09-28 17:35:16 +04:00