diff --git a/docs/content/Agents/basics.mdx b/docs/content/Agents/basics.mdx index 2f29d546..7f9cccec 100644 --- a/docs/content/Agents/basics.mdx +++ b/docs/content/Agents/basics.mdx @@ -68,6 +68,8 @@ A multi-phase agent designed for in-depth research tasks: Includes budget controls for max steps, timeout, and token limits to keep research bounded. +Research steps can't stop to ask you anything, so a tool action that needs approval, a connection or the user's app isn't run there; the step is told why and carries on. The same goes for actions an API, widget or public-link user can't take on the owner's accounts. + **Best for:** Complex questions that require multi-step investigation, gathering information from multiple sources, and producing structured reports with citations. ### 4. Workflow Agent @@ -111,6 +113,7 @@ Once an agent is created, you can: * Modify any of its configuration settings (name, description, source, prompt, tools, type). * **Generate a Public Link:** From the edit screen, you can create a shareable public link that allows others to import and use your agent. * **Get a Webhook URL:** You can also obtain a Webhook URL for the agent. This allows external applications or services to trigger the agent and receive responses programmatically, enabling powerful integrations and automations. +* **Share it with a team:** Viewers chat with it; editors can also change it. The share dialog lists what the agent uses and whose access each item runs with. See [Sharing agents and what they use](/Deploying/Access-Control#sharing-agents-and-what-they-use). * **Use it via API:** Every agent exposes an API key that can be used with the native [Agent API](/Agents/api) or the [OpenAI-Compatible API](/Agents/openai-compatible) so you can drop DocsGPT Agents into any tool that already speaks the chat completions protocol. ## Seeding Premade Agents from YAML diff --git a/docs/content/Agents/guardrails.mdx b/docs/content/Agents/guardrails.mdx index eb645725..cf0e8c1a 100644 --- a/docs/content/Agents/guardrails.mdx +++ b/docs/content/Agents/guardrails.mdx @@ -172,7 +172,7 @@ Open the agent in the builder and expand the **Guardrails** section. Checks that cannot run without settings (`denylist`, `url`, `policy`, and `pii` with no entities selected) are marked *Not configured* and block saving until they are filled in, so a half-configured control can never be published as if it were protecting you. -Guardrails are the **agent owner's** policy. Team members with edit access can see the configuration but cannot change it — an editor who could clear a control would silently strip protection from everyone using the agent. Controls required by the instance floor (below) appear locked and cannot be removed. +Guardrails, like the agent's limits, are policy that the owner and the agent's team **editors** can change. Viewers don't see the configuration. To keep a control that nobody who edits the agent can remove, require it in the [instance floor](#the-instance-floor): floor controls appear locked for everyone. See [Sharing agents and what they use](/Deploying/Access-Control#sharing-agents-and-what-they-use) for what else viewers and editors can do. Guardrails also apply to a **draft** agent in the builder preview, which is the natural place to try a control before publishing. diff --git a/docs/content/Deploying/Access-Control.mdx b/docs/content/Deploying/Access-Control.mdx index 208c378e..dc62ef77 100644 --- a/docs/content/Deploying/Access-Control.mdx +++ b/docs/content/Deploying/Access-Control.mdx @@ -138,17 +138,95 @@ Four resource types can be shared: **agents**, **sources**, **prompts**, and **t Sharing rules: -- Only the **owner** of a resource can share it. +- Only the **owner** of a resource can share it, unless they turn on **Editors can share** in the share dialog's **Access settings**. - A share targets either the **whole team** or a **single member**. - Each share carries an access level: **`viewer`** (read-only) or **`editor`** (read and modify). -- `editor` is not the same as owner — an editor can change a resource but cannot delete it or re-share it. -- Shared tools run server-side with the **owner's** credentials; a grantee never sees the owner's secrets. +- `editor` is not the same as owner — an editor can change a resource but cannot re-share it unless the owner turns on **Editors can share**, or delete it unless the owner turns on **Editors can delete** (agents and sources only; editors never delete tools or prompts). +- Shared tools run server-side with the **owner's** credentials, or with each member's own account when a connected tool is shared that way; a grantee never sees the owner's secrets. +- A wiki's editors can edit its pages, but only its owner decides whether API and widget users can edit it through an agent (see [Wiki sources](/Sources/Wiki-sources#edits-from-the-api-widget-and-public-links)). + +### Sharing agents and what they use + +Sharing an agent lets people use it; it doesn't share its tools, sources or prompt. Each stays with whoever owns it, and people reach them only through the agent. + +- **Viewers** chat with the agent. They don't see its configuration, and they never open its share dialog. +- **Editors** open its edit page and change it: its instructions, model, tools, sources and prompt, and its [guardrails](/Agents/guardrails) and limits. They can share it or delete it only when you turn on **Editors can share** or **Editors can delete**. + +Every tool, source and prompt on the agent runs with one person's access for everyone who uses the agent. The exception is a connected tool shared as **Each person's own**, which uses the account of whoever is chatting. The agent's share dialog lists them under **What this agent uses**, for you and its editors. For a workflow agent, the list also includes the tools and sources on its nodes. + +| The item | Runs with | The list says | +| --- | --- | --- | +| Yours, or shared with you | Your access | **Your access** (an editor sees **The owner's access**) | +| Added by an editor while you can't use it | The editor's access; they are its [sponsor](#resources-an-editor-adds-to-someone-elses-agent). Once you can use it, it runs with yours again | **dana@example.com's access** | +| A tool with saved credentials (an API key, an MCP sign-in) | Its owner's credentials | **Your saved credentials**, or **dana@example.com's saved credentials** for a teammate's tool | +| A connected tool shared as **Your account** | That account on the service | **Your Notion account**, or **dana@example.com's Notion account** for a teammate's tool | +| A connected tool shared as **Each person's own** | The account of whoever is chatting; they connect it the first time. API and widget users run the agent as its owner, so they get the owner's account | **Each person's own Notion account (API and widget: yours)** | + +The list and the edit page name a person only when you know them already: you, the agent's owner, anyone who shares a team with you and, when you own the agent, whoever sponsored something on it. An item's owner is named for its account or credentials, or as the person to ask about it, only when you can also see that item. Anyone else shows as someone else, for example **Someone else's access**. + +An editor becomes a sponsor only after confirming who will reach the resource through the agent. A resource whose sponsor or owner loses access stops running and is marked **Stopped** in the list, with the reason (see [When a resource stops working](#when-a-resource-stops-working)). + +Some runs are limited, because nobody can approve an action for you there: + +- **API key, website widget and public link.** A write action on your connected accounts or saved credentials runs only if you allow it under **Access details > Actions API, widget and public-link users can take as you**. Only the owner can change that list. It offers every such write on the agent, including tools an editor sponsored and the tools on a workflow's nodes. The share dialog marks a tool **API changes off** when none of its writes are allowed, **Some API changes off** when only some are, and **Changes off by an admin** when an admin turned off changes through its service. On a tool shared as **Each person's own**, public-link users act with their own account, so the list doesn't limit them there; a write that needs approval still asks them first. See [Agents used through an API key](/Guides/Connectors#agents-used-through-an-api-key). +- **Wikis.** API and widget users edit a wiki only when its owner turns on **Let API and widget users edit this wiki** in its **Wiki settings**. Public-link users edit only wikis they can edit themselves, and approve each edit (see [Wiki sources](/Sources/Wiki-sources#edits-from-the-api-widget-and-public-links)). +- **Research agents.** A research step can't stop to ask, so it skips any action that would need approval or a connection, and any write the caller may not make on your accounts. The step is told why and carries on (see [Research Agent](/Agents/basics#3-research-agent)). + +### Resources an editor adds to someone else's agent + +An agent (and its workflow) runs as its owner, so its tools, sources and prompts are checked against the owner's access. When a team editor adds one the owner can't use, it runs with the **editor's** access instead, for everyone who uses the agent: members of the teams it is shared with, anyone with its API key or website widget, its public link and its webhook. The editor becomes that resource's **sponsor**. + +- Only someone who **owns** the resource, or has **`editor`** access to it through any team, can sponsor it. `viewer` access lets you use a resource in your own agents, but not extend it to another agent's users: the save is refused with `403` and `code: "sponsor_not_allowed"`. +- Sponsoring is never implied. A save that would make you a new sponsor is refused with `409` until you confirm it. In DocsGPT, a dialog names each resource and who will reach it through the agent. Through the API, send the save again with `confirm_sponsor` listing every resource from the response as `":"`. A `confirm_sponsor` entry for anything the save doesn't ask you to sponsor is refused with `400`. +- A sponsored resource stops running when its sponsor can no longer edit the agent, or no longer owns or edits the resource. It stays on the agent but does nothing, and it doesn't pass to whoever saves the agent next. The agent's edit page says why it stopped. An editor who may sponsor it can choose **Run … with my access** there; after they confirm in the same dialog that names who reaches the agent, it runs with their access from their next save. Through the API, include its key in `confirm_sponsor` on any save. Otherwise, someone removes it. +- A resource that is removed and later added again needs a new confirmation, even if its old sponsor could still sponsor it. +- Editors can remove any tool, source or prompt from the agent or its workflow nodes, including the owner's private ones they can't open. +- Workflow nodes follow the same rules, through `PUT /api/workflows/`. The owner's own saves are checked too: a node can't name a tool or source its owner can't use. +- The confirmation covers the audience the agent has at that moment. If the owner later shares the agent with more teams, or turns on a public link, sponsored resources reach those people too, and their sponsors aren't asked again. +- `audience.teams` lists every team the agent is shared with, including teams where only some members were given access. + +A save that needs confirmation returns: + +```json +{ + "success": false, + "code": "sponsor_confirmation_required", + "message": "These resources would run with your access for everyone who uses this agent. Confirm to add them.", + "resources": [{ "key": "tool:", "type": "tool", "id": "", "name": "Jira" }], + "audience": { "teams": ["Support"], "api_key": true, "public_link": false, "webhook": false } +} +``` + +Owners and editors see sponsored resources on the agent's edit page and in `resource_sponsors` from `GET /api/get_agent` (and `GET /api/workflows/`). Viewers get an empty list. Each entry has the resource (`key`, `type`, `id`, `name`), the sponsor (`user_id`, `label`, both `null` for someone you don't know; see [Sharing agents and what they use](#sharing-agents-and-what-they-use)), `state` (`active` or `inactive`), `reason` when inactive (`sponsor_cannot_edit_agent` or `sponsor_cannot_edit_resource`), and `can_confirm`, which says whether you may take an inactive one over. + + +After upgrading, resources sponsored by someone with only `viewer` access to them stop running. An editor who owns or can edit such a resource can choose **Run … with my access** on the agent's edit page to start it again. + + +### When a resource stops working + +Every run checks each tool, source and prompt on the agent (and each tool and source on its workflow nodes) against the owner's access, or the sponsor's. One that no longer passes is left out of the run; a prompt falls back to the default prompt. A connected tool whose account needs attention is different: it stays on the agent but can't run until someone fixes the account. In a chat, calling it shows a card asking to connect the account; in a scheduled run, a webhook or an API call, the call is refused. The agent's edit page, and the workflow builder for node resources, lists each resource that stopped or can't run, with the reason and what you can do: + +| Reason | What happened | What you can do | +| --- | --- | --- | +| `deleted` | The tool was deleted. Left out of runs. | Remove it. | +| `owner_lost_access` | The owner can no longer use it, for example a team stopped sharing it with them. Left out of runs. | Ask its owner to share it again, choose **Run … with my access** if you may sponsor it, or remove it. | +| `sponsor_cannot_edit_agent`, `sponsor_cannot_edit_resource` | Its sponsor lost access (see above). Left out of runs. | Choose **Run … with my access** if you may sponsor it, or remove it. | +| `connection_needs_reconnect` | The account the tool uses was disconnected or needs signing in again. It can't run until someone signs in again. | If it is your account, choose **Reconnect**; otherwise ask the tool's owner. | +| `connection_removed` | The tool's connection was removed but the tool was kept, and it has no credentials of its own. It can't run until the account is connected again. | Ask the tool's owner to connect the account again, or remove it. | +| `connector_disabled` | An admin turned the service off. It can't run until it is turned back on. | Ask an admin to turn it back on, or remove it. | + +Only tools that run on their owner's account (owner mode) are checked this way. A tool shared in member mode runs on each person's own account, so the owner's account doesn't decide whether it runs: it is listed as running, with the note that each person uses their own account, and whoever runs it is asked to connect their own account when they need to. + +**Remove** takes the item off the form; save to store it. Through the API, `GET /api/get_agent` and `GET /api/workflows/` return `resource_states` to owners and editors (an empty list to everyone else): one entry per attached resource with `key`, `type`, `id`, `name`, `state` (`active` or `stopped`), `reason`, `note` (`per_user_account` for a running member-mode tool), `sponsor` (`{user_id, label}`), `contact_role` and `contact`, `connection`, `can_confirm` and `can_reconnect`. `contact_role` is `resource_owner` when the resource's owner can fix it; `contact` names them (`{user_id, label}`) when you know them, and is `null` otherwise. `connection` names the service; its `id` comes only with `can_reconnect`, and the account's own name only for its owner. `runs_as` is the live sponsor a running resource runs as (`null` when it runs as the owner). A running tool also has `credential_mode` (`owner` or `member` for a connected tool, else `null`), `account` (whose saved credentials or `owner`-mode connection it uses), `owner_credential_writes` (its write actions on those credentials, which the API write allowlist covers) and `writes_allowed` (`false` when an admin turned off changes through its service). When anything can be taken over, the response also carries `sponsor_audience`, the same shape as `audience` above. People are named by the rule in [Sharing agents and what they use](#sharing-agents-and-what-they-use): `sponsor`, `runs_as` and `account` have `user_id` and `label` set to `null` for someone you don't know, and `contact` is `null`. A resource's name is shown only for a resource that runs, that someone sponsored, that you can see yourself, or that is on an agent (agent saves check every reference). If the run state can't be worked out, the read still succeeds with an empty list. + +The state comes from the checks a run uses, so a source, prompt or tool the page shows as running is one a run uses, and one shown as stopped is left out of runs or can't run until it's fixed, as the table says. Each resource a run leaves out is logged as `resource_stopped` with the agent or workflow, the resource's type and id, and the reason. ## Audit log Access-control actions are appended to the `auth_events` table alongside the [authentication events](/Deploying/OIDC-SSO#login-auditing). This includes admin actions — `admin_user_activated` / `admin_user_deactivated`, `admin_sessions_revoked`, `role_granted` / `role_revoked` (with `metadata.source` = `manual` or `oidc_group`), `quota_policy_set` / `quota_policy_deleted` — and team events (`team.create`, `team.member_add`, `team.member_role`, `team.member_remove`, `team.share`, `team.unshare`, `team.transfer_owner`, `team.delete`). -Data-plane actions are recorded too: `source.created` / `source.deleted` / `source.reingested`, `agent.created` / `agent.updated` / `agent.deleted` / `agent.key_regenerated`, and `conversation.deleted` / `conversation.deleted_all`. +Data-plane actions are recorded too: `source.created` / `source.deleted` / `source.reingested` / `source.wiki_settings_updated`, `agent.created` / `agent.updated` / `agent.deleted` / `agent.key_regenerated`, and `conversation.deleted` / `conversation.deleted_all`. Every row carries `actor_id` (who did it) and `target_id` (the user it was done to, or `NULL` when the event is not about a user), so "everything this admin did" is a single query: diff --git a/docs/content/Deploying/Settings-Reference.mdx b/docs/content/Deploying/Settings-Reference.mdx index b08af68a..e6635aa2 100644 --- a/docs/content/Deploying/Settings-Reference.mdx +++ b/docs/content/Deploying/Settings-Reference.mdx @@ -34,7 +34,13 @@ Signing key for session tokens and other signed capabilities. Required on every Type `str`, default `default-docsgpt-encryption-key`. -Key used to encrypt stored credentials such as tool and connector secrets. +Key used to encrypt stored credentials such as tool and connector secrets. Set your own value before connecting services on a multi-user install; the default is public. + +### `ENCRYPTION_SECRET_KEY_PREVIOUS` + +Type `str`, default unset. + +Previous ENCRYPTION_SECRET_KEY, tried when a stored credential was encrypted with it. Set it while rotating the key, run `docsgpt connectors reencrypt`, then remove it. ### `INTERNAL_KEY` @@ -1134,7 +1140,25 @@ Confluence Cloud OAuth client secret. Type `str`, default unset. -GitHub PAT with read access to repositories. +Instance-wide GitHub token for the public-repository upload. It raises GitHub's rate limit and is never used to read a private repository; users connect their own GitHub account for those. + +### `GITHUB_CLIENT_ID` + +Type `str`, default unset. + +GitHub App client id. With the secret and slug, offers Sign in with GitHub next to tokens. + +### `GITHUB_CLIENT_SECRET` + +Type `str`, default unset. + +GitHub App client secret. + +### `GITHUB_APP_SLUG` + +Type `str`, default unset. + +GitHub App URL name (github.com/apps/<slug>), for the link where users choose repositories. ### `MCP_OAUTH_REDIRECT_URI` diff --git a/docs/content/Guides/Connectors.mdx b/docs/content/Guides/Connectors.mdx new file mode 100644 index 00000000..fa82bfcb --- /dev/null +++ b/docs/content/Guides/Connectors.mdx @@ -0,0 +1,264 @@ +--- +title: Connectors +description: Connect DocsGPT to the services your team uses. One connection can sync content into Knowledge and give agents tools, with credentials encrypted on the server. +--- + +import { Callout } from 'nextra/components' +import { Steps } from 'nextra/components' + +# Connectors + +A connector is a service DocsGPT can connect to: Google Drive, SharePoint, Confluence, GitHub, Amazon S3, Reddit, Brave Search, Telegram, ntfy, PostgreSQL, curated MCP servers (Notion, Linear, Atlassian, Sentry, Asana, Stripe), and any MCP server or OpenAPI spec you add yourself. A **connection** is one signed-in account or one saved API key for a connector. + +Connections are the single place credentials live. A synced source and an agent tool both point at a connection, so you sign in once, reconnect once when a token expires, and disconnect in one place. + +## Using connectors + +Open **Settings > Connectors**. Filters narrow the list by category, and **Connected** shows what you already use. + +1. Pick a connector. Services with OAuth open the provider's sign-in page in a pop-up; API-key services ask for the key. +2. Choose what the connection sets up. Tool services create their tools right away. Services that can sync ask whether to **Sync into Knowledge**: off by default, on when you started from **Add knowledge** or from Knowledge's **Connect a service**. Left off, the account is still connected and you can sync later with **Sync more content** on its page. Turned on, pick files or folders, how often to sync (never, daily, weekly or monthly) and, under **Advanced retrieval settings**, the same chunking and retrieval options as an upload. +3. Adjust what the tools may do. Each action is **Read** or **Write**, and each can be **Always allow**, **Needs approval** or **Off**. Writes default to needing approval. + +**Knowledge** (formerly Sources) is what the assistant searches: uploads, links, and content synced from your connections. The **Add knowledge** and **Add Tool** dialogs, the composer's Knowledge and Tools menus, and the agent builder all start the same flow, and list tools grouped by the connection they use. A service offered two ways appears as one card: Confluence syncs pages into Knowledge and, through **Jira & Confluence**, lets agents search and update Jira and Confluence. GitHub and Linear do both from one connection: see [GitHub](#github) and [Linear](#linear). + +A connector's page is the one place to manage it: its accounts (**Reconnect**, **Rename**, **Disconnect**), the Knowledge it syncs (**Sync now**), and its tools (on or off, and what each action may do). On the Knowledge and Tools pages, anything from a connection has **Manage connection** in its menu, which opens that page. Disconnecting keeps synced content but stops its sync. + +### Several accounts of one service + +You can connect the same service more than once, for example two Telegram bots. Give each account a name when you connect it (**Name this account**) or later with **Rename**. When you have more than one account of a service, its tools are listed as "Telegram · Alerts bot", and the assistant sees each account's actions under that account's name, so it can pick the right one. + +### Fixed values + +Under **Customize**, each action lists its parameters. A parameter is either left to the AI (**Let AI decide**) or set to **Always use** a value. A fixed value is sent on every call; the AI never sees it and can't change it, even when a prompt asks it to. Only the tool's owner can change fixed values. + +A Telegram connection can hold a **Default chat ID**. When it's set, messages always go to that chat. To find a chat's id, add the bot to the chat, send it a message, and open `https://api.telegram.org/bot/getUpdates`. The same bot with another default chat is a separate connection. When team members use their own accounts, each member's messages go to their own default chat. + +### When a connection needs attention + +If a provider rejects a token that DocsGPT cannot refresh, the connection is marked **Reconnect needed**, its synced sources pause, and you get a notification. Reconnecting the same account resumes them. In a chat, a tool whose connection is missing or expired shows a **Connect** prompt instead of failing; the answer continues once you connect. + +### Sharing a tool with a team + +Sharing a tool that uses a connection asks whose account team members use: + +- **Your account.** Everyone acts as your account on that service. When the tool can write, you confirm this before sharing, and a member's write actions always need approval. +- **Each person's own.** Each person uses their own account, and sees a **Connect** prompt the first time they use the tool. Through an agent's API key or widget, the agent runs as its owner, so the tool uses the agent owner's account. + +An editor you let share the tool sees your choice as **The owner's account** and can't change it. An agent's share dialog shows the same words for each connected tool on the agent (see [Sharing agents and what they use](/Deploying/Access-Control#sharing-agents-and-what-they-use)). + +Tools from OAuth MCP servers default to each member's own account. An admin can force one mode for a connector. + +### Agents used through an API key + +An agent called with its API key (the website widget, the API) runs with its owner's connections, and nobody can approve an action there. It can read through those connections, but it can't take write actions on them unless the owner allows each one under **Access details > Actions API, widget and public-link users can take as you**. The same goes for tools that hold the owner's own credentials without a connection: an API tool action that sends a header or query value the owner saved (a key or token), or an MCP server the owner signed in to. The list offers every such write on the agent, including tools a team editor sponsored and the tools on a workflow's nodes. Only the owner can change that list; team editors can't. The owner previewing their own agent in DocsGPT is not limited. + +The same list covers people who open the agent from its public link without being on a team it's shared with, and anything they schedule from that chat. They can't approve write actions on the owner's accounts or credentials, so those run only when the owner allows them there. On a tool where each person connects their own account, they act with their own account, so the list doesn't limit them there; a write that needs approval still asks them first. + +Schedules that an API or widget chat set up before this version still run as the owner's own, without this list, until they fire or expire. Schedules set from a public link follow the list from their next run. + +Wikis have their own switch rather than an entry in this list: API and widget users can read a wiki the agent uses but edit it only when the wiki's owner turns on **Let API and widget users edit this wiki** in its **Wiki settings**. Public-link visitors edit only wikis they can edit themselves, and approve each edit (see [Wiki sources](/Sources/Wiki-sources#edits-from-the-api-widget-and-public-links)). + +A webhook is the owner's own automation (its URL is a secret), so its runs are not limited by this list. Nobody can approve during a webhook run, so it still skips every action that needs approval. + +### What answers show + +Tool calls name the service that ran them: "Searched Notion", "Read from Google Drive", "Used Linear: create issue". Citations from a synced source read "From Google Drive". Answers never show which account was used. + +## Admin setup + +Admins manage connectors in **Admin > Connectors**. It lists every connector with its status, how many connections use it, and: + +- **Enabled** turns a connector off for everyone. Members no longer see it (except to manage a connection they already have), its tools stop working, and its sources stop syncing until you turn it back on. Existing connections are kept. +- **Shared tools use** is **The sharer decides per share** (default), **Always the sharer's account** or **Always each person's own account**. Connectors that only sync content have no tools and show **No tools**. +- The **MCP server** row decides whether members can add their own MCP servers. Presets are switched one by one. +- **Setup guide** on an OAuth connector shows the redirect URI to register and which server settings are still missing. GitHub works without setup and shows **Tokens only** until its optional GitHub App settings are in place. +- **Write access** has one switch per connector whose tools can opt into changes, currently **Let agents make changes through GitHub** (on by default). Turned off, members no longer see the option, the API refuses it, and every GitHub tool only reads, including ones already set up for changes: they call the read-only endpoint and their write actions are refused with a reason. + +A connector that still needs server settings starts turned off and is hidden from members; its switch stays disabled until the settings are present. It turns on once they are, unless you switched it off. On a phone, the page lists the connectors and opens each one's controls in a panel. + +### Encryption key + +Credentials are encrypted with a key derived from `ENCRYPTION_SECRET_KEY` and bound to the connection's owner. **Set your own value before anyone connects a service.** + +```env +ENCRYPTION_SECRET_KEY=a-long-random-value +``` + +When authentication is on (`AUTH_TYPE` set), DocsGPT refuses to store new credentials while the key is the public default, and Admin > Connectors shows a warning. A single-user local install keeps working and logs a warning at startup. + +To rotate the key, move the old value to `ENCRYPTION_SECRET_KEY_PREVIOUS`, set the new one, restart the API and worker, then run: + +```bash +docsgpt connectors reencrypt +``` + +The command prints how many connections it rewrote. Connections it cannot decrypt with either key are marked **Reconnect needed** for their owners. Once it has run, you can remove `ENCRYPTION_SECRET_KEY_PREVIOUS`. + +### Redirect URIs + +OAuth connectors return to one callback, set by `CONNECTOR_REDIRECT_BASE_URI` (default `http://127.0.0.1:7091/api/connectors/callback`). Register it with each provider exactly as set, without query parameters. MCP servers use `MCP_OAUTH_REDIRECT_URI`, which is derived from the same base when unset. Admin > Connectors shows both values with copy buttons. + +If the frontend runs on a different origin from the API, list it in `CONNECTOR_ALLOWED_ORIGINS` so the sign-in pop-up can hand the result back. + +### Google Drive + + + +### Create OAuth credentials + +In the [Google Cloud Console](https://console.cloud.google.com/), enable the **Google Drive API**, then create an **OAuth client ID** of type **Web application**. Add the redirect URI from Admin > Connectors under **Authorized redirect URIs**. + +### Set the server settings + +```env +GOOGLE_CLIENT_ID=your-client-id +GOOGLE_CLIENT_SECRET=your-client-secret +``` + +To offer Google's own file picker, also build the frontend with `VITE_GOOGLE_CLIENT_ID` (and optionally `VITE_GOOGLE_PICKER_API_KEY`). Without them, members browse files in DocsGPT's picker. + +### Publish the app + +Publish the OAuth consent screen, or make it an **Internal** Workspace app. + + +Apps left in **Testing** get refresh tokens that expire after seven days, which stops background sync. + + + + +### SharePoint and OneDrive + +Register an app in [Microsoft Entra ID](https://entra.microsoft.com/) with a **Web** redirect URI from Admin > Connectors, create a client secret, and grant the delegated Microsoft Graph permissions `Files.Read`, `Sites.Read.All` and `User.Read`. + +```env +MICROSOFT_CLIENT_ID=your-application-id +MICROSOFT_CLIENT_SECRET=your-client-secret +MICROSOFT_TENANT_ID=common # or your tenant id for a single-tenant app +``` + +See [SharePoint / OneDrive](/Guides/Integrations/sharepoint-connector) for tenant options. + +### Confluence + +Create an **OAuth 2.0 (3LO)** app in the [Atlassian developer console](https://developer.atlassian.com/console/myapps/), add the redirect URI from Admin > Connectors as its callback URL, and add the scopes `read:page:confluence`, `read:space:confluence`, `read:attachment:confluence` and `read:me`. See [Confluence](/Guides/Integrations/confluence-connector) for details. + +```env +CONFLUENCE_CLIENT_ID=your-client-id +CONFLUENCE_CLIENT_SECRET=your-client-secret +``` + +### GitHub + +One GitHub connection syncs repositories into Knowledge and gives agents GitHub's own [MCP server](https://github.com/github/github-mcp-server) as a tool. When connecting, members choose both on one screen: **Let agents use GitHub** (on by default) and **Sync into Knowledge** with a repository to sync. A connection set up without tools can add them later from its page. + +- **Knowledge.** The repository picker lists what the connection can read. Each synced repository is one source, read with the connection's token and synced on the chosen schedule. +- **Tools.** By default the tool is the read-only endpoint `https://api.githubcopilot.com/mcp/readonly`: agents can read code, issues, pull requests and more, and cannot change anything unless the connection [lets them make changes](#letting-agents-make-changes). Its actions are discovered when the tool is created; **Refresh tools** re-reads them. + +Members connect in one of two ways. + +#### A personal access token (no admin setup) + +Always available. Create a [fine-grained personal access token](https://github.com/settings/personal-access-tokens/new): + + + +### Choose the repositories + +Under **Repository access**, pick **Only select repositories** (or all repositories) for the owner whose repositories DocsGPT should read. + +### Grant read access + +Under **Repository permissions**, set **Contents** to **Read-only**. **Metadata** is read-only by default. Add read-only access to **Issues** and **Pull requests** if agents should read those through the tools, or **Read and write** if agents should also [make changes](#letting-agents-make-changes). + +### Paste it into DocsGPT + +DocsGPT checks the token with GitHub and names the connection after the account. A token that expires, or that GitHub stops accepting, marks the connection **Reconnect needed**; reconnect with a new token. + + + +A classic token with the `repo` scope also works, but it can read every repository the account can, so prefer a fine-grained one. + +#### Sign in with GitHub (a GitHub App) + +When an admin registers a GitHub App, members can also **Sign in with GitHub**. Each member then chooses the repositories on GitHub, when they install the app, and DocsGPT reads only those. Tokens last eight hours and are renewed automatically. + + + +### Register the app + +In GitHub, open **Settings > Developer settings > GitHub Apps > New GitHub App** (under an organization's settings to let its members install it). + +- **Callback URL**: the redirect URI from Admin > Connectors (`CONNECTOR_REDIRECT_BASE_URI`). +- Turn on **Request user authorization (OAuth) during installation**. Choosing repositories from DocsGPT then returns to DocsGPT, which reloads the repository list. +- Leave **Expire user authorization tokens** on (DocsGPT refreshes them). +- **Webhook**: turn off **Active**; DocsGPT does not use webhooks. +- **Repository permissions**: **Contents** read-only (**Metadata** read-only is added automatically). Add **Issues** and **Pull requests** read-only for the tools, or read and write if members should be able to let agents [make changes](#letting-agents-make-changes). A member's token can only do what both the app and the member may do, so an app without write permissions keeps every agent read-only. When you add permissions later, each installation has to accept them on GitHub first. +- **Where can this GitHub App be installed?**: **Any account** for members outside your organization, otherwise **Only on this account**. + +### Create a client secret + +On the app's page, generate a client secret. You do not need a private key: DocsGPT only uses user tokens. + +### Set the server settings + +```env +GITHUB_CLIENT_ID=Iv23li... # the app's Client ID +GITHUB_CLIENT_SECRET=your-client-secret +GITHUB_APP_SLUG=docsgpt-acme # from the app's public link, github.com/apps/ +``` + +Restart the API and the worker. **Sign in with GitHub** appears next to the token option. + + + +#### Letting agents make changes + +Under **Let agents use GitHub**, **Also let agents make changes (issues, comments, pull requests)** is off by default. Turned on, the tool uses GitHub's full endpoint `https://api.githubcopilot.com/mcp/` instead, which adds write actions such as creating issues, commenting and opening pull requests. The same switch is on the connection's page, on its GitHub tool; switching re-reads the actions from the other endpoint and keeps the permissions and fixed values of actions both endpoints have. Either way the tool sends the token only to `api.githubcopilot.com`. + +An action is a **Write** unless GitHub marks it read-only, and writes default to **Needs approval**, so each change asks first until you choose **Always allow** for it. As everywhere, an agent called with its API key takes a write action only if the owner allows it in **Access details**, and an admin can turn changes off for everyone (see [Admin setup](#admin-setup)). + +GitHub still checks the token: changes need a token, or a GitHub App, with write access to what they touch: + +| To let agents | Repository permission | +| --- | --- | +| Create and edit issues, comment on issues | **Issues**: Read and write | +| Open, update and review pull requests, comment on them | **Pull requests**: Read and write | +| Create or edit files, push commits, create branches | **Contents**: Read and write | + +Leave **Contents** read-only unless agents should edit files. Without a permission, GitHub refuses that action and the agent reports the error. + +#### Public repositories and GITHUB_ACCESS_TOKEN + +The **GitHub** tile under **Upload & web** in **Add knowledge** still ingests a public repository from its URL, without an account. `GITHUB_ACCESS_TOKEN`, if set, is used only there and only for public repositories, to raise GitHub's rate limit: DocsGPT checks that the repository is public first, because that token belongs to the server and not to the member asking. For a private repository, the form links to the member's GitHub connection. + + +Earlier versions read any repository `GITHUB_ACCESS_TOKEN` could see, private ones included. Sources made that way from private repositories stop syncing; recreate them from a GitHub connection. + + +### Amazon S3 + +S3 needs no server settings. Each member connects with an access key that can list and read the bucket (`s3:ListBucket`, `s3:GetObject`), then picks a bucket and optional path prefix. A custom endpoint URL connects S3-compatible storage such as MinIO or Cloudflare R2. + +### MCP presets + +Notion, Linear, Atlassian, Sentry, Asana and Stripe are remote MCP servers that support OAuth with dynamic client registration, so they need no server settings. Members sign in with their own account. The presets ship in `docsgpt/connectors/presets/mcp.yaml`. + +### Linear + +One Linear sign-in gives agents Linear's tools and syncs Linear into Knowledge. After signing in, members turn on **Sync into Knowledge** and choose what to sync, or leave it off to keep only the tools; a connection's page adds more with **Sync more content**. + +- **Teams and projects.** Each picked team or project brings its issues. An issue in both is synced once. +- **Include comments** (on by default) adds each issue's comments to it. +- **Include project documents** syncs the Linear documents of the picked projects. + +Each issue becomes one document: its identifier and title, state, assignee, priority, labels, description and comments. It's filed under its team's key (`ENG/ENG-123.md`) and answers cite it with its Linear link. Archived issues are left out. + +Linear's MCP server issues its own sign-in tokens, which work only with that server, so DocsGPT reads Linear through the same MCP tools the agents use. Nobody registers a Linear OAuth app. Tokens are renewed automatically. If renewing fails, the connection is marked **Reconnect needed** and its sources pause until you sign in again. + +Each sync reads the source again in full, up to 500 issues and 100 documents, most recently updated first. For a large workspace, pick teams or projects rather than everything, and sync daily or weekly. + + +Background sync runs on the Celery worker and beat. Keep both running, as in the bundled Compose and Kubernetes files. + diff --git a/docs/content/Guides/Integrations/confluence-connector.mdx b/docs/content/Guides/Integrations/confluence-connector.mdx index d9702fd2..1bdc1ad8 100644 --- a/docs/content/Guides/Integrations/confluence-connector.mdx +++ b/docs/content/Guides/Integrations/confluence-connector.mdx @@ -10,6 +10,10 @@ import { Steps } from 'nextra/components' Connect your Confluence Cloud workspace to upload and process pages directly as an external knowledge base. Supports page content and attachments (PDFs, Office files, text files, images, and more). Authentication is handled via Atlassian OAuth 2.0 with automatic token refresh. + +Members connect this service from **Settings > Connectors**, and one sign-in serves every source that uses it. Admins can check what is still missing in **Admin > Connectors**. See [Connectors](/Guides/Connectors). + + ## Setup @@ -18,8 +22,8 @@ Connect your Confluence Cloud workspace to upload and process pages directly as 1. Go to [developer.atlassian.com/console/myapps](https://developer.atlassian.com/console/myapps/) and click **Create** > **OAuth 2.0 integration** 2. Under **Authorization**, add a callback URL: - - Local: `http://localhost:7091/api/connectors/callback?provider=confluence` - - Production: `https://yourdomain.com/api/connectors/callback?provider=confluence` + - Local: `http://127.0.0.1:7091/api/connectors/callback` (the default `CONNECTOR_REDIRECT_BASE_URI`; Admin > Connectors shows the exact value to copy) + - Production: `https://yourdomain.com/api/connectors/callback` (the value of `CONNECTOR_REDIRECT_BASE_URI`, registered as-is) ### Step 2: Configure Permissions @@ -56,7 +60,7 @@ VITE_CONFLUENCE_CLIENT_ID=your-atlassian-client-id ### Step 5: Restart and Use -Restart your application, then go to the upload section in DocsGPT and select **Confluence** as the source. You'll be redirected to Atlassian to sign in, then can browse spaces and select pages to process. +Restart your application, then go to **Settings > Connectors** and pick **Confluence**. You'll be redirected to Atlassian to sign in, then can browse spaces and select pages to process. @@ -64,6 +68,6 @@ Restart your application, then go to the upload section in DocsGPT and select ** - **Option not appearing** — Verify `VITE_CONFLUENCE_CLIENT_ID` is set in the frontend `.env`, then restart. - **Sign-in popup closes but the account never connects** — The frontend origin is not allowed to receive the result. Add it to `CONNECTOR_ALLOWED_ORIGINS` in the backend `.env`. -- **Authentication failed** — Check that the callback URL matches exactly, including `?provider=confluence`. +- **Authentication failed** — Check that the callback URL matches exactly and equals `CONNECTOR_REDIRECT_BASE_URI`, with no query parameters. - **No accessible sites** — Ensure the authenticating user has access to at least one Confluence Cloud site. - **Permission denied** — Verify that the Confluence API scopes are enabled in your Atlassian app settings. diff --git a/docs/content/Guides/Integrations/google-drive-connector.mdx b/docs/content/Guides/Integrations/google-drive-connector.mdx index 3014c784..3abe5dcf 100644 --- a/docs/content/Guides/Integrations/google-drive-connector.mdx +++ b/docs/content/Guides/Integrations/google-drive-connector.mdx @@ -10,6 +10,10 @@ import { Steps } from 'nextra/components' Connect your Google Drive account to upload and process files directly as an external knowledge base. Supports Google Workspace files (Docs, Sheets, Slides), Office files, PDFs, text files, CSVs, images, and more. Authentication is handled via Google OAuth 2.0 with automatic token refresh. + +Members connect this service from **Settings > Connectors**, and one sign-in serves every source that uses it. Admins can check what is still missing in **Admin > Connectors**. See [Connectors](/Guides/Connectors). + + ## Setup @@ -26,8 +30,8 @@ Connect your Google Drive account to upload and process files directly as an ext 3. Select **Web application** as the application type 4. Add your DocsGPT URL to **Authorized JavaScript origins** (e.g. `http://localhost:3000`) 5. Add your callback URL to **Authorized redirect URIs**: - - Local: `http://localhost:7091/api/connectors/callback?provider=google_drive` - - Production: `https://yourdomain.com/api/connectors/callback?provider=google_drive` + - Local: `http://127.0.0.1:7091/api/connectors/callback` (the default `CONNECTOR_REDIRECT_BASE_URI`; Admin > Connectors shows the exact value to copy) + - Production: `https://yourdomain.com/api/connectors/callback` (the value of `CONNECTOR_REDIRECT_BASE_URI`, registered as-is) 6. Click **Create** and copy the **Client ID** and **Client Secret** ### Step 3: Configure Environment Variables @@ -39,7 +43,7 @@ GOOGLE_CLIENT_ID=your-google-client-id GOOGLE_CLIENT_SECRET=your-google-client-secret ``` -Add to your frontend `.env` file: +Optionally, to use Google's own file picker instead of DocsGPT's, add to your frontend `.env` file: ```env VITE_GOOGLE_CLIENT_ID=your-google-client-id @@ -49,23 +53,23 @@ VITE_GOOGLE_CLIENT_ID=your-google-client-id |----------|-------------|----------| | `GOOGLE_CLIENT_ID` | OAuth Client ID from GCP Credentials | Yes | | `GOOGLE_CLIENT_SECRET` | OAuth Client Secret from GCP Credentials | Yes | -| `VITE_GOOGLE_CLIENT_ID` | Same Client ID, used by the frontend to show the Google Drive option | Yes | +| `VITE_GOOGLE_CLIENT_ID` | Same Client ID, used by the frontend for Google's file picker | No | | `CONNECTOR_ALLOWED_ORIGINS` | Comma-separated frontend origins allowed to receive the sign-in result, e.g. `https://docsgpt.example.com`. Not needed when the frontend shares the API origin, or in local dev when the callback is on `localhost`/`127.0.0.1` and the frontend runs on port 5173 | When the frontend is on its own origin | -Make sure to use the same Google Client ID in both backend and frontend configurations. +If you set `VITE_GOOGLE_CLIENT_ID`, use the same Client ID as the backend. Publish the OAuth consent screen (or use an internal Workspace app): apps left in Testing get refresh tokens that expire after seven days, which stops background sync. ### Step 4: Restart and Use -Restart your application, then go to the upload section in DocsGPT and select **Google Drive** as the source. You'll be redirected to Google to sign in, then can browse and select files to process. +Restart your application, then go to **Settings > Connectors** and pick **Google Drive**. You'll be redirected to Google to sign in, then can browse and select files to process. ## Troubleshooting -- **Option not appearing** — Verify `VITE_GOOGLE_CLIENT_ID` is set in the frontend `.env`, then restart. -- **Authentication failed** — Check that the redirect URI matches exactly, including `?provider=google_drive`. Ensure the Google Drive API is enabled. +- **Google Drive is not offered** — `GOOGLE_CLIENT_ID` or `GOOGLE_CLIENT_SECRET` is missing from the backend `.env`, so the connector stays off. Admin > Connectors lists which one. +- **Authentication failed** — Check that the redirect URI matches exactly and equals `CONNECTOR_REDIRECT_BASE_URI`, with no query parameters. Ensure the Google Drive API is enabled. - **Sign-in popup closes but the account never connects** — The frontend origin is not allowed to receive the result. Add it to `CONNECTOR_ALLOWED_ORIGINS` in the backend `.env`. - **Permission denied** — Verify the OAuth consent screen is configured and the user has access to the target files. - **Files not processing** — Check backend logs and verify that backend environment variables are correctly set. diff --git a/docs/content/Guides/Integrations/mcp-tool-integration.mdx b/docs/content/Guides/Integrations/mcp-tool-integration.mdx index d33e1b94..083bc1a3 100644 --- a/docs/content/Guides/Integrations/mcp-tool-integration.mdx +++ b/docs/content/Guides/Integrations/mcp-tool-integration.mdx @@ -22,11 +22,11 @@ Only needed if your MCP servers use OAuth authentication: MCP_OAUTH_REDIRECT_URI=https://yourdomain.com/api/mcp_server/callback ``` -If not set, falls back to `API_URL/api/mcp_server/callback`. +If not set, it is derived from the host of `CONNECTOR_REDIRECT_BASE_URI`, then from `API_URL`. ### Step 2: Add an MCP Server -Go to **Settings** > **Tools** > **Add Tool** > **MCP Server**. Enter the server URL, select an auth type, and click **Test Connection** to verify, then **Save**. +Go to **Settings** > **Connectors** and pick a preset (Notion, Linear, Atlassian, Sentry, Asana, Stripe): **Sign in to Notion** opens the service's sign-in, and its tools are ready when you come back. For any other server, choose **Add custom connector** > **MCP server** and enter its URL and authentication; scopes and the timeout are under **Show advanced**. Enter the server URL, select an auth type, and click **Test Connection** to verify, then **Save**. ### Step 3: Enable for Your Agent @@ -34,6 +34,8 @@ In your agent configuration, enable the MCP tools you want the agent to use. +Presets need no URL or form: they sign in with OAuth in one step. Admins can turn presets off one by one, and turn off custom MCP servers, in **Admin > Connectors**. See [Connectors](/Guides/Connectors). + ## Authentication Types | Auth Type | Config Fields | diff --git a/docs/content/Guides/Integrations/sharepoint-connector.mdx b/docs/content/Guides/Integrations/sharepoint-connector.mdx index 58829f38..9b38cc65 100644 --- a/docs/content/Guides/Integrations/sharepoint-connector.mdx +++ b/docs/content/Guides/Integrations/sharepoint-connector.mdx @@ -10,6 +10,10 @@ import { Steps } from 'nextra/components' Connect your SharePoint or OneDrive account to upload and process files directly as an external knowledge base. Supports Office files, PDFs, text files, CSVs, images, and more. Authentication is handled via Microsoft Entra ID (Azure AD) with automatic token refresh. + +Members connect this service from **Settings > Connectors**, and one sign-in serves every source that uses it. Admins can check what is still missing in **Admin > Connectors**. See [Connectors](/Guides/Connectors). + + ## Setup @@ -18,8 +22,8 @@ Connect your SharePoint or OneDrive account to upload and process files directly 1. Go to the [Azure Portal](https://portal.azure.com/) > **Microsoft Entra ID** > **App registrations** > **New registration** 2. Set **Redirect URI** (Web) to: - - Local: `http://localhost:7091/api/connectors/callback?provider=share_point` - - Production: `https://yourdomain.com/api/connectors/callback?provider=share_point` + - Local: `http://127.0.0.1:7091/api/connectors/callback` (the default `CONNECTOR_REDIRECT_BASE_URI`; Admin > Connectors shows the exact value to copy) + - Production: `https://yourdomain.com/api/connectors/callback` (the value of `CONNECTOR_REDIRECT_BASE_URI`, registered as-is) ### Step 2: Configure API Permissions @@ -53,13 +57,13 @@ MICROSOFT_TENANT_ID=your-azure-ad-tenant-id ### Step 5: Restart and Use -Restart your application, then go to the upload section in DocsGPT and select **SharePoint / OneDrive** as the source. You'll be redirected to Microsoft to sign in, then can browse and select files to process. +Restart your application, then go to **Settings > Connectors** and pick **SharePoint**. You'll be redirected to Microsoft to sign in, then can browse and select files to process. ## Troubleshooting - **Option not appearing** — Verify `MICROSOFT_CLIENT_ID` and `MICROSOFT_CLIENT_SECRET` are set, then restart. -- **Authentication failed** — Check that the redirect URI matches exactly, including `?provider=share_point`. +- **Authentication failed** — Check that the redirect URI matches exactly and equals `CONNECTOR_REDIRECT_BASE_URI`, with no query parameters. - **Sign-in popup closes but the account never connects** — The frontend origin is not allowed to receive the result. Add it to `CONNECTOR_ALLOWED_ORIGINS` in the backend `.env`. - **Permission denied** — Ensure admin consent is granted and the user has access to the target files. diff --git a/docs/content/Guides/_meta.js b/docs/content/Guides/_meta.js index 33cca51f..a9122c32 100644 --- a/docs/content/Guides/_meta.js +++ b/docs/content/Guides/_meta.js @@ -1,4 +1,8 @@ export default { + "Connectors": { + "title": "🔌 Connectors", + "href": "/Guides/Connectors" + }, "Customising-prompts": { "title": "️💻 Customising Prompts", "href": "/Guides/Customising-prompts" diff --git a/docs/content/Sources/Wiki-sources.mdx b/docs/content/Sources/Wiki-sources.mdx index 04c8b568..43afb37a 100644 --- a/docs/content/Sources/Wiki-sources.mdx +++ b/docs/content/Sources/Wiki-sources.mdx @@ -90,7 +90,29 @@ PUT /api/sources//wiki/page # create or overwrite a page ( Human edits are stamped with `human` provenance and trigger the same re-embed as agent edits. Read access follows source sharing (owner or anyone the source is shared with); writing requires owner or team `editor` access. +## Edits from the API, widget and public links + +An agent called with its API key (the website widget, the API) runs as its owner, so it could change any wiki the owner can edit. By default it can't: API and widget users can still read the wiki through the agent, but the agent isn't offered the create, edit, delete or rename actions in those chats, and the Wiki tool refuses them if it's asked anyway. + +The wiki's owner can allow such edits. On the **Sources** page, open the wiki's menu, choose **Wiki settings**, and turn on **Let API and widget users edit this wiki**. The change applies from the next message, including in chats that are already open. Only the owner sees it; team editors can't change it. You and the wiki's editors can always edit it in DocsGPT, and so can agents you or they use there. + +The switch doesn't cover public links. People who open an agent from its public link run it as themselves, so they can only edit wikis they could edit anyway (their own, or ones shared with them as an editor). Because the agent's prompt and sources belong to someone else, each of their wiki edits waits for their approval in the chat before it runs. A research agent can't stop to ask, so in a public-link chat it reads the wiki but doesn't edit it. + +Scheduled and webhook runs don't get the Wiki tool, so they read a wiki through search but never edit it. + + +With authentication off (`AUTH_TYPE=None`, the default), every request, including one from the widget or the API, runs as the same `local` user who owns the agents. The switch then can't tell outside callers from you and has no effect, so turn authentication on if others can reach the widget or the API. + + +From the API, only a signed-in session can change the setting; a personal access token can read it but not change it: + +```text +GET /api/sources//wiki/settings # {"allow_outside_edits": false, ...} +PUT /api/sources//wiki/settings # owner only; body {"allow_outside_edits": true} +``` + ## Related - [Per-Source Configuration](/Sources/Per-source-configuration) — exposure and retrieval settings a wiki uses. -- [Access Control & Teams](/Deploying/Access-Control) — sharing a wiki with a team. +- [Access Control & Teams](/Deploying/Access-Control) — sharing a wiki with a team, and [what an agent's users reach](/Deploying/Access-Control#sharing-agents-and-what-they-use) when you share the agent. +- [Connectors](/Guides/Connectors#agents-used-through-an-api-key) — what else an agent can and can't do for API, widget and public-link users. diff --git a/docs/content/Sources/_meta.js b/docs/content/Sources/_meta.js index 06a1d99a..cc800b43 100644 --- a/docs/content/Sources/_meta.js +++ b/docs/content/Sources/_meta.js @@ -1,4 +1,8 @@ export default { + "Connectors": { + "title": "🔌 Synced Sources (Connectors)", + "href": "/Guides/Connectors" + }, "Per-source-configuration": { "title": "🎛️ Per-Source Configuration", "href": "/Sources/Per-source-configuration" diff --git a/docs/content/Tools/basics.mdx b/docs/content/Tools/basics.mdx index b578eb8c..995f5216 100644 --- a/docs/content/Tools/basics.mdx +++ b/docs/content/Tools/basics.mdx @@ -59,7 +59,7 @@ DocsGPT includes a suite of pre-built tools designed to expand its capabilities { title: 'Telegram Bot', link: 'https://github.com/arc53/DocsGPT/blob/main/docsgpt/agents/tools/telegram.py', - description: 'Allows DocsGPT to send messages or images to Telegram chats via a Telegram Bot. Requires a bot token and chat ID.' + description: 'Allows DocsGPT to send messages or images to Telegram chats via a Telegram Bot. Requires a bot token; an optional default chat ID sends every message to one chat.' }, { title: 'PostgreSQL Database', @@ -121,7 +121,8 @@ Interacting with tools in DocsGPT is designed to be intuitive: 2. **Configuration in UI:** * Tools are generally managed and configured within the DocsGPT application's settings, found under a "Tools" section in the GUI. - * For tools that interact with external services (like Brave Search, Telegram, or any service via the API Tool), you might need to provide authentication credentials (e.g., API keys, tokens) or specific endpoint information during the tool's setup in the UI. + * Tools for external services (Brave Search, Telegram, ntfy, PostgreSQL, MCP servers) are set up from **Settings > Connectors**. You connect the service once; its credentials stay encrypted on the server and never reach the browser, and the tools it provides are grouped under that connection. Each action is marked **Read** or **Write** and can be set to **Always allow**, **Needs approval** or **Off**. See [Connectors](/Guides/Connectors). + * When a tool's connection is missing or expired, the chat shows a **Connect** prompt and continues once you connect. 3. **Prompt Engineering for Tools:** While the LLM aims to intelligently use tools, for more complex or reliable agent-like behaviors, you might need to customize the system prompts. Modifying the prompt can guide the LLM on when and how to prioritize or chain tools to achieve specific outcomes, especially if you're building an agent designed to perform a certain sequence of actions every time. For more on this, see [Customising Prompts](/Guides/Customising-prompts). diff --git a/docs/content/upgrading.mdx b/docs/content/upgrading.mdx index c5397390..d1208947 100644 --- a/docs/content/upgrading.mdx +++ b/docs/content/upgrading.mdx @@ -11,6 +11,23 @@ import { Callout } from 'nextra/components' **Upgrading from 0.16.x?** User data moved from MongoDB to Postgres in 0.17.0. Follow the [Postgres Migration guide](/Deploying/Postgres-Migration) before running `docker compose pull` or `git pull` — existing deployments will not start cleanly without it. +## Connectors: set ENCRYPTION_SECRET_KEY first + +Service credentials (OAuth tokens for Google Drive, SharePoint, Confluence and MCP servers, and API keys for tools) now live on **connections** and are encrypted with a key derived from `ENCRYPTION_SECRET_KEY`. Migration `0040_connections` runs on startup, encrypts the stored tokens and removes their plaintext copies. + + + **Multi-user installs** (any `AUTH_TYPE`): set `ENCRYPTION_SECRET_KEY` to your own value **before** you upgrade, in the environment of the API, the worker and anything that runs migrations. The migration encrypts with the key it sees; changing it afterwards makes every connection ask its owner to reconnect. While the key is still the public default, DocsGPT refuses to store new credentials and Admin > Connectors shows a warning. + + +If you already used a key and want to change it, see [rotating the key](/Guides/Connectors#encryption-key). Single-user local installs keep working with the default and log a warning at startup. + +Other changes: + +- Register `CONNECTOR_REDIRECT_BASE_URI` exactly as set (no `?provider=` query) as the redirect URI of each OAuth app. Admin > Connectors shows it. +- `VITE_GOOGLE_CLIENT_ID` is optional now; without it, members pick Drive files in DocsGPT's own picker. +- Synced sources run as their connection, without a browser session. Keep Celery beat running for scheduled syncs. +- Downgrading to `0037_request_traces` decrypts the tokens back and gives each tool its key again. + ## pip installs: data home moved An installed package (`pip install docsgpt`, pipx, `uv tool`) used to keep its data home, meaning `.env`, `inputs/`, `indexes/` and `models/`, in the directory you ran `docsgpt api` and `docsgpt worker` from. It is now `~/.docsgpt/server` (`/opt/docsgpt` for root on Linux). Either move those files there, or set `DOCSGPT_HOME` to the old directory in the environment of both commands. Both commands print the data home they use, and point out a `.env` in the working directory that they no longer read. Source checkouts and the Docker images are not affected. diff --git a/docsgpt/agents/headless_runner.py b/docsgpt/agents/headless_runner.py index cddf693e..021efb21 100644 --- a/docsgpt/agents/headless_runner.py +++ b/docsgpt/agents/headless_runner.py @@ -15,15 +15,17 @@ from docsgpt.api.answer.services.prompt_renderer import ( resolve_prompt_skeleton, ) from docsgpt.api.answer.services.stream_processor import ( + authorized_agent_sources, authorized_prompt_id, get_prompt, ) -from docsgpt.api.user.resource_access import ref_principal from docsgpt.core.settings import settings +from docsgpt.guardrails.config import AgentConfig from docsgpt.quotas.service import QuotaExceededError, QuotaService +from docsgpt.retriever.dispatcher import build_dispatcher from docsgpt.retriever.retriever_creator import RetrieverCreator -from docsgpt.storage.db.repositories.sources import SourcesRepository from docsgpt.storage.db.session import db_readonly +from docsgpt.storage.db.source_config import SourceConfig logger = logging.getLogger(__name__) @@ -74,6 +76,8 @@ def run_agent_headless( endpoint: str = "headless", chat_history: Optional[List[Dict[str, Any]]] = None, conversation_id: Optional[str] = None, + external_caller: bool = False, + public_link_caller: bool = False, request_id: Optional[str] = None, trace_user_id: Optional[str] = None, ) -> Dict[str, Any]: @@ -86,6 +90,11 @@ def run_agent_headless( ``trace_user_id`` owns the trace when the run belongs to someone other than the agent's owner (a schedule a user set on a shared agent), so the trace is visible wherever that user sees the run; it defaults to the owner. + ``external_caller`` (a schedule set through the API) and + ``public_link_caller`` (a schedule a public-link user set) mark a run for + someone who can't approve for the owner: writes on the owner's accounts + and credentials then run only when the agent's API write allowlist has + them. Raises: QuotaExceededError: If the agent owner's usage quota is exhausted. @@ -108,6 +117,8 @@ def run_agent_headless( endpoint=endpoint, chat_history=chat_history, conversation_id=conversation_id, + external_caller=external_caller, + public_link_caller=public_link_caller, ) if outcome.get("error"): status = tracing.STATUS_ERROR @@ -128,6 +139,8 @@ def _run_agent_headless( endpoint: str = "headless", chat_history: Optional[List[Dict[str, Any]]] = None, conversation_id: Optional[str] = None, + external_caller: bool = False, + public_link_caller: bool = False, ) -> Dict[str, Any]: from docsgpt.core.model_utils import ( get_api_key_for_provider, @@ -148,20 +161,22 @@ def _run_agent_headless( raise QuotaExceededError(exceeded) retriever_kind = agent_config.get("retriever", "classic") - source_id = agent_config.get("source_id") or agent_config.get("source") - source_active: Any = {} - if source_id: + # Every source a chat with this agent searches: the primary and the + # extras, each owned or team-shared to the owner, else attached by an + # editor who still qualifies. + sources_row = dict(agent_config) + sources_row["source_id"] = agent_config.get("source_id") or agent_config.get("source") + primary, source_docs = None, [] + if sources_row["source_id"] or sources_row.get("extra_source_ids"): with db_readonly() as conn: - # Owned or team-shared to the owner, else attached by an editor - # who still qualifies; read unscoped once authorized. - src_row = ( - SourcesRepository(conn).get_by_id(str(source_id)) - if ref_principal(conn, "agent", agent_config, "source", str(source_id)) - else None - ) - if src_row: - source_active = str(src_row["id"]) - retriever_kind = src_row.get("retriever", retriever_kind) + primary, source_docs = authorized_agent_sources(conn, sources_row) + if primary: + retriever_kind = primary.get("retriever") or retriever_kind + per_source = [ + {"id": str(doc["id"]), "retrieval": SourceConfig.parse(doc.get("config")).retrieval} + for doc in source_docs + ] + source_active: Any = [entry["id"] for entry in per_source] or {} source = {"active_docs": source_active} # ``chunks=0`` switches retrieval off; only a missing value takes the default. raw_chunks = agent_config.get("chunks") @@ -196,8 +211,7 @@ def _run_agent_headless( system_api_key = get_api_key_for_provider(provider or settings.LLM_PROVIDER) doc_token_limit = calculate_doc_token_budget(model_id=model_id, user_id=owner) - retriever = RetrieverCreator.create_retriever( - retriever_kind, + retriever_kwargs: Dict[str, Any] = dict( source=source, chat_history=chat_history or [], prompt=prompt, @@ -208,6 +222,13 @@ def _run_agent_headless( agent_id=agent_id, decoded_token=decoded_token, ) + # Routed per source like a chat's pre-fetch, so each source keeps its own + # retriever and retrieval settings. + retriever = build_dispatcher( + lambda: RetrieverCreator.create_retriever(retriever_kind, **retriever_kwargs), + sources=per_source, + **retriever_kwargs, + ) retrieved_docs: List[Dict[str, Any]] = [] try: docs = retriever.search(query) @@ -223,6 +244,9 @@ def _run_agent_headless( agent_id=agent_id, headless=True, tool_allowlist=list(tool_allowlist or []), + external_caller=external_caller, + public_link_caller=public_link_caller, + api_write_allowlist=AgentConfig.parse(agent_config.get("config")).api_write_allowlist, ) if conversation_id: tool_executor.conversation_id = str(conversation_id) diff --git a/docsgpt/agents/research_agent.py b/docsgpt/agents/research_agent.py index 859bc872..c59f1513 100644 --- a/docsgpt/agents/research_agent.py +++ b/docsgpt/agents/research_agent.py @@ -6,7 +6,7 @@ from typing import Any, Dict, Generator, List, Optional from docsgpt import tracing from docsgpt.agents.base import BaseAgent -from docsgpt.agents.tool_executor import ToolExecutor +from docsgpt.agents.tool_executor import ToolExecutor, journal_refused_call from docsgpt.agents.tools.graph_search import add_graph_search_tool from docsgpt.agents.tools.internal_search import add_internal_search_tool from docsgpt.agents.tools.wiki import add_wiki_tool @@ -582,24 +582,31 @@ class ResearchAgent(BaseAgent): search_returned_empty = False for call in tool_calls: - gen = executor.execute( - tools_dict, call, self.llm.__class__.__name__ - ) - result = None - call_id = None - while True: - try: - event = next(gen) - # Log tool_call status events instead of discarding them - if isinstance(event, dict) and event.get("type") == "tool_call": - logger.debug( - "Tool %s status: %s", - event.get("data", {}).get("action_name", ""), - event.get("data", {}).get("status", ""), - ) - except StopIteration as e: - result, call_id = e.value - break + # A step runs inside one turn and nobody can answer a pause here, + # so a call that would pause (approval, a connection, the client, + # an outside caller's write on the owner's account) is refused. + refusal = self._refuse_paused_call(tools_dict, call, executor) + if refusal is not None: + result, call_id = refusal + else: + gen = executor.execute( + tools_dict, call, self.llm.__class__.__name__ + ) + result = None + call_id = None + while True: + try: + event = next(gen) + # Log tool_call status events instead of discarding them + if isinstance(event, dict) and event.get("type") == "tool_call": + logger.debug( + "Tool %s status: %s", + event.get("data", {}).get("action_name", ""), + event.get("data", {}).get("status", ""), + ) + except StopIteration as e: + result, call_id = e.value + break # Detect empty search results for refinement is_search = "search" in (call.name or "").lower() @@ -636,6 +643,60 @@ class ResearchAgent(BaseAgent): return messages, search_returned_empty + def _refuse_paused_call( + self, tools_dict: Dict, call, executor: ToolExecutor + ) -> Optional[tuple[str, str]]: + """Refuse a call ``check_pause`` would pause on, as a headless run does. + + Args: + tools_dict: The step's tools. + call: The model's tool call. + executor: The run's executor. + + Returns: + ``(tool result, call id)`` when the call is refused, else None. + """ + pause_info = executor.check_pause( + tools_dict, call, self.llm.__class__.__name__ + ) + if not pause_info: + return None + pause_type = pause_info.get("pause_type") + if pause_type == "headless_denied": + reason = pause_info.get("deny_reason") or "This tool can't run here." + result = f"Tool denied: {reason}" + journal_error = f"headless: {reason}" if executor.headless else f"denied: {reason}" + if executor.headless: + executor.headless_denials.append(pause_info) + elif pause_info.get("connection_required"): + result = ( + "Tool not run: its service needs to be connected first, and a " + "research step can't wait for that." + ) + journal_error = "research: connection required" + elif pause_type == "requires_client_execution": + result = ( + "Tool not run: it runs in the user's app, which a research step " + "can't reach." + ) + journal_error = "research: client-side tool" + else: + result = ( + "Tool not run: this action needs the user's approval, which a " + "research step can't ask for. Tell the user it needs their " + "approval in a regular chat." + ) + journal_error = "research: approval required" + logger.info( + "research_step_tool_refused", + extra={ + "action_name": pause_info.get("action_name"), + "pause_type": pause_type, + }, + ) + journal_refused_call(executor, pause_info, journal_error) + return result, pause_info["call_id"] + def _collect_step_sources(self): """Register the search tools' docs (internal search and graph pages) with CitationManager.""" for doc in self._search_tool_docs(): diff --git a/docsgpt/agents/tool_executor.py b/docsgpt/agents/tool_executor.py index dd9e6f0b..38a69958 100644 --- a/docsgpt/agents/tool_executor.py +++ b/docsgpt/agents/tool_executor.py @@ -15,6 +15,7 @@ from docsgpt.agents.default_tools import ( synthesized_default_tools, ) from docsgpt import tracing +from docsgpt.agents.tool_pins import iter_parameters, llm_fills, resolve_arguments, sent_arguments from docsgpt.agents.tools.tool_action_parser import ToolActionParser from docsgpt.agents.tools.tool_manager import ToolManager from docsgpt.guardrails.types import Stage as GuardrailStage, resolve_tool_result @@ -188,6 +189,16 @@ def _requires_approval(tool: Dict, action: Dict) -> bool: return bool((tool.get("config") or {}).get("require_approval")) +def _account_slug(account: Optional[str], limit: int = 24) -> str: + """An account name as a function-name suffix: ``Ops: on-call!`` → ``ops_on_call``. + + Empty when nothing ASCII is left (a name in another script); the caller + then numbers the duplicates instead. + """ + slug = re.sub(r"[^a-z0-9]+", "_", str(account or "").lower()).strip("_") + return slug[:limit].rstrip("_") + + def _sanitize_tool_prefix(tool_name: Optional[str]) -> str: """Reduce a tool name to characters allowed in function-call names.""" return re.sub(r"[^a-zA-Z0-9_-]+", "_", str(tool_name or "")).strip("_") @@ -472,6 +483,36 @@ def _mark_failed( logger.exception("tool_call_attempts failed-write failed for %s", call_id) +def journal_refused_call(executor: Any, pause_info: Dict, error: str) -> None: + """Journal a tool call that was refused instead of paused, as failed. + + A headless run and a research step can't pause for anyone, so a call + ``check_pause`` would pause on is answered with a refusal. Journaling it + keeps the refusal visible to the reconciler and tool analytics. + + Args: + executor: The run's ``ToolExecutor``. + pause_info: What ``check_pause`` returned for the call. + error: The failure recorded on the journal row. + """ + if _record_proposed( + pause_info["call_id"], + pause_info["tool_name"], + pause_info["action_name"], + pause_info.get("arguments") or {}, + tool_id=pause_info.get("tool_id"), + message_id=getattr(executor, "message_id", None), + user_id=getattr(executor, "user", None), + agent_id=getattr(executor, "agent_id", None), + ): + _mark_failed( + pause_info["call_id"], + error, + message_id=getattr(executor, "message_id", None), + user_id=getattr(executor, "user", None), + ) + + class ToolExecutor: """Handles tool discovery, preparation, and execution. @@ -487,6 +528,9 @@ class ToolExecutor: *, headless: bool = False, tool_allowlist: Optional[List[str]] = None, + external_caller: bool = False, + public_link_caller: bool = False, + api_write_allowlist: Optional[List[str]] = None, ): self.user_api_key = user_api_key self.user = user @@ -497,6 +541,15 @@ class ToolExecutor: self.headless = bool(headless) # Tool-instance ids pre-authorized for headless approval-gated execution. self.tool_allowlist: set = {str(x) for x in tool_allowlist} if tool_allowlist else set() + # Someone calling the agent with its API key (widget, API): the run + # uses the owner's accounts and nobody can approve, so writes on a + # connected account run only when the owner allowlisted them. + self.external_caller = bool(external_caller) + # Someone who reaches the agent only through its public link: they + # may not approve writes on the owner's account either, so those run + # only when allowlisted. Their own account (member mode) is theirs. + self.public_link_caller = bool(public_link_caller) + self.api_write_allowlist: set = {str(x) for x in api_write_allowlist or []} # Set by BaseAgent._prepare_tools when the agent has tool-stage controls. self.guardrail_engine = None self.tool_calls: List[Dict] = [] @@ -505,9 +558,15 @@ class ToolExecutor: # get_tools() resolves EXACTLY these ids — builtin synthetic ids and # user_tools rows alike — with no defaults mixed in. None = unscoped. self.allowed_tool_ids: Optional[List[str]] = None + # Who an explicit tool-id scope resolves as: the workflow owner for a + # node, so whoever runs it gets the owner's tools. None = ``user``. + self.tool_owner: Optional[str] = None # Tool id -> the user to resolve it as, for a workflow node's tools # sponsored by an editor (see resource_access.active_sponsor). self.tool_principals: Dict[str, str] = {} + # The workflow row a node's tools belong to, so a dropped one is + # logged with why it doesn't run. + self.tool_holder: Optional[Dict] = None self.conversation_id: Optional[str] = None # Set by the workflow engine for agent nodes so run-scoped tools # (artifact_generator / code_executor) address artifacts by the @@ -528,6 +587,11 @@ class ToolExecutor: self._tool_to_name: Dict[Tuple[str, str], str] = {} # Filled by the LLMHandler.handle_tool_calls headless loop. self.headless_denials: List[Dict] = [] + # Per-turn connection resolution for connection-backed tools, keyed + # by tool row id, so check_pause and execute share one lookup. + self._connections: Dict[str, Any] = {} + # Tool parameters those connections set (Telegram's default chat). + self._connection_params: Dict[str, Dict] = {} def get_tools(self) -> Dict[str, Dict]: """Load tool configs from DB based on user context. @@ -559,27 +623,45 @@ class ToolExecutor: """Resolve an explicit tool-id scope — exactly these ids, no defaults. Used by workflow agent nodes: the node's configured tools (builtin - synthetic ids like Artifact/Code Executor/Read Document, or the user's - ``user_tools`` rows) are the node's WHOLE toolset. An unresolvable id - is dropped with a warning rather than failing the node. + synthetic ids like Artifact/Code Executor/Read Document, or the + ``user_tools`` rows of ``tool_owner``) are the node's WHOLE toolset. + Rows resolve as the workflow owner, then as the editor who attached + them, never as whoever runs the workflow — the same rule as an agent's + own tools. An unresolvable id is dropped with a warning rather than + failing the node. """ if not tool_ids: return {} + principal = self.tool_owner or self.user with db_readonly() as conn: tools_repo = UserToolsRepository(conn) tools: List[Dict] = [] for tid in tool_ids: - row = resolve_tool_by_id(tid, self.user, user_tools_repo=tools_repo) + row = resolve_tool_by_id(tid, principal, user_tools_repo=tools_repo) if row is None and str(tid) in self.tool_principals: row = resolve_tool_by_id(tid, self.tool_principals[str(tid)], user_tools_repo=tools_repo) if row is None: - logger.warning("tool id %s did not resolve; dropped from scoped toolset", tid) + self._log_dropped_scoped_tool(conn, tid, principal) continue if self.headless and is_headless_excluded_tool(row.get("name")): continue tools.append(row) return {str(tool["id"]): tool for tool in tools} + def _log_dropped_scoped_tool(self, conn, tool_id: str, principal: Optional[str]) -> None: + """Log a node tool the scoped toolset leaves out, with why when the workflow is known.""" + # Lazy: docsgpt.api's package import pulls in every route module. + from docsgpt.api.user.resource_access import log_stopped, resolve_holder_tool + + holder = {**(self.tool_holder or {}), "user_id": principal} + reason = None + try: + _row, access = resolve_holder_tool(conn, "workflow", holder, tool_id) + reason = access.reason + except Exception: + logger.exception("Could not tell why tool %s does not resolve", tool_id) + log_stopped("workflow", holder, "tool", tool_id, reason) + def _get_tools_by_api_key(self, api_key: str) -> Dict[str, Dict]: """Resolve an agent's toolset — exactly ``agents.tools``, no defaults.""" # Per-operation session: the answer pipeline spans a long-lived @@ -595,13 +677,14 @@ class ToolExecutor: row = resolve_tool_by_id(tid, owner, user_tools_repo=tools_repo) if row is None: # A tool the owner can't use runs as the editor who - # attached it, while they still qualify. + # attached it, while they still qualify: the same check + # the agent page's run state uses. # Lazy: docsgpt.api's package import pulls in every route module. - from docsgpt.api.user.resource_access import active_sponsor + from docsgpt.api.user.resource_access import log_stopped, resolve_holder_tool - sponsor = active_sponsor(conn, "agent", agent_data, "tool", str(tid)) - if sponsor: - row = resolve_tool_by_id(tid, sponsor, user_tools_repo=tools_repo) + row, access = resolve_holder_tool(conn, "agent", agent_data, tid, tools_repo=tools_repo) + if row is None: + log_stopped("agent", agent_data, "tool", tid, access.reason) if row is None: continue # Workflow-only builtins (read_document) never resolve for a @@ -791,15 +874,36 @@ class ToolExecutor: self._tool_to_name = {} all_llm_names: set = set() + # Connection tools that share an action name are told apart by what + # they connect to: the service ("search" on Notion and on Linear), or + # the account when one service is connected twice (two Telegram bots). + connected: Dict[int, Tuple[str, str]] = {} + for index, (tool_id, _tool_name, action_name, _action, is_client) in enumerate(entries): + if name_counts[action_name] > 1 and not is_client: + names = self._connection_names(tools_dict[tool_id]) + if names: + connected[index] = names + per_service = Counter((entries[i][2], service) for i, (service, _account) in connected.items()) + result = [] - for tool_id, tool_name, action_name, action, is_client in entries: + for index, (tool_id, tool_name, action_name, action, is_client) in enumerate(entries): + service, account = connected.get(index, (None, None)) + # The account is named only where it is what tells tools apart. + if service is not None and per_service[(action_name, service)] < 2: + account = None + slug = _account_slug(account or service) if name_counts[action_name] == 1 and len(action_name) <= _MAX_LLM_NAME_LEN: llm_name = action_name else: # An over-long unique name skips the prefix — it needs # truncation, not disambiguation. prefix = _sanitize_tool_prefix(tool_name) if name_counts[action_name] > 1 else "" - base = f"{prefix}_{action_name}" if prefix and not action_name.startswith(f"{prefix}_") else action_name + if slug: + base = f"{action_name}_{slug}" + elif prefix and not action_name.startswith(f"{prefix}_"): + base = f"{prefix}_{action_name}" + else: + base = action_name base = base[:_MAX_LLM_NAME_LEN] # A duplicated bare name stays ambiguous, and a candidate # must not steal a unique action's name or one already taken. @@ -818,31 +922,54 @@ class ToolExecutor: if is_client: params = action.get("parameters", {}) else: - params = self._build_tool_parameters(action) + params = self._build_tool_parameters( + action, hidden=set(self._connection_parameters(tools_dict[tool_id])), + ) + description = action.get("description", "") + if account: + description = f"{description} ({service} account: {account})".strip() + elif service: + description = f"{description} ({service})".strip() result.append( { "type": "function", "function": { "name": llm_name, - "description": action.get("description", ""), + "description": description, "parameters": params, }, } ) return result - def _build_tool_parameters(self, action: Dict) -> Dict: + def _connection_names(self, tool_data: Dict) -> Optional[Tuple[str, str]]: + """``(service, account)`` a connection tool runs with, e.g. ``("Telegram", "Alerts bot")``.""" + if not tool_data.get("connection_id") or tool_data.get("client_side"): + return None + resolved = self._resolve_connection(tool_data) + if resolved is None or resolved.row is None: + return None + from docsgpt.connectors.service import account_name + + return resolved.connector_name or tool_data.get("name") or "", account_name(resolved.row) + + def _build_tool_parameters(self, action: Dict, hidden: Optional[set] = None) -> Dict: + """The JSON schema the model sees for ``action``. + + Parameters the model does not fill (fixed values) and ``hidden`` ones + (values the connection fixes) are left out, so the model is never + asked for them. + """ params = {"type": "object", "properties": {}, "required": []} - for param_type in ["query_params", "headers", "body", "parameters"]: - if param_type in action and action[param_type].get("properties"): - for k, v in action[param_type]["properties"].items(): - if v.get("filled_by_llm", True): - params["properties"][k] = { - key: value for key, value in v.items() if key not in ("filled_by_llm", "value", "required") - } - if v.get("required", False): - params["required"].append(k) + for _section, k, v in iter_parameters(action): + if not llm_fills(v) or (hidden and k in hidden): + continue + params["properties"][k] = { + key: value for key, value in v.items() if key not in ("filled_by_llm", "value", "required") + } + if v.get("required", False): + params["required"].append(k) return params def _guardrail_tool_result(self, result: Any, tool_name: str, action_name: str) -> Any: @@ -866,6 +993,70 @@ class ToolExecutor: return result return resolve_tool_result(result, decision) + def _resolve_connection(self, tool_data: Dict): + """The connection a connection-backed tool runs with this turn, or None.""" + if not tool_data.get("connection_id") or tool_data.get("client_side"): + return None + key = str(tool_data.get("id") or tool_data.get("connection_id")) + if key not in self._connections: + from docsgpt.connectors.resolve import resolve_connection + + try: + self._connections[key] = resolve_connection(tool_data, self.user) + except Exception: + logger.exception("connection resolution failed for tool %s", key) + self._connections[key] = None + return self._connections[key] + + def _connection_parameters(self, tool_data: Dict) -> Dict: + """Tool parameters the connection a call runs with sets, e.g. Telegram's default chat. + + Resolved like the credentials: in member mode each member's own + connection (and so their own chat) applies. Only connectors with + such fields are looked up. + """ + if not tool_data.get("connection_id") or tool_data.get("client_side"): + return {} + from docsgpt.connectors import catalog + + definition = catalog.definition_for_tool(tool_data.get("name") or "") + if definition is None or not catalog.parameter_fields(definition.key): + return {} + key = str(tool_data.get("id") or tool_data.get("connection_id")) + if key not in self._connection_params: + resolved = self._resolve_connection(tool_data) + params: Dict = {} + if resolved is not None and resolved.available: + from docsgpt.connectors.service import connection_parameters + + try: + params = connection_parameters(resolved.row) + except Exception: + logger.exception("connection parameters failed for tool %s", key) + self._connection_params[key] = params + return self._connection_params[key] + + @staticmethod + def _connection_payload(resolved) -> Dict: + """What the chat's Connect card needs; never an account or a secret. + + ``connection_id`` is only the caller's own connection, which the card + reconnects in place; an owner's account (``owner_account``) is not + the caller's to reconnect. + """ + payload = { + "connector_key": resolved.connector_key, + "connector_name": resolved.connector_name, + "status": ( + "missing" if resolved.row is None + else (resolved.row.get("status") or "reconnect_needed") + ), + "owner_account": bool(resolved.delegated), + } + if resolved.row is not None and not resolved.delegated: + payload["connection_id"] = str(resolved.row["id"]) + return payload + def check_pause(self, tools_dict: Dict, call, llm_class_name: str) -> Optional[Dict]: """Return a pending-action dict (approval / client / headless_denied) or None. @@ -912,6 +1103,41 @@ class ToolExecutor: "thought_signature": getattr(call, "thought_signature", None), } + # A tool whose connection needs signing in pauses on a Connect card + # (the approval card's connection variant) instead of failing; the + # user connects, then continues, and the pending call resumes. + resolved = self._resolve_connection(tool_data) + if resolved is not None and not resolved.available: + if self.headless or self.external_caller: + return { + "call_id": call_id, + "name": llm_name, + "tool_name": tool_data.get("name", "unknown"), + "tool_id": tool_id, + "action_name": action_name, + "llm_name": llm_name, + "arguments": arguments, + "pause_type": "headless_denied", + "deny_reason": ( + f"{resolved.connector_name or 'This service'} needs to be connected " + "before this tool can run." + ), + "error_type": "connection_required", + "thought_signature": getattr(call, "thought_signature", None), + } + return { + "call_id": call_id, + "name": llm_name, + "tool_name": tool_data.get("name", "unknown"), + "tool_id": tool_id, + "action_name": action_name, + "llm_name": llm_name, + "arguments": arguments, + "pause_type": "awaiting_approval", + "connection_required": self._connection_payload(resolved), + "thought_signature": getattr(call, "thought_signature", None), + } + # Approval required if tool_data["name"] == "api_tool": action_data = tool_data.get("config", {}).get("actions", {}).get(action_name, {}) @@ -948,6 +1174,74 @@ class ToolExecutor: or require_approval ) + # An admin forbade changes through this connector (GitHub): its tool + # already calls the read-only endpoint, so say why instead of failing. + if resolved is not None and not resolved.writes_allowed: + from docsgpt.connectors.permissions import ACCESS_WRITE, action_access + + if action_access(tool_data.get("name"), action_data) == ACCESS_WRITE: + return { + "call_id": call_id, + "name": llm_name, + "tool_name": tool_data.get("name", "unknown"), + "tool_id": tool_id, + "action_name": action_name, + "llm_name": llm_name, + "arguments": arguments, + "pause_type": "headless_denied", + "deny_reason": ( + f"An admin turned off changes through {resolved.connector_name or 'this service'}. " + "It can only look things up." + ), + "error_type": "tool_not_allowed", + "thought_signature": getattr(call, "thought_signature", None), + } + + # A member running someone else's account (a shared tool in owner + # mode) always confirms write actions, whatever the owner chose for + # themselves. + if not require_approval and resolved is not None and resolved.delegated: + from docsgpt.connectors.permissions import ACCESS_WRITE, action_access + + require_approval = action_access(tool_data.get("name"), action_data) == ACCESS_WRITE + + # An API-key caller writes with the owner's credentials only with the + # owner's say-so: nobody can approve in a widget, and "Always allow" + # was the owner's choice for themselves, not for anyone with the key. + # A public-link user is a stranger to the owner, so their approval + # can't stand in for the owner's either. + if (self.external_caller or self.public_link_caller) and self._on_owner_credentials( + tool_data, resolved, action_name + ): + from docsgpt.connectors.permissions import ACCESS_WRITE, action_access + + if action_access(tool_data.get("name"), action_data) == ACCESS_WRITE: + entry = f"{tool_data.get('id') or tool_id}:{action_name}" + if entry in self.api_write_allowlist: + return None + route = "for API or widget callers" if self.external_caller else "from its public link" + target = ( + f"the owner's {resolved.connector_name} account" + if resolved is not None and resolved.connector_name + else "the owner's credentials" + ) + return { + "call_id": call_id, + "name": llm_name, + "tool_name": tool_data.get("name", "unknown"), + "tool_id": tool_id, + "action_name": action_name, + "llm_name": llm_name, + "arguments": arguments, + "pause_type": "headless_denied", + "deny_reason": ( + f"This agent can't take this action with {target} {route}. " + "The owner can allow it in the agent's Access details." + ), + "error_type": "tool_not_allowed", + "thought_signature": getattr(call, "thought_signature", None), + } + if require_approval: if self.headless: tool_row_id = str(tool_data.get("id") or tool_id) @@ -982,6 +1276,12 @@ class ToolExecutor: "pause_type": "awaiting_approval", "thought_signature": getattr(call, "thought_signature", None), } + # The card shows what will be sent: fixed values replace what the + # model asked for. ``arguments`` stays as the model sent it, since + # resuming replays it to the model, which never sees fixed values. + sent = sent_arguments(action_data, arguments, self._connection_parameters(tool_data)) + if action_data and sent != arguments: + payload["sent_arguments"] = sent # Surface the device id so the approval UI can offer a # "don't ask again" sticky-pattern action for remote devices. if tool_data.get("name") == "remote_device": @@ -992,6 +1292,35 @@ class ToolExecutor: return None + def _on_owner_credentials(self, tool_data: Dict, resolved, action_name: Optional[str] = None) -> bool: + """Whether a call would act with credentials the caller doesn't hold. + + The connection's account when there is one, else the tool owner's + stored credentials (see ``holds_owner_credentials``). An API-key + caller and any scheduled run hold none of their own: the run acts as + the owner. A public-link user in the app holds their own account only. + + Args: + tool_data: The ``user_tools`` row being called. + resolved: The connection ``resolve_connection`` picked, or None. + action_name: The action called; an API tool action carries its + own headers and query values. + + Returns: + True when the call would use someone else's credentials. + """ + from docsgpt.connectors.permissions import holds_owner_credentials + + if resolved is not None: + holder = (resolved.row or {}).get("user_id") + elif holds_owner_credentials(tool_data, action_name): + holder = tool_data.get("user_id") + else: + return False + if self.external_caller or self.headless: + return True + return holder != self.user + def _remote_device_requires_approval( self, tool_data: Dict, @@ -1302,6 +1631,12 @@ class ToolExecutor: "arguments": call_args, } tool_data = tools_dict[tool_id] + # Name the service a connection-backed tool used, so the chip can show + # its logo ("Searched Notion"). Never the account behind it. + resolved = self._resolve_connection(tool_data) + if resolved is not None and resolved.connector_key: + tool_call_data["connector_key"] = resolved.connector_key + tool_call_data["connector_name"] = resolved.connector_name # Surface the device id on remote_device tool-call events so the # approval UI can wire up the sticky "don't ask again" button. if tool_data.get("name") == "remote_device": @@ -1347,35 +1682,40 @@ class ToolExecutor: else next(action for action in tool_data["actions"] if action["name"] == action_name) ) - query_params, headers, body, parameters = {}, {}, {}, {} - param_types = { - "query_params": query_params, - "headers": headers, - "body": body, - "parameters": parameters, - } + if "connector_key" in tool_call_data: + from docsgpt.connectors.permissions import action_access - for param_type, target_dict in param_types.items(): - if param_type in action_data and action_data[param_type].get("properties"): - for param, details in action_data[param_type]["properties"].items(): - if param not in call_args and "value" in details and details["value"]: - target_dict[param] = details["value"] - for param, value in call_args.items(): - for param_type, target_dict in param_types.items(): - if param_type in action_data and param in action_data[param_type].get("properties", {}): - target_dict[param] = value + tool_call_data["access"] = action_access(tool_data.get("name"), action_data) + + # Fixed values win over whatever the model sent for the same key. + sections = resolve_arguments(action_data, call_args, self._connection_parameters(tool_data)) + query_params, headers = sections["query_params"], sections["headers"] + body, parameters = sections["body"], sections["parameters"] + # The chat shows what was sent; ``arguments`` keeps what the model + # asked for, which is what a later turn replays to it. + sent = sent_arguments(action_data, call_args, self._connection_parameters(tool_data)) + if sent != (call_args if isinstance(call_args, dict) else {}): + tool_call_data["sent_arguments"] = sent # Load tool (with caching) - tool = self._get_or_load_tool( - tool_data, - tool_id, - action_name, - headers=headers, - query_params=query_params, - ) + from docsgpt.connectors.service import ConnectionUnavailable + + connection_error = None + try: + tool = self._get_or_load_tool( + tool_data, + tool_id, + action_name, + headers=headers, + query_params=query_params, + ) + except ConnectionUnavailable as exc: + tool, connection_error = None, str(exc) if tool is None: - error_message = ( + error_message = connection_error and ( + f"{connection_error}. Ask the user to connect it in Settings > Connectors, then try again." + ) or ( f"Failed to load tool '{tool_data.get('name')}' (tool_id key={tool_id}): missing 'id' on tool row." ) logger.error( @@ -1535,6 +1875,7 @@ class ToolExecutor: return cached tm = ToolManager(config={}) + load_user = self.user if tool_data["name"] == "api_tool": action_config = tool_data["config"]["actions"][action_name] @@ -1549,6 +1890,9 @@ class ToolExecutor: tool_config["body_encoding_rules"] = action_config.get("body_encoding_rules", {}) else: tool_config = tool_data["config"].copy() if tool_data["config"] else {} + # Whose MCP tokens a tool uses is decided by resolving its + # connection below, never by a value stored in its config. + tool_config.pop("connection_id", None) # Credentials are PBKDF2-bound to the tool OWNER's sub, not the # invoker's. Decrypt with the tool row's user_id so a team member # running an owner's shared tool authenticates with the owner's @@ -1557,7 +1901,17 @@ class ToolExecutor: # silently decrypt-failing. Falls back to self.user for the # agentless path where the tool row carries no user_id. tool_owner = tool_data.get("user_id") or self.user - if tool_config.get("encrypted_credentials") and tool_owner: + resolved = self._resolve_connection(tool_data) + if tool_data["name"] == "mcp_tool": + # MCP OAuth tokens are read as the account the call runs on: + # the resolved connection's owner (a member's own in member + # mode), else the tool owner, so a shared server without a + # connection runs on the owner's sign-in like every other + # credential. + load_user = ((resolved.row or {}).get("user_id") if resolved else None) or tool_owner + if resolved is not None: + self._apply_connection(tool_data, tool_id, tool_config, resolved) + elif tool_config.get("encrypted_credentials") and tool_owner: if tool_owner != self.user: # Credential delegation: the invoker is running a shared # tool with the owner's secrets. Audit it (the agent-run @@ -1606,12 +1960,12 @@ class ToolExecutor: # falls back to ``origin_conversation_id`` as the schedule's # conversation home. tool_config["agent_id"] = str(self.agent_id) if self.agent_id else None + if self.external_caller: + # Its runs act as the owner for an API-key caller. + tool_config["created_via"] = "api" if tool_data["name"] == "mcp_tool": tool_config["query_mode"] = True - # MCP OAuth tokens are looked up by user id: a shared server runs on - # the owner's connection, like every other credential. - load_user = (tool_data.get("user_id") or self.user) if tool_data["name"] == "mcp_tool" else self.user tool = tm.load_tool( tool_data["name"], tool_config=tool_config, @@ -1624,10 +1978,83 @@ class ToolExecutor: return tool + def _apply_connection(self, tool_data: Dict, tool_id: str, tool_config: Dict, resolved) -> None: + """Merge a connection's credentials into ``tool_config``. + + A connection only serves the tools its connector provides (a Telegram + bot token never reaches an ntfy tool), and an MCP connection's secret + only goes to the server it was stored for. + + Args: + tool_data: The ``user_tools`` row being run. + tool_id: The tool's key in this run. + tool_config: The config the tool is loaded with, updated in place. + resolved: The connection :func:`resolve_connection` picked. + + Raises: + ConnectionUnavailable: The connection needs signing in again, or + does not belong to this tool's connector or server. + """ + from docsgpt.connectors import catalog, service + from docsgpt.connectors.resolve import audit_delegation + + unavailable = service.ConnectionUnavailable( + f"{resolved.connector_name or 'This service'} needs to be connected", + connection_id=resolved.connection_id, + ) + if not resolved.available or resolved.row is None: + raise unavailable + tool_name = tool_data.get("name") + definition = catalog.get_definition(resolved.connector_key) + if definition is None or tool_name not in definition.tool_templates: + logger.warning( + "tool %s (%s) points at a %s connection", tool_data.get("id") or tool_id, tool_name, + resolved.connector_key, + ) + raise unavailable + if tool_name == "mcp_tool": + # A connection's secret only goes to the server it was stored for: + # a custom server's own URL (legacy rows name it in ``provider``), + # or a preset's or built-in connector's MCP server (GitHub's token + # only ever goes to GitHub's). No server known, nothing is sent. + stored_for = catalog.base_url(resolved.row.get("server_url")) + provider = str(resolved.row.get("provider") or "") + if not stored_for and provider.startswith("mcp:"): + stored_for = catalog.base_url(provider[len("mcp:"):]) + if not stored_for: + stored_for = definition.mcp_base_url or "" + if not stored_for or stored_for != catalog.base_url(tool_config.get("server_url")): + raise unavailable + if service.builtin_mcp_config(definition) is not None: + # Only the connector's own endpoints: GitHub's write one while + # the tool opted in and an admin allows it, else read-only. + tool_config["server_url"] = service.builtin_mcp_url( + definition, tool_config.get("server_url"), resolved.writes_allowed, + ) + audit_delegation( + resolved, + invoker=self.user, + resource_type="tool", + resource_id=str(tool_data.get("id") or tool_id), + agent_id=self.agent_id, + ) + tool_config.pop("encrypted_credentials", None) + if (resolved.row.get("auth_kind") or "") in ("api_key", "none", "oauth"): + # Pasted keys, or the current access token of a built-in OAuth + # sign-in (GitHub's App), refreshed first when it has expired. + credentials = service.access_credentials(resolved.row) + tool_config.update(credentials) + tool_config["auth_credentials"] = credentials + if tool_data.get("name") == "mcp_tool": + # MCP OAuth tokens are read by connection id inside the tool. + tool_config["connection_id"] = resolved.connection_id + # Keys the client needs that are not part of the fixed shape below. They are # small and optional, and are copied only when present so an ordinary tool # call does not grow null columns in every persisted row. - _PRESERVED_TOOL_CALL_KEYS = ("artifacts", "device_id") + _PRESERVED_TOOL_CALL_KEYS = ( + "artifacts", "device_id", "connector_key", "connector_name", "access", "sent_arguments", + ) def get_truncated_tool_calls(self) -> List[Dict]: """Project tool calls into the shape that is streamed and persisted. diff --git a/docsgpt/agents/tool_pins.py b/docsgpt/agents/tool_pins.py new file mode 100644 index 00000000..6a9847c9 --- /dev/null +++ b/docsgpt/agents/tool_pins.py @@ -0,0 +1,334 @@ +"""Fixed ("pinned") tool parameters. + +Every parameter of a stored tool action carries ``filled_by_llm`` and +``value``. A parameter with ``filled_by_llm`` false is hidden from the model; +when it also has a value, that value is *pinned*: it is sent on every call and +the model can neither see nor override it. An empty string means "no value": +the parameter is left out of the call (optional OpenAPI parameters are stored +that way), while ``0`` and ``false`` are real values. + +The run-time rules live in :func:`resolve_arguments`; the helpers below +validate and apply pin changes coming from the API. +""" + +from __future__ import annotations + +import copy +from typing import Any, Iterable, Mapping, Optional + +#: The places an action keeps parameter schemas: ``parameters`` for ordinary +#: tools and MCP, the other three for ``api_tool`` requests. +PARAM_SECTIONS = ("query_params", "headers", "body", "parameters") + +_NUMERIC = {"integer": int, "number": float} + + +def _properties(action: Mapping, section: str) -> dict: + block = action.get(section) + if not isinstance(block, dict): + return {} + properties = block.get("properties") + return properties if isinstance(properties, dict) else {} + + +def iter_parameters(action: Mapping) -> Iterable[tuple[str, str, dict]]: + """``(section, name, details)`` for every parameter schema of ``action``.""" + for section in PARAM_SECTIONS: + for name, details in _properties(action, section).items(): + if isinstance(details, dict): + yield section, name, details + + +def llm_fills(details: Mapping) -> bool: + """Whether the model fills this parameter (the default when unset).""" + return bool(details.get("filled_by_llm", True)) + + +def has_value(details: Mapping) -> bool: + """Whether a parameter carries a value: anything but a missing, None or empty string.""" + value = details.get("value") + return value is not None and value != "" + + +def is_pinned(details: Mapping) -> bool: + """Whether the parameter is hidden from the model and always sent with its value.""" + return not llm_fills(details) and has_value(details) + + +def pinned_names(action: Mapping) -> list[str]: + """Names of the parameters of ``action`` that carry a pinned value.""" + return [name for _section, name, details in iter_parameters(action) if is_pinned(details)] + + +def resolve_arguments( + action: Mapping, + llm_arguments: Mapping[str, Any], + connection_pins: Optional[Mapping[str, Any]] = None, +) -> dict[str, dict]: + """The arguments a call runs with, per parameter section. + + Only parameters the action declares are passed. For each one: + + * a value the connection fixes (``connection_pins``, e.g. Telegram's + default chat) always wins; + * a parameter hidden from the model takes its stored value, or is left + out when it has none; whatever the model sent for it is ignored, so a + model that names the key anyway (a mistake, or a prompt injection) + changes nothing; + * any other parameter takes the model's value, falling back to its + stored default. + + Args: + action: The stored action (``parameters`` / ``query_params`` / + ``headers`` / ``body`` schemas with ``filled_by_llm`` and ``value``). + llm_arguments: The arguments the model sent. + connection_pins: Parameter name to a value the connection fixes. + + Returns: + Section name to ``{parameter: value}`` for every section in + :data:`PARAM_SECTIONS` (empty dicts included). + """ + connection_pins = connection_pins or {} + resolved: dict[str, dict] = {section: {} for section in PARAM_SECTIONS} + for section, name, details in iter_parameters(action): + target = resolved[section] + if name in connection_pins: + target[name] = connection_pins[name] + elif not llm_fills(details): + if has_value(details): + target[name] = details["value"] + elif name in llm_arguments: + target[name] = llm_arguments[name] + elif has_value(details): + target[name] = details["value"] + return resolved + + +#: What the chat shows in place of a value the owner fixed. +FIXED_MASK = "(fixed)" + + +def sent_arguments( + action: Mapping, + llm_arguments: Mapping[str, Any], + connection_pins: Optional[Mapping[str, Any]] = None, +) -> dict: + """What a call sends, flattened for the chat (the approval card, a finished call). + + It follows :func:`resolve_arguments`, but any value that comes from the + stored action rather than from the model shows as :data:`FIXED_MASK`: + a fixed value, and a default the model left alone, may be a secret (an + api_tool query value is decrypted into the action before the call), and + the chat is shown to whoever runs the agent, over the API or a widget + too. A value the connection sets (Telegram's default chat) is the + account's own setting and shows as it is. Headers are left out. + + Args: + action: The stored action, with any secrets merged back. + llm_arguments: The arguments the model sent. + connection_pins: Parameter name to a value the connection fixes. + + Returns: + Parameter name to the value shown for it. + """ + connection_pins = connection_pins or {} + shown: dict = {} + for section, name, details in iter_parameters(action): + if section == "headers": + continue + if name in connection_pins: + shown[name] = connection_pins[name] + elif not llm_fills(details): + if has_value(details): + shown[name] = FIXED_MASK + elif name in llm_arguments: + shown[name] = llm_arguments[name] + elif has_value(details): + shown[name] = FIXED_MASK + return shown + + +def coerce_value(details: Mapping, value: Any) -> Any: + """Check a value for a pin against the parameter's declared type. + + Strings typed into a form are converted for ``integer``, ``number`` and + ``boolean`` parameters. + + Raises: + ValueError: The value does not fit the type, or is empty. + """ + if value is None or value == "": + raise ValueError("A fixed value cannot be empty") + kind = details.get("type") + if isinstance(kind, list): + kind = next((k for k in kind if k != "null"), None) + if kind in _NUMERIC: + if isinstance(value, bool): + raise ValueError(f"Expected a {kind}") + try: + number = _NUMERIC[kind](value) + except (TypeError, ValueError): + raise ValueError(f"Expected a {kind}") from None + if kind == "integer" and isinstance(value, float) and not value.is_integer(): + raise ValueError("Expected an integer") + return number + if kind == "boolean": + if isinstance(value, bool): + return value + if isinstance(value, str) and value.strip().lower() in ("true", "false"): + return value.strip().lower() == "true" + raise ValueError("Expected true or false") + if kind == "array": + if not isinstance(value, list): + raise ValueError("Expected a list") + return value + if kind == "object": + if not isinstance(value, dict): + raise ValueError("Expected an object") + return value + if kind in (None, "string"): + if not isinstance(value, (str, int, float, bool)): + raise ValueError("Expected text") + return str(value) if kind == "string" else value + return value + + +def set_pins(action: Mapping, pins: Mapping[str, Any]) -> dict: + """Return ``action`` with ``pins`` applied. + + Args: + action: A stored action. + pins: Parameter name to the value to always use, or None to let the + model decide again. + + Raises: + ValueError: A parameter does not exist on the action, or a value does + not fit its type. + """ + updated = copy.deepcopy(dict(action)) + known = {name: details for _section, name, details in iter_parameters(updated)} + for name, value in pins.items(): + details = known.get(name) + if details is None: + raise ValueError(f"Unknown parameter: {name}") + if value is None: + details["filled_by_llm"] = True + details["value"] = "" + else: + details["value"] = coerce_value(details, value) + details["filled_by_llm"] = False + return updated + + +class PinChangeRefused(PermissionError): + """Someone other than the tool's owner tried to change a fixed value.""" + + +_ACTION_KEYS = ("active", "require_approval", "description") +_PARAM_KEYS = ("filled_by_llm", "value", "description") + + +def _pin_state(details: Mapping) -> tuple: + return (llm_fills(details), details.get("value") if has_value(details) else None) + + +def merge_submitted_actions(stored: list, submitted: Any, *, may_change_pins: bool) -> list: + """Validate a full list of actions sent by a client against the stored ones. + + The stored actions are the schema: a client can switch actions on and off, + change approval and descriptions and (the owner only) fixed values, but + cannot add actions or parameters, or change a parameter's type. + + Args: + stored: The tool's stored actions. + submitted: What the client sent (untrusted). + may_change_pins: False for a team editor: any change to + ``filled_by_llm`` or ``value`` is refused. + + Returns: + The actions to store: every stored action, updated from the + submission. + + Raises: + ValueError: The submission is malformed or names an unknown action or + parameter, or a fixed value does not fit its parameter. + PinChangeRefused: ``may_change_pins`` is False and a fixed value changed. + """ + if not isinstance(submitted, list): + raise ValueError("actions must be a list") + by_name = {a.get("name"): a for a in stored if isinstance(a, dict) and a.get("name")} + seen: set = set() + updates: dict[str, dict] = {} + for entry in submitted: + if not isinstance(entry, dict) or not isinstance(entry.get("name"), str): + raise ValueError("Each action needs a name") + name = entry["name"] + if name not in by_name: + raise ValueError(f"Unknown action: {name}") + if name in seen: + raise ValueError(f"Duplicate action: {name}") + seen.add(name) + updates[name] = entry + + merged = [] + for action in stored: + if not isinstance(action, dict) or action.get("name") not in updates: + merged.append(action) + continue + entry = updates[action["name"]] + result = copy.deepcopy(action) + for key in _ACTION_KEYS: + if key in entry: + value = entry[key] + if key == "description" and not isinstance(value, str): + raise ValueError("description must be text") + result[key] = bool(value) if key != "description" else value + for section in PARAM_SECTIONS: + sent = _properties(entry, section) + if not sent: + continue + properties = _properties(result, section) + for param, sent_details in sent.items(): + details = properties.get(param) + if details is None or not isinstance(details, dict): + raise ValueError(f"Unknown parameter {param} on {action['name']}") + if not isinstance(sent_details, dict): + raise ValueError(f"Parameter {param} must be an object") + before = _pin_state(details) + for key in _PARAM_KEYS: + if key in sent_details: + details[key] = sent_details[key] + details["filled_by_llm"] = llm_fills(details) + if _pin_state(details) == before: + continue + if not may_change_pins: + raise PinChangeRefused("Only the tool's owner can change fixed values") + if is_pinned(details): + details["value"] = coerce_value(details, details["value"]) + merged.append(result) + return merged + + +def carry_pins_between(previous: Optional[list], fresh: list) -> list: + """:func:`carry_pins` for whole action lists, matching actions by name.""" + old = {a.get("name"): a for a in previous or [] if isinstance(a, dict)} + return [carry_pins(old.get(a.get("name")), a) if isinstance(a, dict) else a for a in fresh] + + +def carry_pins(previous: Optional[Mapping], fresh: Mapping) -> dict: + """Copy fixed values from an action's old copy onto its re-discovered one. + + Parameters that still exist keep ``filled_by_llm`` and ``value``; removed + ones are gone with the new schema. + """ + result = copy.deepcopy(dict(fresh)) + if not previous: + return result + old = {(section, name): details for section, name, details in iter_parameters(previous)} + for section, name, details in iter_parameters(result): + before = old.get((section, name)) + if before is None: + continue + details["filled_by_llm"] = llm_fills(before) + details["value"] = before.get("value", "") + return result diff --git a/docsgpt/agents/tools/brave.py b/docsgpt/agents/tools/brave.py index 532e310f..c58d784a 100644 --- a/docsgpt/agents/tools/brave.py +++ b/docsgpt/agents/tools/brave.py @@ -136,6 +136,7 @@ class BraveSearchTool(Tool): return [ { "name": "brave_web_search", + "access": "read", "description": ( "Search the web with Brave Search. Returns result titles, " "URLs, and snippets. Use it for current events or " @@ -163,6 +164,7 @@ class BraveSearchTool(Tool): }, { "name": "brave_image_search", + "access": "read", "description": ( "Search for images with Brave Search. Returns image " "titles, page URLs, and thumbnail URLs." diff --git a/docsgpt/agents/tools/mcp_tool.py b/docsgpt/agents/tools/mcp_tool.py index 061f4c6d..d429428d 100644 --- a/docsgpt/agents/tools/mcp_tool.py +++ b/docsgpt/agents/tools/mcp_tool.py @@ -1,10 +1,11 @@ import asyncio import base64 import concurrent.futures +import hashlib import json import logging import time -from typing import Any, Dict, List, Optional +from typing import Any, Callable, Dict, List, Optional from urllib.parse import parse_qs, urlparse from fastmcp import Client @@ -15,7 +16,12 @@ from fastmcp.client.transports import ( StreamableHttpTransport, ) from mcp.client.auth import OAuthClientProvider, TokenStorage -from mcp.shared.auth import OAuthClientInformationFull, OAuthClientMetadata, OAuthToken +from mcp.shared.auth import ( + AuthorizationCodeResult, + OAuthClientInformationFull, + OAuthClientMetadata, + OAuthToken, +) from pydantic import AnyHttpUrl, ValidationError from redis import Redis @@ -32,12 +38,54 @@ logger = logging.getLogger(__name__) _mcp_clients_cache = {} +def forget_cached_clients(*identities: str) -> None: + """Drop cached OAuth clients signed in as any of ``identities``. + + OAuth cache keys name the connection or user whose tokens the client + holds (see ``MCPTool._generate_cache_key``). + + Args: + identities: Connection ids and user ids. + """ + markers = tuple(f"#oauth:{identity}:" for identity in identities if identity) + for key in [k for k in list(_mcp_clients_cache) if any(marker in k for marker in markers)]: + _mcp_clients_cache.pop(key, None) + +# A token expiry long past: a stored token of unknown age is renewed before use. +_EXPIRED = 1.0 + +_ANNOTATION_HINTS = ("readOnlyHint", "destructiveHint", "idempotentHint", "openWorldHint") + + +def _secret_fingerprint(secret: str) -> str: + """A short digest of a whole secret, for telling cached clients apart. + + A prefix of the secret itself is not enough: every fine-grained GitHub + token starts with ``github_pat_``, so two users' tokens would share (and + reuse) one cached client carrying the first user's token. + """ + return hashlib.sha256(secret.encode("utf-8")).hexdigest()[:16] + + +def _annotation_hints(annotations: Any) -> Dict[str, bool]: + """The boolean MCP tool annotation hints, from a model or a dict.""" + if annotations is None: + return {} + if hasattr(annotations, "model_dump"): + annotations = annotations.model_dump() + if not isinstance(annotations, dict): + return {} + return {k: annotations[k] for k in _ANNOTATION_HINTS if isinstance(annotations.get(k), bool)} + + class MCPTool(Tool): """ MCP Tool Connect to remote Model Context Protocol (MCP) servers to access dynamic tools and resources. """ + connection_id: Optional[str] = None + def __init__(self, config: Dict[str, Any], user_id: Optional[str] = None): """ Initialize the MCP Tool with configuration. @@ -76,6 +124,10 @@ class MCPTool(Tool): self.oauth_scopes = config.get("oauth_scopes", []) self.oauth_task_id = config.get("oauth_task_id", None) self.oauth_client_name = config.get("oauth_client_name", "DocsGPT-MCP") + # The connection whose OAuth tokens this tool uses. Set by the tool + # executor for connection-backed tools, so a shared tool in ``owner`` + # mode signs in with its owner's account rather than the invoker's. + self.connection_id = config.get("connection_id") self.redirect_uri = self._resolve_redirect_uri(config.get("redirect_uri")) # Pulled out of ``config`` (rather than left in ``self.config``) # because it is a callable supplied by the OAuth worker — not @@ -105,13 +157,16 @@ class MCPTool(Tool): raise ValueError(f"Invalid MCP server URL: {exc}") from exc def _resolve_redirect_uri(self, configured_redirect_uri: Optional[str]) -> str: - if configured_redirect_uri: - return configured_redirect_uri.rstrip("/") - + # The operator's setting wins over the page's own origin: a page opened + # on a plain-HTTP address would otherwise register a callback that + # servers such as Linear refuse. explicit = settings.MCP_OAUTH_REDIRECT_URI if explicit: return explicit.rstrip("/") + if configured_redirect_uri: + return configured_redirect_uri.rstrip("/") + connector_base = settings.CONNECTOR_REDIRECT_BASE_URI if connector_base: parsed = urlparse(connector_base) @@ -125,7 +180,9 @@ class MCPTool(Tool): auth_key = "" if self.auth_type == "oauth": scopes_str = ",".join(self.oauth_scopes) if self.oauth_scopes else "none" - oauth_identity = self.user_id or self.oauth_task_id or "anonymous" + # A connection-backed tool shares a client only with calls that + # use the same connection's tokens. + oauth_identity = self.connection_id or self.user_id or self.oauth_task_id or "anonymous" auth_key = ( f"oauth:{oauth_identity}:{self.oauth_client_name}:{scopes_str}:{self.redirect_uri}" ) @@ -133,10 +190,10 @@ class MCPTool(Tool): token = self.auth_credentials.get( "bearer_token", "" ) or self.auth_credentials.get("access_token", "") - auth_key = f"bearer:{token[:10]}..." if token else "bearer:none" + auth_key = f"bearer:{_secret_fingerprint(token)}" if token else "bearer:none" elif self.auth_type == "api_key": api_key = self.auth_credentials.get("api_key", "") - auth_key = f"apikey:{api_key[:10]}..." if api_key else "apikey:none" + auth_key = f"apikey:{_secret_fingerprint(api_key)}" if api_key else "apikey:none" elif self.auth_type == "basic": username = self.auth_credentials.get("username", "") auth_key = f"basic:{username}" @@ -165,6 +222,7 @@ class MCPTool(Tool): redis_client=redis_client, redirect_uri=self.redirect_uri, user_id=self.user_id, + connection_id=self.connection_id, ) else: auth = DocsGPTOAuth( @@ -175,6 +233,7 @@ class MCPTool(Tool): task_id=self.oauth_task_id, user_id=self.user_id, redirect_publish=self.oauth_redirect_publish, + connection_id=self.connection_id, ) elif self.auth_type == "bearer": token = self.auth_credentials.get( @@ -245,6 +304,9 @@ class MCPTool(Tool): } if hasattr(tool, "inputSchema"): tool_dict["inputSchema"] = tool.inputSchema + annotations = _annotation_hints(getattr(tool, "annotations", None)) + if annotations: + tool_dict["annotations"] = annotations tools_dict.append(tool_dict) elif isinstance(tool, dict): tools_dict.append(tool) @@ -493,7 +555,7 @@ class MCPTool(Tool): def _test_oauth_connection(self) -> Dict: storage = DBTokenStorage( - server_url=self.server_url, user_id=self.user_id, + server_url=self.server_url, user_id=self.user_id, connection_id=self.connection_id, ) loop = asyncio.new_event_loop() try: @@ -580,6 +642,11 @@ class MCPTool(Tool): "description": tool.get("description", ""), "parameters": parameters_schema, } + # ``readOnlyHint`` / ``destructiveHint`` decide whether the action + # is a read (always allowed) or a write (needs approval). + annotations = _annotation_hints(tool.get("annotations")) + if annotations: + action["annotations"] = annotations actions.append(action) return actions @@ -670,6 +737,10 @@ class MCPTool(Tool): } +class MCPReauthorizationRequired(Exception): + """An MCP sign-in expired and could not be renewed: its owner must sign in again.""" + + class DocsGPTOAuth(OAuthClientProvider): """ Custom OAuth handler for DocsGPT that uses frontend redirect instead of browser. @@ -688,6 +759,7 @@ class DocsGPTOAuth(OAuthClientProvider): additional_client_metadata: dict[str, Any] | None = None, skip_redirect_validation: bool = False, redirect_publish=None, + connection_id: Optional[str] = None, ): self.redirect_uri = redirect_uri self.redis_client = redis_client @@ -717,19 +789,70 @@ class DocsGPTOAuth(OAuthClientProvider): server_url=self.server_base_url, user_id=self.user_id, expected_redirect_uri=None if skip_redirect_validation else redirect_uri, + connection_id=connection_id, ) + # The SDK checks the server's protected-resource metadata against a + # resource derived from this URL, so it gets the full MCP endpoint: + # Linear and Sentry publish ``https://host/mcp``, and an origin-only + # URL fails that check before sign-in. Tokens stay keyed by origin. super().__init__( - server_url=self.server_base_url, + server_url=mcp_url.rstrip("/") or self.server_base_url, client_metadata=client_metadata, storage=storage, redirect_handler=self.redirect_handler, callback_handler=self.callback_handler, ) + self.context.prepare_token_auth = self._one_client_authentication(self.context.prepare_token_auth) self.auth_url = None self.extracted_state = None + @staticmethod + def _one_client_authentication(prepare: Callable) -> Callable: + """Wrap the SDK's token-request auth so the client authenticates one way. + + With ``client_secret_basic`` the SDK sets the Basic header but leaves + ``client_id`` in the body, which servers such as Linear reject as a + second method; RFC 6749 names the client in the body only when it does + not authenticate otherwise. + + Args: + prepare: The SDK context's ``prepare_token_auth``. + + Returns: + The same function, minus ``client_id`` in the body under Basic auth. + """ + + def prepare_token_auth( + data: dict[str, str], headers: dict[str, str] | None = None + ) -> tuple[dict[str, str], dict[str, str]]: + data, headers = prepare(data, headers) + if headers.get("Authorization", "").startswith("Basic "): + data = {k: v for k, v in data.items() if k != "client_id"} + return data, headers + + return prepare_token_auth + + async def _initialize(self) -> None: + """Load the stored sign-in along with when its access token expires. + + The SDK learns a token's expiry only from a token response in this + process, so a token read back from the connection gets the expiry + saved with it, and an expired one is renewed with its refresh token + before it is sent. One saved before expiries were kept is of unknown + age and is renewed once. + """ + await super()._initialize() + tokens = self.context.current_tokens + if tokens is None: + return + expires_at = getattr(self.context.storage, "expires_at", None) + if expires_at: + self.context.token_expiry_time = float(expires_at) + elif tokens.refresh_token and tokens.expires_in: + self.context.token_expiry_time = _EXPIRED + def _process_auth_url(self, authorization_url: str) -> tuple[str, str]: """Process authorization URL to extract state""" try: @@ -771,8 +894,13 @@ class DocsGPTOAuth(OAuthClientProvider): exc_info=True, ) - async def callback_handler(self) -> tuple[str, str | None]: - """Wait for auth code from Redis using the state value.""" + async def callback_handler(self) -> AuthorizationCodeResult: + """Wait for auth code from Redis using the state value. + + Returns: + The code, the state it came back with, and the RFC 9207 issuer + when the authorization server sent one. + """ if not self.redis_client or not self.extracted_state: raise Exception("Redis client or state not configured for OAuth") poll_interval = 1 @@ -785,15 +913,22 @@ class DocsGPTOAuth(OAuthClientProvider): if code_data: code = code_data.decode() returned_state = self.extracted_state + iss_key = f"{self.redis_prefix}iss:{self.extracted_state}" + iss_data = self.redis_client.get(iss_key) self.redis_client.delete(code_key) + self.redis_client.delete(iss_key) self.redis_client.delete( f"{self.redis_prefix}auth_url:{self.extracted_state}" ) self.redis_client.delete( f"{self.redis_prefix}state:{self.extracted_state}" ) - return code, returned_state + return AuthorizationCodeResult( + code=code, + state=returned_state, + iss=iss_data.decode() if iss_data else None, + ) error_key = f"{self.redis_prefix}error:{self.extracted_state}" error_data = self.redis_client.get(error_key) if error_data: @@ -825,26 +960,39 @@ class NonInteractiveOAuth(DocsGPTOAuth): super().__init__(**kwargs) async def redirect_handler(self, authorization_url: str) -> None: - raise Exception( + raise MCPReauthorizationRequired( "OAuth session expired — please re-authorize this MCP server in tool settings." ) - async def callback_handler(self) -> tuple[str, str | None]: - raise Exception( + async def callback_handler(self) -> AuthorizationCodeResult: + raise MCPReauthorizationRequired( "OAuth session expired — please re-authorize this MCP server in tool settings." ) class DBTokenStorage(TokenStorage): + """MCP OAuth tokens and client registration, kept encrypted on the connection. + + Reads and writes go through ``docsgpt.connectors.service``, which stores + them in the connection's owner-bound ``encrypted_credentials``. A tool + that runs with a specific connection (``owner`` mode on a shared tool) + passes its ``connection_id``; otherwise the invoking user's connection + for the server's base URL is used. + """ + def __init__( self, server_url: str, user_id: str, expected_redirect_uri: Optional[str] = None, + connection_id: Optional[str] = None, ): self.server_url = server_url self.user_id = user_id self.expected_redirect_uri = expected_redirect_uri + self.connection_id = connection_id + # When the stored access token expires (epoch seconds), once read. + self.expires_at: Optional[float] = None @staticmethod def get_base_url(url: str) -> str: @@ -855,31 +1003,18 @@ class DBTokenStorage(TokenStorage): return f"mcp:{self.get_base_url(self.server_url)}" def _fetch_session_data(self) -> dict: - """Read the JSONB ``session_data`` blob for this MCP server row.""" - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - from docsgpt.storage.db.session import db_readonly + """The decrypted ``tokens`` / ``client_info`` for this MCP server.""" + from docsgpt.connectors import service - base_url = self.get_base_url(self.server_url) - with db_readonly() as conn: - row = ConnectorSessionsRepository(conn).get_by_user_and_server_url( - self.user_id, base_url, - ) - if not row: - return {} - data = row.get("session_data") or {} - if isinstance(data, str): - try: - data = json.loads(data) - except ValueError: - return {} - return data if isinstance(data, dict) else {} + return service.read_mcp_secrets( + self.user_id, self.get_base_url(self.server_url), self.connection_id, + ) async def get_tokens(self) -> OAuthToken | None: data = await asyncio.to_thread(self._fetch_session_data) if not data or "tokens" not in data: return None + self.expires_at = data.get("tokens_expires_at") try: return OAuthToken.model_validate(data["tokens"]) except ValidationError as e: @@ -887,22 +1022,19 @@ class DBTokenStorage(TokenStorage): return None def _merge(self, patch: dict) -> None: - """Shallow-merge ``patch`` into this row's ``session_data``. + """Merge ``patch`` into the connection's secrets; ``None`` drops a key.""" + from docsgpt.connectors import service - Threads ``server_url`` through to the repository so it lands in - the scalar column — ``get_by_user_and_server_url`` needs that to - resolve the row (``NULL = 'https://...'`` is UNKNOWN in SQL). - """ - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, + status = None + if patch.get("tokens"): + status = service.STATUS_CONNECTED + service.update_mcp_secrets( + self.user_id, + self.get_base_url(self.server_url), + patch, + connection_id=self.connection_id, + status=status, ) - from docsgpt.storage.db.session import db_session - - base_url = self.get_base_url(self.server_url) - with db_session() as conn: - ConnectorSessionsRepository(conn).merge_session_data( - self.user_id, self._pg_provider(), base_url, patch, - ) def _delete(self) -> None: from docsgpt.storage.db.repositories.connector_sessions import ( @@ -918,7 +1050,10 @@ class DBTokenStorage(TokenStorage): async def set_tokens(self, tokens: OAuthToken) -> None: base_url = self.get_base_url(self.server_url) token_dump = tokens.model_dump() - await asyncio.to_thread(self._merge, {"tokens": token_dump}) + # Kept beside the token: a later process cannot tell from + # ``expires_in`` alone whether it has expired. + self.expires_at = time.time() + tokens.expires_in if tokens.expires_in else None + await asyncio.to_thread(self._merge, {"tokens": token_dump, "tokens_expires_at": self.expires_at}) logger.info("Saved tokens for %s", base_url) async def get_client_info(self) -> OAuthClientInformationFull | None: @@ -1003,7 +1138,7 @@ class MCPOAuthManager: self.redis_prefix = redis_prefix def handle_oauth_callback( - self, state: str, code: str, error: Optional[str] = None + self, state: str, code: str, error: Optional[str] = None, iss: Optional[str] = None ) -> bool: """ Handle OAuth callback from provider. @@ -1012,6 +1147,7 @@ class MCPOAuthManager: state: The state parameter from OAuth callback code: The authorization code from OAuth callback error: Error message if OAuth failed + iss: The RFC 9207 issuer from the callback, if the server sent one Returns: True if successful, False otherwise @@ -1023,6 +1159,9 @@ class MCPOAuthManager: error_key = f"{self.redis_prefix}error:{state}" self.redis_client.setex(error_key, 300, error) raise Exception(f"OAuth error received: {error}") + # The issuer goes first: the waiting sign-in reads it once the code lands. + if iss: + self.redis_client.setex(f"{self.redis_prefix}iss:{state}", 300, iss) code_key = f"{self.redis_prefix}code:{state}" self.redis_client.setex(code_key, 300, code) diff --git a/docsgpt/agents/tools/ntfy.py b/docsgpt/agents/tools/ntfy.py index 8d52d3f5..d719211f 100644 --- a/docsgpt/agents/tools/ntfy.py +++ b/docsgpt/agents/tools/ntfy.py @@ -89,6 +89,7 @@ class NtfyTool(Tool): return [ { "name": "ntfy_send_message", + "access": "write", "description": ( "Send a push notification to an ntfy topic on the " "configured server. Provide the message text; title and " diff --git a/docsgpt/agents/tools/postgres.py b/docsgpt/agents/tools/postgres.py index 35aaab81..d465faee 100644 --- a/docsgpt/agents/tools/postgres.py +++ b/docsgpt/agents/tools/postgres.py @@ -136,6 +136,7 @@ class PostgresTool(Tool): return [ { "name": "postgres_execute_sql", + "access": "write", "description": "Execute an SQL query against the PostgreSQL database and return the results. Use this tool to interact with the database, e.g., retrieve specific data or perform updates. Only SELECT queries will return data, other queries will return execution status.", "parameters": { "type": "object", @@ -151,6 +152,7 @@ class PostgresTool(Tool): }, { "name": "postgres_get_schema", + "access": "read", "description": "Retrieve the schema of the PostgreSQL database, including tables and their columns. Use this to understand the database structure before executing queries. db_name is 'default' if not provided.", "parameters": { "type": "object", diff --git a/docsgpt/agents/tools/scheduler.py b/docsgpt/agents/tools/scheduler.py index cc0b64ed..bb30d675 100644 --- a/docsgpt/agents/tools/scheduler.py +++ b/docsgpt/agents/tools/scheduler.py @@ -44,6 +44,9 @@ class SchedulerTool(Tool): self.user_id: Optional[str] = user_id self.agent_id: Optional[str] = cfg.get("agent_id") self.conversation_id: Optional[str] = cfg.get("conversation_id") + # ``api`` when an API-key caller asked (set by the executor, never the + # model): the schedule's runs then keep that caller's write limits. + self.created_via: str = "api" if cfg.get("created_via") == "api" else "chat" def execute_action(self, action_name: str, **kwargs: Any) -> str: """Dispatch on the LLM-supplied action name.""" @@ -191,7 +194,7 @@ class SchedulerTool(Tool): timezone=tz or "UTC", tool_allowlist=allowlist, origin_conversation_id=self.conversation_id, - created_via="chat", + created_via=self.created_via, ) except Exception as exc: logger.exception("schedule_task create failed: %s", exc) diff --git a/docsgpt/agents/tools/telegram.py b/docsgpt/agents/tools/telegram.py index cf8ca67a..2dcca301 100644 --- a/docsgpt/agents/tools/telegram.py +++ b/docsgpt/agents/tools/telegram.py @@ -11,12 +11,22 @@ class TelegramTool(Tool): """ Telegram Bot A flexible Telegram tool for performing various actions (e.g., sending messages, images). - Requires a bot token and chat ID for configuration + Requires a bot token; a default chat ID set on the connection is used when no chat is named """ def __init__(self, config): self.config = config self.token = config.get("token", "") + self.default_chat_id = config.get("chat_id") or None + + def _no_chat(self): + return { + "status": "error", + "error": ( + "No chat to send to. Name a chat_id, or set a default chat ID on the Telegram " + "connection in Settings > Connectors." + ), + } def execute_action(self, action_name, **kwargs): actions = { @@ -27,14 +37,20 @@ class TelegramTool(Tool): raise ValueError(f"Unknown action: {action_name}") return actions[action_name](**kwargs) - def _send_message(self, text, chat_id): + def _send_message(self, text, chat_id=None): + chat_id = chat_id or self.default_chat_id + if not chat_id: + return self._no_chat() logger.debug("Sending Telegram message to chat_id=%s", chat_id) url = f"https://api.telegram.org/bot{self.token}/sendMessage" payload = {"chat_id": chat_id, "text": text} response = requests.post(url, data=payload, timeout=100) return {"status_code": response.status_code, "message": "Message sent"} - def _send_image(self, image_url, chat_id): + def _send_image(self, image_url, chat_id=None): + chat_id = chat_id or self.default_chat_id + if not chat_id: + return self._no_chat() logger.debug("Sending Telegram image to chat_id=%s", chat_id) url = f"https://api.telegram.org/bot{self.token}/sendPhoto" payload = {"chat_id": chat_id, "photo": image_url} @@ -45,6 +61,7 @@ class TelegramTool(Tool): return [ { "name": "telegram_send_message", + "access": "write", "description": ( "Send a text message to the configured Telegram chat via " "the bot. Compose the final message text before sending." @@ -67,6 +84,7 @@ class TelegramTool(Tool): }, { "name": "telegram_send_image", + "access": "write", "description": ( "Send an image to the configured Telegram chat. Requires " "a publicly accessible image URL." diff --git a/docsgpt/agents/tools/wiki.py b/docsgpt/agents/tools/wiki.py index 372532f8..f398b6f7 100644 --- a/docsgpt/agents/tools/wiki.py +++ b/docsgpt/agents/tools/wiki.py @@ -3,6 +3,7 @@ from typing import Any, Dict, List, Optional from docsgpt.agents.tools.base import Tool from docsgpt.agents.tools.path_utils import validate_tool_path +from docsgpt.storage.db.repositories.sources import SourcesRepository from docsgpt.storage.db.repositories.wiki_pages import ( WikiPageConflict, WikiPagesRepository, @@ -21,6 +22,11 @@ MAX_WIKI_PAGE_BYTES = 1_000_000 _WRITE_ACTIONS = frozenset({"create", "str_replace", "insert", "delete", "rename"}) +OUTSIDE_EDITS_DENIED = ( + "Error: This wiki's owner doesn't let API or widget users edit it, " + "so it can't be changed from here. You can still read it." +) + class WikiTool(Tool): """Wiki @@ -39,6 +45,9 @@ class WikiTool(Tool): self.config = config self.source_id: Optional[str] = config.get("source_id") self.source_owner_id: Optional[str] = config.get("source_owner_id") + # An API-key or widget run (it acts as the agent's owner): it writes + # only while the wiki's owner allows such edits. + self.outside_caller: bool = bool(config.get("outside_caller")) decoded_token = config.get("decoded_token") or {} self.updated_by: Optional[str] = ( (decoded_token.get("sub") if decoded_token else None) @@ -228,8 +237,30 @@ class WikiTool(Tool): return message if access is None or not access.can("edit"): return message + if self.outside_caller and not self._outside_edits_allowed(): + return OUTSIDE_EDITS_DENIED return None + def _outside_edits_allowed(self) -> bool: + """Read the wiki's live ``wiki_outside_edits`` setting. + + Read on every write rather than trusted from the run's setup, so the + owner turning it off stops a conversation that is already going. + Fails closed on a missing row or a failed lookup. + + Returns: + bool: Whether an API-key or widget run may edit the wiki. + """ + try: + with db_readonly() as conn: + row = SourcesRepository(conn).get_by_id(str(self.source_id)) + except Exception: + logger.exception( + "Wiki outside-edits check failed for source %s", self.source_id + ) + return False + return outside_edits_allowed(row) + def get_config_requirements(self) -> Dict[str, Any]: return {} @@ -500,12 +531,21 @@ class WikiTool(Tool): return f"Renamed: {validated_old} -> {validated_new}" -def build_wiki_tool_entry() -> Dict[str, Any]: - """Build the synthetic tools_dict entry for the WikiTool.""" +def build_wiki_tool_entry(writes_allowed: bool = True, approval_required: bool = False) -> Dict[str, Any]: + """Build the synthetic tools_dict entry for the WikiTool. + + Args: + writes_allowed: False offers the model only ``wiki_view``. + approval_required: Every write action waits for the caller's approval. + """ entry = {"name": "wiki"} entry["actions"] = [ - {**action, "active": True} for action in _wiki_actions_metadata() + {**action, "active": True} + for action in _wiki_actions_metadata() + if writes_allowed or action["name"] == "wiki_view" ] + if approval_required: + _require_write_approval(entry) return entry @@ -513,11 +553,23 @@ def _wiki_actions_metadata() -> List[Dict[str, Any]]: return WikiTool().get_actions_metadata() +def _require_write_approval(entry: Dict[str, Any]) -> None: + for action in entry.get("actions") or []: + if action.get("name") != "wiki_view": + action["require_approval"] = True + + +def outside_edits_allowed(source_row: Optional[Dict[str, Any]]) -> bool: + """Whether a wiki's owner lets API-key and widget runs edit it.""" + return bool(source_row and source_row.get("wiki_outside_edits")) + + def build_wiki_tool_config( source_id: str, source_owner_id: str, decoded_token: Optional[Dict] = None, user: Optional[str] = None, + outside_caller: bool = False, ) -> Dict[str, Any]: """Build the config dict passed to the injected WikiTool.""" return { @@ -525,6 +577,7 @@ def build_wiki_tool_config( "source_owner_id": source_owner_id, "decoded_token": decoded_token, "user": user, + "outside_caller": bool(outside_caller), } @@ -534,15 +587,66 @@ def add_wiki_tool(tools_dict: Dict, config: Dict) -> None: Mirrors ``add_internal_search_tool``: the entry carries ``id=WIKI_TOOL_ID`` so the executor can resolve the synthetic (DB-rowless) tool, and a ``config`` the executor copies into the loaded tool. Mutates ``tools_dict`` in place. + ``writes_allowed=False`` (an API-key or widget run on a wiki whose owner + hasn't allowed their edits) offers only ``wiki_view``; the tool still + refuses writes itself from ``outside_caller``. ``approval_required`` (a + public-link run) puts every write behind the caller's approval. """ if not config or not config.get("source_id") or not config.get("source_owner_id"): return - entry = build_wiki_tool_entry() + entry = build_wiki_tool_entry( + writes_allowed=config.get("writes_allowed", True) is not False, + approval_required=bool(config.get("approval_required")), + ) entry["id"] = WIKI_TOOL_ID entry["config"] = build_wiki_tool_config( source_id=config["source_id"], source_owner_id=config["source_owner_id"], decoded_token=config.get("decoded_token"), user=config.get("user"), + outside_caller=bool(config.get("outside_caller")), ) tools_dict[WIKI_TOOL_ID] = entry + + +def apply_resume_caller_rules( + tools_dict: Dict, *, outside_caller: bool, public_link_caller: bool +) -> None: + """Hold a resumed run's saved WikiTool entry to its caller's rules. + + A paused turn is resumed by whoever sends its tool actions, which may not + be who started it: a widget key can resume the owner's own chat. So the + saved entry is tightened here rather than trusted. An API-key or widget + run gets ``outside_caller`` and, unless the wiki's owner allows their + edits (read live), only ``wiki_view``; a public-link run approves every + write. Never loosens an entry. Mutates ``tools_dict`` in place. + + Args: + tools_dict: The resumed run's tools. + outside_caller: The saved state or the resuming request is an + API-key or widget caller. + public_link_caller: The saved state or the resuming request reached + the agent through its public link. + """ + entry = tools_dict.get(WIKI_TOOL_ID) if isinstance(tools_dict, dict) else None + if not isinstance(entry, dict): + return + if public_link_caller: + _require_write_approval(entry) + if not outside_caller: + return + config = entry.get("config") + if not isinstance(config, dict): + config = {} + entry["config"] = config + config["outside_caller"] = True + allowed = False + source_id = config.get("source_id") + if source_id: + try: + with db_readonly() as conn: + allowed = outside_edits_allowed(SourcesRepository(conn).get_by_id(str(source_id))) + except Exception: + logger.exception("Wiki outside-edits check failed for source %s", source_id) + if not allowed: + entry["actions"] = [a for a in entry.get("actions") or [] if a.get("name") == "wiki_view"] diff --git a/docsgpt/agents/workflows/node_agent.py b/docsgpt/agents/workflows/node_agent.py index 45215317..7681e430 100644 --- a/docsgpt/agents/workflows/node_agent.py +++ b/docsgpt/agents/workflows/node_agent.py @@ -20,6 +20,8 @@ class _WorkflowNodeMixin: api_key: str, tool_ids: Optional[List[str]] = None, tool_principals: Optional[Dict[str, str]] = None, + tool_owner: Optional[str] = None, + tool_holder: Optional[dict] = None, **kwargs, ): super().__init__( @@ -35,8 +37,12 @@ class _WorkflowNodeMixin: # (Artifact / Code Executor / Read Document) and ``user_tools`` rows # alike, and an empty list means the node's LLM gets no tools. self.tool_executor.allowed_tool_ids = [str(t) for t in (tool_ids or [])] - # Tools the owner can't use resolve as the editor who attached them. + # The node's tools are the workflow owner's whoever runs it; tools the + # owner can't use resolve as the editor who attached them. + self.tool_executor.tool_owner = tool_owner self.tool_executor.tool_principals = dict(tool_principals or {}) + # The workflow row, so a dropped tool is logged with why. + self.tool_executor.tool_holder = tool_holder class WorkflowNodeClassicAgent(_WorkflowNodeMixin, ClassicAgent): diff --git a/docsgpt/agents/workflows/workflow_engine.py b/docsgpt/agents/workflows/workflow_engine.py index 22017726..a700991d 100644 --- a/docsgpt/agents/workflows/workflow_engine.py +++ b/docsgpt/agents/workflows/workflow_engine.py @@ -405,7 +405,9 @@ class WorkflowEngine: "model_user_id": getattr(self.agent, "model_user_id", None), "api_key": node_api_key, "tool_ids": node_config.tools, + "tool_owner": self._workflow_owner_id(), "tool_principals": self._node_tool_principals(node_config.tools), + "tool_holder": getattr(self.agent, "workflow_row", None), "prompt": node_prompt, "chat_history": self.agent.chat_history, "decoded_token": self.agent.decoded_token, @@ -445,9 +447,9 @@ class WorkflowEngine: # the tool_call_attempts primary key and drop the later journal rows. node_executor = getattr(node_agent, "tool_executor", None) if node_executor is not None: - node_executor.message_id = getattr( - getattr(self.agent, "tool_executor", None), "message_id", None - ) + run_executor = getattr(self.agent, "tool_executor", None) + node_executor.message_id = getattr(run_executor, "message_id", None) + self._inherit_caller_policy(node_executor, run_executor) # Run-scope the node agent's tools so artifact_generator / code_executor # address artifacts by this workflow run: a short ref (A1) created by one # node resolves for edit_artifact in a later node within the same run. Only @@ -1322,6 +1324,34 @@ class WorkflowEngine: docs_together = "\n\n".join(docs_together_parts) if docs_together_parts else None return docs, docs_together + @staticmethod + def _inherit_caller_policy(node_executor: Any, run_executor: Any) -> None: + """Give a node's executor the caller rules of the run that started it. + + A node's tools run for the same caller as the workflow agent: a + scheduled run still can't pause, and an API-key or public-link caller + still can't write on the owner's account unless it is allowlisted. + + Args: + node_executor: The node agent's ``ToolExecutor``. + run_executor: The workflow agent's ``ToolExecutor``, if any. + """ + if run_executor is None: + return + for attr in ("headless", "external_caller", "public_link_caller"): + setattr(node_executor, attr, bool(getattr(run_executor, attr, False))) + for attr in ("tool_allowlist", "api_write_allowlist"): + setattr(node_executor, attr, set(getattr(run_executor, attr, None) or ())) + + def _workflow_owner_id(self) -> Optional[str]: + """The workflow's owner, whom node tools and sources run as. + + Returns: + The owner's user id, or None when the run has none. + """ + resolve_owner = getattr(self.agent, "_resolve_owner_id", None) + return (resolve_owner() if callable(resolve_owner) else None) or self._resolve_user_id() + def _node_tool_principals(self, tool_ids) -> Dict[str, str]: """Node tool id -> the editor to resolve it as, for sponsored tools. @@ -1373,33 +1403,26 @@ class WorkflowEngine: if not sources: return [] ids = sources if isinstance(sources, list) else [sources] - resolve_owner = getattr(self.agent, "_resolve_owner_id", None) - owner = (resolve_owner() if callable(resolve_owner) else None) or ( - self._resolve_user_id() - ) + owner = self._workflow_owner_id() if not owner: logger.warning("Workflow node sources dropped: no owner to authorize.") return [] - from docsgpt.api.user.resource_access import active_sponsor - from docsgpt.api.user.team_sharing import can_access + from docsgpt.api.user.resource_access import log_stopped, ref_access from docsgpt.storage.db.session import db_readonly - workflow_row = getattr(self.agent, "workflow_row", None) + # The same check the workflow page's run state uses: the owner, else + # the editor who attached it while they still qualify. + holder = {**(getattr(self.agent, "workflow_row", None) or {}), "user_id": owner} allowed = [] try: with db_readonly() as conn: for sid in ids: - if sid and ( - can_access(conn, "source", str(sid), owner) - or active_sponsor(conn, "workflow", workflow_row, "source", str(sid)) - ): + access = ref_access(conn, "workflow", holder, "source", str(sid)) if sid else None + if access is not None and access.principal: allowed.append(sid) else: - logger.warning( - "Workflow node source %s dropped: %s has no access.", - sid, owner, - ) + log_stopped("workflow", holder, "source", sid, access.reason if access else None) except Exception: logger.exception("Workflow node source authorization failed; dropping all.") return [] diff --git a/docsgpt/alembic/versions/0040_connections.py b/docsgpt/alembic/versions/0040_connections.py new file mode 100644 index 00000000..33a18cc6 --- /dev/null +++ b/docsgpt/alembic/versions/0040_connections.py @@ -0,0 +1,557 @@ +"""0040 connections — connector_sessions becomes the connections table. + +``connector_sessions`` already holds one row per signed-in account (OAuth +ingest providers) or per MCP server. This migration names what each row is +and lets sources and tools point at the row they use: + +* ``connector_key`` is the catalog entry (``google_drive``, ``custom_mcp``, + ``telegram``), ``auth_kind`` how the row signs in, ``display_name`` and + ``account_label`` what the Connectors page shows. +* ``sources.connection_id`` and ``user_tools.connection_id`` link the + resources a connection feeds. ``ON DELETE SET NULL`` keeps a source's + indexed content when its connection is removed. + +Connections own their credentials. Every secret of a connection (OAuth +tokens, MCP OAuth tokens and the dynamic client registration, API keys) +moves into ``encrypted_credentials``, a v2 envelope from +``docsgpt.security.encryption`` bound to the owner. Plain columns +(``status``, ``has_refresh_token``, ``scopes``, ``expires_at``) keep status +checks from ever decrypting. The unique index gains ``account_label`` so one +user can connect several accounts of the same service, and +``credential_mode`` says whose account a shared tool or source uses. + +Backfill (idempotent, only fills NULLs or unconverted rows): + +1. ``connector_key``, ``auth_kind``, ``display_name`` from ``provider``. +2. ``account_label`` from ``user_email`` for OAuth rows. +3. ``sources.connection_id`` for ``connector:file`` sources, matched to the + owner's only row for ``remote_data->>'provider'``. +4. ``user_tools.connection_id`` for OAuth MCP tools, matched to the owner's + row for the tool's server base URL. +5. API-key tools (Brave, Telegram, ntfy, PostgreSQL, custom MCP with a key, + bearer token or basic auth) get one connection per distinct credential, + re-encrypted into v2. The tool keeps its v1 copy for one release so a + rollback still works; the executor prefers the connection. Downgrade + writes a v1 copy back to every tool on an API-key connection, including + tools added after the upgrade. +6. ``token_info`` and the secret parts of ``session_data`` (``tokens``, + ``client_info``) are encrypted into ``encrypted_credentials`` and removed + from the plaintext columns. This needs ``ENCRYPTION_SECRET_KEY`` set to + the value the app will run with. Downgrade decrypts them back. +7. ``credential_mode`` is ``owner`` everywhere except OAuth MCP tools, which + resolved each invoking member's own token before this migration and keep + doing so (``member``); owners can switch them in the share dialog. + +``connector_policies`` holds the admin's per-connector switches (enabled, +forced credential mode). ``enabled`` NULL means the default: on when the +connector has the server settings it needs, off (and hidden from members) +until then. The instance-wide "Allow custom MCP servers" switch +is the ``connectors.allow_custom_mcp`` key in ``app_metadata`` (absent means +allowed). + +Revision ID: 0040_connections +Revises: 0039_resource_sponsors +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0040_connections" +down_revision: Union[str, None] = "0039_resource_sponsors" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +_OAUTH_PROVIDERS = { + "google_drive": "Google Drive", + "share_point": "SharePoint", + "confluence": "Confluence", +} + + +_TOOL_CONNECTORS = { + "brave": "Brave Search", + "telegram": "Telegram", + "ntfy": "ntfy", + "postgres": "PostgreSQL", +} +_MCP_SECRET_AUTH = ("api_key", "bearer", "basic") +_BATCH = 500 + + +def upgrade() -> None: + _upgrade_links() + _upgrade_credentials() + op.execute( + """ + CREATE TABLE IF NOT EXISTS connector_policies ( + connector_key TEXT PRIMARY KEY, + enabled BOOLEAN, + credential_mode TEXT NOT NULL DEFAULT 'choose' + CONSTRAINT connector_policies_credential_mode_chk + CHECK (credential_mode IN ('choose', 'owner', 'member')), + updated_by TEXT, + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() + ); + """ + ) + + +def _upgrade_links() -> None: + op.execute( + """ + ALTER TABLE connector_sessions + ADD COLUMN IF NOT EXISTS connector_key TEXT, + ADD COLUMN IF NOT EXISTS display_name TEXT, + ADD COLUMN IF NOT EXISTS account_label TEXT, + ADD COLUMN IF NOT EXISTS auth_kind TEXT, + ADD COLUMN IF NOT EXISTS updated_at TIMESTAMPTZ NOT NULL DEFAULT now(); + """ + ) + op.execute( + "ALTER TABLE sources ADD COLUMN IF NOT EXISTS connection_id UUID " + "REFERENCES connector_sessions(id) ON DELETE SET NULL;" + ) + op.execute( + "ALTER TABLE user_tools ADD COLUMN IF NOT EXISTS connection_id UUID " + "REFERENCES connector_sessions(id) ON DELETE SET NULL;" + ) + op.execute("CREATE INDEX IF NOT EXISTS sources_connection_idx ON sources (connection_id);") + op.execute("CREATE INDEX IF NOT EXISTS user_tools_connection_idx ON user_tools (connection_id);") + + # 1 + 2: name the existing rows. + for provider, name in _OAUTH_PROVIDERS.items(): + op.execute( + f""" + UPDATE connector_sessions SET + connector_key = COALESCE(connector_key, '{provider}'), + auth_kind = COALESCE(auth_kind, 'oauth'), + display_name = COALESCE(display_name, '{name}'), + account_label = COALESCE(account_label, user_email) + WHERE provider = '{provider}'; + """ + ) + op.execute( + """ + UPDATE connector_sessions SET + connector_key = COALESCE(connector_key, 'custom_mcp'), + auth_kind = COALESCE(auth_kind, 'mcp_oauth'), + display_name = COALESCE( + display_name, + regexp_replace(COALESCE(server_url, substr(provider, 5)), '^https?://', '') + ) + WHERE provider LIKE 'mcp:%'; + """ + ) + + # 3: connector sources point at the owner's only session for the provider. + op.execute( + """ + UPDATE sources s SET connection_id = cs.id + FROM connector_sessions cs + WHERE s.connection_id IS NULL + AND s.type = 'connector:file' + AND cs.user_id = s.user_id + AND cs.provider = s.remote_data->>'provider' + AND ( + SELECT count(*) FROM connector_sessions c2 + WHERE c2.user_id = s.user_id AND c2.provider = cs.provider + ) = 1; + """ + ) + + # 4: OAuth MCP tools point at the owner's session for the server's base URL. + op.execute( + """ + UPDATE user_tools t SET connection_id = cs.id + FROM connector_sessions cs + WHERE t.connection_id IS NULL + AND t.name = 'mcp_tool' + AND t.config->>'auth_type' = 'oauth' + AND cs.user_id = t.user_id + AND cs.provider = 'mcp:' || substring(t.config->>'server_url' from '^(https?://[^/]+)'); + """ + ) + + +def _credential_hint(credentials: dict) -> str: + """``…abcd``: the last four characters of the first secret, never more.""" + for value in credentials.values(): + if isinstance(value, str) and len(value) >= 8: + return "\u2026" + value[-4:] + return "\u2026" + + +def _upgrade_credentials() -> None: + op.execute( + """ + ALTER TABLE connector_sessions + ADD COLUMN IF NOT EXISTS encrypted_credentials TEXT, + ADD COLUMN IF NOT EXISTS has_refresh_token BOOLEAN NOT NULL DEFAULT false, + ADD COLUMN IF NOT EXISTS scopes JSONB NOT NULL DEFAULT '[]'::jsonb, + ADD COLUMN IF NOT EXISTS last_error TEXT, + ADD COLUMN IF NOT EXISTS last_used_at TIMESTAMPTZ; + """ + ) + for table in ("sources", "user_tools"): + op.execute( + f"ALTER TABLE {table} ADD COLUMN IF NOT EXISTS credential_mode TEXT NOT NULL DEFAULT 'owner' " + f"CONSTRAINT {table}_credential_mode_chk CHECK (credential_mode IN ('owner', 'member'));" + ) + op.execute("DROP INDEX IF EXISTS connector_sessions_user_endpoint_uidx;") + op.execute( + "CREATE UNIQUE INDEX IF NOT EXISTS connector_sessions_account_uidx ON connector_sessions " + "(user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, ''));" + ) + + # Statuses become connected / pending / reconnect_needed / disconnected / + # error. MCP rows never had one; name them before their tokens are sealed. + op.execute("UPDATE connector_sessions SET status = 'connected' WHERE status = 'authorized';") + op.execute( + """ + UPDATE connector_sessions SET status = + CASE WHEN session_data ? 'tokens' THEN 'connected' ELSE 'pending' END + WHERE provider LIKE 'mcp:%' AND status IS NULL; + """ + ) + + bind = op.get_bind() + _encrypt_session_secrets(bind) + _link_api_key_tools(bind) + + # 7: keep OAuth MCP tools on each member's own token, as before. + op.execute( + """ + UPDATE user_tools SET credential_mode = 'member' + WHERE name = 'mcp_tool' AND config->>'auth_type' = 'oauth'; + """ + ) + + +def _encrypt_session_secrets(bind) -> None: + """6: move plaintext tokens into the owner-bound v2 envelope.""" + import json + + from sqlalchemy import text + + from docsgpt.security.encryption import encrypt_json + + while True: + rows = bind.execute( + text( + """ + SELECT id, user_id, token_info, token_info IS NOT NULL AS has_token_info, session_data + FROM connector_sessions + WHERE encrypted_credentials IS NULL + AND (token_info IS NOT NULL OR session_data ? 'tokens' OR session_data ? 'client_info') + LIMIT :batch + """ + ), + {"batch": _BATCH}, + ).fetchall() + if not rows: + return + for row in rows: + token_info = _token_info_object(row.token_info) + session_data = dict(row.session_data or {}) + secrets = {} + if token_info is not None: + secrets["token_info"] = token_info + elif row.has_token_info: + # A value the app never read as a token (a bare string, a number, + # JSON ``null``): kept apart from ``token_info`` so downgrade can + # restore it, and the row is not selected again. + secrets["legacy_token_info"] = row.token_info + for key in ("tokens", "client_info"): + if key in session_data: + secrets[key] = session_data.pop(key) + tokens = secrets.get("tokens") if isinstance(secrets.get("tokens"), dict) else {} + has_refresh = bool((token_info or {}).get("refresh_token") or tokens.get("refresh_token")) + scopes = (token_info or {}).get("scopes") or [] + if isinstance(scopes, str): + scopes = scopes.split() + bind.execute( + text( + """ + UPDATE connector_sessions SET + encrypted_credentials = :blob, + has_refresh_token = :has_refresh, + scopes = CAST(:scopes AS jsonb), + token_info = NULL, + session_data = CAST(:session_data AS jsonb) + WHERE id = :id + """ + ), + { + "blob": encrypt_json(secrets, row.user_id), + "has_refresh": has_refresh, + "scopes": json.dumps(list(scopes)), + "session_data": json.dumps(session_data), + "id": row.id, + }, + ) + + +def _token_info_object(value): + """``token_info`` as an object, decoding one stored as a JSON string. + + Args: + value: The decoded ``token_info`` column value. + + Returns: + The object, or ``None`` when the value is not (and does not encode) one. + """ + import json + + if isinstance(value, str): + try: + value = json.loads(value) + except ValueError: + return None + return value if isinstance(value, dict) else None + + +def _link_api_key_tools(bind) -> None: + """5: one connection per distinct API credential, linked from its tools.""" + from urllib.parse import urlparse + + from sqlalchemy import text + + from docsgpt.security.encryption import ( + CredentialDecryptionError, + decrypt_credentials, + decrypt_json, + encrypt_json, + ) + + rows = bind.execute( + text( + """ + SELECT id, user_id, name, custom_name, display_name, config FROM user_tools + WHERE connection_id IS NULL + AND config ? 'encrypted_credentials' + AND ( + name = ANY(:names) + OR (name = 'mcp_tool' AND config->>'auth_type' = ANY(:mcp_auth)) + ) + """ + ), + {"names": list(_TOOL_CONNECTORS), "mcp_auth": list(_MCP_SECRET_AUTH)}, + ).fetchall() + for row in rows: + config = row.config or {} + credentials = decrypt_credentials(config.get("encrypted_credentials") or "", row.user_id) + if not credentials: + # Written with a different key; the tool keeps its v1 copy. + continue + if row.name == "mcp_tool": + parsed = urlparse(config.get("server_url") or "") + server_url = f"{parsed.scheme}://{parsed.netloc}" if parsed.netloc else None + connector_key = "custom_mcp" + display_name = row.custom_name or row.display_name or parsed.netloc or "MCP server" + else: + server_url = None + connector_key = row.name + display_name = _TOOL_CONNECTORS[row.name] + # The hint is not an identity: reuse a connection only when it holds + # the same credentials, and give a different key its own label. + hint = _credential_hint(credentials) + label, suffix = hint, 1 + while True: + existing = bind.execute( + text( + """ + SELECT id, encrypted_credentials FROM connector_sessions + WHERE user_id = :user_id AND provider = :provider + AND COALESCE(server_url, '') = COALESCE(:server_url, '') + AND COALESCE(account_label, '') = :label + """ + ), + {"user_id": row.user_id, "provider": connector_key, "server_url": server_url, "label": label}, + ).fetchone() + if existing is None: + break + try: + stored = decrypt_json(existing.encrypted_credentials or "", row.user_id).get("credentials") + except CredentialDecryptionError: + stored = None + if stored == credentials: + break + suffix += 1 + label = f"{hint} ({suffix})" + if existing is None: + connection_id = bind.execute( + text( + """ + INSERT INTO connector_sessions ( + user_id, provider, server_url, connector_key, display_name, account_label, + auth_kind, status, encrypted_credentials, session_data + ) VALUES ( + :user_id, :provider, :server_url, :provider, :display_name, :label, + 'api_key', 'connected', :blob, '{}'::jsonb + ) RETURNING id + """ + ), + { + "user_id": row.user_id, + "provider": connector_key, + "server_url": server_url, + "display_name": display_name, + "label": label, + "blob": encrypt_json({"credentials": credentials}, row.user_id), + }, + ).scalar() + else: + connection_id = existing.id + bind.execute( + text("UPDATE user_tools SET connection_id = :cid WHERE id = :id"), + {"cid": connection_id, "id": row.id}, + ) + + +def _downgrade_credentials() -> None: + """Decrypt the envelopes back into the pre-0038 plaintext columns.""" + + from sqlalchemy import text + + + bind = op.get_bind() + has_envelope = bind.execute( + text( + "SELECT 1 FROM information_schema.columns " + "WHERE table_name = 'connector_sessions' AND column_name = 'encrypted_credentials'" + ) + ).first() + if has_envelope is not None: + _decrypt_back(bind) + _restore_account_index() + + +def _decrypt_back(bind) -> None: + import json + + from sqlalchemy import text + + from docsgpt.security.encryption import CredentialDecryptionError, decrypt_json, encrypt_credentials + + # API-key connections go away, so their tools get a v1 copy back: tools + # added after the upgrade never had one, and a reconnect may have changed + # the key since the backfill. + linked = bind.execute( + text( + "SELECT t.id, t.user_id, c.encrypted_credentials FROM user_tools t " + "JOIN connector_sessions c ON c.id = t.connection_id " + "WHERE c.auth_kind = 'api_key' AND c.encrypted_credentials IS NOT NULL" + ) + ).fetchall() + for row in linked: + try: + credentials = decrypt_json(row.encrypted_credentials, row.user_id).get("credentials") + except CredentialDecryptionError: + continue + if not credentials: + continue + bind.execute( + text( + "UPDATE user_tools SET config = COALESCE(config, '{}'::jsonb) " + "|| jsonb_build_object('encrypted_credentials', CAST(:blob AS text)) WHERE id = :id" + ), + {"blob": encrypt_credentials(credentials, row.user_id), "id": row.id}, + ) + bind.execute(text("UPDATE user_tools SET connection_id = NULL WHERE connection_id IN " + "(SELECT id FROM connector_sessions WHERE auth_kind = 'api_key')")) + bind.execute(text("DELETE FROM connector_sessions WHERE auth_kind = 'api_key'")) + rows = bind.execute( + text( + "SELECT id, user_id, session_data, encrypted_credentials FROM connector_sessions " + "WHERE encrypted_credentials IS NOT NULL" + ) + ).fetchall() + for row in rows: + try: + secrets = decrypt_json(row.encrypted_credentials, row.user_id) + except CredentialDecryptionError: + continue + session_data = dict(row.session_data or {}) + for key in ("tokens", "client_info"): + if key in secrets: + session_data[key] = secrets[key] + bind.execute( + text( + "UPDATE connector_sessions SET token_info = CAST(:token_info AS jsonb), " + "session_data = CAST(:session_data AS jsonb) WHERE id = :id" + ), + { + "token_info": _restored_token_info(secrets), + "session_data": json.dumps(session_data), + "id": row.id, + }, + ) + + +def _restored_token_info(secrets: dict): + """The ``token_info`` JSON downgrade writes back, ``None`` for SQL NULL. + + Args: + secrets: The decrypted envelope. + + Returns: + The JSON text of ``token_info`` or of a preserved non-object value. + """ + import json + + for key in ("token_info", "legacy_token_info"): + if key in secrets: + return json.dumps(secrets[key]) + return None + + +def _restore_account_index() -> None: + # Several accounts per provider cannot survive the old unique index: keep + # the most recently updated one. + op.execute( + """ + DELETE FROM connector_sessions c USING connector_sessions newer + WHERE c.user_id = newer.user_id AND c.provider = newer.provider + AND COALESCE(c.server_url, '') = COALESCE(newer.server_url, '') + AND (c.updated_at, c.id) < (newer.updated_at, newer.id); + """ + ) + op.execute("DROP INDEX IF EXISTS connector_sessions_account_uidx;") + op.execute( + "CREATE UNIQUE INDEX IF NOT EXISTS connector_sessions_user_endpoint_uidx " + "ON connector_sessions (user_id, COALESCE(server_url, ''), provider);" + ) + for table in ("sources", "user_tools"): + op.execute(f"ALTER TABLE {table} DROP COLUMN IF EXISTS credential_mode;") + op.execute( + """ + ALTER TABLE connector_sessions + DROP COLUMN IF EXISTS last_used_at, + DROP COLUMN IF EXISTS last_error, + DROP COLUMN IF EXISTS scopes, + DROP COLUMN IF EXISTS has_refresh_token, + DROP COLUMN IF EXISTS encrypted_credentials; + """ + ) + + +def downgrade() -> None: + op.execute("DROP TABLE IF EXISTS connector_policies;") + _downgrade_credentials() + op.execute("DROP INDEX IF EXISTS user_tools_connection_idx;") + op.execute("DROP INDEX IF EXISTS sources_connection_idx;") + op.execute("ALTER TABLE user_tools DROP COLUMN IF EXISTS connection_id;") + op.execute("ALTER TABLE sources DROP COLUMN IF EXISTS connection_id;") + op.execute( + """ + ALTER TABLE connector_sessions + DROP COLUMN IF EXISTS updated_at, + DROP COLUMN IF EXISTS auth_kind, + DROP COLUMN IF EXISTS account_label, + DROP COLUMN IF EXISTS display_name, + DROP COLUMN IF EXISTS connector_key; + """ + ) diff --git a/docsgpt/alembic/versions/0041_connection_account_name.py b/docsgpt/alembic/versions/0041_connection_account_name.py new file mode 100644 index 00000000..fffc34fd --- /dev/null +++ b/docsgpt/alembic/versions/0041_connection_account_name.py @@ -0,0 +1,32 @@ +"""0041 connection account name — what the user calls an account. + +``account_label`` identifies an account: the email an OAuth sign-in returns, +or a hint of a pasted key. Signing in again finds the connection by it, so it +cannot be renamed. ``account_name`` is the name the user gives the account +("Alerts bot", "Work"), shown instead of the label and used to tell two +accounts of one service apart, for people and for the model. NULL means the +user never named it. + +Idempotent both ways. + +Revision ID: 0041_connection_account_name +Revises: 0040_connections +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0041_connection_account_name" +down_revision: Union[str, None] = "0040_connections" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.execute("ALTER TABLE connector_sessions ADD COLUMN IF NOT EXISTS account_name TEXT;") + + +def downgrade() -> None: + op.execute("ALTER TABLE connector_sessions DROP COLUMN IF EXISTS account_name;") diff --git a/docsgpt/alembic/versions/0042_schedule_created_via_api.py b/docsgpt/alembic/versions/0042_schedule_created_via_api.py new file mode 100644 index 00000000..975c0f08 --- /dev/null +++ b/docsgpt/alembic/versions/0042_schedule_created_via_api.py @@ -0,0 +1,40 @@ +"""0042 schedule created_via api — schedules set by an API-key caller. + +A schedule the agent sets from a widget or API chat is stored under the +agent's owner, like the run itself. ``created_via = 'api'`` records that +it came from outside the app, so its runs keep that caller's limits: no +writes on the owner's accounts or credentials unless the owner allowed +them in the agent's API write allowlist. + +Idempotent both ways. Downgrade folds ``api`` back into ``chat``. + +Revision ID: 0042_schedule_created_via_api +Revises: 0041_connection_account_name +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0042_schedule_created_via_api" +down_revision: Union[str, None] = "0041_connection_account_name" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.execute("ALTER TABLE schedules DROP CONSTRAINT IF EXISTS schedules_created_via_chk;") + op.execute( + "ALTER TABLE schedules ADD CONSTRAINT schedules_created_via_chk " + "CHECK (created_via IN ('chat', 'ui', 'api'));" + ) + + +def downgrade() -> None: + op.execute("ALTER TABLE schedules DROP CONSTRAINT IF EXISTS schedules_created_via_chk;") + op.execute("UPDATE schedules SET created_via = 'chat' WHERE created_via = 'api';") + op.execute( + "ALTER TABLE schedules ADD CONSTRAINT schedules_created_via_chk " + "CHECK (created_via IN ('chat', 'ui'));" + ) diff --git a/docsgpt/alembic/versions/0043_wiki_outside_edits.py b/docsgpt/alembic/versions/0043_wiki_outside_edits.py new file mode 100644 index 00000000..6968ab14 --- /dev/null +++ b/docsgpt/alembic/versions/0043_wiki_outside_edits.py @@ -0,0 +1,32 @@ +"""0043 wiki outside edits — the wiki owner's say on API and widget edits. + +An agent run from its API key or widget acts as the agent's owner, so it +could rewrite any wiki the owner can edit. ``wiki_outside_edits`` records +whether the wiki's owner allows that; it is off by default, so such runs can +still read the wiki but not change it. + +Idempotent both ways. + +Revision ID: 0043_wiki_outside_edits +Revises: 0042_schedule_created_via_api +""" + +from typing import Sequence, Union + +from alembic import op + + +revision: str = "0043_wiki_outside_edits" +down_revision: Union[str, None] = "0042_schedule_created_via_api" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + op.execute( + "ALTER TABLE sources ADD COLUMN IF NOT EXISTS wiki_outside_edits BOOLEAN NOT NULL DEFAULT false;" + ) + + +def downgrade() -> None: + op.execute("ALTER TABLE sources DROP COLUMN IF EXISTS wiki_outside_edits;") diff --git a/docsgpt/api/admin/__init__.py b/docsgpt/api/admin/__init__.py index 1dcdd686..bf47f250 100644 --- a/docsgpt/api/admin/__init__.py +++ b/docsgpt/api/admin/__init__.py @@ -1,5 +1,6 @@ from .routes import admin_ns from . import quotas # noqa: F401 (registers the quota resources on admin_ns) from . import activity # noqa: F401 (registers the activity resources on admin_ns) +from . import connectors # noqa: F401 (registers the connector policy resources on admin_ns) __all__ = ["admin_ns"] diff --git a/docsgpt/api/admin/connectors.py b/docsgpt/api/admin/connectors.py new file mode 100644 index 00000000..fe360cfc --- /dev/null +++ b/docsgpt/api/admin/connectors.py @@ -0,0 +1,159 @@ +"""Admin > Connectors: turn connectors on or off and set sharing policy. + +Every resource here is behind ``@admin_required`` (the frontend guard is +cosmetic). The page also shows what each OAuth connector still needs from +the server (settings and the redirect URI to register), never their values. +""" + +from __future__ import annotations + +import logging + +from flask import jsonify, make_response, request +from flask_restx import Resource +from sqlalchemy import text + +from docsgpt.api.admin.routes import _actor, admin_ns +from docsgpt.api.user.authz import admin_required +from docsgpt.connectors import catalog, service +from docsgpt.core.settings import settings +from docsgpt.security.encryption import is_default_encryption_key +from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository +from docsgpt.storage.db.repositories.connector_policies import ( + ALLOW_CUSTOM_MCP_KEY, + CREDENTIAL_POLICIES, + ConnectorPoliciesRepository, + allow_writes_key, +) +from docsgpt.storage.db.session import db_readonly, db_session + +logger = logging.getLogger(__name__) + + +def _connection_counts(conn) -> dict[str, int]: + rows = conn.execute( + text( + "SELECT connector_key, provider, server_url, count(*) AS n FROM connector_sessions " + "WHERE COALESCE(status, '') <> 'pending' GROUP BY connector_key, provider, server_url" + ) + ).fetchall() + counts: dict[str, int] = {} + for row in rows: + key = catalog.connector_key_for_row(dict(row._mapping)) + if key: + counts[key] = counts.get(key, 0) + int(row.n) + return counts + + +def _mcp_redirect_uri() -> str: + from docsgpt.agents.tools.mcp_tool import MCPTool + + return MCPTool._resolve_redirect_uri(MCPTool.__new__(MCPTool), None) + + +@admin_ns.route("/admin/connectors") +class AdminConnectorsResource(Resource): + @admin_required + def get(self): + """Every connector with its policy, setup state and connection count.""" + with db_readonly() as conn: + policies = ConnectorPoliciesRepository(conn).all() + loaded = service.load_policies(conn) + allow_custom = service.custom_mcp_allowed(conn) + counts = _connection_counts(conn) + connectors = [] + for definition in catalog.all_definitions(): + policy = policies.get(definition.key) or {} + connectors.append( + { + "key": definition.key, + "name": definition.name, + "icon": definition.icon, + "category": definition.category, + "publisher": definition.publisher, + "auth_kind": definition.auth_kind, + "capabilities": list(definition.capabilities), + "enabled": service.connector_is_enabled(policies, definition.key), + "credential_mode": policy.get("credential_mode", "choose"), + "configured": definition.configured, + "required_settings": [ + {"name": name, "set": bool(getattr(settings, name, None))} + for name in definition.required_settings + ], + # Optional: they add a second sign-in (GitHub's App) to a + # connector that already works without them. + "oauth_settings": [ + {"name": name, "set": bool(getattr(settings, name, None))} + for name in definition.oauth_settings + ], + "oauth_configured": definition.oauth_configured, + "connection_count": counts.get(definition.key, 0), + "docs_url": definition.docs_url, + "mcp_url": definition.mcp_url, + # Whether members may let agents make changes (GitHub); + # None where the connector offers no such choice. + "allow_writes": ( + service.writes_allowed(loaded, definition.key) if definition.mcp_write_url else None + ), + } + ) + return make_response( + jsonify( + { + "success": True, + "connectors": connectors, + "allow_custom_mcp": allow_custom, + "default_encryption_key": is_default_encryption_key(), + "oauth_redirect_uri": settings.CONNECTOR_REDIRECT_BASE_URI, + "mcp_redirect_uri": _mcp_redirect_uri(), + } + ), + 200, + ) + + @admin_required + def put(self): + """Update policies: {policies: {key: {enabled?, credential_mode?, allow_writes?}}, allow_custom_mcp?}.""" + body = request.get_json(silent=True) or {} + updates = body.get("policies") or {} + if not isinstance(updates, dict): + return make_response(jsonify({"success": False, "message": "policies must be an object"}), 400) + for key, change in updates.items(): + if catalog.get_definition(key) is None or not isinstance(change, dict): + return make_response(jsonify({"success": False, "message": f"Unknown connector: {key}"}), 400) + mode = change.get("credential_mode") + if mode is not None and mode not in CREDENTIAL_POLICIES: + return make_response(jsonify({"success": False, "message": "Unknown credential mode"}), 400) + if "enabled" in change and not isinstance(change["enabled"], bool): + return make_response(jsonify({"success": False, "message": "enabled must be true or false"}), 400) + if "allow_writes" in change: + if not catalog.get_definition(key).mcp_write_url: + return make_response( + jsonify({"success": False, "message": f"{key} has no write access to allow"}), 400, + ) + if not isinstance(change["allow_writes"], bool): + return make_response( + jsonify({"success": False, "message": "allow_writes must be true or false"}), 400, + ) + allow_custom = body.get("allow_custom_mcp") + if allow_custom is not None and not isinstance(allow_custom, bool): + return make_response(jsonify({"success": False, "message": "allow_custom_mcp must be a boolean"}), 400) + actor = _actor() + with db_session() as conn: + repo = ConnectorPoliciesRepository(conn) + for key, change in updates.items(): + if "allow_writes" in change: + value = "true" if change["allow_writes"] else "false" + AppMetadataRepository(conn).set(allow_writes_key(key), value) + if set(change) == {"allow_writes"}: + continue + repo.upsert( + key, + enabled=change.get("enabled"), + credential_mode=change.get("credential_mode"), + updated_by=actor, + ) + if allow_custom is not None: + AppMetadataRepository(conn).set(ALLOW_CUSTOM_MCP_KEY, "true" if allow_custom else "false") + logger.info("connector_policies_updated", extra={"admin": actor, "connectors": sorted(updates)}) + return self.get() diff --git a/docsgpt/api/answer/routes/base.py b/docsgpt/api/answer/routes/base.py index ca92d3cf..9eb57c1d 100644 --- a/docsgpt/api/answer/routes/base.py +++ b/docsgpt/api/answer/routes/base.py @@ -1057,6 +1057,17 @@ class BaseAnswerResource: "llm_name": getattr(agent, "llm_name", settings.LLM_PROVIDER), "api_key": getattr(agent, "api_key", None), "user_api_key": user_api_key, + # An API-key caller stays one after a + # resume (owner-account write rules). + "external_api_caller": getattr( + agent.tool_executor, "external_caller", False, + ), + "public_link_caller": getattr( + agent.tool_executor, "public_link_caller", False, + ), + "api_write_allowlist": sorted( + getattr(agent.tool_executor, "api_write_allowlist", set()), + ), "agent_id": agent_id, "agent_type": agent.__class__.__name__, "prompt": getattr(agent, "prompt", ""), diff --git a/docsgpt/api/answer/services/stream_processor.py b/docsgpt/api/answer/services/stream_processor.py index b453aff0..f0b96aa3 100644 --- a/docsgpt/api/answer/services/stream_processor.py +++ b/docsgpt/api/answer/services/stream_processor.py @@ -4,13 +4,12 @@ import json import logging import uuid from pathlib import Path -from typing import Any, Callable, Dict, Optional, Set, TypeVar +from typing import Any, Callable, Dict, List, Optional, Set, Tuple, TypeVar from flask import after_this_request from docsgpt import tracing from docsgpt.agents.agent_creator import AgentCreator -from docsgpt.agents.default_tools import synthesized_default_tools from docsgpt.api.answer.services.compression import CompressionOrchestrator from docsgpt.api.answer.services.compression.token_counter import TokenCounter from docsgpt.api.answer.services.compression.types import is_compression_summary_row @@ -28,7 +27,9 @@ from docsgpt.core.model_utils import ( get_provider_from_model_id, validate_model_id, ) +from docsgpt.agents.tools.wiki import apply_resume_caller_rules, outside_edits_allowed from docsgpt.core.settings import settings +from docsgpt.guardrails.config import AgentConfig from sqlalchemy import text as sql_text from docsgpt.storage.db.base_repository import looks_like_uuid, row_to_dict @@ -37,8 +38,6 @@ from docsgpt.storage.db.repositories.attachments import AttachmentsRepository from docsgpt.storage.db.repositories.prompts import PromptsRepository from docsgpt.storage.db.repositories.sources import SourcesRepository from docsgpt.storage.db.repositories.team_scope import TeamScopeRepository -from docsgpt.storage.db.repositories.user_tools import UserToolsRepository -from docsgpt.storage.db.repositories.users import UsersRepository from docsgpt.api.user.team_sharing import can_access from docsgpt.storage.db.session import db_readonly, db_session from docsgpt.storage.db.source_config import SourceConfig @@ -52,6 +51,25 @@ from docsgpt.utils import ( logger = logging.getLogger(__name__) +def is_external_api_caller(data: Dict[str, Any], decoded_token: Optional[Dict], owner: Optional[str]) -> bool: + """Whether a request calls an agent with its API key on someone else's behalf. + + Widget and API requests carry the agent's key and run as its owner. The + owner previewing their own agent in the app sends the key too, but is + signed in as that owner. In local mode without auth everyone is the same + user, so there is no one else to tell apart. + + Args: + data: The request body. + decoded_token: The caller's token before the key's owner replaces it. + owner: The agent owner's user id. + """ + if not data.get("api_key"): + return False + caller = (decoded_token or {}).get("sub") + return not caller or caller != owner + + def _clamp_chunks(value: int) -> int: """Bound top-k to the range ``RetrievalConfig`` enforces, keeping 0. @@ -111,6 +129,10 @@ def get_prompt(prompt_id: str, prompts_collection=None) -> str: _PROMPT_PRESETS_WITHOUT_ROW = ("reduce",) +# Tools whose approval is decided per call from live state, not the stored +# ``require_approval`` flags (see ``ToolExecutor.check_pause``). +_LIVE_APPROVAL_TOOLS = frozenset({"remote_device", "code_executor"}) + def authorized_prompt_id(prompt_id: Any, principal: Optional[str], agent: Optional[dict] = None) -> Any: """``prompt_id`` if ``principal`` (or the agent's sponsor) may use it, else ``"default"``. @@ -134,20 +156,27 @@ def authorized_prompt_id(prompt_id: Any, principal: Optional[str], agent: Option pid = str(prompt_id) if is_composed_preset(pid) or pid in _PROMPT_PRESETS_WITHOUT_ROW: return prompt_id - from docsgpt.api.user.resource_access import active_sponsor, resolve + from docsgpt.api.user.resource_access import ( + REASON_OWNER_LOST_ACCESS, + log_stopped, + ref_access, + ) + # The same check the agent page's run state uses: the principal, else a + # live sponsor on the agent. + holder = {**agent, "user_id": principal} if agent and agent.get("id") else {"user_id": principal} try: with db_readonly() as conn: - ra = resolve(conn, "prompt", pid, principal) if principal else None - usable = ra is not None and ra.can("use") - if not usable and agent and agent.get("id"): - usable = active_sponsor(conn, "agent", agent, "prompt", pid) is not None + access = ref_access(conn, "agent", holder, "prompt", pid) if principal else None except Exception: logger.exception("Prompt access check failed for %s", pid) - usable = False - if usable: + access = None + if access is not None and access.principal: return prompt_id - logger.info("prompt %s not usable by %s; using the default prompt", pid, principal) + log_stopped( + "agent" if agent else "chat", holder, "prompt", pid, + access.reason if access is not None else REASON_OWNER_LOST_ACCESS, + ) return "default" @@ -158,14 +187,48 @@ def _agent_source_doc(conn: Any, sources_repo: Any, agent: dict, source_id: Any) editor who attached it while they still qualify. Read unscoped once authorized: an owner-scoped read misses a team-shared source. """ - from docsgpt.api.user.resource_access import ref_principal + from docsgpt.api.user.resource_access import log_stopped, ref_access - if not ref_principal(conn, "agent", agent, "source", str(source_id)): - logger.info("agent %s source %s not usable; skipped", agent.get("id"), source_id) + access = ref_access(conn, "agent", agent, "source", str(source_id)) + if not access.principal: + log_stopped("agent", agent, "source", source_id, access.reason) return None return sources_repo.get_by_id(str(source_id)) +def authorized_agent_sources(conn: Any, agent: dict) -> Tuple[Optional[dict], List[dict]]: + """The source rows an agent run retrieves from: primary first, then extras. + + Each is authorized like :func:`_agent_source_doc` (the owner, else the + editor who attached it), and a source listed twice appears once. + + Args: + conn: An open database connection. + agent: The ``agents`` row. + + Returns: + The primary source row (None when unset or not usable) and every + usable row in run order. + """ + sources_repo = SourcesRepository(conn) + primary: Optional[dict] = None + rows: List[dict] = [] + seen: set = set() + refs = [(True, agent.get("source_id"))] + refs.extend((False, sid) for sid in agent.get("extra_source_ids") or []) + for is_primary, sid_raw in refs: + if not sid_raw: + continue + source_doc = _agent_source_doc(conn, sources_repo, agent, sid_raw) + if not source_doc or str(source_doc["id"]) in seen: + continue + if is_primary: + primary = source_doc + seen.add(str(source_doc["id"])) + rows.append(source_doc) + return primary, rows + + def _wiki_write_owner(conn: Any, source_id: str, caller: str) -> Optional[str]: """The owner id to write a wiki source as, when ``caller`` may edit it.""" from docsgpt.api.user.resource_access import resolve @@ -231,12 +294,26 @@ class StreamProcessor: request_data: Dict[str, Any], decoded_token: Optional[Dict[str, Any]], trace_source: str = "stream", + *, + external_caller: bool = False, ): + """Bind a request to its processor. + + Args: + request_data: The request body. + decoded_token: The caller's token; ``/v1`` passes the agent owner's. + trace_source: The entry point the trace is stored under. + external_caller: Set by the server for requests authenticated by + an agent's API key whose token is the owner's (``/v1``), so + the run keeps the key holder's write limits. Never read from + the request body. + """ # Legacy attribute retained as None for any external callers that # introspect the processor; all DB access uses per-op connections. self.prompts_collection = None self.data = request_data self.decoded_token = decoded_token + self.external_caller = bool(external_caller) self.initial_user_id = ( self.decoded_token.get("sub") if self.decoded_token is not None else None ) @@ -250,6 +327,9 @@ class StreamProcessor: self.retriever_config = {} self.is_shared_usage = False self.shared_token = None + # Set by _get_agent_key: the caller reaches the agent only through its + # public link (not its owner, no team grant). + self.public_link_usage = False self.agent_id = self.data.get("agent_id") # Set by _get_agent_key once access checks pass; read for keyless runs. self._authorized_agent_row: Optional[Dict[str, Any]] = None @@ -325,7 +405,13 @@ class StreamProcessor: share it with the rest of the turn. It is always generated here, never taken from the request body: request quotas count distinct request ids, so a client-chosen id would let every call count as one. + + A request with neither a token nor an agent API key has nobody to run + for: nothing is set up (no pre-fetch runs the agent's tools), None is + returned and the route answers 401. """ + if not self.decoded_token and not self.data.get("api_key"): + return None if not getattr(self, "request_id", None): self.request_id = str(uuid.uuid4()) self.initialize() @@ -682,9 +768,10 @@ class StreamProcessor: # Team-shared agents are runnable by any member with a grant # (viewer is enough to run). Resolved live against team_members # on the SAME connection so a revoked grant/membership denies on - # the next call; resolution failure fails closed. + # the next call; resolution failure fails closed. Checked on a + # public agent too: a teammate there is not a link user. is_team_shared = False - if not (is_owner or is_shared_with_user) and user_id: + if not is_owner and user_id: try: is_team_shared = TeamScopeRepository(conn).can_read( user_id, "agent", str(agent["id"]) @@ -697,6 +784,7 @@ class StreamProcessor: if not (is_owner or is_shared_with_user or is_team_shared): raise Exception("Unauthorized access to the agent") + self.public_link_usage = not (is_owner or is_team_shared) # Authorized. Keep the row so _configure_agent can read fields that # do not depend on an API key — a draft agent has key = NULL, and # the builder preview runs exactly that path. @@ -729,7 +817,6 @@ class StreamProcessor: agent = AgentsRepository(conn).find_by_key(api_key) if not agent: raise Exception("Invalid API Key, please generate a new key", 401) - sources_repo = SourcesRepository(conn) # The repo dict uses "user_id" — the streaming path expects # a "user" key (legacy Mongo shape) for identity propagation. data: Dict[str, Any] = dict(agent) @@ -739,68 +826,29 @@ class StreamProcessor: # ``_configure_source`` ignores an empty ``data["sources"]``, # so the primary must appear in the union too — not only in # the legacy ``data["source"]`` slot. - sources_list: list = [] - seen: set = set() - primary_id = agent.get("source_id") - # ``sources`` row may have NULL ``retriever``/``chunks`` — - # fall back to the agent's value (``dict.get`` returns None - # even when the key exists with value None). - if primary_id: - source_doc = _agent_source_doc(conn, sources_repo, agent, primary_id) - if source_doc: - sid = str(source_doc["id"]) - data["source"] = sid - src_retriever = source_doc.get("retriever") - if src_retriever: - data["retriever"] = src_retriever - src_chunks = source_doc.get("chunks") - if src_chunks is not None: - data["chunks"] = src_chunks - sources_list.append( - { - "id": sid, - "retriever": src_retriever or "classic", - "chunks": ( - src_chunks if src_chunks is not None - else data.get("chunks", "6") - ), - # Per-source behaviour contract (lenient read). - "retrieval": SourceConfig.parse( - source_doc.get("config") - ).retrieval, - } - ) - seen.add(sid) - else: - data["source"] = None - else: - data["source"] = None - - for sid_raw in agent.get("extra_source_ids") or []: - if not sid_raw: - continue - source_doc = _agent_source_doc(conn, sources_repo, agent, sid_raw) - if not source_doc: - continue - sid = str(source_doc["id"]) - if sid in seen: - continue - src_retriever = source_doc.get("retriever") - src_chunks = source_doc.get("chunks") - sources_list.append( - { - "id": sid, - "retriever": src_retriever or "classic", - "chunks": ( - src_chunks if src_chunks is not None - else data.get("chunks", "6") - ), - "retrieval": SourceConfig.parse( - source_doc.get("config") - ).retrieval, - } - ) - seen.add(sid) + primary, source_docs = authorized_agent_sources(conn, agent) + # ``sources`` row may have NULL ``retriever``/``chunks`` — fall back to + # the agent's value (``dict.get`` returns None even when the key + # exists with value None). The primary's own values win for the agent. + data["source"] = str(primary["id"]) if primary else None + if primary: + if primary.get("retriever"): + data["retriever"] = primary["retriever"] + if primary.get("chunks") is not None: + data["chunks"] = primary["chunks"] + sources_list: list = [ + { + "id": str(source_doc["id"]), + "retriever": source_doc.get("retriever") or "classic", + "chunks": ( + source_doc["chunks"] if source_doc.get("chunks") is not None + else data.get("chunks", "6") + ), + # Per-source behaviour contract (lenient read). + "retrieval": SourceConfig.parse(source_doc.get("config")).retrieval, + } + for source_doc in source_docs + ] data["sources"] = sources_list data["default_model_id"] = data.get("default_model_id", "") return data @@ -987,6 +1035,9 @@ class StreamProcessor: agent_id, self.initial_user_id ) self.agent_id = str(agent_id) if agent_id else None + self.agent_config["public_link_caller"] = bool( + self.agent_id and getattr(self, "public_link_usage", False) + ) # Determine the effective API key (explicit > agent-derived) effective_key = self.data.get("api_key") or self.agent_key @@ -1024,6 +1075,13 @@ class StreamProcessor: ) # Set identity context + owner = self._agent_data.get("user") + self.agent_config["external_api_caller"] = getattr(self, "external_caller", False) or ( + is_external_api_caller(self.data, self.decoded_token, owner) + ) + self.agent_config["api_write_allowlist"] = AgentConfig.parse( + self._agent_data.get("config") + ).api_write_allowlist if self.data.get("api_key"): # External API key: use the key owner's identity self.initial_user_id = self._agent_data.get("user") @@ -1210,10 +1268,25 @@ class StreamProcessor: writable wiki source; the first match wins and the scan stops there so this runs at most one owner+source lookup per chat on the hot path. Returns None when no writable wiki source is present. + + An API-key or widget run (``outside_caller``) acts as the agent's + owner, so it gets the edit actions only when the wiki's owner turned + on ``wiki_outside_edits``; otherwise ``writes_allowed`` is False and + the tool offers only ``wiki_view``. A public-link visitor runs as + themselves, so they reach only wikis they may edit anyway; the switch + doesn't apply to them, but each of their edits waits for their + approval (``approval_required``), so the agent's prompt or sources + can't steer the model into changing their wiki unasked. """ caller = self.decoded_token.get("sub") if self.decoded_token else None if not caller: return None + # Processors built without __init__ (tests, resume helpers) lack these. + run_config = getattr(self, "agent_config", None) or {} + outside_caller = bool( + run_config.get("external_api_caller") or getattr(self, "external_caller", False) + ) + approval_required = bool(run_config.get("public_link_caller")) wiki_config: Optional[Dict[str, Any]] = None try: @@ -1237,6 +1310,9 @@ class StreamProcessor: "source_owner_id": owner, "decoded_token": self.decoded_token, "user": caller, + "outside_caller": outside_caller, + "writes_allowed": not outside_caller or outside_edits_allowed(source_doc), + "approval_required": approval_required, } break except Exception: @@ -1343,7 +1419,16 @@ class StreamProcessor: return None, None def pre_fetch_tools(self) -> Optional[Dict[str, Any]]: - """Pre-fetch tool data for template rendering before agent creation""" + """Pre-fetch tool data for template rendering before agent creation. + + Runs the actions the prompt template names on the toolset the agent + run gets, so a teammate or public-link user renders the owner's + prompt with the owner's tools, never their own. + + Returns: + Action results keyed by tool name and tool id, or None when + nothing was fetched. + """ if not settings.ENABLE_TOOL_PREFETCH: logger.info( "Tool pre-fetching disabled globally via ENABLE_TOOL_PREFETCH setting" @@ -1359,17 +1444,17 @@ class StreamProcessor: try: user_id = self.initial_user_id or "local" - agentless = self.agent_id is None - with db_readonly() as conn: - user_tools = UserToolsRepository(conn).list_active_for_user(user_id) - user_doc = ( - UsersRepository(conn).get(user_id) if agentless else None - ) - - default_docs = ( - synthesized_default_tools(user_doc) if agentless else [] + outside_caller = bool( + self.agent_config.get("external_api_caller") or self.agent_config.get("public_link_caller") ) - tool_docs = list(user_tools) + default_docs + # The same toolset the run gets: an agent's own tools (resolved as + # its owner, or the editor who attached them), else the caller's + # tools plus defaults. Explicit rows first, so they claim names. + run_tools = [ + tool for tool in self._run_tool_executor().get_tools().values() + if isinstance(tool, dict) and not tool.get("client_side") + ] + tool_docs = sorted(run_tools, key=lambda tool: bool(tool.get("default"))) if not tool_docs: return None @@ -1397,6 +1482,15 @@ class StreamProcessor: continue required_actions = None + owner = tool_doc.get("user_id") + if owner and (owner != user_id or outside_caller): + # Someone else's tool (a widget or API run carries the + # owner's id but isn't the owner): pre-fetch asks nobody, + # so only what the run would do without asking. + required_actions = self._unasked_actions(tool_doc, required_actions) + if not required_actions: + continue + tool_data = self._fetch_tool_data(tool_doc, required_actions) if tool_data: # Explicit rows claim the name key; a default tool takes @@ -1413,6 +1507,57 @@ class StreamProcessor: logger.warning(f"Failed to pre-fetch tools: {type(e).__name__}") return None + @staticmethod + def _unasked_actions( + tool_doc: Dict[str, Any], required_actions: Optional[Set[Optional[str]]] + ) -> Set[Optional[str]]: + """The required actions of someone else's tool that run without asking. + + A tool on someone else's connected account runs on their account or + needs the caller's own connection, a tool that decides approval per + call (a remote device, the code executor) can't be judged from its + stored flags, an approval-gated action waits for a person, and a + write with the owner's credentials needs the owner's say-so; + pre-fetch has none of these, so all are left out. + + Args: + tool_doc: The tool row, owned by someone other than the caller. + required_actions: Action names the template needs; None, or a set + holding None, means all of them. + + Returns: + The action names to run, empty when there are none. + """ + from docsgpt.connectors.permissions import owner_credential_writes, tool_actions + + if tool_doc.get("connection_id") or tool_doc.get("name") in _LIVE_APPROVAL_TOOLS: + return set() + owner_writes = set(owner_credential_writes(tool_doc)) + unasked = { + action.get("name") for action in tool_actions(tool_doc) + if action.get("name") and action.get("active", True) and not action.get("require_approval") + and action.get("name") not in owner_writes + } + if required_actions is None or None in required_actions: + return unasked + return {name for name in required_actions if name in unasked} + + def _run_tool_executor(self): + """A ``ToolExecutor`` resolving the toolset this turn's agent run gets. + + Returns: + ToolExecutor: Built with the run's key, user and agent. + """ + from docsgpt.agents.tool_executor import ToolExecutor + + user = self.decoded_token.get("sub") if self.decoded_token else None + return ToolExecutor( + user_api_key=self.agent_config.get("user_api_key"), + user=user, + decoded_token=self.decoded_token, + agent_id=self.agent_id, + ) + def _enabled_tool_names(self) -> Optional[set]: """Resolve the tool names enabled for this turn, for ``tools.enabled`` gating. @@ -1422,15 +1567,7 @@ class StreamProcessor: (keeps the section) rather than hiding guidance when resolution breaks. """ try: - from docsgpt.agents.tool_executor import ToolExecutor - - user = self.decoded_token.get("sub") if self.decoded_token else None - tool_executor = ToolExecutor( - user_api_key=self.agent_config.get("user_api_key"), - user=user, - decoded_token=self.decoded_token, - agent_id=self.agent_id, - ) + tool_executor = self._run_tool_executor() client_tools = self.data.get("client_tools") if client_tools: tool_executor.client_tools = client_tools @@ -1638,21 +1775,37 @@ class StreamProcessor: from docsgpt.llm.handlers.handler_creator import LLMHandlerCreator from docsgpt.llm.llm_creator import LLMCreator + # Who is resuming, classified from this request alone: the saved state + # says who paused the turn, but anyone holding the agent's key (a + # widget key is public) can send the tool actions that resume it. + request_key = self.data.get("api_key") + original_token = self.decoded_token + key_agent = None + if request_key: + with db_readonly() as conn: + key_agent = AgentsRepository(conn).find_by_key(request_key) + key_owner = ( + (key_agent.get("user_id") or key_agent.get("user")) if key_agent else None + ) + request_external = bool(getattr(self, "external_caller", False)) or ( + bool(request_key) and is_external_api_caller(self.data, original_token, key_owner) + ) + request_public_link = False + named_agent = self.data.get("agent_id") + if named_agent and not request_key: + try: + self._get_agent_key(str(named_agent), self.initial_user_id) + except Exception as exc: + raise ValueError("This conversation can't be resumed with that agent") from exc + request_public_link = bool(getattr(self, "public_link_usage", False)) + # api_key-in-body auth carries no JWT, so initial_user_id is None — but # the state was saved under the agent owner. Resolve the owner so the # lookup / mark_resuming / delete_state key on the same id. (No-op for # v1, which already passes an owner-scoped decoded_token.) - if self.initial_user_id is None and self.data.get("api_key"): - with db_readonly() as conn: - agent_doc = AgentsRepository(conn).find_by_key(self.data["api_key"]) - owner = ( - (agent_doc.get("user_id") or agent_doc.get("user")) - if agent_doc - else None - ) - if owner: - self.initial_user_id = owner - self.decoded_token = {"sub": owner} + if self.initial_user_id is None and key_owner: + self.initial_user_id = key_owner + self.decoded_token = {"sub": key_owner} cont_service = ContinuationService() state = claimed_state or cont_service.claim_state( @@ -1661,6 +1814,21 @@ class StreamProcessor: if not state: raise ValueError("No pending tool state found for this conversation") + # A request that names an agent (by key or id) resumes only that + # agent's turn; the claim goes back so its rightful caller can resume. + saved_agent = str((state.get("agent_config") or {}).get("agent_id") or "").lower() + targets = [] + if request_key: + targets.append(str((key_agent or {}).get("id") or (key_agent or {}).get("_id") or "")) + if named_agent: + targets.append(str(named_agent)) + if any(target.lower() != saved_agent or not target for target in targets): + try: + cont_service.release_claim(conversation_id, self.initial_user_id) + except Exception: + logger.warning("Failed to release a refused resume claim", exc_info=True) + raise ValueError("This conversation belongs to a different agent") + messages = state["messages"] pending_tool_calls = state["pending_tool_calls"] tools_dict = state["tools_dict"] @@ -1696,11 +1864,20 @@ class StreamProcessor: if callable(importer): importer(agent_config.get("responses_state")) llm_handler = LLMHandlerCreator.create_handler(llm_name or "default") + # Outside if either who paused the turn or who resumes it is. + resume_external = bool(agent_config.get("external_api_caller")) or request_external + resume_public_link = bool(agent_config.get("public_link_caller")) or request_public_link + apply_resume_caller_rules( + tools_dict, outside_caller=resume_external, public_link_caller=resume_public_link, + ) tool_executor = ToolExecutor( user_api_key=user_api_key, user=self.initial_user_id, decoded_token=self.decoded_token, agent_id=agent_id, + external_caller=resume_external, + public_link_caller=resume_public_link, + api_write_allowlist=agent_config.get("api_write_allowlist"), ) tool_executor.conversation_id = conversation_id # Restore client tools so they stay available for subsequent LLM calls @@ -1878,6 +2055,9 @@ class StreamProcessor: user=user, decoded_token=self.decoded_token, agent_id=self.agent_id, + external_caller=bool(self.agent_config.get("external_api_caller")), + public_link_caller=bool(self.agent_config.get("public_link_caller")), + api_write_allowlist=self.agent_config.get("api_write_allowlist"), ) tool_executor.conversation_id = self.conversation_id # Pass client-side tools so they get merged in get_tools() diff --git a/docsgpt/api/connector/connections.py b/docsgpt/api/connector/connections.py new file mode 100644 index 00000000..1aa03e8e --- /dev/null +++ b/docsgpt/api/connector/connections.py @@ -0,0 +1,722 @@ +"""Connectors catalog and connections API. + +``/api/connectors/catalog`` lists every service DocsGPT can connect to, with +whether the server is set up for it and the caller's connection summary. +``/api/connections`` lists and manages the caller's connections. Responses +never include tokens or secrets. +""" + +from __future__ import annotations + +from flask import current_app, jsonify, make_response, request +from flask_restx import Namespace, Resource + +import uuid + +from docsgpt.api import api +from docsgpt.api.user.authz import ROLE_ADMIN, has_role +from docsgpt.connectors import catalog, service +from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository +from docsgpt.storage.db.session import db_readonly, db_session +from docsgpt.security.encryption import CredentialDecryptionError + +_FREQUENCIES = ("never", "daily", "weekly", "monthly") + +connections_ns = Namespace("connections", description="Connectors and connections", path="/api") +api.add_namespace(connections_ns) + + +def _user_id() -> str | None: + token = getattr(request, "decoded_token", None) + return token.get("sub") if isinstance(token, dict) else None + + +def _unauthorized(): + return make_response(jsonify({"success": False, "error": "Unauthorized"}), 401) + + +def _not_found(): + return make_response(jsonify({"success": False, "error": "Connection not found"}), 404) + + +@connections_ns.route("/connectors/catalog") +class ConnectorCatalog(Resource): + @api.doc(description="Every connector with its availability and the caller's connection summary") + def get(self): + user_id = _user_id() + if not user_id: + return _unauthorized() + try: + with db_readonly() as conn: + entries = service.catalog_for_user( + conn, user_id, is_admin=has_role(request.decoded_token, ROLE_ADMIN), + ) + except Exception as err: + current_app.logger.error(f"Error building connector catalog: {err}", exc_info=True) + return make_response(jsonify({"success": False, "error": "Failed to load connectors"}), 500) + return make_response(jsonify({"success": True, "connectors": entries}), 200) + + +def _json_body() -> dict: + body = request.get_json(silent=True) + return body if isinstance(body, dict) else {} + + +def _owned(conn, connection_id: str, user_id: str): + return ConnectorSessionsRepository(conn).get_for_user(connection_id, user_id) + + +def _error(message: str, status: int, **extra): + return make_response(jsonify({"success": False, "error": message, **extra}), status) + + +@connections_ns.route("/connections") +class ConnectionsList(Resource): + @api.doc( + description=( + "Create a connection from pasted credentials: " + "{connector_key, credentials, label?}. Same credentials reuse the same connection." + ) + ) + def post(self): + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + definition = catalog.get_definition(body.get("connector_key")) + if definition is None or definition.auth_kind != "api_key": + return _error("This connector does not take pasted credentials", 400) + if definition.missing_settings: + return _error("This connector needs admin setup", 400, code="needs_setup") + credentials = body.get("credentials") + if not isinstance(credentials, dict): + return _error("credentials must be an object", 400) + label = body.get("label") or None + if definition.key == "github": + # Check the token now, not at the first sync, and name the + # connection after the account rather than a hint of the token. + from docsgpt.connectors import github + + try: + label = label or github.token_account(credentials.get("access_token")) + except github.TokenRejected as err: + return _error(str(err), 400, code="invalid_credentials") + except service.TransientConnectionError as err: + return _error(str(err), 502) + try: + with db_session() as conn: + row, created = service.create_api_key_connection( + conn, user_id, definition, credentials, label=label, + ) + except service.EncryptionKeyNotConfigured as err: + return _error(str(err), 400, code="encryption_key_default") + except service.ConnectorDisabled as err: + return _error(str(err), 403, code="disabled") + except ValueError as err: + return _error(str(err), 400) + except Exception as err: + current_app.logger.error(f"Error creating connection: {err}", exc_info=True) + return _error("Failed to create connection", 500) + return make_response( + jsonify( + { + "success": True, + "created": created, + "connection": service.serialize_connection(row), + "setup": dict(definition.setup), + } + ), + 201 if created else 200, + ) + + @api.doc(description="The caller's connections with status and linked resource counts") + def get(self): + user_id = _user_id() + if not user_id: + return _unauthorized() + try: + with db_readonly() as conn: + connections = service.list_connections(conn, user_id) + except Exception as err: + current_app.logger.error(f"Error listing connections: {err}", exc_info=True) + return make_response(jsonify({"success": False, "error": "Failed to load connections"}), 500) + return make_response(jsonify({"success": True, "connections": connections}), 200) + + +@connections_ns.route("/connections/") +class ConnectionDetail(Resource): + @api.doc(description="One connection with the sources it syncs and the tools it provides") + def get(self, connection_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + try: + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get_for_user(connection_id, user_id) + if row is None: + return _not_found() + detail = service.connection_detail(conn, row) + except Exception as err: + current_app.logger.error(f"Error loading connection: {err}", exc_info=True) + return make_response(jsonify({"success": False, "error": "Failed to load connection"}), 500) + return make_response(jsonify({"success": True, "connection": detail}), 200) + + @api.doc(description="Name an account: {name}. An empty name clears it. Owner only.") + def patch(self, connection_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + name = _json_body().get("name") + if not isinstance(name, str) or len(name.strip()) > service.ACCOUNT_NAME_MAX: + return _error(f"name must be text of at most {service.ACCOUNT_NAME_MAX} characters", 400) + with db_session() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + connection = service.rename_connection(conn, row, name) + return make_response(jsonify({"success": True, "connection": connection}), 200) + + @api.doc( + description=( + "Remove a connection: {sources: keep | delete, tools: delete | keep}. " + "Kept sources keep their content and stop syncing." + ) + ) + def delete(self, connection_id: str): + from docsgpt.api.user.sources.routes import delete_source + + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + sources_mode = body.get("sources", "keep") + tools_mode = body.get("tools", "delete") + if sources_mode not in ("keep", "delete") or tools_mode not in ("keep", "delete"): + return _error("sources and tools must be keep or delete", 400) + try: + with db_session() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + to_delete = service.remove_connection(conn, row, sources=sources_mode, tools=tools_mode) + failed = [str(doc["id"]) for doc in to_delete if not delete_source(user_id, doc)] + except Exception as err: + current_app.logger.error(f"Error removing connection: {err}", exc_info=True) + return _error("Failed to remove connection", 500) + return make_response(jsonify({"success": True, "failed_sources": failed}), 200) + + +@connections_ns.route("/connections//disconnect") +class ConnectionDisconnect(Resource): + @api.doc(description="Delete a connection's stored credentials; its sources and tools stay") + def post(self, connection_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + try: + with db_session() as conn: + row = ConnectorSessionsRepository(conn).get_for_user(connection_id, user_id) + if row is None: + return _not_found() + connection = service.disconnect(conn, row) + except Exception as err: + current_app.logger.error(f"Error disconnecting connection: {err}", exc_info=True) + return make_response(jsonify({"success": False, "error": "Failed to disconnect"}), 500) + return make_response(jsonify({"success": True, "connection": connection}), 200) + + +@connections_ns.route("/connections//setup") +class ConnectionSetup(Resource): + @api.doc( + description=( + "Apply the connect wizard's choices: {create_tools, allow_writes?, tool_permissions?, " + "sync?: {items, frequency, name?, config?}}. allow_writes points GitHub's tool at its write " + "endpoint. config is the synced source's retrieval settings, validated like an upload's. Honours an Idempotency-Key header for the sync." + ) + ) + def post(self, connection_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + create_tools = body.get("create_tools", True) + allow_writes = body.get("allow_writes", False) + if not isinstance(allow_writes, bool): + return _error("allow_writes must be true or false", 400) + try: + mcp_actions = None + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + discover = bool(row) and create_tools and service.needs_mcp_discovery(conn, row) + forbidden = bool(row) and allow_writes and not service.writes_allowed( + service.load_policies(conn), catalog.connector_key_for_row(row), + ) + if row is None: + return _not_found() + if service.normalize_status(row) != service.STATUS_CONNECTED: + return _error("Reconnect before setting up", 409, code="reconnect") + if forbidden: + return _error("Changes through this connector are turned off by an admin", 403, + code="writes_forbidden") + if discover: + # GitHub's tool is its MCP server: read its actions before + # the write transaction, not while holding it open. + from docsgpt.connectors.mcp import discover_builtin_actions + + try: + mcp_actions = discover_builtin_actions(user_id, row, writes=allow_writes) + except service.ConnectionUnavailable: + return _error("Reconnect before setting up", 409, code="reconnect") + except Exception as err: + current_app.logger.warning(f"Could not list the MCP server's tools: {err}") + return _error("The service's tools could not be reached. Try again.", 502, + code="tools_unavailable") + with db_session() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + tools = [] + if create_tools: + tools = service.ensure_connection_tools( + conn, user_id, row, permissions=body.get("tool_permissions") or None, + mcp_actions=mcp_actions, mcp_writes=allow_writes, + ) + account_parameters = service.connection_parameters(row) + tool_payload = [service.serialize_tool(tool, account_parameters) for tool in tools] + sources = [] + if body.get("sync"): + started = _start_sync(user_id, row, body["sync"]) + if isinstance(started, tuple): + return _error(*started) + sources.append(started) + except Exception as err: + current_app.logger.error(f"Error setting up connection: {err}", exc_info=True) + return _error("Failed to set up connection", 500) + return make_response(jsonify({"success": True, "tools": tool_payload, "sources": sources}), 200) + + +def _start_sync(user_id: str, row: dict, sync: dict): + """Queue the first ingest of a source synced from ``row``. + + Args: + user_id: The connection's owner. + row: The connection row. + sync: ``{items, frequency, name?, config?}``. ``config`` is the source's + retrieval settings (a ``SourceConfig``), applied as an upload's are. + + Returns: + The source summary, or ``(message, status)`` on a bad request. + """ + from docsgpt.api.user.sources.upload import ( + _claim_task_or_get_cached, + _derive_source_id, + _parse_source_config, + _read_idempotency_key, + _scoped_idempotency_key, + ) + from docsgpt.api.user.tasks import ingest_connector_task, ingest_remote + + definition = catalog.get_definition(catalog.connector_key_for_row(row)) + if definition is None or not definition.sync_ingestor: + return ("This connector does not sync content", 400) + items = sync.get("items") or {} + if not isinstance(items, dict): + return ("items must be an object", 400) + frequency = sync.get("frequency") or definition.default_sync_frequency + if frequency not in _FREQUENCIES: + return ("Unknown sync frequency", 400) + name = (sync.get("name") or "").strip() + source_config, config_error = _parse_source_config(sync.get("config")) + if config_error is not None: + return ("Invalid source config", 400) + if definition.sync_ingestor == "github": + from docsgpt.parser.remote.github_loader import GitHubLoader + + repo = GitHubLoader.normalize_repo(str(items.get("repo_url") or "")) + if not repo: + return ("Pick a GitHub repository", 400) + items = {**items, "repo_url": repo} + name = name or repo + elif definition.sync_ingestor == "linear": + from docsgpt.connectors import linear + + try: + items = linear.normalize_selection(items) + except ValueError as err: + return (str(err), 400) + name = name or linear.selection_name(items) + name = name or definition.name + # Validate before claiming the idempotency key: a rejected request must + # leave the key free for the corrected retry. + if definition.auth_kind == "oauth": + file_ids = [str(i) for i in items.get("file_ids") or [] if i] + folder_ids = [str(i) for i in items.get("folder_ids") or [] if i] + if not file_ids and not folder_ids: + return ("Pick at least one file or folder", 400) + task_fn = ingest_connector_task + kwargs = { + "job_name": name, + "user": user_id, + "source_type": definition.sync_ingestor, + "connection_id": str(row["id"]), + "file_ids": file_ids, + "folder_ids": folder_ids, + "recursive": bool(items.get("recursive", True)), + "sync_frequency": frequency, + "config": source_config, + } + else: + if definition.sync_ingestor == "linear": + # The teams and projects picked, read with the connection's MCP sign-in. + source_data = items + else: + fields = {f.key for f in definition.setup_fields} + source_data = {k: v for k, v in items.items() if k in fields and v not in (None, "")} + missing = [f.label for f in definition.setup_fields if f.required and f.key not in source_data] + if missing: + return (f"Missing: {', '.join(missing)}", 400) + task_fn = ingest_remote + kwargs = { + "source_data": source_data, + "job_name": name, + "user": user_id, + "loader": definition.sync_ingestor, + "connection_id": str(row["id"]), + "sync_frequency": frequency, + "config": source_config, + } + idempotency_key, _ = _read_idempotency_key() + scoped_key = _scoped_idempotency_key(idempotency_key, user_id) + task_id = None + if scoped_key: + task_id, cached = _claim_task_or_get_cached(scoped_key, "connection_setup_sync") + if cached is not None: + return {"id": cached.get("source_id"), "task_id": cached.get("task_id"), "name": name} + source_id = str(_derive_source_id(scoped_key)) if scoped_key else str(uuid.uuid4()) + options = {"task_id": task_id} if task_id else {} + task = task_fn.apply_async( + kwargs={**kwargs, "idempotency_key": scoped_key, "source_id": source_id}, **options, + ) + return {"id": source_id, "task_id": task_id or task.id, "name": name, "sync_frequency": frequency} + + +@connections_ns.route("/connections//repositories") +class ConnectionRepositories(Resource): + @api.doc( + description=( + "GitHub: the repositories the connection can read, for the sync picker. " + "install_url is where a GitHub App sign-in chooses more repositories." + ) + ) + def get(self, connection_id: str): + from docsgpt.connectors import github + + user_id = _user_id() + if not user_id: + return _unauthorized() + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None or catalog.connector_key_for_row(row) != "github": + return _not_found() + app_sign_in = (row.get("auth_kind") or "") == "oauth" + try: + token = service.access_credentials(row).get("access_token") + repositories = github.list_repositories(token or "", app=app_sign_in) + except service.ConnectionUnavailable: + return _error("Reconnect to continue", 409, code="reconnect") + except github.TokenRejected as err: + service.mark_reconnect_needed(connection_id, str(err)) + return _error("Reconnect to continue", 409, code="reconnect") + except service.TransientConnectionError: + return _error("GitHub is not responding. Try again.", 503) + except Exception as err: + current_app.logger.error(f"Error listing GitHub repositories: {err}", exc_info=True) + return _error("Failed to list repositories", 502) + install_url = None + if app_sign_in: + from docsgpt.core.settings import settings + + slug = settings.GITHUB_APP_SLUG + install_url = f"https://github.com/apps/{slug}/installations/new" if slug else None + return make_response( + jsonify({"success": True, "repositories": repositories, "install_url": install_url}), 200, + ) + + +@connections_ns.route("/connections//linear") +class LinearWorkspace(Resource): + @api.doc( + description=( + "Linear: the teams and projects the connection can see, for the sync picker. " + "Read through Linear's MCP server with the connection's sign-in." + ) + ) + def get(self, connection_id: str): + from docsgpt.connectors import linear, mcp + + user_id = _user_id() + if not user_id: + return _unauthorized() + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None or catalog.connector_key_for_row(row) != linear.LINEAR_CONNECTOR: + return _not_found() + try: + workspace = mcp.run_connection_session(row, linear.mcp_url(), linear.list_workspace) + except service.ConnectionUnavailable: + return _error("Reconnect to continue", 409, code="reconnect") + except service.TransientConnectionError: + return _error("Linear is not responding. Try again.", 503) + except Exception as err: + current_app.logger.error(f"Error listing Linear teams: {err}", exc_info=True) + return _error("Failed to list Linear teams", 502) + return make_response(jsonify({"success": True, **workspace}), 200) + + +@connections_ns.route("/connections//reconnect") +class ConnectionReconnect(Resource): + @api.doc( + description=( + "OAuth: returns an authorization URL for the same account. " + "API key: accepts {credentials} and replaces the stored ones." + ) + ) + def post(self, connection_id: str): + from docsgpt.api.connector.routes import build_authorization + + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + try: + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + key = catalog.connector_key_for_row(row) + definition = catalog.get_definition(key) + auth_kind = row.get("auth_kind") or (definition.auth_kind if definition else None) + if auth_kind == "oauth": + started = build_authorization(row["provider"], user_id, connection_id) + return make_response(jsonify({"success": True, "kind": "oauth", **started}), 200) + if auth_kind == "mcp_oauth": + # The MCP client runs the OAuth dance (dynamic registration, + # PKCE); the frontend starts it through /api/mcp_server/test. + return make_response( + jsonify({"success": True, "kind": "mcp_oauth", "server_url": row.get("server_url")}), 200, + ) + credentials = body.get("credentials") + if not isinstance(credentials, dict) or not credentials: + return _error("credentials are required", 400) + service.ensure_can_store_credentials() + with db_session() as conn: + locked = ConnectorSessionsRepository(conn).get_for_update(connection_id) + # read_secrets, not load_secrets: flagging an unreadable row + # would write it from a second transaction while this one + # holds its lock. The new credentials replace it anyway. + try: + stored = service.read_secrets(locked) + except CredentialDecryptionError: + stored = {} + merged = {**(stored.get("credentials") or {}), **{k: v for k, v in credentials.items() if v}} + service.write_secrets( + conn, locked, {**stored, "credentials": merged}, + status=service.STATUS_CONNECTED, last_error=None, + ) + service.resume_sources(conn, connection_id) + connection = service.serialize_connection(ConnectorSessionsRepository(conn).get(connection_id)) + except service.EncryptionKeyNotConfigured as err: + return _error(str(err), 400, code="encryption_key_default") + except Exception as err: + current_app.logger.error(f"Error reconnecting: {err}", exc_info=True) + return _error("Failed to reconnect", 500) + return make_response(jsonify({"success": True, "kind": "api_key", "connection": connection}), 200) + + +@connections_ns.route("/connections//picker-token") +class ConnectionPickerToken(Resource): + @api.doc(description="A short-lived access token for a browser-side file picker. Owner only; never a refresh token.") + def post(self, connection_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None or (row.get("auth_kind") or "oauth") != "oauth": + return _not_found() + try: + token = service.picker_token(connection_id) + except service.ConnectionUnavailable: + return _error("Reconnect to continue", 409, code="reconnect") + except service.TransientConnectionError: + return _error("The provider is not responding. Try again.", 503) + return make_response(jsonify({"success": True, **token}), 200) + + +@connections_ns.route("/connections/claim") +class ConnectionClaim(Resource): + @api.doc( + description=( + "One-time link of a legacy browser session token ({provider, session_token}) " + "to the caller's connection. Removed next release." + ) + ) + def post(self): + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + provider, token = body.get("provider"), body.get("session_token") + if not provider or not token: + return _error("provider and session_token are required", 400) + with db_readonly() as conn: + row = service.claim_session_token(conn, user_id, str(provider), str(token)) + if row is None: + return _not_found() + return make_response(jsonify({"success": True, "connection_id": str(row["id"])}), 200) + + +@connections_ns.route("/connections//tools//permissions") +class ConnectionToolPermissions(Resource): + @api.doc(description="Set per-action permissions: {permissions: {action: always | ask | off}}") + def put(self, connection_id: str, tool_id: str): + from docsgpt.connectors.permissions import PERMISSIONS + + user_id = _user_id() + if not user_id: + return _unauthorized() + permissions = _json_body().get("permissions") + if not isinstance(permissions, dict) or any(p not in PERMISSIONS for p in permissions.values()): + return _error("permissions must map action names to always, ask or off", 400) + with db_session() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + tool = service.set_tool_permissions(conn, user_id, connection_id, tool_id, permissions) + if tool is None: + return _not_found() + payload = service.serialize_tool(tool, service.connection_parameters(row)) + return make_response(jsonify({"success": True, "tool": payload}), 200) + + +@connections_ns.route("/connections//tools//parameters") +class ConnectionToolParameters(Resource): + @api.doc( + description=( + "Fix or release an action's parameters: {action, parameters: {name: value | null}}. " + "A value is sent on every call and hidden from the model; null lets the model decide. Owner only." + ) + ) + def put(self, connection_id: str, tool_id: str): + user_id = _user_id() + if not user_id: + return _unauthorized() + body = _json_body() + action, pins = body.get("action"), body.get("parameters") + if not isinstance(action, str) or not isinstance(pins, dict) or not pins: + return _error("Send the action and a map of parameters to values or null", 400) + with db_session() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + try: + tool = service.set_tool_parameters(conn, user_id, connection_id, tool_id, action, pins) + except ValueError as err: + return _error(str(err), 400) + if tool is None: + return _not_found() + payload = service.serialize_tool(tool, service.connection_parameters(row)) + return make_response(jsonify({"success": True, "tool": payload}), 200) + + +@connections_ns.route("/connections//refresh-tools") +class ConnectionRefreshTools(Resource): + @api.doc(description="MCP: re-scan the server's actions and return what was added and removed") + def post(self, connection_id: str): + from docsgpt.connectors.mcp import refresh_mcp_tools + + user_id = _user_id() + if not user_id: + return _unauthorized() + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + try: + diff = refresh_mcp_tools(user_id, row) + except service.ConnectionUnavailable: + return _error("Reconnect to continue", 409, code="reconnect") + except Exception as err: + current_app.logger.error(f"Error refreshing MCP tools: {err}", exc_info=True) + return _error("Failed to refresh tools", 502) + return make_response(jsonify({"success": True, **diff}), 200) + + +@connections_ns.route("/connections//writes") +class ConnectionWrites(Resource): + @api.doc( + description=( + "GitHub: let agents make changes through this connection, or only read: {allow}. " + "Re-reads the tool's actions from the matching endpoint. Owner only." + ) + ) + def put(self, connection_id: str): + from docsgpt.connectors.mcp import NoMcpTool, set_builtin_writes + + user_id = _user_id() + if not user_id: + return _unauthorized() + allow = _json_body().get("allow") + if not isinstance(allow, bool): + return _error("allow must be true or false", 400) + with db_readonly() as conn: + row = _owned(conn, connection_id, user_id) + if row is None: + return _not_found() + try: + result = set_builtin_writes(user_id, row, allow) + except service.WritesForbidden as err: + return _error(str(err), 403, code="writes_forbidden") + except NoMcpTool as err: + return _error(str(err), 409, code="no_tools") + except ValueError as err: + return _error(str(err), 400) + except service.ConnectionUnavailable: + return _error("Reconnect to continue", 409, code="reconnect") + except Exception as err: + current_app.logger.error(f"Error switching write access: {err}", exc_info=True) + return _error("The service's tools could not be reached. Try again.", 502, code="tools_unavailable") + return make_response(jsonify({"success": True, **result}), 200) + + +@connections_ns.route("/connections/tools//credential-mode") +class ToolCredentialMode(Resource): + @api.doc( + description=( + "Whose account a shared connection-backed tool uses: {mode: owner | member}. " + "Owner only; refused when an admin forces a mode for the connector." + ) + ) + def put(self, tool_id: str): + from docsgpt.connectors.resolve import MODE_MEMBER, MODE_OWNER + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + user_id = _user_id() + if not user_id: + return _unauthorized() + mode = _json_body().get("mode") + if mode not in (MODE_OWNER, MODE_MEMBER): + return _error("mode must be owner or member", 400) + with db_session() as conn: + tools = UserToolsRepository(conn) + tool = tools.get_any(tool_id, user_id) + if tool is None or tool.get("user_id") != user_id or not tool.get("connection_id"): + return _error("Tool not found", 404) + connection = ConnectorSessionsRepository(conn).get(str(tool["connection_id"])) + forced = service.forced_credential_mode( + conn, catalog.connector_key_for_row(connection) if connection else None, + ) + if forced and forced != mode: + return _error("An admin sets this for every share", 409, code="forced", mode=forced) + tools.update(str(tool["id"]), user_id, {"credential_mode": mode}) + return make_response(jsonify({"success": True, "mode": mode}), 200) diff --git a/docsgpt/api/connector/routes.py b/docsgpt/api/connector/routes.py index dccdda16..a2d10c06 100644 --- a/docsgpt/api/connector/routes.py +++ b/docsgpt/api/connector/routes.py @@ -1,7 +1,6 @@ import base64 import html import json -import uuid from typing import Optional from urllib.parse import urlencode, urlsplit @@ -22,6 +21,7 @@ from docsgpt.api.user.sources.access import load_source from docsgpt.api.user.tasks import ( ingest_connector_task, ) +from docsgpt.connectors import service from docsgpt.core.settings import settings from docsgpt.parser.connectors.connector_creator import ConnectorCreator from docsgpt.storage.db.repositories.connector_sessions import ( @@ -101,17 +101,25 @@ def _js_literal(value) -> str: def _render_callback_page( status: str, message: str, provider_raw: str, session_token: str = "", user_email: str = "", + connection_id: str = "", ): """Popup page that reports an OAuth result to the opener on allowed origins only.""" status = status if status in ("success", "error", "cancelled") else "error" # The script only carries server-side values: the provider key comes from the # supported-connector list rather than the request, and no request text is posted. provider_key = next( - (key for key in ConnectorCreator.get_supported_connectors() if key == provider_raw.lower()), None, + (key for key in ConnectorCreator.get_auth_providers() if key == provider_raw.lower()), None, ) payload = None if provider_key and status == "success" and session_token: - payload = {"type": f"{provider_key}_auth_success", "session_token": session_token, "user_email": user_email} + payload = { + "type": f"{provider_key}_auth_success", + # The connection id is what current frontends use; the session + # token is kept for one release for frontends from before it. + "connection_id": connection_id, + "session_token": session_token, + "user_email": user_email, + } elif provider_key and status == "error": # The frontend shows its own localized failure message; cancellations are # reported when the popup closes. @@ -171,6 +179,42 @@ def _render_callback_page( +def build_authorization( + provider: str, user_id: str, connection_id: Optional[str] = None, *, install: bool = False, +) -> dict: + """Start an OAuth sign-in for ``provider`` and return its authorization URL. + + Args: + provider: The connector, e.g. ``google_drive`` or ``github``. + user_id: The caller. + connection_id: The caller's connection to sign in again, if any. + install: GitHub: send the user to install the GitHub App (where the + repositories it can read are chosen) instead of straight to + authorization. GitHub returns to the callback with the same + state when the app requests authorization during installation. + + Raises: + service.EncryptionKeyNotConfigured: See ``ensure_can_store_credentials``. + service.ConnectionUnavailable: ``connection_id`` is not the caller's. + ValueError: ``install`` for a provider with no installation page. + """ + service.ensure_can_store_credentials() + with db_session() as conn: + session_row = service.begin_oauth(conn, user_id, provider, connection_id) + state = base64.urlsafe_b64encode( + json.dumps({"provider": provider, "object_id": str(session_row["id"])}).encode() + ).decode() + auth = ConnectorCreator.create_auth(provider) + if install and not hasattr(auth, "get_installation_url"): + raise ValueError(f"{provider} has no installation page") + url = auth.get_installation_url(state=state) if install else auth.get_authorization_url(state=state) + return { + "authorization_url": url, + "state": state, + "callback_origin": _origin_of(settings.CONNECTOR_REDIRECT_BASE_URI), + } + + @connectors_ns.route("/api/connectors/auth") class ConnectorAuth(Resource): @api.doc(description="Get connector OAuth authorization URL", params={"provider": "Connector provider (e.g., google_drive)"}) @@ -180,27 +224,26 @@ class ConnectorAuth(Resource): if not provider: return make_response(jsonify({"success": False, "error": "Missing provider"}), 400) - if not ConnectorCreator.is_supported(provider): + if not (ConnectorCreator.is_supported(provider) or ConnectorCreator.has_auth(provider)): return make_response(jsonify({"success": False, "error": f"Unsupported provider: {provider}"}), 400) decoded_token = request.decoded_token if not decoded_token: return make_response(jsonify({"success": False, "error": "Unauthorized"}), 401) user_id = decoded_token.get('sub') - - with db_session() as conn: - session_row = ConnectorSessionsRepository(conn).upsert( - user_id, provider, status="pending", + try: + started = build_authorization( + provider, user_id, request.args.get("connection_id") or None, + install=request.args.get("install") in ("1", "true"), ) - session_pg_id = str(session_row["id"]) - state_dict = { - "provider": provider, - "object_id": session_pg_id, - } - state = base64.urlsafe_b64encode(json.dumps(state_dict).encode()).decode() - - auth = ConnectorCreator.create_auth(provider) - authorization_url = auth.get_authorization_url(state=state) + except service.EncryptionKeyNotConfigured as err: + return make_response( + jsonify({"success": False, "error": str(err), "code": "encryption_key_default"}), 400, + ) + except service.ConnectionUnavailable: + return make_response(jsonify({"success": False, "error": "Connection not found"}), 404) + except service.ConnectorDisabled as err: + return make_response(jsonify({"success": False, "error": str(err), "code": "disabled"}), 403) # The popup drops results for origins outside the allowlist, which the # user only sees as a cancelled sign-in; name the missing origin here. request_origin = _origin_of(request.headers.get("Origin")) @@ -209,12 +252,7 @@ class ConnectorAuth(Resource): f"Connector sign-in requested from {request_origin}, which cannot receive the result; " "add it to CONNECTOR_ALLOWED_ORIGINS" ) - return make_response(jsonify({ - "success": True, - "authorization_url": authorization_url, - "state": state, - "callback_origin": _origin_of(settings.CONNECTOR_REDIRECT_BASE_URI), - }), 200) + return make_response(jsonify({"success": True, **started}), 200) except Exception as e: current_app.logger.error(f"Error generating connector auth URL: {e}", exc_info=True) return make_response(jsonify({"success": False, "error": "Failed to generate authorization URL"}), 500) @@ -233,12 +271,24 @@ class ConnectorsCallback(Resource): state = request.args.get('state') error = request.args.get('error') + if not state and request.args.get('installation_id'): + # The GitHub App was installed from GitHub itself, not from a + # DocsGPT sign-in: there is no state to tie the code to a user, + # so the code is ignored and the user goes back to DocsGPT. + return _render_callback_page( + "success", + "The GitHub App is installed. Return to DocsGPT and refresh the repository list.", + "github", + ) + state_dict = json.loads(base64.urlsafe_b64decode(state.encode()).decode()) provider = state_dict.get("provider") state_object_id = state_dict.get("object_id") # Validate provider - if not provider or not isinstance(provider, str) or not ConnectorCreator.is_supported(provider): + if not provider or not isinstance(provider, str) or not ( + ConnectorCreator.is_supported(provider) or ConnectorCreator.has_auth(provider) + ): return redirect(build_callback_redirect({ "status": "error", "message": "Invalid provider" @@ -270,16 +320,16 @@ class ConnectorsCallback(Resource): auth = ConnectorCreator.create_auth(provider) token_info = auth.exchange_code_for_tokens(authorization_code) - session_token = str(uuid.uuid4()) - try: if provider == "google_drive": credentials = auth.create_credentials_from_token_info(token_info) - service = auth.build_drive_service(credentials) - user_info = service.about().get(fields="user").execute() + drive_service = auth.build_drive_service(credentials) + user_info = drive_service.about().get(fields="user").execute() user_email = user_info.get('user', {}).get('emailAddress', 'Connected User') else: - user_email = token_info.get('user_info', {}).get('email', 'Connected User') + # GitHub names the account by its login, the others by email. + user_info = token_info.get('user_info') or {} + user_email = user_info.get('email') or user_info.get('login') or 'Connected User' except Exception as e: current_app.logger.warning(f"Could not get user info: {e}") @@ -289,29 +339,23 @@ class ConnectorsCallback(Resource): # ``object_id`` in the OAuth state is the PG session row # UUID (new flow) or a legacy Mongo ObjectId (pre-cutover - # issued state). Try UUID update first; fall back to - # legacy id path. - patch = { - "session_token": session_token, - "token_info": sanitized_token_info, - "user_email": user_email, - "status": "authorized", - } + # issued state). with db_session() as conn: repo = ConnectorSessionsRepository(conn) - if state_object_id: - value = str(state_object_id) - updated = False - if len(value) == 36 and "-" in value: - updated = repo.update(value, patch) - if not updated: - repo.update_by_legacy_id(value, patch) + value = str(state_object_id or "") + state_row = repo.get(value) or (repo.get_by_legacy_id(value) if value else None) + if state_row is None or state_row.get("provider") != provider: + raise ValueError("OAuth state names no pending connection") + connection = service.complete_oauth( + conn, state_row, provider, sanitized_token_info, user_email, + ) # Render instead of redirecting so the session token never # lands in a URL (browser history, access logs, Referer). return _render_callback_page( "success", "Authentication successful", provider, - session_token=session_token, user_email=user_email, + session_token=connection.get("session_token") or "", user_email=user_email, + connection_id=str(connection["id"]), ) except Exception as e: @@ -334,7 +378,8 @@ class ConnectorsCallback(Resource): class ConnectorFiles(Resource): @api.expect(api.model("ConnectorFilesModel", { "provider": fields.String(required=True), - "session_token": fields.String(required=True), + "connection_id": fields.String(required=False), + "session_token": fields.String(required=False, description="Legacy; use connection_id"), "folder_id": fields.String(required=False), "limit": fields.Integer(required=False), "page_token": fields.String(required=False), @@ -345,26 +390,29 @@ class ConnectorFiles(Resource): try: data = request.get_json() provider = data.get('provider') - session_token = data.get('session_token') limit = data.get('limit', 10) - if not provider or not session_token: - return make_response(jsonify({"success": False, "error": "provider and session_token are required"}), 400) + if not provider or not (data.get('connection_id') or data.get('session_token')): + return make_response( + jsonify({"success": False, "error": "provider and connection_id are required"}), 400, + ) decoded_token = request.decoded_token if not decoded_token: return make_response(jsonify({"success": False, "error": "Unauthorized"}), 401) user = decoded_token.get('sub') - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token( - session_token, - ) - if not owns_connector_session(session, user, provider): + session = service.resolve_request_connection(user, provider, data) + if session is None: return make_response(jsonify({"success": False, "error": "Invalid or unauthorized session"}), 401) - loader = ConnectorCreator.create_connector(provider, session_token) + try: + loader = ConnectorCreator.create_connector(provider, connection_id=str(session["id"])) + except service.ConnectionUnavailable: + return make_response( + jsonify({"success": False, "error": "Reconnect to continue", "reconnect": True}), 401, + ) - generic_keys = {'provider', 'session_token'} + generic_keys = {'provider', 'session_token', 'connection_id'} input_config = { k: v for k, v in data.items() if k not in generic_keys } @@ -409,65 +457,49 @@ class ConnectorFiles(Resource): @connectors_ns.route("/api/connectors/validate-session") class ConnectorValidateSession(Resource): - @api.expect(api.model("ConnectorValidateSessionModel", {"provider": fields.String(required=True), "session_token": fields.String(required=True)})) - @api.doc(description="Validate connector session token and return user info and access token") + @api.expect(api.model("ConnectorValidateSessionModel", { + "provider": fields.String(required=True), + "connection_id": fields.String(required=False), + "session_token": fields.String(required=False, description="Legacy; use connection_id"), + })) + @api.doc(description="Validate a connection and return the account and a short-lived access token") def post(self): try: - data = request.get_json() + data = request.get_json() or {} provider = data.get('provider') - session_token = data.get('session_token') - if not provider or not session_token: - return make_response(jsonify({"success": False, "error": "provider and session_token are required"}), 400) + if not provider or not (data.get('connection_id') or data.get('session_token')): + return make_response( + jsonify({"success": False, "error": "provider and connection_id are required"}), 400, + ) decoded_token = request.decoded_token if not decoded_token: return make_response(jsonify({"success": False, "error": "Unauthorized"}), 401) user = decoded_token.get('sub') - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token( - session_token, - ) - if not owns_connector_session(session, user, provider) or not session.get("token_info"): + session = service.resolve_request_connection(user, provider, data) + if session is None: return make_response(jsonify({"success": False, "error": "Invalid or expired session"}), 401) - - token_info = session["token_info"] - auth = ConnectorCreator.create_auth(provider) - is_expired = auth.is_token_expired(token_info) - - if is_expired and token_info.get('refresh_token'): - try: - refreshed_token_info = auth.refresh_access_token(token_info.get('refresh_token')) - sanitized_token_info = auth.sanitize_token_info(refreshed_token_info) - with db_session() as conn: - repo = ConnectorSessionsRepository(conn) - row = repo.get_by_session_token(session_token) - if row: - repo.update(str(row["id"]), {"token_info": sanitized_token_info}) - token_info = sanitized_token_info - is_expired = False - except Exception as refresh_error: - current_app.logger.error(f"Failed to refresh token: {refresh_error}") - - if is_expired: + try: + token = service.picker_token(str(session["id"])) + except service.ConnectionUnavailable: return make_response(jsonify({ "success": False, "expired": True, "error": "Session token has expired. Please reconnect." }), 401) + except service.TransientConnectionError: + return make_response( + jsonify({"success": False, "error": "The provider is not responding. Try again."}), 503, + ) - _base_fields = {"access_token", "refresh_token", "token_uri", "expiry"} - provider_extras = {k: v for k, v in token_info.items() if k not in _base_fields} - - response_data = { + return make_response(jsonify({ "success": True, "expired": False, - "user_email": session.get('user_email', 'Connected User'), - "access_token": token_info.get('access_token'), - **provider_extras, - } - - return make_response(jsonify(response_data), 200) + "connection_id": str(session["id"]), + "user_email": session.get('account_label') or session.get('user_email') or 'Connected User', + **token, + }), 200) except Exception as e: current_app.logger.error(f"Error validating connector session: {e}", exc_info=True) return make_response(jsonify({"success": False, "error": "Failed to validate session"}), 500) @@ -484,15 +516,13 @@ class ConnectorDisconnect(Resource): try: data = request.get_json() provider = data.get('provider') - session_token = data.get('session_token') if not provider: return make_response(jsonify({"success": False, "error": "provider is required"}), 400) - if session_token: + session = service.resolve_request_connection(decoded_token.get('sub'), provider, data) + if session is not None: with db_session() as conn: - ConnectorSessionsRepository(conn).delete_by_session_token( - session_token, decoded_token.get('sub'), - ) + service.disconnect(conn, session) return make_response(jsonify({"success": True}), 200) except Exception as e: @@ -500,38 +530,51 @@ class ConnectorDisconnect(Resource): return make_response(jsonify({"success": False, "error": "Failed to disconnect session"}), 500) -def _owner_connector_session(conn, owner_id: str, provider: str) -> Optional[dict]: - """The owner's usable connector session for ``provider``, or None. +def _owner_connector_session( + conn, owner_id: str, provider: str, connection_id: Optional[str] = None, +) -> Optional[dict]: + """The owner's usable connection for ``provider``, or None. Used when a team editor syncs a shared connector source: the sync runs - with the owner's account. A session with no token, no stored credentials, - or an expired access token that can't be refreshed counts as missing. + with the owner's account, the source's own connection when it has one, + else the owner's first connection for the provider. A connection that is + not connected (signed out, flagged for reconnect, holding no + credentials), cannot be decrypted, or has an expired access token that + can't be refreshed counts as missing. Args: conn: Open database connection. owner_id: The source owner's ``sub``. provider: The source's connector provider. + connection_id: The source's own connection, if it names one. Returns: - Optional[dict]: The session row, or None when the owner must reconnect. + Optional[dict]: The connection row, or None when the owner must reconnect. """ - candidates = [ - s for s in ConnectorSessionsRepository(conn).list_for_user(owner_id) - if owns_connector_session(s, owner_id, provider) - and s.get("session_token") and s.get("token_info") - ] - if not candidates: - return None - session = candidates[0] - token_info = session["token_info"] - if not token_info.get("refresh_token"): + from docsgpt.security.encryption import CredentialDecryptionError + + repo = ConnectorSessionsRepository(conn) + if connection_id: + row = repo.get_for_user(str(connection_id), owner_id) + candidates = [row] if owns_connector_session(row, owner_id, provider) else [] + else: + candidates = [s for s in repo.list_for_user(owner_id) if owns_connector_session(s, owner_id, provider)] + for session in candidates: + if service.normalize_status(session) != service.STATUS_CONNECTED: + continue try: - if ConnectorCreator.create_auth(provider).is_token_expired(token_info): - return None - except Exception: - # Providers without an expiry check leave the verdict to the sync. - pass - return session + token_info = service.read_secrets(session).get("token_info") or {} + except CredentialDecryptionError: + continue + if isinstance(token_info, dict) and token_info and not token_info.get("refresh_token"): + try: + if ConnectorCreator.create_auth(provider).is_token_expired(token_info): + continue + except Exception: + # Providers without an expiry check leave the verdict to the sync. + pass + return session + return None @connectors_ns.route("/api/connectors/sync") @@ -541,11 +584,12 @@ class ConnectorSync(Resource): "ConnectorSyncModel", { "source_id": fields.String(required=True, description="Source ID to sync"), - "session_token": fields.String( + "connection_id": fields.String( required=False, - description="The owner's connector session token (ignored for team editors, " - "whose sync uses the owner's session)", - ) + description="Connection to sync with; defaults to the source's own (ignored for " + "team editors, whose sync uses the owner's connection)", + ), + "session_token": fields.String(required=False, description="Legacy; use connection_id") }, ) ) @@ -558,13 +602,12 @@ class ConnectorSync(Resource): try: data = request.get_json() or {} source_id = data.get('source_id') - session_token = data.get('session_token') if not source_id: return make_response( jsonify({ "success": False, - "error": "source_id and session_token are required" + "error": "source_id is required" }), 400 ) @@ -582,14 +625,6 @@ class ConnectorSync(Resource): ) owner_id = ra.owner_id is_owner = ra.access == "owner" - if is_owner and not session_token: - return make_response( - jsonify({ - "success": False, - "error": "source_id and session_token are required" - }), - 400 - ) remote_data = source.get('remote_data') or {} if isinstance(remote_data, str): @@ -609,17 +644,27 @@ class ConnectorSync(Resource): 400 ) + source_connection = str(source['connection_id']) if source.get('connection_id') else None if is_owner: - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token(session_token) - if not owns_connector_session(session, user_id, source_type): + lookup = dict(data) + if not (lookup.get('connection_id') or lookup.get('session_token')): + if not source_connection: + return make_response( + jsonify({"success": False, "error": "connection_id is required"}), + 400, + ) + lookup['connection_id'] = source_connection + session = service.resolve_request_connection(user_id, source_type, lookup) + if session is None: return make_response( jsonify({"success": False, "error": "Invalid or unauthorized session"}), 401, ) else: + # A grantee can't name a connection: theirs would read the + # source's files with another account. with db_readonly() as conn: - session = _owner_connector_session(conn, owner_id, source_type) + session = _owner_connector_session(conn, owner_id, source_type, source_connection) if session is None: message = ( "The owner needs to reconnect this source's account " @@ -629,7 +674,6 @@ class ConnectorSync(Resource): jsonify({"success": False, "error": message, "message": message}), 409, ) - session_token = session["session_token"] # Extract configuration from remote_data file_ids = remote_data.get('file_ids', []) @@ -641,7 +685,7 @@ class ConnectorSync(Resource): job_name=source.get('name'), user=owner_id, source_type=source_type, - session_token=session_token, + connection_id=str(session["id"]), file_ids=file_ids, folder_ids=folder_ids, recursive=recursive, diff --git a/docsgpt/api/pat/rules.py b/docsgpt/api/pat/rules.py index 80223f81..afb2b0bd 100644 --- a/docsgpt/api/pat/rules.py +++ b/docsgpt/api/pat/rules.py @@ -224,6 +224,7 @@ RULES: dict[tuple[str, str], Rule] = { ("/api/get_chunks", "GET"): _rule("sources:read", (QUERY, "id")), ("/api/sources//wiki/pages", "GET"): _rule("sources:read", (VIEW, "source_id")), ("/api/sources//wiki/page", "GET"): _rule("sources:read", (VIEW, "source_id")), + ("/api/sources//wiki/settings", "GET"): _rule("sources:read", (VIEW, "source_id")), ("/api/sources//graph", "GET"): _rule("sources:read", (VIEW, "source_id")), ("/api/sources//graph/node/", "GET"): _rule( "sources:read", (VIEW, "source_id") @@ -357,12 +358,15 @@ DENIED: dict[str, tuple[str, ...]] = { "/api/teams//grants": ("POST", "DELETE"), "/api/teams//transfer_owner": ("*",), "/api/resource_settings": ("PUT",), + # Who may edit a wiki from outside the app is the owner's call in a session. + "/api/sources//wiki/settings": ("PUT",), "/swagger.json": ("*",), } DENIED_PREFIXES = ( "/api/admin/", "/api/auth/oidc/", "/api/connectors/", + "/api/connections", "/api/devices", "/scim/", "/static/", diff --git a/docsgpt/api/user/agents/portability.py b/docsgpt/api/user/agents/portability.py index 289f2512..39cd778b 100644 --- a/docsgpt/api/user/agents/portability.py +++ b/docsgpt/api/user/agents/portability.py @@ -38,6 +38,7 @@ from docsgpt.agents.default_tools import ( from docsgpt.api import api from docsgpt.api.pat.rules import allowed_ids from docsgpt.api.user.resource_access import AccessDenied, require +from docsgpt.connectors.resolve import carry_removed_connection from docsgpt.core.model_utils import validate_model_id from docsgpt.core.url_validation import SSRFError, validate_url from docsgpt.security.safe_url import UnsafeUserUrlError, validate_user_base_url @@ -1138,7 +1139,8 @@ def _create_tool_from_spec(conn, user: str, tool: dict, secrets: dict, warnings: warnings.append(f"Tool type '{tool_type}' not available on this instance; skipped") return None config_requirements = inst.get_config_requirements() or {} - config = dict(tool.get("config") or {}) + # An imported tool starts with no note that a connection was removed. + config = carry_removed_connection(dict(tool.get("config") or {}), None) config.update(secrets or {}) if tool_type == "api_tool": label = tool.get("display_name") or tool.get("name") or tool_type @@ -1491,18 +1493,24 @@ def _apply_workflow( if existing_id: existing = wf_repo.get(str(existing_id), user) if existing is not None: + from docsgpt.api.user.resource_access import prune_sponsors + from docsgpt.api.user.workflows.routes import _node_refs + pg_workflow_id = str(existing["id"]) next_version = get_workflow_graph_version(existing) + 1 _write_graph(conn, pg_workflow_id, next_version, nodes_data, edges_data) - wf_repo.update( - pg_workflow_id, - user, - { - "name": name, - "description": description, - "current_graph_version": next_version, - }, - ) + workflow_fields = { + "name": name, + "description": description, + "current_graph_version": next_version, + } + # Sponsors of node resources the file dropped go too, so a stale + # record can't vouch for the resource if someone adds it back. + sponsors = existing.get("resource_sponsors") or {} + pruned = prune_sponsors(sponsors, _node_refs(nodes_data)) + if pruned != sponsors: + workflow_fields["resource_sponsors"] = pruned + wf_repo.update(pg_workflow_id, user, workflow_fields) WorkflowNodesRepository(conn).delete_other_versions(pg_workflow_id, next_version) WorkflowEdgesRepository(conn).delete_other_versions(pg_workflow_id, next_version) return pg_workflow_id @@ -1512,6 +1520,25 @@ def _apply_workflow( return str(created["id"]) +def _prune_agent_sponsors(agents_repo: AgentsRepository, agent_id: str, user: str) -> None: + """Drop sponsor records of resources the imported agent no longer references. + + Args: + agents_repo: Repository on the import's connection. + agent_id: The updated agent. + user: Its owner. + """ + from docsgpt.api.user.resource_access import agent_refs, prune_sponsors + + row = agents_repo.get(agent_id, user) + if not row: + return + sponsors = row.get("resource_sponsors") or {} + pruned = prune_sponsors(sponsors, agent_refs(row)) + if pruned != sponsors: + agents_repo.update(agent_id, user, {"resource_sponsors": pruned}) + + def apply_import(conn, user: str, doc: dict, resolution: Optional[dict] = None) -> dict: """Create or update an agent from a parsed YAML doc. @@ -1648,6 +1675,7 @@ def apply_import(conn, user: str, doc: dict, resolution: Optional[dict] = None) # Only a brand-new agent (the create path below) starts as a draft. fields = {**authoritative, **optional, "name": spec.get("name")} if agents_repo.update(str(target["agent_id"]), user, fields): + _prune_agent_sponsors(agents_repo, str(target["agent_id"]), user) if orphaned_workflow_id and not agents_repo.count_by_workflow( orphaned_workflow_id, user ): diff --git a/docsgpt/api/user/agents/routes.py b/docsgpt/api/user/agents/routes.py index 2715b842..619362d0 100644 --- a/docsgpt/api/user/agents/routes.py +++ b/docsgpt/api/user/agents/routes.py @@ -29,13 +29,19 @@ from docsgpt.storage.db.base_repository import canonical_uuid, looks_like_uuid from docsgpt.api.user.resource_access import ( AccessDenied, agent_refs, + best_effort, + cached_resolves, delete_settings, + parse_confirmations, payload_for, + plan_sponsors, require, resolve, + resource_states, settings_many, + sponsor_audience, sponsor_details, - sponsors_after_save, + sponsor_refusal, ) from docsgpt.api.user.team_sharing import ( can_access, @@ -483,6 +489,27 @@ def _build_create_kwargs(data: dict, *, image_url: str, agent_type: str) -> dict return kwargs +def keep_owner_only_config(config: dict, existing_agent: dict, is_team_editor: bool) -> dict: + """Keep the parts of an agent's config only its owner may change. + + ``api_write_allowlist`` lets anyone with the agent's API key act on the + owner's connected accounts, so a team editor's update keeps the stored + value whatever it sends. + + Args: + config: The normalized config from the request. + existing_agent: The agent row being updated. + is_team_editor: The caller edits through a team share, not as owner. + + Returns: + The config to store. + """ + if not is_team_editor: + return config + stored = AgentConfig.parse(existing_agent.get("config")).api_write_allowlist + return {**config, "api_write_allowlist": stored} + + def normalize_agent_config(raw): """Validate an inbound ``config`` payload, returning the normalized dict. @@ -537,15 +564,29 @@ class GetAgent(Resource): user = decoded_token["sub"] agent = None sponsored: list = [] - with db_readonly() as conn: + states: list = [] + audience = None + with db_readonly() as conn, cached_resolves(): # Anyone who can see the agent reads it (a viewer needs it to # chat); what they get back is trimmed by their actions. ra = resolve(conn, "agent", agent_id, user) if ra is not None: agent = AgentsRepository(conn).get_by_id(ra.resource_id) - # Edit-page detail: who vouches for resources the owner can't use. - if agent and ra.can("view"): - sponsored = sponsor_details(conn, "agent", agent) + # Edit-page detail: who vouches for resources the owner can't + # use, and which attached resources stopped running and why, + # with their names. Only for people who may edit the agent: + # the names can be an editor's private resources. Run state + # never fails the read. + if agent and ra.can("edit"): + sponsored = sponsor_details(conn, "agent", agent, viewer=user) + states = best_effort( + conn, "the agent's resource states", + lambda: resource_states(conn, "agent", agent, agent_refs(agent), user), [], + ) + audience = best_effort( + conn, "the agent's audience", + lambda: sponsor_audience(conn, "agent", agent, states, sponsored), None, + ) if not agent: return {"status": "Not found"}, 404 is_owner = ra.access == "owner" @@ -557,6 +598,9 @@ class GetAgent(Resource): access=ra.payload(), ) data["resource_sponsors"] = sponsored + data["resource_states"] = states + if audience is not None: + data["sponsor_audience"] = audience return make_response(jsonify(data), 200) except Exception as e: current_app.logger.error(f"Agent fetch error: {e}", exc_info=True) @@ -1035,14 +1079,11 @@ class UpdateAgent(Resource): ) pg_agent_id = str(existing_agent["id"]) existing_image = existing_agent.get("image", "") or "" - image_url, image_error = handle_image_upload( - request, - existing_image, - existing_agent.get("user_id") or user, - storage, - ) - if image_error: - return image_error + # The image is stored only once the save is known to go + # ahead, so a refused save (a sponsor confirmation round + # trip, a validation error) leaves no orphaned file. + image_file = request.files.get("image") + has_new_image = bool(image_file and image_file.filename) update_fields: dict = {} allowed_fields = [ @@ -1153,7 +1194,9 @@ class UpdateAgent(Resource): exc, ) return _reject(INVALID_CONFIG_MESSAGE, user, field) - update_fields["config"] = normalized_config or {} + update_fields["config"] = keep_owner_only_config( + normalized_config or {}, existing_agent, is_team_editor, + ) elif field == "limited_token_mode": raw_value = data.get("limited_token_mode", False) bool_value = ( @@ -1288,9 +1331,7 @@ class UpdateAgent(Resource): f"Field '{field}' cannot be empty", user, field ) update_fields[field] = value - if image_url and image_url != existing_image: - update_fields["image"] = image_url - if not update_fields: + if not update_fields and not has_new_image: return _reject("No valid update data provided", user) newly_generated_key = None @@ -1406,18 +1447,6 @@ class UpdateAgent(Resource): 403, ) - # A resource the owner can't use runs as the editor who - # attached it (its sponsor); record who that is. - after_save = dict(existing_agent) - for ref_field in ("source_id", "extra_source_ids", "prompt_id", "tools"): - if ref_field in update_fields: - after_save[ref_field] = update_fields[ref_field] - sponsors = sponsors_after_save( - conn, "agent", existing_agent, owner_id, user, agent_refs(after_save) - ) - if sponsors != (existing_agent.get("resource_sponsors") or {}): - update_fields["resource_sponsors"] = sponsors - # Guardrails and the pooled quota are policy: an unchanged # value re-sent by a full-form save is fine, a change needs # ``edit_policy``. @@ -1426,6 +1455,46 @@ class UpdateAgent(Resource): AccessDenied(403, "Your access doesn't allow changing guardrails or limits") ) + # A resource the owner can't use runs as the editor who + # attached it (its sponsor). Only someone who owns or edits + # it may sponsor it, and only after confirming: the save is + # refused (409) until ``confirm_sponsor`` lists every + # resource it would newly sponsor. + after_save = dict(existing_agent) + for ref_field in ("source_id", "extra_source_ids", "prompt_id", "tools"): + if ref_field in update_fields: + after_save[ref_field] = update_fields[ref_field] + plan = plan_sponsors( + conn, + "agent", + existing_agent, + owner_id, + user, + agent_refs(after_save), + previous_refs=agent_refs(existing_agent), + confirmed=parse_confirmations(data.get("confirm_sponsor")), + ) + refusal = sponsor_refusal( + conn, "agent", existing_agent, plan, api_key=bool(update_fields.get("key")) or None + ) + if refusal is not None: + body, status = refusal + return make_response(jsonify(body), status) + if plan.sponsors != (existing_agent.get("resource_sponsors") or {}): + update_fields["resource_sponsors"] = plan.sponsors + + if has_new_image: + image_url, image_error = handle_image_upload( + request, + existing_image, + existing_agent.get("user_id") or user, + storage, + ) + if image_error: + return image_error + if image_url and image_url != existing_image: + update_fields["image"] = image_url + # Apply update. Owner writes use the dual-key guard; team-editor # writes go by-id (already authorized) with an optimistic-lock # check when the client supplies the row's expected updated_at. diff --git a/docsgpt/api/user/resource_access.py b/docsgpt/api/user/resource_access.py index 7d3f5627..952fec8f 100644 --- a/docsgpt/api/user/resource_access.py +++ b/docsgpt/api/user/resource_access.py @@ -22,11 +22,14 @@ from __future__ import annotations import json import logging +from contextlib import contextmanager +from contextvars import ContextVar from dataclasses import dataclass, field from typing import Iterable, Optional from sqlalchemy import Connection, text +from docsgpt.connectors.permissions import holds_owner_credentials, owner_credential_writes from docsgpt.storage.db.base_repository import canonical_uuid, looks_like_uuid from docsgpt.storage.db.repositories.agents import AgentsRepository from docsgpt.storage.db.repositories.prompts import PromptsRepository @@ -295,10 +298,41 @@ def payload_for(resource_type: str, access: Optional[str], settings: Optional[di } +# Per-read memo of :func:`resolve`, on only inside :func:`cached_resolves`. +_RESOLVE_CACHE: ContextVar[Optional[dict]] = ContextVar("resource_access_resolve_cache", default=None) + + +@contextmanager +def cached_resolves(): + """Remember :func:`resolve` answers for the rest of one read. + + A page read asks about the same resources several times (sponsor details, + run state, names); inside this block each ``(type, id, user)`` is looked + up once. Only for reads: a write must see a grant change at once. + """ + token = _RESOLVE_CACHE.set({}) + try: + yield + finally: + _RESOLVE_CACHE.reset(token) + + def resolve( conn: Connection, resource_type: str, resource_id: str, user_id: str ) -> Optional[ResourceAccess]: """The caller's access to a resource, or None when they can't see it.""" + cache = _RESOLVE_CACHE.get() + if cache is None: + return _resolve_uncached(conn, resource_type, resource_id, user_id) + key = (resource_type, str(resource_id or "").lower(), user_id) + if key not in cache: + cache[key] = _resolve_uncached(conn, resource_type, resource_id, user_id) + return cache[key] + + +def _resolve_uncached( + conn: Connection, resource_type: str, resource_id: str, user_id: str +) -> Optional[ResourceAccess]: repo_cls = _REPO_FOR_TYPE.get(resource_type) if repo_cls is None or not resource_id or not user_id: return None @@ -348,15 +382,34 @@ def require( # # An agent (or workflow) runs as its owner, so a source, prompt or tool it # references is authorized against the owner. When a team editor attaches one -# the owner can't use, the editor becomes its *sponsor*: the holder row's +# the owner can't use, the editor may become its *sponsor*: the holder row's # ``resource_sponsors`` maps ``":"`` to the editor's id, and at run # time the resource is authorized as the sponsor while they can still edit the -# holder and still use the resource. A tool still runs with its own row's -# credentials (the tool owner's), whoever the principal is. +# holder and still may sponsor the resource. A tool still runs with its own +# row's credentials (the tool owner's), whoever the principal is. +# +# Sponsoring extends a resource to everyone who uses the holder, so it takes +# more than being able to use it: the sponsor must own the resource or have +# ``edit`` on it (``can_sponsor_ref``). It is never implied: a save that would +# make the caller a new sponsor needs their explicit confirmation +# (``plan_sponsors``), and when a sponsor loses access the resource stops +# rather than passing to whoever saves next. # Action a principal needs on a referenced resource for a holder to run it. REF_USE_ACTION = {"source": "use", "prompt": "use", "tool": "use_in_own"} +# Action a user needs on a resource to sponsor it (owners have every action). +SPONSOR_ACTION = "edit" + +# Why a recorded sponsorship doesn't run (``sponsor_details`` ``reason``). +REASON_CANNOT_EDIT_HOLDER = "sponsor_cannot_edit_agent" +REASON_CANNOT_EDIT_RESOURCE = "sponsor_cannot_edit_resource" + +# Error codes of a save the sponsor rules refuse. +CODE_CONFIRMATION_REQUIRED = "sponsor_confirmation_required" +CODE_NOT_ALLOWED = "sponsor_not_allowed" +CODE_UNEXPECTED_CONFIRMATION = "sponsor_confirmation_unexpected" + def sponsor_key(resource_type: str, resource_id: str) -> str: """The ``resource_sponsors`` key for one referenced resource.""" @@ -382,6 +435,28 @@ def can_use_ref(conn: Connection, resource_type: str, resource_id: str, user_id: return ra is not None and ra.can(REF_USE_ACTION[resource_type]) +def can_sponsor_ref(conn: Connection, resource_type: str, resource_id: str, user_id: Optional[str]) -> bool: + """Whether ``user_id`` may extend ``resource_id`` to someone else's agent. + + Using a resource is not enough to hand it to another agent's audience: + the sponsor must own it or reach it with ``edit`` through a team grant + (any team). + + Args: + conn: Open database connection. + resource_type: ``source``, ``prompt`` or ``tool``. + resource_id: The referenced id. + user_id: The would-be sponsor. + + Returns: + True when the user owns the resource or may edit it. + """ + if not user_id or not resource_id: + return False + ra = resolve(conn, resource_type, str(resource_id), user_id) + return ra is not None and ra.can(SPONSOR_ACTION) and ra.can(REF_USE_ACTION[resource_type]) + + def _holder_editable_by(conn: Connection, holder_type: str, holder: dict, user_id: str) -> bool: """Whether ``user_id`` may still edit the agent or workflow ``holder``. @@ -392,11 +467,7 @@ def _holder_editable_by(conn: Connection, holder_type: str, holder: dict, user_i ra = resolve(conn, "agent", str(holder["id"]), user_id) return ra is not None and ra.can("edit") if holder_type == "workflow": - agent_ids = conn.execute( - text("SELECT id FROM agents WHERE workflow_id = CAST(:wid AS uuid) AND user_id = :owner"), - {"wid": str(holder["id"]), "owner": holder.get("user_id")}, - ).scalars().all() - for agent_id in agent_ids: + for agent_id in _workflow_agent_ids(conn, holder): ra = resolve(conn, "agent", str(agent_id), user_id) if ra is not None and ra.can("edit"): return True @@ -404,6 +475,51 @@ def _holder_editable_by(conn: Connection, holder_type: str, holder: dict, user_i raise ValueError(f"Unknown sponsor holder type: {holder_type}") +def _workflow_agent_ids(conn: Connection, workflow: dict) -> list: + """The ids of the workflow owner's agents that run ``workflow``.""" + return conn.execute( + text("SELECT id FROM agents WHERE workflow_id = CAST(:wid AS uuid) AND user_id = :owner"), + {"wid": str(workflow["id"]), "owner": workflow.get("user_id")}, + ).scalars().all() + + +def sponsor_state( + conn: Connection, holder_type: str, holder: Optional[dict], resource_type: str, resource_id: str +) -> tuple[Optional[str], Optional[str]]: + """The recorded sponsor of one reference and why it doesn't run, if it doesn't. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row (needs ``id``, ``user_id``, ``resource_sponsors``). + resource_type: ``source``, ``prompt`` or ``tool``. + resource_id: The referenced id. + + Returns: + ``(sponsor, reason)``: ``sponsor`` is the recorded user or None; + ``reason`` is None while the sponsorship runs, else + :data:`REASON_CANNOT_EDIT_HOLDER` or :data:`REASON_CANNOT_EDIT_RESOURCE`. + With no usable record both are None. + """ + if not holder or not resource_id: + return None, None + recorded = holder.get("resource_sponsors") or {} + sponsor = recorded.get(sponsor_key(resource_type, str(resource_id))) or recorded.get( + sponsor_key(resource_type, str(resource_id).lower()) + ) + if not sponsor or sponsor == holder.get("user_id"): + return None, None + try: + if not _holder_editable_by(conn, holder_type, holder, sponsor): + return sponsor, REASON_CANNOT_EDIT_HOLDER + if not can_sponsor_ref(conn, resource_type, str(resource_id), sponsor): + return sponsor, REASON_CANNOT_EDIT_RESOURCE + except Exception: + logger.exception("Sponsor check failed for %s %s", resource_type, resource_id) + return sponsor, REASON_CANNOT_EDIT_RESOURCE + return sponsor, None + + def active_sponsor( conn: Connection, holder_type: str, holder: Optional[dict], resource_type: str, resource_id: str ) -> Optional[str]: @@ -418,20 +534,10 @@ def active_sponsor( Returns: The sponsor's id when one is recorded, still edits the holder and - can still use the resource; else None. + still may sponsor the resource (owns or edits it); else None. """ - if not holder or not resource_id: - return None - sponsor = (holder.get("resource_sponsors") or {}).get(sponsor_key(resource_type, str(resource_id))) - if not sponsor or sponsor == holder.get("user_id"): - return None - try: - if not _holder_editable_by(conn, holder_type, holder, sponsor): - return None - return sponsor if can_use_ref(conn, resource_type, str(resource_id), sponsor) else None - except Exception: - logger.exception("Sponsor check failed for %s %s", resource_type, resource_id) - return None + sponsor, reason = sponsor_state(conn, holder_type, holder, resource_type, resource_id) + return sponsor if sponsor and reason is None else None def ref_principal( @@ -441,12 +547,148 @@ def ref_principal( The owner when they may use it (the default), else a live sponsor. """ + return ref_access(conn, holder_type, holder, resource_type, resource_id).principal + + +# --- Run state of attached resources ---------------------------------------- +# +# One check decides whether an attached resource runs, for the run and for the +# edit page alike (``ref_access``, ``resolve_holder_tool``), so the page never +# says a resource runs when the run drops it, or the other way round. + +# Why an attached resource doesn't run (``resource_states`` ``reason``), next +# to the sponsor reasons above. +REASON_DELETED = "deleted" +REASON_OWNER_LOST_ACCESS = "owner_lost_access" +REASON_CONNECTION_NEEDS_RECONNECT = "connection_needs_reconnect" +REASON_CONNECTION_REMOVED = "connection_removed" +REASON_CONNECTOR_DISABLED = "connector_disabled" + +_REF_TABLES = {"source": "sources", "prompt": "prompts", "tool": "user_tools"} + + +@dataclass(frozen=True) +class RefAccess: + """Who a holder runs one referenced resource as, or why it doesn't run. + + Attributes: + principal: The user it is authorized as (the holder's owner or a live + sponsor); None when it doesn't run. + reason: None while it runs; else :data:`REASON_DELETED`, + :data:`REASON_OWNER_LOST_ACCESS` or a sponsor reason. + sponsor: The recorded sponsor, running or not. + """ + + principal: Optional[str] + reason: Optional[str] = None + sponsor: Optional[str] = None + + +def _ref_exists(conn: Connection, resource_type: str, resource_id: str) -> bool: + """Whether a row with this id exists, whoever owns it.""" + table = _REF_TABLES.get(resource_type) + if table is None or not looks_like_uuid(str(resource_id)): + return False + return conn.execute( + text(f"SELECT 1 FROM {table} WHERE id = CAST(:id AS uuid)"), {"id": str(resource_id)} + ).first() is not None + + +def ref_access( + conn: Connection, holder_type: str, holder: Optional[dict], resource_type: str, resource_id: str +) -> RefAccess: + """Whether and as whom a holder runs one referenced resource. + + The run and the edit page both ask this. The owner runs it when they may + use it, else a live sponsor does; otherwise it is stopped, because the row + is gone, a recorded sponsor no longer qualifies, or the owner lost access. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row (needs ``user_id``; ``id`` and + ``resource_sponsors`` for sponsors). + resource_type: ``source``, ``prompt`` or ``tool``. + resource_id: The referenced id. + + Returns: + RefAccess: The principal, or the reason it doesn't run. + """ if not holder: - return None + return RefAccess(None, REASON_OWNER_LOST_ACCESS) + rid = str(resource_id) owner = holder.get("user_id") - if can_use_ref(conn, resource_type, str(resource_id), owner): - return owner - return active_sponsor(conn, holder_type, holder, resource_type, resource_id) + if can_use_ref(conn, resource_type, rid, owner): + return RefAccess(owner) + sponsor, sponsor_reason = sponsor_state(conn, holder_type, holder, resource_type, rid) + if sponsor and sponsor_reason is None: + return RefAccess(sponsor, None, sponsor) + if not _ref_exists(conn, resource_type, rid): + return RefAccess(None, REASON_DELETED, sponsor) + return RefAccess(None, sponsor_reason or REASON_OWNER_LOST_ACCESS, sponsor) + + +def resolve_holder_tool( + conn: Connection, holder_type: str, holder: Optional[dict], tool_id: str, *, tools_repo=None +) -> tuple[Optional[dict], RefAccess]: + """The tool row a holder runs ``tool_id`` with, and its access. + + Builtin and default tool ids resolve to their synthesized rows. A + ``user_tools`` row resolves as the holder's owner, else as its live + sponsor (see :func:`ref_access`); the row is the tool owner's either way. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + tool_id: The referenced tool id. + tools_repo: A ``UserToolsRepository`` on ``conn`` to reuse. + + Returns: + ``(row, access)``: the row, None when it doesn't run, and why. + """ + # Lazy: default_tools imports this module lazily too. + from docsgpt.agents.default_tools import resolve_tool_by_id + + repo = tools_repo or UserToolsRepository(conn) + owner = (holder or {}).get("user_id") + row = resolve_tool_by_id(tool_id, owner, user_tools_repo=repo) + if row is not None: + return row, RefAccess(owner) + access = ref_access(conn, holder_type, holder, "tool", str(tool_id)) + if access.principal: + row = resolve_tool_by_id(tool_id, access.principal, user_tools_repo=repo) + if row is None: + reason = access.reason or REASON_DELETED + return None, RefAccess(None, reason, access.sponsor) + return row, access + + +def log_stopped( + holder_type: str, holder: Optional[dict], resource_type: str, resource_id, reason: Optional[str] +) -> None: + """Log one attached resource a run leaves out, greppable by ``resource_stopped``. + + Args: + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + resource_type: ``source``, ``prompt`` or ``tool``. + resource_id: The referenced id. + reason: Why it doesn't run. + """ + holder_id = str((holder or {}).get("id") or "") + logger.info( + "resource_stopped holder=%s:%s type=%s id=%s reason=%s", + holder_type, holder_id, resource_type, resource_id, reason, + extra={ + "event": "resource_stopped", + "holder_type": holder_type, + "holder_id": holder_id, + "resource_type": resource_type, + "resource_id": str(resource_id), + "reason": reason, + }, + ) def _sponsorable(resource_type: str, resource_id: str) -> bool: @@ -471,21 +713,85 @@ def agent_refs(agent: dict) -> list[tuple[str, str]]: return refs -def sponsors_after_save( +def parse_confirmations(raw) -> set[str]: + """``confirm_sponsor`` from a request as canonical ``":"`` keys. + + Accepts a list, a JSON-encoded list (form posts) or a comma-separated + string. Unknown shapes and malformed entries are dropped, so they can + never confirm anything. + + Args: + raw: The request value, or None. + + Returns: + set: The confirmed keys, ids lowercased like stored refs. + """ + if raw is None or raw == "": + return set() + if isinstance(raw, str): + try: + raw = json.loads(raw) + except (ValueError, TypeError): + raw = raw.split(",") + if isinstance(raw, str): + raw = [raw] + if not isinstance(raw, (list, tuple)): + return set() + out: set[str] = set() + for item in raw: + resource_type, _, resource_id = str(item).strip().partition(":") + if resource_type in REF_USE_ACTION and resource_id: + out.add(sponsor_key(resource_type, canonical_uuid(resource_id.strip()))) + return out + + +@dataclass +class SponsorPlan: + """What a save does to a holder's sponsors, and what stops it. + + Attributes: + sponsors: The ``resource_sponsors`` map to store. + needs_confirmation: ``(type, id)`` refs the save would newly have the + caller sponsor without their confirmation; the save must be + refused with :data:`CODE_CONFIRMATION_REQUIRED`. + not_allowed: Newly attached ``(type, id)`` refs the owner can't use + and the caller may not sponsor; the save must be refused. + unexpected: Confirmed keys this save has no sponsorship for. + """ + + sponsors: dict = field(default_factory=dict) + needs_confirmation: list = field(default_factory=list) + not_allowed: list = field(default_factory=list) + unexpected: list = field(default_factory=list) + + +def plan_sponsors( conn: Connection, holder_type: str, holder: Optional[dict], owner_id: str, caller: str, refs: Iterable[tuple[str, str]], -) -> dict[str, str]: - """The ``resource_sponsors`` map to store after ``caller`` saves ``refs``. + previous_refs: Iterable[tuple[str, str]] = (), + confirmed: Iterable[str] = (), +) -> SponsorPlan: + """Work out the sponsors after ``caller`` saves ``refs``. - Per referenced resource the owner can't use: a recorded sponsor who still - qualifies is kept; otherwise the caller takes it over when they can use it - (a new attachment, or one whose sponsor lost access); otherwise the old - record is kept so the editor can show who added it. Resources the owner - can use, presets and builtin tools need no sponsor. Removed refs drop out. + Per referenced resource the owner can't use: + + * a recorded sponsor who still qualifies is kept; + * the caller becomes the sponsor only when they may sponsor it + (:func:`can_sponsor_ref`) and listed its key in ``confirmed``; + * a newly attached one the caller may sponsor but didn't confirm goes to + ``needs_confirmation``; one they may not sponsor goes to ``not_allowed``; + * one already attached keeps its old record even when that sponsor lost + access: it stays stopped until someone confirms taking it over; + * a newly attached one ignores any record left from before it was + removed, so a stale sponsor never vouches for it again. + + Resources the owner can use, presets and builtin tools need no sponsor. + Removed refs drop out. A confirmed key that names none of the refs the + caller could sponsor lands in ``unexpected``. Args: conn: Open database connection. @@ -494,65 +800,674 @@ def sponsors_after_save( owner_id: The holder's owner. caller: The user saving. refs: Every ``(type, id)`` the holder references after the save. + previous_refs: Every ``(type, id)`` it referenced before the save. + confirmed: ``":"`` keys the caller agreed to sponsor. Returns: - dict: ``":" -> user_id``. + SponsorPlan: The map to store and anything that refuses the save. """ previous = (holder or {}).get("resource_sponsors") or {} - out: dict[str, str] = {} + before = {sponsor_key(t, str(i).lower()) for t, i in previous_refs} + confirmed = set(confirmed) + plan = SponsorPlan() + eligible: set[str] = set() + seen: set[str] = set() for resource_type, resource_id in refs: + resource_id = str(resource_id).lower() key = sponsor_key(resource_type, resource_id) - if key in out or not _sponsorable(resource_type, resource_id): + if key in seen or not _sponsorable(resource_type, resource_id): continue + seen.add(key) if can_use_ref(conn, resource_type, resource_id, owner_id): continue - if active_sponsor(conn, holder_type, holder, resource_type, resource_id): - out[key] = previous[key] - elif caller != owner_id and can_use_ref(conn, resource_type, resource_id, caller): - out[key] = caller - elif previous.get(key): - out[key] = previous[key] + is_new = key not in before + caller_may = caller != owner_id and can_sponsor_ref(conn, resource_type, resource_id, caller) + if caller_may: + eligible.add(key) + # A record only vouches for a resource that stayed attached: one left + # behind by a path that dropped the resource never covers it again. + live = None if is_new else active_sponsor(conn, holder_type, holder, resource_type, resource_id) + if live: + plan.sponsors[key] = live + elif caller_may and key in confirmed: + plan.sponsors[key] = caller + elif is_new and caller != owner_id: + (plan.needs_confirmation if caller_may else plan.not_allowed).append((resource_type, resource_id)) + elif not is_new and previous.get(key): + plan.sponsors[key] = previous[key] + plan.unexpected = sorted(confirmed - eligible) + return plan + + +def prune_sponsors(sponsors: Optional[dict], refs: Iterable[tuple[str, str]]) -> dict: + """``sponsors`` without the keys of resources no longer referenced. + + For paths that rewrite a holder's references without going through + :func:`plan_sponsors` (YAML import, a workflow graph written by import). + + Args: + sponsors: The stored ``resource_sponsors`` map. + refs: Every ``(type, id)`` the holder references now. + + Returns: + dict: The map to store. + """ + keep = {sponsor_key(t, str(i).lower()) for t, i in refs} + return {k: v for k, v in (sponsors or {}).items() if k.lower() in keep} + + +def ref_names(conn: Connection, refs: Iterable[tuple[str, str]]) -> dict[str, str]: + """Display names of referenced resources, looked up by id (owner-agnostic). + + Args: + conn: Open database connection. + refs: ``(type, id)`` pairs. + + Returns: + dict: ``":" -> name`` for the ones found. + """ + queries = { + "source": "SELECT id, name FROM sources WHERE id = ANY(CAST(:ids AS uuid[]))", + "prompt": "SELECT id, name FROM prompts WHERE id = ANY(CAST(:ids AS uuid[]))", + "tool": ( + "SELECT id, COALESCE(NULLIF(custom_name, ''), NULLIF(display_name, ''), name) " + "FROM user_tools WHERE id = ANY(CAST(:ids AS uuid[]))" + ), + } + by_type: dict[str, list[str]] = {} + for resource_type, resource_id in refs: + if resource_type in queries and looks_like_uuid(str(resource_id)): + by_type.setdefault(resource_type, []).append(str(resource_id)) + out: dict[str, str] = {} + for resource_type, ids in by_type.items(): + for rid, name in conn.execute(text(queries[resource_type]), {"ids": ids}).fetchall(): + if name: + out[sponsor_key(resource_type, str(rid))] = name return out -def sponsor_details(conn: Connection, holder_type: str, holder: dict) -> list[dict]: +def holder_audience(conn: Connection, holder_type: str, holder: dict, *, api_key: Optional[bool] = None) -> dict: + """Who reaches a holder's resources: its teams and outside entry points. + + For a workflow, the union over the owner's agents that run it. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + api_key: Override for an agent whose key this save creates. + + Returns: + dict: ``teams`` (names, sorted), and booleans ``api_key`` (API and + widget), ``public_link`` and ``webhook``. + """ + if holder_type == "agent": + agents = [holder] + else: + ids = [str(a) for a in _workflow_agent_ids(conn, holder)] + agents = [a for a in (AgentsRepository(conn).get_by_id(i) for i in ids) if a] + grants_repo = TeamResourceGrantsRepository(conn) + teams: set[str] = set() + for agent in agents: + teams.update(g.get("team_name") for g in grants_repo.list_for_resource("agent", str(agent["id"]))) + has_key = any(a.get("key") for a in agents) + return { + "teams": sorted(t for t in teams if t), + "api_key": bool(has_key if api_key is None else api_key or has_key), + "public_link": any(a.get("shared") and a.get("shared_token") for a in agents), + "webhook": any(a.get("incoming_webhook_token") for a in agents), + } + + +def sponsor_refusal( + conn: Connection, holder_type: str, holder: dict, plan: SponsorPlan, *, api_key: Optional[bool] = None +) -> Optional[tuple[dict, int]]: + """The error body and status for a save ``plan`` refuses, or None. + + In order: 403 :data:`CODE_NOT_ALLOWED` (a new resource the caller may not + sponsor), 400 :data:`CODE_UNEXPECTED_CONFIRMATION`, then 409 + :data:`CODE_CONFIRMATION_REQUIRED` listing what the caller would sponsor + and the holder's audience, so the client can ask and retry with + ``confirm_sponsor``. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row before the save. + plan: The result of :func:`plan_sponsors`. + api_key: Passed to :func:`holder_audience`. + + Returns: + ``(body, status)`` or None when the save may go ahead. + """ + def _resources(pairs: list) -> list[dict]: + names = ref_names(conn, pairs) + return [ + { + "key": sponsor_key(t, i), + "type": t, + "id": i, + "name": names.get(sponsor_key(t, i)), + } + for t, i in pairs + ] + + if plan.not_allowed: + return { + "success": False, + "code": CODE_NOT_ALLOWED, + "message": ( + "You can't add a resource the owner can't use unless you own it or can edit it." + ), + "resources": _resources(plan.not_allowed), + }, 403 + if plan.unexpected: + return { + "success": False, + "code": CODE_UNEXPECTED_CONFIRMATION, + "message": "confirm_sponsor lists resources this save doesn't ask you to sponsor.", + "unexpected": plan.unexpected, + }, 400 + if plan.needs_confirmation: + return { + "success": False, + "code": CODE_CONFIRMATION_REQUIRED, + "message": ( + "These resources would run with your access for everyone who uses this agent. " + "Confirm to add them." + ), + "resources": _resources(plan.needs_confirmation), + "audience": holder_audience(conn, holder_type, holder, api_key=api_key), + }, 409 + return None + + +def sponsor_details( + conn: Connection, holder_type: str, holder: dict, viewer: Optional[str] = None +) -> list[dict]: """The holder's sponsored resources for its edit page. Args: conn: Open database connection. holder_type: ``agent`` or ``workflow``. holder: The holder row. + viewer: The user reading the page; sets ``can_confirm``. Returns: - list: ``{type, id, user_id, label, active}`` per sponsored resource; - ``label`` is the sponsor's email when on file, ``active`` whether it - runs (the sponsor still edits the holder and can use the resource). + list: Per sponsored resource ``{key, type, id, name, user_id, label, + state, reason, active, can_confirm}``. ``user_id`` and ``label`` (the + sponsor's email when on file) are both None for a sponsor the reader + doesn't know (see :func:`people_named_to`); ``state`` is ``active`` + or ``inactive``, and + ``reason`` (None while active) is :data:`REASON_CANNOT_EDIT_HOLDER` + or :data:`REASON_CANNOT_EDIT_RESOURCE`; ``active`` mirrors ``state``. + ``can_confirm`` says whether ``viewer`` may take an inactive one + over by confirming it on their next save. """ sponsors = holder.get("resource_sponsors") or {} if not sponsors: return [] - user_ids = sorted({u for u in sponsors.values() if u}) - labels = dict( + named = {user_id for user_id, _ in people_named_to(conn, viewer, holder, [(u, None) for u in sponsors.values()])} + labels = _user_labels(conn, named) + entries = [] + for key, user_id in sponsors.items(): + resource_type, _, resource_id = key.partition(":") + if resource_type in REF_USE_ACTION and resource_id: + entries.append((key, resource_type, resource_id, user_id)) + names = ref_names(conn, [(t, i) for _, t, i, _ in entries]) + viewer_edits = bool( + viewer + and viewer != holder.get("user_id") + and _holder_editable_by(conn, holder_type, holder, viewer) + ) + out = [] + for key, resource_type, resource_id, user_id in entries: + _, reason = sponsor_state(conn, holder_type, holder, resource_type, resource_id) + active = reason is None + out.append( + { + "key": key, + "type": resource_type, + "id": resource_id, + "name": names.get(key), + **(_person(user_id, labels, user_id in named) or {"user_id": None, "label": None}), + "state": "active" if active else "inactive", + "reason": reason, + "active": active, + "can_confirm": bool( + not active + and viewer_edits + and can_sponsor_ref(conn, resource_type, resource_id, viewer) + ), + } + ) + return out + + +def holder_editable_by(conn: Connection, holder_type: str, holder: dict, user_id: Optional[str]) -> bool: + """Whether ``user_id`` may edit the agent or workflow ``holder``. + + Its owner always may; anyone else needs ``edit`` on the agent (for a + workflow, on one of its owner's agents that run it). + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + user_id: The reader. + + Returns: + bool: Whether they may edit it. + """ + if not user_id or not holder: + return False + if user_id == holder.get("user_id"): + return True + return _holder_editable_by(conn, holder_type, holder, user_id) + + +def _ref_rows(conn: Connection, refs: Iterable[tuple[str, str]]) -> dict[str, dict]: + """``":" -> {name, user_id}`` for referenced rows, owner-agnostic, per type in one query.""" + queries = { + "source": "SELECT id, name, user_id FROM sources WHERE id = ANY(CAST(:ids AS uuid[]))", + "prompt": "SELECT id, name, user_id FROM prompts WHERE id = ANY(CAST(:ids AS uuid[]))", + "tool": ( + "SELECT id, COALESCE(NULLIF(custom_name, ''), NULLIF(display_name, ''), name), user_id " + "FROM user_tools WHERE id = ANY(CAST(:ids AS uuid[]))" + ), + } + by_type: dict[str, list[str]] = {} + for resource_type, resource_id in refs: + if resource_type in queries and looks_like_uuid(str(resource_id)): + by_type.setdefault(resource_type, []).append(str(resource_id)) + out: dict[str, dict] = {} + for resource_type, ids in by_type.items(): + for rid, name, owner in conn.execute(text(queries[resource_type]), {"ids": ids}).fetchall(): + out[sponsor_key(resource_type, str(rid))] = {"name": name, "user_id": owner} + return out + + +def _user_labels(conn: Connection, user_ids: Iterable[Optional[str]]) -> dict[str, str]: + """``user_id -> email`` for the ones with an email on file.""" + ids = sorted({u for u in user_ids if u}) + if not ids: + return {} + return dict( conn.execute( text( "SELECT user_id, email FROM users WHERE user_id = ANY(:ids) " "AND email IS NOT NULL AND email <> ''" ), - {"ids": user_ids}, + {"ids": ids}, ).fetchall() - ) if user_ids else {} - out = [] - for key, user_id in sponsors.items(): - resource_type, _, resource_id = key.partition(":") - if resource_type not in REF_USE_ACTION or not resource_id: + ) + + +# ``resource_states`` ``note`` for a running tool on each caller's own account. +NOTE_PER_USER_ACCOUNT = "per_user_account" + +# ``resource_states`` ``contact_role``: whom to ask, when someone can fix it. +CONTACT_RESOURCE_OWNER = "resource_owner" + + +# What ``resource_states`` says about how a tool runs, for anything that +# isn't a running tool. +_NO_RUN_DETAILS = { + "note": None, + "credential_mode": None, + "account": None, + "owner_credential_writes": [], + "writes_allowed": True, +} + + +def _tool_run_state(conn: Connection, tool: dict, owner: Optional[str], policies_box: list) -> tuple: + """``(reason, connection, run)`` for a tool the holder runs as ``owner``. + + Resolved the way the run resolves it. Only an owner-mode tool's account + is judged (see ``connection_stop_reason``); a member-mode one runs on + each caller's own account. + + ``connection`` is ``(id, connector_key, name)`` for a tool with a + connection or one that lost it, else None. ``run`` is what + ``resource_states`` says about a tool that runs, :data:`_NO_RUN_DETAILS` + for one that doesn't: ``note`` (:data:`NOTE_PER_USER_ACCOUNT` in member + mode), ``credential_mode`` (``owner`` or ``member`` after any mode an + admin forces, for a tool with a connection), ``account`` (the tool's + owner, whose saved credentials or owner-mode connection it acts with), + the ``owner_credential_writes`` outside callers need allowlisted, and + ``writes_allowed`` (False when an admin turned off changes through its + connector). + """ + from docsgpt.connectors import catalog, service + from docsgpt.connectors.resolve import ( + MODE_MEMBER, + REMOVED_CONNECTION_KEY, + connection_stop_reason, + resolve_connection, + ) + + if not tool.get("connection_id"): + reason = connection_stop_reason(tool, None) + if reason is not None: + marker = (tool.get("config") or {}).get(REMOVED_CONNECTION_KEY) + key = marker if isinstance(marker, str) and marker else None + definition = catalog.get_definition(key) if key else catalog.definition_for_tool(tool.get("name") or "") + connection = (None, definition.key if definition else key, definition.name if definition else None) + return reason, connection, dict(_NO_RUN_DETAILS) + mode, writes_allowed, connection = None, True, None + else: + if not policies_box: + policies_box.append(service.load_policies(conn)) + resolved = resolve_connection(tool, owner, conn=conn, policies=policies_box[0]) + connection = (resolved.connection_id, resolved.connector_key, resolved.connector_name) + reason = connection_stop_reason(tool, resolved) + if reason is not None: + return reason, connection, dict(_NO_RUN_DETAILS) + mode, writes_allowed = resolved.mode, resolved.writes_allowed + member = mode == MODE_MEMBER + return None, connection, { + "note": NOTE_PER_USER_ACCOUNT if member else None, + "credential_mode": mode, + "account": tool.get("user_id") if not member and holds_owner_credentials(tool) else None, + "owner_credential_writes": owner_credential_writes(tool) if writes_allowed else [], + "writes_allowed": writes_allowed, + } + + +def _connection_payload(connection: Optional[tuple], reader_owns: bool, can_reconnect: bool) -> Optional[dict]: + """The ``connection`` a reader may see: the id only to reconnect it, its own name only to its owner.""" + if connection is None: + return None + from docsgpt.connectors import catalog + + connection_id, connector_key, name = connection + if not reader_owns: + # The owner's name for the account (a custom MCP server's label, say) + # is theirs; others see the service's own name. + definition = catalog.get_definition(connector_key) if connector_key else None + name = definition.name if definition else None + return {"id": connection_id if can_reconnect else None, "connector_key": connector_key, "name": name} + + +def people_named_to( + conn: Connection, + viewer: Optional[str], + holder: dict, + people: Iterable[tuple[Optional[str], Optional[tuple[str, str]]]], +) -> set: + """Which people an agent or workflow page may name to its reader. + + The one rule for every person the page names (a sponsor, whom a + resource runs as, whom to ask, whose credentials a tool uses): only + someone the reader knows already. That is the reader, the holder's + owner, anyone who sponsored something on the reader's own holder, and + anyone who shares a team with the reader; a person named as the owner + of one resource only when, besides sharing a team, the reader can see + that resource too. Anyone else is left unnamed. + + Args: + conn: Open database connection. + viewer: The user reading the page. + holder: The agent or workflow row (``user_id``, ``resource_sponsors``). + people: ``(user_id, ref)`` pairs; ``ref`` is the ``(type, id)`` the + person is named as the owner of, or None. + + Returns: + set: The ``(user_id, ref)`` pairs that may be named. + """ + pairs = {(user_id, ref) for user_id, ref in people if user_id} + owner = holder.get("user_id") + known = {u for u in (viewer, owner) if u} + if viewer and viewer == owner: + known.update(u for u in (holder.get("resource_sponsors") or {}).values() if u) + rest = sorted({user_id for user_id, _ in pairs if user_id not in known}) + teammates: set = set() + if viewer and rest: + teammates = {row[0] for row in conn.execute( + text( + "SELECT DISTINCT theirs.user_id FROM team_members mine " + "JOIN team_members theirs ON theirs.team_id = mine.team_id " + "WHERE mine.user_id = :viewer AND theirs.user_id = ANY(:ids)" + ), + {"viewer": viewer, "ids": rest}, + ).fetchall()} + named = set() + for user_id, ref in pairs: + if user_id in known or ( + user_id in teammates and (ref is None or resolve(conn, ref[0], ref[1], viewer) is not None) + ): + named.add((user_id, ref)) + return named + + +def _person(user_id: Optional[str], labels: dict, named: bool) -> Optional[dict]: + """``{user_id, label}`` for someone the reader may see named, both None otherwise.""" + if not user_id: + return None + if not named: + return {"user_id": None, "label": None} + return {"user_id": user_id, "label": labels.get(user_id) or user_id} + + +def resource_states( + conn: Connection, + holder_type: str, + holder: dict, + refs: Iterable[tuple[str, str]], + viewer: Optional[str], +) -> list[dict]: + """Whether each resource a holder references runs, for its edit page. + + Uses the run's own checks (:func:`ref_access`, :func:`resolve_holder_tool` + and an owner-mode tool's connection as the run resolves it for the + owner), so a source or prompt shown as stopped is one the run leaves out, + and a tool shown as stopped is left out or can't run until someone acts: + a tool whose owner-mode account needs attention is still offered to the + model, and pauses (or, with nobody to ask, is refused) when called. + Presets and builtin tools always run and are not listed. Only for readers + who may edit the holder: the caller checks that. + + A name is given only for a resource that runs, that the reader can see + themselves, that someone sponsored on the holder, or that is attached to + an agent (agent saves check every reference), so a reference to someone + else's resource never reveals its name. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + refs: The ``(type, id)`` resources it references. + viewer: The user reading the page. + + Returns: + list: Per resource ``{key, type, id, name, state, reason, note, + sponsor, contact, contact_role, connection, runs_as, credential_mode, + account, owner_credential_writes, writes_allowed, can_confirm, + can_reconnect}``. ``state`` is + ``active`` or ``stopped``; ``reason`` (None while active) is one of + ``deleted``, ``owner_lost_access``, the sponsor reasons, + ``connection_needs_reconnect``, ``connection_removed`` or + ``connector_disabled``. ``sponsor`` (``{user_id, label}``) is the + recorded sponsor. ``contact_role`` is ``resource_owner`` when the + resource's owner (not the reader) can fix it, and ``contact`` names + them, or is None. ``connection`` (``{id, connector_key, name}``) + names the service of a connected tool, and of one whose connection + reason stopped it; ``id`` only when the reader may reconnect it, and + the account's own name only for its owner. ``runs_as`` (``{user_id, + label}``) is the live sponsor a running item runs as, None when it + runs as the owner or doesn't run. + For a running tool, ``note`` is ``per_user_account`` when it runs on + each caller's own account; ``credential_mode`` is ``owner`` or + ``member`` when it has a connection (else None); ``account`` is whose + saved credentials or ``owner``-mode connection it acts with (the + tool's owner); ``owner_credential_writes`` names its write actions on + those credentials (what the API write allowlist covers); and + ``writes_allowed`` is False when an admin turned off changes through + its connector. Other items get None, None, None, ``[]`` and True. + Every person is named only to a reader who knows them + (:func:`people_named_to`): ``sponsor``, ``runs_as`` and ``account`` + are then ``{user_id: None, label: None}``, and ``contact`` None. + ``can_confirm``: the reader may take it over on their next save; + ``can_reconnect``: the reader owns the connection that needs signing + in again. + """ + owner = holder.get("user_id") + seen: set[str] = set() + pairs: list[tuple[str, str]] = [] + for resource_type, resource_id in refs: + rid = str(resource_id).lower() + key = sponsor_key(resource_type, rid) + if key in seen or resource_type not in REF_USE_ACTION or not _sponsorable(resource_type, rid): continue - out.append( - { - "type": resource_type, - "id": resource_id, - "user_id": user_id, - "label": labels.get(user_id) or user_id, - "active": active_sponsor(conn, holder_type, holder, resource_type, resource_id) is not None, - } + seen.add(key) + pairs.append((resource_type, rid)) + if not pairs: + return [] + rows = _ref_rows(conn, pairs) + recorded = {k.lower(): v for k, v in (holder.get("resource_sponsors") or {}).items()} + viewer_edits = bool(viewer and viewer != owner and _holder_editable_by(conn, holder_type, holder, viewer)) + tools_repo = UserToolsRepository(conn) + policies_box: list = [] + entries: list[dict] = [] + + def reader_sees(resource_type: str, rid: str) -> bool: + return bool(viewer) and resolve(conn, resource_type, rid, viewer) is not None + + for resource_type, rid in pairs: + key = sponsor_key(resource_type, rid) + info = rows.get(key) or {} + connection = None + run = dict(_NO_RUN_DETAILS) + if resource_type == "tool": + tool_row, access = resolve_holder_tool(conn, holder_type, holder, rid, tools_repo=tools_repo) + reason = access.reason + if tool_row is not None and reason is None: + reason, connection, run = _tool_run_state(conn, tool_row, owner, policies_box) + else: + access = ref_access(conn, holder_type, holder, resource_type, rid) + reason = access.reason + sponsor = recorded.get(key) + sponsor = sponsor if sponsor and sponsor != owner else None + # Who it runs as now: a live sponsor, or None for the owner (a + # recorded sponsor stops mattering once the owner can use it). + runs_as = access.principal if reason is None and access.principal != owner else None + resource_owner = info.get("user_id") + can_confirm = bool( + reason in (REASON_OWNER_LOST_ACCESS, REASON_CANNOT_EDIT_HOLDER, REASON_CANNOT_EDIT_RESOURCE) + and viewer_edits + and can_sponsor_ref(conn, resource_type, rid, viewer) ) - return out + # Whoever owns the resource can share it again or fix its account. + contact = ( + resource_owner + if reason in (REASON_OWNER_LOST_ACCESS, REASON_CONNECTION_NEEDS_RECONNECT, REASON_CONNECTION_REMOVED) + and resource_owner + and resource_owner != viewer + else None + ) + name_visible = bool(reason is None or sponsor or holder_type == "agent" or reader_sees(resource_type, rid)) + can_reconnect = bool( + reason == REASON_CONNECTION_NEEDS_RECONNECT + and connection + and connection[0] + and viewer + and viewer == resource_owner + ) + entries.append({ + "key": key, + "type": resource_type, + "id": rid, + "name": info.get("name") if name_visible else None, + "state": "active" if reason is None else "stopped", + "reason": reason, + "note": run["note"], + "sponsor": sponsor, + "contact": contact, + "contact_role": CONTACT_RESOURCE_OWNER if contact else None, + "connection": _connection_payload(connection, bool(viewer) and viewer == resource_owner, can_reconnect), + "runs_as": runs_as, + "credential_mode": run["credential_mode"], + "account": run["account"], + "owner_credential_writes": run["owner_credential_writes"], + "writes_allowed": run["writes_allowed"], + "can_confirm": can_confirm, + "can_reconnect": can_reconnect, + }) + # Every person is named by the one rule (:func:`people_named_to`); whom + # to ask and whose credentials a tool uses are named as the owner of + # that resource. + people = [ + (entry[field_name], (entry["type"], entry["id"]) if field_name in ("contact", "account") else None) + for entry in entries + for field_name in ("sponsor", "runs_as", "contact", "account") + ] + named = people_named_to(conn, viewer, holder, people) + labels = _user_labels(conn, [user_id for user_id, ref in named]) + for entry in entries: + ref = (entry["type"], entry["id"]) + entry["sponsor"] = _person(entry["sponsor"], labels, (entry["sponsor"], None) in named) + entry["runs_as"] = _person(entry["runs_as"], labels, (entry["runs_as"], None) in named) + entry["account"] = _person(entry["account"], labels, (entry["account"], ref) in named) + # Whom to ask stays a role for someone the reader doesn't know. + contact = _person(entry["contact"], labels, (entry["contact"], ref) in named) + entry["contact"] = contact if contact and contact["user_id"] else None + return entries + + +def best_effort(conn: Connection, label: str, compute, default): + """``compute()``, or ``default`` when it fails, without failing the read. + + Runs inside a savepoint, so a failed query doesn't poison the rest of + the read's transaction. + + Args: + conn: Open database connection. + label: What is computed, for the log line. + compute: A no-argument callable. + default: What to return when it raises. + """ + try: + with conn.begin_nested(): + return compute() + except Exception: + logger.exception("Could not work out %s; left out of the read", label) + return default + + +def named_ref_keys(states: list[dict]) -> set[str]: + """Keys of the references whose names :func:`resource_states` gave the reader. + + The same rule decides which node resources a workflow read names in its + ``ref_details``, so the two never disagree. + + Args: + states: :func:`resource_states` for the reader. + + Returns: + set: ``":"`` keys, ids lowercased. + """ + return {state["key"] for state in states if state.get("name") is not None} + + +def sponsor_audience( + conn: Connection, holder_type: str, holder: dict, states: list[dict], sponsors: list[dict] +) -> Optional[dict]: + """The holder's audience when the reader may take something over, else None. + + A take-over runs the resource with the reader's access for everyone who + uses the holder, so the page shows them who that is before they agree. + + Args: + conn: Open database connection. + holder_type: ``agent`` or ``workflow``. + holder: The holder row. + states: :func:`resource_states` for the reader. + sponsors: :func:`sponsor_details` for the reader. + + Returns: + dict or None: :func:`holder_audience`, when any item has ``can_confirm``. + """ + if not any(item.get("can_confirm") for item in [*states, *sponsors]): + return None + return holder_audience(conn, holder_type, holder) diff --git a/docsgpt/api/user/scheduler_worker.py b/docsgpt/api/user/scheduler_worker.py index fe581759..0183c633 100644 --- a/docsgpt/api/user/scheduler_worker.py +++ b/docsgpt/api/user/scheduler_worker.py @@ -233,6 +233,33 @@ def _scheduler_still_allowed(schedule: Dict[str, Any], agent_config: Dict[str, A return resolve(conn, "agent", str(agent_id), user_id) is not None +def _schedule_caller_rules(schedule: Dict[str, Any], agent_config: Dict[str, Any]) -> Dict[str, bool]: + """The caller rules a scheduled run keeps, though it acts as the owner. + + A schedule set through the agent's API key is an external caller's; one + a user without a team grant set (they reach the agent by its public + link) is a public-link caller's. Neither can approve for the owner, so + the run writes on the owner's accounts and credentials only as far as + the agent's API write allowlist allows. + + Args: + schedule: The schedule row. + agent_config: The agent row (or the agentless ephemeral config). + + Returns: + ``external_caller`` and ``public_link_caller`` flags for the run. + """ + public = False + user_id = schedule.get("user_id") + agent_id = agent_config.get("id") + if agent_id and user_id and agent_config.get("user_id") != user_id: + from docsgpt.api.user.resource_access import resolve + + with get_engine().connect() as conn: + public = resolve(conn, "agent", str(agent_id), user_id) is None + return {"external_caller": schedule.get("created_via") == "api", "public_link_caller": public} + + def execute_scheduled_run_body(run_id: str, celery_task_id: Optional[str]) -> Dict[str, Any]: """Execute one scheduled run by id; returns a result dict for tracing.""" if not settings.POSTGRES_URI: @@ -324,6 +351,7 @@ def execute_scheduled_run_body(run_id: str, celery_task_id: Optional[str]) -> Di # to the user who scheduled it (not always the agent's owner). request_id=str(run_id), trace_user_id=run.get("user_id") or schedule.get("user_id"), + **_schedule_caller_rules(schedule, agent_config), ) except SoftTimeLimitExceeded: timed_out = True diff --git a/docsgpt/api/user/sources/routes.py b/docsgpt/api/user/sources/routes.py index fab6968a..423d621e 100644 --- a/docsgpt/api/user/sources/routes.py +++ b/docsgpt/api/user/sources/routes.py @@ -74,6 +74,12 @@ def _get_provider_from_remote_data(remote_data): return None +def _connection_id(row: dict) -> str | None: + """The connection a source syncs from, as a string id, or None.""" + value = row.get("connection_id") + return str(value) if value else None + + def _with_access(entry: dict, access: Optional[str], switches: Optional[dict]) -> dict: """Add ``access`` + ``allowed_actions`` to a listed source row. @@ -150,6 +156,7 @@ class CombinedJson(Resource): "config": SourceConfig.parse(index.get("config")).model_dump(), "ownership": ownership, "team_access": team_access, + "connectionId": _connection_id(index), } return _with_access( entry, @@ -247,6 +254,7 @@ class PaginatedSources(Resource): "team_access": ( None if owned else team_shared.get(str(doc["id"])) ), + "connectionId": _connection_id(doc), } paginated_docs.append( _with_access( @@ -269,6 +277,70 @@ class PaginatedSources(Resource): return make_response(jsonify({"success": False}), 400) +def delete_source(owner: str, doc: dict, *, actor: Optional[str] = None) -> bool: + """Delete a source's index, stored files and row. Returns whether it worked. + + Args: + owner: The source's owner; the row is deleted as them. + doc: The source row, already authorised for the caller. + actor: Who asked for the delete (a team editor, say), when not the + owner. Recorded on the audit event. + """ + actor = actor or owner + storage = StorageCreator.get_storage() + resolved_id = str(doc["id"]) + source_id = resolved_id + + try: + if settings.VECTOR_STORE == "faiss": + index_path = f"indexes/{resolved_id}" + # index.pkl is the legacy sidecar; index.json the current one. + # Older sources have only the former, so clear whichever exist. + for index_file in ("index.faiss", "index.json", "index.pkl"): + if storage.file_exists(f"{index_path}/{index_file}"): + storage.delete_file(f"{index_path}/{index_file}") + else: + vectorstore = VectorCreator.create_vectorstore( + settings.VECTOR_STORE, source_id=source_id + ) + vectorstore.delete_index() + if "file_path" in doc and doc["file_path"]: + file_path = doc["file_path"] + if storage.is_directory(file_path): + files = storage.list_files(file_path) + for f in files: + storage.delete_file(f) + else: + storage.delete_file(file_path) + except FileNotFoundError: + pass + except Exception as err: + current_app.logger.error( + f"Error deleting files and indexes: {err}", exc_info=True + ) + return False + try: + with db_session() as conn: + # The AFTER DELETE trigger drops the source's team grants; the + # owner switches have no FK, so clear them here. + SourcesRepository(conn).delete(resolved_id, owner) + delete_settings(conn, "source", resolved_id) + record_event( + conn, + "source.deleted", + actor=actor, + source_id=resolved_id, + name=doc.get("name"), + owner=owner if owner != actor else None, + ) + except Exception as err: + current_app.logger.error( + f"Error deleting source row: {err}", exc_info=True + ) + return False + return True + + @sources_ns.route("/delete_old") class DeleteOldIndexes(Resource): @api.doc( @@ -295,56 +367,7 @@ class DeleteOldIndexes(Resource): except Exception as err: current_app.logger.error(f"Error looking up source: {err}", exc_info=True) return make_response(jsonify({"success": False}), 400) - owner = ra.owner_id - storage = StorageCreator.get_storage() - resolved_id = str(doc["id"]) - - try: - if settings.VECTOR_STORE == "faiss": - index_path = f"indexes/{resolved_id}" - # index.pkl is the legacy sidecar; index.json the current one. - # Older sources have only the former, so clear whichever exist. - for index_file in ("index.faiss", "index.json", "index.pkl"): - if storage.file_exists(f"{index_path}/{index_file}"): - storage.delete_file(f"{index_path}/{index_file}") - else: - vectorstore = VectorCreator.create_vectorstore( - settings.VECTOR_STORE, source_id=resolved_id - ) - vectorstore.delete_index() - if "file_path" in doc and doc["file_path"]: - file_path = doc["file_path"] - if storage.is_directory(file_path): - files = storage.list_files(file_path) - for f in files: - storage.delete_file(f) - else: - storage.delete_file(file_path) - except FileNotFoundError: - pass - except Exception as err: - current_app.logger.error( - f"Error deleting files and indexes: {err}", exc_info=True - ) - return make_response(jsonify({"success": False}), 400) - try: - with db_session() as conn: - # The AFTER DELETE trigger drops the source's team grants; the - # owner switches have no FK, so clear them here. - SourcesRepository(conn).delete(resolved_id, owner) - delete_settings(conn, "source", resolved_id) - record_event( - conn, - "source.deleted", - actor=user, - source_id=resolved_id, - name=doc.get("name"), - owner=owner if owner != user else None, - ) - except Exception as err: - current_app.logger.error( - f"Error deleting source row: {err}", exc_info=True - ) + if not delete_source(ra.owner_id, doc, actor=user): return make_response(jsonify({"success": False}), 400) return make_response(jsonify({"success": True}), 200) @@ -467,6 +490,8 @@ class SyncSource(Resource): sync_frequency=doc.get("sync_frequency", "never"), retriever=doc.get("retriever", "classic"), doc_id=str(doc["id"]), + # S3 and GitHub sources made from a connection read with its keys. + connection_id=str(doc["connection_id"]) if doc.get("connection_id") else None, ) except Exception as err: current_app.logger.error( @@ -1031,6 +1056,90 @@ class WikiPage(Resource): ) +def _wiki_settings_body(doc: dict, ra) -> dict: + """The wiki settings response: the stored switch plus the caller's access.""" + return { + "success": True, + "allow_outside_edits": bool(doc.get("wiki_outside_edits")), + **ra.payload(), + } + + +@sources_ns.route("/sources//wiki/settings") +class WikiSettings(Resource): + @api.doc( + description="A wiki's settings. Anyone who can see the wiki may read " + "them; returns allow_outside_edits plus the caller's access." + ) + def get(self, source_id): + decoded_token = request.decoded_token + if not decoded_token: + return make_response(jsonify({"success": False}), 401) + user = decoded_token.get("sub") + try: + with db_readonly() as conn: + try: + doc, ra = load_source(conn, source_id, user, "use") + except AccessDenied as err: + return denied_response(err) + except Exception as err: + current_app.logger.error( + f"Error reading wiki settings for {source_id}: {err}", exc_info=True + ) + return make_response(jsonify({"success": False}), 400) + return make_response(jsonify(_wiki_settings_body(doc, ra)), 200) + + @api.doc( + description="Change a wiki's settings (owner only, manage_settings). " + "Body: {\"allow_outside_edits\": bool}: whether runs from an agent's " + "API key or widget may edit the wiki." + ) + def put(self, source_id): + decoded_token = request.decoded_token + if not decoded_token: + return make_response(jsonify({"success": False}), 401) + user = decoded_token.get("sub") + data = request.get_json(silent=True) or {} + allowed = data.get("allow_outside_edits") + if not isinstance(allowed, bool): + return make_response( + jsonify( + {"success": False, "message": "allow_outside_edits must be true or false"} + ), + 400, + ) + try: + with db_session() as conn: + try: + doc, ra = load_source(conn, source_id, user, "manage_settings") + except AccessDenied as err: + return denied_response(err) + if SourceConfig.parse(doc.get("config")).kind != "wiki": + return make_response( + jsonify({"success": False, "message": "Source is not a wiki"}), 400 + ) + if not SourcesRepository(conn).set_wiki_outside_edits( + str(doc["id"]), ra.owner_id, allowed + ): + return make_response( + jsonify({"success": False, "message": "Source not found"}), 404 + ) + record_event( + conn, + "source.wiki_settings_updated", + actor=user, + source_id=str(doc["id"]), + allow_outside_edits=allowed, + ) + doc["wiki_outside_edits"] = allowed + except Exception as err: + current_app.logger.error( + f"Error updating wiki settings for {source_id}: {err}", exc_info=True + ) + return make_response(jsonify({"success": False}), 400) + return make_response(jsonify(_wiki_settings_body(doc, ra)), 200) + + def _source_is_blank(doc): """True when a source has no ingested files to convert into pages.""" structure = doc.get("directory_structure") or {} diff --git a/docsgpt/api/user/sources/upload.py b/docsgpt/api/user/sources/upload.py index 4fc52e7e..3e155649 100644 --- a/docsgpt/api/user/sources/upload.py +++ b/docsgpt/api/user/sources/upload.py @@ -29,7 +29,6 @@ from docsgpt.security.zip_archive import ( ) from docsgpt.storage.db.repositories.connector_sessions import ( ConnectorSessionsRepository, - owns_connector_session, ) from docsgpt.storage.db.repositories.idempotency import IdempotencyRepository from docsgpt.storage.db.repositories.sources import SourcesRepository @@ -452,6 +451,45 @@ class UploadFile(Resource): return make_response(jsonify(response_payload), 200) +def _remote_credentials(user, source, config): + """Split an S3 / Reddit request into loader config and the connection holding its keys. + + A request naming a ``connection_id`` uses that connection's stored keys. + A request carrying keys (the form before connections) stores them on a + connection, so they are entered once and never land in + ``sources.remote_data``. When a multi-user install still runs on the + public default encryption key, the keys stay with the source as before. + + Returns: + ``(source_data, connection_id, error_response)``. + """ + from docsgpt.connectors import catalog, service + + definition = catalog.get_definition(source) + credential_keys = {f.key for f in definition.credential_fields} + public = {k: v for k, v in config.items() if k not in credential_keys and k != "connection_id"} + connection_id = config.get("connection_id") + if connection_id: + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get_for_user(str(connection_id), user) + if row is None or catalog.connector_key_for_row(row) != source: + return None, None, make_response( + jsonify({"success": False, "error": "Invalid or unauthorized connection"}), 401, + ) + return public, str(row["id"]), None + provided = {k: config[k] for k in credential_keys if config.get(k) not in (None, "")} + if not provided: + return config, None, None + try: + with db_session() as conn: + row, _ = service.create_api_key_connection(conn, user, definition, provided) + except service.ConnectorDisabled as err: + return None, None, make_response(jsonify({"success": False, "error": str(err)}), 403) + except (service.EncryptionKeyNotConfigured, ValueError): + return config, None, None + return public, str(row["id"]), None + + @sources_upload_ns.route("/remote") class UploadRemote(Resource): @api.expect( @@ -521,32 +559,35 @@ class UploadRemote(Resource): try: config = json.loads(data["data"]) source_data = None + connection_id = None if data["source"] == "github": source_data = config.get("repo_url") elif data["source"] in ["crawler", "url", "sitemap"]: source_data = config.get("url") - elif data["source"] == "reddit": - source_data = config - elif data["source"] == "s3": - source_data = config + elif data["source"] in ("reddit", "s3"): + source_data, connection_id, error = _remote_credentials(user, data["source"], config) + if error is not None: + if scoped_key: + _release_claim(scoped_key) + return error elif data["source"] in ConnectorCreator.get_supported_connectors(): - session_token = config.get("session_token") - if not session_token: + if not (config.get("connection_id") or config.get("session_token")): if scoped_key: _release_claim(scoped_key) return make_response( jsonify( { "success": False, - "error": f"Missing session_token in {data['source']} configuration", + "error": f"Missing connection_id in {data['source']} configuration", } ), 400, ) - with db_readonly() as conn: - connector_session = ConnectorSessionsRepository(conn).get_by_session_token(session_token) - if not owns_connector_session(connector_session, user, data["source"]): + from docsgpt.connectors import service as connection_service + + connector_session = connection_service.resolve_request_connection(user, data["source"], config) + if connector_session is None: if scoped_key: _release_claim(scoped_key) return make_response( @@ -577,7 +618,7 @@ class UploadRemote(Resource): "job_name": data["name"], "user": user, "source_type": data["source"], - "session_token": session_token, + "connection_id": str(connector_session["id"]), "file_ids": file_ids, "folder_ids": folder_ids, "recursive": config.get("recursive", False), @@ -612,6 +653,7 @@ class UploadRemote(Resource): remote_kwargs = { "kwargs": { "source_data": source_data, + "connection_id": connection_id, "job_name": data["name"], "user": user, "loader": data["source"], diff --git a/docsgpt/api/user/tasks.py b/docsgpt/api/user/tasks.py index b34945fe..996c3b49 100644 --- a/docsgpt/api/user/tasks.py +++ b/docsgpt/api/user/tasks.py @@ -79,6 +79,12 @@ DURABLE_TASK = dict( ) +def _connection_unavailable(): + from docsgpt.connectors.service import ConnectionUnavailable + + return ConnectionUnavailable + + def durable_task(**overrides) -> Dict: """Return ``DURABLE_TASK`` with per-task overrides applied. @@ -167,13 +173,16 @@ def ingest( @with_idempotency(task_name="ingest_remote", on_poison=_emit_ingest_poison_event) def ingest_remote( self, source_data, job_name, user, loader, - config=None, idempotency_key=None, source_id=None, + config=None, idempotency_key=None, source_id=None, connection_id=None, + sync_frequency="never", ): resp = remote_worker( self, source_data, job_name, user, loader, + sync_frequency=sync_frequency, config=config, idempotency_key=idempotency_key, source_id=source_id, + connection_id=connection_id, ) return resp @@ -249,6 +258,15 @@ def schedule_syncs(self, frequency): return resp +@celery.task(bind=True, acks_late=True, autoretry_for=(Exception,), max_retries=3, retry_backoff=60, + dont_autoretry_for=(_connection_unavailable(),)) +def sync_connector_source(self, source_id): + """Re-sync one connector source from its connection, with no browser involved.""" + from docsgpt.worker import sync_connector_source as run + + return run(self, source_id) + + @celery.task(bind=True) def sync_source( self, @@ -259,6 +277,7 @@ def sync_source( sync_frequency, retriever, doc_id, + connection_id=None, ): resp = sync( self, @@ -269,6 +288,7 @@ def sync_source( sync_frequency, retriever, doc_id, + connection_id=connection_id, ) return resp @@ -408,7 +428,9 @@ except Exception: pass -@celery.task(**DURABLE_TASK) +# A revoked or disconnected connection will not heal by retrying; the +# service has already paused the source and told its owner to reconnect. +@celery.task(**durable_task(dont_autoretry_for=(DocumentParseError, _connection_unavailable()))) @with_idempotency( task_name="ingest_connector_task", on_poison=_emit_ingest_poison_event, ) @@ -428,6 +450,7 @@ def ingest_connector_task( config=None, idempotency_key=None, source_id=None, + connection_id=None, ): from docsgpt.worker import ingest_connector @@ -437,6 +460,7 @@ def ingest_connector_task( user, source_type, session_token=session_token, + connection_id=connection_id, file_ids=file_ids, folder_ids=folder_ids, recursive=recursive, diff --git a/docsgpt/api/user/tools/mcp.py b/docsgpt/api/user/tools/mcp.py index 5f284c82..1308d4c3 100644 --- a/docsgpt/api/user/tools/mcp.py +++ b/docsgpt/api/user/tools/mcp.py @@ -5,6 +5,7 @@ from urllib.parse import urlencode, urlparse from flask import current_app, jsonify, make_response, redirect, request from flask_restx import Namespace, Resource, fields +from docsgpt.agents.tool_pins import carry_pins_between from docsgpt.agents.tools.mcp_tool import MCPOAuthManager, MCPTool from docsgpt.api import api from docsgpt.api.user.resource_access import AccessDenied, require @@ -18,6 +19,7 @@ from docsgpt.api.user.tools.routes import ( transform_actions, ) from docsgpt.cache import get_redis_instance +from docsgpt.connectors.resolve import REMOVED_CONNECTION_KEY from docsgpt.core.url_validation import SSRFError, validate_url from docsgpt.security.encryption import decrypt_credentials, encrypt_credentials from docsgpt.storage.db.repositories.connector_sessions import ( @@ -36,13 +38,19 @@ def _sanitize_mcp_transport(config): """Normalise and validate the transport_type field. Strips ``command`` / ``args`` keys that are only valid for local STDIO - transports and returns the cleaned transport type string. + transports, ``connection_id``, which only the tool executor sets + (it picks whose MCP tokens the tool uses), and the note that a + connection was removed, which only removing one writes (a save stores + a fresh config, so it drops that note too). Returns the cleaned + transport type string. """ transport_type = (config.get("transport_type") or "auto").lower() if transport_type not in _ALLOWED_TRANSPORTS: raise ValueError(f"Unsupported transport_type: {transport_type}") config.pop("command", None) config.pop("args", None) + config.pop("connection_id", None) + config.pop(REMOVED_CONNECTION_KEY, None) config["transport_type"] = transport_type return transport_type @@ -84,14 +92,140 @@ def _validate_mcp_server_url(config: dict) -> None: raise ValueError(f"Invalid server URL: {exc}") from exc +def _mcp_connection(user, config, auth_type, auth_credentials, display_name): + """The connection an MCP tool runs with; created on first save. + + OAuth servers already have one (the sign-in stored its tokens there). + Key, bearer and basic auth store their secret on a connection; servers + with no auth get a credential-less connection so they still appear on + the Connectors page. Returns None when a multi-user install runs on the + default encryption key, which keeps the legacy per-tool secret. + """ + from docsgpt.connectors import catalog, service + + base = catalog.base_url(config.get("server_url")) + if not base: + return None + if auth_type == "oauth": + with db_readonly() as conn: + row = service._mcp_row(conn, user, base, None) + return str(row["id"]) if row else None + definition = catalog.get_definition("custom_mcp") + host = base.split("://")[-1] + try: + with db_session() as conn: + if auth_credentials: + row, _ = service.create_api_key_connection( + conn, user, definition, auth_credentials, server_url=base, display_name=display_name, + ) + else: + repo = ConnectorSessionsRepository(conn) + row = repo.find_account(user, "custom_mcp", server_url=base, account_label=host) or repo.create( + user, "custom_mcp", connector_key="custom_mcp", auth_kind="none", + display_name=display_name, account_label=host, server_url=base, + ) + except service.EncryptionKeyNotConfigured: + return None + return str(row["id"]) if row else None + + +def _previous_connection(existing_doc, config, owner_id) -> str | None: + """The saved tool's connection, kept only while the server is unchanged. + + An edit that cannot resolve a connection of its own (the default + encryption key blocks storing a new secret) must not carry the old + one over to a different server (its key would be sent there), to a + connection the tool's owner does not own, or to one that signs in + another way. + """ + from docsgpt.connectors import catalog + + connection_id = (existing_doc or {}).get("connection_id") + base = catalog.base_url(config.get("server_url")) + if not connection_id or not base: + return None + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get_for_user(str(connection_id), owner_id) + if row is None or catalog.base_url(row.get("server_url")) != base: + return None + wanted = {"oauth": "mcp_oauth", "none": "none"}.get(config.get("auth_type") or "none", "api_key") + if (row.get("auth_kind") or "") != wanted: + return None + return str(connection_id) + + +def _mcp_policy_error(config: dict): + """A 403 when an admin turned this MCP server's connector off, else None. + + A preset's own switch applies to its server; any other server is a + custom connector and needs "Allow custom MCP servers". + """ + from docsgpt.connectors import catalog, service + + preset = catalog.preset_for_url(config.get("server_url")) + key = preset.key if preset else "custom_mcp" + try: + with db_readonly() as conn: + service.ensure_connector_allowed(conn, key) + except service.ConnectorDisabled: + return make_response( + jsonify({"success": False, "error": "This MCP server is turned off by an admin", "code": "disabled"}), + 403, + ) + except Exception: + # Fail closed: a server whose admin switch cannot be read is not contacted. + current_app.logger.warning("Could not read connector policies", exc_info=True) + return make_response( + jsonify({"success": False, "error": "Could not check whether this MCP server is allowed"}), + 503, + ) + return None + + +def _stored_mcp_credentials(existing_doc: dict, owner_id: str) -> dict: + """The secrets a saved MCP tool authenticates with, for reuse on an edit. + + A legacy tool keeps them encrypted in its config; a connection-backed one + on its key-based connection, read only while that connection is the + owner's and was stored for the tool's own server. + + Args: + existing_doc: The stored ``user_tools`` row. + owner_id: The tool's owner. + + Returns: + The stored credentials, or ``{}`` when there are none to reuse. + """ + from docsgpt.connectors import catalog, service + + existing_config = existing_doc.get("config") or {} + if existing_config.get("encrypted_credentials"): + return decrypt_credentials(existing_config["encrypted_credentials"], owner_id) + connection_id = existing_doc.get("connection_id") + if not connection_id: + return {} + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get_for_user(str(connection_id), owner_id) + if ( + row is None + or (row.get("auth_kind") or "") != "api_key" + or catalog.base_url(row.get("server_url")) != catalog.base_url(existing_config.get("server_url")) + ): + return {} + try: + return service.get_credentials(row) + except service.ConnectionUnavailable: + return {} + + def _existing_mcp_context(tool_id, user, config): """Resolve the stored MCP tool a test/save refers to, and its credentials. With no ``tool_id`` the caller acts on their own new server. With one, the caller needs ``edit_credentials`` on that tool and everything runs as its owner. Stored secrets are write-only, so an empty secret field reuses - the stored one while the origin (scheme, host, port) is unchanged; a new - origin never inherits them. + the stored one (on the tool, or on its key-based connection) while the + origin (scheme, host, port) is unchanged; a new origin never inherits them. A server that is or would become OAuth is the owner's alone (its tokens are the owner's sign-in). @@ -123,9 +257,9 @@ def _existing_mcp_context(tool_id, user, config): jsonify({"success": False, "message": _CREDENTIALS_FOR_NEW_SERVER}), 400 ) credentials = dict(auth_credentials) - existing_encrypted = None if moved else existing_config.get("encrypted_credentials") - if existing_encrypted: - credentials = {**decrypt_credentials(existing_encrypted, ra.owner_id), **auth_credentials} + stored = {} if moved else _stored_mcp_credentials(existing_doc, ra.owner_id) + if stored: + credentials = {**stored, **auth_credentials} return existing_doc, ra.owner_id, ra.access == "owner", moved, credentials @@ -168,6 +302,9 @@ class TestMCPServerConfig(Resource): ) _validate_mcp_server_url(config) + policy_error = _mcp_policy_error(config) + if policy_error is not None: + return policy_error ctx = _existing_mcp_context(data.get("id"), user, config) if not isinstance(ctx, tuple): @@ -183,7 +320,7 @@ class TestMCPServerConfig(Resource): safe_result = { k: v for k, v in result.items() - if k in ("success", "requires_oauth", "auth_url") + if k in ("success", "requires_oauth", "auth_url", "task_id") } return make_response(jsonify(safe_result), 200) @@ -269,6 +406,9 @@ class MCPServerSave(Resource): ) _validate_mcp_server_url(config) + policy_error = _mcp_policy_error(config) + if policy_error is not None: + return policy_error # An existing id is always an update of THAT row, written as its # owner; it never falls through to creating a copy for the caller. @@ -280,11 +420,16 @@ class MCPServerSave(Resource): mcp_config = config.copy() mcp_config["auth_credentials"] = merged_credentials - if auth_type == "oauth": + if auth_type == "oauth" and not config.get("oauth_task_id"): # Only the owner reaches here for an existing server (see - # ``_existing_mcp_context``), and every OAuth save needs the - # sign-in they just completed. - if not config.get("oauth_task_id"): + # ``_existing_mcp_context``). No new handshake: they signed in + # to this server before, so its stored tokens answer the + # discovery, and the save below still needs that connection. + try: + mcp_tool = MCPTool(config=mcp_config, user_id=owner_id) + mcp_tool.discover_tools() + actions_metadata = mcp_tool.get_actions_metadata() + except Exception: return make_response( jsonify( { @@ -294,6 +439,7 @@ class MCPServerSave(Resource): ), 400, ) + elif auth_type == "oauth": redis_client = get_redis_instance() manager = MCPOAuthManager(redis_client) result = manager.get_oauth_status( @@ -334,9 +480,32 @@ class MCPServerSave(Resource): "redirect_uri", ]: storage_config.pop(field, None) - transformed_actions = transform_actions(actions_metadata) + from docsgpt.connectors.permissions import apply_default_permissions + + transformed_actions = apply_default_permissions( + "mcp_tool", transform_actions(actions_metadata), + ) display_name = data["displayName"] + # The connection is the owner's: an editor's save stores the + # secret on the owner's account, as the tool runs as its owner. + connection_id = _mcp_connection( + owner_id, storage_config, auth_type, merged_credentials, display_name, + ) or _previous_connection(existing_doc, storage_config, owner_id) + if auth_type == "oauth" and not connection_id: + # A sign-in server's tokens live on its connection. Without one + # (it was removed, and a client cached before that answered the + # discovery) the tool would be saved unconnected. + return make_response( + jsonify({ + "success": False, + "error": "Not signed in to this server. Sign in again to connect it.", + }), + 400, + ) + if connection_id and auth_type != "oauth": + # The secret lives on the connection only. + storage_config.pop("encrypted_credentials", None) description = f"MCP Server: {storage_config.get('server_url', 'Unknown')}" status_bool = bool(data.get("status", True)) fields_out = { @@ -344,7 +513,7 @@ class MCPServerSave(Resource): "custom_name": display_name, "description": description, "config": storage_config, - "actions": transformed_actions, + "connection_id": connection_id, } updated_message = ( f"MCP server updated successfully! Discovered {len(transformed_actions)} tools." @@ -357,6 +526,8 @@ class MCPServerSave(Resource): # save doesn't flip it. if is_owner: fields_out["status"] = status_bool + # Fixed values the owner set survive a re-save. + fields_out["actions"] = carry_pins_between(existing_doc.get("actions"), transformed_actions) repo.update(str(existing_doc["id"]), owner_id, fields_out) saved_id = str(existing_doc["id"]) response_data = { @@ -375,6 +546,9 @@ class MCPServerSave(Resource): (existing_by_name.get("config") or {}).get("server_url") == storage_config.get("server_url") ): + fields_out["actions"] = carry_pins_between( + existing_by_name.get("actions"), transformed_actions, + ) repo.update(str(existing_by_name["id"]), user, fields_out) saved_id = str(existing_by_name["id"]) response_data = { @@ -393,6 +567,7 @@ class MCPServerSave(Resource): config_requirements={}, actions=transformed_actions, status=status_bool, + connection_id=connection_id, ) saved_id = str(created["id"]) response_data = { @@ -458,7 +633,7 @@ class MCPOAuthCallback(Resource): "/api/connectors/callback-status?status=error&message=Internal+server+error:+Redis+not+available.&provider=mcp_tool" ) manager = MCPOAuthManager(redis_client) - success = manager.handle_oauth_callback(state, code, error) + success = manager.handle_oauth_callback(state, code, error, iss=request.args.get("iss")) if success: return redirect( "/api/connectors/callback-status?status=success&message=Authorization+code+received+successfully.+You+can+close+this+window.&provider=mcp_tool" @@ -505,50 +680,36 @@ class MCPAuthStatus(Resource): jsonify({"success": True, "statuses": {}}), 200 ) - oauth_server_urls: dict = {} + from docsgpt.connectors import service + + # Read from connection status alone: status checks never + # decrypt credentials. statuses: dict = {} for tool in mcp_tools: tool_id = str(tool["id"]) config = tool.get("config") or {} auth_type = config.get("auth_type", "none") - if auth_type == "oauth": - server_url = config.get("server_url", "") - if server_url: - parsed = urlparse(server_url) - base_url = f"{parsed.scheme}://{parsed.netloc}" - oauth_server_urls[tool_id] = (tool.get("user_id") or user, base_url) + row = None + if tool.get("connection_id"): + row = sessions_repo.get(str(tool["connection_id"])) + elif auth_type == "oauth" and config.get("server_url"): + parsed = urlparse(config["server_url"]) + # A team-shared server signs in as its owner. + row = sessions_repo.get_by_user_provider( + tool.get("user_id") or user, + service.mcp_provider(f"{parsed.scheme}://{parsed.netloc}"), + ) + if row is not None: + connected = service.normalize_status(row) == service.STATUS_CONNECTED + if auth_type == "oauth" or not connected: + statuses[tool_id] = "connected" if connected else "needs_auth" else: - statuses[tool_id] = "needs_auth" + statuses[tool_id] = "configured" + elif auth_type == "oauth": + statuses[tool_id] = "needs_auth" else: statuses[tool_id] = "configured" - if oauth_server_urls: - # Look up a session per distinct base URL. MCP sessions - # are stored with ``provider = "mcp:"`` - # and the URL in ``server_url``; reuse the repo's - # per-URL accessor rather than an ad-hoc $in query. - url_has_tokens: dict = {} - for owner_id, base_url in set(oauth_server_urls.values()): - session = sessions_repo.get_by_user_and_server_url( - owner_id, base_url, - ) - tokens = ( - (session or {}).get("session_data", {}) or {} - ).get("tokens", {}) or {} - # MCP code also stashes tokens into token_info on - # the row; consider either present as "connected". - token_info = (session or {}).get("token_info") or {} - url_has_tokens[(owner_id, base_url)] = bool( - tokens.get("access_token") - or token_info.get("access_token") - ) - - for tool_id, key in oauth_server_urls.items(): - if url_has_tokens.get(key): - statuses[tool_id] = "connected" - else: - statuses[tool_id] = "needs_auth" - return make_response(jsonify({"success": True, "statuses": statuses}), 200) except Exception as e: current_app.logger.error( diff --git a/docsgpt/api/user/tools/routes.py b/docsgpt/api/user/tools/routes.py index 25d826e1..d400496f 100644 --- a/docsgpt/api/user/tools/routes.py +++ b/docsgpt/api/user/tools/routes.py @@ -19,6 +19,7 @@ from docsgpt.agents.default_tools import ( WORKFLOW_ONLY_BUILTINS, ) from docsgpt.agents.tool_executor import API_TOOL_SECRET_SECTIONS, API_TOOL_SECRETS_KEY +from docsgpt.agents.tool_pins import iter_parameters, llm_fills, merge_submitted_actions, PinChangeRefused from docsgpt.agents.tools.spec_parser import parse_spec from docsgpt.agents.tools.tool_manager import ToolManager from docsgpt.api import api @@ -33,9 +34,13 @@ from docsgpt.api.user.resource_access import ( settings_many, ) from docsgpt.api.user.team_sharing import visible_with_access +from docsgpt.connectors.catalog import base_url, definition_for_tool +from docsgpt.connectors.resolve import carry_removed_connection +from docsgpt.connectors.service import account_tool_names +from docsgpt.connectors.permissions import owner_credential_writes from docsgpt.core.settings import settings from docsgpt.core.url_validation import SSRFError, validate_url -from docsgpt.security.encryption import decrypt_credentials, encrypt_credentials +from docsgpt.security.encryption import CredentialDecryptionError, decrypt_credentials, encrypt_credentials from docsgpt.storage.db.base_repository import looks_like_uuid from docsgpt.storage.db.repositories.artifacts import ArtifactsRepository from docsgpt.storage.db.repositories.notes import NotesRepository @@ -189,7 +194,14 @@ def _merge_secrets_on_update(new_config, existing_config, config_requirements, u _CREDENTIALS_FOR_NEW_SERVER = "Enter credentials for the new server" _FORBIDDEN_MESSAGE = "Your access to this item doesn't allow that" _MCP_CREDENTIAL_AUTH_TYPES = {"api_key", "bearer", "basic"} -_META_KEYS = ("name", "displayName", "customName", "description", "actions") +# ``name`` (the tool type) and ``actions`` are handled on their own. +_META_KEYS = ("displayName", "customName", "description") +_TYPE_IS_FIXED = "A tool's type can't be changed" +_FIXED_VALUES_OWNER_ONLY = "Only the tool's owner can change fixed values" +_MOVE_THROUGH_MCP_SAVE = ( + "This server signs in through a connection: change its address or sign-in by saving the " + "server again (/api/mcp_server/save)" +) class CredentialsRequired(Exception): @@ -359,6 +371,96 @@ def _api_tool_config_needs_credentials(new_config: dict, existing_config: dict) return False +def _api_tool_param_state(section: str, spec: dict, *, stored: bool) -> tuple: + """Who fills an api_tool parameter and which value it keeps. + + A header / query value is only ever shown masked, so a stored one (sealed, + or legacy plaintext) reads as ``"stored"`` and any value a client sends + is a new one. + + Args: + section: ``headers``, ``query_params`` or ``body``. + spec: The parameter's schema. + stored: Whether ``spec`` comes from the stored config. + + Returns: + ``(filled_by_llm, value marker)``; the marker is None without a value. + """ + value = spec.get("value") + if section in API_TOOL_SECRET_SECTIONS: + if _has_value(value): + marker: Any = "stored" if stored else ("new", value) + else: + marker = "stored" if spec.get("has_value") else None + else: + marker = value if _has_value(value) else None + return llm_fills(spec), marker + + +def _api_tool_fixed_values_changed(new_config: dict, existing_config: dict) -> bool: + """Whether an api_tool save changes a fixed value of an existing action. + + A fixed value may be a secret (an API key in a query) or where a call + goes, so changing who fills a parameter, or its value, or clearing a + stored one is the owner's. Flipping ``filled_by_llm`` on a stored secret + would hand it to the model and show it in the chat. A parameter that + carries no value can be added or removed freely; a new action brings a + URL and is judged by :func:`_api_tool_config_needs_credentials`. + + Args: + new_config: The config the client sent. + existing_config: The stored config. + + Returns: + True when any existing action's fixed values would differ. + """ + old_actions = (existing_config or {}).get("actions") or {} + new_actions = (new_config or {}).get("actions") or {} + if not isinstance(old_actions, dict) or not isinstance(new_actions, dict): + return bool(old_actions) or bool(new_actions) + for name, action in new_actions.items(): + old = old_actions.get(name) + if not isinstance(old, dict) or not isinstance(action, dict): + continue + before = {(s, p): _api_tool_param_state(s, d, stored=True) for s, p, d in iter_parameters(old)} + after = {(s, p): _api_tool_param_state(s, d, stored=False) for s, p, d in iter_parameters(action)} + for key in before.keys() | after.keys(): + if key in before and key in after: + if before[key] != after[key]: + return True + elif (before.get(key) or after.get(key))[1] is not None: + return True + return False + + +def _connection_server_moved(tool_doc: dict, new_config: Optional[dict]) -> bool: + """Whether a config save re-points a connection-backed MCP tool. + + The connection holds the key for one server and one way of signing in; + a tool moved on these routes would keep a connection that no longer + applies (it is refused at run time) and the new key would be stored + where nothing reads it. ``/api/mcp_server/save`` moves a server properly. + + Args: + tool_doc: The stored ``user_tools`` row. + new_config: The incoming ``config``. + + Returns: + True when the base URL or ``auth_type`` would change. + """ + if tool_doc.get("name") != "mcp_tool" or not tool_doc.get("connection_id"): + return False + existing = tool_doc.get("config") or {} + new_config = new_config if isinstance(new_config, dict) else {} + if "server_url" in new_config and base_url(str(new_config.get("server_url") or "").strip()) != base_url( + existing.get("server_url") + ): + return True + return "auth_type" in new_config and (new_config.get("auth_type") or "none") != ( + existing.get("auth_type") or "none" + ) + + def _mcp_origin_changed(new_config: dict, existing_config: dict) -> bool: """Whether a save moves an MCP server to another scheme, host or port.""" old_url = (existing_config or {}).get("server_url") @@ -396,18 +498,40 @@ def check_oauth_mcp_owner_only( raise AccessDenied(403, SHARED_OAUTH_OWNER_ONLY) +def check_api_tool_fixed_values( + ra: ResourceAccess, new_config: Optional[dict], existing_config: Optional[dict] +) -> None: + """Keep an api_tool's fixed header, query and body values with its owner. + + Args: + ra: The caller's access to the tool. + new_config: The incoming ``config``. + existing_config: The stored ``config``. + + Raises: + AccessDenied: 403 when a non-owner changes a fixed value (see + :func:`_api_tool_fixed_values_changed`). + """ + if ra.access != "owner" and _api_tool_fixed_values_changed(new_config or {}, existing_config or {}): + raise AccessDenied(403, _FIXED_VALUES_OWNER_ONLY) + + def _prepare_tool_config(tool_doc: dict, new_config: dict, config_requirements: dict) -> dict: """Validate-free merge of an incoming config with the stored one, as the owner. Handles the three secret stores: ``config_requirements`` secrets (``encrypted_credentials``), api_tool header/query values, and the MCP origin-change rule (a new scheme, host or port drops stored credentials). + A removed connection's note is carried over from the stored config; the + client's copy is ignored. Raises: CredentialsRequired: the MCP origin changed and no new secret arrived. """ owner_id = tool_doc["user_id"] existing_config = tool_doc.get("config") or {} + # The note that its connection was removed is the server's to keep. + new_config = carry_removed_connection(new_config, existing_config) if tool_doc.get("name") == "api_tool": return _seal_api_tool_secrets(new_config, existing_config, owner_id) moved = tool_doc.get("name") == "mcp_tool" and _mcp_origin_changed(new_config, existing_config) @@ -479,8 +603,30 @@ def transform_actions(actions_metadata): return transformed +def _stored_actions(tool_doc: dict) -> list: + """A tool's stored actions, or its class's own for a row stored without any. + + Args: + tool_doc: The ``user_tools`` row. + + Returns: + The action list submitted actions are validated against. + """ + actions = tool_doc.get("actions") or [] + if actions or tool_doc.get("name") in ("mcp_tool", "api_tool"): + return actions + tool_instance = tool_manager.tools.get(tool_doc.get("name")) + if tool_instance is None: + return actions + return transform_actions(copy.deepcopy(tool_instance.get_actions_metadata())) + + tools_ns = Namespace("tools", description="Tool management operations", path="/api") +# Tools the Connectors page adds through "Add custom connector" rather than +# the Add Tool modal. +_CUSTOM_CONNECTOR_TOOLS = {"mcp_tool": "custom_mcp", "api_tool": "custom_openapi"} + @tools_ns.route("/available_tools") class AvailableTools(Resource): @@ -488,6 +634,16 @@ class AvailableTools(Resource): def get(self): if not request.decoded_token: return make_response(jsonify({"success": False}), 401) + from docsgpt.connectors import service as connection_service + + try: + with db_readonly() as conn: + policies = connection_service.load_policies(conn) + except Exception: + # Without the admin's switches, fall back to the defaults (a + # connector is on when its server settings are present). + current_app.logger.warning("Could not read connector policies", exc_info=True) + policies = {} try: tools_metadata = [] for tool_name, tool_instance in tool_manager.tools.items(): @@ -497,6 +653,19 @@ class AvailableTools(Resource): description = lines[1].strip() if len(lines) > 1 else "" config_req = tool_instance.get_config_requirements() actions = tool_instance.get_actions_metadata() + definition = definition_for_tool(tool_name) + if definition is not None: + if not (definition.configured and connection_service.connector_is_enabled( + policies, definition.key, + )): + continue + group, connector_key = "service", definition.key + # One name everywhere: the connector's, not the tool's own. + name = definition.name + elif tool_name in _CUSTOM_CONNECTOR_TOOLS: + group, connector_key = "custom", _CUSTOM_CONNECTOR_TOOLS[tool_name] + else: + group, connector_key = "built_in", None tools_metadata.append( { "name": tool_name, @@ -504,6 +673,8 @@ class AvailableTools(Resource): "description": description, "configRequirements": config_req, "actions": actions, + "group": group, + "connector_key": connector_key, } ) except Exception as err: @@ -534,6 +705,8 @@ class GetTools(Resource): team_shared = visible_with_access(conn, user, "tool") shared_ids = [tid for tid in team_shared if tid not in owned_ids] shared_rows = tools_repo.list_by_ids(shared_ids) + # "Telegram · Alerts bot" when the owner has several bots. + account_names = account_tool_names(conn, [*rows, *shared_rows]) switches = settings_many(conn, "tool", [*owned_ids, *shared_ids]) prefs = UserToolPreferencesRepository(conn).in_chat_many(user, shared_ids) shared_via = _shared_via(conn, user, shared_ids) @@ -542,6 +715,9 @@ class GetTools(Resource): def _shape_tool(row, *, ownership="user", force_strip_secret=False): tool_copy = _row_to_api(row) + # The writes an agent's API write allowlist can cover (read + # from the stored row, before any secret is masked). + tool_copy["owner_credential_writes"] = owner_credential_writes(row) config_req = tool_copy.get("configRequirements", {}) if not config_req: tool_instance = tool_manager.tools.get(tool_copy.get("name")) @@ -556,10 +732,16 @@ class GetTools(Resource): ): tool_copy["config"]["has_encrypted_credentials"] = True tool_copy["config"].pop("encrypted_credentials", None) + if tool_copy.get("connection_id"): + # The secret lives on the connection; the form must not + # ask for it again. + tool_copy.setdefault("config", {})["has_encrypted_credentials"] = True if tool_copy.get("name") == "api_tool": # Header / query-param values are secrets for everyone. tool_copy["config"] = mask_api_tool_config(tool_copy.get("config") or {}) tool_copy["ownership"] = ownership + if str(row["id"]) in account_names: + tool_copy["customName"] = tool_copy["displayName"] = account_names[str(row["id"])] return tool_copy for row in rows: @@ -657,6 +839,9 @@ class CreateTool(Resource): missing_fields = check_required_fields(data, required_fields) if missing_fields: return missing_fields + if isinstance(data.get("config"), dict): + # Only removing a connection notes that it was removed. + data["config"] = carry_removed_connection(data["config"], None) try: if data["name"] == "mcp_tool": server_url = (data.get("config", {}).get("server_url") or "").strip() @@ -680,6 +865,11 @@ class CreateTool(Resource): f"Error getting tool actions: {err}", exc_info=True ) return make_response(jsonify({"success": False}), 400) + definition = definition_for_tool(data["name"]) + if definition is not None: + connected = _create_connected_tool(user, data, definition, tool_instance) + if connected is not None: + return connected try: config_requirements = tool_instance.get_config_requirements() if config_requirements: @@ -722,6 +912,82 @@ class CreateTool(Resource): return make_response(jsonify({"id": new_id}), 200) +def _create_connected_tool(user, data, definition, tool_instance): + """Create a service tool whose secret lives on a connection, not the tool. + + Uses ``connection_id`` when given, otherwise stores the pasted secret on + a connection (reusing an identical one). Returns None to fall back to the + legacy path when a multi-user install still runs on the default key. + """ + from docsgpt.connectors import catalog as connector_catalog + from docsgpt.connectors import service as connection_service + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + config_requirements = tool_instance.get_config_requirements() + public, secrets = connection_service.split_secrets(data.get("config") or {}, config_requirements) + connection_id = data.get("connection_id") + # An existing connection supplies the secrets the request leaves out. + validation_errors = _validate_config( + data.get("config") or {}, config_requirements, has_existing_secrets=bool(connection_id), + ) + if validation_errors: + return make_response( + jsonify({"success": False, "message": "Validation failed", "errors": validation_errors}), 400, + ) + try: + with db_session() as conn: + if connection_id: + connection = ConnectorSessionsRepository(conn).get_for_user(str(connection_id), user) + if connection is None or connector_catalog.connector_key_for_row(connection) != definition.key: + return make_response(jsonify({"success": False, "message": "Connection not found"}), 404) + else: + connection, _ = connection_service.create_api_key_connection(conn, user, definition, secrets) + created = connection_service.create_tool_for_connection( + conn, + user, + connection, + template=data["name"], + display_name=data.get("customName") or data.get("displayName") or definition.name, + config=public, + status=bool(data.get("status", True)), + ) + except connection_service.EncryptionKeyNotConfigured: + return None + except connection_service.ConnectorDisabled as err: + return make_response(jsonify({"success": False, "message": str(err)}), 403) + except ValueError as err: + return make_response(jsonify({"success": False, "message": str(err)}), 400) + return make_response(jsonify({"id": str(created["id"]), "connection_id": str(connection["id"])}), 200) + + +def _update_connection_secrets(conn, user, tool_doc, config, config_requirements): + """Write changed secrets onto the tool's connection; return the tool's public config. + + Returns None when the caller does not own the connection (an editor on a + team share may change actions, never the owner's credentials). + """ + from docsgpt.connectors import service as connection_service + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + public, secrets = connection_service.split_secrets(config or {}, config_requirements) + if secrets: + connection = ConnectorSessionsRepository(conn).get_for_user(str(tool_doc["connection_id"]), user) + if connection is None: + return None + try: + stored = connection_service.read_secrets(connection) + except CredentialDecryptionError: + # Unreadable after a lost key: the new secret replaces it. + stored = {} + credentials = {**(stored.get("credentials") or {}), **secrets} + connection_service.write_secrets( + conn, connection, {**stored, "credentials": credentials}, + status=connection_service.STATUS_CONNECTED, last_error=None, + ) + connection_service.resume_sources(conn, str(connection["id"])) + return public + + @tools_ns.route("/update_tool") class UpdateTool(Resource): @api.expect( @@ -743,6 +1009,16 @@ class UpdateTool(Resource): ) @api.doc(description="Update a tool by ID") def post(self): + """Update a tool's names, actions, config or chat switch. + + The tool type (``name``) never changes. ``actions`` are checked + against the stored ones like ``/api/update_tool_actions``: nothing + is added and fixed values are the owner's. A connection-backed MCP + server is moved only through ``/api/mcp_server/save``. + + Returns: + ``{"success": true}``, or 400 / 403 / 404 with a message. + """ decoded_token = request.decoded_token if not decoded_token: return make_response(jsonify({"success": False}), 401) @@ -817,13 +1093,32 @@ class UpdateTool(Resource): return make_response( jsonify({"success": False, "message": "Tool not found"}), 404, ) - if update_data: + if "name" in data and data["name"] != tool_doc.get("name"): + # The type decides what the config and the connection's + # credentials are used for; it is set once, on create. + return make_response(jsonify({"success": False, "message": _TYPE_IS_FIXED}), 400) + if update_data or "actions" in data: check_action(ra, "edit") + if "actions" in data: + # The stored actions are the schema, and a fixed value is + # the owner's (as on /api/update_tool_actions). + try: + update_data["actions"] = merge_submitted_actions( + _stored_actions(tool_doc), data["actions"], may_change_pins=ra.access == "owner", + ) + except PinChangeRefused as err: + return make_response(jsonify({"success": False, "message": str(err)}), 403) + except ValueError as err: + return make_response(jsonify({"success": False, "message": str(err)}), 400) if "config" in data: - tool_name = tool_doc.get("name", data.get("name")) + tool_name = tool_doc.get("name") existing_config = tool_doc.get("config", {}) or {} if tool_name == "mcp_tool": check_oauth_mcp_owner_only(ra, existing_config, data["config"]) + if _connection_server_moved(tool_doc, data["config"]): + return make_response(jsonify({"success": False, "message": _MOVE_THROUGH_MCP_SAVE}), 400) + if tool_name == "api_tool": + check_api_tool_fixed_values(ra, data["config"], existing_config) if tool_name == "api_tool" and not _api_tool_config_needs_credentials( data["config"], existing_config ): @@ -834,7 +1129,11 @@ class UpdateTool(Resource): config_requirements = ( tool_instance.get_config_requirements() if tool_instance else {} ) - has_existing_secrets = "encrypted_credentials" in existing_config + has_existing_secrets = ( + "encrypted_credentials" in existing_config or bool(tool_doc.get("connection_id")) + ) + # Validate before touching the connection: a rejected + # edit must not have rotated its credentials. if config_requirements: validation_errors = _validate_config( data["config"], config_requirements, @@ -849,6 +1148,20 @@ class UpdateTool(Resource): }), 400, ) + if tool_doc.get("connection_id"): + # The connection may back the owner's other tools too, + # so its secret stays the owner's to change even when + # an editor may change the tool's own credentials. + new_config = _update_connection_secrets( + conn, user, tool_doc, data["config"], config_requirements, + ) + if new_config is None: + return make_response( + jsonify({"success": False, "message": "Only the owner can change the credentials"}), + 403, + ) + data = {**data, "config": new_config} + update_data["config"] = _prepare_tool_config( tool_doc, data["config"], config_requirements ) @@ -891,6 +1204,15 @@ class UpdateToolConfig(Resource): ) @api.doc(description="Update the configuration of a tool") def post(self): + """Replace a tool's config, keeping stored secrets the client left out. + + A connection-backed tool's new key goes to its connection (the + owner's to change) and its server cannot be moved here; an api_tool's + fixed values are the owner's. + + Returns: + ``{"success": true}``, or 400 / 403 / 404 with a message. + """ decoded_token = request.decoded_token if not decoded_token: return make_response(jsonify({"success": False}), 401) @@ -931,12 +1253,18 @@ class UpdateToolConfig(Resource): jsonify({"success": False, "message": "Invalid server URL"}), 400, ) + if _connection_server_moved(tool_doc, data["config"]): + return make_response(jsonify({"success": False, "message": _MOVE_THROUGH_MCP_SAVE}), 400) + if tool_name == "api_tool": + check_api_tool_fixed_values(ra, data["config"], tool_doc.get("config")) tool_instance = tool_manager.tools.get(tool_name) config_requirements = ( tool_instance.get_config_requirements() if tool_instance else {} ) existing_config = tool_doc.get("config", {}) or {} - has_existing_secrets = "encrypted_credentials" in existing_config + has_existing_secrets = ( + "encrypted_credentials" in existing_config or bool(tool_doc.get("connection_id")) + ) if config_requirements: validation_errors = _validate_config( @@ -953,7 +1281,17 @@ class UpdateToolConfig(Resource): 400, ) - final_config = _prepare_tool_config(tool_doc, data["config"], config_requirements) + config = data["config"] + if tool_doc.get("connection_id"): + # The tool runs with its connection's key: a new one goes + # there (the owner's to change), not into this config. + config = _update_connection_secrets(conn, user, tool_doc, config, config_requirements) + if config is None: + return make_response( + jsonify({"success": False, "message": "Only the owner can change the credentials"}), + 403, + ) + final_config = _prepare_tool_config(tool_doc, config, config_requirements) repo.update(str(tool_doc["id"]), ra.owner_id, {"config": final_config}) except AccessDenied as err: @@ -1009,7 +1347,23 @@ class UpdateToolActions(Resource): # ``edit`` covers action on/off, descriptions and approval # (``require_approval``); actions carry no credentials. ra = require(conn, "tool", data["id"], user, "edit") - UserToolsRepository(conn).update(ra.resource_id, ra.owner_id, {"actions": data["actions"]}) + tool_doc = _load_owned_row(conn, ra) + if not tool_doc: + return make_response( + jsonify({"success": False, "message": "Tool not found"}), 404, + ) + # The stored actions are the schema: nothing can be added, + # and a fixed value is the owner's to set (an editor pinning + # a Telegram chat would redirect the owner's bot). + try: + actions = merge_submitted_actions( + _stored_actions(tool_doc), data["actions"], may_change_pins=ra.access == "owner", + ) + except PinChangeRefused as err: + return make_response(jsonify({"success": False, "message": str(err)}), 403) + except ValueError as err: + return make_response(jsonify({"success": False, "message": str(err)}), 400) + UserToolsRepository(conn).update(str(tool_doc["id"]), ra.owner_id, {"actions": actions}) except AccessDenied as err: return denied_response(err) except Exception as err: diff --git a/docsgpt/api/user/workflows/routes.py b/docsgpt/api/user/workflows/routes.py index 3941e786..4cb08bde 100644 --- a/docsgpt/api/user/workflows/routes.py +++ b/docsgpt/api/user/workflows/routes.py @@ -10,12 +10,20 @@ from docsgpt.agents.workflows.cel_evaluator import ( CelEvaluationError, validate_cel_expression, ) +from docsgpt.api.user import resource_access from docsgpt.api.user.resource_access import ( AccessDenied, + best_effort, + cached_resolves, can_use_ref, + named_ref_keys, + parse_confirmations, + plan_sponsors, resolve, + resource_states, + sponsor_audience, sponsor_details, - sponsors_after_save, + sponsor_refusal, ) from docsgpt.storage.db.base_repository import looks_like_uuid from docsgpt.storage.db.repositories.workflow_edges import WorkflowEdgesRepository @@ -125,6 +133,44 @@ def _node_refs(nodes: List[Dict]) -> List[Tuple[str, str]]: return refs +def _node_ref_details(nodes: List[Dict], visible: Optional[Set[str]] = None) -> Dict[str, List[Dict]]: + """Names of the tools and sources the graph's agent nodes reference. + + Looked up by id whoever owns them, so an editor's node pickers can show + (and remove) the owner's private tools and sources. Builtin tool ids + always resolve; any other id only when its ``":"`` key is in + ``visible`` (see ``resource_access.named_ref_keys``), so a node naming + someone else's resource never reveals its name. + + Args: + nodes: Nodes in builder shape. + visible: Keys whose names may be read; None allows every id. + + Returns: + dict: ``tools`` as ``[{id, name, display_name}]`` and ``sources`` as + ``[{id, name}]``, each id once. + """ + from docsgpt.agents.default_tools import is_synthesized_tool_id + from docsgpt.api.user.base import resolve_source_details, resolve_tool_details + + tool_ids: List[str] = [] + source_ids: List[str] = [] + for resource_type, resource_id in _node_refs(nodes): + if ( + visible is not None + and not (resource_type == "tool" and is_synthesized_tool_id(resource_id)) + and f"{resource_type}:{resource_id.lower()}" not in visible + ): + continue + bucket = tool_ids if resource_type == "tool" else source_ids + if resource_id not in bucket: + bucket.append(resource_id) + return { + "tools": resolve_tool_details(tool_ids), + "sources": resolve_source_details(source_ids), + } + + def _new_node_ref_denied( conn, previous_nodes: List[Dict], new_nodes: List[Dict], caller: str ) -> Optional[AccessDenied]: @@ -133,13 +179,16 @@ def _new_node_ref_denied( A workflow runs as its owner, so an editor saving the owner's graph must not reference the owner's private tools or sources: the caller's own access counts (``use_in_own`` for a tool, ``use`` for a source), not the - owner's. Refs already in the stored graph stay, like an agent's. + owner's. The owner's own saves are checked the same way, so no graph + names a resource its owner never could use. Refs already in the stored + graph stay, like an agent's. Args: conn: Open database connection. - previous_nodes: The stored graph's nodes, in builder shape. + previous_nodes: The stored graph's nodes, in builder shape (empty + when creating the workflow). new_nodes: The nodes being saved. - caller: The editor saving. + caller: The user saving, owner or editor. Returns: An :class:`AccessDenied` to return, or None when every new ref is fine. @@ -600,6 +649,9 @@ class WorkflowList(Resource): try: with db_session() as conn: + denied = _new_node_ref_denied(conn, [], nodes_data, user_id) + if denied is not None: + return _denied(denied) repo = WorkflowsRepository(conn) workflow = repo.create(user_id, name, description=description) pg_workflow_id = str(workflow["id"]) @@ -631,16 +683,45 @@ class WorkflowDetail(Resource): edges = WorkflowEdgesRepository(conn).find_by_version( pg_workflow_id, graph_version, ) - sponsored = sponsor_details(conn, "workflow", workflow) + serialized_nodes = [serialize_node(n) for n in nodes] + # Edit-page detail (sponsors, run state, names of node + # resources) only for people who may edit the workflow. + sponsored: list = [] + states: list = [] + audience = None + visible: Optional[Set[str]] = None + with cached_resolves(): + if resource_access.holder_editable_by(conn, "workflow", workflow, user_id): + refs = _node_refs(serialized_nodes) + sponsored = sponsor_details(conn, "workflow", workflow, viewer=user_id) + # Run state never fails the read; node names follow + # the same rule as the state's names. + states = best_effort( + conn, "the workflow's resource states", + lambda: resource_states(conn, "workflow", workflow, refs, user_id), [], + ) + audience = best_effort( + conn, "the workflow's audience", + lambda: sponsor_audience(conn, "workflow", workflow, states, sponsored), None, + ) + visible = named_ref_keys(states) + ref_details = ( + _node_ref_details(serialized_nodes, visible) + if visible is not None + else {"tools": [], "sources": []} + ) except Exception as err: return _workflow_error_response("Failed to fetch workflow", err) return success_response( { "workflow": serialize_workflow(workflow), - "nodes": [serialize_node(n) for n in nodes], + "nodes": serialized_nodes, "edges": [serialize_edge(e) for e in edges], "resource_sponsors": sponsored, + "resource_states": states, + "ref_details": ref_details, + **({"sponsor_audience": audience} if audience is not None else {}), } ) @@ -674,16 +755,35 @@ class WorkflowDetail(Resource): nodes_data = normalize_agent_node_json_schemas(nodes_data) pg_workflow_id = str(workflow["id"]) current_graph_version = get_workflow_graph_version(workflow) - if acting != user_id: - previous_nodes = [ - serialize_node(n) - for n in WorkflowNodesRepository(conn).find_by_version( - pg_workflow_id, current_graph_version, - ) - ] - denied = _new_node_ref_denied(conn, previous_nodes, nodes_data, user_id) - if denied is not None: - return _denied(denied) + previous_nodes = [ + serialize_node(n) + for n in WorkflowNodesRepository(conn).find_by_version( + pg_workflow_id, current_graph_version, + ) + ] + # Every newly referenced node tool or source must be one the + # caller may use, the owner included. + denied = _new_node_ref_denied(conn, previous_nodes, nodes_data, user_id) + if denied is not None: + return _denied(denied) + # A node tool/source the owner can't use runs as the editor + # who attached it (its sponsor): only someone who owns or + # edits it, and only once ``confirm_sponsor`` lists it. + plan = plan_sponsors( + conn, + "workflow", + workflow, + acting, + user_id, + _node_refs(nodes_data), + previous_refs=_node_refs(previous_nodes), + confirmed=parse_confirmations(data.get("confirm_sponsor")), + ) + refusal = sponsor_refusal(conn, "workflow", workflow, plan) + if refusal is not None: + body, status = refusal + body.setdefault("error", body["message"]) + return make_response(jsonify(body), status) next_graph_version = current_graph_version + 1 _write_graph( @@ -695,13 +795,8 @@ class WorkflowDetail(Resource): "description": description, "current_graph_version": next_graph_version, } - # A node tool/source the owner can't use runs as the editor - # who attached it (its sponsor); record who that is. - sponsors = sponsors_after_save( - conn, "workflow", workflow, acting, user_id, _node_refs(nodes_data) - ) - if sponsors != (workflow.get("resource_sponsors") or {}): - workflow_fields["resource_sponsors"] = sponsors + if plan.sponsors != (workflow.get("resource_sponsors") or {}): + workflow_fields["resource_sponsors"] = plan.sponsors repo.update(pg_workflow_id, acting, workflow_fields) WorkflowNodesRepository(conn).delete_other_versions( pg_workflow_id, next_graph_version, diff --git a/docsgpt/api/v1/routes.py b/docsgpt/api/v1/routes.py index 2b943088..4750f8a9 100644 --- a/docsgpt/api/v1/routes.py +++ b/docsgpt/api/v1/routes.py @@ -261,7 +261,9 @@ def chat_completions(): internal_data["persist"] = True try: - processor = StreamProcessor(internal_data, decoded_token, trace_source="v1") + # The token is the owner's, so tell the processor the caller is a key + # holder: their writes on the owner's accounts need the allowlist. + processor = StreamProcessor(internal_data, decoded_token, trace_source="v1", external_caller=True) flush_trace_after_request(processor) # Set when this request took the resume claim, so a refusal can release it. claimed_conversation_id = None diff --git a/docsgpt/app.py b/docsgpt/app.py index 88e2d5cd..1e018bf8 100644 --- a/docsgpt/app.py +++ b/docsgpt/app.py @@ -29,6 +29,7 @@ from docsgpt.api.scim import scim_bp # noqa: E402 from docsgpt.api.user.authz import ROLE_USER, resolve_roles # noqa: E402 from docsgpt.api.user.routes import user # noqa: E402 from docsgpt.api.connector.routes import connector # noqa: E402 +from docsgpt.api.connector import connections as _connections # noqa: E402,F401 from docsgpt.api.v1 import v1_bp # noqa: E402 from docsgpt.celery_init import celery # noqa: E402 from docsgpt.core.secret_key import resolve_jwt_secret_key # noqa: E402 @@ -183,6 +184,27 @@ if settings.AUTH_TYPE == "simple_jwt": print(f"Generated Simple JWT Token: {SIMPLE_JWT_TOKEN}") +def _warn_default_encryption_key() -> None: + """Say when stored credentials are sealed with the public default key.""" + from docsgpt.security.encryption import is_default_encryption_key + + if not is_default_encryption_key(): + return + if settings.AUTH_TYPE: + logging.getLogger(__name__).warning( + "ENCRYPTION_SECRET_KEY is the public default: connecting services is refused until you set your " + "own value (then run `docsgpt connectors reencrypt`)." + ) + else: + logging.getLogger(__name__).warning( + "ENCRYPTION_SECRET_KEY is the public default. Stored connector credentials are only as safe as " + "that key; set your own value before exposing this install." + ) + + +_warn_default_encryption_key() + + @app.route("/") def home(): if request.remote_addr in ("0.0.0.0", "127.0.0.1", "localhost", "172.18.0.1"): @@ -209,6 +231,9 @@ def get_config(): "hybrid_available": settings.VECTOR_STORE == "pgvector", "tts_available": TTSCreator.is_enabled(settings.TTS_PROVIDER), "stt_available": STTCreator.is_enabled(settings.STT_PROVIDER), + # Lets a frontend built before the Connectors page run against this + # backend, and a new frontend hide the page against an older one. + "connectors_enabled": True, } if settings.AUTH_TYPE == "oidc": response["oidc"] = { diff --git a/docsgpt/cli.py b/docsgpt/cli.py index d837642d..c1264dc7 100644 --- a/docsgpt/cli.py +++ b/docsgpt/cli.py @@ -285,6 +285,28 @@ def _add_deploy_commands(commands) -> None: dev.set_defaults(func=_deploy("dev"), deploy=True) +def _connectors(args: argparse.Namespace) -> int: + """``docsgpt connectors reencrypt``: move every credential onto the current key.""" + if getattr(args, "connectors_action", None) != "reencrypt": + print("usage: docsgpt connectors reencrypt", file=sys.stderr) + return 2 + from docsgpt.connectors.service import reencrypt_all + + counts = reencrypt_all() + print( + f"docsgpt: re-encrypted {counts['rewritten']} connection(s), " + f"{counts['current']} already current, {counts['failed']} unreadable", + file=sys.stderr, + ) + if counts["failed"]: + print( + "docsgpt: unreadable connections were marked 'Reconnect needed'; " + "their owners must reconnect them.", + file=sys.stderr, + ) + return 1 if counts["failed"] else 0 + + def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(prog="docsgpt", description="DocsGPT: private AI for agents, assistants and search.") parser.add_argument("--version", action="version", version=f"docsgpt {__version__}") @@ -315,6 +337,14 @@ def build_parser() -> argparse.ArgumentParser: migrate.add_argument("--no-create", dest="create_db", action="store_false", help="fail instead of creating a missing database") migrate.set_defaults(func=_migrate) + connectors = commands.add_parser("connectors", help="manage stored connector credentials") + connector_actions = connectors.add_subparsers(dest="connectors_action", metavar="") + connector_actions.add_parser( + "reencrypt", + help="rewrite every stored credential with ENCRYPTION_SECRET_KEY (after a key rotation)", + ) + connectors.set_defaults(func=_connectors) + for name, (module, help_text) in SCRIPTS.items(): commands.add_parser(name, help=f"{help_text} (docsgpt.scripts.{module})", add_help=False) diff --git a/docsgpt/connectors/__init__.py b/docsgpt/connectors/__init__.py new file mode 100644 index 00000000..fc58c5c0 --- /dev/null +++ b/docsgpt/connectors/__init__.py @@ -0,0 +1,6 @@ +"""Connectors: one place to connect accounts that feed Sources and Tools. + +``catalog`` declares the services, ``service`` manages connections (the rows +of ``connector_sessions``), ``permissions`` classifies tool actions as read +or write. +""" diff --git a/docsgpt/connectors/attribution.py b/docsgpt/connectors/attribution.py new file mode 100644 index 00000000..9719a8a6 --- /dev/null +++ b/docsgpt/connectors/attribution.py @@ -0,0 +1,68 @@ +"""Which connector a retrieved chunk came from, for "From Google Drive" citations. + +Only the connector's key and display name are attached, never the account +behind the connection, so shared conversations show the same attribution +without revealing whose Drive it was. +""" + +from __future__ import annotations + +import logging +from typing import Optional + +from sqlalchemy import text + +from docsgpt.storage.db.base_repository import looks_like_uuid +from docsgpt.storage.db.session import db_readonly + +logger = logging.getLogger(__name__) + + +def connector_labels(source_ids: list[str]) -> dict[str, dict]: + """``{source_id: {"connector_key", "connector_name"}}`` for synced sources. + + Sources that do not come from a connection are left out. Never raises: + attribution is decoration, and a lookup failure must not break an answer. + """ + from docsgpt.connectors import service + + ids = [str(i) for i in source_ids if i and looks_like_uuid(str(i))] + if not ids: + return {} + try: + with db_readonly() as conn: + rows = conn.execute( + text( + "SELECT s.id AS source_id, cs.* FROM sources s " + "JOIN connector_sessions cs ON cs.id = s.connection_id " + "WHERE s.id = ANY(CAST(:ids AS uuid[]))" + ), + {"ids": ids}, + ).fetchall() + except Exception: + logger.warning("connector attribution lookup failed", exc_info=True) + return {} + labels = {} + for row in rows: + data = dict(row._mapping) + public = service.serialize_connection({**data, "id": data["id"]}) + labels[str(data["source_id"])] = { + "connector_key": public["connector_key"], + "connector_name": public["name"], + } + return labels + + +class ConnectorLabelCache: + """Per-search memo so each source is looked up once.""" + + def __init__(self) -> None: + self._labels: dict[str, dict] = {} + + def for_source(self, source_id: Optional[str]) -> dict: + if not source_id: + return {} + key = str(source_id) + if key not in self._labels: + self._labels[key] = connector_labels([key]).get(key, {}) + return self._labels[key] diff --git a/docsgpt/connectors/catalog.py b/docsgpt/connectors/catalog.py new file mode 100644 index 00000000..1e4e66d9 --- /dev/null +++ b/docsgpt/connectors/catalog.py @@ -0,0 +1,522 @@ +"""The connector catalog: every service DocsGPT can connect to. + +A connector is something the user connects once (an OAuth sign-in, an API key +or an MCP server). Each connection it produces can feed Sources (content +synced into DocsGPT) and Tools (actions an agent can take). The definitions +here are declarative: they say how a connector signs in, which server +settings it needs, what it can sync and which tools it creates, so the API +and the frontend never hard-code a list of services. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Iterable, Optional +from urllib.parse import urlparse + +import yaml + +from docsgpt.core.settings import settings + +CATEGORIES = ("files", "knowledge", "projects", "dev", "business", "messaging", "database", "search", "custom") +AUTH_KINDS = ("oauth", "mcp_oauth", "api_key", "none", "mcp") +PUBLISHERS = ("built_in", "preset", "custom") + + +@dataclass(frozen=True) +class CredentialField: + """One field the user fills in to connect an ``api_key`` connector. + + Attributes: + key: Name the credential is stored and passed to the tool or loader + under, e.g. ``token`` or ``aws_access_key_id``. + label: English label; the frontend shows the translated + ``settings.connectors.fields.`` when it has one. + secret: Masked in the form and never returned by the API. + required: Whether connecting fails without it. + parameter: A tool parameter this field sets. When the connection has + a value for it, every call through the connection uses that + value and the model is not asked for the parameter (Telegram's + default chat). + hint: English help shown under the field; the frontend shows the + translated ``settings.connectors.fieldHints._``. + """ + + key: str + label: str + secret: bool = True + required: bool = True + parameter: Optional[str] = None + hint: Optional[str] = None + + def to_dict(self) -> dict: + return { + "key": self.key, + "label": self.label, + "secret": self.secret, + "required": self.required, + "parameter": self.parameter, + "hint": self.hint, + } + + +@dataclass(frozen=True) +class ConnectorDefinition: + """How one connector signs in and what a connection to it provides. + + Attributes: + key: Catalog key, stored on each connection as ``connector_key``. + name: Display name. + description: One line shown on the catalog card. + icon: Frontend asset key (``assets/connectors/.svg``). + category: One of :data:`CATEGORIES`. + auth_kind: ``oauth`` (server-side OAuth app), ``mcp_oauth`` (MCP + OAuth with dynamic client registration), ``api_key`` (fields the + user pastes), ``none`` or ``mcp`` (a custom MCP server whose auth + the user picks). + capabilities: Any of ``sync``, ``read``, ``write``. + credential_fields: Fields asked for by ``api_key`` connectors. + required_settings: Server settings that must be set for the + connector to be usable, e.g. ``GOOGLE_CLIENT_ID``. + sync_ingestor: Ingest loader a synced source uses (``google_drive``, + ``s3``), or None when the connector cannot sync. + default_sync_frequency: Sync frequency preselected in the wizard. + setup_fields: Per-source fields asked when choosing what to sync + (an S3 bucket, Reddit search queries). + tool_templates: ``user_tools`` names created on connect. + setup: What the wizard does after sign-in: ``tools`` is ``auto`` + (created and enabled), ``ask`` or ``off``; ``sync`` likewise. + mcp_url: MCP endpoint for presets, and for a built-in connector whose + tool is its service's own MCP server (GitHub). + mcp_write_url: The same server's endpoint that also offers write + actions, which a connection opts into and an admin can forbid + (GitHub's full server beside its read-only one). ``mcp_url`` + stays the default. + publisher: ``built_in``, ``preset`` or ``custom``. + docs_url: Setup guide for admins. + oauth_scopes: Scopes an MCP preset requests. + part_of: Another connector this one is shown under, for one service + offered two ways (the Atlassian MCP preset under Confluence). + oauth_settings: Server settings that add a second sign-in, OAuth, + to an ``api_key`` connector (GitHub's "Sign in with GitHub" + through a GitHub App). Unlike ``required_settings`` the + connector works without them, with pasted credentials only. + """ + + key: str + name: str + description: str + icon: str + category: str + auth_kind: str + capabilities: tuple[str, ...] = () + credential_fields: tuple[CredentialField, ...] = () + required_settings: tuple[str, ...] = () + sync_ingestor: Optional[str] = None + default_sync_frequency: str = "weekly" + setup_fields: tuple[CredentialField, ...] = () + tool_templates: tuple[str, ...] = () + setup: dict = field(default_factory=lambda: {"tools": "auto", "sync": "ask"}) + mcp_url: Optional[str] = None + mcp_write_url: Optional[str] = None + publisher: str = "built_in" + docs_url: Optional[str] = None + oauth_scopes: tuple[str, ...] = () + part_of: Optional[str] = None + oauth_settings: tuple[str, ...] = () + + @property + def oauth_configured(self) -> bool: + """Whether the optional OAuth sign-in has every server setting it needs.""" + return bool(self.oauth_settings) and all(getattr(settings, name, None) for name in self.oauth_settings) + + @property + def sign_in_methods(self) -> list[str]: + """How a user can connect, preferred first: ``auth_kind``, after OAuth when that is set up.""" + return ["oauth", self.auth_kind] if self.oauth_configured else [self.auth_kind] + + @property + def missing_settings(self) -> list[str]: + """Server settings this connector needs that are not set.""" + return [name for name in self.required_settings if not getattr(settings, name, None)] + + @property + def configured(self) -> bool: + """Whether every required server setting is set.""" + return not self.missing_settings + + @property + def mcp_base_url(self) -> Optional[str]: + """``scheme://host`` of the preset's MCP endpoint (the MCP session key).""" + return base_url(self.mcp_url) if self.mcp_url else None + + def to_dict(self) -> dict: + """Serialise the parts the frontend needs (never server secrets).""" + return { + "key": self.key, + "name": self.name, + "description": self.description, + "icon": self.icon, + "category": self.category, + "auth_kind": self.auth_kind, + "capabilities": list(self.capabilities), + "credential_fields": [f.to_dict() for f in self.credential_fields], + "setup_fields": [f.to_dict() for f in self.setup_fields], + "sync_ingestor": self.sync_ingestor, + "default_sync_frequency": self.default_sync_frequency, + "tool_templates": list(self.tool_templates), + "setup": dict(self.setup), + "mcp_url": self.mcp_url, + "writes_opt_in": bool(self.mcp_write_url), + "publisher": self.publisher, + "docs_url": self.docs_url, + "oauth_scopes": list(self.oauth_scopes), + "part_of": self.part_of, + "sign_in_methods": self.sign_in_methods, + } + + +def base_url(url: Optional[str]) -> str: + """``scheme://netloc`` of ``url``, the key MCP sessions are stored under.""" + parsed = urlparse(url or "") + if not parsed.scheme or not parsed.netloc: + return "" + return f"{parsed.scheme}://{parsed.netloc}" + + +_DOCS = "https://docs.docsgpt.cloud/Guides/Connectors" +GITHUB_MCP_URL = "https://api.githubcopilot.com/mcp/readonly" +GITHUB_MCP_WRITE_URL = "https://api.githubcopilot.com/mcp/" + +# Tool templates any connector's server can use (an MCP server, an OpenAPI +# spec): a tool made from one belongs to its connection, not to a connector. +_GENERIC_TOOL_TEMPLATES = frozenset({"mcp_tool", "api_tool"}) + +_BUILT_IN: tuple[ConnectorDefinition, ...] = ( + ConnectorDefinition( + key="google_drive", + name="Google Drive", + description="Sync Docs, Sheets and PDFs into Knowledge.", + icon="drive", + category="files", + auth_kind="oauth", + capabilities=("sync",), + required_settings=("GOOGLE_CLIENT_ID", "GOOGLE_CLIENT_SECRET"), + sync_ingestor="google_drive", + setup={"tools": "off", "sync": "ask"}, + docs_url=f"{_DOCS}#google-drive", + ), + ConnectorDefinition( + key="share_point", + name="SharePoint", + description="Sync files from SharePoint sites and OneDrive into Knowledge.", + icon="sharepoint", + category="files", + auth_kind="oauth", + capabilities=("sync",), + required_settings=("MICROSOFT_CLIENT_ID", "MICROSOFT_CLIENT_SECRET"), + sync_ingestor="share_point", + setup={"tools": "off", "sync": "ask"}, + docs_url=f"{_DOCS}#sharepoint-and-onedrive", + ), + ConnectorDefinition( + key="confluence", + name="Confluence", + description="Sync Confluence spaces and pages into Knowledge.", + icon="confluence", + category="knowledge", + auth_kind="oauth", + capabilities=("sync",), + required_settings=("CONFLUENCE_CLIENT_ID", "CONFLUENCE_CLIENT_SECRET"), + sync_ingestor="confluence", + setup={"tools": "off", "sync": "ask"}, + docs_url=f"{_DOCS}#confluence", + ), + ConnectorDefinition( + key="github", + name="GitHub", + description="Sync repositories into Knowledge and let agents read code, issues and pull requests.", + icon="github", + category="dev", + # A token works with no admin setup; a GitHub App adds Sign in with GitHub. + auth_kind="api_key", + capabilities=("sync", "read"), + credential_fields=(CredentialField("access_token", "Personal access token"),), + oauth_settings=("GITHUB_CLIENT_ID", "GITHUB_CLIENT_SECRET", "GITHUB_APP_SLUG"), + sync_ingestor="github", + setup_fields=(CredentialField("repo_url", "Repository", secret=False),), + # GitHub's own MCP server, read-only unless the connection opts into + # changes (issues, comments, pull requests) and an admin allows them. + tool_templates=("mcp_tool",), + mcp_url=GITHUB_MCP_URL, + mcp_write_url=GITHUB_MCP_WRITE_URL, + setup={"tools": "ask", "sync": "ask"}, + docs_url=f"{_DOCS}#github", + ), + ConnectorDefinition( + key="s3", + name="Amazon S3", + description="Sync documents from an S3 bucket into Knowledge.", + icon="s3", + category="files", + auth_kind="api_key", + capabilities=("sync",), + credential_fields=( + CredentialField("aws_access_key_id", "Access key ID", secret=False), + CredentialField("aws_secret_access_key", "Secret access key"), + CredentialField("region", "Region", secret=False, required=False), + CredentialField("endpoint_url", "Custom endpoint URL", secret=False, required=False), + ), + sync_ingestor="s3", + setup_fields=( + CredentialField("bucket", "Bucket", secret=False), + CredentialField("prefix", "Path prefix", secret=False, required=False), + ), + setup={"tools": "off", "sync": "ask"}, + docs_url=f"{_DOCS}#amazon-s3", + ), + ConnectorDefinition( + key="reddit", + name="Reddit", + description="Sync Reddit posts that match your searches into Knowledge.", + icon="reddit", + category="search", + auth_kind="api_key", + capabilities=("sync",), + credential_fields=( + CredentialField("client_id", "Client ID", secret=False), + CredentialField("client_secret", "Client secret"), + CredentialField("user_agent", "User agent", secret=False), + ), + sync_ingestor="reddit", + setup_fields=( + CredentialField("search_queries", "Search queries", secret=False), + CredentialField("number_posts", "Number of posts", secret=False), + ), + setup={"tools": "off", "sync": "ask"}, + ), + ConnectorDefinition( + key="brave", + name="Brave Search", + description="Search the web and images with the Brave Search API.", + icon="tool_brave", + category="search", + auth_kind="api_key", + capabilities=("read",), + credential_fields=(CredentialField("token", "API key"),), + tool_templates=("brave",), + setup={"tools": "auto", "sync": "off"}, + ), + ConnectorDefinition( + key="telegram", + name="Telegram", + description="Send messages and images to a Telegram chat.", + icon="tool_telegram", + category="messaging", + auth_kind="api_key", + capabilities=("write",), + credential_fields=( + CredentialField("token", "Bot token"), + CredentialField( + "chat_id", + "Default chat ID", + secret=False, + required=False, + parameter="chat_id", + hint=( + "Optional. Messages go to this chat, and the AI cannot pick another. Add the bot to the " + "chat, send it a message, then find the chat's id in " + "https://api.telegram.org/bot/getUpdates." + ), + ), + ), + tool_templates=("telegram",), + setup={"tools": "auto", "sync": "off"}, + ), + ConnectorDefinition( + key="ntfy", + name="ntfy", + description="Send push notifications through an ntfy server.", + icon="tool_ntfy", + category="messaging", + auth_kind="api_key", + capabilities=("write",), + credential_fields=(CredentialField("token", "Access token"),), + tool_templates=("ntfy",), + setup={"tools": "auto", "sync": "off"}, + ), + ConnectorDefinition( + key="postgres", + name="PostgreSQL", + description="Read the schema and run SQL against a Postgres database.", + icon="tool_postgres", + category="database", + auth_kind="api_key", + capabilities=("read", "write"), + credential_fields=(CredentialField("token", "Connection string"),), + tool_templates=("postgres",), + setup={"tools": "auto", "sync": "off"}, + ), + ConnectorDefinition( + key="custom_mcp", + name="MCP server", + description="Connect any remote Model Context Protocol server.", + icon="tool_mcp_tool", + category="custom", + auth_kind="mcp", + capabilities=("read", "write"), + tool_templates=("mcp_tool",), + setup={"tools": "auto", "sync": "off"}, + publisher="custom", + ), + ConnectorDefinition( + key="custom_openapi", + name="OpenAPI / REST", + description="Import an OpenAPI spec and call its endpoints as tools.", + icon="tool_api_tool", + category="custom", + auth_kind="none", + capabilities=("read", "write"), + tool_templates=("api_tool",), + setup={"tools": "ask", "sync": "off"}, + publisher="custom", + ), +) + +_PRESETS_FILE = Path(__file__).parent / "presets" / "mcp.yaml" + + +def _load_presets(path: Optional[Path] = None) -> tuple[ConnectorDefinition, ...]: + """Read the curated MCP server presets shipped with the repository.""" + path = path or _PRESETS_FILE + if not path.exists(): + return () + with path.open(encoding="utf-8") as fh: + entries = yaml.safe_load(fh) or [] + presets = [] + for entry in entries: + presets.append( + ConnectorDefinition( + key=entry["key"], + name=entry["name"], + description=entry["description"], + icon=entry.get("icon") or "tool_mcp_tool", + category=entry.get("category", "knowledge"), + auth_kind=entry.get("auth_kind", "mcp_oauth"), + capabilities=tuple(entry.get("capabilities") or ("read", "write")), + tool_templates=("mcp_tool",), + # Linear also syncs into Knowledge, read through its MCP server. + sync_ingestor=entry.get("sync_ingestor"), + setup={"tools": "auto", "sync": "ask" if entry.get("sync_ingestor") else "off"}, + mcp_url=entry["mcp_url"], + publisher="preset", + docs_url=entry.get("docs_url"), + oauth_scopes=tuple(entry.get("oauth_scopes") or ()), + part_of=entry.get("part_of"), + ) + ) + return tuple(presets) + + +_REGISTRY: dict[str, ConnectorDefinition] = {} + + +def _registry() -> dict[str, ConnectorDefinition]: + if not _REGISTRY: + for definition in (*_BUILT_IN, *_load_presets()): + _REGISTRY[definition.key] = definition + return _REGISTRY + + +def all_definitions() -> list[ConnectorDefinition]: + """Every catalog entry, built-ins first, then presets.""" + return list(_registry().values()) + + +def get_definition(key: Optional[str]) -> Optional[ConnectorDefinition]: + """The definition for ``key``, or None when it is not in the catalog.""" + if not key: + return None + return _registry().get(key) + + +def preset_for_url(url: Optional[str]) -> Optional[ConnectorDefinition]: + """The MCP preset whose server shares ``url``'s base URL, if any.""" + target = base_url(url) + if not target: + return None + for definition in _registry().values(): + if definition.publisher == "preset" and definition.mcp_base_url == target: + return definition + return None + + +def definition_for_tool(tool_name: str) -> Optional[ConnectorDefinition]: + """The built-in connector that provides the ``user_tools`` template ``tool_name``.""" + if tool_name in _GENERIC_TOOL_TEMPLATES: + return None + for definition in _BUILT_IN: + if definition.publisher == "built_in" and tool_name in definition.tool_templates: + return definition + return None + + +def parameter_fields(key: Optional[str]) -> tuple[CredentialField, ...]: + """The credential fields of connector ``key`` that set a tool parameter.""" + definition = get_definition(key) + if definition is None: + return () + return tuple(f for f in definition.credential_fields if f.parameter) + + +def connector_key_for_row(row: dict) -> Optional[str]: + """Catalog key for a ``connector_sessions`` row. + + Rows written before ``0040_connections`` carry no key; they are named from + ``provider``. A custom MCP row whose server matches a preset is reported + as that preset. + """ + key = row.get("connector_key") + provider = row.get("provider") or "" + if not key or key.startswith("mcp:"): + # ``mcp:`` is a provider value, never a catalog key. + key = None + if provider.startswith("mcp:"): + key = "custom_mcp" + elif get_definition(provider): + key = provider + if key == "custom_mcp": + preset = preset_for_url(row.get("server_url") or provider[4:]) + if preset: + return preset.key + return key + + +def tool_connector_keys() -> set[str]: + """``user_tools`` names that belong to a built-in service connector.""" + return { + name + for definition in _BUILT_IN + if definition.publisher == "built_in" + for name in definition.tool_templates + if name not in _GENERIC_TOOL_TEMPLATES + } + + +def iter_by_category(definitions: Iterable[ConnectorDefinition]) -> dict[str, list[ConnectorDefinition]]: + """Group definitions by category, keeping :data:`CATEGORIES` order.""" + grouped: dict[str, list[ConnectorDefinition]] = {c: [] for c in CATEGORIES} + for definition in definitions: + grouped.setdefault(definition.category, []).append(definition) + return grouped + + +def reset_registry_for_tests() -> None: + """Drop the cached registry so a test can load a different presets file.""" + _REGISTRY.clear() + + +def to_public(definition: ConnectorDefinition, **extra: Any) -> dict: + """Definition dict plus request-specific fields (availability, counts).""" + return {**definition.to_dict(), **extra} diff --git a/docsgpt/connectors/github.py b/docsgpt/connectors/github.py new file mode 100644 index 00000000..a43bdf81 --- /dev/null +++ b/docsgpt/connectors/github.py @@ -0,0 +1,132 @@ +"""GitHub account lookups for the GitHub connector: who a token is, what it can read. + +A connection signs in with a personal access token (``api_key``) or through +the GitHub App (``oauth``). A token lists the repositories it was granted; +an App sign-in lists the repositories of the App's installations the user +can see, which the user chooses on GitHub when installing the App. +""" + +from __future__ import annotations + +from typing import Optional + +import requests + +from docsgpt.connectors.service import TransientConnectionError + +API_URL = "https://api.github.com" +_PAGE_SIZE = 100 +# A picker, not an export: enough for any one person's list. +MAX_REPOSITORIES = 1000 + + +class TokenRejected(ValueError): + """GitHub answered 401: the token is wrong, expired or revoked.""" + + +def _headers(token: str) -> dict: + return { + "Authorization": f"Bearer {token}", + "Accept": "application/vnd.github+json", + "X-GitHub-Api-Version": "2022-11-28", + } + + +def _get(url: str, token: str, params: Optional[dict] = None): + """GET ``url`` as ``token`` and return its JSON. + + Raises: + TokenRejected: 401. + TransientConnectionError: Network trouble, rate limiting or a 5xx. + ValueError: Any other refusal. + """ + try: + response = requests.get(url, headers=_headers(token), params=params, timeout=30) + except requests.RequestException as exc: + raise TransientConnectionError(f"GitHub did not answer: {type(exc).__name__}") from exc + if response.status_code == 401: + raise TokenRejected("GitHub did not accept this token.") + if response.status_code == 429 or response.status_code >= 500 or ( + response.status_code == 403 and response.headers.get("X-RateLimit-Remaining") == "0" + ): + raise TransientConnectionError(f"GitHub is busy ({response.status_code}). Try again.") + if response.status_code >= 400: + raise ValueError(f"GitHub refused the request ({response.status_code}).") + return response.json() + + +def token_account(token: Optional[str]) -> str: + """The login of the account ``token`` belongs to. + + Raises: + TokenRejected: No token, or GitHub did not accept it. + TransientConnectionError: GitHub could not be reached. + """ + if not token or not str(token).strip(): + raise TokenRejected("Paste a GitHub token.") + user = _get(f"{API_URL}/user", str(token).strip()) + login = user.get("login") if isinstance(user, dict) else None + if not login: + raise TokenRejected("GitHub did not say whose token this is.") + return str(login) + + +def _summary(repo: dict) -> dict: + return { + "full_name": repo.get("full_name"), + "private": bool(repo.get("private")), + "description": repo.get("description") or "", + "default_branch": repo.get("default_branch") or "", + "updated_at": repo.get("pushed_at") or repo.get("updated_at"), + "html_url": repo.get("html_url") or "", + } + + +def _paged(url: str, token: str, key: Optional[str] = None, params: Optional[dict] = None): + """Every item of a paginated list endpoint, up to :data:`MAX_REPOSITORIES`.""" + page = 1 + fetched = 0 + while fetched < MAX_REPOSITORIES: + payload = _get(url, token, {**(params or {}), "per_page": _PAGE_SIZE, "page": page}) + items = payload.get(key, []) if key else payload + if not isinstance(items, list): + return + for item in items: + if isinstance(item, dict): + fetched += 1 + yield item + if len(items) < _PAGE_SIZE: + return + page += 1 + + +def list_repositories(token: str, *, app: bool) -> list[dict]: + """Repositories ``token`` can read, most recently pushed first. + + Args: + token: The connection's access token. + app: Whether it is a GitHub App user token, whose repositories are + those of the App's installations. + + Returns: + ``{full_name, private, description, default_branch, updated_at, + html_url}`` per repository. + + Raises: + TokenRejected: GitHub did not accept the token. + TransientConnectionError: GitHub could not be reached. + """ + if app: + raw = [] + for installation in _paged(f"{API_URL}/user/installations", token, "installations"): + raw.extend(_paged( + f"{API_URL}/user/installations/{installation.get('id')}/repositories", token, "repositories", + )) + else: + raw = list(_paged(f"{API_URL}/user/repos", token, params={"sort": "pushed"})) + seen: dict[str, dict] = {} + for repo in raw: + name = repo.get("full_name") + if name and name not in seen: + seen[name] = _summary(repo) + return sorted(seen.values(), key=lambda r: r["updated_at"] or "", reverse=True)[:MAX_REPOSITORIES] diff --git a/docsgpt/connectors/linear.py b/docsgpt/connectors/linear.py new file mode 100644 index 00000000..fec52efe --- /dev/null +++ b/docsgpt/connectors/linear.py @@ -0,0 +1,299 @@ +"""Linear through its MCP server: what a sync can pick, and reading issues and documents. + +Linear's hosted MCP server (``mcp.linear.app``) is its own OAuth issuer, and +the tokens it grants are for that server. Knowledge sync therefore reads +Linear through the same MCP tools the agents use (``list_issues``, +``get_issue``, ``list_comments``, ``list_documents``...), signed in with the +Linear connection, so one sign-in powers both and nobody registers an OAuth +app. Linear publishes no output schema for these tools, so records are read +defensively: a value may be a string or an object with a ``name``. +""" + +from __future__ import annotations + +import json +import re +from typing import Any, AsyncIterator, Optional + +from docsgpt.connectors import catalog + +LINEAR_CONNECTOR = "mcp:linear" +LINEAR_MCP_URL = "https://mcp.linear.app/mcp" +# Enough for a team's working set; a sync is a full read, so it stays bounded. +MAX_ISSUES = 500 +MAX_DOCUMENTS = 100 +MAX_COMMENTS = 100 +# Teams and projects offered in the picker. +MAX_PICKER_ITEMS = 250 +PAGE_SIZE = 100 +_MAX_PAGES = 50 + +_IDENTIFIER = re.compile(r"^[A-Za-z][A-Za-z0-9_]*-\d+$") +_TRUNCATED = re.compile(r"\(truncated\b", re.IGNORECASE) +_LIST_KEYS = ("nodes", "items", "results", "data") +_PRIORITIES = {0: "No priority", 1: "Urgent", 2: "High", 3: "Medium", 4: "Low"} + + +class LinearSyncError(ValueError): + """Linear's MCP server cannot do what the sync needs (a tool or filter is gone).""" + + +def mcp_url() -> str: + """The Linear preset's MCP endpoint.""" + definition = catalog.get_definition(LINEAR_CONNECTOR) + return (definition.mcp_url if definition and definition.mcp_url else None) or LINEAR_MCP_URL + + +# --------------------------------------------------------------------------- +# What a source syncs +# --------------------------------------------------------------------------- + + +def _truthy(value: Any, default: bool) -> bool: + if value is None: + return default + if isinstance(value, str): + return value.strip().lower() not in ("", "0", "false", "no", "off") + return bool(value) + + +def _picked(values: Any, fields: tuple[str, ...]) -> list[dict]: + picked: dict[str, dict] = {} + for value in values if isinstance(values, list) else []: + record = value if isinstance(value, dict) else {"id": value} + item_id = str(record.get("id") or "").strip() + if not item_id: + continue + current = picked.setdefault(item_id, {"id": item_id, **{f: "" for f in fields}}) + for field in fields: + current[field] = current[field] or str(record.get(field) or "").strip() + return list(picked.values()) + + +def normalize_selection(items: Any) -> dict: + """What a Linear source syncs, as it is stored in ``sources.remote_data``. + + Args: + items: The wizard's choice (or a stored ``remote_data``, possibly as + JSON): ``teams`` and ``projects`` as ids or ``{id, key, name}`` + records, ``include_comments`` (on unless turned off) and + ``include_documents`` (the picked projects' documents). + + Returns: + ``{teams: [{id, key, name}], projects: [{id, name}], + include_comments, include_documents}``. + + Raises: + ValueError: Nothing to sync was picked. + """ + if isinstance(items, str): + try: + items = json.loads(items) + except ValueError: + items = {} + items = items if isinstance(items, dict) else {} + selection = { + "teams": _picked(items.get("teams"), ("key", "name")), + "projects": _picked(items.get("projects"), ("name",)), + "include_comments": _truthy(items.get("include_comments"), True), + "include_documents": _truthy(items.get("include_documents"), False), + } + if not selection["teams"] and not selection["projects"]: + raise ValueError("Pick at least one Linear team or project") + return selection + + +def selection_name(selection: dict) -> str: + """``Linear · Engineering, Acme``: a source's default name, from what it syncs.""" + names = [item.get("name") for item in (*selection["teams"], *selection["projects"]) if item.get("name")] + return f"Linear · {', '.join(names)}" if names else "Linear" + + +# --------------------------------------------------------------------------- +# Reading tool answers +# --------------------------------------------------------------------------- + + +def items_of(payload: Any, key: str) -> list[dict]: + """The records in a list tool's answer: ``{key: [...]}``, a GraphQL ``nodes`` list or a bare list.""" + if isinstance(payload, list): + found = payload + elif isinstance(payload, dict): + found = payload.get(key) + if isinstance(found, dict): + found = found.get("nodes") + if not isinstance(found, list): + found = next((payload[k] for k in _LIST_KEYS if isinstance(payload.get(k), list)), None) + if found is None: + lists = [value for value in payload.values() if isinstance(value, list)] + found = lists[0] if len(lists) == 1 else [] + else: + found = [] + return [item for item in found if isinstance(item, dict)] + + +def next_cursor(payload: Any) -> Optional[str]: + """The cursor of the next page, or None on the last one.""" + if not isinstance(payload, dict): + return None + info = payload.get("pageInfo") if isinstance(payload.get("pageInfo"), dict) else payload + if info.get("hasNextPage") is False: + return None + cursor = info.get("cursor") or info.get("nextCursor") or info.get("endCursor") + return str(cursor) if cursor else None + + +def unwrap(payload: Any, key: str) -> dict: + """A single record from a get tool's answer, bare or as ``{key: {...}}``.""" + if isinstance(payload, dict) and isinstance(payload.get(key), dict): + return payload[key] + return payload if isinstance(payload, dict) else {} + + +def name_of(value: Any) -> str: + """A person's, state's or label's name, whether Linear sent a string or an object.""" + if isinstance(value, dict): + value = value.get("name") or value.get("displayName") or value.get("title") or value.get("key") + return str(value).strip() if value not in (None, "") else "" + + +def names_of(values: Any) -> list[str]: + """Label names from a list of strings or objects (or a ``{nodes: [...]}``).""" + if isinstance(values, dict): + values = values.get("nodes") + return [name for name in (name_of(value) for value in values or []) if name] if isinstance(values, list) else [] + + +def priority_of(value: Any) -> str: + """``High``: a priority given as a label, an object or Linear's 0-4 number.""" + if isinstance(value, dict): + return str(value.get("name") or _PRIORITIES.get(value.get("value"), "")).strip() + if isinstance(value, bool): + return "" + if isinstance(value, int): + return _PRIORITIES.get(value, "") if value else "" + return str(value or "").strip() + + +def issue_identifier(issue: dict) -> str: + """``ENG-123``: the identifier people use, which list tools may send as the ``id``.""" + identifier = issue.get("identifier") + if identifier: + return str(identifier) + item_id = str(issue.get("id") or "") + return item_id if _IDENTIFIER.match(item_id) else "" + + +def is_truncated(text: Any) -> bool: + """Whether Linear clipped a description in a list (it marks it ``(truncated…)``).""" + return isinstance(text, str) and bool(_TRUNCATED.search(text)) + + +# --------------------------------------------------------------------------- +# Calling the tools +# --------------------------------------------------------------------------- + + +async def _schema(session: Any, tool: str) -> dict: + schema = await session.input_schema(tool) + if schema is None: + raise LinearSyncError(f"Linear's MCP server no longer offers {tool}") + return schema + + +def argument(schema: dict, *names: str) -> Optional[str]: + """The first of ``names`` the tool takes; the first name when the schema lists no properties.""" + properties = schema.get("properties") if isinstance(schema, dict) else None + if not isinstance(properties, dict) or not properties: + return names[0] + return next((name for name in names if name in properties), None) + + +def _page_size(schema: dict, name: str) -> int: + spec = (schema.get("properties") or {}).get(name) if isinstance(schema, dict) else None + maximum = spec.get("maximum") if isinstance(spec, dict) else None + return min(PAGE_SIZE, int(maximum)) if isinstance(maximum, (int, float)) and maximum > 0 else PAGE_SIZE + + +async def paged( + session: Any, + tool: str, + key: str, + filters: dict, + *, + limit: int, + extra: Optional[dict] = None, +) -> AsyncIterator[dict]: + """Every record a list tool returns for ``filters``, page by page, up to ``limit``. + + Args: + session: The open MCP session. + tool: The list tool, e.g. ``list_issues``. + key: Where its answer holds the records, e.g. ``issues``. + filters: Filters the sync depends on, as ``{(name, alias...): value}``. + A filter the tool does not take is an error, never dropped: a + team's sync must not read the whole workspace. + limit: Most records to yield. + extra: Optional arguments, sent only when the tool takes them. + + Raises: + LinearSyncError: The tool is gone or cannot apply a filter. + """ + schema = await _schema(session, tool) + arguments: dict = {} + for names, value in filters.items(): + name = argument(schema, *names) + if name is None: + raise LinearSyncError(f"Linear's {tool} can no longer filter by {names[0]}") + arguments[name] = value + size_name = argument(schema, "limit", "first") + if size_name: + arguments[size_name] = _page_size(schema, size_name) + properties = schema.get("properties") if isinstance(schema, dict) else None + for name, value in (extra or {}).items(): + if isinstance(properties, dict) and name in properties: + arguments[name] = value + cursor_name = argument(schema, "cursor", "after") + cursor: Optional[str] = None + seen: set[str] = set() + yielded = 0 + for _ in range(_MAX_PAGES): + payload = await session.call(tool, {**arguments, **({cursor_name: cursor} if cursor else {})}) + for record in items_of(payload, key): + yield record + yielded += 1 + if yielded >= limit: + return + cursor = next_cursor(payload) + if not cursor or not cursor_name or cursor in seen: + return + seen.add(cursor) + + +async def list_workspace(session: Any) -> dict: + """The teams and projects a Linear account can see, for the sync picker. + + Returns: + ``{teams: [{id, key, name}], projects: [{id, name, state, teams}]}``, + each sorted by name. + """ + teams = [] + async for team in paged(session, "list_teams", "teams", {}, limit=MAX_PICKER_ITEMS): + if team.get("id"): + teams.append({"id": str(team["id"]), "key": str(team.get("key") or ""), + "name": name_of(team.get("name")) or str(team.get("key") or team["id"])}) + projects = [] + async for project in paged(session, "list_projects", "projects", {}, limit=MAX_PICKER_ITEMS, + extra={"includeArchived": False}): + if project.get("id"): + project_teams = project.get("teams") or ([project["team"]] if project.get("team") else []) + projects.append({ + "id": str(project["id"]), + "name": name_of(project.get("name")) or str(project["id"]), + "state": name_of(project.get("state") or project.get("status")), + "teams": names_of(project_teams), + }) + return { + "teams": sorted(teams, key=lambda t: t["name"].lower()), + "projects": sorted(projects, key=lambda p: p["name"].lower()), + } diff --git a/docsgpt/connectors/mcp.py b/docsgpt/connectors/mcp.py new file mode 100644 index 00000000..6ca3ddf3 --- /dev/null +++ b/docsgpt/connectors/mcp.py @@ -0,0 +1,382 @@ +"""MCP helpers for connections: re-scanning a server's actions, switching GitHub's +writes, and calling a server's tools. + +Agents call an MCP server's tools through ``MCPTool``. Server-side work that +reads a service through its MCP server (syncing Linear into Knowledge) opens +one session with :func:`run_connection_session` and makes all its calls in +it, signed in with the connection's own OAuth tokens. +""" + +from __future__ import annotations + +import asyncio +import concurrent.futures +import json +from typing import Any, Awaitable, Callable, Optional, TypeVar + +from docsgpt.agents.tool_pins import carry_pins +from docsgpt.connectors import catalog, service +from docsgpt.connectors.catalog import ConnectorDefinition +from docsgpt.connectors.permissions import ACCESS_READ, ACCESS_WRITE, apply_default_permissions +from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository +from docsgpt.storage.db.repositories.user_tools import UserToolsRepository +from docsgpt.storage.db.session import db_readonly, db_session + +T = TypeVar("T") + + +class NoMcpTool(LookupError): + """The connection has no MCP tool yet (it was set up without tools).""" + + +def _builtin_definition(connection: dict) -> Optional[ConnectorDefinition]: + """The connection's built-in connector when its tool is that service's MCP server.""" + definition = catalog.get_definition(catalog.connector_key_for_row(connection)) + return definition if service.builtin_mcp_config(definition) is not None else None + + +def _write_endpoint_access(action: dict) -> dict: + """Stamp an action of a write endpoint: a read only when the server marks it read-only. + + GitHub marks its read tools, so a name is no evidence there: from the name + alone ``mark_all_notifications_read`` would pass for a read. + """ + annotations = action.get("annotations") if isinstance(action.get("annotations"), dict) else {} + return {**action, "access": ACCESS_READ if annotations.get("readOnlyHint") is True else ACCESS_WRITE} + + +def _discover(user_id: str, connection: dict, tool: dict) -> list[dict]: + from docsgpt.agents.tools.mcp_tool import MCPTool + + config = {k: v for k, v in (tool.get("config") or {}).items() if k != "encrypted_credentials"} + if (connection.get("auth_kind") or "") in ("api_key", "none", "oauth"): + # Pasted keys, or the access token of a built-in OAuth sign-in (GitHub's App). + config["auth_credentials"] = service.access_credentials(connection) + elif service.normalize_status(connection) != service.STATUS_CONNECTED: + raise service.ConnectionUnavailable("Reconnect to continue", connection_id=str(connection["id"])) + config["connection_id"] = str(connection["id"]) + config["query_mode"] = True + mcp_tool = MCPTool(config, user_id) + mcp_tool.discover_tools() + return mcp_tool.get_actions_metadata() + + +def _classified(connection: dict, config: dict, actions: list[dict]) -> list[dict]: + """``actions`` as read from ``config``'s server, stamped strictly when it is a write endpoint.""" + definition = _builtin_definition(connection) + if definition is None or not definition.mcp_write_url or config.get("server_url") != definition.mcp_write_url: + return actions + return [_write_endpoint_access(action) for action in actions] + + +def discover_builtin_actions(user_id: str, connection: dict, writes: bool = False) -> list[dict]: + """The actions of a built-in connector's MCP server (GitHub's), read with the connection. + + Args: + user_id: The connection's owner. + connection: The connection row. + writes: Read the endpoint that also offers write actions; the caller + checked that an admin allows them. + + Raises: + ValueError: The connector has no MCP server of its own. + service.ConnectionUnavailable: The connection needs reconnecting. + Exception: The server could not be reached or listed. + """ + config = service.builtin_mcp_config(_builtin_definition(connection), writes=writes) + if config is None: + raise ValueError("This connector has no MCP server") + return _classified(connection, config, _discover(user_id, connection, {"config": config})) + + +def _rediscover(user_id: str, connection: dict, tool: dict, config: dict) -> tuple[set, set, dict]: + """Re-read one MCP tool's actions from ``config``'s server and store both. + + Actions that still exist keep the permissions the user chose and the + fixed values of parameters they still have; new ones get the defaults + from their annotations (reads always allowed, writes need approval); + removed ones disappear. + + Returns: + ``(added, removed, tool)``: action names, and the serialized tool. + """ + discovered = _classified(connection, config, _discover(user_id, connection, {**tool, "config": config})) + fresh = apply_default_permissions("mcp_tool", service._transform_actions(discovered)) + previous = {a.get("name"): a for a in (tool.get("actions") or []) if isinstance(a, dict)} + merged = [] + for action in fresh: + old = previous.get(action.get("name")) + if old is not None: + action = { + **carry_pins(old, action), + "active": old.get("active", True), + "require_approval": bool(old.get("require_approval")), + } + merged.append(action) + names = {a.get("name") for a in merged} + fields = {"actions": merged} + if config != (tool.get("config") or {}): + fields["config"] = config + with db_session() as conn: + repo = UserToolsRepository(conn) + repo.update(str(tool["id"]), user_id, fields) + serialized = service.serialize_tool(repo.get_any(str(tool["id"]), user_id)) + return names - set(previous), set(previous) - names, serialized + + +def refresh_mcp_tools(user_id: str, connection: dict) -> dict: + """Re-read the actions of every MCP tool on a connection. + + A built-in connector's tool is re-read from its own endpoint: the write + one while it is set up for writes and an admin allows them, the read-only + one otherwise, which it then keeps. + + Returns: + ``{"added": [...], "removed": [...], "tools": [...]}``. + """ + with db_readonly() as conn: + tools = [t for t in ConnectorSessionsRepository(conn).list_tools(str(connection["id"])) + if t.get("name") == "mcp_tool"] + policies = service.load_policies(conn) if tools else {} + definition = _builtin_definition(connection) + added: set[str] = set() + removed: set[str] = set() + refreshed = [] + for tool in tools: + config = dict(tool.get("config") or {}) + if definition is not None: + config["server_url"] = service.builtin_mcp_url( + definition, config.get("server_url"), service.writes_allowed(policies, definition.key), + ) + tool_added, tool_removed, serialized = _rediscover(user_id, connection, tool, config) + added |= tool_added + removed |= tool_removed + refreshed.append(serialized) + return {"added": sorted(added), "removed": sorted(removed), "tools": refreshed} + + +def set_builtin_writes(user_id: str, connection: dict, allow: bool) -> dict: + """Point a connection's built-in MCP tool (GitHub's) at its write or read-only endpoint. + + The tool's actions are re-read from the new endpoint, keeping the + choices made for actions that exist on both. + + Args: + user_id: The connection's owner. + connection: The connection row. + allow: Let agents make changes (the write endpoint) or only read. + + Returns: + ``{"added", "removed", "tools", "writes"}``. + + Raises: + ValueError: The connector offers no write access. + NoMcpTool: The connection has no such tool yet. + service.WritesForbidden: ``allow`` while an admin forbids writes. + service.ConnectionUnavailable: The connection needs reconnecting. + """ + definition = _builtin_definition(connection) + if definition is None or not definition.mcp_write_url: + raise ValueError("This connector has no write access to turn on") + with db_readonly() as conn: + tools = [t for t in ConnectorSessionsRepository(conn).list_tools(str(connection["id"])) + if t.get("name") == "mcp_tool"] + allowed = service.writes_allowed(service.load_policies(conn), definition.key) + if not tools: + raise NoMcpTool("Add this connection's tools first") + if allow and not allowed: + raise service.WritesForbidden(f"An admin turned off changes through {definition.name}") + added: set[str] = set() + removed: set[str] = set() + refreshed = [] + for tool in tools: + config = {**(tool.get("config") or {}), **service.builtin_mcp_config(definition, writes=allow)} + tool_added, tool_removed, serialized = _rediscover(user_id, connection, tool, config) + added |= tool_added + removed |= tool_removed + refreshed.append(serialized) + return {"added": sorted(added), "removed": sorted(removed), "tools": refreshed, "writes": allow} + + +# --------------------------------------------------------------------------- +# Calling a server's tools from the server side +# --------------------------------------------------------------------------- + +SIGN_IN_EXPIRED = "The sign-in expired and could not be renewed. Reconnect to continue." +_TRANSIENT_WORDS = ("rate limit", "ratelimit", "too many requests", "temporarily", "timed out", "timeout") + + +class MCPToolError(Exception): + """An MCP tool answered with an error (a missing issue, a refused argument).""" + + +class MCPSession: + """One open MCP session: list the server's tools and call them. + + Made by :func:`run_connection_session`. Every call goes over the same + session, so a sync of hundreds of items signs in once. + """ + + def __init__(self, client: Any): + self._client = client + self._schemas: Optional[dict[str, dict]] = None + + async def input_schema(self, name: str) -> Optional[dict]: + """The input schema of tool ``name`` (``{}`` when it has none), or None when there is no such tool.""" + if self._schemas is None: + tools = await self._client.list_tools() + self._schemas = {tool.name: (getattr(tool, "inputSchema", None) or {}) for tool in tools} + return self._schemas.get(name) + + async def call(self, name: str, arguments: dict) -> Any: + """Call tool ``name`` and return what it answered. + + Returns: + Its structured content when it sends some, else its text parsed + as JSON, else the text itself. + + Raises: + MCPToolError: The tool answered with an error. + service.TransientConnectionError: The error says to try again later. + """ + result = await self._client.call_tool(name, arguments, raise_on_error=False) + if getattr(result, "is_error", False): + message = _result_text(result) or "the tool failed" + if any(word in message.lower() for word in _TRANSIENT_WORDS): + raise service.TransientConnectionError(f"{name}: {message}") + raise MCPToolError(f"{name}: {message}") + return tool_result_data(result) + + +def _result_text(result: Any) -> str: + blocks = getattr(result, "content", None) or [] + return "\n".join(str(block.text) for block in blocks if getattr(block, "text", None)).strip() + + +def tool_result_data(result: Any) -> Any: + """What an MCP tool answered: its structured content, its text parsed as JSON, or the text.""" + structured = getattr(result, "structured_content", None) + if isinstance(structured, dict) and structured: + # A tool without an output schema has other values wrapped as ``{"result": ...}``. + return structured["result"] if set(structured) == {"result"} else structured + text = _result_text(result) + try: + return json.loads(text) + except ValueError: + return text + + +def _client_for(connection: dict, server_url: str, timeout: float) -> Any: + """An MCP client for ``server_url`` that signs in with ``connection``'s stored OAuth tokens. + + The tokens are renewed with their refresh token when they expire; when + that fails the client raises ``MCPReauthorizationRequired`` instead of + starting an interactive sign-in. + """ + from fastmcp import Client + + from docsgpt.agents.tools.mcp_tool import MCPTool, NonInteractiveOAuth + + tool = MCPTool( + {"server_url": server_url, "auth_type": "oauth", "transport_type": "http", "timeout": timeout}, + connection["user_id"], + ) + auth = NonInteractiveOAuth( + mcp_url=tool.server_url, + redirect_uri=tool.redirect_uri, + user_id=connection["user_id"], + connection_id=str(connection["id"]), + ) + return Client(tool._create_transport(), auth=auth, timeout=timeout) + + +def _errors(exc: BaseException): + """``exc`` and every exception it wraps: causes, contexts and group members.""" + seen: set[int] = set() + pending: list[Optional[BaseException]] = [exc] + while pending: + current = pending.pop() + if current is None or id(current) in seen: + continue + seen.add(id(current)) + yield current + pending.extend(getattr(current, "exceptions", None) or ()) + pending.extend((current.__cause__, current.__context__)) + + +def _needs_sign_in(exc: BaseException) -> bool: + from docsgpt.agents.tools.mcp_tool import MCPReauthorizationRequired + + return any( + isinstance(error, MCPReauthorizationRequired) or "oauth session expired" in str(error).lower() + for error in _errors(exc) + ) + + +def _is_transient(exc: BaseException) -> bool: + import httpx + + return any( + isinstance(error, (ConnectionError, TimeoutError, httpx.TransportError)) for error in _errors(exc) + ) + + +def _run_coroutine(coroutine: Awaitable[T]) -> T: + """Run ``coroutine`` to the end, on a thread of its own when this one already runs an event loop.""" + try: + asyncio.get_running_loop() + except RuntimeError: + return asyncio.run(coroutine) + with concurrent.futures.ThreadPoolExecutor(max_workers=1) as pool: + return pool.submit(asyncio.run, coroutine).result() + + +def run_connection_session( + connection: dict, + server_url: str, + work: Callable[[MCPSession], Awaitable[T]], + *, + timeout: float = 60.0, +) -> T: + """Open one MCP session signed in with ``connection`` and run ``work`` in it. + + Args: + connection: An MCP OAuth connection (a Linear sign-in). + server_url: The MCP endpoint. It must be the server the connection + signed in to, so its tokens never go anywhere else. + work: Async function given the open :class:`MCPSession`. + timeout: Seconds allowed for each request. + + Returns: + What ``work`` returned. + + Raises: + ValueError: ``server_url`` is not the connection's server. + service.ConnectionUnavailable: The connection needs reconnecting, + also when its sign-in expired and could not be renewed (it is + then flagged, which pauses its sources). + service.TransientConnectionError: The server could not be reached, + or asked to slow down. + MCPToolError: A tool answered with an error. + """ + connection_id = str(connection["id"]) + if catalog.base_url(connection.get("server_url")) != catalog.base_url(server_url): + raise ValueError("A connection's sign-in only goes to its own server") + if service.normalize_status(connection) != service.STATUS_CONNECTED: + raise service.ConnectionUnavailable("Reconnect to continue", connection_id=connection_id) + + async def session_work() -> T: + async with _client_for(connection, server_url, timeout) as client: + return await work(MCPSession(client)) + + try: + return _run_coroutine(session_work()) + except (MCPToolError, service.TransientConnectionError, service.ConnectionUnavailable): + raise + except Exception as exc: + if _needs_sign_in(exc): + service.mark_reconnect_needed(connection_id, SIGN_IN_EXPIRED) + raise service.ConnectionUnavailable(SIGN_IN_EXPIRED, connection_id=connection_id) from exc + if _is_transient(exc): + raise service.TransientConnectionError(f"The MCP server did not answer: {type(exc).__name__}") from exc + raise diff --git a/docsgpt/connectors/permissions.py b/docsgpt/connectors/permissions.py new file mode 100644 index 00000000..fd2ddeaf --- /dev/null +++ b/docsgpt/connectors/permissions.py @@ -0,0 +1,179 @@ +"""Read / write classification and permissions for connection-backed tools. + +Every action a connection's tool offers is either a *read* (it only looks +something up) or a *write* (it changes or sends something). Reads default to +"Always allow"; writes default to "Needs approval". The permission is stored +on the action itself: ``active`` off means "Off", ``require_approval`` means +"Needs approval", neither means "Always allow". +""" + +from __future__ import annotations + +import re +from typing import Optional + +ACCESS_READ = "read" +ACCESS_WRITE = "write" + +PERMISSION_ALWAYS = "always" +PERMISSION_ASK = "ask" +PERMISSION_OFF = "off" +PERMISSIONS = (PERMISSION_ALWAYS, PERMISSION_ASK, PERMISSION_OFF) + +_READ_WORDS = frozenset( + ("search", "query", "find", "list", "get", "read", "fetch", "lookup", "describe", "retrieve") +) +# A name holding any of these is a write even when it also holds a read word +# (``get_or_create_page``, ``search_and_replace``). +_WRITE_WORDS = frozenset(( + "create", "update", "delete", "remove", "set", "send", "post", "put", "patch", "write", "add", "insert", + "upsert", "append", "replace", "edit", "modify", "rename", "move", "copy", "upload", "save", "submit", + "publish", "share", "invite", "assign", "archive", "restore", "reset", "clear", "purge", "drop", "execute", + "run", "invoke", "trigger", "start", "stop", "cancel", "close", "merge", "approve", "reject", "reply", + "comment", "mark", "enable", "disable", "grant", "revoke", "transfer", "import", "sync", "lock", "unlock", +)) +_CAMEL_BOUNDARY = re.compile(r"(?<=[a-z0-9])(?=[A-Z])|(?<=[A-Z])(?=[A-Z][a-z])") +_NON_WORD = re.compile(r"[^A-Za-z0-9]+") + + +def _name_words(name: str) -> set[str]: + """The lower-cased words of an action name (``snake``, ``kebab``, ``camelCase``).""" + return {word.lower() for word in _NON_WORD.split(_CAMEL_BOUNDARY.sub(" ", name)) if word} + + +def action_access(tool_name: Optional[str], action: dict) -> str: + """Whether ``action`` reads or writes. + + Order of evidence: an explicit ``access`` on the action metadata, MCP tool + annotations (``readOnlyHint`` / ``destructiveHint``), the HTTP method of an + API tool action, then the action's name: a read only when one of its + words is a read verb and none is a write verb. + + Args: + tool_name: The ``user_tools`` name the action belongs to. + action: One entry of the tool's ``actions``. + + Returns: + ``read`` or ``write``. + """ + access = action.get("access") + if access in (ACCESS_READ, ACCESS_WRITE): + return access + annotations = action.get("annotations") or {} + if isinstance(annotations, dict): + if annotations.get("readOnlyHint") is True: + return ACCESS_READ + if annotations.get("destructiveHint") is True or annotations.get("readOnlyHint") is False: + return ACCESS_WRITE + method = (action.get("method") or "").upper() + if tool_name == "api_tool" and method: + return ACCESS_READ if method in ("GET", "HEAD", "OPTIONS") else ACCESS_WRITE + words = _name_words(action.get("name") or "") + return ACCESS_READ if words & _READ_WORDS and not words & _WRITE_WORDS else ACCESS_WRITE + + +def action_permission(action: dict) -> str: + """The action's permission: ``always``, ``ask`` or ``off``.""" + if action.get("active") is False: + return PERMISSION_OFF + if action.get("require_approval"): + return PERMISSION_ASK + return PERMISSION_ALWAYS + + +def apply_permission(action: dict, permission: str) -> dict: + """Return ``action`` with ``permission`` written onto its flags.""" + if permission not in PERMISSIONS: + raise ValueError(f"Unknown permission: {permission}") + updated = dict(action) + updated["active"] = permission != PERMISSION_OFF + updated["require_approval"] = permission == PERMISSION_ASK + return updated + + +def apply_default_permissions(tool_name: Optional[str], actions: list[dict]) -> list[dict]: + """Stamp ``access`` on each action and default writes to needing approval.""" + stamped = [] + for action in actions: + access = action_access(tool_name, action) + updated = {**action, "access": access} + if access == ACCESS_WRITE: + updated["require_approval"] = True + stamped.append(updated) + return stamped + + +def tool_actions(tool: dict) -> list[dict]: + """A tool row's actions, each carrying its ``name``. + + An API tool keeps its actions under ``config["actions"]`` keyed by name; + every other tool lists them in ``actions``. + """ + if tool.get("name") == "api_tool": + stored = (tool.get("config") or {}).get("actions") or {} + return [{**(action or {}), "name": name} for name, action in stored.items()] + return [action for action in tool.get("actions") or [] if isinstance(action, dict)] + + +# Where an API tool keeps the values it sends: header and query values are +# the owner's (sealed per action, flagged ``has_value``; legacy rows hold them +# in plain ``value``). Mirrors ``tool_executor.API_TOOL_SECRET_SECTIONS``. +_API_TOOL_SECRET_SECTIONS = ("headers", "query_params") + + +def _api_action_sends_credentials(action: dict) -> bool: + """Whether an API tool action sends a header or query value the owner stored.""" + for section in _API_TOOL_SECRET_SECTIONS: + block = action.get(section) + props = block.get("properties") if isinstance(block, dict) else None + for spec in (props or {}).values(): + if isinstance(spec, dict) and (spec.get("has_value") or spec.get("value") not in (None, "")): + return True + return False + + +def holds_owner_credentials(tool: dict, action_name: Optional[str] = None) -> bool: + """Whether ``tool`` (or its ``action_name``) acts with credentials its owner stored. + + A connection's account, a stored secret, an MCP server the owner signed + in to, or an API tool action that sends a header or query value the + owner saved (a key, a token). Anyone running it acts as the owner there, + whoever they are. An API tool action that sends nothing stored does not. + + Args: + tool: A ``user_tools`` row. + action_name: One action to judge; None judges the tool as a whole. + + Returns: + True when the tool (or that action) runs on the owner's credentials. + """ + config = tool.get("config") or {} + if tool.get("connection_id") or config.get("encrypted_credentials"): + return True + if tool.get("name") == "api_tool": + actions = config.get("actions") or {} + if action_name is not None: + return _api_action_sends_credentials(actions.get(action_name) or {}) + return any(_api_action_sends_credentials(action or {}) for action in actions.values()) + return tool.get("name") == "mcp_tool" and (config.get("auth_type") or "none") != "none" + + +def owner_credential_writes(tool: dict) -> list[str]: + """Names of the tool's write actions that run on its owner's credentials. + + These are what someone who can't approve for the owner (an API-key or + widget caller, a public-link user) may run only when the owner allows + them in the agent's API write allowlist. + + Args: + tool: A ``user_tools`` row. + + Returns: + Action names, empty when the tool holds no owner credentials. + """ + return [ + action["name"] for action in tool_actions(tool) + if action.get("name") and action.get("active") is not False + and action_access(tool.get("name"), action) == ACCESS_WRITE + and holds_owner_credentials(tool, action["name"]) + ] diff --git a/docsgpt/connectors/presets/mcp.yaml b/docsgpt/connectors/presets/mcp.yaml new file mode 100644 index 00000000..f185a518 --- /dev/null +++ b/docsgpt/connectors/presets/mcp.yaml @@ -0,0 +1,77 @@ +# Curated remote MCP servers shown as Connectors catalog cards. +# +# Inclusion rules: the vendor operates the server, it has a stable public URL, +# and it signs in with OAuth plus dynamic client registration (or needs no +# auth), so connecting takes no admin setup. Each entry was checked for OAuth +# authorization-server metadata with a registration endpoint and for a 401 +# challenge on the MCP endpoint. Keep the list short; anything else is a +# custom connector. +# +# Keys look like ``mcp:``. The frontend reads ``settings.connectors +# .descriptions.mcp_`` for a translated description. +# +# A preset with ``sync_ingestor`` also syncs into Knowledge through its own +# MCP server, signed in with the same connection (Linear's issues). + +- key: mcp:notion + name: Notion + description: Search, read and update Notion pages and databases. + icon: notion + category: knowledge + mcp_url: https://mcp.notion.com/mcp + auth_kind: mcp_oauth + capabilities: [read, write] + docs_url: https://developers.notion.com/docs/mcp + +- key: mcp:linear + name: Linear + description: Sync issues and documents into Knowledge, and let agents find, create and update issues. + icon: linear + category: projects + mcp_url: https://mcp.linear.app/mcp + auth_kind: mcp_oauth + capabilities: [sync, read, write] + sync_ingestor: linear + docs_url: https://linear.app/docs/mcp + +- key: mcp:atlassian + name: Jira & Confluence + description: Search and update Jira issues and Confluence pages. + icon: atlassian + category: projects + # Shown on the Confluence card, next to syncing pages into Knowledge. + part_of: confluence + mcp_url: https://mcp.atlassian.com/v1/mcp + auth_kind: mcp_oauth + capabilities: [read, write] + docs_url: https://support.atlassian.com/rovo/docs/getting-started-with-the-atlassian-remote-mcp-server/ + +- key: mcp:sentry + name: Sentry + description: Look up Sentry issues, events and releases. + icon: sentry + category: dev + mcp_url: https://mcp.sentry.dev/mcp + auth_kind: mcp_oauth + capabilities: [read, write] + docs_url: https://docs.sentry.io/product/sentry-mcp/ + +- key: mcp:asana + name: Asana + description: Find and update Asana tasks and projects. + icon: asana + category: projects + mcp_url: https://mcp.asana.com/v2/mcp + auth_kind: mcp_oauth + capabilities: [read, write] + docs_url: https://developers.asana.com/docs/using-asanas-mcp-server + +- key: mcp:stripe + name: Stripe + description: Look up customers, payments and subscriptions in Stripe. + icon: stripe + category: business + mcp_url: https://mcp.stripe.com + auth_kind: mcp_oauth + capabilities: [read, write] + docs_url: https://docs.stripe.com/mcp diff --git a/docsgpt/connectors/resolve.py b/docsgpt/connectors/resolve.py new file mode 100644 index 00000000..1b3119d0 --- /dev/null +++ b/docsgpt/connectors/resolve.py @@ -0,0 +1,279 @@ +"""Which connection a shared tool or source uses at runtime. + +A resource that points at a connection runs either with its owner's account +(``owner`` mode, the default for new shares) or with the invoking member's +own account for the same service (``member`` mode). Resolution never returns +credentials; callers read them from the resolved row through +``docsgpt.connectors.service``. +""" + +from __future__ import annotations + +from datetime import datetime, timezone +import logging +from dataclasses import dataclass +from typing import Optional + +from docsgpt.connectors import catalog, service +from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository +from docsgpt.storage.db.session import db_readonly + +logger = logging.getLogger(__name__) + +MODE_OWNER = "owner" +MODE_MEMBER = "member" + +# Why a connection-backed tool can't run (``connection_stop_reason``). +CONNECTION_NEEDS_RECONNECT = "connection_needs_reconnect" +CONNECTION_REMOVED = "connection_removed" +CONNECTOR_DISABLED = "connector_disabled" + +# Tool config key ``remove_connection`` sets when it keeps a connection's +# tools: the connector key, or True when the connection had none, so they can +# say what they lost. Only the server writes it (see +# :func:`carry_removed_connection`). +REMOVED_CONNECTION_KEY = "removed_connection" + + +def carry_removed_connection(new_config: dict, stored_config: Optional[dict]) -> dict: + """``new_config`` with the stored removed-connection note, never a client's. + + A config save must neither fake the note nor clear it; every path that + writes a tool config from a request passes it through here. + + Args: + new_config: The config about to be stored. + stored_config: The tool's stored config (None or ``{}`` when creating it). + + Returns: + dict: A copy of ``new_config`` carrying the stored note, if any. + """ + out = {k: v for k, v in (new_config or {}).items() if k != REMOVED_CONNECTION_KEY} + marker = (stored_config or {}).get(REMOVED_CONNECTION_KEY) + if marker: + out[REMOVED_CONNECTION_KEY] = marker + return out + + +@dataclass(frozen=True) +class ResolvedConnection: + """The connection a call runs with, or why there is none. + + Attributes: + row: The ``connector_sessions`` row, None when missing. + available: Whether it can be used right now. + connector_key: Catalog key of the service. + connector_name: Name shown to the user ("Notion", or a custom label). + delegated: The row belongs to someone other than the invoker. + writes_allowed: Whether an admin lets agents make changes through a + connector that offers them as an opt-in (GitHub); True elsewhere. + enabled: Whether the connector is switched on (an admin can turn it off). + mode: Whose account the resource runs with, after any mode an admin + forces: :data:`MODE_OWNER` or :data:`MODE_MEMBER`. + """ + + row: Optional[dict] + available: bool + connector_key: Optional[str] + connector_name: Optional[str] + delegated: bool = False + writes_allowed: bool = True + enabled: bool = True + mode: str = MODE_OWNER + + @property + def connection_id(self) -> Optional[str]: + return str(self.row["id"]) if self.row else None + + +def _name_for(row: Optional[dict], fallback_key: Optional[str]) -> Optional[str]: + if row is not None: + return service.serialize_connection(row)["name"] + definition = catalog.get_definition(fallback_key) + return definition.name if definition else None + + +def resolve_connection( + resource: dict, + invoker_user_id: Optional[str], + *, + conn=None, + policies: Optional[dict] = None, +) -> Optional[ResolvedConnection]: + """Pick the connection a tool or source uses for ``invoker_user_id``. + + ``owner`` mode uses ``resource.connection_id``. ``member`` mode uses the + invoker's own connection to the same service (and, for MCP, the same + server), falling back to the owner's when the invoker is the owner. + + Args: + resource: A ``user_tools`` or ``sources`` row. + invoker_user_id: Who is running it. + conn: An open connection to reuse; a read-only one is opened when None. + policies: Connector policies already loaded with ``service.load_policies``, + so a caller resolving many resources loads them once. + + Returns: + None when the resource has no connection at all; otherwise the + resolution, possibly with ``available=False``. + """ + if not resource.get("connection_id"): + return None + if conn is None: + with db_readonly() as own_conn: + return _resolve(own_conn, resource, invoker_user_id, policies) + return _resolve(conn, resource, invoker_user_id, policies) + + +def effective_credential_mode(resource: dict, policies: dict, connector_key: Optional[str]) -> str: + """Whose account a connection-backed resource runs with: ``owner`` or ``member``. + + The resource's own ``credential_mode``, unless an admin forces one mode + for every share of its connector. + + Args: + resource: A ``user_tools`` or ``sources`` row. + policies: Connector policies from ``service.load_policies``. + connector_key: Catalog key of the resource's connection. + + Returns: + :data:`MODE_OWNER` or :data:`MODE_MEMBER`. + """ + policy = (policies.get(connector_key) or {}) if connector_key else {} + if policy.get("credential_mode") in (MODE_OWNER, MODE_MEMBER): + return policy["credential_mode"] + return MODE_MEMBER if resource.get("credential_mode") == MODE_MEMBER else MODE_OWNER + + +def _resolve(conn, resource: dict, invoker_user_id: Optional[str], policies: Optional[dict]) -> ResolvedConnection: + connection_id = resource.get("connection_id") + owner = resource.get("user_id") + repo = ConnectorSessionsRepository(conn) + owned = repo.get(str(connection_id)) + owned_key = catalog.connector_key_for_row(owned) if owned else None + if policies is None: + policies = service.load_policies(conn) + mode = effective_credential_mode(resource, policies, owned_key) + if owned is not None and owner and owned.get("user_id") != owner: + # A resource may only point at its own owner's connection. + logger.warning( + "resource %s points at a connection it does not own", resource.get("id"), + ) + owned = None + row = owned + if mode == MODE_MEMBER and invoker_user_id and invoker_user_id != owner: + row = _member_connection(repo, owned, invoker_user_id) + key = catalog.connector_key_for_row(row or owned or {}) + enabled = service.connector_is_enabled(policies, key) + available = ( + row is not None + and service.normalize_status(row) == service.STATUS_CONNECTED + and enabled + ) + return ResolvedConnection( + row=row, + available=available, + connector_key=key, + connector_name=_name_for(row or owned, key), + delegated=bool(row and invoker_user_id and row.get("user_id") != invoker_user_id), + writes_allowed=_writes_allowed(policies, key), + enabled=enabled, + mode=mode, + ) + + +def connection_stop_reason(tool: dict, resolved: Optional[ResolvedConnection]) -> Optional[str]: + """Why an owner-mode tool's connection keeps it from running, or None. + + Only the connection the owner's account runs on is judged. A member-mode + tool runs on each caller's own account, so the owner's account says + nothing about whether it runs; only an admin turning the service off + stops it for everyone. + + Args: + tool: The ``user_tools`` row. + resolved: What :func:`resolve_connection` returned for it, resolved + for the tool's holder's owner. + + Returns: + :data:`CONNECTION_REMOVED` when its connection was removed and the + tool kept (``remove_connection`` notes it on the tool) with no + credentials of its own, or points at + one it may not use; :data:`CONNECTOR_DISABLED` when an admin turned + the service off; :data:`CONNECTION_NEEDS_RECONNECT` when the account + must sign in again; else None. + """ + if resolved is None: + # Only a tool that had a connection lost it: a tool that never had + # one (a tokenless ntfy, a legacy tool) runs on its own config, and + # so does a kept one its owner gave credentials of its own since. + config = tool.get("config") or {} + removed = config.get(REMOVED_CONNECTION_KEY) and not config.get("encrypted_credentials") + return CONNECTION_REMOVED if removed and not tool.get("connection_id") else None + if not resolved.enabled: + return CONNECTOR_DISABLED + if resolved.mode == MODE_MEMBER: + return None + if resolved.row is None: + return CONNECTION_REMOVED + if not resolved.available: + return CONNECTION_NEEDS_RECONNECT + return None + + +def _writes_allowed(policies: dict, key: Optional[str]) -> bool: + definition = catalog.get_definition(key) if key else None + if definition is None or not definition.mcp_write_url: + return True + return service.writes_allowed(policies, key) + + +def _member_connection(repo: ConnectorSessionsRepository, owned: Optional[dict], invoker: str) -> Optional[dict]: + """The invoker's own connection to the service the owner's connection is for. + + A member with several connected accounts of that service gets the one + most recently used or connected, whichever is later: the account they + are working in, or the one a "Connect to continue" just added (never + used yet). Ties go to the last used, then the last updated. + """ + if owned is None: + return None + candidates = [ + row for row in repo.list_for_user(invoker) + if row.get("provider") == owned.get("provider") + and (row.get("server_url") or "") == (owned.get("server_url") or "") + and service.normalize_status(row) == service.STATUS_CONNECTED + ] + if not candidates: + return None + + def when(value) -> datetime: + if isinstance(value, str): + value = datetime.fromisoformat(value) + if not isinstance(value, datetime): + return datetime.min.replace(tzinfo=timezone.utc) + return value if value.tzinfo else value.replace(tzinfo=timezone.utc) + + def recency(row: dict) -> tuple: + used, updated, created = (when(row.get(field)) for field in ("last_used_at", "updated_at", "created_at")) + return max(used, created), used, updated, created + + return max(candidates, key=recency) + + +def audit_delegation(resolved: ResolvedConnection, *, invoker: Optional[str], resource_type: str, + resource_id: Optional[str], agent_id: Optional[str] = None) -> None: + """Log a call that runs with someone else's account (``owner`` mode).""" + if not resolved.delegated or resolved.row is None: + return + logger.info( + "tool_credential_delegation", + extra={ + "invoker": invoker, + "tool_owner": resolved.row.get("user_id"), + "connection_id": resolved.connection_id, + "resource_type": resource_type, + "resource_id": resource_id, + "agent_id": agent_id, + }, + ) diff --git a/docsgpt/connectors/service.py b/docsgpt/connectors/service.py new file mode 100644 index 00000000..1b6f9ac4 --- /dev/null +++ b/docsgpt/connectors/service.py @@ -0,0 +1,1704 @@ +"""Connection lifecycle: listing, status, and what each connection provides. + +A connection is one ``connector_sessions`` row: a signed-in OAuth account, an +MCP server or a set of API credentials. Sources and tools point at it through +``connection_id``. Everything the API returns about a connection goes through +:func:`serialize_connection`, which never includes credentials. +""" + +from __future__ import annotations + +import json +import sys +from typing import Any, Iterable, Optional + +from docsgpt.connectors import catalog +from docsgpt.connectors.catalog import ConnectorDefinition +from docsgpt.storage.db.repositories.connector_sessions import ( + ConnectorSessionsRepository, + owns_connector_session, +) +from docsgpt.storage.db.session import db_readonly, db_session + +STATUS_CONNECTED = "connected" +STATUS_RECONNECT = "reconnect_needed" +STATUS_DISCONNECTED = "disconnected" +STATUS_ERROR = "error" +STATUS_PENDING = "pending" + +# Worst first: a card shows the worst status among the user's accounts. +_STATUS_SEVERITY = { + STATUS_RECONNECT: 4, + STATUS_ERROR: 3, + STATUS_DISCONNECTED: 2, + STATUS_CONNECTED: 1, + STATUS_PENDING: 0, +} + + +def _json(value: Any) -> Any: + if isinstance(value, str): + try: + return json.loads(value) + except ValueError: + return None + return value + + +def has_credentials(row: dict) -> bool: + """Whether a connection holds usable credentials (without decrypting them).""" + if row.get("encrypted_credentials"): + return True + return _has_plaintext_tokens(row) + + +def _has_plaintext_tokens(row: dict) -> bool: + """Legacy plaintext tokens on a row not yet converted by migration 0040.""" + token_info = _json(row.get("token_info")) or {} + if isinstance(token_info, dict) and token_info.get("access_token"): + return True + session_data = _json(row.get("session_data")) or {} + tokens = session_data.get("tokens") if isinstance(session_data, dict) else None + return bool(isinstance(tokens, dict) and tokens.get("access_token")) + + +def normalize_status(row: dict) -> str: + """Map a row's stored status onto the connection status set. + + Rows written before ``0040_connections`` use ``authorized`` for a + finished OAuth sign-in, and MCP rows carry no status at all; both are + read from whether the row holds credentials. + """ + raw = (row.get("status") or "").lower() + if raw in (STATUS_RECONNECT, STATUS_DISCONNECTED, STATUS_ERROR): + return raw + if raw in ("authorized", STATUS_CONNECTED, "active"): + return STATUS_CONNECTED + if raw == STATUS_PENDING: + # A legacy row whose plaintext tokens landed after the pending mark. + return STATUS_CONNECTED if _has_plaintext_tokens(row) else STATUS_PENDING + return STATUS_CONNECTED if has_credentials(row) else STATUS_PENDING + + +def worst_status(statuses: Iterable[str]) -> Optional[str]: + """The status that needs the most attention, or None for an empty list.""" + ranked = sorted(statuses, key=lambda s: _STATUS_SEVERITY.get(s, 0), reverse=True) + return ranked[0] if ranked else None + + +def account_label(row: dict) -> str: + """What identifies the account to its owner: an email, a workspace or a host.""" + return ( + row.get("account_label") + or row.get("user_email") + or row.get("display_name") + or catalog.base_url(row.get("server_url")).split("://")[-1] + or "" + ) + + +#: Longest name a user can give an account. +ACCOUNT_NAME_MAX = 80 + + +def account_name(row: dict) -> str: + """What the user calls the account, falling back to its label.""" + return (row.get("account_name") or "").strip() or account_label(row) + + +def rename_connection(conn, row: dict, name: str) -> dict: + """Give an account a name (an empty one clears it). + + The name is only a name: the account is still found by its label when + its owner signs in again. + + Args: + conn: Open connection inside a transaction. + row: The connection, already authorised for its owner. + name: The new name; surrounding spaces are dropped. + + Returns: + The connection's public shape after the change. + """ + repo = ConnectorSessionsRepository(conn) + repo.update(str(row["id"]), {"account_name": name.strip() or None}) + return serialize_connection(repo.get(str(row["id"]))) + + +def with_account(name: str, account: str) -> str: + """``Telegram · Alerts bot``: a tool's name with the account it uses.""" + return f"{name} · {account}" if account and account not in name else name + + +def account_tool_names(conn, tools: Iterable[dict]) -> dict[str, str]: + """Names for tools whose owner has several accounts of the tool's service. + + With one Telegram bot a tool is just "Telegram"; with two, each tool + still named after the service becomes "Telegram · ", so people + (and the model) can tell them apart. A name the user chose is kept. + + Args: + conn: Open database connection. + tools: ``user_tools`` rows. + + Returns: + Tool id to its name, only for tools whose name changes. + """ + repo = ConnectorSessionsRepository(conn) + connections: dict[str, dict] = {} + for tool in tools: + connection_id = str(tool.get("connection_id") or "") + if connection_id and connection_id not in connections: + row = repo.get(connection_id) + if row is not None: + connections[connection_id] = row + counts: dict[tuple, int] = {} + for owner in {row["user_id"] for row in connections.values()}: + for row in repo.list_for_user(owner): + if normalize_status(row) != STATUS_PENDING: + key = (owner, catalog.connector_key_for_row(row)) + counts[key] = counts.get(key, 0) + 1 + names = {} + for tool in tools: + row = connections.get(str(tool.get("connection_id") or "")) + if row is None or counts.get((row["user_id"], catalog.connector_key_for_row(row)), 0) < 2: + continue + name = tool.get("custom_name") or tool.get("display_name") or "" + if name == serialize_connection(row)["name"]: + names[str(tool["id"])] = with_account(name, account_name(row)) + return names + + +def _iso(value: Any) -> Optional[str]: + return value.isoformat() if hasattr(value, "isoformat") else value + + +def serialize_connection(row: dict, counts: Optional[dict] = None) -> dict: + """Public shape of a connection. Never includes tokens or secrets.""" + key = catalog.connector_key_for_row(row) + definition = catalog.get_definition(key) + counts = counts or {} + return { + "id": str(row["id"]), + "connector_key": key, + "name": (definition.name if definition and definition.publisher != "custom" else None) + or row.get("display_name") + or (definition.name if definition else key), + "display_name": row.get("display_name"), + "icon": definition.icon if definition else "tool_mcp_tool", + "account_label": account_label(row), + "account_name": (row.get("account_name") or "").strip() or None, + "auth_kind": row.get("auth_kind") or (definition.auth_kind if definition else None), + "status": normalize_status(row), + "server_url": row.get("server_url"), + "last_error": row.get("last_error"), + "created_at": _iso(row.get("created_at")), + "updated_at": _iso(row.get("updated_at")), + "last_used_at": _iso(row.get("last_used_at")), + "source_count": counts.get("sources", 0), + "tool_count": counts.get("tools", 0), + "credential_mode": row.get("credential_mode"), + } + + +def list_connections(conn, user_id: str) -> list[dict]: + """The user's connections, finished sign-ins only, with resource counts.""" + repo = ConnectorSessionsRepository(conn) + rows = [r for r in repo.list_for_user(user_id) if normalize_status(r) != STATUS_PENDING] + counts = repo.resource_counts([str(r["id"]) for r in rows]) + return [serialize_connection(r, counts.get(str(r["id"]))) for r in rows] + + +def serialize_source(row: dict, connection_status: Optional[str] = None) -> dict: + """A source linked to a connection, as the connection drawer lists it.""" + metadata = _json(row.get("metadata")) or {} + sync_state = metadata.get("sync_state") if isinstance(metadata, dict) else None + if connection_status in (STATUS_RECONNECT, STATUS_DISCONNECTED) and not sync_state: + sync_state = "paused_reconnect" + return { + "id": str(row["id"]), + "name": row.get("name"), + "type": row.get("type"), + "last_sync": _iso(row.get("date")), + "sync_frequency": row.get("sync_frequency") or "never", + "sync_state": sync_state or "active", + } + + +def serialize_parameters(action: dict, account_parameters: Optional[dict] = None) -> list[dict]: + """An action's parameters as the connection drawer shows them. + + ``fixed`` parameters are sent with ``value`` on every call and hidden + from the model; the others are left to it. A parameter the connection + itself sets (Telegram's default chat) is fixed with ``set_by: account``. + """ + account_parameters = account_parameters or {} + from docsgpt.agents.tool_pins import is_pinned, iter_parameters + + schema = action.get("parameters") if isinstance(action.get("parameters"), dict) else {} + required = set(schema.get("required") or []) if isinstance(schema.get("required"), list) else set() + parameters = [] + for _section, name, details in iter_parameters(action): + fixed = is_pinned(details) + entry = { + "name": name, + "description": details.get("description") or "", + "type": details.get("type") if isinstance(details.get("type"), str) else "string", + "required": bool(details.get("required")) or name in required, + "fixed": fixed, + "value": details.get("value") if fixed else None, + "set_by": "tool" if fixed else None, + } + if name in account_parameters: + entry.update(fixed=True, value=account_parameters[name], set_by="account") + parameters.append(entry) + return parameters + + +def serialize_tool( + row: dict, account_parameters: Optional[dict] = None, display_name: Optional[str] = None, +) -> dict: + """A tool linked to a connection, with its actions, permissions and parameters. + + Args: + row: The ``user_tools`` row. + account_parameters: Parameters its connection sets, from + :func:`connection_parameters`. + display_name: The name to show instead of the stored one, from + :func:`account_tool_names`. + """ + from docsgpt.connectors.permissions import action_access, action_permission + + actions = [] + for action in _json(row.get("actions")) or []: + if not isinstance(action, dict): + continue + access = action_access(row.get("name"), action) + actions.append( + { + "name": action.get("name"), + "description": action.get("description", ""), + "access": access, + "permission": action_permission(action), + "parameters": serialize_parameters(action, account_parameters), + } + ) + return { + "id": str(row["id"]), + "name": row.get("name"), + "display_name": display_name or row.get("custom_name") or row.get("display_name") or row.get("name"), + "status": bool(row.get("status")), + "credential_mode": row.get("credential_mode") or "owner", + "actions": actions, + } + + +def connection_detail(conn, row: dict) -> dict: + """A connection with the sources and tools it feeds.""" + repo = ConnectorSessionsRepository(conn) + status = normalize_status(row) + sources = [serialize_source(s, status) for s in repo.list_sources(str(row["id"]))] + account_parameters = connection_parameters(row) if status == STATUS_CONNECTED else {} + tool_rows = repo.list_tools(str(row["id"])) + names = account_tool_names(conn, tool_rows) + tools = [serialize_tool(t, account_parameters, names.get(str(t["id"]))) for t in tool_rows] + detail = serialize_connection(row, {"sources": len(sources), "tools": len(tools)}) + detail["sources"] = sources + detail["tools"] = tools + detail["writes"] = builtin_writes(catalog.get_definition(detail["connector_key"]), tool_rows) + return detail + + +def _card_state(definition: ConnectorDefinition, available: bool, disabled: bool, status: Optional[str]) -> str: + if disabled: + return "disabled" + if status in (STATUS_RECONNECT, STATUS_ERROR): + return "reconnect" + if status == STATUS_CONNECTED: + return "connected" + if not available: + return "needs_setup" + if definition.publisher == "custom": + return "custom" + return "available" + + +def catalog_for_user(conn, user_id: str, *, is_admin: bool, policies: Optional[dict] = None) -> list[dict]: + """Catalog entries with availability and the caller's connection summary. + + Args: + conn: Open database connection. + user_id: The caller. + is_admin: Admins see the names of missing server settings; everyone + else only learns that setup is needed. + policies: ``connector_key`` to policy row, when admin policies exist. + """ + if policies is None: + policies = load_policies(conn) + by_key: dict[str, list[str]] = {} + for connection in list_connections(conn, user_id): + by_key.setdefault(connection["connector_key"], []).append(connection["status"]) + + entries = [] + for definition in catalog.all_definitions(): + policy = policies.get(definition.key) or {} + disabled = not connector_is_enabled(policies, definition.key) + missing = definition.missing_settings + available = not missing and not disabled + statuses = by_key.get(definition.key, []) + if not available and not statuses: + # Members only see what they can use, plus what they connected + # before an admin turned it off (to manage or remove it). + continue + status = worst_status(statuses) + entries.append( + { + **definition.to_dict(), + "available": available, + "disabled": disabled, + "needs_setup": bool(missing), + "missing_settings": missing if is_admin else [], + "connected_count": sum(1 for s in statuses if s == STATUS_CONNECTED), + "connection_count": len(statuses), + "status": status, + "state": _card_state(definition, available, disabled, status), + "credential_policy": policy.get("credential_mode") or "choose", + "writes_allowed": writes_allowed(policies, definition.key), + } + ) + return entries + + +def disconnect(conn, row: dict) -> dict: + """Forget a connection's credentials but keep the row and what it feeds. + + Revokes the grant at the provider where that is supported (Google); + a failed revocation never blocks forgetting the tokens here. Sources keep + their indexed content and pause syncing; tools stop working until the + account is reconnected. An MCP server's client registration is kept so + reconnecting skips dynamic client registration. + + Args: + conn: Open database connection inside a transaction. + row: The connection row, already authorised for the caller. + + Returns: + The connection's public shape after the change. + """ + from sqlalchemy import text + + from docsgpt.security.encryption import CredentialDecryptionError + + try: + secrets = read_secrets(row) + except CredentialDecryptionError: + secrets = {} + revoke_at_provider(row, secrets) + _forget_mcp_clients(row) + kept = {"client_info": secrets["client_info"]} if secrets.get("client_info") else {} + write_secrets( + conn, row, kept, status=STATUS_DISCONNECTED, session_token=None, last_error=None, + ) + conn.execute( + text( + "UPDATE sources SET metadata = metadata || '{\"sync_state\": \"paused_reconnect\"}'::jsonb " + "WHERE connection_id = CAST(:id AS uuid)" + ), + {"id": str(row["id"])}, + ) + return serialize_connection(ConnectorSessionsRepository(conn).get(str(row["id"]))) + + +def revoke_at_provider(row: dict, secrets: dict) -> bool: + """Best-effort revocation of an OAuth grant. Returns whether it succeeded. + + Only Google offers a public revocation endpoint for these apps; Microsoft + and Atlassian tokens are simply forgotten locally. + """ + import logging + + import requests + + token_info = secrets.get("token_info") or {} + token = token_info.get("refresh_token") or token_info.get("access_token") + if row.get("provider") != "google_drive" or not token: + return False + try: + response = requests.post( + "https://oauth2.googleapis.com/revoke", + data={"token": token}, + headers={"Content-Type": "application/x-www-form-urlencoded"}, + timeout=10, + ) + return response.status_code == 200 + except requests.RequestException as exc: + logging.getLogger(__name__).warning("Google token revocation failed: %s", type(exc).__name__) + return False + + +# --------------------------------------------------------------------------- +# Credentials +# --------------------------------------------------------------------------- +# +# A connection's secrets are one dict, encrypted as a whole into +# ``encrypted_credentials``: +# +# token_info OAuth tokens of an ingest provider (Drive, SharePoint, Confluence) +# tokens MCP OAuth tokens +# client_info MCP dynamic client registration (client_id / client_secret) +# credentials API keys and other pasted secrets, by credential field key +# +# This module is the only caller of encrypt_json / decrypt_json for +# connections, and get_valid_token_info the only reader of OAuth tokens. + +DECRYPT_ERROR = "Stored credentials could not be decrypted. Reconnect to continue." +_SECRET_SESSION_KEYS = ("tokens", "client_info") + + +class ConnectionUnavailable(ValueError): + """A connection cannot be used until its owner reconnects it. + + Attributes: + connection_id: The connection, when known. + status: ``reconnect_needed``, ``disconnected`` or ``missing``. + """ + + def __init__(self, message: str, *, connection_id: Optional[str] = None, status: str = STATUS_RECONNECT): + super().__init__(message) + self.connection_id = connection_id + self.status = status + + +class TransientConnectionError(Exception): + """The provider failed in a way worth retrying (5xx, rate limit, network).""" + + +class EncryptionKeyNotConfigured(Exception): + """New credentials are refused while a multi-user install uses the public default key.""" + + +def ensure_can_store_credentials() -> None: + """Refuse to store new credentials under the public default encryption key. + + Only multi-user installs (any ``AUTH_TYPE``) are refused; a single-user + local install keeps working and logs a warning at startup instead. + + Raises: + EncryptionKeyNotConfigured: When the key is the default and auth is on. + """ + from docsgpt.core.settings import settings + from docsgpt.security.encryption import is_default_encryption_key + + if settings.AUTH_TYPE and is_default_encryption_key(): + raise EncryptionKeyNotConfigured("Set ENCRYPTION_SECRET_KEY before connecting services.") + + +def read_secrets(row: dict) -> dict: + """Decrypt a connection's secrets. + + Rows not yet converted by migration 0040 (or written by an older process + during a rolling deploy) still carry plaintext ``token_info`` and + ``session_data`` tokens; those are read as they are. + + Raises: + CredentialDecryptionError: The envelope cannot be decrypted. + """ + from docsgpt.security.encryption import decrypt_json + + blob = row.get("encrypted_credentials") + if blob: + return decrypt_json(blob, row["user_id"]) + secrets: dict = {} + token_info = _json(row.get("token_info")) + if isinstance(token_info, dict) and token_info: + secrets["token_info"] = token_info + session_data = _json(row.get("session_data")) or {} + if isinstance(session_data, dict): + for key in _SECRET_SESSION_KEYS: + if key in session_data: + secrets[key] = session_data[key] + return secrets + + +def _has_refresh(secrets: dict) -> bool: + token_info = secrets.get("token_info") or {} + tokens = secrets.get("tokens") or {} + return bool( + (isinstance(token_info, dict) and token_info.get("refresh_token")) + or (isinstance(tokens, dict) and tokens.get("refresh_token")) + ) + + +def write_secrets(conn, row: dict, secrets: dict, **fields: Any) -> None: + """Replace a connection's secrets and clear every plaintext copy. + + Args: + conn: Open connection inside a transaction. + row: The connection row (only ``id``, ``user_id`` and ``session_data`` + are read). + secrets: The complete secrets dict; empty clears the credentials. + **fields: Other columns to update in the same statement. + """ + from docsgpt.security.encryption import encrypt_json + + session_data = _json(row.get("session_data")) or {} + if isinstance(session_data, dict): + session_data = {k: v for k, v in session_data.items() if k not in _SECRET_SESSION_KEYS} + else: + session_data = {} + update = { + "encrypted_credentials": encrypt_json(secrets, row["user_id"]) if secrets else None, + "has_refresh_token": _has_refresh(secrets), + "token_info": None, + "session_data": session_data, + **fields, + } + token_info = secrets.get("token_info") + if isinstance(token_info, dict) and token_info.get("scopes") and "scopes" not in fields: + scopes = token_info["scopes"] + update["scopes"] = scopes.split() if isinstance(scopes, str) else list(scopes) + ConnectorSessionsRepository(conn).update(str(row["id"]), update) + + +def credential_hint(credentials: dict) -> str: + """``…abcd``: the last four characters of the first long secret, never more.""" + for value in credentials.values(): + if isinstance(value, str) and len(value) >= 8: + return "…" + value[-4:] + return "…" + + +def mark_reconnect_needed(connection_id: str, error: str) -> None: + """Flag a connection whose credentials stopped working and tell its owner. + + Pauses syncing on every source the connection feeds (no retry storm) and + publishes a ``connection.reconnect_needed`` user event, which the + frontend shows as a toast with a Reconnect action. Runs in its own + transaction so it sticks even when the caller's transaction rolls back. + """ + from sqlalchemy import text + + from docsgpt.events.publisher import publish_user_event + with db_session() as conn: + repo = ConnectorSessionsRepository(conn) + row = repo.get(connection_id) + if row is None: + return + already = normalize_status(row) == STATUS_RECONNECT + counts = repo.resource_counts([connection_id]).get(str(connection_id), {}) + repo.update(connection_id, {"status": STATUS_RECONNECT, "last_error": error[:500]}) + conn.execute( + text( + "UPDATE sources SET metadata = metadata || '{\"sync_state\": \"paused_reconnect\"}'::jsonb " + "WHERE connection_id = CAST(:id AS uuid)" + ), + {"id": connection_id}, + ) + if not already: + publish_user_event( + row["user_id"], + "connection.reconnect_needed", + { + "connection_id": connection_id, + "connector_key": catalog.connector_key_for_row(row), + "name": serialize_connection(row)["name"], + "source_count": counts.get("sources", 0), + "tool_count": counts.get("tools", 0), + }, + scope={"kind": "connection", "id": connection_id}, + ) + + +def resume_sources(conn, connection_id: str) -> None: + """Lift the reconnect pause from a connection's sources.""" + from sqlalchemy import text + + conn.execute( + text( + "UPDATE sources SET metadata = metadata - 'sync_state' " + "WHERE connection_id = CAST(:id AS uuid) AND metadata ? 'sync_state'" + ), + {"id": connection_id}, + ) + + +def load_secrets(row: dict) -> Optional[dict]: + """Decrypt a row's secrets; on failure flag it for reconnect and return None.""" + from docsgpt.security.encryption import CredentialDecryptionError + + try: + return read_secrets(row) + except CredentialDecryptionError: + mark_reconnect_needed(str(row["id"]), DECRYPT_ERROR) + return None + + +def _is_auth_failure(exc: BaseException) -> bool: + """Whether a refresh failure means the grant is gone (vs. a retryable blip).""" + import requests + + response = getattr(exc, "response", None) + status = getattr(response, "status_code", None) + if isinstance(exc, requests.exceptions.HTTPError) and status is not None: + return status in (400, 401, 403) + if isinstance(exc, (requests.exceptions.ConnectionError, requests.exceptions.Timeout)): + return False + message = str(exc).lower() + if any(word in message for word in ("timed out", "temporarily", "rate limit", "503", "502", "504")): + return False + try: + from google.auth.exceptions import RefreshError, TransportError + + if isinstance(exc, TransportError): + return False + if isinstance(exc, RefreshError): + return True + except ImportError: # pragma: no cover - google libs are core deps + pass + return True + + +def connection_id_for_session_token(session_token: Optional[str]) -> str: + """The connection behind a legacy browser session token. + + Raises: + ConnectionUnavailable: No connection holds that token. + """ + from docsgpt.parser.connectors._auth_utils import session_token_fingerprint + row = None + if session_token: + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get_by_session_token(session_token) + if row is None: + raise ConnectionUnavailable( + f"Invalid session token ({session_token_fingerprint(session_token or '')})", status="missing", + ) + return str(row["id"]) + + +def get_valid_token_info(connection_id: str, *, rejected_access_token: Optional[str] = None) -> dict: + """An unexpired OAuth ``token_info`` for an ingest connection. + + Holds a row lock (``SELECT ... FOR UPDATE``) while it checks expiry and + refreshes, and writes the rotated refresh token back in the same + transaction, so two workers refreshing a rotating token (Microsoft, + Atlassian) never spend the same one twice: the second waits and then + reads the first one's fresh token. + + Args: + connection_id: The connection to use. + rejected_access_token: An access token the provider just answered + with 401. The token is refreshed when it is still the stored one; + when another worker already replaced it, the new one is returned. + + Returns: + The decrypted token info, refreshed if it was about to expire. + + Raises: + ConnectionUnavailable: The connection is missing, disconnected, cannot + be decrypted, or its grant was revoked (it is then flagged + ``reconnect_needed``). + TransientConnectionError: The provider failed in a retryable way. + """ + from docsgpt.parser.connectors.connector_creator import ConnectorCreator + from docsgpt.security.encryption import CredentialDecryptionError + failure: Optional[str] = None + transient: Optional[BaseException] = None + token_info: Optional[dict] = None + with db_session() as conn: + repo = ConnectorSessionsRepository(conn) + row = repo.get_for_update(connection_id) + if row is None: + raise ConnectionUnavailable("Connection not found", connection_id=connection_id, status="missing") + status = normalize_status(row) + if status == STATUS_DISCONNECTED: + raise ConnectionUnavailable( + "Connection is disconnected", connection_id=connection_id, status=STATUS_DISCONNECTED + ) + try: + secrets = read_secrets(row) + except CredentialDecryptionError: + secrets, failure = {}, DECRYPT_ERROR + token_info = secrets.get("token_info") if not failure else None + if not failure and not token_info: + failure = "No stored sign-in for this connection. Reconnect to continue." + if not failure: + auth = ConnectorCreator.create_auth(row["provider"]) + rejected = bool(rejected_access_token) and token_info.get("access_token") == rejected_access_token + if rejected or auth.is_token_expired(token_info): + refresh_token = token_info.get("refresh_token") + if not refresh_token: + failure = "The sign-in expired and cannot be renewed. Reconnect to continue." + else: + try: + refreshed = auth.refresh_access_token(refresh_token) + except Exception as exc: # classified below + if _is_auth_failure(exc): + failure = f"The provider rejected the stored sign-in: {type(exc).__name__}" + else: + transient = exc + else: + merged = {**token_info, **{k: v for k, v in refreshed.items() if v is not None}} + token_info = auth.sanitize_token_info(merged) + write_secrets( + conn, row, {**secrets, "token_info": token_info}, + status=STATUS_CONNECTED, last_error=None, + ) + if not failure and transient is None: + repo.update(connection_id, {"last_used_at": _now()}) + if failure: + mark_reconnect_needed(connection_id, failure) + raise ConnectionUnavailable(failure, connection_id=connection_id) + if transient is not None: + raise TransientConnectionError(str(transient)) from transient + return token_info + + +def _now(): + import datetime + + return datetime.datetime.now(datetime.timezone.utc) + + +def get_credentials(row: dict) -> dict: + """The pasted credentials (API keys) of an ``api_key`` connection. + + Raises: + ConnectionUnavailable: Disconnected, flagged, or undecryptable. + """ + status = normalize_status(row) + if status in (STATUS_DISCONNECTED, STATUS_RECONNECT): + raise ConnectionUnavailable(f"Connection is {status}", connection_id=str(row["id"]), status=status) + secrets = load_secrets(row) + if secrets is None: + raise ConnectionUnavailable(DECRYPT_ERROR, connection_id=str(row["id"])) + return dict(secrets.get("credentials") or {}) + + +def access_credentials(row: dict) -> dict: + """What a loader or tool authenticates with through this connection. + + Pasted credentials for an ``api_key`` connection. An OAuth connection + whose token a loader or tool sends itself (GitHub's App sign-in) gives + its current access token as ``access_token``, refreshed first when it + has expired. + + Raises: + ConnectionUnavailable: Disconnected, flagged, undecryptable, or the + refresh was refused (the connection is then flagged). + TransientConnectionError: The provider failed in a retryable way. + """ + if (row.get("auth_kind") or "") == "oauth": + status = normalize_status(row) + if status in (STATUS_DISCONNECTED, STATUS_RECONNECT): + raise ConnectionUnavailable(f"Connection is {status}", connection_id=str(row["id"]), status=status) + return {"access_token": get_valid_token_info(str(row["id"])).get("access_token")} + return get_credentials(row) + + +def connection_parameters(row: Optional[dict]) -> dict: + """Tool parameters a connection sets, from its credential fields. + + Only fields the catalog marks with ``parameter`` count (Telegram's + default chat); an empty value sets nothing. A connection that cannot be + read sets nothing either: the call then fails on its credentials. + + Args: + row: The connection row, or None. + + Returns: + Parameter name to value. + """ + if row is None: + return {} + fields = catalog.parameter_fields(catalog.connector_key_for_row(row)) + if not fields: + return {} + try: + credentials = get_credentials(row) + except ConnectionUnavailable: + return {} + return { + f.parameter: credentials[f.key] + for f in fields + if credentials.get(f.key) not in (None, "") + } + + +def _api_key_account( + repo: ConnectorSessionsRepository, + user_id: str, + connector_key: str, + server_url: Optional[str], + label: str, + credentials: dict, +) -> tuple[Optional[dict], str]: + """The connection holding exactly ``credentials``, or a free label for a new one. + + A label (a hint of the key, or what the user typed) is not an identity: + two different keys can share it. A row under the label is reused only + when it holds the same credentials; otherwise the label gets a + ``(2)``, ``(3)`` suffix until it names a matching row or no row. + + Returns: + ``(row, label)``: the row to reuse (None to create one) and its label. + """ + from docsgpt.security.encryption import CredentialDecryptionError + + candidate, suffix = label, 1 + while True: + existing = repo.find_account(user_id, connector_key, server_url=server_url, account_label=candidate) + if existing is None: + return None, candidate + try: + if (read_secrets(existing).get("credentials") or {}) == credentials: + return existing, candidate + except CredentialDecryptionError: + pass + suffix += 1 + candidate = f"{label} ({suffix})" + + +def create_api_key_connection( + conn, + user_id: str, + definition, + credentials: dict, + *, + label: Optional[str] = None, + server_url: Optional[str] = None, + display_name: Optional[str] = None, +) -> tuple[dict, bool]: + """Store a connection for pasted credentials, or reuse the matching one. + + The same credentials for the same service map to one connection + ("enter secrets once"): adding a second tool or bucket reuses it. + + Args: + conn: Open connection inside a transaction. + user_id: The owner. + definition: The catalog entry (``api_key`` or ``custom_mcp``). + credentials: Field key to value, as the user entered them. + label: What the account is called; defaults to a hint of the key. + server_url: Base URL, for custom MCP servers. + display_name: Name shown for custom connectors. + + Returns: + ``(row, created)``. + + Raises: + ConnectorDisabled: An admin turned the connector off. Checked first, + so callers that fall back to a legacy path on the other errors + never do so for a disabled connector. A custom MCP server at a + preset's address is that preset (see + :func:`catalog.connector_key_for_row`), so its switch applies. + ValueError: A required credential field is missing. + EncryptionKeyNotConfigured: See :func:`ensure_can_store_credentials`. + """ + preset = catalog.preset_for_url(server_url) if definition.key == "custom_mcp" else None + ensure_connector_allowed(conn, preset.key if preset else definition.key) + fields = {f.key: f for f in definition.credential_fields} + if fields: + missing = [f.label for f in fields.values() if f.required and not str(credentials.get(f.key) or "").strip()] + if missing: + raise ValueError(f"Missing credentials: {', '.join(missing)}") + credentials = {k: v for k, v in credentials.items() if k in fields and v not in (None, "")} + else: + credentials = {k: v for k, v in credentials.items() if v not in (None, "")} + ensure_can_store_credentials() + secret_values = {k: v for k, v in credentials.items() if (fields.get(k).secret if fields.get(k) else True)} + repo = ConnectorSessionsRepository(conn) + existing, account_label = _api_key_account( + repo, user_id, definition.key, server_url, label or credential_hint(secret_values or credentials), credentials, + ) + if existing is not None: + write_secrets(conn, existing, {"credentials": credentials}, status=STATUS_CONNECTED, last_error=None) + resume_sources(conn, str(existing["id"])) + return repo.get(str(existing["id"])), False + from docsgpt.security.encryption import encrypt_json + + row = repo.create( + user_id, + definition.key, + connector_key=definition.key, + auth_kind="api_key", + display_name=display_name or definition.name, + account_label=account_label, + server_url=server_url, + encrypted_credentials=encrypt_json({"credentials": credentials}, user_id), + ) + if row is None: # lost a race with an identical insert + row = repo.find_account(user_id, definition.key, server_url=server_url, account_label=account_label) + return row, False + return row, True + + +# --------------------------------------------------------------------------- +# OAuth sign-in +# --------------------------------------------------------------------------- + + +def begin_oauth(conn, user_id: str, provider: str, connection_id: Optional[str] = None) -> dict: + """The row an OAuth sign-in writes into when it completes. + + Reconnecting passes the connection being reconnected. A new sign-in + gets a pending row with no account label; the callback moves the tokens + onto the existing row for that account if there is one. + + Raises: + ConnectionUnavailable: ``connection_id`` is not the caller's + connection for ``provider``. + """ + from sqlalchemy import text + + repo = ConnectorSessionsRepository(conn) + if connection_id: + row = repo.get_for_user(connection_id, user_id) + if row is None or row.get("provider") != provider: + raise ConnectionUnavailable("Connection not found", connection_id=connection_id, status="missing") + return row + ensure_connector_allowed(conn, provider) + definition = catalog.get_definition(provider) + result = conn.execute( + text( + """ + INSERT INTO connector_sessions (user_id, provider, status, connector_key, auth_kind, display_name) + VALUES (:user_id, :provider, 'pending', :provider, 'oauth', :display_name) + ON CONFLICT (user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, '')) + DO UPDATE SET status = CASE + WHEN connector_sessions.status IN ('connected', 'authorized') THEN connector_sessions.status + ELSE 'pending' END + RETURNING * + """ + ), + {"user_id": user_id, "provider": provider, "display_name": definition.name if definition else provider}, + ) + from docsgpt.storage.db.base_repository import row_to_dict + + return row_to_dict(result.fetchone()) + + +def complete_oauth(conn, state_row: dict, provider: str, token_info: dict, account: str) -> dict: + """Store a finished OAuth sign-in and return the connection it belongs to. + + Signing in to an account that already has a connection updates that + connection (and drops the pending placeholder), so reconnecting from any + entry point heals every source and tool of the account. Reconnecting a + connection but signing in to a different account never rewrites it (its + sources and tools would silently run as the new account): the new + account gets its own connection and the original keeps its status. + + Args: + conn: Open connection inside a transaction. + state_row: The row named in the OAuth ``state``. + provider: ``google_drive``, ``share_point`` or ``confluence``. + token_info: Sanitised token info from the provider. + account: The account's email or name, shown as "Connected as". + + Returns: + The connection now holding the sign-in. + """ + import uuid + + repo = ConnectorSessionsRepository(conn) + target = state_row + definition = catalog.get_definition(provider) + display_name = definition.name if definition else provider + existing = repo.find_account(state_row["user_id"], provider, server_url=None, account_label=account) + if existing is not None and str(existing["id"]) != str(state_row["id"]): + target = existing + if not has_credentials(state_row) and normalize_status(state_row) == STATUS_PENDING: + repo.delete_by_id(str(state_row["id"])) + elif existing is None and state_row.get("account_label"): + # The row belongs to another account (a reconnect that signed in as someone else). + target = repo.create( + state_row["user_id"], provider, connector_key=provider, auth_kind="oauth", + display_name=display_name, account_label=account, status=STATUS_PENDING, + ) or repo.find_account(state_row["user_id"], provider, server_url=None, account_label=account) + write_secrets( + conn, + target, + {"token_info": token_info}, + status=STATUS_CONNECTED, + # Kept for frontends from before connections; new ones use the id. + session_token=str(uuid.uuid4()), + user_email=account, + account_label=account, + connector_key=provider, + auth_kind="oauth", + display_name=display_name, + last_error=None, + ) + resume_sources(conn, str(target["id"])) + return repo.get(str(target["id"])) + + +def picker_token(connection_id: str) -> dict: + """What a browser-side picker needs: a short-lived access token, never the refresh token.""" + token_info = get_valid_token_info(connection_id) + return { + "access_token": token_info.get("access_token"), + "expiry": token_info.get("expiry"), + "allows_shared_content": bool(token_info.get("allows_shared_content")), + } + + +def claim_session_token(conn, user_id: str, provider: str, session_token: str) -> Optional[dict]: + """Link a legacy browser session token to its connection, once. + + Returns the connection when the token belongs to ``user_id`` and + ``provider``. The token itself keeps working for the session-token + routes during this release, so an older tab does not break. + """ + repo = ConnectorSessionsRepository(conn) + row = repo.get_by_session_token(session_token) + if not row or row.get("user_id") != user_id or (row.get("provider") or "").lower() != provider.lower(): + return None + return row + + +def resolve_request_connection(user_id: str, provider: Optional[str], data: dict) -> Optional[dict]: + """The caller's connection a request names, by ``connection_id`` or legacy ``session_token``. + + Returns None unless the caller owns the connection and it belongs to + ``provider`` (when given). + """ + connection_id = data.get("connection_id") + session_token = data.get("session_token") + if not connection_id and not session_token: + return None + with db_readonly() as conn: + repo = ConnectorSessionsRepository(conn) + if connection_id: + row = repo.get_for_user(str(connection_id), user_id) + else: + row = repo.get_by_session_token(session_token) + if not owns_connector_session(row, user_id, provider): + return None + if row is None: + return None + if provider and (row.get("provider") or "").lower() != provider.lower(): + return None + return row + + +# --------------------------------------------------------------------------- +# MCP OAuth token storage +# --------------------------------------------------------------------------- + + +def mcp_provider(base_url: str) -> str: + """``provider`` value MCP OAuth connections are stored under.""" + return f"mcp:{base_url}" + + +def _mcp_row(conn, user_id: str, base_url: str, connection_id: Optional[str], *, lock: bool = False): + from sqlalchemy import text + + from docsgpt.storage.db.base_repository import row_to_dict + + repo = ConnectorSessionsRepository(conn) + if connection_id: + row = repo.get_for_update(connection_id) if lock else repo.get(connection_id) + # A connection's tokens only ever go to its own server. Ownership is + # checked where the id is chosen (the tool executor); clients cannot + # supply one (see ``_sanitize_mcp_transport``). + if row is None or row.get("provider") != mcp_provider(base_url): + return None + return row + result = conn.execute( + text( + "SELECT * FROM connector_sessions WHERE user_id = :user_id AND provider = :provider " + "ORDER BY updated_at DESC LIMIT 1" + (" FOR UPDATE" if lock else "") + ), + {"user_id": user_id, "provider": mcp_provider(base_url)}, + ) + row = result.fetchone() + return row_to_dict(row) if row is not None else None + + +def read_mcp_secrets(user_id: str, base_url: str, connection_id: Optional[str] = None) -> dict: + """The MCP OAuth ``tokens`` and ``client_info`` of a server connection.""" + with db_readonly() as conn: + row = _mcp_row(conn, user_id, base_url, connection_id) + if row is None: + return {} + return load_secrets(row) or {} + + +def update_mcp_secrets( + user_id: str, + base_url: str, + patch: dict, + *, + connection_id: Optional[str] = None, + status: Optional[str] = None, +) -> dict: + """Merge ``patch`` into an MCP connection's secrets (``None`` drops a key). + + Creates the connection on first use, named after the matching preset or + the server's host. A write for a named connection never creates one: a + client or sync still running when its connection was removed would + otherwise bring it back, holding the tokens it just renewed. + + Returns: + The connection row after the update. + + Raises: + ConnectionUnavailable: ``connection_id`` names no connection of this + user for this server (removed, or another server's). + """ + from docsgpt.security.encryption import CredentialDecryptionError + + with db_session() as conn: + repo = ConnectorSessionsRepository(conn) + row = _mcp_row(conn, user_id, base_url, connection_id, lock=True) + if row is None and connection_id: + raise ConnectionUnavailable("Connection not found", connection_id=connection_id, status="missing") + if row is None: + row = repo.merge_session_data(user_id, mcp_provider(base_url), base_url, {}) + try: + secrets = read_secrets(row) + except CredentialDecryptionError: + secrets = {} + for key, value in patch.items(): + if value is None: + secrets.pop(key, None) + else: + secrets[key] = value + fields: dict = {} + if status: + fields["status"] = status + if status == STATUS_CONNECTED: + fields["last_error"] = None + elif not row.get("status"): + fields["status"] = STATUS_CONNECTED if secrets.get("tokens") else STATUS_PENDING + if not row.get("connector_key"): + preset = catalog.preset_for_url(base_url) + fields["connector_key"] = preset.key if preset else "custom_mcp" + fields["auth_kind"] = "mcp_oauth" + fields["display_name"] = preset.name if preset else base_url.split("://")[-1] + write_secrets(conn, row, secrets, **fields) + if fields.get("status") == STATUS_CONNECTED: + resume_sources(conn, str(row["id"])) + return repo.get(str(row["id"])) + + +# --------------------------------------------------------------------------- +# Tools created from a connection +# --------------------------------------------------------------------------- + + +def _transform_actions(actions: list) -> list: + """``transform_actions`` from the tools API: active, LLM-filled parameters.""" + transformed = [] + for action in actions: + action = dict(action) + action["active"] = True + parameters = action.get("parameters") + if isinstance(parameters, dict): + for details in (parameters.get("properties") or {}).values(): + if isinstance(details, dict): + details["filled_by_llm"] = True + details["value"] = "" + transformed.append(action) + return transformed + + +def create_tool_for_connection( + conn, + user_id: str, + connection: dict, + *, + template: Optional[str] = None, + display_name: Optional[str] = None, + config: Optional[dict] = None, + actions: Optional[list] = None, + permissions: Optional[dict] = None, + status: bool = True, +) -> dict: + """Create the tool a connection provides, with read / write defaults. + + Writes default to "Needs approval", reads to "Always allow"; the + ``permissions`` map (action name to ``always`` / ``ask`` / ``off``) + overrides them. Secrets are never copied onto the tool: the executor + reads them from the connection at run time. + + Args: + conn: Open connection inside a transaction. + user_id: The owner of the connection and the new tool. + connection: The connection row. + template: ``user_tools`` name; defaults to the connector's first template. + display_name: Name shown for the tool. + config: Non-secret tool configuration (an MCP server URL, say). + actions: Action metadata; defaults to the tool class's own. + permissions: Per-action permission overrides. + status: Whether the tool starts enabled. + + Returns: + The new ``user_tools`` row. + """ + from docsgpt.agents.tools.tool_manager import ToolManager + from docsgpt.connectors.permissions import apply_default_permissions, apply_permission + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + key = catalog.connector_key_for_row(connection) + definition = catalog.get_definition(key) + template = template or (definition.tool_templates[0] if definition and definition.tool_templates else None) + if not template: + raise ValueError(f"Connector {key} provides no tool") + tool = ToolManager(config={}).tools.get(template) + if tool is None: + raise ValueError(f"Unknown tool template: {template}") + doc = (tool.__doc__ or template).strip().split("\n", 1) + if actions is None: + actions = tool.get_actions_metadata() + actions = apply_default_permissions(template, _transform_actions(actions)) + for index, action in enumerate(actions): + permission = (permissions or {}).get(action.get("name")) + if permission: + actions[index] = apply_permission(action, permission) + name = display_name or (serialize_connection(connection)["name"] if connection else doc[0].strip()) + return UserToolsRepository(conn).create( + user_id, + template, + config=dict(config or {}), + custom_name=name, + display_name=name, + description=doc[1].strip() if len(doc) > 1 else "", + config_requirements=tool.get_config_requirements(), + actions=actions, + status=status, + connection_id=str(connection["id"]), + ) + + +def split_secrets(config: dict, config_requirements: dict) -> tuple[dict, dict]: + """``(public, secrets)``: a tool config split by its secret requirements.""" + secret_keys = {k for k, spec in (config_requirements or {}).items() if spec.get("secret")} + public = {k: v for k, v in config.items() if k not in secret_keys} + secrets = {k: v for k, v in config.items() if k in secret_keys and v not in (None, "")} + return public, secrets + + +def builtin_mcp_config(definition: Optional[ConnectorDefinition], writes: bool = False) -> Optional[dict]: + """Tool config of a built-in connector whose tool is its service's MCP server. + + The token is not part of it: the executor reads it from the connection + at run time and sends it as a bearer token. + + Args: + definition: The connector. + writes: Use the endpoint that also offers write actions, when the + connector has one (GitHub's full server). + + Returns: + The config, or None when the connector has no such tool. + """ + if ( + definition is None + or definition.publisher != "built_in" + or not definition.mcp_url + or "mcp_tool" not in definition.tool_templates + ): + return None + url = definition.mcp_write_url if writes and definition.mcp_write_url else definition.mcp_url + return {"server_url": url, "auth_type": "bearer", "transport_type": "http", "timeout": 30} + + +def builtin_mcp_url(definition: Optional[ConnectorDefinition], stored_url: Optional[str], allowed: bool) -> str: + """The endpoint a built-in connector's MCP tool may call, whatever its config says. + + Only the connector's own endpoints are ever used: the write endpoint when + the tool was set up for writes and an admin allows them, the read-only + one otherwise (another path on the same host included). + + Args: + definition: The connector, with ``mcp_url`` set. + stored_url: The ``server_url`` in the tool's config. + allowed: Whether an admin allows writes through this connector. + """ + if allowed and definition.mcp_write_url and stored_url == definition.mcp_write_url: + return definition.mcp_write_url + return definition.mcp_url or "" + + +def builtin_writes(definition: Optional[ConnectorDefinition], tool_rows: Iterable[dict]) -> Optional[bool]: + """Whether a connection's built-in MCP tool is set up for writes. + + Returns: + None when the connector offers no writes or the connection has no such + tool yet; otherwise whether the tool uses the write endpoint. + """ + if definition is None or not definition.mcp_write_url or builtin_mcp_config(definition) is None: + return None + for tool in tool_rows: + if tool.get("name") == "mcp_tool": + return (_json(tool.get("config")) or {}).get("server_url") == definition.mcp_write_url + return None + + +def needs_mcp_discovery(conn, connection: dict) -> bool: + """Whether creating this connection's tools first needs its MCP server's actions.""" + definition = catalog.get_definition(catalog.connector_key_for_row(connection)) + if builtin_mcp_config(definition) is None: + return False + have = {tool.get("name") for tool in ConnectorSessionsRepository(conn).list_tools(str(connection["id"]))} + return "mcp_tool" not in have + + +def ensure_connection_tools( + conn, + user_id: str, + connection: dict, + permissions: Optional[dict] = None, + mcp_actions: Optional[list] = None, + mcp_writes: bool = False, +) -> list[dict]: + """Create the connector's tools once; later calls return the existing ones. + + This is what makes the setup step idempotent for tools: a retried or + repeated setup never creates a second Telegram tool for the same bot. + + Args: + conn: Open connection inside a transaction. + user_id: The owner. + connection: The connection row. + permissions: Per-action permission overrides. + mcp_actions: The actions of a built-in connector's MCP server + (GitHub's), discovered by the caller outside this transaction. + Without them that tool is not created. + mcp_writes: Point that tool at the endpoint that also offers writes; + the caller checked that an admin allows them. + """ + repo = ConnectorSessionsRepository(conn) + existing = repo.list_tools(str(connection["id"])) + key = catalog.connector_key_for_row(connection) + definition = catalog.get_definition(key) + if not definition or not definition.tool_templates: + return existing + have = {tool.get("name") for tool in existing} + mcp_config = builtin_mcp_config(definition, writes=mcp_writes) + created = [] + for template in definition.tool_templates: + if template in have: + continue + if template == "mcp_tool" and mcp_config is not None and mcp_actions is not None: + created.append(create_tool_for_connection( + conn, user_id, connection, template=template, config=mcp_config, + actions=mcp_actions, permissions=permissions, + )) + continue + if template in ("mcp_tool", "api_tool"): + # Other MCP tools are created by the MCP save flow, which has the + # discovered actions; OpenAPI tools come from an imported spec. + continue + created.append( + create_tool_for_connection(conn, user_id, connection, template=template, permissions=permissions) + ) + return existing + created + + +def _forget_mcp_clients(row: dict) -> None: + """Drop the MCP clients this process cached for a connection's tokens. + + A cached client keeps the tokens it signed in with for a few minutes, so + after the connection is removed or disconnected it would still answer as + signed in (and save a tool with no connection behind it). + """ + if not str(row.get("provider") or "").startswith("mcp:") and row.get("auth_kind") != "mcp_oauth": + return + # Not imported here: a process that never loaded the MCP tool has no + # clients cached, and importing it from this module would be circular. + mcp_tool = sys.modules.get("docsgpt.agents.tools.mcp_tool") + if mcp_tool is not None: + mcp_tool.forget_cached_clients(str(row["id"]), str(row.get("user_id") or "")) + + +def remove_connection(conn, row: dict, *, sources: str = "keep", tools: str = "delete") -> list[dict]: + """Delete a connection, choosing what happens to what it feeds. + + Args: + conn: Open connection inside a transaction. + row: The connection, already authorised for its owner. + sources: ``keep`` (content stays, syncing stops) or ``delete``. + tools: ``delete`` (the default: they cannot work without it) or ``keep``. + + Returns: + The source rows the caller must delete with their indexes (empty + when ``sources`` is ``keep``). Index files live outside the database, + so the caller deletes them after this transaction commits. + """ + from sqlalchemy import text + + from docsgpt.security.encryption import CredentialDecryptionError + + repo = ConnectorSessionsRepository(conn) + connection_id = str(row["id"]) + _forget_mcp_clients(row) + linked_sources = repo.list_sources(connection_id) + try: + revoke_at_provider(row, read_secrets(row)) + except CredentialDecryptionError: + pass + if tools == "delete": + # Team grants go with the tools (delete trigger) and chat preferences + # by FK cascade; the owner's sharing switches have no FK. + conn.execute( + text( + "WITH gone AS (DELETE FROM user_tools WHERE connection_id = CAST(:id AS uuid) " + "AND user_id = :user_id RETURNING id) " + "DELETE FROM resource_share_settings WHERE resource_type = 'tool' " + "AND resource_id IN (SELECT id FROM gone)" + ), + {"id": connection_id, "user_id": row["user_id"]}, + ) + else: + # The link itself goes with the row (ON DELETE SET NULL); note which + # service the tools lost, so an agent can say why they stopped. + conn.execute( + text( + "UPDATE user_tools SET status = false, " + "config = COALESCE(config, '{}'::jsonb) || jsonb_build_object('removed_connection', " + "CAST(:marker AS jsonb)) WHERE connection_id = CAST(:id AS uuid)" + ), + {"id": connection_id, "marker": json.dumps(catalog.connector_key_for_row(row) or True)}, + ) + conn.execute( + text( + "UPDATE sources SET sync_frequency = 'never', metadata = metadata - 'sync_state' " + "WHERE connection_id = CAST(:id AS uuid)" + ), + {"id": connection_id}, + ) + repo.delete_by_id(connection_id) + return linked_sources if sources == "delete" else [] + + +def set_tool_permissions( + conn, user_id: str, connection_id: str, tool_id: str, permissions: dict, +) -> Optional[dict]: + """Apply ``{action: always | ask | off}`` to a tool the user owns. + + Args: + conn: Open database connection. + user_id: The caller, who must own the tool. + connection_id: The connection the tool must belong to. + tool_id: The tool to update. + permissions: Action name to ``always``, ``ask`` or ``off``. + + Returns: + The updated tool, or None (nothing written) when the tool is not the + user's or belongs to another connection. + """ + from docsgpt.connectors.permissions import apply_permission + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + tools = UserToolsRepository(conn) + tool = tools.get_any(tool_id, user_id) + if tool is None or tool.get("user_id") != user_id or str(tool.get("connection_id")) != connection_id: + return None + actions = [ + apply_permission(action, permissions[action.get("name")]) if action.get("name") in permissions else action + for action in (_json(tool.get("actions")) or []) + ] + tools.update(str(tool["id"]), user_id, {"actions": actions}) + return tools.get_any(str(tool["id"]), user_id) + + +def set_tool_parameters( + conn, user_id: str, connection_id: str, tool_id: str, action_name: str, pins: dict, +) -> Optional[dict]: + """Fix or release parameters of one action of a tool the user owns. + + Args: + conn: Open database connection. + user_id: The caller, who must own the tool. + connection_id: The connection the tool must belong to. + tool_id: The tool to update. + action_name: The action whose parameters change. + pins: Parameter name to the value to always use, or None to let the + model decide. + + Returns: + The updated tool, or None (nothing written) when the tool is not the + user's or belongs to another connection. + + Raises: + ValueError: The action or a parameter does not exist, or a value does + not fit its parameter. + """ + from docsgpt.agents.tool_pins import set_pins + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + tools = UserToolsRepository(conn) + tool = tools.get_any(tool_id, user_id) + if tool is None or tool.get("user_id") != user_id or str(tool.get("connection_id")) != connection_id: + return None + actions = _json(tool.get("actions")) or [] + if not any(isinstance(a, dict) and a.get("name") == action_name for a in actions): + raise ValueError(f"Unknown action: {action_name}") + actions = [ + set_pins(action, pins) if isinstance(action, dict) and action.get("name") == action_name else action + for action in actions + ] + tools.update(str(tool["id"]), user_id, {"actions": actions}) + return tools.get_any(str(tool["id"]), user_id) + + +def reencrypt_all(batch_size: int = 500) -> dict: + """Rewrite every connection's credentials with the current key. + + Run after rotating ENCRYPTION_SECRET_KEY with the old value in + ENCRYPTION_SECRET_KEY_PREVIOUS; afterwards the previous key can go. + + Returns: + ``{"rewritten": n, "current": n, "failed": n}``; failed rows (neither + key opens them) are flagged ``reconnect_needed``. + """ + from sqlalchemy import text + + from docsgpt.security.encryption import CredentialDecryptionError, current_key_id, envelope_key_id + + key_id = current_key_id() + counts = {"rewritten": 0, "current": 0, "failed": 0} + failed_ids: list[str] = [] + last_id = "00000000-0000-0000-0000-000000000000" + while True: + with db_session() as conn: + rows = conn.execute( + text( + "SELECT * FROM connector_sessions WHERE encrypted_credentials IS NOT NULL " + "AND id > CAST(:last AS uuid) ORDER BY id LIMIT :batch FOR UPDATE" + ), + {"last": last_id, "batch": batch_size}, + ).fetchall() + if not rows: + break + from docsgpt.storage.db.base_repository import row_to_dict + + for raw in rows: + row = row_to_dict(raw) + last_id = str(row["id"]) + if envelope_key_id(row["encrypted_credentials"]) == key_id: + counts["current"] += 1 + continue + try: + secrets = read_secrets(row) + except CredentialDecryptionError: + counts["failed"] += 1 + failed_ids.append(last_id) + continue + write_secrets(conn, row, secrets) + counts["rewritten"] += 1 + for connection_id in failed_ids: + mark_reconnect_needed(connection_id, DECRYPT_ERROR) + return counts + + +# --------------------------------------------------------------------------- +# Admin policy +# --------------------------------------------------------------------------- + + +class ConnectorDisabled(Exception): + """An admin turned this connector (or custom MCP servers) off.""" + + +class WritesForbidden(Exception): + """An admin forbade write actions through this connector (GitHub's changes).""" + + +def custom_mcp_allowed(conn) -> bool: + """The instance-wide "Allow custom MCP servers" switch (on unless turned off).""" + from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository + from docsgpt.storage.db.repositories.connector_policies import ALLOW_CUSTOM_MCP_KEY + + return AppMetadataRepository(conn).get(ALLOW_CUSTOM_MCP_KEY) != "false" + + +def connector_is_enabled(policies: dict, connector_key: Optional[str]) -> bool: + """Whether a connector is switched on. + + An admin's explicit switch wins. Without one, a connector is on when it + has the server settings it needs, so one that still needs admin setup + starts off and members never see it. + + Args: + policies: ``connector_key`` to policy row, from :func:`load_policies`. + connector_key: The catalog key. + """ + explicit = (policies.get(connector_key) or {}).get("enabled") if connector_key else None + if explicit is not None: + return bool(explicit) + definition = catalog.get_definition(connector_key) if connector_key else None + return definition is None or definition.configured + + +def load_policies(conn) -> dict[str, dict]: + """Every connector's policy, with custom MCP and write switches folded in. + + A connector that offers writes (GitHub) gets ``allow_writes``: False when + an admin forbade them. + """ + from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository, allow_writes_key + + policies = dict(ConnectorPoliciesRepository(conn).all()) + if not custom_mcp_allowed(conn): + policies["custom_mcp"] = {**policies.get("custom_mcp", {}), "enabled": False} + metadata = AppMetadataRepository(conn) + for definition in catalog.all_definitions(): + if definition.mcp_write_url and metadata.get(allow_writes_key(definition.key)) == "false": + policies[definition.key] = {**policies.get(definition.key, {}), "allow_writes": False} + return policies + + +def writes_allowed(policies: dict, connector_key: Optional[str]) -> bool: + """Whether a connection to this connector may be set up for write actions. + + Args: + policies: ``connector_key`` to policy row, from :func:`load_policies`. + connector_key: The catalog key. + + Returns: + False for a connector that offers no writes, or whose writes an admin + forbade. + """ + definition = catalog.get_definition(connector_key) if connector_key else None + if definition is None or not definition.mcp_write_url: + return False + return (policies.get(connector_key) or {}).get("allow_writes") is not False + + +def connector_enabled(conn, row: dict) -> bool: + """Whether an admin left on the connector a connection row belongs to. + + Args: + conn: Open database connection. + row: A ``connector_sessions`` row. + + Returns: + False when the connector (or custom MCP servers) is turned off. + """ + return connector_is_enabled(load_policies(conn), catalog.connector_key_for_row(row)) + + +def ensure_connector_allowed(conn, connector_key: Optional[str]) -> None: + """Refuse a new connection to a connector an admin turned off. + + Raises: + ConnectorDisabled: The connector, or custom MCP servers, are off. + """ + if not connector_key: + return + if not connector_is_enabled(load_policies(conn), connector_key): + raise ConnectorDisabled(f"{connector_key} is turned off by an admin") + + +def forced_credential_mode(conn, connector_key: Optional[str]) -> Optional[str]: + """``owner`` or ``member`` when an admin forces one for this connector.""" + if not connector_key: + return None + mode = (load_policies(conn).get(connector_key) or {}).get("credential_mode") + return mode if mode in ("owner", "member") else None diff --git a/docsgpt/core/settings/auth.py b/docsgpt/core/settings/auth.py index 706a3339..9c6deacb 100644 --- a/docsgpt/core/settings/auth.py +++ b/docsgpt/core/settings/auth.py @@ -29,7 +29,17 @@ class AuthSettings(SettingsGroup): ) ENCRYPTION_SECRET_KEY: str = Field( default="default-docsgpt-encryption-key", - description="Key used to encrypt stored credentials such as tool and connector secrets.", + description=( + "Key used to encrypt stored credentials such as tool and connector secrets. Set your own value before " + "connecting services on a multi-user install; the default is public." + ), + ) + ENCRYPTION_SECRET_KEY_PREVIOUS: Optional[str] = Field( + default=None, + description=( + "Previous ENCRYPTION_SECRET_KEY, tried when a stored credential was encrypted with it. Set it while " + "rotating the key, run `docsgpt connectors reencrypt`, then remove it." + ), ) INTERNAL_KEY: Optional[str] = Field( default=None, description="Internal API key for worker-to-backend authentication." diff --git a/docsgpt/core/settings/connectors.py b/docsgpt/core/settings/connectors.py index b4199300..917c605d 100644 --- a/docsgpt/core/settings/connectors.py +++ b/docsgpt/core/settings/connectors.py @@ -44,7 +44,23 @@ class ConnectorSettings(SettingsGroup): CONFLUENCE_CLIENT_SECRET: Optional[str] = Field(default=None, description="Confluence Cloud OAuth client secret.") # GitHub source. - GITHUB_ACCESS_TOKEN: Optional[str] = Field(default=None, description="GitHub PAT with read access to repositories.") + GITHUB_ACCESS_TOKEN: Optional[str] = Field( + default=None, + description=( + "Instance-wide GitHub token for the public-repository upload. It raises GitHub's rate limit and is never " + "used to read a private repository; users connect their own GitHub account for those." + ), + ) + # GitHub App behind "Sign in with GitHub" on the GitHub connector. + GITHUB_CLIENT_ID: Optional[str] = Field( + default=None, + description="GitHub App client id. With the secret and slug, offers Sign in with GitHub next to tokens.", + ) + GITHUB_CLIENT_SECRET: Optional[str] = Field(default=None, description="GitHub App client secret.") + GITHUB_APP_SLUG: Optional[str] = Field( + default=None, + description="GitHub App URL name (github.com/apps/), for the link where users choose repositories.", + ) MCP_OAUTH_REDIRECT_URI: Optional[str] = Field( default=None, description="Public callback URL for MCP OAuth; unset derives it from CONNECTOR_REDIRECT_BASE_URI." diff --git a/docsgpt/guardrails/config.py b/docsgpt/guardrails/config.py index e7ecf4c9..026049a7 100644 --- a/docsgpt/guardrails/config.py +++ b/docsgpt/guardrails/config.py @@ -212,6 +212,25 @@ class AgentConfig(BaseModel): model_config = ConfigDict(extra="forbid") guardrails: GuardrailsConfig = GuardrailsConfig() + # Write actions on credentials the owner holds (a connected account, a + # saved API tool key, a stored secret, an MCP sign-in) that someone who + # can't approve for the owner may run: an API-key or widget caller, a + # public-link user, and schedules either of them set. Any other such + # write is refused for them. ``tool_id:action``. + api_write_allowlist: List[str] = [] + + @field_validator("api_write_allowlist") + @classmethod + def _check_allowlist(cls, value: List[str]) -> List[str]: + if len(value) > 200: + raise ValueError("api_write_allowlist accepts at most 200 actions") + cleaned = [] + for entry in value: + tool_id, _, action = str(entry).partition(":") + if not tool_id.strip() or not action.strip(): + raise ValueError("api_write_allowlist entries must be 'tool_id:action'") + cleaned.append(f"{tool_id.strip()}:{action.strip()}") + return sorted(set(cleaned)) @classmethod def parse(cls, raw: Optional[dict]) -> "AgentConfig": @@ -221,4 +240,5 @@ class AgentConfig(BaseModel): try: return cls.model_validate(raw) except Exception: + # A bad allowlist falls back to none: the safe side. return cls(guardrails=GuardrailsConfig.parse(raw.get("guardrails"))) diff --git a/docsgpt/llm/handlers/base.py b/docsgpt/llm/handlers/base.py index 0954dd53..f495773c 100644 --- a/docsgpt/llm/handlers/base.py +++ b/docsgpt/llm/handlers/base.py @@ -1191,27 +1191,11 @@ class LLMHandler(ABC): ) if hasattr(agent.tool_executor, "headless_denials"): agent.tool_executor.headless_denials.append(pause_info) - from docsgpt.agents.tool_executor import ( - _mark_failed, - _record_proposed, - ) + from docsgpt.agents.tool_executor import journal_refused_call - if _record_proposed( - pause_info["call_id"], - pause_info["tool_name"], - pause_info["action_name"], - pause_info.get("arguments") or {}, - tool_id=pause_info.get("tool_id"), - message_id=agent.tool_executor.message_id, - user_id=agent.tool_executor.user, - agent_id=agent.tool_executor.agent_id, - ): - _mark_failed( - pause_info["call_id"], - f"headless: {deny_reason}", - message_id=agent.tool_executor.message_id, - user_id=agent.tool_executor.user, - ) + journal_refused_call( + agent.tool_executor, pause_info, f"headless: {deny_reason}" + ) denied_data = { "tool_name": pause_info["tool_name"], "call_id": pause_info["call_id"], @@ -1240,6 +1224,13 @@ class LLMHandler(ABC): # can wire the sticky "don't ask again" button. if pause_info.get("device_id"): pause_data["device_id"] = pause_info["device_id"] + # What will be sent once fixed values replace the model's. + if pause_info.get("sent_arguments") is not None: + pause_data["sent_arguments"] = pause_info["sent_arguments"] + # A connection-backed tool whose account needs signing in: the + # approval card becomes a Connect card. + if pause_info.get("connection_required"): + pause_data["connection_required"] = pause_info["connection_required"] trace_unexecuted_tool_call(call, pause_data) yield {"type": "tool_call", "data": pause_data} pending_actions.append(pause_info) diff --git a/docsgpt/parser/connectors/base.py b/docsgpt/parser/connectors/base.py index 5d2dd706..1b0f3c4d 100644 --- a/docsgpt/parser/connectors/base.py +++ b/docsgpt/parser/connectors/base.py @@ -6,7 +6,7 @@ interface for external knowledge base connectors. """ from abc import ABC, abstractmethod -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Tuple from docsgpt.parser.schema.base import Document @@ -88,18 +88,55 @@ class BaseConnectorLoader(ABC): Abstract base class for connector loaders. Defines the minimal interface that all connector loader - implementations must follow. + implementations must follow. A loader reads its OAuth tokens through + ``docsgpt.connectors.service`` from the connection it was built for, + either directly (``connection_id``, what background sync uses) or through + a legacy browser ``session_token`` that names the connection. """ - + + connection_id: Optional[str] = None + @abstractmethod - def __init__(self, session_token: str): + def __init__(self, session_token: Optional[str] = None, *, connection_id: Optional[str] = None): """ Initialize the connector loader. Args: - session_token: Authentication session token + session_token: Legacy browser session token naming the connection. + connection_id: The connection to read tokens from. """ pass + + @classmethod + def from_connection(cls, connection_id: str) -> "BaseConnectorLoader": + """Build a loader that reads its tokens from ``connection_id``.""" + return cls(connection_id=connection_id) + + def _load_token_info( + self, session_token: Optional[str], connection_id: Optional[str], + ) -> Tuple[str, Dict[str, Any]]: + """Resolve the connection and return ``(connection_id, token_info)``. + + Raises: + ValueError: The connection is missing or needs reconnecting. + """ + from docsgpt.connectors import service + + resolved = connection_id or service.connection_id_for_session_token(session_token) + self.connection_id = resolved + return resolved, service.get_valid_token_info(resolved) + + def _refresh_rejected_token(self, access_token: Optional[str]) -> Dict[str, Any]: + """Token info after the provider answered 401 to ``access_token``. + + Refreshes under the connection's row lock and persists the rotated + refresh token, or returns the token another worker already renewed. + """ + from docsgpt.connectors import service + + if not self.connection_id: + raise ValueError("Loader has no connection to refresh") + return service.get_valid_token_info(self.connection_id, rejected_access_token=access_token) @abstractmethod def load_data(self, inputs: Dict[str, Any]) -> List[Document]: diff --git a/docsgpt/parser/connectors/confluence/auth.py b/docsgpt/parser/connectors/confluence/auth.py index 431e99c7..6a5cf0fe 100644 --- a/docsgpt/parser/connectors/confluence/auth.py +++ b/docsgpt/parser/connectors/confluence/auth.py @@ -6,7 +6,6 @@ from urllib.parse import urlencode import requests from docsgpt.core.settings import settings -from docsgpt.parser.connectors._auth_utils import session_token_fingerprint from docsgpt.parser.connectors.base import BaseConnectorAuth logger = logging.getLogger(__name__) @@ -152,32 +151,6 @@ class ConfluenceAuth(BaseConnectorAuth): except Exception: return True - def get_token_info_from_session(self, session_token: str) -> Dict[str, Any]: - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - from docsgpt.storage.db.session import db_readonly - - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token( - session_token - ) - if not session: - raise ValueError( - f"Invalid session token ({session_token_fingerprint(session_token)})" - ) - - token_info = session.get("token_info") - if not token_info: - raise ValueError("Session missing token information") - - required = ["access_token", "refresh_token", "cloud_id"] - missing = [f for f in required if not token_info.get(f)] - if missing: - raise ValueError(f"Missing required token fields: {missing}") - - return token_info - def sanitize_token_info( self, token_info: Dict[str, Any], **extra_fields ) -> Dict[str, Any]: diff --git a/docsgpt/parser/connectors/confluence/loader.py b/docsgpt/parser/connectors/confluence/loader.py index 38c379dc..646913b0 100644 --- a/docsgpt/parser/connectors/confluence/loader.py +++ b/docsgpt/parser/connectors/confluence/loader.py @@ -44,12 +44,8 @@ def _retry_on_auth_failure(func): "Auth failure in %s, refreshing token and retrying", func.__name__ ) try: - new_token_info = self.auth.refresh_access_token(self.refresh_token) + new_token_info = self._refresh_rejected_token(self.access_token) self.access_token = new_token_info["access_token"] - self.refresh_token = new_token_info.get( - "refresh_token", self.refresh_token - ) - self._persist_refreshed_tokens(new_token_info) except Exception as refresh_err: raise ValueError( f"Authentication failed and could not be refreshed: {refresh_err}" @@ -62,13 +58,15 @@ def _retry_on_auth_failure(func): class ConfluenceLoader(BaseConnectorLoader): - def __init__(self, session_token: str): + def __init__(self, session_token: Optional[str] = None, *, connection_id: Optional[str] = None): self.auth = ConfluenceAuth() self.session_token = session_token - token_info = self.auth.get_token_info_from_session(session_token) + _, token_info = self._load_token_info(session_token, connection_id) + missing = [f for f in ("access_token", "cloud_id") if not token_info.get(f)] + if missing: + raise ValueError(f"Missing required token fields: {missing}") self.access_token = token_info["access_token"] - self.refresh_token = token_info["refresh_token"] self.cloud_id = token_info["cloud_id"] self.base_url = API_V2.format(cloud_id=self.cloud_id) @@ -81,22 +79,6 @@ class ConfluenceLoader(BaseConnectorLoader): "Accept": "application/json", } - def _persist_refreshed_tokens(self, token_info: Dict[str, Any]) -> None: - try: - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - from docsgpt.storage.db.session import db_session - - sanitized = self.auth.sanitize_token_info(token_info) - with db_session() as conn: - repo = ConnectorSessionsRepository(conn) - session = repo.get_by_session_token(self.session_token) - if session: - repo.update(str(session["id"]), {"token_info": sanitized}) - except Exception as e: - logger.warning("Failed to persist refreshed tokens: %s", e) - @_retry_on_auth_failure def load_data(self, inputs: Dict[str, Any]) -> List[Document]: folder_id = inputs.get("folder_id") diff --git a/docsgpt/parser/connectors/connector_creator.py b/docsgpt/parser/connectors/connector_creator.py index 2ac91b34..d688db99 100644 --- a/docsgpt/parser/connectors/connector_creator.py +++ b/docsgpt/parser/connectors/connector_creator.py @@ -1,5 +1,6 @@ from docsgpt.parser.connectors.confluence.auth import ConfluenceAuth from docsgpt.parser.connectors.confluence.loader import ConfluenceLoader +from docsgpt.parser.connectors.github.auth import GitHubAuth from docsgpt.parser.connectors.google_drive.auth import GoogleDriveAuth from docsgpt.parser.connectors.google_drive.loader import GoogleDriveLoader from docsgpt.parser.connectors.share_point.auth import SharePointAuth @@ -20,8 +21,11 @@ class ConnectorCreator: "share_point": SharePointLoader, } + # GitHub signs in here too, but its repositories are read by the remote + # GitHub loader, so it has an auth provider and no connector class. auth_providers = { "confluence": ConfluenceAuth, + "github": GitHubAuth, "google_drive": GoogleDriveAuth, "share_point": SharePointAuth, } @@ -75,6 +79,23 @@ class ConnectorCreator: """ return list(cls.connectors.keys()) + @classmethod + def has_auth(cls, connector_type: str) -> bool: + """Whether ``connector_type`` signs in through the OAuth callback. + + Args: + connector_type: Provider key, e.g. ``google_drive`` or ``github``. + + Returns: + True when an auth provider is registered for it. + """ + return (connector_type or "").lower() in cls.auth_providers + + @classmethod + def get_auth_providers(cls) -> list: + """Provider keys that sign in through the OAuth callback.""" + return list(cls.auth_providers.keys()) + @classmethod def is_supported(cls, connector_type): """ diff --git a/docsgpt/parser/connectors/github/__init__.py b/docsgpt/parser/connectors/github/__init__.py new file mode 100644 index 00000000..26850db0 --- /dev/null +++ b/docsgpt/parser/connectors/github/__init__.py @@ -0,0 +1,9 @@ +"""GitHub App user sign-in for the GitHub connector. + +Repositories are read by the remote GitHub loader +(``docsgpt.parser.remote.github_loader``); this package only signs users in. +""" + +from .auth import GitHubAuth + +__all__ = ["GitHubAuth"] diff --git a/docsgpt/parser/connectors/github/auth.py b/docsgpt/parser/connectors/github/auth.py new file mode 100644 index 00000000..b528c6ac --- /dev/null +++ b/docsgpt/parser/connectors/github/auth.py @@ -0,0 +1,148 @@ +"""Sign in with GitHub through a GitHub App (user access tokens). + +A GitHub App's user token can read what both the user and the app's +installations can see: the repositories are chosen when the user installs +the app, not with OAuth scopes. Tokens last eight hours and come with a +six-month refresh token, unless the app turned token expiry off, in which +case they carry no expiry and never need refreshing. +""" + +from __future__ import annotations + +import datetime +import logging +from typing import Any, Dict, Optional +from urllib.parse import urlencode + +import requests + +from docsgpt.core.settings import settings +from docsgpt.parser.connectors.base import BaseConnectorAuth + +logger = logging.getLogger(__name__) + +API_URL = "https://api.github.com" + + +class GitHubAuth(BaseConnectorAuth): + """OAuth web flow of the GitHub App set in ``GITHUB_CLIENT_ID`` and friends.""" + + AUTH_URL = "https://github.com/login/oauth/authorize" + TOKEN_URL = "https://github.com/login/oauth/access_token" + # Refresh this long before the stated expiry, so a token never lapses mid-request. + EXPIRY_MARGIN = datetime.timedelta(minutes=5) + + def __init__(self): + self.client_id = settings.GITHUB_CLIENT_ID + self.client_secret = settings.GITHUB_CLIENT_SECRET + self.app_slug = settings.GITHUB_APP_SLUG + self.redirect_uri = settings.CONNECTOR_REDIRECT_BASE_URI + if not self.client_id or not self.client_secret: + raise ValueError( + "GitHub App credentials not configured. " + "Please set GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET in settings." + ) + + def get_authorization_url(self, state: Optional[str] = None) -> str: + """The GitHub page that asks the user to authorize the app.""" + params = {"client_id": self.client_id, "redirect_uri": self.redirect_uri, "state": state} + return f"{self.AUTH_URL}?{urlencode({k: v for k, v in params.items() if v})}" + + def get_installation_url(self, state: Optional[str] = None) -> str: + """The GitHub page where the user installs the app and picks repositories. + + With "Request user authorization (OAuth) during installation" on, GitHub + sends the user back to the callback with a code and this ``state``. + """ + base = f"https://github.com/apps/{self.app_slug}/installations/new" + return f"{base}?{urlencode({'state': state})}" if state else base + + def _token_request(self, data: Dict[str, str]) -> Dict[str, Any]: + """POST to the token endpoint; GitHub reports failures as 200 with ``error``.""" + response = requests.post( + self.TOKEN_URL, + data={"client_id": self.client_id, "client_secret": self.client_secret, **data}, + headers={"Accept": "application/json"}, + timeout=30, + ) + response.raise_for_status() + payload = response.json() + if not isinstance(payload, dict) or payload.get("error") or not payload.get("access_token"): + error = payload.get("error") if isinstance(payload, dict) else None + raise ValueError(f"GitHub refused the sign-in: {error or 'no access token returned'}") + return payload + + @staticmethod + def _tokens(payload: Dict[str, Any], refresh_token: Optional[str] = None) -> Dict[str, Any]: + """Token info from a token response; ``expiry`` is None for non-expiring tokens.""" + expires_in = payload.get("expires_in") + expiry = None + if expires_in: + expiry = ( + datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(seconds=int(expires_in)) + ).isoformat() + return { + "access_token": payload["access_token"], + "refresh_token": payload.get("refresh_token") or refresh_token, + "token_uri": GitHubAuth.TOKEN_URL, + "expiry": expiry, + } + + def exchange_code_for_tokens(self, authorization_code: str) -> Dict[str, Any]: + """Trade the callback's code for tokens, plus the account's login. + + Raises: + ValueError: No code, or GitHub refused it. + """ + if not authorization_code: + raise ValueError("Authorization code is required") + payload = self._token_request({"code": authorization_code, "redirect_uri": self.redirect_uri}) + token_info = self._tokens(payload) + token_info["user_info"] = self._fetch_user(token_info["access_token"]) + return token_info + + def refresh_access_token(self, refresh_token: str) -> Dict[str, Any]: + """A new access token (and a new refresh token: GitHub rotates them). + + Raises: + ValueError: The refresh token was refused (expired or revoked). + """ + if not refresh_token: + raise ValueError("Refresh token is required") + payload = self._token_request({"grant_type": "refresh_token", "refresh_token": refresh_token}) + return self._tokens(payload, refresh_token) + + def is_token_expired(self, token_info: Dict[str, Any]) -> bool: + """Whether the access token is (about to be) expired. + + A token with no expiry never expires: the app has token expiry + turned off. + """ + if not token_info or not token_info.get("access_token"): + return True + expiry = token_info.get("expiry") + if not expiry: + return False + try: + expiry_dt = datetime.datetime.fromisoformat(expiry) + except (TypeError, ValueError): + return True + if expiry_dt.tzinfo is None: + expiry_dt = expiry_dt.replace(tzinfo=datetime.timezone.utc) + return datetime.datetime.now(datetime.timezone.utc) >= expiry_dt - self.EXPIRY_MARGIN + + @staticmethod + def _fetch_user(access_token: str) -> Dict[str, Any]: + """``{login, name}`` of the signed-in account; empty when GitHub does not say.""" + try: + response = requests.get( + f"{API_URL}/user", + headers={"Authorization": f"Bearer {access_token}", "Accept": "application/vnd.github+json"}, + timeout=30, + ) + response.raise_for_status() + user = response.json() + except Exception as exc: # the sign-in still works; only its label is missing + logger.warning("Could not read the GitHub account: %s", type(exc).__name__) + return {} + return {"login": user.get("login", ""), "name": user.get("name") or ""} diff --git a/docsgpt/parser/connectors/google_drive/auth.py b/docsgpt/parser/connectors/google_drive/auth.py index ddff3e4e..0a10d203 100644 --- a/docsgpt/parser/connectors/google_drive/auth.py +++ b/docsgpt/parser/connectors/google_drive/auth.py @@ -8,7 +8,6 @@ from googleapiclient.discovery import build from googleapiclient.errors import HttpError from docsgpt.core.settings import settings -from docsgpt.parser.connectors._auth_utils import session_token_fingerprint from docsgpt.parser.connectors.base import BaseConnectorAuth @@ -213,39 +212,6 @@ class GoogleDriveAuth(BaseConnectorAuth): return True - def get_token_info_from_session(self, session_token: str) -> Dict[str, Any]: - try: - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - from docsgpt.storage.db.session import db_readonly - - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token( - session_token - ) - if not session: - raise ValueError( - f"Invalid session token ({session_token_fingerprint(session_token)})" - ) - - token_info = session.get("token_info") - if not token_info: - raise ValueError("Session missing token information") - - required_fields = ["access_token", "refresh_token"] - missing_fields = [field for field in required_fields if field not in token_info or not token_info.get(field)] - if missing_fields: - raise ValueError(f"Missing required token fields: {missing_fields}") - - if 'token_uri' not in token_info: - token_info['token_uri'] = 'https://oauth2.googleapis.com/token' - - return token_info - - except Exception as e: - raise ValueError(f"Failed to retrieve Google Drive token information: {str(e)}") - def validate_credentials(self, credentials: Credentials) -> bool: """ Validate Google Drive credentials by making a test API call. diff --git a/docsgpt/parser/connectors/google_drive/loader.py b/docsgpt/parser/connectors/google_drive/loader.py index 9a8d7a97..f37a6560 100644 --- a/docsgpt/parser/connectors/google_drive/loader.py +++ b/docsgpt/parser/connectors/google_drive/loader.py @@ -48,11 +48,13 @@ class GoogleDriveLoader(BaseConnectorLoader): 'application/vnd.google-apps.spreadsheet': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' } - def __init__(self, session_token: str): + def __init__(self, session_token: Optional[str] = None, *, connection_id: Optional[str] = None): self.auth = GoogleDriveAuth() self.session_token = session_token - token_info = self.auth.get_token_info_from_session(session_token) + _, token_info = self._load_token_info(session_token, connection_id) + # Google refresh tokens do not rotate, so the credentials object may + # renew the access token in memory for the rest of this run. self.credentials = self.auth.create_credentials_from_token_info(token_info) try: diff --git a/docsgpt/parser/connectors/share_point/auth.py b/docsgpt/parser/connectors/share_point/auth.py index ec006740..4188f143 100644 --- a/docsgpt/parser/connectors/share_point/auth.py +++ b/docsgpt/parser/connectors/share_point/auth.py @@ -5,7 +5,6 @@ from typing import Optional, Dict, Any from msal import ConfidentialClientApplication from docsgpt.core.settings import settings -from docsgpt.parser.connectors._auth_utils import session_token_fingerprint from docsgpt.parser.connectors.base import BaseConnectorAuth logger = logging.getLogger(__name__) @@ -76,41 +75,6 @@ class SharePointAuth(BaseConnectorAuth): return self.map_token_response(result) - def get_token_info_from_session(self, session_token: str) -> Dict[str, Any]: - try: - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - from docsgpt.storage.db.session import db_readonly - - with db_readonly() as conn: - session = ConnectorSessionsRepository(conn).get_by_session_token( - session_token - ) - - if not session: - raise ValueError( - f"Invalid session token ({session_token_fingerprint(session_token)})" - ) - - token_info = session.get("token_info") - if not token_info: - raise ValueError("Session missing token information") - - required_fields = ["access_token", "refresh_token"] - missing_fields = [field for field in required_fields if field not in token_info or not token_info.get(field)] - if missing_fields: - raise ValueError(f"Missing required token fields: {missing_fields}") - - if 'token_uri' not in token_info: - token_info['token_uri'] = f"https://login.microsoftonline.com/{settings.MICROSOFT_TENANT_ID}/oauth2/v2.0/token" - - return token_info - - except Exception as e: - logger.error("Failed to retrieve token from session: %s", e) - raise ValueError(f"Failed to retrieve SharePoint token information: {str(e)}") - def is_token_expired(self, token_info: Dict[str, Any]) -> bool: if not token_info: return True diff --git a/docsgpt/parser/connectors/share_point/loader.py b/docsgpt/parser/connectors/share_point/loader.py index 6fa3e2c3..4690b303 100644 --- a/docsgpt/parser/connectors/share_point/loader.py +++ b/docsgpt/parser/connectors/share_point/loader.py @@ -26,8 +26,7 @@ def _retry_on_auth_failure(func): if e.response is not None and e.response.status_code in (401, 403): logging.info(f"Auth failure in {func.__name__}, refreshing token and retrying") try: - new_token_info = self.auth.refresh_access_token(self.refresh_token) - self.access_token = new_token_info.get('access_token') + self._apply_token_info(self._refresh_rejected_token(self.access_token)) except Exception as refresh_error: raise ValueError( f"Authentication failed and could not be refreshed: {refresh_error}" @@ -63,13 +62,12 @@ class SharePointLoader(BaseConnectorLoader): GRAPH_API_BASE = "https://graph.microsoft.com/v1.0" - def __init__(self, session_token: str): + def __init__(self, session_token: Optional[str] = None, *, connection_id: Optional[str] = None): self.auth = SharePointAuth() self.session_token = session_token - token_info = self.auth.get_token_info_from_session(session_token) - self.access_token = token_info.get('access_token') - self.refresh_token = token_info.get('refresh_token') + _, token_info = self._load_token_info(session_token, connection_id) + self._apply_token_info(token_info) self.allows_shared_content = token_info.get('allows_shared_content', False) if not self.access_token: @@ -77,6 +75,10 @@ class SharePointLoader(BaseConnectorLoader): self.next_page_token = None + def _apply_token_info(self, token_info: Dict[str, Any]) -> None: + self.access_token = token_info.get('access_token') + self.expiry = token_info.get('expiry') + def _get_headers(self) -> Dict[str, str]: return { 'Authorization': f'Bearer {self.access_token}', @@ -87,12 +89,14 @@ class SharePointLoader(BaseConnectorLoader): if not self.access_token: raise ValueError("No access token available") - token_info = {'access_token': self.access_token, 'expiry': None} - if self.auth.is_token_expired(token_info): + # The connection service refreshes under a row lock and stores the + # rotated refresh token; refreshing here would spend it and lose it. + if self.auth.is_token_expired({'access_token': self.access_token, 'expiry': self.expiry}): logging.info("Token expired, attempting refresh") try: - new_token_info = self.auth.refresh_access_token(self.refresh_token) - self.access_token = new_token_info.get('access_token') + from docsgpt.connectors import service + + self._apply_token_info(service.get_valid_token_info(self.connection_id)) except Exception: raise ValueError("Failed to refresh access token") diff --git a/docsgpt/parser/remote/github_loader.py b/docsgpt/parser/remote/github_loader.py index 3ba4a55c..9a997c5c 100644 --- a/docsgpt/parser/remote/github_loader.py +++ b/docsgpt/parser/remote/github_loader.py @@ -3,7 +3,7 @@ import logging import mimetypes import time from concurrent.futures import ThreadPoolExecutor -from typing import Dict, List, Optional, Tuple +from typing import Any, Dict, List, Optional, Tuple, Union import requests @@ -39,6 +39,14 @@ SKIP_SUFFIXES = ( ) +class GitHubTokenRejected(PermissionError): + """GitHub answered 401 to a user's own token (revoked or expired).""" + + +class PrivateRepositoryError(PermissionError): + """A repository the instance-wide token may not read for a user.""" + + class GitHubLoader(BaseRemote): """Load a GitHub repository's text files as ``Document`` objects. @@ -48,15 +56,74 @@ class GitHubLoader(BaseRemote): downloads the survivors in parallel. """ - def __init__(self): - self.access_token = settings.GITHUB_ACCESS_TOKEN - self.headers = { - "Authorization": f"token {self.access_token}", - "Accept": "application/vnd.github.v3+json" - } if self.access_token else { - "Accept": "application/vnd.github.v3+json" - } - return + def __init__(self, access_token: Optional[str] = None): + """Create a loader. + + Args: + access_token: A user's own GitHub token (from their GitHub + connection). Without one the instance-wide + ``GITHUB_ACCESS_TOKEN`` is used, and only for public + repositories. + """ + self._user_token = access_token + self._use_token(access_token or settings.GITHUB_ACCESS_TOKEN) + + def _use_token(self, token: Optional[str]) -> None: + """Send ``token`` (or nothing) on every following request.""" + self.access_token = token + self.headers = {"Accept": "application/vnd.github.v3+json"} + if token: + self.headers["Authorization"] = f"Bearer {token}" + + @staticmethod + def _parse_inputs(inputs: Union[str, Dict[str, Any]]) -> Tuple[str, Optional[str]]: + """``(repo_url, token)`` from a URL string or a connection's loader input. + + A source synced from a GitHub connection gets a dict: the repository + (``repo_url`` or ``url``) plus the connection's ``access_token``. + """ + if isinstance(inputs, dict): + repo = inputs.get("repo_url") or inputs.get("url") or "" + token = inputs.get("access_token") or inputs.get("token") + return str(repo), (str(token) if token else None) + return str(inputs or ""), None + + def ensure_instance_token_allowed(self, repo_name: str) -> None: + """Refuse a repository the instance-wide token must not read. + + ``GITHUB_ACCESS_TOKEN`` belongs to the server, not to the user asking, + and may see private repositories. Without a user's own token it is + used only for public ones (for their higher rate limit); anything + else needs the user's GitHub connection. + + Raises: + PrivateRepositoryError: The repository is private, internal, or + not visible to the token. + """ + if self._user_token or not self.access_token: + # A user's own token, or anonymous requests (which only ever see + # public repositories). + return + url = f"https://api.github.com/repos/{repo_name}" + response = requests.get(url, headers=self.headers, timeout=30) + if response.status_code == 401: + # A stale instance token: public repositories still read anonymously. + logger.warning("GitHub rejected GITHUB_ACCESS_TOKEN (401); reading %s anonymously", repo_name) + self._use_token(None) + return + if response.status_code == 200: + try: + data = response.json() + except ValueError: + data = {} + if isinstance(data, dict) and data.get("private") is False and ( + data.get("visibility") or "public" + ) == "public": + return + raise PrivateRepositoryError( + f"{repo_name} is private or could not be found. Connect your GitHub account " + "under Connectors to sync private repositories." + ) def is_text_file(self, file_path: str) -> bool: """Determine if a file is a text file based on extension.""" @@ -344,6 +411,10 @@ class GitHubLoader(BaseRemote): raise # If we can't parse the response, raise the original error response.raise_for_status() + elif response.status_code == 401 and self._user_token: + # The user's own token: a retry without it could only read + # public repositories, which is not what they asked for. + raise GitHubTokenRejected("GitHub rejected the connection's token. Reconnect GitHub to continue.") elif response.status_code == 401 and self.access_token: # An expired or revoked PAT makes even public repos 401, which # is strictly worse than not sending one. Retry unauthenticated @@ -404,14 +475,30 @@ class GitHubLoader(BaseRemote): paths = self.fetch_repo_files(repo_name) return self.select_files([(p, 0) for p in paths]) - def load_data(self, repo_url: str) -> List[Document]: - """Load every ingestable text file in ``repo_url`` as a Document.""" + def load_data(self, inputs: Union[str, Dict[str, Any]]) -> List[Document]: + """Load every ingestable text file of a repository as a Document. + + Args: + inputs: The repository URL, or ``{"repo_url", "access_token"}`` + for a source synced from a GitHub connection. + + Raises: + ValueError: ``inputs`` names no github.com repository. + PrivateRepositoryError: Only the instance-wide token is available + and the repository is not public. + GitHubTokenRejected: GitHub refused the connection's token. + """ + repo_url, token = self._parse_inputs(inputs) + if token: + self._user_token = token + self._use_token(token) repo_name = self.normalize_repo(repo_url) if not repo_name or "/" not in repo_name: raise ValueError( f"Not a valid GitHub repository: {repo_url!r}. " "Expected a github.com URL like https://github.com/owner/name." ) + self.ensure_instance_token_allowed(repo_name) branch = self.get_default_branch(repo_name) files = self._list_candidate_files(repo_name, branch) logger.info( diff --git a/docsgpt/parser/remote/linear_loader.py b/docsgpt/parser/remote/linear_loader.py new file mode 100644 index 00000000..3d40eb07 --- /dev/null +++ b/docsgpt/parser/remote/linear_loader.py @@ -0,0 +1,234 @@ +"""Load Linear issues and documents as Knowledge, through Linear's MCP server. + +A Linear source names teams and projects (see +``docsgpt.connectors.linear.normalize_selection``). Each issue becomes one +document: its identifier and title, state, assignee, priority, labels, +description and, optionally, its comments. With ``include_documents`` the +picked projects' Linear documents come too. Every document carries its +Linear URL as ``source``, so answers cite the issue. + +Each sync reads everything again (up to ``MAX_ISSUES`` issues and +``MAX_DOCUMENTS`` documents): the index of a synced source is rebuilt +whole, so there is nothing to merge an incremental read into. +""" + +from __future__ import annotations + +import logging +import re +from typing import Any, Optional + +from docsgpt.connectors import linear +from docsgpt.connectors.mcp import MCPToolError, run_connection_session +from docsgpt.parser.remote.base import BaseRemote +from docsgpt.parser.schema.base import Document + +logger = logging.getLogger(__name__) + +_UNSAFE_PATH = re.compile(r"[\\/\x00-\x1f]+") + + +def _connection(connection_id: str) -> Optional[dict]: + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + from docsgpt.storage.db.session import db_readonly + + with db_readonly() as conn: + return ConnectorSessionsRepository(conn).get(str(connection_id)) + + +def _segment(value: str, fallback: str) -> str: + """A file tree segment: no slashes or control characters, never empty.""" + cleaned = _UNSAFE_PATH.sub(" ", value or "").strip().strip(".") + return cleaned[:120] or fallback + + +def _day(value: Any) -> str: + return str(value)[:10] if value else "" + + +class LinearLoader(BaseRemote): + """Read a Linear source's issues and documents with its connection's sign-in.""" + + def load_data(self, inputs: dict) -> list[Document]: + """Read what ``inputs`` picks, signed in with its ``connection_id``. + + Args: + inputs: The source's selection plus ``connection_id``, merged in + by the worker from the source's connection. + + Raises: + ValueError: No connection, or nothing picked. + docsgpt.connectors.service.ConnectionUnavailable: The connection + needs reconnecting. + """ + inputs = inputs if isinstance(inputs, dict) else {} + connection_id = inputs.get("connection_id") + connection = _connection(connection_id) if connection_id else None + if connection is None: + raise ValueError("A Linear source needs its Linear connection") + selection = linear.normalize_selection(inputs) + return run_connection_session( + connection, linear.mcp_url(), lambda session: self.collect(session, selection), + ) + + async def collect(self, session: Any, selection: dict) -> list[Document]: + """The documents for ``selection``, read over an open MCP session.""" + issues: dict[str, dict] = {} + filters = [(("team", "teamId"), team["id"]) for team in selection["teams"]] + filters += [(("project", "projectId"), project["id"]) for project in selection["projects"]] + for names, value in filters: + remaining = linear.MAX_ISSUES - len(issues) + if remaining <= 0: + logger.info("Linear sync stopped at %d issues", linear.MAX_ISSUES) + break + async for issue in linear.paged( + session, "list_issues", "issues", {names: value}, limit=remaining, + extra={"orderBy": "updatedAt", "includeArchived": False}, + ): + key = linear.issue_identifier(issue) or str(issue.get("id") or "") + if key: + issues.setdefault(key, issue) + documents = [] + for key, issue in issues.items(): + # One issue Linear will not read (deleted since it was listed, or + # out of the token's reach) loses that detail, not the whole sync. + if linear.is_truncated(issue.get("description")) or issue.get("descriptionTruncated"): + try: + issue = await self._full_issue(session, key, issue) + except MCPToolError as exc: + logger.warning("Linear sync keeps %s as listed: %s", key, exc) + comments = [] + if selection["include_comments"]: + try: + comments = await self._comments(session, key) + except MCPToolError as exc: + logger.warning("Linear sync leaves out the comments of %s: %s", key, exc) + documents.append(self._issue_document(issue, comments, selection)) + if selection["include_documents"]: + documents.extend(await self._project_documents(session, selection["projects"])) + return documents + + async def _full_issue(self, session: Any, key: str, issue: dict) -> dict: + schema = await session.input_schema("get_issue") + if schema is None: + return issue + full = linear.unwrap(await session.call("get_issue", {linear.argument(schema, "id", "issueId"): key}), "issue") + return {**issue, **{k: v for k, v in full.items() if v not in (None, "")}} + + async def _comments(self, session: Any, key: str) -> list[dict]: + """An issue's comments, oldest first as Linear lists them; none when Linear cannot list them.""" + schema = await session.input_schema("list_comments") + names = ("issueId", "id", "issue") + if schema is None or linear.argument(schema, *names) is None: + return [] + return [ + comment async for comment in linear.paged( + session, "list_comments", "comments", {names: key}, limit=linear.MAX_COMMENTS, + ) + ] + + def _issue_document(self, issue: dict, comments: list[dict], selection: dict) -> Document: + identifier = linear.issue_identifier(issue) or str(issue.get("id") or "") + title = str(issue.get("title") or "").strip() or identifier + heading = f"{identifier}: {title}" if identifier else title + facts = [ + ("State", linear.name_of(issue.get("status") or issue.get("state"))), + ("Assignee", linear.name_of(issue.get("assignee"))), + ("Priority", linear.priority_of(issue.get("priority"))), + ("Labels", ", ".join(linear.names_of(issue.get("labels")))), + ("Project", linear.name_of(issue.get("project"))), + ("Team", linear.name_of(issue.get("team"))), + ("Due", _day(issue.get("dueDate"))), + ("Created", _day(issue.get("createdAt"))), + ("Updated", _day(issue.get("updatedAt"))), + ("Link", str(issue.get("url") or "")), + ] + lines = [f"# {heading}", ""] + lines += [f"- {label}: {value}" for label, value in facts if value] + description = str(issue.get("description") or "").strip() + if description: + lines += ["", "## Description", "", description] + written = [c for c in comments if str(c.get("body") or "").strip()] + if written: + lines += ["", "## Comments"] + for comment in written: + author = linear.name_of(comment.get("user") or comment.get("author")) or "Someone" + when = _day(comment.get("createdAt")) + lines += ["", f"**{author}**" + (f" · {when}" if when else ""), str(comment["body"]).strip()] + return Document( + text="\n".join(lines).strip() + "\n", + extra_info={ + "title": heading, + "source": str(issue.get("url") or ""), + "file_path": f"{self._team_folder(issue, identifier, selection)}/{_segment(identifier, 'issue')}.md", + "linear_id": str(issue.get("id") or identifier), + "updated_at": str(issue.get("updatedAt") or ""), + }, + ) + + @staticmethod + def _team_folder(issue: dict, identifier: str, selection: dict) -> str: + """The folder an issue is filed under: its team's key (``ENG``), else its team's name.""" + team = issue.get("team") + key = team.get("key") if isinstance(team, dict) else None + if not key and "-" in identifier: + key = identifier.rsplit("-", 1)[0] + return _segment(str(key or linear.name_of(team) or ""), "Issues") + + async def _project_documents(self, session: Any, projects: list[dict]) -> list[Document]: + """The Linear documents of ``projects``, each read in full, up to ``MAX_DOCUMENTS``.""" + documents: list[Document] = [] + paths: set[str] = set() + get_schema = await session.input_schema("get_document") + for project in projects: + remaining = linear.MAX_DOCUMENTS - len(documents) + if remaining <= 0: + break + async for record in linear.paged( + session, "list_documents", "documents", {("projectId", "project"): project["id"]}, limit=remaining, + ): + content = record.get("content") + if (not content or linear.is_truncated(content)) and get_schema is not None and record.get("id"): + try: + full = linear.unwrap( + await session.call("get_document", {linear.argument(get_schema, "id", "documentId"): + str(record["id"])}), + "document", + ) + except MCPToolError as exc: + # A document that cannot be read is skipped, not the sync. + logger.warning("Linear sync skips document %s: %s", record.get("id"), exc) + continue + record = {**record, **{k: v for k, v in full.items() if v not in (None, "")}} + document = self._linear_document(record, project) + path = document.extra_info["file_path"] + if path in paths: + # Two documents with one title stay two files. + document.extra_info["file_path"] = f"{path[:-3]} ({str(record.get('id') or len(paths))[:8]}).md" + paths.add(document.extra_info["file_path"]) + documents.append(document) + return documents + + @staticmethod + def _linear_document(record: dict, project: dict) -> Document: + title = str(record.get("title") or "").strip() or "Untitled document" + content = str(record.get("content") or "").strip() + project_name = project.get("name") or linear.name_of(record.get("project")) + lines = [f"# {title}", ""] + if project_name: + lines.append(f"- Project: {project_name}") + if record.get("url"): + lines.append(f"- Link: {record['url']}") + if content: + lines += ["", content] + folder = _segment(project_name or "", "Documents") + return Document( + text="\n".join(lines).strip() + "\n", + extra_info={ + "title": title, + "source": str(record.get("url") or ""), + "file_path": f"{folder}/Documents/{_segment(title, 'Document')}.md", + "linear_id": str(record.get("id") or ""), + "updated_at": str(record.get("updatedAt") or ""), + }, + ) diff --git a/docsgpt/parser/remote/remote_creator.py b/docsgpt/parser/remote/remote_creator.py index b6f396c5..585ea9b0 100644 --- a/docsgpt/parser/remote/remote_creator.py +++ b/docsgpt/parser/remote/remote_creator.py @@ -5,6 +5,7 @@ from docsgpt.parser.remote.crawler_loader import CrawlerLoader from docsgpt.parser.remote.web_loader import WebLoader from docsgpt.parser.remote.reddit_loader import RedditPostsLoaderRemote from docsgpt.parser.remote.github_loader import GitHubLoader +from docsgpt.parser.remote.linear_loader import LinearLoader from docsgpt.parser.remote.s3_loader import S3Loader @@ -26,6 +27,7 @@ class RemoteCreator: "reddit": RedditPostsLoaderRemote, "github": GitHubLoader, "s3": S3Loader, + "linear": LinearLoader, } @classmethod @@ -88,5 +90,5 @@ def normalize_remote_data(source_type, remote_data): return json.dumps(remote_data) return remote_data - # s3's loader accepts a dict or JSON string; pass it through unchanged. + # s3's and linear's loaders accept a dict or JSON string; pass it through unchanged. return remote_data diff --git a/docsgpt/retriever/base.py b/docsgpt/retriever/base.py index a1b49d9c..897ab622 100644 --- a/docsgpt/retriever/base.py +++ b/docsgpt/retriever/base.py @@ -2,6 +2,17 @@ from abc import ABC, abstractmethod class BaseRetriever(ABC): + @property + def _connector_labels(self): + """Per-retriever memo of each source's connector ("From Google Drive").""" + cache = self.__dict__.get("_connector_label_cache") + if cache is None: + from docsgpt.connectors.attribution import ConnectorLabelCache + + cache = ConnectorLabelCache() + self.__dict__["_connector_label_cache"] = cache + return cache + def __init__(self): pass diff --git a/docsgpt/retriever/classic_rag.py b/docsgpt/retriever/classic_rag.py index 76f343c3..3e8779be 100644 --- a/docsgpt/retriever/classic_rag.py +++ b/docsgpt/retriever/classic_rag.py @@ -385,7 +385,11 @@ class ClassicRAG(BaseRetriever): doc_tokens = num_tokens_from_string(doc_text_with_header) if cumulative_tokens + doc_tokens < token_budget: - entry = {"text": page_content, **labels} + entry = { + "text": page_content, + **labels, + **self._connector_labels.for_source(vectorstore_id), + } if self.include_scores: entry["score"] = score entry["score_kind"] = score_kind diff --git a/docsgpt/retriever/graph_rag.py b/docsgpt/retriever/graph_rag.py index 4b12e1b3..48fa0fd5 100644 --- a/docsgpt/retriever/graph_rag.py +++ b/docsgpt/retriever/graph_rag.py @@ -584,7 +584,7 @@ class GraphRAGRetriever(BaseRetriever): doc_tokens = num_tokens_from_string(f"{labels['filename']}\n{text}") if cumulative_tokens + doc_tokens >= token_budget: break - docs.append({"text": text, **labels}) + docs.append({"text": text, **labels, **self._connector_labels.for_source(source_id)}) cumulative_tokens += doc_tokens return docs diff --git a/docsgpt/security/encryption.py b/docsgpt/security/encryption.py index aa77069d..5793fc7c 100644 --- a/docsgpt/security/encryption.py +++ b/docsgpt/security/encryption.py @@ -1,11 +1,17 @@ import base64 +import functools +import hashlib +import hmac import json import logging import os +from typing import Optional from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives import hashes from cryptography.hazmat.primitives.ciphers import algorithms, Cipher, modes +from cryptography.hazmat.primitives.ciphers.aead import AESGCM +from cryptography.hazmat.primitives.kdf.hkdf import HKDF from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC from docsgpt.core.settings import settings @@ -86,3 +92,134 @@ def _pad_data(data: bytes) -> bytes: def _unpad_data(data: bytes) -> bytes: padding_len = data[-1] return data[:-padding_len] + + +# --------------------------------------------------------------------------- +# Envelope v2: connection credentials +# --------------------------------------------------------------------------- +# +# ``v2::`` +# +# AES-256-GCM, so a tampered blob fails to decrypt instead of returning +# garbage. The master key is derived once per process from +# ENCRYPTION_SECRET_KEY (PBKDF2, cached); each record gets its own key from +# HKDF(master, salt, owner id), which keeps the v1 owner binding without +# paying 100k PBKDF2 iterations on every token read in the worker. The owner +# id is also the GCM associated data, so a blob copied onto another user's +# row does not decrypt. ``key_id`` names the master key, so a blob written +# under ENCRYPTION_SECRET_KEY_PREVIOUS is still readable during a rotation. + +_V2_PREFIX = "v2" +_V2_MASTER_SALT = b"docsgpt-credentials-v2" +_V2_ITERATIONS = 200_000 +_V2_SALT_BYTES = 16 +_V2_NONCE_BYTES = 12 +DEFAULT_ENCRYPTION_KEY = "default-docsgpt-encryption-key" + + +class CredentialDecryptionError(Exception): + """A stored credential could not be decrypted (wrong key, tampering, bad format).""" + + +@functools.lru_cache(maxsize=8) +def _master_key(secret: str) -> bytes: + kdf = PBKDF2HMAC( + algorithm=hashes.SHA256(), + length=32, + salt=_V2_MASTER_SALT, + iterations=_V2_ITERATIONS, + backend=default_backend(), + ) + return kdf.derive(secret.encode()) + + +def _key_id(master: bytes) -> str: + return hmac.new(master, b"docsgpt-key-id", hashlib.sha256).hexdigest()[:8] + + +def _record_key(master: bytes, owner_id: str, salt: bytes) -> bytes: + return HKDF( + algorithm=hashes.SHA256(), + length=32, + salt=salt, + info=b"docsgpt-v2|" + owner_id.encode(), + backend=default_backend(), + ).derive(master) + + +def _candidate_keys() -> dict[str, bytes]: + """Master keys this process can decrypt with, by key id (current first).""" + keys: dict[str, bytes] = {} + for secret in (settings.ENCRYPTION_SECRET_KEY, settings.ENCRYPTION_SECRET_KEY_PREVIOUS): + if secret: + master = _master_key(secret) + keys.setdefault(_key_id(master), master) + return keys + + +def current_key_id() -> str: + """Key id of ENCRYPTION_SECRET_KEY, as written into new v2 blobs.""" + return _key_id(_master_key(settings.ENCRYPTION_SECRET_KEY)) + + +def is_default_encryption_key() -> bool: + """Whether ENCRYPTION_SECRET_KEY is still the public default.""" + return settings.ENCRYPTION_SECRET_KEY == DEFAULT_ENCRYPTION_KEY + + +def encrypt_json(data: dict, owner_id: str) -> str: + """Encrypt ``data`` for ``owner_id`` into a v2 envelope. + + Args: + data: JSON-serialisable credentials. + owner_id: The user the credentials belong to; decryption needs it. + + Returns: + The ``v2::`` string. + """ + master = _master_key(settings.ENCRYPTION_SECRET_KEY) + key_id = _key_id(master) + salt = os.urandom(_V2_SALT_BYTES) + nonce = os.urandom(_V2_NONCE_BYTES) + key = _record_key(master, owner_id, salt) + plaintext = json.dumps(data, separators=(",", ":")).encode() + ciphertext = AESGCM(key).encrypt(nonce, plaintext, owner_id.encode()) + payload = base64.b64encode(salt + nonce + ciphertext).decode() + return f"{_V2_PREFIX}:{key_id}:{payload}" + + +def envelope_key_id(blob: str) -> Optional[str]: + """The key id a v2 blob was written with, or None for anything else.""" + parts = (blob or "").split(":", 2) + if len(parts) != 3 or parts[0] != _V2_PREFIX: + return None + return parts[1] + + +def decrypt_json(blob: str, owner_id: str) -> dict: + """Decrypt a v2 envelope written for ``owner_id``. + + Raises: + CredentialDecryptionError: The blob is malformed, was written with a + key this process does not have, belongs to another owner, or was + tampered with. + """ + key_id = envelope_key_id(blob) + if key_id is None: + raise CredentialDecryptionError("Not a v2 credential envelope") + master = _candidate_keys().get(key_id) + if master is None: + raise CredentialDecryptionError("Credential was encrypted with an unknown key") + try: + raw = base64.b64decode(blob.split(":", 2)[2].encode(), validate=True) + salt = raw[:_V2_SALT_BYTES] + nonce = raw[_V2_SALT_BYTES:_V2_SALT_BYTES + _V2_NONCE_BYTES] + ciphertext = raw[_V2_SALT_BYTES + _V2_NONCE_BYTES:] + key = _record_key(master, owner_id, salt) + plaintext = AESGCM(key).decrypt(nonce, ciphertext, owner_id.encode()) + data = json.loads(plaintext.decode()) + except Exception as exc: + raise CredentialDecryptionError("Credential could not be decrypted") from exc + if not isinstance(data, dict): + raise CredentialDecryptionError("Credential payload is not an object") + return data diff --git a/docsgpt/services/search_service.py b/docsgpt/services/search_service.py index 3fd55e4e..7d6b0fb9 100644 --- a/docsgpt/services/search_service.py +++ b/docsgpt/services/search_service.py @@ -65,7 +65,8 @@ def _authorized_source_ids(conn, agent: Dict[str, Any], source_ids: List[str]) - source_ids: Ids extracted from that row. A source the owner can't read still searches while the editor who - attached it (its sponsor) can edit the agent and read the source. + attached it (its sponsor) can edit the agent and still owns or edits + the source. Returns: list: The subset the agent's owner (or a live sponsor) may read. diff --git a/docsgpt/storage/db/models.py b/docsgpt/storage/db/models.py index 34841da6..4d5d750f 100644 --- a/docsgpt/storage/db/models.py +++ b/docsgpt/storage/db/models.py @@ -233,6 +233,11 @@ user_tools_table = Table( Column("created_at", DateTime(timezone=True), nullable=False, server_default=func.now()), Column("updated_at", DateTime(timezone=True), nullable=False, server_default=func.now()), Column("legacy_mongo_id", Text), + Column( + "connection_id", UUID(as_uuid=True), ForeignKey("connector_sessions.id", ondelete="SET NULL"), + ), + # Whose account a shared resource runs with: the owner's, or each member's. + Column("credential_mode", Text, nullable=False, server_default="owner"), ) # A grantee's personal "In my chats" switch for a tool shared with them @@ -371,6 +376,13 @@ sources_table = Table( Column("created_at", DateTime(timezone=True), nullable=False, server_default=func.now()), Column("updated_at", DateTime(timezone=True), nullable=False, server_default=func.now()), Column("legacy_mongo_id", Text), + Column( + "connection_id", UUID(as_uuid=True), ForeignKey("connector_sessions.id", ondelete="SET NULL"), + ), + # Whose account a shared resource runs with: the owner's, or each member's. + Column("credential_mode", Text, nullable=False, server_default="owner"), + # A wiki's owner lets API-key and widget runs edit it (off: read only). + Column("wiki_outside_edits", Boolean, nullable=False, server_default="false"), ) agents_table = Table( @@ -621,6 +633,34 @@ connector_sessions_table = Table( Column("expires_at", DateTime(timezone=True)), Column("created_at", DateTime(timezone=True), nullable=False, server_default=func.now()), Column("legacy_mongo_id", Text), + # Added in ``0040_connections``: each row is a connection (one signed-in + # account, one MCP server or one set of API credentials). + Column("connector_key", Text), + Column("display_name", Text), + Column("account_label", Text), + Column("auth_kind", Text), + Column("updated_at", DateTime(timezone=True), nullable=False, server_default=func.now()), + # Every secret of the connection, as one owner-bound v2 envelope. + Column("encrypted_credentials", Text), + Column("has_refresh_token", Boolean, nullable=False, server_default="false"), + Column("scopes", JSONB, nullable=False, server_default="[]"), + Column("last_error", Text), + Column("last_used_at", DateTime(timezone=True)), + # Added in ``0041_connection_account_name``: what the user calls the + # account; ``account_label`` stays its identity. + Column("account_name", Text), +) + + +connector_policies_table = Table( + "connector_policies", + metadata, + Column("connector_key", Text, primary_key=True), + # NULL: on when the connector has its server settings (see connectors.service). + Column("enabled", Boolean), + Column("credential_mode", Text, nullable=False, server_default="choose"), + Column("updated_by", Text), + Column("updated_at", DateTime(timezone=True), nullable=False, server_default=func.now()), ) diff --git a/docsgpt/storage/db/redaction.py b/docsgpt/storage/db/redaction.py index cb1e4754..483c7564 100644 --- a/docsgpt/storage/db/redaction.py +++ b/docsgpt/storage/db/redaction.py @@ -33,13 +33,17 @@ _SECRET_SUBSTRINGS = ( "credential", "authorization", "bearer", + # Connection secrets: an OAuth token_info blob, MCP token and client + # registration dicts (``client_secret`` is covered by ``secret``). + "token_info", + "client_info", ) def is_secret_key(key: str) -> bool: """True when ``key`` names a credential that must not be persisted/returned.""" k = key.lower() - if k == "token": + if k in ("token", "tokens"): return True return any(s in k for s in _SECRET_SUBSTRINGS) diff --git a/docsgpt/storage/db/repositories/connector_policies.py b/docsgpt/storage/db/repositories/connector_policies.py new file mode 100644 index 00000000..3ac628fc --- /dev/null +++ b/docsgpt/storage/db/repositories/connector_policies.py @@ -0,0 +1,72 @@ +"""Repository for ``connector_policies``: the admin's per-connector switches.""" + +from __future__ import annotations + +from typing import Optional + +from sqlalchemy import Connection, text + +from docsgpt.storage.db.base_repository import row_to_dict + +CREDENTIAL_POLICIES = ("choose", "owner", "member") +ALLOW_CUSTOM_MCP_KEY = "connectors.allow_custom_mcp" + + +def allow_writes_key(connector_key: str) -> str: + """``app_metadata`` key of the admin's "let agents make changes" switch for a connector. + + Only connectors whose tools opt into writes (GitHub) have one. It lives in + ``app_metadata`` like the custom MCP switch: ``"false"`` forbids writes, + anything else (or nothing) allows them. + """ + return f"connectors.{connector_key}.allow_writes" + + +class ConnectorPoliciesRepository: + """Whether a connector is enabled and which credential mode it forces.""" + + def __init__(self, conn: Connection) -> None: + self._conn = conn + + def all(self) -> dict[str, dict]: + """Every stored policy, by connector key. Missing keys use the defaults.""" + result = self._conn.execute(text("SELECT * FROM connector_policies")) + return {row["connector_key"]: row for row in (row_to_dict(r) for r in result.fetchall())} + + def get(self, connector_key: str) -> Optional[dict]: + row = self._conn.execute( + text("SELECT * FROM connector_policies WHERE connector_key = :key"), {"key": connector_key}, + ).fetchone() + return row_to_dict(row) if row is not None else None + + def upsert( + self, + connector_key: str, + *, + enabled: Optional[bool] = None, + credential_mode: Optional[str] = None, + updated_by: Optional[str] = None, + ) -> dict: + """Set one connector's policy; fields left None keep their value. + + A new row leaves ``enabled`` NULL unless it is given, so changing only + the credential mode never switches a connector on. + """ + if credential_mode is not None and credential_mode not in CREDENTIAL_POLICIES: + raise ValueError(f"unknown credential mode: {credential_mode!r}") + row = self._conn.execute( + text( + """ + INSERT INTO connector_policies (connector_key, enabled, credential_mode, updated_by) + VALUES (:key, :enabled, COALESCE(:mode, 'choose'), :by) + ON CONFLICT (connector_key) DO UPDATE SET + enabled = COALESCE(:enabled, connector_policies.enabled), + credential_mode = COALESCE(:mode, connector_policies.credential_mode), + updated_by = :by, + updated_at = now() + RETURNING * + """ + ), + {"key": connector_key, "enabled": enabled, "mode": credential_mode, "by": updated_by}, + ).fetchone() + return row_to_dict(row) diff --git a/docsgpt/storage/db/repositories/connector_sessions.py b/docsgpt/storage/db/repositories/connector_sessions.py index 7a95b9d1..1bfd25ac 100644 --- a/docsgpt/storage/db/repositories/connector_sessions.py +++ b/docsgpt/storage/db/repositories/connector_sessions.py @@ -9,8 +9,13 @@ Shape notes: unique constraint on ``session_token``. * MCP sessions key off ``server_url`` instead — a single user may have multiple MCP servers, one row each. The composite unique index - ``(user_id, COALESCE(server_url, ''), provider)`` makes both patterns - coexist without collision. + ``(user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, ''))`` + makes both patterns coexist, and lets one user connect several accounts + of the same provider (each row carries its own ``account_label``). +* Every secret lives in ``encrypted_credentials``, written and read only by + ``docsgpt.connectors.service``; ``token_info`` and the ``tokens`` / + ``client_info`` keys of ``session_data`` are legacy plaintext that + migration 0040 moved into it. * ``session_data`` remains a catch-all JSONB for driver-specific state (tokens that don't fit anywhere else, per-provider scratch data). Promoted columns (``session_token``, ``user_email``, ``status``, @@ -24,14 +29,16 @@ from typing import Any, Optional from sqlalchemy import Connection, text -from docsgpt.storage.db.base_repository import row_to_dict +from docsgpt.storage.db.base_repository import looks_like_uuid, row_to_dict from docsgpt.storage.db.serialization import PGNativeJSONEncoder _UPDATABLE_SCALARS = { "server_url", "session_token", "user_email", "status", "expires_at", + "connector_key", "display_name", "account_label", "auth_kind", + "encrypted_credentials", "has_refresh_token", "last_error", "last_used_at", "account_name", } -_UPDATABLE_JSONB = {"session_data", "token_info"} +_UPDATABLE_JSONB = {"session_data", "token_info", "scopes"} def _jsonb(value: Any) -> Any: @@ -70,7 +77,8 @@ class ConnectorSessionsRepository: ) -> dict: """Insert or update a connector session row. - Conflict key is ``(user_id, COALESCE(server_url, ''), provider)`` + Conflict key is the account index + ``(user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, ''))`` so MCP rows (per-server) and OAuth rows (per-provider) both get idempotent upsert semantics. """ @@ -86,7 +94,7 @@ class ConnectorSessionsRepository: :status, CAST(:token_info AS jsonb), CAST(:session_data AS jsonb), :expires_at, :legacy_mongo_id ) - ON CONFLICT (user_id, COALESCE(server_url, ''), provider) + ON CONFLICT (user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, '')) DO UPDATE SET session_token = COALESCE(EXCLUDED.session_token, connector_sessions.session_token), user_email = COALESCE(EXCLUDED.user_email, connector_sessions.user_email), @@ -176,11 +184,154 @@ class ConnectorSessionsRepository: def list_for_user(self, user_id: str) -> list[dict]: result = self._conn.execute( - text("SELECT * FROM connector_sessions WHERE user_id = :user_id"), + text("SELECT * FROM connector_sessions WHERE user_id = :user_id ORDER BY created_at"), {"user_id": user_id}, ) return [row_to_dict(r) for r in result.fetchall()] + def create( + self, + user_id: str, + provider: str, + *, + connector_key: str, + auth_kind: str, + display_name: Optional[str] = None, + account_label: Optional[str] = None, + server_url: Optional[str] = None, + status: str = "connected", + encrypted_credentials: Optional[str] = None, + has_refresh_token: bool = False, + ) -> Optional[dict]: + """Insert a connection; return None when that account already exists.""" + result = self._conn.execute( + text( + """ + INSERT INTO connector_sessions ( + user_id, provider, server_url, connector_key, auth_kind, display_name, + account_label, status, encrypted_credentials, has_refresh_token, session_data + ) + VALUES ( + :user_id, :provider, :server_url, :connector_key, :auth_kind, :display_name, + :account_label, :status, :encrypted_credentials, :has_refresh_token, '{}'::jsonb + ) + ON CONFLICT (user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, '')) + DO NOTHING + RETURNING * + """ + ), + { + "user_id": user_id, + "provider": provider, + "server_url": server_url, + "connector_key": connector_key, + "auth_kind": auth_kind, + "display_name": display_name, + "account_label": account_label, + "status": status, + "encrypted_credentials": encrypted_credentials, + "has_refresh_token": has_refresh_token, + }, + ) + row = result.fetchone() + return row_to_dict(row) if row is not None else None + + def find_account( + self, user_id: str, provider: str, *, server_url: Optional[str], account_label: Optional[str], + ) -> Optional[dict]: + """The connection for one account, matching the account unique index.""" + result = self._conn.execute( + text( + "SELECT * FROM connector_sessions WHERE user_id = :user_id AND provider = :provider " + "AND COALESCE(server_url, '') = COALESCE(:server_url, '') " + "AND COALESCE(account_label, '') = COALESCE(:account_label, '')" + ), + {"user_id": user_id, "provider": provider, "server_url": server_url, "account_label": account_label}, + ) + row = result.fetchone() + return row_to_dict(row) if row is not None else None + + def delete_by_id(self, connection_id: str) -> bool: + """Delete a connection row. Linked sources and tools keep existing (SET NULL).""" + if not looks_like_uuid(connection_id): + return False + result = self._conn.execute( + text("DELETE FROM connector_sessions WHERE id = CAST(:id AS uuid)"), {"id": str(connection_id)}, + ) + return result.rowcount > 0 + + def get(self, connection_id: str) -> Optional[dict]: + """Fetch a connection by id, whoever owns it. Callers authorise.""" + if not looks_like_uuid(connection_id): + return None + result = self._conn.execute( + text("SELECT * FROM connector_sessions WHERE id = CAST(:id AS uuid)"), + {"id": str(connection_id)}, + ) + row = result.fetchone() + return row_to_dict(row) if row is not None else None + + def get_for_user(self, connection_id: str, user_id: str) -> Optional[dict]: + """Fetch a connection only when ``user_id`` owns it.""" + row = self.get(connection_id) + if row is None or row.get("user_id") != user_id: + return None + return row + + def get_for_update(self, connection_id: str) -> Optional[dict]: + """Fetch and row-lock a connection until the transaction ends. + + Token refresh holds this lock so two workers refreshing a rotating + refresh token (Microsoft, Atlassian) cannot both spend it. + """ + if not looks_like_uuid(connection_id): + return None + result = self._conn.execute( + text("SELECT * FROM connector_sessions WHERE id = CAST(:id AS uuid) FOR UPDATE"), + {"id": str(connection_id)}, + ) + row = result.fetchone() + return row_to_dict(row) if row is not None else None + + def resource_counts(self, connection_ids: list[str]) -> dict[str, dict[str, int]]: + """Number of sources and tools linked to each connection id.""" + ids = [str(i) for i in connection_ids if looks_like_uuid(str(i))] + counts: dict[str, dict[str, int]] = {i: {"sources": 0, "tools": 0} for i in ids} + if not ids: + return counts + for table, key in (("sources", "sources"), ("user_tools", "tools")): + result = self._conn.execute( + text( + f"SELECT connection_id, count(*) FROM {table} " + "WHERE connection_id = ANY(CAST(:ids AS uuid[])) GROUP BY connection_id" + ), + {"ids": ids}, + ) + for connection_id, count in result.fetchall(): + counts[str(connection_id)][key] = int(count) + return counts + + def list_sources(self, connection_id: str) -> list[dict]: + """Sources synced from a connection, newest first.""" + result = self._conn.execute( + text( + "SELECT id, name, type, date, sync_frequency, metadata, remote_data, user_id, file_path " + "FROM sources WHERE connection_id = CAST(:id AS uuid) ORDER BY date DESC" + ), + {"id": str(connection_id)}, + ) + return [row_to_dict(r) for r in result.fetchall()] + + def list_tools(self, connection_id: str) -> list[dict]: + """Tools a connection provides, oldest first.""" + result = self._conn.execute( + text( + "SELECT * FROM user_tools WHERE connection_id = CAST(:id AS uuid) ORDER BY created_at" + ), + {"id": str(connection_id)}, + ) + return [row_to_dict(r) for r in result.fetchall()] + def update(self, session_id: str, fields: dict) -> bool: """Partial update by PG UUID.""" filtered = { @@ -198,6 +349,7 @@ class ConnectorSessionsRepository: else: set_clauses.append(f"{col} = :{col}") params[col] = val + set_clauses.append("updated_at = now()") result = self._conn.execute( text( f"UPDATE connector_sessions SET {', '.join(set_clauses)} " @@ -263,7 +415,7 @@ class ConnectorSessionsRepository: Notes: The conflict target matches the table's composite unique - constraint ``(user_id, COALESCE(server_url, ''), provider)`` + index ``(user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, ''))`` so MCP's per-URL rows and OAuth's single-row-per-user rows both upsert idempotently. """ @@ -284,7 +436,7 @@ class ConnectorSessionsRepository: :user_id, :provider, :server_url, CAST(:patch AS jsonb) ) - ON CONFLICT (user_id, COALESCE(server_url, ''), provider) + ON CONFLICT (user_id, provider, COALESCE(server_url, ''), COALESCE(account_label, '')) DO UPDATE SET server_url = COALESCE(EXCLUDED.server_url, connector_sessions.server_url), session_data = diff --git a/docsgpt/storage/db/repositories/sources.py b/docsgpt/storage/db/repositories/sources.py index 2406af23..d22451ae 100644 --- a/docsgpt/storage/db/repositories/sources.py +++ b/docsgpt/storage/db/repositories/sources.py @@ -399,6 +399,30 @@ class SourcesRepository: ) self._conn.execute(stmt) + def set_wiki_outside_edits(self, source_id: str, user_id: str, allowed: bool) -> bool: + """Record whether API-key and widget runs may edit this wiki. + + Kept out of :meth:`update`'s columns so no route that forwards a + request body can change it; only the owner-checked wiki settings + route calls this. + + Args: + source_id: The source's UUID. + user_id: The owner's id; the row is scoped to it. + allowed: The new value. + + Returns: + bool: Whether a row was updated. + """ + t = sources_table + result = self._conn.execute( + t.update() + .where(t.c.id == source_id) + .where(t.c.user_id == user_id) + .values(wiki_outside_edits=bool(allowed), updated_at=func.now()) + ) + return result.rowcount > 0 + def get_by_legacy_id( self, legacy_mongo_id: str, user_id: Optional[str] = None, ) -> Optional[dict]: diff --git a/docsgpt/storage/db/repositories/user_tools.py b/docsgpt/storage/db/repositories/user_tools.py index ff235858..0c8e5ca7 100644 --- a/docsgpt/storage/db/repositories/user_tools.py +++ b/docsgpt/storage/db/repositories/user_tools.py @@ -23,8 +23,11 @@ from docsgpt.storage.db.base_repository import looks_like_uuid, row_to_dict _JSONB_COLUMNS = {"config", "config_requirements", "actions"} -_SCALAR_COLUMNS = {"name", "custom_name", "display_name", "description", "status"} -_ALLOWED_COLUMNS = _SCALAR_COLUMNS | _JSONB_COLUMNS +_SCALAR_COLUMNS = {"name", "custom_name", "display_name", "description", "status", "credential_mode"} +# Set by server code only (tool creation, connection setup); route handlers +# must not pass client input here. +_UUID_COLUMNS = {"connection_id"} +_ALLOWED_COLUMNS = _SCALAR_COLUMNS | _JSONB_COLUMNS | _UUID_COLUMNS def _encode_jsonb(value: Any) -> Any: @@ -60,6 +63,8 @@ class UserToolsRepository: status: bool = True, extra: Optional[dict] = None, legacy_mongo_id: Optional[str] = None, + connection_id: Optional[str] = None, + credential_mode: str = "owner", ) -> dict: """Insert a new tool row. ``extra`` is merged into the config JSONB.""" cfg = config or {} @@ -70,14 +75,16 @@ class UserToolsRepository: """ INSERT INTO user_tools ( user_id, name, custom_name, display_name, description, - config, config_requirements, actions, status, legacy_mongo_id + config, config_requirements, actions, status, legacy_mongo_id, + connection_id, credential_mode ) VALUES ( :user_id, :name, :custom_name, :display_name, :description, CAST(:config AS jsonb), CAST(:config_requirements AS jsonb), CAST(:actions AS jsonb), - :status, :legacy_mongo_id + :status, :legacy_mongo_id, + CAST(:connection_id AS uuid), :credential_mode ) RETURNING * """ @@ -93,6 +100,8 @@ class UserToolsRepository: "actions": _encode_jsonb(actions or []), "status": status, "legacy_mongo_id": legacy_mongo_id, + "connection_id": str(connection_id) if connection_id else None, + "credential_mode": credential_mode, }, ) return row_to_dict(result.fetchone()) @@ -213,6 +222,9 @@ class UserToolsRepository: if col in _JSONB_COLUMNS: set_clauses.append(f"{col} = CAST(:{col} AS jsonb)") params[col] = _encode_jsonb(val) + elif col in _UUID_COLUMNS: + set_clauses.append(f"{col} = CAST(:{col} AS uuid)") + params[col] = str(val) if val else None else: set_clauses.append(f"{col} = :{col}") params[col] = val diff --git a/docsgpt/vectorstore/embeddings_local.py b/docsgpt/vectorstore/embeddings_local.py index 5a236e3b..97cfe939 100644 --- a/docsgpt/vectorstore/embeddings_local.py +++ b/docsgpt/vectorstore/embeddings_local.py @@ -20,6 +20,7 @@ from the repository itself rather than assumed. import json import logging +import os import threading from dataclasses import replace from typing import Any, List, Optional @@ -39,6 +40,16 @@ _register_lock = threading.Lock() _FALLBACK_ONNX_FILE = "onnx/model.onnx" _FALLBACK_POOLING = "mean" +# The files FastEmbed fetches with every model besides its graph (see +# ``ModelManagement.download_files_from_huggingface``). +_FASTEMBED_SUPPORT_FILES = ( + "config.json", + "tokenizer.json", + "tokenizer_config.json", + "special_tokens_map.json", + "preprocessor_config.json", +) + # Sentence-transformers records how a model turns token vectors into one # vector, and whether it normalises the result, as files in the repository. # Reading them is the difference between running a model and running something @@ -244,6 +255,51 @@ def _spec_for(model_name: str) -> EmbeddingModel: ) +def _complete_model_cache(repo: str, cache_dir: Optional[str]) -> None: + """Download a model's graph when its cached snapshot lacks it. + + FastEmbed first loads a model from the cache and only downloads when that + fails, and a snapshot of the repository counts as cached whatever it + holds. The chunker caches only ``tokenizer.json`` there (see + ``docsgpt/parser/tokenization.py``), so FastEmbed then finds no graph and + fails to load. Fetching the files it needs first makes either order work. + Offline, nothing is fetched and FastEmbed reports what is missing; a + failed download is logged and left to FastEmbed's own sources too. + + Args: + repo: The model's FastEmbed name. + cache_dir: ``EMBEDDINGS_CACHE_DIR``, or None for FastEmbed's default. + """ + from fastembed import TextEmbedding + from fastembed.common.utils import define_cache_dir + from huggingface_hub import hf_hub_download, snapshot_download + + lowered = repo.lower() + description = next( + (d for d in TextEmbedding._list_supported_models() if str(getattr(d, "model", "")).lower() == lowered), + None, + ) + source = getattr(getattr(description, "sources", None), "hf", None) + model_file = getattr(description, "model_file", None) + if not source or not isinstance(model_file, str): + return # Not fetched from Hugging Face: FastEmbed handles it. + needed = [model_file, *(getattr(description, "additional_files", None) or [])] + cache = str(define_cache_dir(cache_dir)) + try: + for filename in needed: + hf_hub_download(source, filename, cache_dir=cache, local_files_only=True) + return + except Exception: # noqa: BLE001 -- any miss means the snapshot is incomplete + pass + if os.environ.get("HF_HUB_OFFLINE", "").strip().upper() in {"1", "TRUE", "YES", "ON"}: + return + logger.warning("The cached %s has no %s; downloading the model files.", source, ", ".join(needed)) + try: + snapshot_download(repo_id=source, allow_patterns=[*_FASTEMBED_SUPPORT_FILES, *needed], cache_dir=cache) + except Exception as exc: # noqa: BLE001 -- best effort: FastEmbed still tries its own sources + logger.warning("Could not complete the cached %s (%s); leaving the download to FastEmbed.", source, exc) + + def _pad_to_longest_in_batch(model: Any) -> None: """Undo a fixed padding width baked into a model's ``tokenizer.json``.""" # FastEmbed enables padding only when the tokenizer declares none, so a @@ -295,6 +351,7 @@ class EmbeddingsWrapper: cache_dir = settings.EMBEDDINGS_CACHE_DIR if cache_dir: init_kwargs["cache_dir"] = cache_dir + _complete_model_cache(self.spec.repo, cache_dir or None) self.model = TextEmbedding(**init_kwargs) except Exception as exc: raise RuntimeError( diff --git a/docsgpt/worker.py b/docsgpt/worker.py index f3311859..2c3fe9ca 100755 --- a/docsgpt/worker.py +++ b/docsgpt/worker.py @@ -32,6 +32,7 @@ from docsgpt.parser.file.image_parser import ( VISION_CONVERTIBLE_MIME_TYPES, convert_image_to_png, ) +from docsgpt.parser.remote.github_loader import GitHubTokenRejected from docsgpt.parser.remote.remote_creator import ( RemoteCreator, normalize_remote_data, @@ -1237,6 +1238,7 @@ def remote_worker( config=None, idempotency_key=None, source_id=None, + connection_id=None, ): safe_user = safe_filename(user) full_path = os.path.join(directory, safe_user, uuid.uuid4().hex) @@ -1283,7 +1285,26 @@ def remote_worker( self.update_state(state="PROGRESS", meta={"current": 1}) logging.info("Initializing remote loader with type: %s", loader) remote_loader = RemoteCreator.create_loader(loader) - raw_docs = remote_loader.load_data(source_data) + loader_input = source_data + if connection_id: + loader_input = _with_connection_credentials(source_data, connection_id) + if loader_input is None: + from docsgpt.connectors.service import ConnectionUnavailable + + raise ConnectionUnavailable("Reconnect to continue", connection_id=str(connection_id)) + try: + raw_docs = remote_loader.load_data(loader_input) + except GitHubTokenRejected as exc: + # A revoked token pauses the connection's sources until the + # owner reconnects, instead of failing on every schedule. + from docsgpt.connectors import service as connection_service + + if not connection_id: + raise + connection_service.mark_reconnect_needed(str(connection_id), str(exc)) + raise connection_service.ConnectionUnavailable( + str(exc), connection_id=str(connection_id), + ) from exc cfg = SourceConfig.parse(config) chunker = ChunkerCreator.create_chunker( @@ -1414,6 +1435,8 @@ def remote_worker( f"Failed to update last_sync for source {source_id_for_events}: {upd_err}" ) upload_index(full_path, file_data) + if connection_id: + _link_source_to_connection(source_id_for_events, str(connection_id)) publish_user_event( user, "source.ingest.completed", @@ -1467,6 +1490,7 @@ def sync( retriever, doc_id=None, directory="temp", + connection_id=None, ): try: remote_worker( @@ -1480,6 +1504,7 @@ def sync( sync_frequency, "sync", doc_id, + connection_id=connection_id, ) except Exception as e: logging.error(f"Error during sync: {e}", exc_info=True) @@ -1487,6 +1512,10 @@ def sync( return {"status": "success"} +# Remote loaders that can only read with a connection (no public fallback). +_CONNECTION_ONLY_LOADERS = frozenset({"linear"}) + + def sync_worker(self, frequency): from sqlalchemy import text as sql_text @@ -1494,7 +1523,7 @@ def sync_worker(self, frequency): with db_readonly() as conn: result = conn.execute( sql_text( - "SELECT id, name, user_id, type, remote_data, retriever " + "SELECT id, name, user_id, type, remote_data, retriever, connection_id, metadata " "FROM sources WHERE sync_frequency = :freq" ), {"freq": frequency}, @@ -1511,9 +1540,38 @@ def sync_worker(self, frequency): sync_counts["total_sync_count"] += 1 - # Connector sources have no RemoteCreator loader and need an OAuth - # token to sync, which a scheduled task lacks — skip them. + # Connector sources sync from their connection, whose token the + # worker can refresh. Legacy ones with no connection still need the + # browser, so they are skipped as before. if source_type and source_type.startswith("connector"): + if doc.get("connection_id"): + from docsgpt.api.user.tasks import sync_connector_source as sync_task + + sync_task.delay(doc_id) + sync_counts["sync_dispatched"] += 1 + else: + sync_counts["sync_skipped"] += 1 + continue + + metadata = doc.get("metadata") + if isinstance(metadata, str): + try: + metadata = json.loads(metadata) + except ValueError: + metadata = {} + if ( + doc.get("connection_id") + and isinstance(metadata, dict) + and metadata.get("sync_state") == "paused_reconnect" + ): + # An S3 or GitHub source whose connection needs reconnecting: + # it resumes when the owner reconnects, rather than failing (and + # notifying) on every schedule until then. + sync_counts["sync_skipped"] += 1 + continue + if source_type in _CONNECTION_ONLY_LOADERS and not doc.get("connection_id"): + # Linear is read only with a connection's sign-in; its + # connection was removed and the content kept. sync_counts["sync_skipped"] += 1 continue @@ -1525,7 +1583,8 @@ def sync_worker(self, frequency): continue resp = sync( - self, source_data, name, user, source_type, frequency, retriever, doc_id + self, source_data, name, user, source_type, frequency, retriever, doc_id, + connection_id=str(doc["connection_id"]) if doc.get("connection_id") else None, ) sync_counts[ "sync_success" if resp["status"] == "success" else "sync_failure" @@ -1533,7 +1592,7 @@ def sync_worker(self, frequency): return { key: sync_counts[key] for key in [ - "total_sync_count", "sync_success", "sync_failure", "sync_skipped", + "total_sync_count", "sync_success", "sync_failure", "sync_skipped", "sync_dispatched", ] } @@ -2161,6 +2220,132 @@ def _webhook_tool_allowlist(agent_config): return [] +def _with_connection_credentials(source_data, connection_id: str): + """Loader input with the connection's stored keys merged in, or None. + + S3, Reddit and GitHub sources made from a connection keep their keys on + the connection only, never in ``sources.remote_data``. A JSON string + stays a JSON string and a dict a dict; any other string (a GitHub + repository URL) becomes ``{"url": ...}`` next to the keys. An MCP + sign-in (Linear) gets its ``connection_id`` instead: its tokens stay + with the MCP client, which renews them. Returns None when the + connection is gone, needs reconnecting, or its connector is turned off. + """ + from docsgpt.connectors import service + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + with db_readonly() as conn: + row = ConnectorSessionsRepository(conn).get(str(connection_id)) + enabled = row is not None and service.connector_enabled(conn, row) + if row is None or not enabled: + return None + if (row.get("auth_kind") or "") == "mcp_oauth": + if service.normalize_status(row) != service.STATUS_CONNECTED: + return None + credentials = {"connection_id": str(row["id"])} + else: + try: + credentials = service.access_credentials(row) + except service.ConnectionUnavailable: + return None + if isinstance(source_data, str): + try: + parsed = json.loads(source_data) + except ValueError: + parsed = None + if isinstance(parsed, dict): + return json.dumps({**parsed, **credentials}) + return {"url": source_data, **credentials} + return {**dict(source_data or {}), **credentials} + + +def _link_source_to_connection(source_id: str, connection_id: str) -> None: + """Point a source at the connection it syncs from, and lift any reconnect pause.""" + from sqlalchemy import text as sql_text + + try: + with db_session() as conn: + conn.execute( + sql_text( + "UPDATE sources SET connection_id = CAST(:cid AS uuid), " + "metadata = metadata - 'sync_state' WHERE id = CAST(:sid AS uuid)" + ), + {"cid": str(connection_id), "sid": str(source_id)}, + ) + except Exception: + logging.warning("Could not link source %s to connection %s", source_id, connection_id, exc_info=True) + + +def sync_connector_source(self, source_id: str) -> Dict[str, Any]: + """Re-download and re-index a connector source from its connection. + + Runs as the connection owner with no browser: the connection service + refreshes the token under a row lock. When the grant was revoked the + service flags the connection and pauses its sources, and this returns + ``paused`` rather than failing again on every schedule. + + Args: + self: The bound Celery task. + source_id: The source to sync. + + Returns: + ``{"status": "success" | "paused" | "disabled" | "skipped"}`` plus + the ingest result. + """ + from docsgpt.connectors.service import ConnectionUnavailable, connector_enabled, normalize_status + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + with db_readonly() as conn: + from sqlalchemy import text as sql_text + + row = conn.execute( + sql_text( + "SELECT id, name, user_id, remote_data, retriever, sync_frequency, config, connection_id " + "FROM sources WHERE id = CAST(:id AS uuid)" + ), + {"id": str(source_id)}, + ).fetchone() + source = dict(row._mapping) if row else None + connection = ( + ConnectorSessionsRepository(conn).get(str(source["connection_id"])) + if source and source.get("connection_id") + else None + ) + enabled = connection is not None and connector_enabled(conn, connection) + if not source or not connection: + return {"status": "skipped"} + if not enabled: + return {"status": "disabled"} + if normalize_status(connection) != "connected": + return {"status": "paused"} + remote_data = source.get("remote_data") or {} + if isinstance(remote_data, str): + try: + remote_data = json.loads(remote_data) + except json.JSONDecodeError: + remote_data = {} + provider = remote_data.get("provider") or connection.get("provider") + try: + result = ingest_connector( + self, + source.get("name"), + source.get("user_id"), + provider, + connection_id=str(connection["id"]), + file_ids=remote_data.get("file_ids") or [], + folder_ids=remote_data.get("folder_ids") or [], + recursive=remote_data.get("recursive", True), + retriever=source.get("retriever") or "classic", + operation_mode="sync", + doc_id=str(source["id"]), + sync_frequency=source.get("sync_frequency") or "never", + config=source.get("config") or None, + ) + except ConnectionUnavailable: + return {"status": "paused"} + return {"status": "success", "result": result} + + def ingest_connector( self, job_name: str, @@ -2177,6 +2362,7 @@ def ingest_connector( config=None, idempotency_key=None, source_id=None, + connection_id=None, ) -> Dict[str, Any]: """ Ingestion for internal knowledge bases (GoogleDrive, etc.). @@ -2185,7 +2371,8 @@ def ingest_connector( job_name: Name of the ingestion job user: User identifier source_type: Type of remote source ("google_drive", "dropbox", etc.) - session_token: Authentication token for the service + session_token: Legacy browser session token naming the connection + connection_id: The connection whose account the files are read with file_ids: List of file IDs to download folder_ids: List of folder IDs to download recursive: Whether to recursively download folders @@ -2247,8 +2434,8 @@ def ingest_connector( meta={"current": 10, "status": "Initializing connector"}, ) - if not session_token: - raise ValueError(f"{source_type} connector requires session_token") + if not session_token and not connection_id: + raise ValueError(f"{source_type} connector requires a connection") if not ConnectorCreator.is_supported(source_type): raise ValueError( @@ -2256,8 +2443,9 @@ def ingest_connector( ) remote_loader = ConnectorCreator.create_connector( - source_type, session_token + source_type, session_token, connection_id=connection_id ) + connection_id = remote_loader.connection_id # Create a clean config for storage api_source_config = { @@ -2411,6 +2599,8 @@ def ingest_connector( ) upload_index(vector_store_path, file_data) + if connection_id: + _link_source_to_connection(source_id_for_events, connection_id) # Ensure we mark the task as complete self.update_state( diff --git a/frontend/DESIGN.md b/frontend/DESIGN.md index b2c8b657..aece8ffc 100644 --- a/frontend/DESIGN.md +++ b/frontend/DESIGN.md @@ -647,10 +647,12 @@ description use a Switch in a SettingRow instead. ### Switch, TimePicker and Calendar (`ui/switch.tsx`, `ui/time-picker.tsx`, `ui/calendar.tsx`) `Switch` (Radix) is an on/off setting, placed inside a `SettingRow` that names -it. A tile's own on/off (the tool tile's "In my chats") is the one exception: -a `Label text-muted-foreground text-xs font-normal` + `Switch` pair at the -tile's bottom-right. For a shared tool it is the caller's own preference, -never a switch that turns the tool off for everyone. The track is `primary` when on and `bg-input` when off, with a white +it. A tool's own "In my chats" on/off is the one exception. On a Tools page +tile it is a bare `Switch` at the tile's bottom-right, named only by its +`aria-label` (no visible label); in a connection's drawer, a tool row ends in +a `Label text-muted-foreground text-xs font-normal` + `Switch` pair. Either way +it is the caller's own preference, never a switch that turns the tool off for +everyone. The track is `primary` when on and `bg-input` when off, with a white thumb in both themes (see Elevation). `TimePicker` picks a time of day as two `SelectTrigger`s, hours and minutes, @@ -806,9 +808,10 @@ overlay. The app has one `ToastViewport`, mounted in `App.tsx`; it is the live region (`role="status"`, `aria-live="polite"`) and the fixed stack, so `Toast` cards carry no role and no toast renders its own rail or positioning. Top -to bottom it holds `TeamNotificationToast`, `ToolApprovalToast`, -`UploadToast` and `ActionToast`, and it moves to the bottom-left while any -agent preview drawer is open (workflow or classic). A new toast component returns only its +to bottom it holds `TeamNotificationToast`, `ConnectionHealthToast` (a +connection that needs reconnecting, with a Reconnect action), +`ToolApprovalToast`, `UploadToast` and `ActionToast`, and it moves to the +bottom-left while any agent preview drawer is open (workflow or classic). A new toast component returns only its `Toast` cards and is added to that viewport. A page that reports the result of an action (the admin Users actions) dispatches `showActionToast({ variant: 'success' | 'destructive', message })` from @@ -888,7 +891,8 @@ also refuses Logs, Schedules and Pin on a draft), never on `ownership`, (`components/ViewOnlyNotice`: an `Alert role="note"` with `Lock` first and `common.viewOnlyNotice`) as its first child. Where only part of an editable form is locked (a tool's credentials when the owner turned off "Editors can - change credentials"), the same notice passes its own `message`. There is no Save; Cancel becomes a lone Close. + change credentials", or when the tool runs on the owner's connection, whose + secret only the owner changes), the same notice passes its own `message`. There is no Save; Cancel becomes a lone Close. Fields are `disabled` (a group: `
`); long text the user reads or copies (a prompt, a chunk) is `readOnly`, so it keeps full contrast and scrolls. Titles and menu items swap Edit and @@ -1670,6 +1674,7 @@ list stays reviewable. | `settings/PersonalAccessTokens.tsx` | `shadcn/no-restyle` | Token scope chips are identifiers, so the `neutral` Badge is set in mono (`font-mono`). One disable. | | `agents/workflow/WorkflowPreview.tsx` | `shadcn/no-restyle` | A step's state changes (keys and values the app serialised) are `neutral` Badges set in mono (`font-mono`), like token scopes. One disable. | | `settings/traces/TraceChips.tsx` | `shadcn/no-restyle` | Trace stat chips (durations, counts) use tabular figures so they don't jitter between rows (`tabular-nums` on Badge). One disable. | +| `connectors/ConnectorSetupNotice.tsx` | `shadcn/no-restyle` | The setup-guide link inside the needs-setup warning Alert keeps the Alert's status colour (`text-current` on `link inline`). One disable. | | `modals/MCPServerModal.tsx` | `shadcn/no-restyle` | The authorization link inside the test-result Alert keeps the Alert's status colour (`text-current` on `link inline`). One disable. | | `components/MermaidRenderer.tsx` | `shadcn/no-restyle` | The zoom readout between − and + is a `link inline` Button on the `bg-black/70` overlay; it keeps the overlay's white 12px regular text (`text-xs font-normal text-current`). One disable, beside the two zoom-button ones above. | | `conversation/SharedConversation.tsx` | `shadcn/no-restyle` | The "DocsGPT" link sits in the `/share/:id` page's regular-weight byline (`font-normal`). One disable. | diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 20f2ed11..65d84787 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -43,6 +43,7 @@ import Notification from './components/Notification'; import ToolApprovalToast from './notifications/ToolApprovalToast'; import TeamNotificationToast from './notifications/TeamNotificationToast'; import ActionToast from './notifications/ActionToast'; +import ConnectionHealthToast from './notifications/ConnectionHealthToast'; function AuthWrapper({ children }: { children: React.ReactNode }) { const { t } = useTranslation(); @@ -128,6 +129,7 @@ function MainLayout() { onMouseDown={(e) => e.stopPropagation()} > + diff --git a/frontend/src/Hero.test.tsx b/frontend/src/Hero.test.tsx new file mode 100644 index 00000000..ca1dfe97 --- /dev/null +++ b/frontend/src/Hero.test.tsx @@ -0,0 +1,121 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter, Route, Routes } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => { + if (key === 'demo') + return [1, 2, 3, 4].map((n) => ({ + header: `demo${n}`, + query: `q${n}`, + })); + return opts?.names ? `${key}:${opts.names}` : key; + }, + }), +})); + +vi.mock('./api/services/modelService', () => ({ + default: { + getModels: vi.fn(async () => ({ ok: true, json: async () => ({}) })), + transformModels: () => [], + }, +})); +vi.mock('./hooks', () => ({ useDarkTheme: () => [false] })); + +import connectorsReducer from './connectors/connectorsSlice'; +import Hero from './Hero'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const connector = (key: string, name: string, publisher = 'built_in') => ({ + key, + name, + publisher, + available: true, +}); + +describe('Hero connect card', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (connections: unknown[]) => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: ( + state = { + token: null, + selectedModel: null, + availableModels: [], + modelsLoading: false, + }, + ) => state, + }, + preloadedState: { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [ + connector('google_drive', 'Google Drive'), + connector('mcp:notion', 'Notion', 'preset'), + connector('custom_mcp', 'MCP server', 'custom'), + ], + connections, + }, + }, + } as Parameters[0]); + await act(async () => { + root.render( + + + + } /> + CONNECTORS} + /> + + + , + ); + }); + }; + + const card = () => + container.querySelector( + '[data-testid="hero-connect-card"]', + ); + + it('offers connecting the services this install has', async () => { + await render([]); + expect(card()?.textContent).toContain( + 'connectHero.body:Google Drive, Notion', + ); + // It replaces the last demo card. + expect(container.textContent).not.toContain('demo4'); + await act(async () => card()!.click()); + expect(container.textContent).toContain('CONNECTORS'); + }); + + it('keeps the demo cards once something is connected', async () => { + await render([{ id: 'c1', status: 'connected' }]); + expect(card()).toBeNull(); + expect(container.textContent).toContain('demo4'); + }); +}); diff --git a/frontend/src/Hero.tsx b/frontend/src/Hero.tsx index bba6aebd..3b706776 100644 --- a/frontend/src/Hero.tsx +++ b/frontend/src/Hero.tsx @@ -1,5 +1,6 @@ import { useEffect, useRef } from 'react'; import { useTranslation } from 'react-i18next'; +import { useNavigate } from 'react-router-dom'; import { useDispatch, useSelector } from 'react-redux'; import modelService from './api/services/modelService'; @@ -14,6 +15,13 @@ import { SelectTrigger, SelectValue, } from './components/ui/select'; +import { + selectConnections, + selectConnectorCatalog, + selectConnectorsEnabled, + selectConnectorsLoaded, +} from './connectors/connectorsSlice'; +import { connectorName } from './connectors/i18n'; import { useDarkTheme } from './hooks'; import { selectAvailableModels, @@ -145,10 +153,31 @@ export default function Hero({ }) { const { t } = useTranslation(); const [isDarkTheme] = useDarkTheme(); + const navigate = useNavigate(); const demos = t('demo', { returnObjects: true }) as Array<{ header: string; query: string; }>; + // Nothing connected yet: the last card offers connecting a service, named + // after what this install can actually connect. + const connectorsEnabled = useSelector(selectConnectorsEnabled); + const connectorsLoaded = useSelector(selectConnectorsLoaded); + const connections = useSelector(selectConnections); + const catalog = useSelector(selectConnectorCatalog); + const connectable = catalog.filter( + (connector) => connector.available && connector.publisher !== 'custom', + ); + const offerConnect = + connectorsEnabled && + connectorsLoaded && + connections.length === 0 && + connectable.length > 0; + const connectNames = connectable + .slice(0, 2) + .map((connector) => connectorName(t, connector)) + .join(', '); + const cards = (demos ?? []).filter((demo) => demo.header && demo.query); + const shown = offerConnect ? cards.slice(0, 3) : cards; return (
@@ -170,7 +199,7 @@ export default function Hero({ {/* Demo Buttons Section */}
- {demos?.map( + {shown.map( (demo: { header: string; query: string }, key: number) => demo.header && demo.query && ( @@ -199,6 +228,31 @@ export default function Hero({ ), )} + {offerConnect && ( + + )}
diff --git a/frontend/src/Navigation.tsx b/frontend/src/Navigation.tsx index 2344ca16..f853e514 100644 --- a/frontend/src/Navigation.tsx +++ b/frontend/src/Navigation.tsx @@ -56,6 +56,7 @@ import { getSectionForPath, type Section, } from './navigation/sections'; +import ConnectionHealthDot from './connectors/ConnectionHealthDot'; import { useSidebarLevel } from './navigation/SidebarLevelProvider'; import { useSectionContext } from './navigation/useSectionContext'; import { useLastAppPath } from './navigation/useLastAppPath'; @@ -798,6 +799,7 @@ export default function Navigation({ navOpen, setNavOpen }: NavigationProps) { aria-hidden />

{t('settings.label')}

+ diff --git a/frontend/src/admin/Connectors.test.tsx b/frontend/src/admin/Connectors.test.tsx new file mode 100644 index 00000000..0d4ac1eb --- /dev/null +++ b/frontend/src/admin/Connectors.test.tsx @@ -0,0 +1,358 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); + +const getAdmin = vi.fn(); +const updateAdmin = vi.fn(); +vi.mock('../api/services/connectorsService', () => ({ + default: { + getAdmin: (...args: unknown[]) => getAdmin(...args), + updateAdmin: (...args: unknown[]) => updateAdmin(...args), + }, +})); + +import actionToastReducer, { + selectActionToast, +} from '../notifications/actionToastSlice'; +import { prefSlice } from '../preferences/preferenceSlice'; +import Connectors from './Connectors'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const connector = (overrides: Record = {}) => ({ + key: 'google_drive', + name: 'Google Drive', + icon: 'google-drive', + publisher: 'built_in', + auth_kind: 'oauth', + capabilities: ['sync'], + enabled: true, + credential_mode: 'choose', + configured: false, + required_settings: [ + { name: 'GOOGLE_CLIENT_ID', set: true }, + { name: 'GOOGLE_CLIENT_SECRET', set: false }, + ], + connection_count: 3, + docs_url: null, + mcp_url: null, + ...overrides, +}); + +const MCP_ROW = connector({ + key: 'custom_mcp', + name: 'MCP server', + icon: 'mcp', + publisher: 'custom', + auth_kind: 'mcp', + capabilities: ['read', 'write'], + configured: true, + required_settings: [], + connection_count: 0, +}); +const NOTION = { + key: 'mcp_notion', + name: 'Notion', + icon: 'notion', + publisher: 'preset', + auth_kind: 'mcp_oauth', + capabilities: ['read', 'write'], + configured: true, + required_settings: [], + connection_count: 0, +}; + +const payload = (overrides: Record = {}) => ({ + success: true, + connectors: [ + connector(), + connector({ + key: 'mcp_notion', + name: 'Notion', + icon: 'notion', + publisher: 'preset', + auth_kind: 'mcp_oauth', + capabilities: ['read', 'write'], + configured: true, + required_settings: [], + connection_count: 0, + }), + MCP_ROW, + ], + allow_custom_mcp: true, + default_encryption_key: false, + oauth_redirect_uri: 'https://docs.example/api/connectors/callback', + mcp_redirect_uri: 'https://docs.example/api/mcp_server/callback', + ...overrides, +}); + +describe('Admin Connectors', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + getAdmin.mockReset(); + updateAdmin.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(() => { + act(() => root.unmount()); + container.remove(); + }); + + let store: ReturnType; + const makeStore = () => + configureStore({ + reducer: { + preference: prefSlice.reducer, + actionToast: actionToastReducer, + }, + }); + + const render = async () => { + store = makeStore(); + await act(async () => { + root.render( + + + , + ); + }); + }; + + it('lists each connector with its setup state and redirect URIs', async () => { + getAdmin.mockResolvedValue(payload()); + await render(); + const rows = Array.from(container.querySelectorAll('tbody tr')); + expect(rows).toHaveLength(3); + expect(rows[0].textContent).toContain('Google Drive'); + expect(rows[0].textContent).toContain('Needs setup'); + expect(rows[0].textContent).toContain('Setup guide'); + expect(rows[1].textContent).toContain('Preset'); + expect(rows[1].textContent).toContain('Ready'); + expect(rows[1].textContent).not.toContain('Setup guide'); + expect(container.textContent).toContain( + 'https://docs.example/api/connectors/callback', + ); + }); + + it('shows GitHub as ready, with its optional GitHub App settings in the guide', async () => { + const GITHUB = connector({ + key: 'github', + name: 'GitHub', + icon: 'github', + auth_kind: 'api_key', + capabilities: ['sync', 'read'], + configured: true, + required_settings: [], + oauth_settings: [ + { name: 'GITHUB_CLIENT_ID', set: true }, + { name: 'GITHUB_CLIENT_SECRET', set: false }, + { name: 'GITHUB_APP_SLUG', set: false }, + ], + oauth_configured: false, + connection_count: 0, + }); + getAdmin.mockResolvedValue(payload({ connectors: [GITHUB] })); + await render(); + const row = container.querySelector('tbody tr')!; + expect(row.textContent).toContain('Ready'); + expect(row.textContent).toContain('Tokens only'); + const guide = Array.from(row.querySelectorAll('button')).find( + (b) => b.textContent === 'Setup guide', + )!; + await act(async () => guide.click()); + const text = document.body.textContent ?? ''; + expect(text).toContain('Sign in with GitHub'); + expect(text).toContain('GITHUB_APP_SLUG'); + expect(text).toContain( + 'Request user authorization (OAuth) during installation', + ); + expect(text).toContain('https://docs.example/api/connectors/callback'); + }); + + it('warns when credentials use the default encryption key', async () => { + getAdmin.mockResolvedValue(payload({ default_encryption_key: true })); + await render(); + expect(container.textContent).toContain( + 'Set ENCRYPTION_SECRET_KEY before connecting services.', + ); + expect(container.textContent).toContain('docsgpt connectors reencrypt'); + }); + + it('saves a connector toggle as a policy', async () => { + getAdmin.mockResolvedValue(payload()); + updateAdmin.mockResolvedValue( + payload({ connectors: [connector({ ...NOTION, enabled: false })] }), + ); + await render(); + const toggle = () => + container.querySelector( + 'table [aria-label="Notion enabled"]', + )!; + await act(async () => toggle().click()); + expect(updateAdmin).toHaveBeenCalledWith( + { policies: { mcp_notion: { enabled: false } } }, + null, + ); + expect(toggle().getAttribute('aria-checked')).toBe('false'); + }); + + it('keeps a connector that needs setup off, and says why', async () => { + getAdmin.mockResolvedValue(payload()); + await render(); + const drive = container.querySelector( + 'table [aria-label="Google Drive enabled"]', + )!; + expect(drive.disabled).toBe(true); + }); + + it('shows no sharing policy for a sync-only connector', async () => { + getAdmin.mockResolvedValue(payload()); + await render(); + const [drive, notion] = Array.from(container.querySelectorAll('tbody tr')); + expect(drive.textContent).toContain('No tools'); + expect( + notion.querySelector('[aria-label="Notion sharing policy"]'), + ).not.toBeNull(); + }); + + it('turns custom MCP servers off from their own row', async () => { + getAdmin.mockResolvedValue(payload()); + updateAdmin.mockResolvedValue(payload({ allow_custom_mcp: false })); + await render(); + const mcp = () => + container.querySelector( + 'table [aria-label="MCP server enabled"]', + )!; + await act(async () => mcp().click()); + expect(updateAdmin).toHaveBeenCalledWith( + { allow_custom_mcp: false, policies: { custom_mcp: { enabled: false } } }, + null, + ); + expect(mcp().getAttribute('aria-checked')).toBe('false'); + expect(container.querySelector('#allow-custom-mcp')).toBeNull(); + }); + + it('lists connectors on phones and opens their controls in a sheet', async () => { + getAdmin.mockResolvedValue(payload()); + await render(); + const rows = Array.from( + container.querySelectorAll( + '[data-slot="list-row"] button', + ), + ); + expect(rows).toHaveLength(3); + expect(rows[0].textContent).toContain('Off · 3 connections · No tools'); + await act(async () => rows[1].click()); + expect( + document.body.querySelector('[data-slot="sheet-content"]'), + ).not.toBeNull(); + expect(document.body.textContent).toContain('Shared tools use'); + }); + + it('reports a failed save in a toast and keeps the page', async () => { + getAdmin.mockResolvedValue(payload()); + updateAdmin.mockResolvedValue({ success: false }); + await render(); + await act(async () => + container + .querySelector( + 'table [aria-label="Notion enabled"]', + )! + .click(), + ); + expect(updateAdmin).toHaveBeenCalledWith( + { policies: { mcp_notion: { enabled: false } } }, + null, + ); + expect(selectActionToast(store.getState())).toMatchObject({ + variant: 'destructive', + message: 'Could not save the change.', + }); + expect(container.querySelectorAll('tbody tr')).toHaveLength(3); + }); + + it('saves one change at a time so a late response cannot win', async () => { + getAdmin.mockResolvedValue(payload()); + const pending: ((value: unknown) => void)[] = []; + updateAdmin.mockImplementation( + () => new Promise((resolve) => pending.push(resolve)), + ); + await render(); + const notion = () => + container.querySelector( + 'table [aria-label="Notion enabled"]', + )!; + const mcp = () => + container.querySelector( + 'table [aria-label="MCP server enabled"]', + )!; + await act(async () => notion().click()); + await act(async () => mcp().click()); + // The second save waits for the first. + expect(updateAdmin).toHaveBeenCalledTimes(1); + const off = connector({ ...NOTION, enabled: false }); + await act(async () => pending[0](payload({ connectors: [off, MCP_ROW] }))); + expect(updateAdmin).toHaveBeenCalledTimes(2); + await act(async () => + pending[1]( + payload({ connectors: [off, MCP_ROW], allow_custom_mcp: false }), + ), + ); + expect(notion().getAttribute('aria-checked')).toBe('false'); + expect(mcp().getAttribute('aria-checked')).toBe('false'); + }); + + describe('write access', () => { + const GITHUB = connector({ + key: 'github', + name: 'GitHub', + icon: 'github', + auth_kind: 'api_key', + capabilities: ['sync', 'read'], + configured: true, + required_settings: [], + allow_writes: true, + }); + const writes = () => + container.querySelector('#allow-writes-github'); + + it('lets an admin forbid changes through GitHub', async () => { + getAdmin.mockResolvedValue(payload({ connectors: [GITHUB, MCP_ROW] })); + updateAdmin.mockResolvedValue( + payload({ + connectors: [{ ...GITHUB, allow_writes: false }, MCP_ROW], + }), + ); + await render(); + expect(writes()!.getAttribute('aria-checked')).toBe('true'); + await act(async () => writes()!.click()); + expect(updateAdmin).toHaveBeenCalledWith( + { policies: { github: { allow_writes: false } } }, + null, + ); + expect(writes()!.getAttribute('aria-checked')).toBe('false'); + }); + + it('has no write access section without such a connector', async () => { + getAdmin.mockResolvedValue(payload()); + await render(); + expect(container.textContent).not.toContain('Write access'); + }); + }); + + it('offers a retry when loading fails', async () => { + getAdmin.mockRejectedValue(new Error('offline')); + await render(); + expect(container.textContent).toContain('Failed to load connectors.'); + }); +}); diff --git a/frontend/src/admin/Connectors.tsx b/frontend/src/admin/Connectors.tsx new file mode 100644 index 00000000..ea629c6c --- /dev/null +++ b/frontend/src/admin/Connectors.tsx @@ -0,0 +1,609 @@ +import { ExternalLink, Info, TriangleAlert } from 'lucide-react'; +import { useCallback, useEffect, useRef, useState } from 'react'; +import { useDispatch, useSelector } from 'react-redux'; + +import connectorsService from '../api/services/connectorsService'; +import CopyButton from '../components/CopyButton'; +import PageToolbar from '../components/PageToolbar'; +import { Alert, AlertDescription, AlertTitle } from '../components/ui/alert'; +import { Badge } from '../components/ui/badge'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { FormField } from '../components/ui/form-field'; +import { ListRow, ListRows } from '../components/ui/list-row'; +import { + Sheet, + SheetContent, + SheetDescription, + SheetTitle, +} from '../components/ui/sheet'; +import { + DescriptionItem, + DescriptionList, +} from '../components/ui/description-list'; +import { LoadingState } from '../components/ui/loading-state'; +import { Modal } from '../components/ui/modal'; +import { SectionHeader } from '../components/ui/section-header'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '../components/ui/select'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import { + Table, + TableBody, + TableCell, + TableContainer, + TableHead, + TableHeader, + TableRow, +} from '../components/ui/table'; +import { + Tooltip, + TooltipContent, + TooltipTrigger, +} from '../components/ui/tooltip'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { showActionToast } from '../notifications/actionToastSlice'; +import { selectToken } from '../preferences/preferenceSlice'; +import { LoadError, fmtNumber } from './AdminUI'; + +type Policy = 'choose' | 'owner' | 'member'; + +type AdminConnector = { + key: string; + name: string; + icon: string; + publisher: 'built_in' | 'preset' | 'custom'; + auth_kind: string; + capabilities: string[]; + enabled: boolean; + credential_mode: Policy; + configured: boolean; + required_settings: { name: string; set: boolean }[]; + /** Optional settings that add a second sign-in (GitHub's App). */ + oauth_settings?: { name: string; set: boolean }[]; + oauth_configured?: boolean; + connection_count: number; + docs_url: string | null; + mcp_url: string | null; + /** + * Whether members may let agents make changes through it (GitHub's write + * tools); null where the connector offers no such choice. + */ + allow_writes?: boolean | null; +}; + +type AdminConnectorsData = { + success: boolean; + connectors: AdminConnector[]; + allow_custom_mcp: boolean; + default_encryption_key: boolean; + oauth_redirect_uri: string; + mcp_redirect_uri: string; +}; + +const POLICY_LABELS: Record = { + choose: 'The sharer decides per share', + member: "Always each person's own account", + owner: "Always the sharer's account", +}; + +const hasTools = (connector: AdminConnector) => + connector.capabilities.some((capability) => capability !== 'sync'); + +const hasSetupGuide = (connector: AdminConnector) => + connector.required_settings.length > 0 || + (connector.oauth_settings?.length ?? 0) > 0; + +/** Works with pasted tokens, but its optional OAuth sign-in is not set up yet. */ +const tokensOnly = (connector: AdminConnector) => + (connector.oauth_settings?.length ?? 0) > 0 && !connector.oauth_configured; + +function SettingsList({ + settings, +}: { + settings: { name: string; set: boolean }[]; +}) { + return ( + + {settings.map((setting) => ( + {setting.name}} + > + + {setting.set ? 'Set' : 'Missing'} + + + ))} + + ); +} + +function CodeRow({ value }: { value: string }) { + return ( + + 
+        {value}
+
+ + + ); +} + +function SetupGuide({ + connector, + redirectUri, + onClose, +}: { + connector: AdminConnector; + redirectUri: string; + onClose: () => void; +}) { + const oauthSettings = connector.oauth_settings ?? []; + const optionalOAuth = + connector.required_settings.length === 0 && oauthSettings.length > 0; + return ( + !open && onClose()} + title={`Set up ${connector.name}`} + description={ + optionalOAuth + ? `Members can already connect ${connector.name} with their own access tokens. To also offer Sign in with ${connector.name}, register a GitHub App, then set these server settings and restart the API and the worker.` + : 'Register DocsGPT as an OAuth app with the provider, then set these server settings and restart the API and the worker.' + } + footer={ + + } + > +
+
+ + +
+ {connector.required_settings.length > 0 && ( +
+ + +
+ )} + {oauthSettings.length > 0 && ( +
+ + +
+ )} + {connector.key === 'github' && ( + + + + In the GitHub App, give repository permissions Contents and + Metadata read-only access, and turn on Request user authorization + (OAuth) during installation so choosing repositories returns to + DocsGPT. For agents to make changes, also give Issues and Pull + requests read and write access (Contents read and write only if + agents should edit files). GITHUB_APP_SLUG is the name in the + app's public link (github.com/apps/<slug>). + + + )} + {connector.key === 'google_drive' && ( + + + + Publish the Google OAuth app (or use an internal Workspace app). + Apps left in Testing get refresh tokens that expire after seven + days, which stops background sync. + + + )} + {connector.docs_url && ( + + )} +
+ + ); +} + +/** + * Admin > Connectors: which connectors members may use, whose account a + * shared tool runs with, and what each OAuth connector still needs from the + * server. English only, like the rest of the admin pages. + */ +export default function Connectors() { + const dispatch = useDispatch(); + const token = useSelector(selectToken); + const [data, setData] = useState(null); + const [loading, setLoading] = useState(true); + const [guide, setGuide] = useState(null); + const [detailKey, setDetailKey] = useState(null); + + const load = useCallback(async () => { + setLoading(true); + try { + setData(await connectorsService.getAdmin(token)); + } catch { + setData({ success: false } as AdminConnectorsData); + } finally { + setLoading(false); + } + }, [token]); + + useEffect(() => { + load(); + }, [load]); + + // Saves run one after another: each response is a full snapshot, so an + // older one arriving last would otherwise put back a stale policy. + const saveQueue = useRef>(Promise.resolve()); + const save = (body: { + policies?: Record< + string, + { enabled?: boolean; credential_mode?: Policy; allow_writes?: boolean } + >; + allow_custom_mcp?: boolean; + }) => { + const failed = () => + dispatch( + showActionToast({ + variant: 'destructive', + message: 'Could not save the change.', + }), + ); + saveQueue.current = saveQueue.current.then(async () => { + try { + const next = await connectorsService.updateAdmin(body, token); + if (next?.success) setData(next); + else failed(); + } catch { + failed(); + } + }); + return saveQueue.current; + }; + + if (data === null && loading) return ; + if (!data?.success) + return ; + + // The sheet (phones) always shows the connector's latest saved state. + const detail = data.connectors.find((c) => c.key === detailKey) ?? null; + const writable = data.connectors.filter( + (connector) => + connector.allow_writes !== null && connector.allow_writes !== undefined, + ); + + // The custom MCP row is the one switch for members' own MCP servers. + const isCustomMcp = (connector: AdminConnector) => + connector.key === 'custom_mcp'; + const enabledOf = (connector: AdminConnector) => + connector.enabled && (!isCustomMcp(connector) || data.allow_custom_mcp); + const setEnabled = (connector: AdminConnector, on: boolean) => + save( + isCustomMcp(connector) + ? { allow_custom_mcp: on, policies: { custom_mcp: { enabled: on } } } + : { policies: { [connector.key]: { enabled: on } } }, + ); + + const statusBadge = (connector: AdminConnector) => ( + + + {connector.configured ? 'Ready' : 'Needs setup'} + + {connector.configured && tokensOnly(connector) && ( + Tokens only + )} + + ); + + const summary = (connector: AdminConnector) => + [ + // The badge beside it says whether setup is missing; this is what + // members get. + connector.configured && enabledOf(connector) ? 'On' : 'Off', + `${fmtNumber(connector.connection_count)} ${ + connector.connection_count === 1 ? 'connection' : 'connections' + }`, + hasTools(connector) + ? POLICY_LABELS[connector.credential_mode] + : 'No tools', + ].join(' · '); + + const enabledSwitch = (connector: AdminConnector) => { + const control = ( + setEnabled(connector, checked === true)} + /> + ); + // Off until its server settings exist: switching it on would do nothing. + if (connector.configured) return control; + return ( + + + + {control} + + + Add its server settings first + + ); + }; + + const policyControl = (connector: AdminConnector, fullWidth = false) => + hasTools(connector) ? ( + + ) : ( + No tools + ); + + return ( +
+ + + {data.default_encryption_key && ( + + + + Set ENCRYPTION_SECRET_KEY before connecting services. + + + Stored credentials are encrypted with ENCRYPTION_SECRET_KEY, which + still has its public default value. Set your own, keep the old one + in ENCRYPTION_SECRET_KEY_PREVIOUS and run{' '} + + docsgpt connectors reencrypt + + . + + + )} + +
+ +
+
+ + Google Drive, SharePoint, Confluence and the GitHub App + + +
+
+ MCP servers + +
+
+
+ +
+ + {/* Phones: a list; each row opens the connector's controls. */} + + + {data.connectors.map((connector) => ( + + } + title={connector.name} + description={summary(connector)} + trailing={statusBadge(connector)} + > +
+ + {writable.length > 0 && ( +
+ + + {writable.map((connector) => ( + + + save({ + policies: { + [connector.key]: { allow_writes: checked === true }, + }, + }) + } + /> + + ))} + +
+ )} + + {detail && ( + !open && setDetailKey(null)}> + +
+
+ + {detail.name} +
+ {summary(detail)} + + + {enabledSwitch(detail)} + + + {hasTools(detail) && ( + + {policyControl(detail, true)} + + )} + {hasSetupGuide(detail) && ( + + )} +
+ + + )} + + {guide && ( + setGuide(null)} + /> + )} +
+ ); +} diff --git a/frontend/src/admin/index.tsx b/frontend/src/admin/index.tsx index 5a39bb09..296ebb51 100644 --- a/frontend/src/admin/index.tsx +++ b/frontend/src/admin/index.tsx @@ -8,6 +8,7 @@ import Admins from './Admins'; import Activity from './Activity'; import Overview from './Overview'; import Quotas from './Quotas'; +import Connectors from './Connectors'; import Usage from './Usage'; import Users from './Users'; @@ -39,6 +40,7 @@ export default function Admin() { } /> } /> } /> + } /> } /> } /> diff --git a/frontend/src/agents/ApiWriteAllowlist.test.tsx b/frontend/src/agents/ApiWriteAllowlist.test.tsx new file mode 100644 index 00000000..a484b9c8 --- /dev/null +++ b/frontend/src/agents/ApiWriteAllowlist.test.tsx @@ -0,0 +1,444 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => { + if (!opts) return key; + const params = Object.entries(opts) + .filter(([k]) => k !== 'interpolation' && k !== 'count') + .map(([k, v]) => `${k}=${v}`) + .join(','); + return params ? `${key}(${params})` : key; + }, + i18n: { language: 'en' }, + }), +})); + +const getUserTools = vi.fn(); +const getAgent = vi.fn(); +const getWorkflow = vi.fn(); +const updateAgent = vi.fn(); +vi.mock('../api/services/userService', () => ({ + default: { + getUserTools: (...args: unknown[]) => getUserTools(...args), + getAgent: (...args: unknown[]) => getAgent(...args), + getWorkflow: (...args: unknown[]) => getWorkflow(...args), + updateAgent: (...args: unknown[]) => updateAgent(...args), + }, +})); + +import actionToastReducer, { + selectActionToast, +} from '../notifications/actionToastSlice'; +import { prefSlice } from '../preferences/preferenceSlice'; +import ApiWriteAllowlist from './ApiWriteAllowlist'; +import type { Mock } from 'vitest'; + +import type { Agent, AgentConfig, ResourceState } from './types'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const TOOLS = { + tools: [ + { + id: 'tg', + displayName: 'Telegram', + connection_id: 'c1', + owner_credential_writes: ['telegram_send_message'], + actions: [ + { + name: 'telegram_send_message', + description: 'Sends a message.', + access: 'write', + active: true, + }, + { name: 'telegram_read', access: 'read', active: true }, + ], + }, + { + id: 'memory', + displayName: 'Memory', + connection_id: null, + owner_credential_writes: [], + actions: [{ name: 'memory_write', access: 'write', active: true }], + }, + { + id: 'crm', + displayName: 'CRM API', + connection_id: null, + owner_credential_writes: ['create_lead'], + actions: [], + }, + ], +}; + +// What the agent read says each tool may write on stored credentials: the +// owner's own tools, one an editor sponsored that the owner can't see, and +// one that stopped. +const state = (over: Partial): ResourceState => ({ + key: `tool:${over.id}`, + type: 'tool', + id: 'x', + name: 'Tool', + state: 'active', + reason: null, + owner_credential_writes: [], + ...over, +}); + +const STATES: ResourceState[] = [ + state({ + id: 'tg', + name: 'Telegram', + owner_credential_writes: ['telegram_send_message', 'telegram_pin_message'], + }), + state({ id: 'memory', name: 'Memory' }), + state({ + id: 'crm', + name: 'CRM API', + owner_credential_writes: ['create_lead'], + }), + state({ + id: 'bob-jira', + name: 'Bob Jira', + runs_as: { user_id: 'bob', label: 'bob@example.com' }, + owner_credential_writes: ['create_issue'], + }), + state({ + id: 'gone', + name: 'Gone', + state: 'stopped', + reason: 'deleted', + owner_credential_writes: ['delete_all'], + }), +]; + +const readAgent = (tools: string[], extra: Partial = {}) => ({ + ok: true, + json: async () => ({ + id: 'agent-1', + agent_type: 'classic', + tools, + resource_states: STATES.filter((s) => tools.includes(s.id)), + ...extra, + }), +}); + +const agent = (overrides: Partial = {}): Agent => + ({ + id: 'agent-1', + tools: ['tg', 'memory'], + config: { guardrails: { controls: [] } }, + ...overrides, + }) as unknown as Agent; + +describe('ApiWriteAllowlist', () => { + let container: HTMLDivElement; + let root: Root; + let store: ReturnType; + const makeStore = () => + configureStore({ + reducer: { + preference: prefSlice.reducer, + actionToast: actionToastReducer, + }, + }); + + beforeEach(() => { + getUserTools.mockResolvedValue({ json: async () => TOOLS }); + getAgent + .mockReset() + .mockImplementation(async () => readAgent(currentTools)); + getWorkflow.mockReset(); + updateAgent.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + let currentTools: string[] = []; + const render = async ( + a: Agent, + { + onConfigChange = vi.fn<(config: AgentConfig) => void>(), + getSavedConfig, + defaultOpen, + }: { + onConfigChange?: Mock<(config: AgentConfig) => void>; + getSavedConfig?: () => Agent['config']; + defaultOpen?: boolean; + } = {}, + ) => { + currentTools = a.tools ?? []; + store = makeStore(); + await act(async () => { + root.render( + + + , + ); + }); + return onConfigChange; + }; + + const K = 'modals.agentDetails.apiWrites'; + const disclosure = () => + Array.from(container.querySelectorAll('button')).find((b) => + b.textContent?.includes(`${K}.title`), + )!; + const expand = async () => { + if (disclosure().getAttribute('aria-expanded') === 'false') + await act(async () => disclosure().click()); + }; + const groups = () => + Array.from(container.querySelectorAll('[data-tool]')); + const groupTitles = () => + groups().map((g) => g.querySelector('h4')?.textContent); + const group = (id: string) => + container.querySelector(`[data-tool="${id}"]`)!; + const choice = (id: string, value: 'off' | 'all') => + group(id).querySelector(`[data-choice="${value}"]`)!; + const customize = async (id: string) => { + const button = Array.from(group(id).querySelectorAll('button')).find( + (b) => b.getAttribute('aria-expanded') === 'false', + ); + if (button) await act(async () => button.click()); + }; + const actionSwitch = (id: string, label: string) => { + const row = Array.from( + group(id).querySelectorAll('[data-slot="setting-row"]'), + ).find((r) => r.querySelector('label')?.textContent === label)!; + return row.querySelector('[role="switch"]')!; + }; + const summary = () => + container.querySelector('[data-testid="api-writes-summary"]')?.textContent; + const savedConfig = () => + JSON.parse( + (updateAgent.mock.calls.at(-1)![1] as FormData).get('config') as string, + ); + + it('starts folded to a summary line, and nothing is allowed by default', async () => { + await render(agent()); + expect(disclosure().getAttribute('aria-expanded')).toBe('false'); + expect(summary()).toBe(`${K}.summaryNone`); + expect(groups()).toHaveLength(0); + await expand(); + expect(disclosure().getAttribute('aria-expanded')).toBe('true'); + expect(choice('tg', 'off').getAttribute('data-state')).toBe('on'); + }); + + it('opens straight away when asked to', async () => { + await render(agent(), { defaultOpen: true }); + expect(disclosure().getAttribute('aria-expanded')).toBe('true'); + expect(groupTitles()).toEqual(['Telegram']); + }); + + it('names the tools that can make changes and counts what is allowed', async () => { + await render( + agent({ + tools: ['tg', 'crm', 'memory'], + config: { + api_write_allowlist: ['tg:telegram_send_message', 'crm:create_lead'], + }, + } as Partial), + ); + expect(summary()).toBe( + `${K}.summaryTools(tools=Telegram and CRM API) · ${K}.summaryCount(allowed=2,formatted=3)`, + ); + }); + + it("groups only writes on the owner's credentials by tool, each action under Customize", async () => { + await render(agent()); + await expand(); + expect(groupTitles()).toEqual(['Telegram']); + expect(container.textContent).not.toContain('Telegram send message'); + await customize('tg'); + const labels = Array.from( + group('tg').querySelectorAll('[data-slot="setting-row"] label'), + ).map((l) => l.textContent); + expect(labels).toEqual(['Telegram send message', 'Telegram pin message']); + expect(group('tg').textContent).toContain('Sends a message.'); + expect( + actionSwitch('tg', 'Telegram send message').getAttribute('aria-checked'), + ).toBe('false'); + }); + + it("allows all of a tool's changes at once, without dropping the rest of the config", async () => { + updateAgent.mockResolvedValue({ ok: true }); + const onConfigChange = await render(agent()); + await expand(); + await act(async () => choice('tg', 'all').click()); + const expected = { + guardrails: { controls: [] }, + api_write_allowlist: [ + 'tg:telegram_send_message', + 'tg:telegram_pin_message', + ], + }; + expect(updateAgent).toHaveBeenCalledTimes(1); + expect(savedConfig()).toEqual(expected); + expect(onConfigChange).toHaveBeenCalledWith(expected); + expect(choice('tg', 'all').getAttribute('data-state')).toBe('on'); + expect(summary()).toBe( + `${K}.summaryTools(tools=Telegram) · ${K}.summaryCount(allowed=2,formatted=2)`, + ); + }); + + it('locks the choices while a save is in flight, so saves never overlap', async () => { + let finish: (value: { ok: boolean }) => void = () => undefined; + updateAgent.mockReturnValue( + new Promise((resolve) => { + finish = resolve; + }), + ); + await render(agent()); + await expand(); + await act(async () => choice('tg', 'all').click()); + expect(choice('tg', 'off').hasAttribute('disabled')).toBe(true); + await act(async () => choice('tg', 'off').click()); + expect(updateAgent).toHaveBeenCalledTimes(1); + await act(async () => finish({ ok: true })); + expect(choice('tg', 'off').hasAttribute('disabled')).toBe(false); + }); + + it('turns a tool off again, keeping entries of tools no longer listed', async () => { + updateAgent.mockResolvedValue({ ok: true }); + await render( + agent({ + config: { + api_write_allowlist: [ + 'tg:telegram_send_message', + 'tg:telegram_pin_message', + 'old:gone_action', + ], + }, + } as Partial), + ); + await expand(); + await act(async () => choice('tg', 'off').click()); + expect(savedConfig().api_write_allowlist).toEqual(['old:gone_action']); + }); + + // Every tool has a Customize link; each names its tool to screen readers. + it('ties each Customize link to its tool', async () => { + await render(agent({ tools: ['tg', 'crm'] }), { defaultOpen: true }); + for (const [id, name] of [ + ['tg', 'Telegram'], + ['crm', 'CRM API'], + ]) { + const link = Array.from(group(id).querySelectorAll('button')).find( + (b) => b.getAttribute('aria-expanded') === 'false', + )!; + const describedBy = link.getAttribute('aria-describedby'); + expect(describedBy).toBeTruthy(); + expect(document.getElementById(describedBy!)?.textContent).toBe(name); + } + }); + + it('allows one action under Customize, leaving the tool choice mixed', async () => { + updateAgent.mockResolvedValue({ ok: true }); + await render(agent()); + await expand(); + await customize('tg'); + await act(async () => actionSwitch('tg', 'Telegram pin message').click()); + expect(savedConfig().api_write_allowlist).toEqual([ + 'tg:telegram_pin_message', + ]); + expect( + actionSwitch('tg', 'Telegram pin message').getAttribute('aria-checked'), + ).toBe('true'); + expect(choice('tg', 'off').getAttribute('data-state')).toBe('off'); + expect(choice('tg', 'all').getAttribute('data-state')).toBe('off'); + }); + + it('saves on top of the last saved config, not unsaved form edits', async () => { + updateAgent.mockResolvedValue({ ok: true }); + const saved = { guardrails: { controls: [] } } as unknown as NonNullable< + Agent['config'] + >; + const draft = agent({ + config: { guardrails: { controls: [{ id: 'unsaved' }] } }, + } as unknown as Partial); + const onConfigChange = await render(draft, { + getSavedConfig: () => saved, + }); + await expand(); + await act(async () => choice('tg', 'all').click()); + const expected = { + guardrails: { controls: [] }, + api_write_allowlist: [ + 'tg:telegram_send_message', + 'tg:telegram_pin_message', + ], + }; + expect(savedConfig()).toEqual(expected); + expect(onConfigChange).toHaveBeenCalledWith(expected); + }); + + it('puts the choice back and says so when saving fails', async () => { + updateAgent.mockResolvedValue({ ok: false }); + await render(agent()); + await expand(); + await act(async () => choice('tg', 'all').click()); + expect(choice('tg', 'off').getAttribute('data-state')).toBe('on'); + expect(summary()).toBe(`${K}.summaryNone`); + expect(selectActionToast(store.getState())?.variant).toBe('destructive'); + }); + + it('lists writes on stored credentials of tools without a connection', async () => { + await render(agent({ tools: ['crm'] }), { defaultOpen: true }); + expect(groupTitles()).toEqual(['CRM API']); + await customize('crm'); + expect( + group('crm').querySelector('[data-slot="setting-row"] label') + ?.textContent, + ).toBe('Create lead'); + }); + + it("lists a sponsor's tool the owner can't see, but not a stopped one", async () => { + await render(agent({ tools: ['bob-jira', 'gone'] }), { + defaultOpen: true, + }); + expect(groupTitles()).toEqual(['Bob Jira']); + }); + + it("lists writes of a workflow agent's node tools", async () => { + getAgent.mockResolvedValue( + readAgent([], { agent_type: 'workflow', workflow: 'w1' }), + ); + getWorkflow.mockResolvedValue({ + ok: true, + json: async () => ({ + data: { + resource_states: [ + state({ + id: 'node-tool', + name: 'Node Slack', + owner_credential_writes: ['post_message'], + }), + ], + }, + }), + }); + await render(agent({ tools: [] }), { defaultOpen: true }); + expect(groupTitles()).toEqual(['Node Slack']); + }); + + it('renders nothing for an agent without connected tools', async () => { + await render(agent({ tools: ['memory'] })); + expect(container.textContent).toBe(''); + }); +}); diff --git a/frontend/src/agents/ApiWriteAllowlist.tsx b/frontend/src/agents/ApiWriteAllowlist.tsx new file mode 100644 index 00000000..138ecff0 --- /dev/null +++ b/frontend/src/agents/ApiWriteAllowlist.tsx @@ -0,0 +1,354 @@ +import { ChevronRight } from 'lucide-react'; +import { useEffect, useId, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; + +import userService from '../api/services/userService'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { SectionHeader } from '../components/ui/section-header'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import { ToggleGroup, ToggleGroupItem } from '../components/ui/toggle-group'; +import { actionTitle } from '../connectors/i18n'; +import { cn } from '../lib/utils'; +import { showActionToast } from '../notifications/actionToastSlice'; +import { selectToken } from '../preferences/preferenceSlice'; +import { formatCount, intlLocale } from '../utils/dateTimeUtils'; +import type { Agent, AgentConfig } from './types'; +import useAgentResourceStates from './useAgentResourceStates'; + +type WriteAction = { + entry: string; + action: string; + description: string; +}; + +/** One running tool and its writes on the owner's credentials. */ +type ToolWrites = { + id: string; + name: string; + actions: WriteAction[]; +}; + +type UserTool = { + id: string; + actions?: { name: string; description?: string }[]; +}; + +/** The tool-wide choice: every write off, every write on, or mixed (''). */ +type ToolChoice = 'off' | 'all' | ''; + +const K = 'modals.agentDetails.apiWrites'; + +/** + * One tool's writes: an Off / All choice for all of them at once, and each + * write on its own switch under Customize, like a connector's permissions. + */ +function ToolAllowlist({ + tool, + allowed, + onChange, + disabled = false, +}: { + tool: ToolWrites; + allowed: Set; + onChange: (entries: string[], allow: boolean) => void; + /** While a save is in flight, so saves never overlap. */ + disabled?: boolean; +}) { + const { t } = useTranslation(); + const [unfolded, setUnfolded] = useState(false); + // Names the tool for its Customize link, which reads the same in each. + const titleId = useId(); + const allowedCount = tool.actions.filter((item) => + allowed.has(item.entry), + ).length; + const choice: ToolChoice = + allowedCount === 0 + ? 'off' + : allowedCount === tool.actions.length + ? 'all' + : ''; + const plain = { interpolation: { escapeValue: false } }; + return ( +
+
+ {tool.name}} + description={t(`${K}.toolCount`, { + allowed: formatCount(allowedCount), + formatted: formatCount(tool.actions.length), + })} + /> +
+ + value && + onChange( + tool.actions.map((item) => item.entry), + value === 'all', + ) + } + > + {(['off', 'all'] as const).map((value) => ( + + {t(`${K}.choice.${value}`)} + + ))} + +
+
+ + {unfolded && ( + + {tool.actions.map((item) => { + const id = `api-write-${item.entry}`; + return ( + {actionTitle(item.action)} + } + description={item.description || undefined} + > + onChange([item.entry], checked)} + /> + + ); + })} + + )} +
+ ); +} + +/** + * The write actions on the owner's accounts and stored credentials that + * anyone reaching this agent through its API key, widget or its public link + * may run. Nobody there can approve an action for the owner, so + * the server refuses every other such write. The server names these writes + * per running tool in the agent's `resource_states` (`owner_credential_writes`), + * which covers tools an editor sponsored and a workflow agent's node tools + * too; the owner's own tool list only adds action descriptions. + * + * The section starts folded to a one-line summary (which tools can make + * changes, and how many of all the writes are allowed). Open, each tool has + * one Off / All choice with its writes one by one under Customize. Entries + * are `tool_id:action`. + * + * A change saves at once, on top of the agent's last saved config + * (`getSavedConfig`), so edits still pending in the form are not saved with + * it. `onConfigChange` receives the config as saved. + */ +export default function ApiWriteAllowlist({ + agent, + onConfigChange, + getSavedConfig, + defaultOpen = false, +}: { + agent: Agent; + onConfigChange?: (config: AgentConfig) => void; + getSavedConfig?: () => AgentConfig | undefined; + /** Start open (Access details opened to allow changes). */ + defaultOpen?: boolean; +}) { + const { t, i18n } = useTranslation(); + const dispatch = useDispatch(); + const token = useSelector(selectToken); + const [open, setOpen] = useState(defaultOpen); + // One save at a time: overlapping saves can land out of order, and a + // failed one would put back a list that drops the later choice. + const [saving, setSaving] = useState(false); + + const [descriptions, setDescriptions] = useState>({}); + const [allowed, setAllowed] = useState( + agent.config?.api_write_allowlist ?? [], + ); + const loaded = useAgentResourceStates(agent.id, (agent.tools ?? []).join()); + + useEffect(() => { + setAllowed(agent.config?.api_write_allowlist ?? []); + }, [agent.config?.api_write_allowlist]); + + // Descriptions of the actions of tools the owner can open themselves. + useEffect(() => { + let cancelled = false; + userService + .getUserTools(token) + .then((response: Response) => response.json()) + .then((data: { tools?: UserTool[] }) => { + if (cancelled) return; + const found: Record = {}; + for (const tool of data.tools ?? []) + for (const action of tool.actions ?? []) + if (action.description) + found[`${tool.id}:${action.name}`] = action.description; + setDescriptions(found); + }) + .catch(() => undefined); + return () => { + cancelled = true; + }; + }, [token]); + + const tools: ToolWrites[] = (loaded?.items ?? []) + .filter( + (item) => + item.type === 'tool' && + item.state === 'active' && + (item.owner_credential_writes ?? []).length > 0, + ) + .map((item) => ({ + id: item.id, + name: item.name || t('agents.form.sponsors.unknownItem'), + actions: (item.owner_credential_writes ?? []).map((name) => ({ + entry: `${item.id}:${name}`, + action: name, + description: descriptions[`${item.id}:${name}`] ?? '', + })), + })); + + if (tools.length === 0) return null; + + const allowedSet = new Set(allowed); + const total = tools.reduce((sum, tool) => sum + tool.actions.length, 0); + const allowedCount = tools.reduce( + (sum, tool) => + sum + tool.actions.filter((item) => allowedSet.has(item.entry)).length, + 0, + ); + const changing = tools + .filter((tool) => tool.actions.some((item) => allowedSet.has(item.entry))) + .map((tool) => tool.name); + const summary = + changing.length === 0 + ? t(`${K}.summaryNone`) + : [ + t(`${K}.summaryTools`, { + count: changing.length, + tools: new Intl.ListFormat(intlLocale(i18n.language), { + type: 'conjunction', + }).format(changing), + interpolation: { escapeValue: false }, + }), + t(`${K}.summaryCount`, { + allowed: formatCount(allowedCount), + formatted: formatCount(total), + }), + ].join(' · '); + + /** Allow or refuse these entries, and save the list. */ + const change = async (entries: string[], allow: boolean) => { + if (saving) return; + setSaving(true); + const previous = allowed; + const next = allow + ? [...allowed, ...entries.filter((entry) => !allowed.includes(entry))] + : allowed.filter((entry) => !entries.includes(entry)); + setAllowed(next); + const config: AgentConfig = { + ...((getSavedConfig ? getSavedConfig() : agent.config) ?? {}), + api_write_allowlist: next, + }; + const form = new FormData(); + form.append('config', JSON.stringify(config)); + try { + const response = await userService.updateAgent( + agent.id ?? '', + form, + token, + ); + if (!response.ok) throw new Error('save failed'); + onConfigChange?.(config); + } catch { + setAllowed(previous); + dispatch( + showActionToast({ + variant: 'destructive', + message: t(`${K}.saveFailed`), + }), + ); + } finally { + setSaving(false); + } + }; + + return ( +
+ {/* A collapsible group in a modal: the inline disclosure toggle. */} +
+ +

+ {summary} +

+
+ {open && ( + <> +

+ {t(`${K}.description`)} +

+ + {tools.map((tool) => ( + + ))} + + + )} +
+ ); +} diff --git a/frontend/src/agents/NewAgent.test.tsx b/frontend/src/agents/NewAgent.test.tsx index 0132d64b..b022dc96 100644 --- a/frontend/src/agents/NewAgent.test.tsx +++ b/frontend/src/agents/NewAgent.test.tsx @@ -5,7 +5,10 @@ import { MemoryRouter, Route, Routes } from 'react-router-dom'; import type { MultiSelectPopoverItem } from '../components/MultiSelectPopover'; vi.mock('react-i18next', () => ({ - useTranslation: () => ({ t: (key: string) => key }), + useTranslation: () => ({ + t: (key: string) => key, + i18n: { language: 'en' }, + }), })); const mockState = { @@ -27,12 +30,25 @@ const mocks = vi.hoisted(() => { dispatch: vi.fn(), getAgent: vi.fn(() => jsonResponse({})), createAgent: vi.fn(() => jsonResponse({ message: 'Name is taken' }, false)), + tools: null as unknown[] | null, + connections: [] as unknown[], + catalog: [] as unknown[], deleteAgent: vi.fn(() => jsonResponse({})), + updateAgent: vi.fn< + (id: string, data: FormData, token: string | null) => Promise + >(() => jsonResponse({})), guardrailsProps: vi.fn(), + detailsProps: vi.fn(), + shareProps: vi.fn(), + reconnect: vi.fn(), }; }); const { jsonResponse } = mocks; +vi.mock('../connectors/SignInAgainNotice', () => ({ + default: () => null, + useSignInAgain: () => ({ reconnect: mocks.reconnect, modals: null }), +})); vi.mock('react-redux', () => ({ useSelector: (selector: (state: unknown) => unknown) => selector(mockState), useDispatch: () => mocks.dispatch, @@ -42,7 +58,7 @@ vi.mock('../api/services/userService', () => ({ default: { getUserTools: () => jsonResponse({ - tools: [ + tools: mocks.tools ?? [ { id: 'tool-1', name: 'remote_device', @@ -54,7 +70,7 @@ vi.mock('../api/services/userService', () => ({ getAgentFolders: () => jsonResponse({ folders: [] }), getAgent: mocks.getAgent, createAgent: mocks.createAgent, - updateAgent: () => jsonResponse({}), + updateAgent: mocks.updateAgent, deleteAgent: mocks.deleteAgent, createPrompt: () => jsonResponse({}), }, @@ -69,6 +85,14 @@ vi.mock('../api/services/devicesService', () => ({ }, })); +vi.mock('../api/services/connectorsService', () => ({ + default: { + listConnections: () => Promise.resolve({ connections: mocks.connections }), + getCatalog: () => + Promise.resolve({ success: true, connectors: mocks.catalog }), + }, +})); + vi.mock('../api/services/modelService', () => ({ default: { getModels: () => jsonResponse({ models: [] }), @@ -82,14 +106,23 @@ vi.mock('../components/MultiSelectPopover', () => ({ MultiSelectPopover: ({ trigger, items, + onToggle, }: { trigger: React.ReactNode; items: MultiSelectPopoverItem[]; + onToggle: (id: string) => void; }) => (
{trigger} {items.map((item) => ( -
{item.descriptionNode}
+
+ {item.descriptionNode} +
))}
), @@ -106,8 +139,18 @@ vi.mock('./components/GuardrailsSection', () => ({ guardrailsIncomplete: () => false, })); vi.mock('../upload/Upload', () => ({ default: () => null })); -vi.mock('../modals/AgentDetailsModal', () => ({ default: () => null })); -vi.mock('../teams/ShareToTeamModal', () => ({ default: () => null })); +vi.mock('../modals/AgentDetailsModal', () => ({ + default: (props: unknown) => { + mocks.detailsProps(props); + return null; + }, +})); +vi.mock('../teams/ShareToTeamModal', () => ({ + default: (props: unknown) => { + mocks.shareProps(props); + return null; + }, +})); vi.mock('../modals/ConfirmationModal', () => ({ default: ({ modalState, @@ -123,6 +166,24 @@ vi.mock('../modals/ConfirmationModal', () => ({ ) : null, })); vi.mock('../preferences/PromptsModal', () => ({ default: () => null })); +vi.mock('./components/SponsorConfirmModal', () => ({ + default: ({ + confirmation, + onConfirm, + }: { + confirmation: { resources: { key: string; name: string }[] } | null; + onConfirm: (keys: string[]) => void; + }) => + confirmation ? ( + + ) : null, +})); vi.mock('../navigation/SectionPills', () => ({ default: () =>
, })); @@ -176,6 +237,11 @@ describe('NewAgent form', () => { mocks.dispatch.mockClear(); mocks.getAgent.mockClear(); mocks.createAgent.mockClear(); + mocks.updateAgent.mockReset(); + mocks.updateAgent.mockImplementation(() => jsonResponse({})); + mocks.tools = null; + mocks.connections = []; + mocks.catalog = []; }); const render = async () => { @@ -263,6 +329,312 @@ describe('NewAgent form', () => { expect(description.className).toContain('sm:col-start-2'); }); + it('lists tool groups as built-in, default, one per connection, then custom', async () => { + mocks.tools = [ + { id: 'custom', name: 'api_tool', display_name: 'My API' }, + { + id: 'linear', + name: 'mcp_tool', + display_name: 'Linear', + connection_id: 'c-lin', + }, + { id: 'memory', name: 'memory', display_name: 'Memory', builtin: true }, + { + id: 'notion', + name: 'mcp_tool', + display_name: 'Notion', + connection_id: 'c-not', + }, + { + id: 'reader', + name: 'read_webpage', + display_name: 'Reader', + default: true, + }, + ]; + mocks.connections = [ + { id: 'c-lin', name: 'Linear', account_label: 'a@x', icon: 'linear' }, + { id: 'c-not', name: 'Notion', account_label: 'b@x', icon: 'notion' }, + ]; + await render(); + const groups = Array.from( + container.querySelectorAll('[data-testid="picker"] [data-group]'), + ).map((item) => item.getAttribute('data-group')); + const order = groups.filter((g, i) => groups.indexOf(g) === i); + expect(order).toEqual([ + 'agents.form.toolsPopup.groupBuiltin', + 'agents.form.toolsPopup.groupDefault', + 'agents.form.toolsPopup.groupConnection', + 'agents.form.toolsPopup.groupCustom', + ]); + }); + + // A teammate's connection is never in the caller's list; its tool still + // belongs with the services, named from the catalog. + it("groups a teammate's connected tool under its service, before custom", async () => { + mocks.tools = [ + { id: 'custom', name: 'api_tool', display_name: 'My API' }, + { + id: 'shared-tg', + name: 'telegram', + displayName: 'Telegram', + connection_id: 'owner-conn', + access: 'viewer', + allowed_actions: ['use', 'use_in_own'], + }, + ]; + mocks.catalog = [ + { + key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + publisher: 'built_in', + tool_templates: ['telegram'], + }, + ]; + await render(); + const groups = Array.from( + container.querySelectorAll('[data-testid="picker"] [data-group]'), + ).map((item) => item.getAttribute('data-group')); + expect(groups).toEqual(['Telegram', 'agents.form.toolsPopup.groupCustom']); + }); + + const renderEdit = async (agent: Record) => { + mocks.getAgent.mockImplementationOnce(() => + jsonResponse({ + id: 'agent-1', + name: 'Shared', + description: 'd', + status: 'published', + agent_type: 'classic', + prompt_id: 'default', + access: 'editor', + allowed_actions: ['edit', 'view'], + ...agent, + }), + ); + await act(async () => { + root.render( + + + } + /> + + , + ); + }); + }; + + // An editor doesn't list the owner's private tools, but must still be able + // to take one off the agent. + it("adds a remove-only row for an attached tool the editor can't list", async () => { + mocks.tools = [{ id: 'mine', name: 'api_tool', display_name: 'My API' }]; + await renderEdit({ + tools: ['owners'], + tool_details: [ + { id: 'owners', name: 'jira', display_name: 'Owner Jira' }, + ], + }); + const toolPicker = Array.from( + container.querySelectorAll('[data-testid="picker"]'), + ).find((picker) => picker.textContent?.includes('Owner Jira'))!; + const groups = Array.from(toolPicker.querySelectorAll('[data-group]')).map( + (item) => item.getAttribute('data-group'), + ); + expect(groups).toEqual([ + 'agents.form.toolsPopup.groupCustom', + 'agents.form.toolsPopup.groupAttached', + ]); + }); + + it('asks before sponsoring and retries the save with the confirmation', async () => { + mocks.updateAgent.mockImplementationOnce(() => + Promise.resolve( + new Response( + JSON.stringify({ + success: false, + code: 'sponsor_confirmation_required', + message: 'confirm', + resources: [ + { key: 'tool:t1', type: 'tool', id: 't1', name: 'Jira' }, + ], + audience: { + teams: ['Support'], + api_key: true, + public_link: false, + webhook: false, + }, + }), + { status: 409 }, + ), + ), + ); + await renderEdit({}); + const name = container.querySelector( + 'input[placeholder="agents.form.placeholders.agentName"]', + )!; + await act(async () => setNativeValue(name, 'Renamed')); + await act(async () => buttonByText('agents.form.buttons.save').click()); + const confirm = container.querySelector( + '[data-testid="confirm-sponsor"]', + )!; + expect(confirm.textContent).toBe('Jira'); + // The refusal is a question, not an error. + expect(container.querySelector('[data-variant="destructive"]')).toBeNull(); + expect(mocks.updateAgent).toHaveBeenCalledTimes(1); + const first = mocks.updateAgent.mock.calls[0][1]; + expect(first.get('confirm_sponsor')).toBeNull(); + + await act(async () => confirm.click()); + expect(mocks.updateAgent).toHaveBeenCalledTimes(2); + const retried = mocks.updateAgent.mock.calls[1][1]; + expect(JSON.parse(retried.get('confirm_sponsor') as string)).toEqual([ + 'tool:t1', + ]); + expect( + container.querySelector('[data-testid="confirm-sponsor"]'), + ).toBeNull(); + }); + + const rename = async (value: string) => { + const name = container.querySelector( + 'input[placeholder="agents.form.placeholders.agentName"]', + )!; + await act(async () => setNativeValue(name, value)); + }; + + it('drops the remove-only row once the removal is saved', async () => { + mocks.tools = [{ id: 'mine', name: 'api_tool', display_name: 'My API' }]; + await renderEdit({ + tools: ['owners'], + tool_details: [ + { id: 'owners', name: 'jira', display_name: 'Owner Jira' }, + ], + }); + await act(async () => + container + .querySelector('[data-toggle="owners"]')! + .click(), + ); + // Still listed until saved, so it can be put back. + expect(container.querySelector('[data-item="owners"]')).not.toBeNull(); + await act(async () => buttonByText('agents.form.buttons.save').click()); + expect(mocks.updateAgent).toHaveBeenCalledTimes(1); + expect(container.querySelector('[data-item="owners"]')).toBeNull(); + }); + + const stoppedTool = { + key: 'tool:t1', + type: 'tool', + id: 't1', + name: 'Jira', + state: 'stopped', + reason: 'sponsor_cannot_edit_agent', + sponsor: { user_id: 'bob', label: 'bob@example.com' }, + can_confirm: true, + }; + const withStoppedTool = (item: Record = {}) => ({ + tools: ['t1'], + tool_details: [{ id: 't1', name: 'jira', display_name: 'Jira' }], + resource_states: [{ ...stoppedTool, ...item }], + sponsor_audience: { + teams: ['Sales'], + api_key: false, + public_link: false, + webhook: false, + }, + }); + + it('asks before taking over a stopped item and sends it with the next save', async () => { + await renderEdit(withStoppedTool()); + const save = buttonByText('agents.form.buttons.save'); + expect(save.disabled).toBe(true); + await act(async () => + buttonByText('agents.form.sponsors.takeOver').click(), + ); + // Nothing is taken over until the reader agrees in the dialog. + expect(save.disabled).toBe(true); + const confirm = container.querySelector( + '[data-testid="confirm-sponsor"]', + )!; + expect(confirm.textContent).toBe('Jira'); + await act(async () => confirm.click()); + expect(container.textContent).toContain( + 'agents.form.sponsors.takeOverPending', + ); + expect(save.disabled).toBe(false); + await act(async () => save.click()); + const sent = mocks.updateAgent.mock.calls[0][1]; + expect(JSON.parse(sent.get('confirm_sponsor') as string)).toEqual([ + 'tool:t1', + ]); + // The saved agent is fetched again for its fresh details. + expect(mocks.getAgent).toHaveBeenCalledTimes(2); + }); + + it('removes a stopped item from the agent', async () => { + await renderEdit(withStoppedTool({ reason: 'deleted', sponsor: null })); + expect(container.textContent).toContain( + 'agents.form.resourceStates.reason.deleted', + ); + await act(async () => + buttonByText('agents.form.resourceStates.remove').click(), + ); + // Off the form, so off the notice too. + expect(container.textContent).not.toContain( + 'agents.form.resourceStates.reason.deleted', + ); + await act(async () => buttonByText('agents.form.buttons.save').click()); + const sent = mocks.updateAgent.mock.calls[0][1]; + expect(JSON.parse(sent.get('tools') as string)).toEqual([]); + }); + + it('reconnects the account of a stopped tool in place', async () => { + mocks.reconnect.mockClear(); + await renderEdit( + withStoppedTool({ + reason: 'connection_needs_reconnect', + sponsor: null, + can_confirm: false, + can_reconnect: true, + connection: { id: 'c1', connector_key: 'telegram', name: 'Telegram' }, + }), + ); + await act(async () => + buttonByText('settings.connectors.status.reconnect').click(), + ); + expect(mocks.reconnect).toHaveBeenCalledWith( + { id: 'c1', connector_key: 'telegram' }, + undefined, + ); + }); + + it('explains an outdated confirmation and reloads the sponsors', async () => { + mocks.updateAgent.mockImplementationOnce(() => + Promise.resolve( + new Response( + JSON.stringify({ + success: false, + code: 'sponsor_confirmation_unexpected', + message: 'raw english', + unexpected: ['tool:t1'], + }), + { status: 400 }, + ), + ), + ); + await renderEdit({}); + await rename('Renamed'); + await act(async () => buttonByText('agents.form.buttons.save').click()); + expect(container.textContent).toContain( + 'agents.form.sponsors.confirmationOutdated', + ); + expect(container.textContent).not.toContain('raw english'); + expect(mocks.getAgent).toHaveBeenCalledTimes(2); + }); + it('labels every picker with a floating label', async () => { await render(); const labels = Array.from( @@ -700,6 +1072,67 @@ describe('NewAgent gating by role', () => { expect(token.disabled).toBe(true); }); + it('keeps the form clean after the API write allowlist saves', async () => { + await renderEdit('owner', OWNER); + const details = () => + mocks.detailsProps.mock.calls.at(-1)![0] as { + onConfigChange: (config: Record) => void; + getSavedConfig: () => Record | undefined; + }; + const saved = { + ...details().getSavedConfig(), + api_write_allowlist: ['tool-1:send'], + }; + await act(async () => details().onConfigChange(saved)); + expect(details().getSavedConfig()).toEqual(saved); + expect(buttonByText('agents.form.buttons.cancel')).toBeUndefined(); + }); + + // "What this agent uses" in Share sends the owner to the API write + // allowlist, which opens unfolded; opened any other way it starts folded. + it('opens Access details on the allowlist from Share', async () => { + await renderEdit('owner', OWNER); + const details = () => + mocks.detailsProps.mock.calls.at(-1)![0] as { + modalState: string; + openApiWrites?: boolean; + setModalState: (state: string) => void; + }; + await menuLabels(); + await act(async () => + Array.from(document.querySelectorAll('[role="menuitem"]')) + .find((item) => item.textContent === 'agents.shareWithTeam')! + .click(), + ); + const share = mocks.shareProps.mock.calls.at(-1)![0] as { + onOpenAccessDetails?: () => void; + }; + mocks.shareProps.mockClear(); + await act(async () => share.onOpenAccessDetails!()); + expect(mocks.shareProps).not.toHaveBeenCalled(); + expect(details().modalState).toBe('ACTIVE'); + expect(details().openApiWrites).toBe(true); + await act(async () => details().setModalState('INACTIVE')); + expect(details().openApiWrites).toBe(false); + }); + + it('gives Share no way to Access details without manage_access_details', async () => { + await renderEdit( + 'owner', + OWNER.filter((a) => a !== 'manage_access_details'), + ); + await menuLabels(); + await act(async () => + Array.from(document.querySelectorAll('[role="menuitem"]')) + .find((item) => item.textContent === 'agents.shareWithTeam')! + .click(), + ); + const share = mocks.shareProps.mock.calls.at(-1)![0] as { + onOpenAccessDetails?: () => void; + }; + expect(share.onOpenAccessDetails).toBeUndefined(); + }); + it('hides Access details without manage_access_details', async () => { await renderEdit( 'editor', diff --git a/frontend/src/agents/NewAgent.tsx b/frontend/src/agents/NewAgent.tsx index d5ef020f..c79c91e8 100644 --- a/frontend/src/agents/NewAgent.tsx +++ b/frontend/src/agents/NewAgent.tsx @@ -52,6 +52,14 @@ import { } from '../components/MultiSelectPopover'; import SourcesPopoverFooter from '../components/SourcesPopoverFooter'; import ToolIcon from '../components/ToolIcon'; +import connectorsService from '../api/services/connectorsService'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { connectionNeedsSignIn } from '../connectors/connectorsSlice'; +import SignInAgainNotice, { + useSignInAgain, +} from '../connectors/SignInAgainNotice'; +import { toolServiceOf } from '../connectors/toolService'; +import type { Connection, ConnectorDefinition } from '../connectors/types'; import AgentDetailsModal from '../modals/AgentDetailsModal'; import ShareToTeamModal from '../teams/ShareToTeamModal'; import ConfirmationModal from '../modals/ConfirmationModal'; @@ -91,8 +99,19 @@ import { resetPreview, selectPreviewStatus } from './agentPreviewSlice'; import AgentPageToolbar, { LastUsedMeta } from './components/AgentPageToolbar'; import AgentPreviewSheet from './components/AgentPreviewSheet'; import SectionShell from '../navigation/SectionShell'; -import SponsoredResourcesNotice from './components/SponsoredResourcesNotice'; -import { Agent, ResourceSponsor, ToolSummary } from './types'; +import ResourceStatusNotice, { + type NamedResource, + unnamedResourceLabel, +} from './components/ResourceStatusNotice'; +import { + confirmTakeOver, + readSponsorRefusal, + saveWithSponsorConsent, + sponsorNotAllowedMessage, + withAttachedToolRows, +} from './sponsorConsent'; +import { useSponsorPrompt } from './useSponsorPrompt'; +import { Agent, ResourceState, ToolSummary } from './types'; import WorkflowBuilder from './workflow/WorkflowBuilder'; import type { Model } from '../models/types'; @@ -121,7 +140,7 @@ const extractApiError = async ( }; export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { - const { t } = useTranslation(); + const { t, i18n } = useTranslation(); const navigate = useNavigate(); const dispatch = useDispatch(); const { agentId } = useParams(); @@ -165,6 +184,19 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { const [imageFile, setImageFile] = useState(null); const [userTools, setUserTools] = useState([]); const [rawUserTools, setRawUserTools] = useState([]); + // Connections behind the picker's tools that need signing in again. + const [brokenToolConnections, setBrokenToolConnections] = useState< + { connection: Connection; mcpToolId?: string }[] + >([]); + const [toolsReloadKey, setToolsReloadKey] = useState(0); + // Bumped after a reconnect so the agent's run state is read again. + const [detailsReloadKey, setDetailsReloadKey] = useState(0); + const signInAgain = useSignInAgain({ + onConnected: () => { + setToolsReloadKey((key) => key + 1); + setDetailsReloadKey((key) => key + 1); + }, + }); const [availableModels, setAvailableModels] = useState([]); const [isSourcePopupOpen, setIsSourcePopupOpen] = useState(false); const [isToolsPopupOpen, setIsToolsPopupOpen] = useState(false); @@ -175,12 +207,22 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { new Set(), ); const [selectedTools, setSelectedTools] = useState([]); + // Tools on the agent when it loaded: the owner's private ones don't come + // back in the caller's own tool list, so the picker adds a row for each. + const [attachedTools, setAttachedTools] = useState([]); + // Asks before a save makes what the caller added run with their access. + const sponsorPrompt = useSponsorPrompt(); + // Keys of stopped items the caller chose to keep running with their + // access; sent as ``confirm_sponsor`` with the next save. + const [takeovers, setTakeovers] = useState([]); const [selectedModelIds, setSelectedModelIds] = useState>( new Set(), ); const [deleteConfirmation, setDeleteConfirmation] = useState('INACTIVE'); const [agentDetails, setAgentDetails] = useState('INACTIVE'); + // Access details opened from Share to allow changes: its allowlist unfolds. + const [detailsOnApiWrites, setDetailsOnApiWrites] = useState(false); const [shareModalOpen, setShareModalOpen] = useState(false); const [addPromptModal, setAddPromptModal] = useState('INACTIVE'); const [hasChanges, setHasChanges] = useState(false); @@ -286,19 +328,18 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { // Name of a tool/source/prompt that runs with an editor's access, from the // same owner-agnostic details the pickers show. const resolveSponsoredName = useCallback( - (sponsor: ResourceSponsor): string => { + (sponsor: NamedResource): string => { + if (sponsor.name) return sponsor.name; if (sponsor.type === 'source') return resolveSourceLabel(sponsor.id); if (sponsor.type === 'prompt') { return ( prompts.find((prompt) => prompt.id === sponsor.id)?.name || agent.prompt_name || - t('agents.form.sponsors.unknownItem') + unnamedResourceLabel(t, sponsor) ); } const tool = selectedTools.find((item) => item.id === sponsor.id); - return tool - ? getToolDisplayName(tool) - : t('agents.form.sponsors.unknownItem'); + return tool ? getToolDisplayName(tool) : unnamedResourceLabel(t, sponsor); }, [agent.prompt_name, prompts, resolveSourceLabel, selectedTools, t], ); @@ -322,6 +363,17 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { return [...items, ...unlisted]; }, [resolveSourceLabel, selectedSourceIds, sourceDocs, t]); + // The caller's tools, plus a remove-only row for each tool on the agent + // they can't list (the owner's private tools on a shared agent). + const toolItems = useMemo( + () => + withAttachedToolRows(userTools, attachedTools, { + group: t('agents.form.toolsPopup.groupAttached'), + description: t('agents.form.toolsPopup.attachedHint'), + }), + [attachedTools, t, userTools], + ); + const selectedSourceNames = useMemo( () => Array.from(selectedSourceIds) @@ -405,6 +457,156 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { } }; + // Fresh sponsor details and run state after a save or a reconnect, + // without touching unsaved fields. + const refreshSponsors = useCallback( + async (id?: string) => { + if (!id) return; + try { + const response = await userService.getAgent(id, token); + if (!response.ok) return; + const data = await response.json(); + const details: Partial = { + resource_sponsors: data.resource_sponsors ?? [], + resource_states: data.resource_states ?? [], + sponsor_audience: data.sponsor_audience, + }; + setAgent((prev) => ({ ...prev, ...details })); + if (initialAgentRef.current) + initialAgentRef.current = { ...initialAgentRef.current, ...details }; + } catch { + // The notice keeps what it showed. + } + }, + [token], + ); + + useEffect(() => { + if (detailsReloadKey > 0) void refreshSponsors(agent.id); + // Only a reconnect asks for this; agent.id is read when it does. + }, [detailsReloadKey, refreshSponsors]); + + /** The message for a refused save. */ + const saveFailureMessage = async ( + response: Response, + fallback: string, + ): Promise => { + const refusal = await readSponsorRefusal(response); + if (refusal?.kind === 'notAllowed') + return sponsorNotAllowedMessage(t, i18n.language, refusal.resources); + if (refusal?.kind === 'unexpected') { + // Someone changed the agent since the caller chose; show them now. + setTakeovers([]); + void refreshSponsors(agent.id); + return t('agents.form.sponsors.confirmationOutdated'); + } + return extractApiError(response, fallback); + }; + + /** + * Create or update the agent, asking whenever the server wants the + * caller's confirmation to run what they added with their access. + * Resolves to null when they decline. + */ + const sendAgent = (formData: FormData) => + saveWithSponsorConsent( + (confirm) => { + if (confirm.length > 0) + formData.set('confirm_sponsor', JSON.stringify(confirm)); + else formData.delete('confirm_sponsor'); + return effectiveMode === 'new' + ? userService.createAgent(formData, token) + : userService.updateAgent(agent.id || '', formData, token); + }, + sponsorPrompt.ask, + takeovers, + ); + + /** Bookkeeping after an update the server accepted. */ + const afterSaved = (id?: string) => { + // What is on the agent now is what the tool picker keeps rows for. + setAttachedTools(selectedTools); + if (effectiveMode !== 'new') { + setTakeovers([]); + void refreshSponsors(id); + } + }; + + const undoTakeover = (key: string) => + setTakeovers((prev) => prev.filter((k) => k !== key)); + + // Stopped items still on the agent in the form: one removed here (not yet + // saved) leaves the notice. + const stoppedResources = useMemo(() => { + const toolIds = new Set( + selectedTools.map((tool) => String(tool?.id).toLowerCase()), + ); + const sourceIds = new Set( + Array.from(selectedSourceIds, (id) => id.toLowerCase()), + ); + const promptId = String(agent.prompt_id || '').toLowerCase(); + return (agent.resource_states ?? []).filter( + (item) => + item.state === 'stopped' && + (item.type === 'tool' + ? toolIds.has(item.id) + : item.type === 'source' + ? sourceIds.has(item.id) + : promptId === item.id), + ); + }, [ + agent.resource_states, + agent.prompt_id, + selectedSourceIds, + selectedTools, + ]); + + /** Take a stopped item off the agent; the next save stores it. */ + const removeResource = (item: ResourceState) => { + if (item.type === 'tool') + setSelectedTools((prev) => + prev.filter((tool) => String(tool?.id).toLowerCase() !== item.id), + ); + else if (item.type === 'source') + setSelectedSourceIds( + (prev) => + new Set( + Array.from(prev).filter((id) => id.toLowerCase() !== item.id), + ), + ); + else setAgent((prev) => ({ ...prev, prompt_id: 'default' })); + undoTakeover(item.key); + }; + + /** + * Ask before a stopped item runs with the caller's access, naming who + * reaches it through the agent; on yes the next save confirms it. + */ + const takeOverResource = async (item: ResourceState) => { + const agreed = await confirmTakeOver( + sponsorPrompt.ask, + item, + resolveSponsoredName(item), + agent.sponsor_audience, + ); + if (agreed) + setTakeovers((prev) => + prev.includes(item.key) ? prev : [...prev, item.key], + ); + }; + + /** Sign a stopped tool's connection in again, in place where possible. */ + const reconnectResource = (item: ResourceState) => { + const connection = item.connection; + if (!connection?.id || !connection.connector_key) return; + const isMcp = + rawUserTools.find((tool) => tool.id === item.id)?.name === 'mcp_tool'; + signInAgain.reconnect( + { id: connection.id, connector_key: connection.connector_key }, + isMcp ? item.id : undefined, + ); + }; + const handleSaveDraft = async () => { const formData = new FormData(); formData.append('name', agent.name); @@ -467,13 +669,11 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { try { setDraftLoading(true); setSubmitError(null); - const response = - effectiveMode === 'new' - ? await userService.createAgent(formData, token) - : await userService.updateAgent(agent.id || '', formData, token); + const response = await sendAgent(formData); + if (!response) return; if (!response.ok) { setSubmitError( - await extractApiError( + await saveFailureMessage( response, t('agents.form.errors.saveDraftFailed'), ), @@ -481,6 +681,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { return; } const data = await response.json(); + afterSaved(data.id || agent.id); const updatedAgent = { ...agent, @@ -560,13 +761,11 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { try { setPublishLoading(true); setSubmitError(null); - const response = - effectiveMode === 'new' - ? await userService.createAgent(formData, token) - : await userService.updateAgent(agent.id || '', formData, token); + const response = await sendAgent(formData); + if (!response) return; if (!response.ok) { setSubmitError( - await extractApiError( + await saveFailureMessage( response, t('agents.form.errors.publishFailed'), ), @@ -584,6 +783,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { }; setAgent(updatedAgent); initialAgentRef.current = updatedAgent; + afterSaved(updatedAgent.id); // The saved agent is what the preview talks to; start its chat over. dispatch(resetPreview()); @@ -618,12 +818,26 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { useEffect(() => { const getTools = async () => { - const [toolsResponse, devicesResult] = await Promise.all([ - userService.getUserTools(token), - // Tolerate failures here: the picker should still render the - // tool list even if /api/devices returns an error or 401. - devicesService.list(token).catch(() => ({ devices: [] })), - ]); + const [toolsResponse, devicesResult, connectionsResult, catalogResult] = + await Promise.all([ + userService.getUserTools(token), + // Tolerate failures here: the picker should still render the + // tool list even if /api/devices returns an error or 401. + devicesService.list(token).catch(() => ({ devices: [] })), + connectorsService + .listConnections(token) + .catch(() => ({ connections: [] })), + // Names a teammate's connected tool, whose connection the caller + // never sees. + connectorsService.getCatalog(token).catch(() => ({ connectors: [] })), + ]); + const ownConnections = (connectionsResult?.connections ?? + []) as Connection[]; + const catalog = (catalogResult?.connectors ?? + []) as ConnectorDefinition[]; + const connectionsById = new Map( + ownConnections.map((c) => [c.id, c]), + ); if (!toolsResponse.ok) throw new Error('Failed to fetch tools'); const data = await toolsResponse.json(); // Hide workflow-only builtins (e.g. read_document) from the classic @@ -642,21 +856,54 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { !Number.isNaN(seen) && Date.now() - seen < onlineWindowMs; devicesById.set(d.id, { online, last_seen_at: d.last_seen_at }); }); - // Group ordering: builtins -> defaults -> user tools (sorted via the - // MultiSelectPopover first-appearance grouping). + // Group ordering: builtins -> defaults -> one group per connection + // (the service and its account; a teammate's, only the service) -> + // custom tools, via the MultiSelectPopover first-appearance grouping. + const serviceOf = (tool: UserToolType) => + toolServiceOf(tool, ownConnections, catalog); + const connectionOf = (tool: UserToolType) => serviceOf(tool)?.connection; + const rank = (tool: UserToolType) => + tool.builtin ? 0 : tool.default ? 1 : tool.connection_id ? 2 : 3; const groupFor = (tool: UserToolType): string => { if (tool.builtin) return t('agents.form.toolsPopup.groupBuiltin'); if (tool.default) return t('agents.form.toolsPopup.groupDefault'); + const service = serviceOf(tool); + if (service?.connection) + return t('agents.form.toolsPopup.groupConnection', { + name: service.connection.name, + account: service.connection.account_label, + interpolation: { escapeValue: false }, + }); + if (service) return service.name; return t('agents.form.toolsPopup.groupCustom'); }; - const tools: MultiSelectPopoverItem[] = visibleTools.map( - (tool: UserToolType) => { + const tools: MultiSelectPopoverItem[] = [...visibleTools] + .sort( + (a, b) => + rank(a) - rank(b) || + // Keeps each connection's tools together. + (rank(a) === 2 ? groupFor(a).localeCompare(groupFor(b)) : 0), + ) + .map((tool: UserToolType) => { + const connection = connectionOf(tool); + const serviceIcon = serviceOf(tool)?.icon; const base: MultiSelectPopoverItem = { id: tool.id, label: getToolDisplayName(tool), - icon: , + icon: serviceIcon ? ( + + ) : ( + + ), group: groupFor(tool), }; + if (connectionNeedsSignIn(connection)) { + base.descriptionNode = ( +

+ {t('settings.connectors.health.signInAgain')} +

+ ); + } if (tool.name === 'remote_device') { const deviceId = (tool.config?.device_id as string) || ''; const meta = devicesById.get(deviceId); @@ -673,19 +920,25 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { ); } return base; - }, - ); - const groupOrder = [ - t('agents.form.toolsPopup.groupBuiltin'), - t('agents.form.toolsPopup.groupDefault'), - t('agents.form.toolsPopup.groupCustom'), - ]; - tools.sort( - (a, b) => - groupOrder.indexOf(a.group || '') - groupOrder.indexOf(b.group || ''), - ); + }); setUserTools(tools); setRawUserTools(visibleTools); + setBrokenToolConnections( + Array.from(connectionsById.values()) + .filter(connectionNeedsSignIn) + .flatMap((connection) => { + const own = visibleTools.filter( + (tool) => tool.connection_id === connection.id, + ); + if (own.length === 0) return []; + return [ + { + connection, + mcpToolId: own.find((tool) => tool.name === 'mcp_tool')?.id, + }, + ]; + }), + ); }; const getModels = async () => { const response = await modelService.getModels(token); @@ -710,7 +963,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { }; getTools(); getModels(); - }, [token, mode]); + }, [token, mode, toolsReloadKey]); // Validate folder_id from URL against user's folders useEffect(() => { @@ -755,7 +1008,10 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { const agentSourceIds = selectedSourceIdsFromAgent(data); setSelectedSourceIds(new Set(agentSourceIds)); - if (data.tool_details) setSelectedTools(data.tool_details); + if (data.tool_details) { + setSelectedTools(data.tool_details); + setAttachedTools(data.tool_details); + } if (data.status === 'draft') setEffectiveMode('draft'); if (data.json_schema) { const jsonText = JSON.stringify(data.json_schema, null, 2); @@ -865,9 +1121,10 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { const isChanged = !isEqual(agent, initialAgentRef.current) || imageFile !== null || - jsonSchemaText !== initialJsonSchemaText; + jsonSchemaText !== initialJsonSchemaText || + takeovers.length > 0; setHasChanges(isChanged); - }, [agent, dispatch, effectiveMode, imageFile, jsonSchemaText]); + }, [agent, dispatch, effectiveMode, imageFile, jsonSchemaText, takeovers]); const isPublished = agent.status === 'published'; // What the caller's role allows on this agent (`allowed_actions` from the @@ -882,6 +1139,9 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { // Page-level actions live in the ⋯ beside the title. Until the agent is // published the preview can only say "Publish to preview", so Preview is a // menu item then and a toolbar button after. + const canOpenAccessDetails = + modeConfig[effectiveMode].showAccessDetails && + can(agent, 'manage_access_details'); const menuOptions = [ ...(isPublished ? [] @@ -892,8 +1152,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { onClick: () => setPreviewOpen(true), }, ]), - ...(modeConfig[effectiveMode].showAccessDetails && - can(agent, 'manage_access_details') + ...(canOpenAccessDetails ? [ { label: t('agents.form.buttons.accessDetails'), @@ -981,6 +1240,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { /> } > + {signInAgain.modals} {agent.agent_type === 'workflow' && } tool.id)} onToggle={(id) => { const exists = selectedTools.find((t) => t.id === id); @@ -1125,12 +1385,13 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { setSelectedTools(selectedTools.filter((t) => t.id !== id)); return; } - const item = userTools.find((t) => t.id === id); + const item = toolItems.find((t) => t.id === id); const raw = rawUserTools.find((t) => t.id === id); + const attached = attachedTools.find((t) => t.id === id); if (!item) return; setSelectedTools([ ...selectedTools, - { + attached ?? { id: item.id, name: raw?.name || item.label, display_name: item.label, @@ -1141,6 +1402,24 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { 'agents.form.toolsPopup.searchPlaceholder', )} emptyMessage={t('agents.form.toolsPopup.noOptionsMessage')} + footer={ + brokenToolConnections.length > 0 ? ( + connection, + )} + onReconnect={(connection) => { + setIsToolsPopupOpen(false); + signInAgain.reconnect( + connection, + brokenToolConnections.find( + (entry) => entry.connection.id === connection.id, + )?.mcpToolId, + ); + }} + /> + ) : undefined + } trigger={
- void takeOverResource(item)} + onUndoTakeover={undoTakeover} + onRemove={removeResource} + onReconnect={reconnectResource} /> @@ -1550,6 +1835,7 @@ export default function NewAgent({ mode }: { mode: 'new' | 'edit' | 'draft' }) { )} + {sponsorPrompt.modal} { + setAgentDetails(state); + if (state === 'INACTIVE') setDetailsOnApiWrites(false); + }} + openApiWrites={detailsOnApiWrites} onKeyRegenerated={(key) => setAgent((prev) => ({ ...prev, key }))} + onConfigChange={(config) => { + // The allowlist is saved already: record it on the saved snapshot + // too, and keep any unsaved form edits to the rest of the config. + if (initialAgentRef.current) + initialAgentRef.current = { ...initialAgentRef.current, config }; + setAgent((prev) => ({ + ...prev, + config: { + ...(prev.config ?? {}), + api_write_allowlist: config.api_write_allowlist, + }, + })); + }} + getSavedConfig={() => initialAgentRef.current?.config ?? agent.config} /> {shareModalOpen && agent.id && ( setShareModalOpen(false)} + onOpenAccessDetails={ + canOpenAccessDetails + ? () => { + setShareModalOpen(false); + setDetailsOnApiWrites(true); + setAgentDetails('ACTIVE'); + } + : undefined + } /> )} {uploadModalState === 'ACTIVE' && ( diff --git a/frontend/src/agents/components/AgentUsesSection.test.tsx b/frontend/src/agents/components/AgentUsesSection.test.tsx new file mode 100644 index 00000000..d68c3692 --- /dev/null +++ b/frontend/src/agents/components/AgentUsesSection.test.tsx @@ -0,0 +1,593 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('react-redux', () => ({ + useSelector: (selector: (s: unknown) => unknown) => + selector({ preference: { token: 'tok' } }), +})); + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => { + if (!opts) return key; + const params = Object.entries(opts) + .filter(([k]) => k !== 'defaultValue' && k !== 'interpolation') + .map(([k, v]) => `${k}=${v}`) + .join(','); + return params ? `${key}(${params})` : key; + }, + i18n: { language: 'en' }, + }), +})); + +const getAgent = vi.fn(); +const getWorkflow = vi.fn(); +vi.mock('../../api/services/userService', () => ({ + default: { + getAgent: (...a: unknown[]) => getAgent(...a), + getWorkflow: (...a: unknown[]) => getWorkflow(...a), + }, +})); + +import type { Agent, ResourceState } from '../types'; +import AgentUsesSection from './AgentUsesSection'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const K = 'settings.teams.share.uses'; + +const ok = (body: unknown) => ({ ok: true, json: async () => body }); + +const item = (over: Partial): ResourceState => ({ + key: `${over.type ?? 'tool'}:${over.id ?? 't1'}`, + type: 'tool', + id: 't1', + name: 'Item', + state: 'active', + reason: null, + ...over, +}); + +const agentWith = ( + states: ResourceState[], + over: Partial = {}, +): Agent => ({ + id: 'a1', + name: 'A', + description: '', + image: '', + source: '', + chunks: '6', + retriever: '', + prompt_id: '', + tools: [], + agent_type: 'classic', + status: 'published', + access: 'owner', + allowed_actions: ['edit', 'share', 'use'], + resource_states: states, + ...over, +}); + +const flush = async () => { + for (let i = 0; i < 6; i += 1) { + await act(async () => { + await Promise.resolve(); + }); + } +}; + +describe('AgentUsesSection', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + getAgent.mockReset(); + getWorkflow.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(() => { + act(() => root.unmount()); + container.remove(); + }); + + const render = async ( + agent: Agent | null, + readerId = 'me', + onOpenAccessDetails?: () => void, + ) => { + getAgent.mockResolvedValue(agent ? ok(agent) : { ok: false }); + act(() => { + root.render( + , + ); + }); + await flush(); + }; + + const toggle = () => + Array.from(container.querySelectorAll('button')).find((b) => + b.textContent?.includes(`${K}.title`), + ); + + const open = async () => { + const button = toggle(); + if (button?.getAttribute('aria-expanded') === 'false') { + act(() => button.click()); + await flush(); + } + }; + + const rowOf = (name: string) => + Array.from(container.querySelectorAll('li')).find((li) => + li.textContent?.includes(name), + ); + + it('starts collapsed and lists what the agent uses when opened', async () => { + await render(agentWith([item({ name: 'Docs', type: 'source', id: 's1' })])); + expect(toggle()?.getAttribute('aria-expanded')).toBe('false'); + expect(rowOf('Docs')).toBeUndefined(); + await open(); + expect(toggle()?.getAttribute('aria-expanded')).toBe('true'); + expect(rowOf('Docs')?.textContent).toContain(`${K}.access.you`); + }); + + it('names whose access each item runs with, for the owner', async () => { + await render( + agentWith([ + item({ id: 't1', name: 'Mine' }), + item({ + id: 't2', + name: 'Bobs', + runs_as: { user_id: 'bob', label: 'bob@example.com' }, + }), + item({ + id: 't3', + name: 'Slack tool', + credential_mode: 'member', + note: 'per_user_account', + connection: { id: 'c1', connector_key: 'slack', name: 'Slack' }, + }), + item({ + id: 't4', + name: 'Notion tool', + credential_mode: 'owner', + account: { user_id: 'me', label: 'me@example.com' }, + connection: { id: 'c2', connector_key: 'notion', name: 'Notion' }, + }), + item({ + id: 't5', + name: 'Jira tool', + credential_mode: 'owner', + account: { user_id: 'carol', label: 'carol@example.com' }, + connection: { id: 'c3', connector_key: 'jira', name: 'Jira' }, + }), + item({ type: 'prompt', id: 'p1', name: 'Tone' }), + ]), + ); + await open(); + expect(rowOf('Mine')?.textContent).toContain(`${K}.access.you`); + expect(rowOf('Bobs')?.textContent).toContain( + `${K}.access.person(person=bob@example.com)`, + ); + // API and widget callers run it as the owner: the owner's account. + expect(rowOf('Slack tool')?.textContent).toContain( + `${K}.access.member(service=Slack)`, + ); + expect(rowOf('Notion tool')?.textContent).toContain( + `${K}.access.yourAccount(service=Notion)`, + ); + expect(rowOf('Jira tool')?.textContent).toContain( + `${K}.access.personAccount(service=Jira,person=carol@example.com)`, + ); + expect(rowOf('Tone')?.textContent).toContain(`${K}.access.you`); + }); + + it("says the owner's access to an editor, and theirs where they sponsor", async () => { + await render( + agentWith( + [ + item({ id: 't1', name: 'Owners' }), + item({ + id: 't2', + name: 'Editors', + runs_as: { user_id: 'me', label: 'me@example.com' }, + }), + item({ + id: 't3', + name: 'Slack tool', + credential_mode: 'member', + note: 'per_user_account', + connection: { id: null, connector_key: 'slack', name: 'Slack' }, + }), + ], + { access: 'editor', allowed_actions: ['edit', 'share', 'use'] }, + ), + ); + await open(); + expect(rowOf('Owners')?.textContent).toContain(`${K}.access.owner`); + expect(rowOf('Editors')?.textContent).toContain(`${K}.access.you`); + expect(rowOf('Slack tool')?.textContent).toContain( + `${K}.access.memberShared(service=Slack)`, + ); + }); + + it('follows who it runs as now, not a sponsor on record', async () => { + await render( + agentWith([ + item({ + id: 't1', + name: 'Once sponsored', + sponsor: { user_id: 'bob', label: 'bob@example.com' }, + runs_as: null, + }), + ]), + ); + await open(); + expect(rowOf('Once sponsored')?.textContent).toContain(`${K}.access.you`); + expect(rowOf('Once sponsored')?.textContent).not.toContain('bob'); + }); + + it('says someone else for a sponsor the reader does not know', async () => { + await render( + agentWith( + [ + item({ + id: 't1', + name: 'Runs as a stranger', + sponsor: { user_id: null, label: null }, + runs_as: { user_id: null, label: null }, + }), + item({ + id: 't2', + name: 'Stopped', + state: 'stopped', + reason: 'sponsor_cannot_edit_agent', + sponsor: { user_id: null, label: null }, + }), + ], + { access: 'editor', allowed_actions: ['edit', 'share', 'use'] }, + ), + ); + await open(); + expect(rowOf('Runs as a stranger')?.textContent).toContain( + `${K}.access.other`, + ); + expect(rowOf('Stopped')?.textContent).toContain( + 'agents.form.resourceStates.reason.sponsorCannotEditAgentOther', + ); + }); + + it('reads the per_user_account note for a tool each person connects', async () => { + await render( + agentWith([ + item({ + id: 't1', + name: 'Slack tool', + credential_mode: 'member', + note: 'per_user_account', + connection: { id: 'c1', connector_key: 'slack', name: 'Slack' }, + }), + ]), + ); + await open(); + expect(rowOf('Slack tool')?.textContent).toContain( + `${K}.access.member(service=Slack)`, + ); + }); + + it('names whose saved credentials a tool without a connection uses', async () => { + await render( + agentWith([ + item({ + id: 't1', + name: 'My API', + account: { user_id: 'me', label: 'me@example.com' }, + }), + item({ + id: 't2', + name: 'Carols API', + account: { user_id: 'carol', label: 'carol@example.com' }, + }), + item({ + id: 't3', + name: 'Hidden API', + account: { user_id: null, label: null }, + }), + item({ + id: 't4', + name: 'Hidden Jira', + credential_mode: 'owner', + account: { user_id: null, label: null }, + connection: { id: null, connector_key: 'jira', name: 'Jira' }, + }), + ]), + ); + await open(); + expect(rowOf('My API')?.textContent).toContain( + `${K}.access.yourCredentials`, + ); + expect(rowOf('Carols API')?.textContent).toContain( + `${K}.access.personCredentials(person=carol@example.com)`, + ); + expect(rowOf('Hidden API')?.textContent).toContain( + `${K}.access.otherCredentials`, + ); + expect(rowOf('Hidden Jira')?.textContent).toContain( + `${K}.access.otherAccount(service=Jira)`, + ); + }); + + it('opens by itself and marks a stopped item with why it stopped', async () => { + await render( + agentWith([ + item({ id: 't1', name: 'Gone', state: 'stopped', reason: 'deleted' }), + item({ id: 't2', name: 'Fine' }), + ]), + ); + expect(toggle()?.getAttribute('aria-expanded')).toBe('true'); + const gone = rowOf('Gone'); + expect(gone?.textContent).toContain(`${K}.stopped`); + expect(gone?.textContent).toContain( + 'agents.form.resourceStates.reason.deleted(name=Gone', + ); + expect(gone?.textContent).not.toContain(`${K}.access.you`); + expect(rowOf('Fine')?.textContent).not.toContain(`${K}.stopped`); + }); + + it('flags writes outside callers cannot make until they are allowed', async () => { + await render( + agentWith( + [ + item({ + id: 't1', + name: 'Blocked', + owner_credential_writes: ['send', 'delete'], + }), + item({ + id: 't2', + name: 'Allowed', + owner_credential_writes: ['send'], + }), + item({ id: 't3', name: 'Reads', owner_credential_writes: [] }), + ], + { config: { api_write_allowlist: ['t1:send', 't2:send'] } }, + ), + ); + await open(); + // One of two writes still blocked: some, not all. + expect(rowOf('Blocked')?.textContent).toContain(`${K}.writesSomeOff`); + expect(rowOf('Allowed')?.textContent).not.toContain(`${K}.writes`); + expect(rowOf('Reads')?.textContent).not.toContain(`${K}.writes`); + const alert = container.querySelector('[data-slot="alert"]'); + expect(alert?.textContent).toContain(`${K}.writesNote`); + expect(alert?.textContent).not.toContain(`${K}.writesNoteMemberTail`); + }); + + it('marks a tool with every write blocked', async () => { + await render( + agentWith([ + item({ id: 't1', name: 'Blocked', owner_credential_writes: ['a'] }), + ]), + ); + await open(); + expect(rowOf('Blocked')?.textContent).toContain(`${K}.writesOff`); + expect(rowOf('Blocked')?.textContent).not.toContain(`${K}.writesSomeOff`); + }); + + it('names only API and widget users for tools each person connects', async () => { + await render( + agentWith([ + item({ + id: 't1', + name: 'Slack tool', + credential_mode: 'member', + note: 'per_user_account', + connection: { id: null, connector_key: 'slack', name: 'Slack' }, + owner_credential_writes: ['send'], + }), + ]), + ); + await open(); + const alert = container.querySelector('[data-slot="alert"]'); + expect(alert?.textContent).toBe(`${K}.writesNoteApi`); + }); + + it('says public-link users use their own account on mixed tools', async () => { + await render( + agentWith([ + item({ id: 't1', name: 'Mine', owner_credential_writes: ['a'] }), + item({ + id: 't2', + name: 'Slack tool', + credential_mode: 'member', + note: 'per_user_account', + owner_credential_writes: ['send'], + }), + ]), + ); + await open(); + const alert = container.querySelector('[data-slot="alert"]'); + expect(alert?.textContent).toBe( + `${K}.writesNote ${K}.writesNoteMemberTail`, + ); + }); + + it('says when an admin turned changes off', async () => { + await render( + agentWith([ + item({ + id: 't1', + name: 'GitHub tool', + writes_allowed: false, + owner_credential_writes: [], + }), + ]), + ); + await open(); + expect(rowOf('GitHub tool')?.textContent).toContain(`${K}.adminOff`); + expect(container.querySelector('[data-slot="alert"]')).toBeNull(); + }); + + it('tells an editor the owner allows the writes', async () => { + await render( + agentWith( + [item({ id: 't1', name: 'Blocked', owner_credential_writes: ['x'] })], + { access: 'editor', allowed_actions: ['edit', 'share'] }, + ), + ); + await open(); + expect( + container.querySelector('[data-slot="alert"]')?.textContent, + ).toContain(`${K}.writesNoteEditor`); + }); + + it('shows no writes note when every write is allowed', async () => { + await render( + agentWith([ + item({ id: 't1', name: 'Plain', owner_credential_writes: [] }), + ]), + ); + await open(); + expect(container.querySelector('[data-slot="alert"]')).toBeNull(); + }); + + // "Allow" on a connector is the in-chat permission; the badge is about + // the agent's own API write allowlist, so only that list clears it. + it('drops the badge and the note once the write is on the allowlist', async () => { + const linear = item({ + id: 'lin', + name: 'Linear', + connection: { id: 'c1', connector_key: 'mcp:linear', name: 'Linear' }, + credential_mode: 'owner', + owner_credential_writes: ['create_issue'], + }); + await render(agentWith([linear])); + await open(); + expect(rowOf('Linear')?.textContent).toContain(`${K}.writesOff`); + + act(() => root.unmount()); + root = createRoot(container); + await render( + agentWith([linear], { + config: { api_write_allowlist: ['LIN:create_issue'] }, + }), + ); + await open(); + expect(rowOf('Linear')?.textContent).not.toContain(`${K}.writes`); + expect(container.querySelector('[data-slot="alert"]')).toBeNull(); + }); + + it('opens Access details from the note for the owner', async () => { + const openDetails = vi.fn(); + await render( + agentWith([ + item({ id: 't1', name: 'Blocked', owner_credential_writes: ['x'] }), + ]), + 'me', + openDetails, + ); + await open(); + const button = Array.from( + container.querySelectorAll( + '[data-slot="alert"] button', + ), + ).find((b) => b.textContent === `${K}.openAccessDetails`); + expect(button).toBeDefined(); + act(() => button!.click()); + expect(openDetails).toHaveBeenCalledTimes(1); + }); + + // Access details lists the writes only once the agent has an API key, so + // without one the note says to create it there first. + it('says the allowlist needs an API key when the agent has none', async () => { + const blocked = [ + item({ id: 't1', name: 'Blocked', owner_credential_writes: ['x'] }), + ]; + await render(agentWith(blocked), 'me', vi.fn()); + await open(); + const alert = () => container.querySelector('[data-slot="alert"]'); + expect(alert()?.textContent).toContain(`${K}.accessDetailsNeedsKey`); + + act(() => root.unmount()); + root = createRoot(container); + await render(agentWith(blocked, { key: 'abcd...wxyz' }), 'me', vi.fn()); + await open(); + expect(alert()?.textContent).not.toContain(`${K}.accessDetailsNeedsKey`); + expect(alert()?.textContent).toContain(`${K}.openAccessDetails`); + }); + + it('offers no Access details button to an editor or without a way there', async () => { + const blocked = [ + item({ id: 't1', name: 'Blocked', owner_credential_writes: ['x'] }), + ]; + await render( + agentWith(blocked, { access: 'editor', allowed_actions: ['edit'] }), + 'me', + vi.fn(), + ); + await open(); + expect(container.querySelector('[data-slot="alert"] button')).toBeNull(); + + act(() => root.unmount()); + root = createRoot(container); + await render(agentWith(blocked)); + await open(); + expect(container.querySelector('[data-slot="alert"] button')).toBeNull(); + }); + + it('includes the resources of a workflow agent’s nodes', async () => { + getWorkflow.mockResolvedValue( + ok({ + success: true, + data: { + resource_states: [ + item({ id: 'n1', name: 'Node tool' }), + item({ id: 't1', name: 'Shared tool' }), + ], + }, + }), + ); + await render( + agentWith([item({ id: 't1', name: 'Shared tool' })], { + agent_type: 'workflow', + workflow: 'w1', + }), + ); + await open(); + expect(getWorkflow).toHaveBeenCalledWith('w1', 'tok'); + expect(rowOf('Node tool')).toBeDefined(); + expect( + Array.from(container.querySelectorAll('li')).filter((li) => + li.textContent?.includes('Shared tool'), + ), + ).toHaveLength(1); + }); + + it('renders nothing for someone who may not edit the agent', async () => { + await render( + agentWith([item({ name: 'Hidden' })], { + access: 'viewer', + allowed_actions: ['use'], + }), + ); + expect(container.innerHTML).toBe(''); + }); + + it('renders nothing when the agent uses nothing listed', async () => { + await render(agentWith([])); + expect(container.innerHTML).toBe(''); + }); + + it('renders nothing when the agent fails to load', async () => { + await render(null); + expect(container.innerHTML).toBe(''); + }); +}); diff --git a/frontend/src/agents/components/AgentUsesSection.tsx b/frontend/src/agents/components/AgentUsesSection.tsx new file mode 100644 index 00000000..0b5a8868 --- /dev/null +++ b/frontend/src/agents/components/AgentUsesSection.tsx @@ -0,0 +1,259 @@ +import { + ChevronRight, + Database, + ScrollText, + TriangleAlert, + Wrench, +} from 'lucide-react'; +import { useState } from 'react'; +import { useTranslation } from 'react-i18next'; + +import { Alert, AlertDescription } from '@/components/ui/alert'; +import { Badge } from '@/components/ui/badge'; +import { Button } from '@/components/ui/button'; +import { Card } from '@/components/ui/card'; +import { ListRow, ListRows } from '@/components/ui/list-row'; +import { cn } from '@/lib/utils'; + +import { can, isOwner } from '../../utils/accessUtils'; +import type { ResourceState } from '../types'; +import useAgentResourceStates from '../useAgentResourceStates'; +import { reasonKey } from './ResourceStatusNotice'; + +type AgentUsesSectionProps = { + /** The agent the share dialog is for. */ + agentId: string; + /** The reader's user id, to say "your" for what runs as them. */ + readerId?: string; + /** + * Opens the agent's Access details, where its owner allows changes + * through the API, widget and public links. Without it the note only + * says where to go. + */ + onOpenAccessDetails?: () => void; +}; + +const TYPE_ICONS: Record = { + tool: Wrench, + source: Database, + prompt: ScrollText, +}; + +const K = 'settings.teams.share.uses'; +const plain = { interpolation: { escapeValue: false } }; + +/** Write actions outside callers can't take: not in the API write allowlist. */ +function blockedWrites(item: ResourceState, allowlist: string[]): string[] { + const allowed = new Set(allowlist.map((entry) => entry.toLowerCase())); + return (item.owner_credential_writes ?? []).filter( + (action) => !allowed.has(`${item.id}:${action}`.toLowerCase()), + ); +} + +/** + * "What this agent uses" in the agent's share dialog: each attached tool, + * source and prompt (and a workflow agent's node tools and sources), with + * whose access, account or saved credentials it runs with for the people + * the agent is shared with. + * + * Built from `resource_states` on the agent read (and the workflow read), + * which only owners and editors get; the section is hidden from anyone else, + * and while it loads or when it fails. A stopped item says why. A tool with + * writes on stored credentials that aren't in the API write allowlist is + * marked (all or some of them), since API and widget users, and public-link + * users on the owner's accounts, can't make those changes; so is a tool an + * admin allows no changes through. That allowlist is not the connector's + * Allow (the in-chat permission), so the note says so and, for the owner, + * opens Access details where the list is set. + */ +export default function AgentUsesSection({ + agentId, + readerId, + onOpenAccessDetails, +}: AgentUsesSectionProps) { + const { t } = useTranslation(); + const loaded = useAgentResourceStates(agentId); + // Like Access settings: open once by itself when something stopped, then + // follow the reader's clicks. + const [open, setOpen] = useState(null); + + if (!loaded || !can(loaded.agent, 'edit') || loaded.items.length === 0) + return null; + + const { agent, items } = loaded; + const expanded = open ?? items.some((item) => item.state === 'stopped'); + const ownerReads = isOwner(agent); + // Without a user id (authentication off) the one local user is everyone. + const isYou = (userId: string) => + readerId ? userId === readerId : ownerReads; + // A tool each person connects runs on their own account (the notice's + // note), except for API and widget users. + const perUser = (item: ResourceState) => item.note === 'per_user_account'; + const allowlist = agent.config?.api_write_allowlist ?? []; + const nameOf = (item: ResourceState) => + item.name || t('agents.form.sponsors.unknownItem'); + + // Whose account or saved credentials a tool acts with. + const credentialsLabel = (item: ResourceState): string | null => { + const service = item.connection?.name; + const suffix = service ? '' : 'NoService'; + const named = service ? { ...plain, service } : plain; + if (perUser(item)) + // API and widget callers run the agent as its owner, so they use the + // owner's own account. + return t( + `${K}.access.member${ownerReads ? '' : 'Shared'}${suffix}`, + named, + ); + const account = item.account; + if (!account) return null; + // A connection names its service; saved credentials (an API key, an + // MCP sign-in) have none. + const connected = item.credential_mode === 'owner'; + const key = connected ? `Account${suffix}` : 'Credentials'; + const opts = connected ? named : plain; + if (!account.user_id) return t(`${K}.access.other${key}`, opts); + if (isYou(account.user_id)) return t(`${K}.access.your${key}`, opts); + return t(`${K}.access.person${key}`, { + ...opts, + person: account.label || account.user_id, + }); + }; + + const accessLabel = (item: ResourceState): string => { + const credentials = credentialsLabel(item); + if (credentials) return credentials; + if (item.runs_as) { + const { user_id: userId, label } = item.runs_as; + if (!userId) return t(`${K}.access.other`); + return isYou(userId) + ? t(`${K}.access.you`) + : t(`${K}.access.person`, { ...plain, person: label || userId }); + } + return ownerReads ? t(`${K}.access.you`) : t(`${K}.access.owner`); + }; + + const stoppedText = (item: ResourceState) => + t(reasonKey(item.reason, ownerReads, Boolean(item.sponsor?.user_id)), { + ...plain, + name: nameOf(item), + service: + item.connection?.name || + t('agents.form.resourceStates.serviceFallback'), + person: item.sponsor?.label || item.sponsor?.user_id, + }); + + const badgeFor = (item: ResourceState) => { + if (item.state === 'stopped') + return {t(`${K}.stopped`)}; + if (item.writes_allowed === false) + return {t(`${K}.adminOff`)}; + const blocked = blockedWrites(item, allowlist).length; + if (blocked === 0) return null; + const all = blocked === (item.owner_credential_writes ?? []).length; + return ( + + {t(all ? `${K}.writesOff` : `${K}.writesSomeOff`)} + + ); + }; + + // Public-link users act with their own account on a tool each person + // connects, so only API and widget users are held back there. + const blocked = items.filter( + (item) => + item.state === 'active' && + item.writes_allowed !== false && + blockedWrites(item, allowlist).length > 0, + ); + const blockedMember = blocked.some(perUser); + const blockedOwned = blocked.some((item) => !perUser(item)); + const editor = ownerReads ? '' : 'Editor'; + const writesNote = blockedOwned + ? [ + t(`${K}.writesNote${editor}`), + blockedMember ? t(`${K}.writesNoteMemberTail`) : '', + ] + .filter(Boolean) + .join(' ') + : blockedMember + ? t(`${K}.writesNoteApi${editor}`) + : null; + + return ( +
+ {/* The same inline disclosure as Access settings. */} + + {expanded && ( + <> +

{t(`${K}.intro`)}

+ + + {items.map((item) => { + const Icon = TYPE_ICONS[item.type] ?? Wrench; + const description = + item.state === 'stopped' + ? stoppedText(item) + : accessLabel(item); + return ( + + + + {writesNote && ( + + + {writesNote} + {/* Only the owner sets the allowlist. */} + {ownerReads && onOpenAccessDetails && ( +
+ {/* Access details lists these writes once there is a key + (a draft gets one when it is published). */} + {!agent.key && ( +

{t(`${K}.accessDetailsNeedsKey`)}

+ )} + +
+ )} + + )} + + )} +
+ ); +} diff --git a/frontend/src/agents/components/ResourceStatusNotice.test.tsx b/frontend/src/agents/components/ResourceStatusNotice.test.tsx new file mode 100644 index 00000000..a548b76d --- /dev/null +++ b/frontend/src/agents/components/ResourceStatusNotice.test.tsx @@ -0,0 +1,395 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +import i18n from 'i18next'; +import { initReactI18next } from 'react-i18next'; + +import type { Agent, ResourceSponsor, ResourceState } from '../types'; +import ResourceStatusNotice, { + unnamedResourceLabel, +} from './ResourceStatusNotice'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const baseAgent: Agent = { + name: 'A', + description: 'd', + image: '', + source: '', + chunks: '6', + retriever: '', + prompt_id: 'default', + tools: [], + agent_type: 'classic', + status: 'published', +}; + +const editorAgent: Agent = { + ...baseAgent, + access: 'editor', + allowed_actions: ['edit'], +}; + +const sponsor = (over: Partial): ResourceSponsor => ({ + type: 'tool', + id: 't1', + user_id: 'bob', + label: 'bob@example.com', + active: true, + ...over, +}); + +const stoppedItem = (over: Partial): ResourceState => ({ + key: 'tool:t1', + type: 'tool', + id: 't1', + state: 'stopped', + reason: 'deleted', + ...over, +}); + +describe('ResourceStatusNotice', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const handlers = () => ({ + onTakeOver: vi.fn(), + onUndoTakeover: vi.fn(), + onRemove: vi.fn(), + onReconnect: vi.fn(), + }); + + const render = async ( + agent: Agent, + stopped: ResourceState[] = [], + takeovers: string[] = [], + ) => { + const spies = handlers(); + await act(async () => { + root.render( + item.name || `name-${item.id}`} + takeovers={takeovers} + {...spies} + />, + ); + }); + return spies; + }; + + const buttons = () => Array.from(container.querySelectorAll('button')); + const buttonWith = (text: string) => + buttons().filter((b) => b.textContent?.includes(text)); + const alerts = () => + Array.from(container.querySelectorAll('[data-slot="alert"]')); + + it('renders nothing for the owner when everything runs', async () => { + await render(baseAgent); + expect(container.innerHTML).toBe(''); + }); + + it('tells an editor their own items run with their access', async () => { + await render(editorAgent); + expect(alerts()).toHaveLength(1); + expect(container.textContent).toContain('agents.form.sponsors.attachNote'); + expect(container.textContent).not.toContain('publicLinkNote'); + }); + + it('adds the public-link warning when the agent has a link', async () => { + await render({ ...editorAgent, shared: true }); + expect(container.textContent).toContain( + 'agents.form.sponsors.publicLinkNote', + ); + }); + + it('leaves the attach note out where the page asks it to', async () => { + const spies = handlers(); + await act(async () => { + root.render( + item.id} + showAttachNote={false} + {...spies} + />, + ); + }); + expect(container.innerHTML).toBe(''); + }); + + it('does not show the attach note to a viewer', async () => { + await render({ ...baseAgent, access: 'viewer', allowed_actions: ['use'] }); + expect(container.innerHTML).toBe(''); + }); + + it('joins names for the app language and does not escape them', async () => { + await i18n.use(initReactI18next).init({ + lng: 'jp', + resources: { + jp: { + translation: { + agents: { + form: { sponsors: { addedBy: '{{person}}: {{names}}' } }, + }, + }, + }, + }, + }); + try { + await render({ + ...baseAgent, + resource_sponsors: [ + sponsor({ id: 'docs/a' }), + sponsor({ id: 'docs/b' }), + ], + }); + const expected = new Intl.ListFormat('ja', { + type: 'conjunction', + }).format(['name-docs/a', 'name-docs/b']); + expect(container.textContent).toBe(`bob@example.com: ${expected}`); + } finally { + await i18n.changeLanguage('en'); + } + }); + + it('lists running sponsored items apart from stopped ones, in one notice', async () => { + await render( + { + ...baseAgent, + resource_sponsors: [ + sponsor({ id: 't1' }), + // Stopped sponsorships come through resource_states instead. + sponsor({ id: 's1', type: 'source', active: false }), + ], + }, + [stoppedItem({ key: 'source:s1', type: 'source', id: 's1' })], + ); + const [running, stopped] = alerts(); + expect(alerts()).toHaveLength(2); + expect(running.getAttribute('role')).toBe('note'); + expect(running.textContent).toContain('agents.form.sponsors.addedBy'); + expect(stopped.getAttribute('data-variant')).toBe('warning'); + expect(stopped.textContent).toContain('agents.form.resourceStates.title'); + expect(stopped.querySelectorAll('li')).toHaveLength(1); + }); + + it.each([ + ['deleted', 'reason.deleted'], + ['sponsor_cannot_edit_agent', 'reason.sponsorCannotEditAgent'], + ['sponsor_cannot_edit_resource', 'reason.sponsorCannotEditItem'], + ['connection_needs_reconnect', 'reason.connectionNeedsReconnect'], + ['connection_removed', 'reason.connectionRemoved'], + ['connector_disabled', 'reason.connectorDisabled'], + ] as const)('says why an item stopped: %s', async (reason, key) => { + await render(baseAgent, [stoppedItem({ reason })]); + expect(container.textContent).toContain( + `agents.form.resourceStates.${key}`, + ); + }); + + it('says someone else for a sponsor the reader does not know', async () => { + await render( + { + ...editorAgent, + resource_sponsors: [ + sponsor({ id: 't1', user_id: null, label: null }), + sponsor({ id: 't2', user_id: null, label: null }), + ], + }, + [ + stoppedItem({ + reason: 'sponsor_cannot_edit_resource', + sponsor: { user_id: null, label: null }, + }), + stoppedItem({ + key: 'tool:t3', + id: 't3', + reason: 'sponsor_cannot_edit_agent', + sponsor: { user_id: 'bob', label: 'bob@example.com' }, + }), + ], + ); + const text = container.textContent ?? ''; + expect(text).toContain('agents.form.sponsors.addedByOther'); + expect(text).not.toContain('agents.form.sponsors.addedBy.'); + expect(text).toContain( + 'agents.form.resourceStates.reason.sponsorCannotEditItemOther', + ); + // A sponsor the reader knows is named. + expect(text).toMatch(/reason\.sponsorCannotEditAgent(?!Other)/); + }); + + it('tells the owner they lost access, and an editor that the owner did', async () => { + const item = stoppedItem({ reason: 'owner_lost_access' }); + await render(baseAgent, [item]); + expect(container.textContent).toContain('reason.ownerLostAccessYou'); + await render(editorAgent, [item]); + expect(container.textContent).not.toContain('reason.ownerLostAccessYou'); + expect(container.textContent).toContain('reason.ownerLostAccess'); + }); + + it('says whom to ask when the reader cannot fix it', async () => { + await render(baseAgent, [ + stoppedItem({ + key: 'source:s1', + type: 'source', + id: 's1', + reason: 'owner_lost_access', + contact: { user_id: 'carol', label: 'carol@example.com' }, + }), + stoppedItem({ key: 'tool:t2', id: 't2', reason: 'connector_disabled' }), + stoppedItem({ key: 'tool:t3', id: 't3', reason: 'deleted' }), + ]); + const [shareAgain, admin, deleted] = Array.from( + container.querySelectorAll('li'), + ); + expect(shareAgain.textContent).toContain( + 'agents.form.resourceStates.ask.shareAgain', + ); + expect(admin.textContent).toContain('agents.form.resourceStates.ask.admin'); + expect(deleted.textContent).not.toContain('agents.form.resourceStates.ask'); + }); + + it("asks the item's owner without naming them when the reader may not know them", async () => { + await render(editorAgent, [ + stoppedItem({ + reason: 'owner_lost_access', + contact: null, + contact_role: 'resource_owner', + }), + stoppedItem({ + key: 'tool:t2', + id: 't2', + reason: 'connection_needs_reconnect', + contact: null, + contact_role: 'resource_owner', + }), + stoppedItem({ + key: 'tool:t3', + id: 't3', + reason: 'connection_removed', + contact: null, + contact_role: 'resource_owner', + }), + ]); + const [lost, reconnect, removed] = Array.from( + container.querySelectorAll('li'), + ); + expect(lost.textContent).toContain( + 'agents.form.resourceStates.ask.shareAgainOwner', + ); + expect(reconnect.textContent).toContain( + 'agents.form.resourceStates.ask.signInAgainOwner', + ); + expect(removed.textContent).toContain( + 'agents.form.resourceStates.ask.connectAgainOwner', + ); + }); + + it('names a nameless item by its kind and a short id', () => { + const t = ((key: string, options?: Record) => + `${key}|${options?.id}`) as never; + expect( + unnamedResourceLabel(t, { + type: 'source', + id: '0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0', + }), + ).toBe('agents.form.resourceStates.unnamed.source|0f1e2d3c'); + }); + + it('notes that a stopped prompt falls back to the default', async () => { + await render(baseAgent, [ + stoppedItem({ key: 'prompt:p1', type: 'prompt', id: 'p1' }), + ]); + expect(container.textContent).toContain( + 'agents.form.resourceStates.promptFallback', + ); + }); + + it('offers Reconnect only when the reader owns the connection', async () => { + const connection = { id: 'c1', connector_key: 'telegram', name: 'Tg' }; + const spies = await render(baseAgent, [ + stoppedItem({ + reason: 'connection_needs_reconnect', + connection, + can_reconnect: true, + }), + stoppedItem({ + key: 'tool:t2', + id: 't2', + reason: 'connection_needs_reconnect', + connection, + can_reconnect: false, + contact: { user_id: 'bob', label: 'bob@example.com' }, + }), + ]); + const reconnect = buttonWith('settings.connectors.status.reconnect'); + expect(reconnect).toHaveLength(1); + await act(async () => reconnect[0].click()); + expect(spies.onReconnect).toHaveBeenCalledWith( + expect.objectContaining({ key: 'tool:t1' }), + ); + expect(container.textContent).toContain( + 'agents.form.resourceStates.ask.signInAgain', + ); + }); + + it('offers a take-over only for items the reader may confirm', async () => { + const spies = await render(editorAgent, [ + stoppedItem({ reason: 'sponsor_cannot_edit_agent', can_confirm: true }), + stoppedItem({ + key: 'tool:t2', + id: 't2', + reason: 'sponsor_cannot_edit_agent', + can_confirm: false, + }), + ]); + const takeOver = buttonWith('agents.form.sponsors.takeOver'); + expect(takeOver).toHaveLength(1); + await act(async () => takeOver[0].click()); + expect(spies.onTakeOver).toHaveBeenCalledWith( + expect.objectContaining({ key: 'tool:t1' }), + ); + }); + + it('shows a pending take-over with an undo', async () => { + const spies = await render( + editorAgent, + [stoppedItem({ reason: 'owner_lost_access', can_confirm: true })], + ['tool:t1'], + ); + expect(container.textContent).toContain( + 'agents.form.sponsors.takeOverPending', + ); + await act(async () => + buttonWith('agents.form.sponsors.undoTakeOver')[0].click(), + ); + expect(spies.onUndoTakeover).toHaveBeenCalledWith('tool:t1'); + }); + + it('removes an item', async () => { + const spies = await render(baseAgent, [stoppedItem({})]); + const [remove] = buttonWith('agents.form.resourceStates.remove'); + expect(remove.getAttribute('aria-label')).toBe( + 'agents.form.resourceStates.removeLabel', + ); + await act(async () => remove.click()); + expect(spies.onRemove).toHaveBeenCalledWith( + expect.objectContaining({ key: 'tool:t1' }), + ); + }); +}); diff --git a/frontend/src/agents/components/ResourceStatusNotice.tsx b/frontend/src/agents/components/ResourceStatusNotice.tsx new file mode 100644 index 00000000..de5d5bb4 --- /dev/null +++ b/frontend/src/agents/components/ResourceStatusNotice.tsx @@ -0,0 +1,298 @@ +import type { TFunction } from 'i18next'; +import { Info, TriangleAlert, UserRound } from 'lucide-react'; +import { useTranslation } from 'react-i18next'; + +import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert'; +import { Button } from '@/components/ui/button'; + +import { can, isOwner } from '../../utils/accessUtils'; +import { intlLocale } from '../../utils/dateTimeUtils'; +import type { + Agent, + ResourceSponsor, + ResourceState, + ResourceStateReason, +} from '../types'; + +/** What the notice needs to name an item. */ +export type NamedResource = Pick; + +type ResourceStatusNoticeProps = { + /** The agent the page edits: the reader's access, its link, its sponsors. */ + agent: Agent; + /** Stopped items still attached in the form (`resource_states`). */ + stopped: ResourceState[]; + /** Display name of a tool, source or prompt. */ + resolveName: (item: NamedResource) => string; + /** Keys of stopped items the reader chose to run with their access on save. */ + takeovers?: string[]; + /** Ask the reader to run one item with their access. */ + onTakeOver?: (item: ResourceState) => void; + /** Drop a pending take-over. */ + onUndoTakeover?: (key: string) => void; + /** Take one item off the agent (or the workflow's nodes). */ + onRemove?: (item: ResourceState) => void; + /** Sign the item's connection in again. */ + onReconnect?: (item: ResourceState) => void; + /** Tell an editor that what they add runs with their access (the agent form). */ + showAttachNote?: boolean; +}; + +/** + * The message key that says why an item stopped. `sponsorNamed` is false + * when the read doesn't name the sponsor, who is then "someone else". + */ +export function reasonKey( + reason: ResourceStateReason | null, + ownerReads: boolean, + sponsorNamed = true, +) { + const other = sponsorNamed ? '' : 'Other'; + switch (reason) { + case 'deleted': + return 'agents.form.resourceStates.reason.deleted'; + case 'owner_lost_access': + return ownerReads + ? 'agents.form.resourceStates.reason.ownerLostAccessYou' + : 'agents.form.resourceStates.reason.ownerLostAccess'; + case 'sponsor_cannot_edit_agent': + return `agents.form.resourceStates.reason.sponsorCannotEditAgent${other}`; + case 'sponsor_cannot_edit_resource': + return `agents.form.resourceStates.reason.sponsorCannotEditItem${other}`; + case 'connection_needs_reconnect': + return 'agents.form.resourceStates.reason.connectionNeedsReconnect'; + case 'connection_removed': + return 'agents.form.resourceStates.reason.connectionRemoved'; + case 'connector_disabled': + return 'agents.form.resourceStates.reason.connectorDisabled'; + default: + return 'agents.form.resourceStates.reason.unknown'; + } +} + +/** A name for an item the reader may not see: its kind and a short id. */ +export function unnamedResourceLabel( + t: TFunction, + item: Pick, +): string { + return t(`agents.form.resourceStates.unnamed.${item.type}`, { + id: item.id.slice(0, 8), + }); +} + +/** + * Whom to ask, as the key that says so; null when the reader can act. The + * person is named when the read names them (`contact`), else only as the + * item's owner. + */ +function askKey(item: ResourceState): string | null { + if (item.reason === 'connector_disabled') + return 'agents.form.resourceStates.ask.admin'; + const suffix = item.contact + ? '' + : item.contact_role === 'resource_owner' + ? 'Owner' + : null; + if (suffix === null) return null; + if (item.reason === 'owner_lost_access') + return `agents.form.resourceStates.ask.shareAgain${suffix}`; + if (item.reason === 'connection_needs_reconnect') + return `agents.form.resourceStates.ask.signInAgain${suffix}`; + if (item.reason === 'connection_removed') + return `agents.form.resourceStates.ask.connectAgain${suffix}`; + return null; +} + +/** + * Items grouped by the person who added them, in first-seen order; people + * the reader doesn't know form one group. + */ +function groupByPerson(sponsors: ResourceSponsor[]) { + const groups = new Map(); + for (const sponsor of sponsors) { + const key = sponsor.label || sponsor.user_id || ''; + groups.set(key, [...(groups.get(key) ?? []), sponsor]); + } + return Array.from(groups.values()); +} + +/** + * One notice for what runs on the agent with someone else's access and what + * stopped running. + * + * An editor sees up front that what they attach runs with their access for + * everyone who uses the agent, and everyone who may edit it sees who added + * what (`resource_sponsors`). Each attached tool, source or prompt that no + * longer runs (`resource_states`) is listed with the reason in plain words and + * what the reader can do: sign its account in again, run it with their own + * access, take it off, or whom to ask. + */ +export default function ResourceStatusNotice({ + agent, + stopped, + resolveName, + takeovers = [], + onTakeOver, + onUndoTakeover, + onRemove, + onReconnect, + showAttachNote: attachNoteWanted = true, +}: ResourceStatusNoticeProps) { + const { t, i18n } = useTranslation(); + const ownerReads = isOwner(agent); + const showAttachNote = attachNoteWanted && !ownerReads && can(agent, 'edit'); + const running = groupByPerson( + (agent.resource_sponsors ?? []).filter((s) => s.active), + ); + + if (!showAttachNote && running.length === 0 && stopped.length === 0) + return null; + + const listFormat = new Intl.ListFormat(intlLocale(i18n.language), { + type: 'conjunction', + }); + const plain = { interpolation: { escapeValue: false } }; + + return ( +
+ {showAttachNote && ( + + + + {t('agents.form.sponsors.attachNote')} + {agent.shared ? ` ${t('agents.form.sponsors.publicLinkNote')}` : ''} + + + )} + {running.length > 0 && ( + + + + {running.map((items) => ( +

+ {t( + items[0].user_id + ? 'agents.form.sponsors.addedBy' + : 'agents.form.sponsors.addedByOther', + { + ...plain, + person: items[0].label || items[0].user_id, + names: listFormat.format(items.map(resolveName)), + }, + )} +

+ ))} + + + )} + {stopped.length > 0 && ( + + + {t('agents.form.resourceStates.title')} + +
    + {stopped.map((item) => { + const name = resolveName(item); + const service = + item.connection?.name || + t('agents.form.resourceStates.serviceFallback'); + const ask = askKey(item); + const pending = takeovers.includes(item.key); + return ( +
  • +

    + {t( + reasonKey( + item.reason, + ownerReads, + Boolean(item.sponsor?.user_id), + ), + { + ...plain, + name, + service, + person: item.sponsor?.label || item.sponsor?.user_id, + }, + )} + {ask + ? ` ${t(ask, { ...plain, person: item.contact?.label })}` + : ''} + {item.type === 'prompt' + ? ` ${t('agents.form.resourceStates.promptFallback')}` + : ''} +

    + {pending ? ( +
    + + {t('agents.form.sponsors.takeOverPending', { + ...plain, + name, + })} + + +
    + ) : ( +
    + {item.can_reconnect && onReconnect && ( + + )} + {item.can_confirm && onTakeOver && ( + + )} + {onRemove && ( + + )} +
    + )} +
    • + ); + })} +
+ + + )} +
+ ); +} diff --git a/frontend/src/agents/components/SponsorConfirmModal.test.tsx b/frontend/src/agents/components/SponsorConfirmModal.test.tsx new file mode 100644 index 00000000..423f612d --- /dev/null +++ b/frontend/src/agents/components/SponsorConfirmModal.test.tsx @@ -0,0 +1,128 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('../../hooks', () => ({ + useMediaQuery: () => ({ isMobile: false, isDesktop: true }), +})); + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'teams' in opts ? `${key}:${opts.teams}` : key, + i18n: { language: 'en' }, + }), +})); + +import type { SponsorConfirmation } from '../sponsorConsent'; +import SponsorConfirmModal from './SponsorConfirmModal'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const confirmation = ( + over: Partial = {}, +): SponsorConfirmation => ({ + resources: [ + { key: 'tool:t1', type: 'tool', id: 't1', name: 'Jira' }, + { key: 'source:s1', type: 'source', id: 's1', name: null }, + ], + audience: { + teams: ['Support', 'Sales'], + api_key: false, + public_link: false, + webhook: false, + ...over, + }, +}); + +describe('SponsorConfirmModal', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + value: SponsorConfirmation | null, + onConfirm = vi.fn(), + onCancel = vi.fn(), + ) => { + await act(async () => { + root.render( + , + ); + }); + return { onConfirm, onCancel }; + }; + + const dialogText = () => + document.querySelector('[data-slot="modal-content"]')?.textContent ?? ''; + + const button = (label: string) => + Array.from(document.querySelectorAll('button')).find( + (b) => b.textContent === label, + ) as HTMLButtonElement; + + it('renders nothing without a pending confirmation', async () => { + await render(null); + expect(document.querySelector('[data-slot="modal-content"]')).toBeNull(); + }); + + it('names each resource, with a fallback for an unnamed one', async () => { + await render(confirmation()); + const text = dialogText(); + expect(text).toContain('agents.form.sponsorConfirm.title'); + expect(text).toContain('Jira'); + expect(text).toContain('agents.form.sponsorConfirm.types.tool'); + expect(text).toContain('agents.form.sponsors.unknownItem'); + expect(text).toContain('agents.form.sponsorConfirm.types.source'); + }); + + it('lists the teams and only the outside entry points that are on', async () => { + await render(confirmation({ public_link: true })); + const text = dialogText(); + expect(text).toContain( + 'agents.form.sponsorConfirm.audienceTeams:Support and Sales', + ); + expect(text).toContain('agents.form.sponsorConfirm.audiencePublicLink'); + expect(text).not.toContain('agents.form.sponsorConfirm.audienceApiKey'); + expect(text).not.toContain('agents.form.sponsorConfirm.audienceWebhook'); + }); + + it('shows the API and webhook lines when the agent has them', async () => { + await render(confirmation({ teams: [], api_key: true, webhook: true })); + const text = dialogText(); + expect(text).not.toContain('audienceTeams'); + expect(text).toContain('agents.form.sponsorConfirm.audienceApiKey'); + expect(text).toContain('agents.form.sponsorConfirm.audienceWebhook'); + }); + + it('confirms with every resource key', async () => { + const { onConfirm, onCancel } = await render(confirmation()); + await act(async () => { + button('agents.form.sponsorConfirm.confirm').click(); + }); + expect(onConfirm).toHaveBeenCalledWith(['tool:t1', 'source:s1']); + expect(onCancel).not.toHaveBeenCalled(); + }); + + it('cancels without confirming', async () => { + const { onConfirm, onCancel } = await render(confirmation()); + await act(async () => { + button('cancel').click(); + }); + expect(onCancel).toHaveBeenCalled(); + expect(onConfirm).not.toHaveBeenCalled(); + }); +}); diff --git a/frontend/src/agents/components/SponsorConfirmModal.tsx b/frontend/src/agents/components/SponsorConfirmModal.tsx new file mode 100644 index 00000000..30b0252b --- /dev/null +++ b/frontend/src/agents/components/SponsorConfirmModal.tsx @@ -0,0 +1,162 @@ +import { + Database, + Info, + KeyRound, + Link2, + ScrollText, + UsersRound, + Webhook, + Wrench, +} from 'lucide-react'; +import { useTranslation } from 'react-i18next'; + +import { Alert, AlertDescription } from '@/components/ui/alert'; +import { Card } from '@/components/ui/card'; +import { ListRow, ListRows } from '@/components/ui/list-row'; +import { Modal, ModalActions } from '@/components/ui/modal'; +import { SectionHeader } from '@/components/ui/section-header'; + +import { intlLocale } from '../../utils/dateTimeUtils'; +import type { SponsorConfirmation, SponsorResource } from '../sponsorConsent'; + +type SponsorConfirmModalProps = { + /** The pending request from a refused save; null keeps the modal closed. */ + confirmation: SponsorConfirmation | null; + /** Retry the save with these `confirm_sponsor` keys. */ + onConfirm: (keys: string[]) => void; + onCancel: () => void; + pending?: boolean; +}; + +const TYPE_ICONS: Record = { + tool: Wrench, + source: Database, + prompt: ScrollText, +}; + +function IconSquare({ icon: Icon }: { icon: typeof Wrench }) { + return ( + + + + ); +} + +/** + * Asks an editor before a save makes tools, sources or prompts the agent's + * owner can't use run with the editor's access, and says who will reach them + * through the agent. Confirming retries the save with `confirm_sponsor`. + */ +export default function SponsorConfirmModal({ + confirmation, + onConfirm, + onCancel, + pending = false, +}: SponsorConfirmModalProps) { + const { t, i18n } = useTranslation(); + if (!confirmation) return null; + + const { resources, audience } = confirmation; + const listFormat = new Intl.ListFormat(intlLocale(i18n.language), { + type: 'conjunction', + }); + const audienceRows: { key: string; icon: typeof Wrench; text: string }[] = []; + if (audience.teams.length > 0) { + audienceRows.push({ + key: 'teams', + icon: UsersRound, + text: t('agents.form.sponsorConfirm.audienceTeams', { + teams: listFormat.format(audience.teams), + interpolation: { escapeValue: false }, + }), + }); + } + if (audience.api_key) { + audienceRows.push({ + key: 'api', + icon: KeyRound, + text: t('agents.form.sponsorConfirm.audienceApiKey'), + }); + } + if (audience.public_link) { + audienceRows.push({ + key: 'link', + icon: Link2, + text: t('agents.form.sponsorConfirm.audiencePublicLink'), + }); + } + if (audience.webhook) { + audienceRows.push({ + key: 'webhook', + icon: Webhook, + text: t('agents.form.sponsorConfirm.audienceWebhook'), + }); + } + if (audienceRows.length === 0) { + audienceRows.push({ + key: 'editors', + icon: UsersRound, + text: t('agents.form.sponsorConfirm.audienceEditors'), + }); + } + + return ( + { + if (!open) onCancel(); + }} + isPerformingTask={pending} + title={t('agents.form.sponsorConfirm.title')} + description={t('agents.form.sponsorConfirm.description')} + footer={ + onConfirm(resources.map((item) => item.key))} + pending={pending} + /> + } + > +
+ + + {resources.map((item) => ( + } + title={item.name || t('agents.form.sponsors.unknownItem')} + description={t(`agents.form.sponsorConfirm.types.${item.type}`)} + /> + ))} + + +
+ +
    + {audienceRows.map(({ key, icon: Icon, text }) => ( +
  • + + {text} +
    • + ))} +
+
+ + + + {t('agents.form.sponsorConfirm.stopNote')} + + +
+ + ); +} diff --git a/frontend/src/agents/components/SponsoredResourcesNotice.test.tsx b/frontend/src/agents/components/SponsoredResourcesNotice.test.tsx deleted file mode 100644 index ef92086e..00000000 --- a/frontend/src/agents/components/SponsoredResourcesNotice.test.tsx +++ /dev/null @@ -1,136 +0,0 @@ -import { act } from 'react'; -import { createRoot, type Root } from 'react-dom/client'; - -import i18n from 'i18next'; -import { initReactI18next } from 'react-i18next'; - -import type { Agent, ResourceSponsor } from '../types'; -import SponsoredResourcesNotice from './SponsoredResourcesNotice'; - -Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); - -const baseAgent: Agent = { - name: 'A', - description: 'd', - image: '', - source: '', - chunks: '6', - retriever: '', - prompt_id: 'default', - tools: [], - agent_type: 'classic', - status: 'published', -}; - -const sponsor = (over: Partial): ResourceSponsor => ({ - type: 'tool', - id: 't1', - user_id: 'bob', - label: 'bob@example.com', - active: true, - ...over, -}); - -describe('SponsoredResourcesNotice', () => { - let container: HTMLDivElement; - let root: Root; - - beforeEach(() => { - container = document.createElement('div'); - document.body.appendChild(container); - root = createRoot(container); - }); - - afterEach(async () => { - await act(async () => root.unmount()); - container.remove(); - }); - - const render = async (agent: Agent) => { - await act(async () => { - root.render( - `name-${s.id}`} - />, - ); - }); - }; - - const alerts = () => - Array.from(container.querySelectorAll('[data-slot="alert"]')); - - it('renders nothing for the owner with no sponsored items', async () => { - await render(baseAgent); - expect(container.innerHTML).toBe(''); - }); - - it('tells an editor their own items run with their access', async () => { - await render({ ...baseAgent, access: 'editor', allowed_actions: ['edit'] }); - expect(alerts()).toHaveLength(1); - expect(container.textContent).toContain('agents.form.sponsors.attachNote'); - expect(container.textContent).not.toContain('publicLinkNote'); - }); - - it('adds the public-link warning when the agent has a link', async () => { - await render({ - ...baseAgent, - shared: true, - access: 'editor', - allowed_actions: ['edit'], - }); - expect(container.textContent).toContain( - 'agents.form.sponsors.publicLinkNote', - ); - }); - - it('does not show the attach note to a viewer', async () => { - await render({ ...baseAgent, access: 'viewer', allowed_actions: ['use'] }); - expect(container.innerHTML).toBe(''); - }); - - it('joins names for the app language and does not escape them', async () => { - await i18n.use(initReactI18next).init({ - lng: 'jp', - resources: { - jp: { - translation: { - agents: { - form: { sponsors: { addedBy: '{{person}}: {{names}}' } }, - }, - }, - }, - }, - }); - try { - await render({ - ...baseAgent, - resource_sponsors: [ - sponsor({ id: 'docs/a' }), - sponsor({ id: 'docs/b' }), - ], - }); - const expected = new Intl.ListFormat('ja', { - type: 'conjunction', - }).format(['name-docs/a', 'name-docs/b']); - expect(container.textContent).toBe(`bob@example.com: ${expected}`); - } finally { - await i18n.changeLanguage('en'); - } - }); - - it('splits running and no-longer-running items', async () => { - await render({ - ...baseAgent, - resource_sponsors: [ - sponsor({ id: 't1' }), - sponsor({ id: 's1', type: 'source', active: false }), - ], - }); - const [running, stopped] = alerts(); - expect(running.getAttribute('role')).toBe('note'); - expect(running.textContent).toContain('agents.form.sponsors.addedBy'); - expect(stopped.getAttribute('data-variant')).toBe('warning'); - expect(stopped.textContent).toContain('agents.form.sponsors.unavailable'); - }); -}); diff --git a/frontend/src/agents/components/SponsoredResourcesNotice.tsx b/frontend/src/agents/components/SponsoredResourcesNotice.tsx deleted file mode 100644 index 42c6c687..00000000 --- a/frontend/src/agents/components/SponsoredResourcesNotice.tsx +++ /dev/null @@ -1,97 +0,0 @@ -import { Info, TriangleAlert, UserRound } from 'lucide-react'; -import { useTranslation } from 'react-i18next'; - -import { Alert, AlertDescription } from '@/components/ui/alert'; - -import { can, isOwner } from '../../utils/accessUtils'; -import { intlLocale } from '../../utils/dateTimeUtils'; -import type { Agent, ResourceSponsor } from '../types'; - -type SponsoredResourcesNoticeProps = { - agent: Agent; - /** Display name of a sponsored tool, source or prompt. */ - resolveName: (sponsor: ResourceSponsor) => string; -}; - -/** Sponsored items grouped by the person who added them, in first-seen order. */ -function groupByPerson(sponsors: ResourceSponsor[]) { - const groups = new Map(); - for (const sponsor of sponsors) { - const key = sponsor.label || sponsor.user_id; - groups.set(key, [...(groups.get(key) ?? []), sponsor]); - } - return Array.from(groups.entries()); -} - -/** - * Tools, sources and prompts on the agent that run with an editor's access - * rather than the owner's (`resource_sponsors` from GET /api/get_agent). - * - * An editor sees up front that what they attach runs with their access for - * everyone who uses the agent. Everyone who may view the config sees who - * added what, and which items no longer run because that person lost access. - */ -export default function SponsoredResourcesNotice({ - agent, - resolveName, -}: SponsoredResourcesNoticeProps) { - const { t, i18n } = useTranslation(); - const sponsors = agent.resource_sponsors ?? []; - const showAttachNote = !isOwner(agent) && can(agent, 'edit'); - const active = groupByPerson(sponsors.filter((s) => s.active)); - const inactive = groupByPerson(sponsors.filter((s) => !s.active)); - - if (!showAttachNote && sponsors.length === 0) return null; - - const listFormat = new Intl.ListFormat(intlLocale(i18n.language), { - type: 'conjunction', - }); - const names = (items: ResourceSponsor[]) => - listFormat.format(items.map((item) => resolveName(item))); - - return ( -
- {showAttachNote && ( - - - - {t('agents.form.sponsors.attachNote')} - {agent.shared ? ` ${t('agents.form.sponsors.publicLinkNote')}` : ''} - - - )} - {active.length > 0 && ( - - - - {active.map(([person, items]) => ( -

- {t('agents.form.sponsors.addedBy', { - interpolation: { escapeValue: false }, - person, - names: names(items), - })} -

- ))} - - - )} - {inactive.length > 0 && ( - - - - {inactive.map(([person, items]) => ( -

- {t('agents.form.sponsors.unavailable', { - interpolation: { escapeValue: false }, - person, - names: names(items), - })} -

- ))} - - - )} -
- ); -} diff --git a/frontend/src/agents/sponsorConsent.test.tsx b/frontend/src/agents/sponsorConsent.test.tsx new file mode 100644 index 00000000..6a0202bf --- /dev/null +++ b/frontend/src/agents/sponsorConsent.test.tsx @@ -0,0 +1,219 @@ +import type { MultiSelectPopoverItem } from '../components/MultiSelectPopover'; +import { + confirmTakeOver, + NO_AUDIENCE, + readSponsorRefusal, + saveWithSponsorConsent, + withAttachedOptions, + withAttachedToolRows, +} from './sponsorConsent'; + +const json = (status: number, body: unknown) => + new Response(JSON.stringify(body), { + status, + headers: { 'Content-Type': 'application/json' }, + }); + +describe('readSponsorRefusal', () => { + const resources = [{ key: 'tool:t1', type: 'tool', id: 't1', name: 'Jira' }]; + const audience = { + teams: ['Support'], + api_key: true, + public_link: false, + webhook: false, + }; + + it('reads a confirmation request and leaves the body readable', async () => { + const response = json(409, { + success: false, + code: 'sponsor_confirmation_required', + message: 'm', + resources, + audience, + }); + expect(await readSponsorRefusal(response)).toEqual({ + kind: 'confirm', + confirmation: { resources, audience }, + }); + expect((await response.json()).message).toBe('m'); + }); + + it('reads a refusal the caller may not sponsor', async () => { + const response = json(403, { + code: 'sponsor_not_allowed', + message: 'no', + resources, + }); + expect(await readSponsorRefusal(response)).toEqual({ + kind: 'notAllowed', + resources, + }); + }); + + it('ignores other failures', async () => { + expect( + await readSponsorRefusal(json(409, { code: 'stale_write' })), + ).toBeNull(); + expect(await readSponsorRefusal(json(403, { message: 'x' }))).toBeNull(); + expect( + await readSponsorRefusal(new Response('', { status: 502 })), + ).toBeNull(); + }); +}); + +describe('withAttachedToolRows', () => { + const labels = { group: 'On this agent', description: 'Remove only' }; + const listed: MultiSelectPopoverItem[] = [ + { id: 'mine', label: 'My tool', group: 'Custom' }, + ]; + + it('adds a removable row for an attached tool the caller cannot list', () => { + const items = withAttachedToolRows( + listed, + [ + { id: 'mine', name: 'api_tool', display_name: 'My tool' }, + { id: 'owners', name: 'jira', display_name: 'Owner Jira' }, + ], + labels, + ); + expect(items.map((item) => item.id)).toEqual(['mine', 'owners']); + const locked = items[1]; + expect(locked.label).toBe('Owner Jira'); + expect(locked.group).toBe('On this agent'); + expect(locked.description).toBe('Remove only'); + expect(locked.disabled).toBeFalsy(); + }); + + it('keeps one row per tool when it is listed or repeated', () => { + const tool = { id: 'owners', name: 'jira', display_name: 'Owner Jira' }; + const items = withAttachedToolRows(listed, [tool, tool], labels); + expect(items.map((item) => item.id)).toEqual(['mine', 'owners']); + }); +}); + +describe('readSponsorRefusal: stale confirmations', () => { + it('reads a confirmation that names something the save no longer needs', async () => { + const response = json(400, { + code: 'sponsor_confirmation_unexpected', + unexpected: ['tool:gone'], + }); + expect(await readSponsorRefusal(response)).toEqual({ kind: 'unexpected' }); + }); + + it('ignores an ordinary 400', async () => { + expect(await readSponsorRefusal(json(400, { message: 'bad' }))).toBeNull(); + }); +}); + +describe('saveWithSponsorConsent', () => { + const confirmation = { + resources: [ + { key: 'tool:t1', type: 'tool' as const, id: 't1', name: 'Jira' }, + ], + audience: { teams: [], api_key: false, public_link: false, webhook: false }, + }; + const refused = () => + json(409, { + code: 'sponsor_confirmation_required', + ...confirmation, + }); + + it('sends once when nothing needs confirming', async () => { + const send = vi.fn(async () => json(200, { success: true })); + const ask = vi.fn(); + const response = await saveWithSponsorConsent(send, ask, ['tool:x']); + expect(response?.status).toBe(200); + expect(send).toHaveBeenCalledTimes(1); + expect(send).toHaveBeenCalledWith(['tool:x']); + expect(ask).not.toHaveBeenCalled(); + }); + + it('asks, then resends with the agreed keys and any it already had', async () => { + const send = vi + .fn<(keys: string[]) => Promise>() + .mockResolvedValueOnce(refused()) + .mockResolvedValueOnce(json(200, { success: true })); + const ask = vi.fn(async () => ['tool:t1']); + const response = await saveWithSponsorConsent(send, ask, ['source:s1']); + expect(ask).toHaveBeenCalledWith(confirmation); + expect(send.mock.calls.map(([keys]) => keys)).toEqual([ + ['source:s1'], + ['source:s1', 'tool:t1'], + ]); + expect(response?.status).toBe(200); + }); + + it('returns null when the caller declines, without resending', async () => { + const send = vi.fn(async () => refused()); + const response = await saveWithSponsorConsent(send, async () => null); + expect(response).toBeNull(); + expect(send).toHaveBeenCalledTimes(1); + }); + + // A workflow save answers in the workflow routes' shape; the round trip + // is the same. + it('passes other failures straight back', async () => { + const failure = json(400, { success: false, error: 'Workflow invalid' }); + const send = vi.fn(async () => failure); + expect(await saveWithSponsorConsent(send, vi.fn())).toBe(failure); + }); +}); + +describe('withAttachedOptions', () => { + it('adds a labelled option for a selected id the caller cannot list', () => { + const options = withAttachedOptions( + [{ value: 'mine', label: 'Mine' }], + [ + { id: 'owners', label: 'Owner tool' }, + { id: 'unselected', label: 'Elsewhere' }, + ], + ['mine', 'owners'], + (name) => `${name} (added)`, + ); + expect(options).toEqual([ + { value: 'mine', label: 'Mine' }, + { value: 'owners', label: 'Owner tool (added)' }, + ]); + }); + + it('falls back to the id when no name is known', () => { + const options = withAttachedOptions([], [], ['x1'], (name) => name); + expect(options).toEqual([{ value: 'x1', label: 'x1' }]); + }); +}); + +describe('confirmTakeOver', () => { + const item = { key: 'tool:t1', type: 'tool' as const, id: 't1' }; + const audience = { + teams: ['Sales'], + api_key: true, + public_link: false, + webhook: false, + }; + + it('shows the item and who reaches it, and agrees when confirmed', async () => { + const ask = vi.fn(() => Promise.resolve(['tool:t1'])); + await expect(confirmTakeOver(ask, item, 'Jira', audience)).resolves.toBe( + true, + ); + expect(ask).toHaveBeenCalledWith({ + resources: [{ ...item, name: 'Jira' }], + audience, + }); + }); + + it('does not agree when the caller cancels', async () => { + const ask = vi.fn(() => Promise.resolve(null)); + await expect(confirmTakeOver(ask, item, 'Jira', audience)).resolves.toBe( + false, + ); + }); + + it('asks with an empty audience when the read sent none', async () => { + const ask = vi.fn(() => Promise.resolve(['tool:t1'])); + await confirmTakeOver(ask, item, 'Jira', undefined); + expect(ask).toHaveBeenCalledWith( + expect.objectContaining({ audience: NO_AUDIENCE }), + ); + }); +}); diff --git a/frontend/src/agents/sponsorConsent.tsx b/frontend/src/agents/sponsorConsent.tsx new file mode 100644 index 00000000..d91ee7cb --- /dev/null +++ b/frontend/src/agents/sponsorConsent.tsx @@ -0,0 +1,217 @@ +import type { TFunction } from 'i18next'; + +import ToolIcon from '../components/ToolIcon'; +import type { MultiSelectPopoverItem } from '../components/MultiSelectPopover'; +import { intlLocale } from '../utils/dateTimeUtils'; +import { getToolDisplayName } from '../utils/toolUtils'; +import type { ToolSummary } from './types'; + +/** A resource a save would have run with the caller's access. */ +export type SponsorResource = { + /** `":"`, the value `confirm_sponsor` takes. */ + key: string; + type: 'tool' | 'source' | 'prompt'; + id: string; + name: string | null; +}; + +/** Who reaches the agent's resources (409 `audience`). */ +export type SponsorAudience = { + /** Names of the teams the agent is shared with. */ + teams: string[]; + /** The agent has an API key (the API and the website widget). */ + api_key: boolean; + public_link: boolean; + webhook: boolean; +}; + +/** An audience with nobody in it, for a read that sent none. */ +export const NO_AUDIENCE: SponsorAudience = { + teams: [], + api_key: false, + public_link: false, + webhook: false, +}; + +/** What a save asks the caller to agree to before it goes ahead. */ +export type SponsorConfirmation = { + resources: SponsorResource[]; + audience: SponsorAudience; +}; + +export type SponsorRefusal = + | { kind: 'confirm'; confirmation: SponsorConfirmation } + | { kind: 'notAllowed'; resources: SponsorResource[] } + /** The save confirmed something it no longer needs (the agent changed). */ + | { kind: 'unexpected' }; + +/** + * The sponsor refusal a failed agent or workflow save carries, if any. + * + * A save that would run a resource the owner can't use with the caller's + * access answers 409 `sponsor_confirmation_required` until it is retried + * with `confirm_sponsor`; one the caller may not sponsor at all answers 403 + * `sponsor_not_allowed`; one whose confirmation names something it no + * longer needs answers 400 `sponsor_confirmation_unexpected`. Reads a + * clone, so the caller can still read the body for any other error. + */ +export async function readSponsorRefusal( + response: Response, +): Promise { + if (![400, 403, 409].includes(response.status)) return null; + if (typeof response.clone !== 'function') return null; + let body: { + code?: string; + resources?: SponsorResource[]; + audience?: SponsorAudience; + }; + try { + body = await response.clone().json(); + } catch { + return null; + } + if (body?.code === 'sponsor_confirmation_required' && body.audience) { + return { + kind: 'confirm', + confirmation: { + resources: body.resources ?? [], + audience: body.audience, + }, + }; + } + if (body?.code === 'sponsor_not_allowed') { + return { kind: 'notAllowed', resources: body.resources ?? [] }; + } + if (body?.code === 'sponsor_confirmation_unexpected') { + return { kind: 'unexpected' }; + } + return null; +} + +/** + * Send an agent or workflow save, asking the caller whenever the server + * wants their confirmation to sponsor something, and resending with it. + * + * Args: + * send: Sends the save with these `confirm_sponsor` keys. + * ask: Shows the confirmation; resolves to the agreed keys, or null. + * confirmed: Keys already agreed (items the caller chose to take over). + * + * Returns: + * The last response, or null when the caller declined. + */ +export async function saveWithSponsorConsent( + send: (confirm: string[]) => Promise, + ask: (confirmation: SponsorConfirmation) => Promise, + confirmed: string[] = [], +): Promise { + let keys = Array.from(new Set(confirmed)); + let response = await send(keys); + // The agent can change between rounds; three asks is plenty. + for (let round = 0; round < 3; round += 1) { + const refusal = await readSponsorRefusal(response); + if (refusal?.kind !== 'confirm') return response; + const agreed = await ask(refusal.confirmation); + if (!agreed) return null; + keys = Array.from(new Set([...keys, ...agreed])); + response = await send(keys); + } + return response; +} + +/** + * A MultiSelect's options plus one for each selected id the caller can't + * list (the owner's private tool or source on a workflow node), so it can + * be removed. Selecting only removes it; nothing opens. + * + * Args: + * options: The caller's own options. + * attached: Known names of referenced resources, by id. + * selected: The ids selected now. + * format: Turns a name into the option's label. + */ +export function withAttachedOptions( + options: { value: string; label: string }[], + attached: { id: string; label: string }[], + selected: string[], + format: (name: string) => string, +): { value: string; label: string }[] { + const listed = new Set(options.map((option) => option.value)); + const names = new Map(attached.map((item) => [item.id, item.label])); + const extra = selected + .filter((id, index) => !listed.has(id) && selected.indexOf(id) === index) + .map((id) => ({ value: id, label: format(names.get(id) || id) })); + return extra.length ? [...options, ...extra] : options; +} + +/** The error for a save that attached resources the caller may not sponsor. */ +export function sponsorNotAllowedMessage( + t: TFunction, + language: string, + resources: SponsorResource[], +): string { + const names = new Intl.ListFormat(intlLocale(language), { + type: 'conjunction', + }).format( + resources.map((item) => item.name || t('agents.form.sponsors.unknownItem')), + ); + return t('agents.form.sponsors.notAllowed', { + names, + interpolation: { escapeValue: false }, + }); +} + +/** + * The tool picker's rows plus one per attached tool the caller can't list. + * + * An editor of someone else's agent doesn't see the owner's private tools in + * their own list, so without a row of its own such a tool reads as selected + * with nothing to switch it off. The row only removes it (and puts it back + * before saving); it offers nothing to open or configure. + */ +export function withAttachedToolRows( + items: MultiSelectPopoverItem[], + attached: ToolSummary[], + labels: { group: string; description: string }, +): MultiSelectPopoverItem[] { + const seen = new Set(items.map((item) => item.id)); + const extra: MultiSelectPopoverItem[] = []; + for (const tool of attached) { + if (!tool?.id || seen.has(tool.id)) continue; + seen.add(tool.id); + extra.push({ + id: tool.id, + label: getToolDisplayName(tool), + icon: , + group: labels.group, + description: labels.description, + }); + } + return extra.length ? [...items, ...extra] : items; +} + +/** + * Ask the caller to run one stopped item with their access, naming who + * reaches it through the agent (the read's `sponsor_audience`). + * + * Args: + * ask: Shows the confirmation; resolves to the agreed keys, or null. + * item: The stopped item. + * name: Its display name. + * audience: Who reaches the agent's resources. + * + * Returns: + * Whether they agreed; the next save then sends its key. + */ +export async function confirmTakeOver( + ask: (confirmation: SponsorConfirmation) => Promise, + item: Pick, + name: string, + audience: SponsorAudience | undefined, +): Promise { + const agreed = await ask({ + resources: [{ key: item.key, type: item.type, id: item.id, name }], + audience: audience ?? NO_AUDIENCE, + }); + return Boolean(agreed?.includes(item.key)); +} diff --git a/frontend/src/agents/types/index.ts b/frontend/src/agents/types/index.ts index 8b9cd82b..236c8e21 100644 --- a/frontend/src/agents/types/index.ts +++ b/frontend/src/agents/types/index.ts @@ -1,3 +1,4 @@ +import type { SponsorAudience } from '../sponsorConsent'; import type { AccessFields } from '../../utils/accessUtils'; export type ToolSummary = { @@ -6,15 +7,98 @@ export type ToolSummary = { display_name: string; }; +/** Why a sponsored item stopped running (`ResourceSponsor.reason`). */ +export type ResourceSponsorReason = + 'sponsor_cannot_edit_agent' | 'sponsor_cannot_edit_resource'; + /** A tool, source or prompt that runs with the editor's access who added it. */ export type ResourceSponsor = { + /** `":"`, the value `confirm_sponsor` takes. */ + key?: string; type: 'tool' | 'source' | 'prompt'; id: string; - user_id: string; + /** The item's name, looked up whoever owns it. */ + name?: string | null; + /** Null, with `label`, for someone the reader doesn't know. */ + user_id: string | null; /** The person's email when on file, else their user id. */ - label: string; - /** False once that person can no longer edit the agent or use the item. */ + label: string | null; + state?: 'active' | 'inactive'; + /** Null while it runs; else whether the person lost the agent or the item. */ + reason?: ResourceSponsorReason | null; + /** False once that person can no longer edit the agent, or own or edit the item. */ active: boolean; + /** The reader may take a stopped item over by confirming it on a save. */ + can_confirm?: boolean; +}; + +/** Why an attached item doesn't run (`ResourceState.reason`). */ +export type ResourceStateReason = + | 'deleted' + | 'owner_lost_access' + | ResourceSponsorReason + | 'connection_needs_reconnect' + | 'connection_removed' + | 'connector_disabled'; + +/** Something to know about an item that runs (`ResourceState.note`). */ +export type ResourceStateNote = 'per_user_account'; + +/** + * A person the page names: their email when on file, else their user id. + * Both are null for someone the reader doesn't know. + */ +export type ResourcePerson = { user_id: string | null; label: string | null }; + +/** + * Whether an attached tool, source or prompt runs (`resource_states` on the + * agent and workflow reads, for people who may edit them). + */ +export type ResourceState = { + /** `":"`, the value `confirm_sponsor` takes. */ + key: string; + type: 'tool' | 'source' | 'prompt'; + id: string; + /** Null when the reader may not see it. */ + name?: string | null; + state: 'active' | 'stopped'; + /** Null while it runs. */ + reason: ResourceStateReason | null; + /** `per_user_account`: it runs on each person's own account. */ + note?: ResourceStateNote | null; + /** Who it ran with the access of, when someone else added it. */ + sponsor?: ResourcePerson | null; + /** Someone other than the reader who can fix it, when the reader knows them. */ + contact?: ResourcePerson | null; + /** Who can fix it (`resource_owner`), named or not. */ + contact_role?: 'resource_owner' | null; + /** + * The service of a connected tool, or of one a connection reason stopped. + * `id` only when the reader may reconnect + * it; the account's own name only for its owner. + */ + connection?: { + id: string | null; + connector_key: string | null; + name: string | null; + } | null; + /** The live sponsor a running item runs as; null for the owner. */ + runs_as?: ResourcePerson | null; + /** A running connected tool's mode: the owner's account or each person's own. */ + credential_mode?: 'owner' | 'member' | null; + /** + * Whose saved credentials or owner-mode connection a running tool uses + * (the tool's owner); both null when the reader may not see who. + */ + account?: ResourcePerson | null; + /** Its write actions on credentials its owner stored (the API write allowlist's). */ + owner_credential_writes?: string[]; + /** False when an admin turned off changes through its connector. */ + writes_allowed?: boolean; + /** The reader may run it with their access by confirming on a save. */ + can_confirm?: boolean; + /** The reader owns the connection that needs signing in again. */ + can_reconnect?: boolean; }; export type Agent = { @@ -55,6 +139,12 @@ export type Agent = { // Resources the owner can't use that run as the editor who attached them // (GET /api/get_agent, callers who may view the config). resource_sponsors?: ResourceSponsor[]; + // Whether each attached tool, source and prompt runs, and why not + // (GET /api/get_agent, callers who may edit the agent). + resource_states?: ResourceState[]; + // Who reaches the agent's resources, sent when the caller may take one + // over, so the take-over confirmation can say who it extends to. + sponsor_audience?: SponsorAudience; created_at?: string; updated_at?: string; last_used_at?: string; @@ -96,6 +186,8 @@ export type GuardrailsConfig = { export type AgentConfig = { guardrails?: GuardrailsConfig; + /** `tool_id:action` writes on the owner's accounts API-key callers may run. */ + api_write_allowlist?: string[]; }; export type GuardrailCheckInfo = { diff --git a/frontend/src/agents/types/schedule.ts b/frontend/src/agents/types/schedule.ts index 02241354..f277d97c 100644 --- a/frontend/src/agents/types/schedule.ts +++ b/frontend/src/agents/types/schedule.ts @@ -37,7 +37,7 @@ export type Schedule = { model_id?: string | null; token_budget?: number | null; origin_conversation_id?: string | null; - created_via: 'chat' | 'ui'; + created_via: 'chat' | 'ui' | 'api'; consecutive_failure_count: number; created_at: string; updated_at: string; diff --git a/frontend/src/agents/useAgentResourceStates.ts b/frontend/src/agents/useAgentResourceStates.ts new file mode 100644 index 00000000..75089db4 --- /dev/null +++ b/frontend/src/agents/useAgentResourceStates.ts @@ -0,0 +1,75 @@ +import { useEffect, useState } from 'react'; +import { useSelector } from 'react-redux'; + +import userService from '../api/services/userService'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { Agent, ResourceState } from './types'; + +/** The agent as read, and every tool, source and prompt it runs. */ +export type AgentResources = { agent: Agent; items: ResourceState[] }; + +/** The agent's own items, then its workflow nodes' ones it doesn't have. */ +export function mergeStates( + own: ResourceState[], + nodes: ResourceState[], +): ResourceState[] { + const seen = new Set(own.map((item) => item.key.toLowerCase())); + return [...own, ...nodes.filter((item) => !seen.has(item.key.toLowerCase()))]; +} + +/** + * Reads an agent and the run state of what it uses: `resource_states` from + * the agent read, plus the workflow read's for a workflow agent's node + * tools and sources. Only owners and editors get the states; everyone else + * gets an empty list. + * + * Args: + * agentId: The agent to read; nothing is read without one. + * reloadKey: Read again when this changes (the agent's saved tools). + * + * Returns: + * The agent and its items, or null while loading or when the agent + * can't be read. A failed workflow read leaves the agent's own items. + */ +export default function useAgentResourceStates( + agentId: string | undefined, + reloadKey = '', +): AgentResources | null { + const token = useSelector(selectToken); + const [loaded, setLoaded] = useState(null); + + useEffect(() => { + let cancelled = false; + setLoaded(null); + if (!agentId) return; + const load = async () => { + let agent: Agent; + try { + const response = await userService.getAgent(agentId, token); + if (!response.ok) return; + agent = await response.json(); + } catch { + return; + } + let items = agent.resource_states ?? []; + if (agent.agent_type === 'workflow' && agent.workflow) { + try { + const response = await userService.getWorkflow(agent.workflow, token); + if (response.ok) { + const body = await response.json(); + items = mergeStates(items, body?.data?.resource_states ?? []); + } + } catch { + // The agent's own items still show. + } + } + if (!cancelled) setLoaded({ agent, items }); + }; + void load(); + return () => { + cancelled = true; + }; + }, [agentId, reloadKey, token]); + + return loaded; +} diff --git a/frontend/src/agents/useSponsorPrompt.test.tsx b/frontend/src/agents/useSponsorPrompt.test.tsx new file mode 100644 index 00000000..d41cb026 --- /dev/null +++ b/frontend/src/agents/useSponsorPrompt.test.tsx @@ -0,0 +1,108 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('../hooks', () => ({ + useMediaQuery: () => ({ isMobile: false, isDesktop: true }), +})); + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string) => key, + i18n: { language: 'en' }, + }), +})); + +import { saveWithSponsorConsent } from './sponsorConsent'; +import { useSponsorPrompt } from './useSponsorPrompt'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const json = (status: number, body: unknown) => + new Response(JSON.stringify(body), { status }); + +const refused = () => + json(409, { + code: 'sponsor_confirmation_required', + resources: [{ key: 'tool:t1', type: 'tool', id: 't1', name: 'Jira' }], + audience: { teams: ['Ops'], api_key: false, public_link: false }, + }); + +/** A workflow save as the builder does it: one call, one outcome. */ +function Harness({ + send, + onOutcome, +}: { + send: (keys: string[]) => Promise; + onOutcome: (outcome: string) => void; +}) { + const prompt = useSponsorPrompt(); + const save = async () => { + const response = await saveWithSponsorConsent(send, prompt.ask); + onOutcome(!response ? 'cancelled' : response.ok ? 'saved' : 'failed'); + }; + return ( + <> + + + ); + } + + return ( +
+
+ {children} +
+ setOpen(false)} + label={t('agents.close')} + icon={X} + /> +
+
+
+ ); +} diff --git a/frontend/src/agents/workflow/nodeResources.test.ts b/frontend/src/agents/workflow/nodeResources.test.ts new file mode 100644 index 00000000..70856dcc --- /dev/null +++ b/frontend/src/agents/workflow/nodeResources.test.ts @@ -0,0 +1,62 @@ +import type { Node } from 'reactflow'; + +import type { ResourceState } from '../types'; +import { + nodeResourceIds, + stoppedNodeResources, + withoutNodeResource, +} from './nodeResources'; + +const agentNode = (id: string, config: Record): Node => ({ + id, + type: 'agent', + position: { x: 0, y: 0 }, + data: { title: id, config }, +}); + +const nodes: Node[] = [ + { id: 'start', type: 'start', position: { x: 0, y: 0 }, data: {} }, + agentNode('a1', { tools: ['T1', 't2'], sources: ['s1'] }), + agentNode('a2', { tools: ['t1'], sources: [] }), +]; + +const state = (over: Partial): ResourceState => ({ + key: 'tool:t1', + type: 'tool', + id: 't1', + state: 'stopped', + reason: 'deleted', + ...over, +}); + +describe('nodeResources', () => { + it('collects every agent node tool and source, lowercased', () => { + const ids = nodeResourceIds(nodes); + expect([...ids.tool].sort()).toEqual(['t1', 't2']); + expect([...ids.source]).toEqual(['s1']); + }); + + it('keeps only stopped items still on the canvas', () => { + const states = [ + state({}), + state({ key: 'tool:t2', id: 't2', state: 'active', reason: null }), + state({ key: 'source:gone', type: 'source', id: 'gone' }), + ]; + expect( + stoppedNodeResources(states, nodeResourceIds(nodes)).map((s) => s.key), + ).toEqual(['tool:t1']); + }); + + it('takes a tool off every agent node and leaves the rest alone', () => { + const next = withoutNodeResource(nodes, { type: 'tool', id: 't1' }); + expect(next[1].data.config.tools).toEqual(['t2']); + expect(next[2].data.config.tools).toEqual([]); + expect(next[0]).toBe(nodes[0]); + expect(next[1].data.config.sources).toEqual(['s1']); + expect(nodeResourceIds(next).tool.has('t1')).toBe(false); + }); + + it('leaves nodes untouched for a prompt', () => { + expect(withoutNodeResource(nodes, { type: 'prompt', id: 'p' })).toBe(nodes); + }); +}); diff --git a/frontend/src/agents/workflow/nodeResources.ts b/frontend/src/agents/workflow/nodeResources.ts new file mode 100644 index 00000000..e6ea2a83 --- /dev/null +++ b/frontend/src/agents/workflow/nodeResources.ts @@ -0,0 +1,85 @@ +import type { Node } from 'reactflow'; + +import type { ResourceState } from '../types'; + +/** Tool and source ids a workflow's agent nodes reference, lowercased. */ +export type NodeResourceIds = { tool: Set; source: Set }; + +const KEYS = { tool: 'tools', source: 'sources' } as const; + +function idsOf(value: unknown): string[] { + if (Array.isArray(value)) return value.map(String); + return value ? [String(value)] : []; +} + +/** + * The tool and source ids the agent nodes reference now. + * + * Args: + * nodes: The canvas nodes (an agent node keeps them in `data.config`). + */ +export function nodeResourceIds(nodes: Node[]): NodeResourceIds { + const ids: NodeResourceIds = { tool: new Set(), source: new Set() }; + for (const node of nodes) { + if (node.type !== 'agent') continue; + const config = (node.data?.config ?? {}) as Record; + for (const type of ['tool', 'source'] as const) + idsOf(config[KEYS[type]]).forEach((id) => + ids[type].add(id.toLowerCase()), + ); + } + return ids; +} + +/** + * Stopped node resources still on the canvas, so one removed here (not yet + * saved) leaves the notice. + * + * Args: + * states: `resource_states` from the workflow read. + * ids: What the nodes reference now. + */ +export function stoppedNodeResources( + states: ResourceState[], + ids: NodeResourceIds, +): ResourceState[] { + return states.filter( + (item) => + item.state === 'stopped' && + item.type !== 'prompt' && + ids[item.type].has(item.id.toLowerCase()), + ); +} + +/** + * The nodes with one tool or source taken off every agent node. + * + * Args: + * nodes: The canvas nodes. + * item: The tool or source to take off. + */ +export function withoutNodeResource( + nodes: Node[], + item: Pick, +): Node[] { + if (item.type === 'prompt') return nodes; + const key = KEYS[item.type]; + const id = item.id.toLowerCase(); + return nodes.map((node) => { + const config = node.data?.config; + if (node.type !== 'agent' || !config || config[key] === undefined) + return node; + const kept = idsOf(config[key]).filter((ref) => ref.toLowerCase() !== id); + if (kept.length === idsOf(config[key]).length) return node; + return { + ...node, + data: { + ...node.data, + config: { + ...config, + [key]: Array.isArray(config[key]) ? kept : (kept[0] ?? ''), + }, + }, + }; + }); +} diff --git a/frontend/src/agents/workflow/panels/AgentPanel.attached.test.tsx b/frontend/src/agents/workflow/panels/AgentPanel.attached.test.tsx new file mode 100644 index 00000000..73c0bfa2 --- /dev/null +++ b/frontend/src/agents/workflow/panels/AgentPanel.attached.test.tsx @@ -0,0 +1,119 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import type { Node } from 'reactflow'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'name' in opts ? `${key}:${opts.name}` : key, + }), +})); + +const pickers = vi.hoisted( + () => [] as { options: { value: string; label: string }[] }[], +); +vi.mock('@/components/ui/multi-select', () => ({ + MultiSelect: (props: { + options: { value: string; label: string }[]; + selected: string[]; + onChange: (next: string[]) => void; + }) => { + pickers.push(props); + return ( +
+ {props.options.map((option) => ( + + ))} +
+ ); + }, +})); +vi.mock('../components/PromptTextArea', () => ({ + default: () => null, + extractUpstreamVariables: () => [], +})); +vi.mock('../components/NodeDocumentsControl', () => ({ default: () => null })); + +import AgentPanel from './AgentPanel'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +describe('AgentPanel attached resources', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + pickers.length = 0; + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const node = { + id: 'a1', + type: 'agent', + position: { x: 0, y: 0 }, + data: { config: { tools: ['mine', 'owners'], sources: ['owner-src'] } }, + } as Node; + + const render = async (onUpdate = vi.fn()) => { + await act(async () => { + root.render( + , + ); + }); + return onUpdate; + }; + + // An editor's pickers don't list the owner's private tools and sources; + // those still need a row so they can be taken off the node. + it('adds remove-only options for attached tools and sources', async () => { + const onUpdate = await render(); + const labels = pickers.flatMap((p) => p.options.map((o) => o.label)); + expect(labels).toContain( + 'agents.workflow.builder.attachedOption:Owner Jira', + ); + expect(labels).toContain( + 'agents.workflow.builder.attachedOption:Owner docs', + ); + await act(async () => + container + .querySelector('[data-option="owners"]')! + .click(), + ); + expect(onUpdate).toHaveBeenCalledWith({ + config: { tools: ['mine'], sources: ['owner-src'] }, + }); + }); +}); diff --git a/frontend/src/agents/workflow/panels/AgentPanel.tsx b/frontend/src/agents/workflow/panels/AgentPanel.tsx index b7ddfc89..2004432a 100644 --- a/frontend/src/agents/workflow/panels/AgentPanel.tsx +++ b/frontend/src/agents/workflow/panels/AgentPanel.tsx @@ -23,6 +23,7 @@ import { Textarea } from '@/components/ui/textarea'; import { cn } from '@/lib/utils'; import { getToolDisplayName } from '../../../utils/toolUtils'; +import { withAttachedOptions } from '../../sponsorConsent'; import NodeDocumentsControl from '../components/NodeDocumentsControl'; import PromptTextArea from '../components/PromptTextArea'; import { @@ -42,6 +43,13 @@ interface AgentPanelProps extends NodePanelBodyProps { availableModels: Model[]; availableTools: UserTool[]; sourceOptions: { value: string; label: string }[]; + /** + * Names of the tools and sources the workflow's nodes reference, whoever + * owns them: a selected one missing from the caller's own options (the + * owner's private tool) still gets an option, so it can be removed. + */ + attachedTools?: { id: string; label: string }[]; + attachedSources?: { id: string; label: string }[]; /** Upstream file variables the agent can read. */ documentOptions: { value: string; label: string }[]; /** The structured-output schema as typed. */ @@ -101,6 +109,8 @@ export default function AgentPanel({ availableModels, availableTools, sourceOptions, + attachedTools = [], + attachedSources = [], documentOptions, jsonSchemaText, jsonSchemaError, @@ -120,6 +130,23 @@ export default function AgentPanel({ const builtinModels = availableModels.filter((m) => m.source !== 'user'); const userModels = availableModels.filter((m) => m.source === 'user'); const schemaInvalid = jsonSchemaText.trim() !== '' && jsonSchemaError; + const attachedLabel = (name: string) => + t('agents.workflow.builder.attachedOption', { ...NO_ESCAPE, name }); + const toolOptions = withAttachedOptions( + availableTools.map((tool) => ({ + value: tool.id, + label: getToolDisplayName(tool), + })), + attachedTools, + config.tools || [], + attachedLabel, + ); + const nodeSourceOptions = withAttachedOptions( + sourceOptions, + attachedSources, + config.sources || [], + attachedLabel, + ); return (
@@ -248,7 +275,7 @@ export default function AgentPanel({ labelSurface="background" > updateConfig({ sources: newSources })} placeholder={t('agents.form.placeholders.selectSources')} @@ -261,10 +288,7 @@ export default function AgentPanel({ labelSurface="background" > ({ - value: tool.id, - label: getToolDisplayName(tool), - }))} + options={toolOptions} selected={config.tools || []} onChange={(newTools) => updateConfig({ tools: newTools })} placeholder={t('agents.form.placeholders.selectTools')} diff --git a/frontend/src/api/endpoints.ts b/frontend/src/api/endpoints.ts index 9d1a4c83..5eae77de 100644 --- a/frontend/src/api/endpoints.ts +++ b/frontend/src/api/endpoints.ts @@ -88,6 +88,7 @@ const endpoints = { WIKI_PAGES: (id: string) => `/api/sources/${id}/wiki/pages`, WIKI_PAGE: (id: string, path: string) => `/api/sources/${id}/wiki/page?path=${encodeURIComponent(path)}`, + WIKI_SETTINGS: (id: string) => `/api/sources/${id}/wiki/settings`, GET_AVAILABLE_TOOLS: '/api/available_tools', GET_USER_TOOLS: '/api/get_tools', CREATE_TOOL: '/api/create_tool', @@ -96,11 +97,44 @@ const endpoints = { DELETE_TOOL: '/api/delete_tool', PARSE_SPEC: '/api/parse_spec', SYNC_CONNECTOR: '/api/connectors/sync', - CONNECTOR_AUTH: (provider: string) => - `/api/connectors/auth?provider=${provider}`, + CONNECTOR_AUTH: ( + provider: string, + connectionId?: string, + install?: boolean, + ) => + `/api/connectors/auth?provider=${encodeURIComponent(provider)}${ + connectionId ? `&connection_id=${encodeURIComponent(connectionId)}` : '' + }${install ? '&install=1' : ''}`, CONNECTOR_FILES: '/api/connectors/files', CONNECTOR_VALIDATE_SESSION: '/api/connectors/validate-session', CONNECTOR_DISCONNECT: '/api/connectors/disconnect', + CONNECTORS_CATALOG: '/api/connectors/catalog', + CONNECTIONS: '/api/connections', + CONNECTION: (id: string) => `/api/connections/${encodeURIComponent(id)}`, + CONNECTION_DISCONNECT: (id: string) => + `/api/connections/${encodeURIComponent(id)}/disconnect`, + CONNECTION_SETUP: (id: string) => + `/api/connections/${encodeURIComponent(id)}/setup`, + CONNECTION_RECONNECT: (id: string) => + `/api/connections/${encodeURIComponent(id)}/reconnect`, + CONNECTION_PICKER_TOKEN: (id: string) => + `/api/connections/${encodeURIComponent(id)}/picker-token`, + CONNECTION_REFRESH_TOOLS: (id: string) => + `/api/connections/${encodeURIComponent(id)}/refresh-tools`, + CONNECTION_TOOL_PERMISSIONS: (id: string, toolId: string) => + `/api/connections/${encodeURIComponent(id)}/tools/${encodeURIComponent(toolId)}/permissions`, + CONNECTION_REPOSITORIES: (id: string) => + `/api/connections/${encodeURIComponent(id)}/repositories`, + CONNECTION_LINEAR: (id: string) => + `/api/connections/${encodeURIComponent(id)}/linear`, + CONNECTION_TOOL_PARAMETERS: (id: string, toolId: string) => + `/api/connections/${encodeURIComponent(id)}/tools/${encodeURIComponent(toolId)}/parameters`, + CONNECTION_WRITES: (id: string) => + `/api/connections/${encodeURIComponent(id)}/writes`, + CONNECTIONS_CLAIM: '/api/connections/claim', + TOOL_CREDENTIAL_MODE: (toolId: string) => + `/api/connections/tools/${encodeURIComponent(toolId)}/credential-mode`, + ADMIN_CONNECTORS: '/api/admin/connectors', GET_CHUNKS: ( docId: string, page: number, diff --git a/frontend/src/api/services/connectorsService.ts b/frontend/src/api/services/connectorsService.ts new file mode 100644 index 00000000..d310e563 --- /dev/null +++ b/frontend/src/api/services/connectorsService.ts @@ -0,0 +1,175 @@ +import type { SourceConfig } from '../../models/misc'; +import apiClient, { throttledApiClient } from '../client'; +import endpoints from '../endpoints'; + +// apiClient resolves to the raw fetch Response; parse it here so the slice and +// components get plain data. A non-2xx body still parses (``success: false``). +const json = async (response: Response) => { + try { + return await response.json(); + } catch { + return { success: false }; + } +}; + +export type ConnectionSetupBody = { + create_tools?: boolean; + /** GitHub: let agents make changes, not only read. */ + allow_writes?: boolean; + tool_permissions?: Record; + sync?: { + items: Record; + frequency?: string; + name?: string; + /** The source's retrieval settings, as an upload sends them. */ + config?: SourceConfig; + }; +}; + +export type RemoveConnectionBody = { + sources: 'keep' | 'delete'; + tools: 'keep' | 'delete'; +}; + +const connectorsService = { + getCatalog: async (token: string | null) => + json( + await throttledApiClient.get(endpoints.USER.CONNECTORS_CATALOG, token), + ), + listConnections: async (token: string | null) => + json(await throttledApiClient.get(endpoints.USER.CONNECTIONS, token)), + getConnection: async (id: string, token: string | null) => + json(await apiClient.get(endpoints.USER.CONNECTION(id), token)), + createConnection: async ( + body: { + connector_key: string; + credentials: Record; + label?: string; + }, + token: string | null, + ) => json(await apiClient.post(endpoints.USER.CONNECTIONS, body, token)), + setup: async ( + id: string, + body: ConnectionSetupBody, + token: string | null, + idempotencyKey?: string, + ) => + json( + await apiClient.post( + endpoints.USER.CONNECTION_SETUP(id), + body, + token, + idempotencyKey ? { 'Idempotency-Key': idempotencyKey } : {}, + ), + ), + reconnect: async ( + id: string, + body: { credentials?: Record }, + token: string | null, + ) => + json( + await apiClient.post( + endpoints.USER.CONNECTION_RECONNECT(id), + body, + token, + ), + ), + /** Name an account ("Alerts bot"); an empty name clears it. */ + renameConnection: async (id: string, name: string, token: string | null) => + json(await apiClient.patch(endpoints.USER.CONNECTION(id), { name }, token)), + disconnect: async (id: string, token: string | null) => + json( + await apiClient.post(endpoints.USER.CONNECTION_DISCONNECT(id), {}, token), + ), + remove: async ( + id: string, + body: RemoveConnectionBody, + token: string | null, + ) => json(await apiClient.delete(endpoints.USER.CONNECTION(id), token, body)), + pickerToken: async (id: string, token: string | null) => + json( + await apiClient.post( + endpoints.USER.CONNECTION_PICKER_TOKEN(id), + {}, + token, + ), + ), + repositories: async (id: string, token: string | null) => + json( + await apiClient.get(endpoints.USER.CONNECTION_REPOSITORIES(id), token), + ), + /** Linear: the teams and projects a connection can sync. */ + linearWorkspace: async (id: string, token: string | null) => + json(await apiClient.get(endpoints.USER.CONNECTION_LINEAR(id), token)), + refreshTools: async (id: string, token: string | null) => + json( + await apiClient.post( + endpoints.USER.CONNECTION_REFRESH_TOOLS(id), + {}, + token, + ), + ), + /** GitHub: let agents make changes through a connection, or only read. */ + setWrites: async (id: string, allow: boolean, token: string | null) => + json( + await apiClient.put( + endpoints.USER.CONNECTION_WRITES(id), + { allow }, + token, + ), + ), + setToolPermissions: async ( + id: string, + toolId: string, + permissions: Record, + token: string | null, + ) => + json( + await apiClient.put( + endpoints.USER.CONNECTION_TOOL_PERMISSIONS(id, toolId), + { permissions }, + token, + ), + ), + /** Fix parameters to a value (`null` lets the AI decide again). */ + setToolParameters: async ( + id: string, + toolId: string, + action: string, + parameters: Record, + token: string | null, + ) => + json( + await apiClient.put( + endpoints.USER.CONNECTION_TOOL_PARAMETERS(id, toolId), + { action, parameters }, + token, + ), + ), + setCredentialMode: async ( + toolId: string, + mode: 'owner' | 'member', + token: string | null, + ) => + json( + await apiClient.put( + endpoints.USER.TOOL_CREDENTIAL_MODE(toolId), + { mode }, + token, + ), + ), + getAdmin: async (token: string | null) => + json(await apiClient.get(endpoints.USER.ADMIN_CONNECTORS, token)), + updateAdmin: async (body: Record, token: string | null) => + json(await apiClient.put(endpoints.USER.ADMIN_CONNECTORS, body, token)), + claim: async (provider: string, sessionToken: string, token: string | null) => + json( + await apiClient.post( + endpoints.USER.CONNECTIONS_CLAIM, + { provider, session_token: sessionToken }, + token, + ), + ), +}; + +export default connectorsService; diff --git a/frontend/src/api/services/userService.ts b/frontend/src/api/services/userService.ts index 46062aa6..cb075a7b 100644 --- a/frontend/src/api/services/userService.ts +++ b/frontend/src/api/services/userService.ts @@ -1,4 +1,3 @@ -import { getSessionToken } from '../../utils/providerUtils'; import apiClient, { throttledApiClient } from '../client'; import endpoints from '../endpoints'; @@ -187,6 +186,17 @@ const userService = { token: string | null, ): Promise => apiClient.put(endpoints.USER.WIKI_PAGE(sourceId, data.path), data, token), + getWikiSettings: ( + sourceId: string, + token: string | null, + ): Promise => + apiClient.get(endpoints.USER.WIKI_SETTINGS(sourceId), token), + updateWikiSettings: ( + sourceId: string, + data: { allow_outside_edits: boolean }, + token: string | null, + ): Promise => + apiClient.put(endpoints.USER.WIKI_SETTINGS(sourceId), data, token), getAvailableTools: (token: string | null): Promise => apiClient.get(endpoints.USER.GET_AVAILABLE_TOOLS, token), getUserTools: (token: string | null): Promise => @@ -286,24 +296,19 @@ const userService = { apiClient.post(endpoints.USER.MCP_SAVE_SERVER, data, token), getMCPAuthStatus: (token: string | null): Promise => throttledApiClient.get(endpoints.USER.MCP_AUTH_STATUS, token), - syncConnector: ( - docId: string, + // The source's own connection syncs it; no browser token is involved. + syncConnector: (docId: string, token: string | null): Promise => + apiClient.post(endpoints.USER.SYNC_CONNECTOR, { source_id: docId }, token), + getConnectorAuthUrl: ( provider: string, token: string | null, - ): Promise => { - const sessionToken = getSessionToken(provider); - return apiClient.post( - endpoints.USER.SYNC_CONNECTOR, - { - source_id: docId, - session_token: sessionToken, - provider: provider, - }, + connectionId?: string, + install?: boolean, + ): Promise => + apiClient.get( + endpoints.USER.CONNECTOR_AUTH(provider, connectionId, install), token, - ); - }, - getConnectorAuthUrl: (provider: string, token: string | null): Promise => - apiClient.get(endpoints.USER.CONNECTOR_AUTH(provider), token), + ), getConnectorFiles: ( data: any, token: string | null, @@ -316,28 +321,6 @@ const userService = { {}, signal, ), - validateConnectorSession: ( - provider: string, - token: string | null, - ): Promise => - apiClient.post( - endpoints.USER.CONNECTOR_VALIDATE_SESSION, - { - provider, - session_token: getSessionToken(provider), - }, - token, - ), - disconnectConnector: ( - provider: string, - sessionToken: string, - token: string | null, - ): Promise => - apiClient.post( - endpoints.USER.CONNECTOR_DISCONNECT, - { provider, session_token: sessionToken }, - token, - ), textToSpeech: ( text: string, token: string | null, diff --git a/frontend/src/assets/confluence.svg b/frontend/src/assets/confluence.svg index 183807ad..34631723 100644 --- a/frontend/src/assets/confluence.svg +++ b/frontend/src/assets/confluence.svg @@ -1,4 +1,4 @@ - + diff --git a/frontend/src/assets/connectors/asana.svg b/frontend/src/assets/connectors/asana.svg new file mode 100644 index 00000000..3b2ba44c --- /dev/null +++ b/frontend/src/assets/connectors/asana.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/connectors/atlassian.svg b/frontend/src/assets/connectors/atlassian.svg new file mode 100644 index 00000000..76d35504 --- /dev/null +++ b/frontend/src/assets/connectors/atlassian.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/connectors/github.svg b/frontend/src/assets/connectors/github.svg new file mode 100644 index 00000000..ffa82ef8 --- /dev/null +++ b/frontend/src/assets/connectors/github.svg @@ -0,0 +1 @@ + diff --git a/frontend/src/assets/connectors/linear.svg b/frontend/src/assets/connectors/linear.svg new file mode 100644 index 00000000..583413fe --- /dev/null +++ b/frontend/src/assets/connectors/linear.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/connectors/notion.svg b/frontend/src/assets/connectors/notion.svg new file mode 100644 index 00000000..5a8188cb --- /dev/null +++ b/frontend/src/assets/connectors/notion.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/connectors/sentry.svg b/frontend/src/assets/connectors/sentry.svg new file mode 100644 index 00000000..4f08902e --- /dev/null +++ b/frontend/src/assets/connectors/sentry.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/connectors/stripe.svg b/frontend/src/assets/connectors/stripe.svg new file mode 100644 index 00000000..3ffcab80 --- /dev/null +++ b/frontend/src/assets/connectors/stripe.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/frontend/src/assets/drive.svg b/frontend/src/assets/drive.svg index 8ec4dc9b..ea0c1607 100644 --- a/frontend/src/assets/drive.svg +++ b/frontend/src/assets/drive.svg @@ -1,3 +1,3 @@ - + diff --git a/frontend/src/assets/reddit.svg b/frontend/src/assets/reddit.svg index 705ef638..61d78727 100644 --- a/frontend/src/assets/reddit.svg +++ b/frontend/src/assets/reddit.svg @@ -1,4 +1,4 @@ - - + + diff --git a/frontend/src/assets/s3.svg b/frontend/src/assets/s3.svg index 3fdc41ff..c40cb06f 100644 --- a/frontend/src/assets/s3.svg +++ b/frontend/src/assets/s3.svg @@ -1,7 +1,7 @@ - - - - - + + + + + diff --git a/frontend/src/assets/sharepoint.svg b/frontend/src/assets/sharepoint.svg index 9a332f8e..8aa88c99 100644 --- a/frontend/src/assets/sharepoint.svg +++ b/frontend/src/assets/sharepoint.svg @@ -1,6 +1,6 @@ - + diff --git a/frontend/src/components/ConnectorAuth.tsx b/frontend/src/components/ConnectorAuth.tsx index 0f25a5d1..b3e03a80 100644 --- a/frontend/src/components/ConnectorAuth.tsx +++ b/frontend/src/components/ConnectorAuth.tsx @@ -9,27 +9,41 @@ import { isTrustedConnectorMessage } from '../utils/connectorAuthUtils'; import { Alert, AlertDescription } from './ui/alert'; import { Button } from './ui/button'; -interface ConnectorAuthProps { +interface ConnectorAuthOptions { provider: string; - onSuccess: (data: { session_token: string; user_email: string }) => void; + /** Reconnect this connection (the same account) rather than add one. */ + connectionId?: string; + /** + * GitHub: open the GitHub App's installation page (where the repositories + * it can read are chosen) instead of the sign-in page. + */ + install?: boolean; + onSuccess: (data: { connection_id: string; user_email: string }) => void; onError: (error: string) => void; +} + +interface ConnectorAuthProps extends ConnectorAuthOptions { label?: string; + /** The service's glyph on the sign-in button (e.g. a `ConnectorIcon`). */ + icon?: React.ReactNode; isConnected?: boolean; userEmail?: string; onDisconnect?: () => void; errorMessage?: string; } -const ConnectorAuth: React.FC = ({ +/** + * The OAuth pop-up sign-in: returns the function to call from a click. It + * opens the pop-up inside the gesture, points it at the provider and reports + * the connection it created (or an error) once the callback page answers. + */ +export function useConnectorAuth({ provider, + connectionId, + install = false, onSuccess, onError, - label, - isConnected = false, - userEmail = '', - onDisconnect, - errorMessage, -}) => { +}: ConnectorAuthOptions): () => void { const { t } = useTranslation(); const token = useSelector(selectToken); const completedRef = useRef(false); @@ -76,7 +90,7 @@ const ConnectorAuth: React.FC = ({ cleanup(); authWindowRef.current = null; onSuccess({ - session_token: event.data.session_token, + connection_id: event.data.connection_id, user_email: event.data.user_email || t('modals.uploadDoc.connectors.auth.connectedUser'), @@ -120,6 +134,8 @@ const ConnectorAuth: React.FC = ({ const authResponse = await userService.getConnectorAuthUrl( provider, token, + connectionId, + install, ); if (!mountedRef.current) { authWindow.close(); @@ -205,6 +221,29 @@ const ConnectorAuth: React.FC = ({ }; }, []); + return handleAuth; +} + +const ConnectorAuth: React.FC = ({ + provider, + connectionId, + onSuccess, + onError, + label, + icon, + isConnected = false, + userEmail = '', + onDisconnect, + errorMessage, +}) => { + const { t } = useTranslation(); + const handleAuth = useConnectorAuth({ + provider, + connectionId, + onSuccess, + onError, + }); + return ( <> {errorMessage && ( @@ -238,12 +277,7 @@ const ConnectorAuth: React.FC = ({ ) : ( )} diff --git a/frontend/src/components/ConnectorTree.tsx b/frontend/src/components/ConnectorTree.tsx index ffbe2bb2..48186371 100644 --- a/frontend/src/components/ConnectorTree.tsx +++ b/frontend/src/components/ConnectorTree.tsx @@ -93,7 +93,7 @@ const ConnectorTree: React.FC = ({ setSyncProgress(0); try { - const response = await userService.syncConnector(docId, provider, token); + const response = await userService.syncConnector(docId, token); const data = await response.json(); if (data.success) { diff --git a/frontend/src/components/FilePicker.test.tsx b/frontend/src/components/FilePicker.test.tsx index 6ac53d29..0fb425d9 100644 --- a/frontend/src/components/FilePicker.test.tsx +++ b/frontend/src/components/FilePicker.test.tsx @@ -1,21 +1,26 @@ +import { configureStore } from '@reduxjs/toolkit'; import { act } from 'react'; import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; -const validateConnectorSession = vi.fn(); const getConnectorFiles = vi.fn(); vi.mock('../api/services/userService', () => ({ default: { - validateConnectorSession: (...args: unknown[]) => - validateConnectorSession(...args), getConnectorFiles: (...args: unknown[]) => getConnectorFiles(...args), - disconnectConnector: vi.fn(), }, })); -vi.mock('../utils/providerUtils', () => ({ - getSessionToken: () => 'session-token', - setSessionToken: vi.fn(), - removeSessionToken: vi.fn(), +const pickerToken = vi.fn(); +const disconnect = vi.fn(); +vi.mock('../api/services/connectorsService', () => ({ + default: { + pickerToken: (...args: unknown[]) => pickerToken(...args), + disconnect: (...args: unknown[]) => disconnect(...args), + getCatalog: vi.fn().mockResolvedValue({ success: true, connectors: [] }), + listConnections: vi + .fn() + .mockResolvedValue({ success: true, connections: [] }), + }, })); vi.mock('../components/ConnectorAuth', () => ({ default: () => null })); @@ -24,17 +29,38 @@ vi.mock('react-i18next', () => ({ useTranslation: () => ({ t: (key: string) => key }), })); +import connectorsReducer from '../connectors/connectorsSlice'; +import type { Connection } from '../connectors/types'; import { FilePicker } from './FilePicker'; Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); +const connection = (overrides: Partial = {}): Connection => ({ + id: 'conn-1', + connector_key: 'share_point', + name: 'SharePoint', + display_name: 'SharePoint', + icon: 'sharepoint', + account_label: 'lena@meridian.example', + auth_kind: 'oauth', + status: 'connected', + server_url: null, + last_error: null, + created_at: null, + updated_at: null, + last_used_at: null, + source_count: 0, + tool_count: 0, + ...overrides, +}); + describe('FilePicker', () => { let container: HTMLDivElement; let root: Root; beforeEach(() => { - validateConnectorSession.mockReset(); getConnectorFiles.mockReset(); + pickerToken.mockReset(); container = document.createElement('div'); document.body.appendChild(container); root = createRoot(container); @@ -45,43 +71,70 @@ describe('FilePicker', () => { container.remove(); }); - it('shows an expired session as a destructive alert', async () => { - validateConnectorSession.mockResolvedValue({ ok: false }); + const render = async (provider: string, connections: Connection[]) => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + preloadedState: { + connectors: { + enabled: true, + catalog: [], + connections, + loading: false, + loaded: true, + failed: false, + }, + preference: { token: null }, + }, + }); await act(async () => { root.render( - undefined} - />, + + undefined} + /> + , ); }); + }; + + it('shows a connection that needs signing in as a destructive alert', async () => { + getConnectorFiles.mockResolvedValue({ + json: async () => ({ success: false, reconnect: true }), + }); + await render('google_drive', [ + connection({ connector_key: 'google_drive' }), + ]); const alert = container.querySelector('[role="alert"]'); expect(alert?.textContent).toContain('filePicker.sessionExpiredFor'); expect(alert?.querySelector('svg')).not.toBeNull(); }); + it('lists files with the connection id, never a session token', async () => { + getConnectorFiles.mockResolvedValue({ + json: async () => ({ success: true, files: [], next_page_token: null }), + }); + await render('google_drive', [ + connection({ connector_key: 'google_drive' }), + ]); + const body = getConnectorFiles.mock.calls[0][0]; + expect(body.connection_id).toBe('conn-1'); + expect(body).not.toHaveProperty('session_token'); + }); + async function renderSharePoint() { - validateConnectorSession.mockResolvedValue({ - ok: true, - json: async () => ({ - success: true, - user_email: 'lena@meridian.example', - allows_shared_content: true, - }), + pickerToken.mockResolvedValue({ + success: true, + allows_shared_content: true, }); getConnectorFiles.mockResolvedValue({ json: async () => ({ success: true, files: [], next_page_token: null }), }); - await act(async () => { - root.render( - undefined} - />, - ); - }); + await render('share_point', [connection()]); } it('renders the drive switch as underline tabs with the active one marked', async () => { @@ -112,4 +165,22 @@ describe('FilePicker', () => { expect(page?.getAttribute('title')).toBe('filePicker.myFiles'); expect(trail?.querySelector('button[disabled]')).toBeNull(); }); + + it('offers an account switch when several accounts are connected', async () => { + getConnectorFiles.mockResolvedValue({ + json: async () => ({ success: true, files: [], next_page_token: null }), + }); + pickerToken.mockResolvedValue({ success: true }); + await render('share_point', [ + connection(), + connection({ id: 'conn-2', account_label: 'ops@meridian.example' }), + ]); + expect(container.textContent).toContain('filePicker.account'); + }); + + it('shows nothing to browse without a connection', async () => { + await render('confluence', []); + expect(getConnectorFiles).not.toHaveBeenCalled(); + expect(container.querySelector('[data-slot="tabs-list"]')).toBeNull(); + }); }); diff --git a/frontend/src/components/FilePicker.tsx b/frontend/src/components/FilePicker.tsx index f9570bc0..3b36a89b 100644 --- a/frontend/src/components/FilePicker.tsx +++ b/frontend/src/components/FilePicker.tsx @@ -10,12 +10,24 @@ import { useTranslation } from 'react-i18next'; import userService from '../api/services/userService'; import { formatBytes } from '../utils/stringUtils'; import { formatDateTime } from '../utils/dateTimeUtils'; -import { - getSessionToken, - setSessionToken, - removeSessionToken, -} from '../utils/providerUtils'; +import { useDispatch, useSelector } from 'react-redux'; +import connectorsService from '../api/services/connectorsService'; import ConnectorAuth from '../components/ConnectorAuth'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { connectorIconKey } from '../connectors/i18n'; +import { + loadConnectors, + selectConnections, +} from '../connectors/connectorsSlice'; +import type { AppDispatch } from '../store'; +import { FormField } from './ui/form-field'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from './ui/select'; import SearchInput from './SearchInput'; import { Alert, AlertDescription } from './ui/alert'; import { @@ -75,6 +87,15 @@ interface CloudFilePickerProps { selectedFolderIds?: string[], ) => void; onDisconnect?: () => void; + /** Called with the first item's name when the selection goes from empty to one. */ + onFirstPickName?: (name: string) => void; + /** + * The connection (signed-in account) to browse. Left out, the picker uses + * the first connected account for ``provider`` and lets the user switch. + */ + connectionId?: string | null; + /** Reports the account the picker is browsing, so the upload can name it. */ + onConnectionChange?: (connectionId: string | null) => void; provider: string; token: string | null; initialSelectedFiles?: string[]; @@ -84,11 +105,15 @@ interface CloudFilePickerProps { export const FilePicker: React.FC = ({ onSelectionChange, onDisconnect, + onFirstPickName, + connectionId: controlledConnectionId, + onConnectionChange, provider, token, initialSelectedFiles = [], }) => { const { t } = useTranslation(); + const dispatch = useDispatch(); const PROVIDER_CONFIG = { google_drive: { displayName: 'Drive', @@ -113,6 +138,23 @@ export const FilePicker: React.FC = ({ ); }; + const allConnections = useSelector(selectConnections); + const accounts = allConnections.filter( + (connection) => + connection.connector_key === provider && + connection.status === 'connected', + ); + const [chosenConnectionId, setChosenConnectionId] = useState( + null, + ); + const activeConnectionId = + controlledConnectionId ?? + (chosenConnectionId && + accounts.some((account) => account.id === chosenConnectionId) + ? chosenConnectionId + : (accounts[0]?.id ?? null)); + const activeAccount = accounts.find((a) => a.id === activeConnectionId); + const [files, setFiles] = useState([]); const [selectedFiles, setSelectedFiles] = useState(initialSelectedFiles); @@ -131,14 +173,18 @@ export const FilePicker: React.FC = ({ ]); const [searchQuery, setSearchQuery] = useState(''); const [authError, setAuthError] = useState(''); - const [isConnected, setIsConnected] = useState(false); - const [userEmail, setUserEmail] = useState(''); const [allowsSharedContent, setAllowsSharedContent] = useState(false); const [activeTab, setActiveTab] = useState<'my_files' | 'shared'>('my_files'); + const isConnected = !!activeConnectionId; + const scrollContainerRef = useRef(null); const abortControllerRef = useRef(null); + useEffect(() => { + onConnectionChange?.(activeConnectionId); + }, [activeConnectionId]); + const isFolder = (file: CloudFile) => { return ( file.isFolder || @@ -149,7 +195,7 @@ export const FilePicker: React.FC = ({ const loadCloudFiles = useCallback( async ( - sessionToken: string, + connectionId: string, folderId: string | null, pageToken?: string, searchQuery = '', @@ -169,7 +215,7 @@ export const FilePicker: React.FC = ({ try { const body: Record = { provider: provider, - session_token: sessionToken, + connection_id: connectionId, folder_id: folderId, limit: 10, page_token: pageToken, @@ -190,7 +236,14 @@ export const FilePicker: React.FC = ({ setNextPageToken(data.next_page_token); setHasMoreFiles(!!data.next_page_token); } else { - console.error('Error loading files:', data.error); + if (data.reconnect) { + setAuthError( + t('filePicker.sessionExpiredFor', { + provider: getProviderConfig(provider).displayName, + }), + ); + dispatch(loadConnectors({ token })); + } if (!pageToken) { setFiles([]); } @@ -210,72 +263,26 @@ export const FilePicker: React.FC = ({ [token, provider], ); - const validateAndLoadFiles = useCallback(async () => { - const sessionToken = getSessionToken(provider); - if (!sessionToken) { - setIsConnected(false); - return; - } - - try { - const validateResponse = await userService.validateConnectorSession( - provider, - token, - ); - - if (!validateResponse.ok) { - removeSessionToken(provider); - setIsConnected(false); - setAuthError( - t('filePicker.sessionExpiredFor', { - provider: getProviderConfig(provider).displayName, - }), - ); - return; - } - - const validateData = await validateResponse.json(); - if (validateData.success) { - setUserEmail( - validateData.user_email || - t('modals.uploadDoc.connectors.auth.connectedUser'), - ); - setIsConnected(true); - setAuthError(''); - if (provider === 'share_point') { - setAllowsSharedContent(validateData.allows_shared_content ?? false); - } - - setFiles([]); - setNextPageToken(null); - setHasMoreFiles(false); - setCurrentFolderId(null); - setActiveTab('my_files'); - setFolderPath([ - { - id: null, - name: getProviderConfig(provider).rootName, - }, - ]); - loadCloudFiles(sessionToken, null, undefined, ''); - } else { - removeSessionToken(provider); - setIsConnected(false); - setAuthError( - validateData.error || - t('modals.uploadDoc.connectors.googleDrive.sessionExpiredGeneric'), - ); - } - } catch (error) { - console.error('Error validating session:', error); - setAuthError(t('modals.uploadDoc.connectors.googleDrive.validateFailed')); - setIsConnected(false); - } - }, [provider, token, loadCloudFiles]); - + // Browse from the root whenever the account changes. useEffect(() => { - validateAndLoadFiles(); - }, [validateAndLoadFiles]); + setFiles([]); + setNextPageToken(null); + setHasMoreFiles(false); + setCurrentFolderId(null); + setActiveTab('my_files'); + setAllowsSharedContent(false); + setFolderPath([{ id: null, name: getProviderConfig(provider).rootName }]); + if (!activeConnectionId) return; + setAuthError(''); + loadCloudFiles(activeConnectionId, null, undefined, ''); + if (provider === 'share_point') { + // Work and school accounts can browse "Shared with me" too. + connectorsService + .pickerToken(activeConnectionId, token) + .then((data) => setAllowsSharedContent(!!data?.allows_shared_content)) + .catch(() => undefined); + } + }, [activeConnectionId, provider, loadCloudFiles]); const handleScroll = useCallback(() => { const scrollContainer = scrollContainerRef.current; @@ -284,17 +291,20 @@ export const FilePicker: React.FC = ({ const { scrollTop, scrollHeight, clientHeight } = scrollContainer; const isNearBottom = scrollHeight - scrollTop - clientHeight < 50; - if (isNearBottom && hasMoreFiles && !isLoading && nextPageToken) { - const sessionToken = getSessionToken(provider); - if (sessionToken) { - loadCloudFiles( - sessionToken, - currentFolderId, - nextPageToken, - searchQuery, - activeTab === 'shared' && !currentFolderId, - ); - } + if ( + isNearBottom && + hasMoreFiles && + !isLoading && + nextPageToken && + activeConnectionId + ) { + loadCloudFiles( + activeConnectionId, + currentFolderId, + nextPageToken, + searchQuery, + activeTab === 'shared' && !currentFolderId, + ); } }, [ hasMoreFiles, @@ -302,7 +312,7 @@ export const FilePicker: React.FC = ({ nextPageToken, currentFolderId, searchQuery, - provider, + activeConnectionId, loadCloudFiles, activeTab, ]); @@ -322,10 +332,9 @@ export const FilePicker: React.FC = ({ }, []); const debouncedLoadFiles = useDebouncedCallback((query: string) => { - const sessionToken = getSessionToken(provider); - if (sessionToken) { + if (activeConnectionId) { loadCloudFiles( - sessionToken, + activeConnectionId, currentFolderId, undefined, query, @@ -350,9 +359,8 @@ export const FilePicker: React.FC = ({ setFolderPath((prev) => [...prev, { id: folderId, name: folderName }]); setSearchQuery(''); - const sessionToken = getSessionToken(provider); - if (sessionToken) { - loadCloudFiles(sessionToken, folderId, undefined, '', false); + if (activeConnectionId) { + loadCloudFiles(activeConnectionId, folderId, undefined, '', false); } }; @@ -366,10 +374,9 @@ export const FilePicker: React.FC = ({ setCurrentFolderId(newFolderId); setSearchQuery(''); - const sessionToken = getSessionToken(provider); - if (sessionToken) { + if (activeConnectionId) { loadCloudFiles( - sessionToken, + activeConnectionId, newFolderId, undefined, '', @@ -397,13 +404,20 @@ export const FilePicker: React.FC = ({ : getProviderConfig(provider).rootName, }, ]); - const sessionToken = getSessionToken(provider); - if (sessionToken) { - loadCloudFiles(sessionToken, null, undefined, '', tab === 'shared'); + if (activeConnectionId) { + loadCloudFiles(activeConnectionId, null, undefined, '', tab === 'shared'); } }; const handleFileSelect = (fileId: string, isFolder: boolean) => { + if ( + selectedFiles.length === 0 && + selectedFolders.length === 0 && + onFirstPickName + ) { + const picked = files.find((file) => file.id === fileId); + if (picked) onFirstPickName(picked.name); + } if (isFolder) { const newSelectedFolders = selectedFolders.includes(fileId) ? selectedFolders.filter((id) => id !== fileId) @@ -419,6 +433,13 @@ export const FilePicker: React.FC = ({ } }; + const switchAccount = (connectionId: string) => { + setChosenConnectionId(connectionId); + setSelectedFiles([]); + setSelectedFolders([]); + onSelectionChange([], []); + }; + return (
{authError && ( @@ -428,45 +449,60 @@ export const FilePicker: React.FC = ({ )} + {accounts.length > 1 && !controlledConnectionId && ( + + + + )} + + } onSuccess={(data) => { - setUserEmail( - data.user_email || - t('modals.uploadDoc.connectors.auth.connectedUser'), - ); - setIsConnected(true); setAuthError(''); - - if (data.session_token) { - setSessionToken(provider, data.session_token); - validateAndLoadFiles(); - } + dispatch(loadConnectors({ token })); + if (data.connection_id) setChosenConnectionId(data.connection_id); }} onError={(error) => { setAuthError(error); - setIsConnected(false); }} isConnected={isConnected} - userEmail={userEmail} + userEmail={ + activeAccount?.account_label || + t('modals.uploadDoc.connectors.auth.connectedUser') + } onDisconnect={() => { - const sessionToken = getSessionToken(provider); - if (sessionToken) { - userService - .disconnectConnector(provider, sessionToken, token) + if (activeConnectionId) { + connectorsService + .disconnect(activeConnectionId, token) .catch((err) => console.error( `Error disconnecting from ${getProviderConfig(provider).displayName}:`, err, ), - ); + ) + .finally(() => dispatch(loadConnectors({ token }))); } - - removeSessionToken(provider); - setIsConnected(false); + setChosenConnectionId(null); setAllowsSharedContent(false); setActiveTab('my_files'); setFiles([]); diff --git a/frontend/src/components/GoogleDrivePicker.tsx b/frontend/src/components/GoogleDrivePicker.tsx index e172040f..5d0b7818 100644 --- a/frontend/src/components/GoogleDrivePicker.tsx +++ b/frontend/src/components/GoogleDrivePicker.tsx @@ -9,14 +9,16 @@ const useDrivePicker = (( drivePickerImport as unknown as { default?: typeof drivePickerImport } ).default ?? drivePickerImport) as typeof drivePickerImport; -import userService from '../api/services/userService'; -import ConnectorAuth from './ConnectorAuth'; +import { useDispatch, useSelector } from 'react-redux'; + +import connectorsService from '../api/services/connectorsService'; import { - getSessionToken, - setSessionToken, - removeSessionToken, - validateProviderSession, -} from '../utils/providerUtils'; + loadConnectors, + selectConnections, +} from '../connectors/connectorsSlice'; +import type { AppDispatch } from '../store'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import ConnectorAuth from './ConnectorAuth'; import SkeletonLoader from './SkeletonLoader'; import { Button } from './ui/button'; import { SectionHeader } from './ui/section-header'; @@ -32,90 +34,88 @@ interface PickerFile { interface GoogleDrivePickerProps { token: string | null; + /** The Drive connection to pick from; defaults to the first connected one. */ + connectionId?: string | null; + /** Reports the account the picker uses, so the upload can name it. */ + onConnectionChange?: (connectionId: string | null) => void; onSelectionChange: (fileIds: string[], folderIds?: string[]) => void; + /** Called with the first item's name when the selection goes from empty to one. */ + onFirstPickName?: (name: string) => void; } const GoogleDrivePicker: React.FC = ({ token, + connectionId: controlledConnectionId, + onConnectionChange, onSelectionChange, + onFirstPickName, }) => { const { t } = useTranslation(); + const dispatch = useDispatch(); + const connections = useSelector(selectConnections); + const [chosenConnectionId, setChosenConnectionId] = useState( + null, + ); + const accounts = connections.filter( + (c) => c.connector_key === 'google_drive' && c.status === 'connected', + ); + const activeConnectionId = + controlledConnectionId ?? + (chosenConnectionId && accounts.some((a) => a.id === chosenConnectionId) + ? chosenConnectionId + : (accounts[0]?.id ?? null)); + const activeAccount = accounts.find((a) => a.id === activeConnectionId); + const isConnected = !!activeConnectionId; const [selectedFiles, setSelectedFiles] = useState([]); const [selectedFolders, setSelectedFolders] = useState([]); const [isLoading, setIsLoading] = useState(false); - const [userEmail, setUserEmail] = useState(''); - const [isConnected, setIsConnected] = useState(false); const [authError, setAuthError] = useState(''); - const [accessToken, setAccessToken] = useState(null); const [isValidating, setIsValidating] = useState(false); const [openPicker] = useDrivePicker(); useEffect(() => { - const sessionToken = getSessionToken('google_drive'); - if (sessionToken) { - setIsValidating(true); - setIsConnected(true); // Optimistically set as connected for skeleton - validateSession(sessionToken); - } - }, [token]); + onConnectionChange?.(activeConnectionId); + }, [activeConnectionId]); - const validateSession = async (sessionToken: string) => { + // The Picker runs in the browser and needs an access token. It is fetched + // per use and kept in memory only; the refresh token never leaves the server. + const fetchAccessToken = async (): Promise => { + if (!activeConnectionId) return null; + setIsValidating(true); try { - const validateResponse = await validateProviderSession( + const data = await connectorsService.pickerToken( + activeConnectionId, token, - 'google_drive', ); - - if (!validateResponse.ok) { - setIsConnected(false); + if (!data?.success) { setAuthError( t('modals.uploadDoc.connectors.googleDrive.sessionExpired'), ); - setIsValidating(false); - return false; - } - - const validateData = await validateResponse.json(); - if (validateData.success) { - setUserEmail( - validateData.user_email || - t('modals.uploadDoc.connectors.auth.connectedUser'), - ); - setIsConnected(true); - setAuthError(''); - setAccessToken(validateData.access_token || null); - setIsValidating(false); - return true; - } else { - setIsConnected(false); - setAuthError( - validateData.error || - t('modals.uploadDoc.connectors.googleDrive.sessionExpiredGeneric'), - ); - setIsValidating(false); - return false; + dispatch(loadConnectors({ token })); + return null; } + setAuthError(''); + return data.access_token ?? null; } catch (error) { - console.error('Error validating session:', error); + console.error('Error fetching the picker token:', error); setAuthError(t('modals.uploadDoc.connectors.googleDrive.validateFailed')); - setIsConnected(false); + return null; + } finally { setIsValidating(false); - return false; } }; const handleOpenPicker = async () => { setIsLoading(true); - const sessionToken = getSessionToken('google_drive'); - - if (!sessionToken) { + if (!activeConnectionId) { setAuthError(t('modals.uploadDoc.connectors.googleDrive.noSession')); setIsLoading(false); return; } + const accessToken = await fetchAccessToken(); if (!accessToken) { setAuthError(t('modals.uploadDoc.connectors.googleDrive.noAccessToken')); setIsLoading(false); @@ -191,6 +191,13 @@ const GoogleDrivePicker: React.FC = ({ ); return [...prevFolders, ...uniqueNewFolders]; }); + if ( + selectedFiles.length === 0 && + selectedFolders.length === 0 && + docs.length > 0 + ) { + onFirstPickName?.(docs[0].name); + } onSelectionChange( [...selectedFiles, ...newFiles].map((file) => file.id), [...selectedFolders, ...newFolders].map((folder) => folder.id), @@ -206,25 +213,17 @@ const GoogleDrivePicker: React.FC = ({ }; const handleDisconnect = async () => { - const sessionToken = getSessionToken('google_drive'); - if (sessionToken) { + if (activeConnectionId) { try { - await userService.disconnectConnector( - 'google_drive', - sessionToken, - token, - ); + await connectorsService.disconnect(activeConnectionId, token); } catch (err) { console.error('Error disconnecting from Google Drive:', err); } + dispatch(loadConnectors({ token })); } - - removeSessionToken('google_drive'); - setIsConnected(false); + setChosenConnectionId(null); setSelectedFiles([]); setSelectedFolders([]); - setAccessToken(null); - setUserEmail(''); setAuthError(''); onSelectionChange([], []); }; @@ -241,25 +240,20 @@ const GoogleDrivePicker: React.FC = ({ } onSuccess={(data) => { - setUserEmail( - data.user_email || - t('modals.uploadDoc.connectors.auth.connectedUser'), - ); - setIsConnected(true); setAuthError(''); - - if (data.session_token) { - setSessionToken('google_drive', data.session_token); - validateSession(data.session_token); - } + dispatch(loadConnectors({ token })); + if (data.connection_id) setChosenConnectionId(data.connection_id); }} onError={(error) => { setAuthError(error); - setIsConnected(false); }} isConnected={isConnected} - userEmail={userEmail} + userEmail={ + activeAccount?.account_label || + t('modals.uploadDoc.connectors.auth.connectedUser') + } onDisconnect={handleDisconnect} errorMessage={authError} /> diff --git a/frontend/src/components/MessageInput.test.tsx b/frontend/src/components/MessageInput.test.tsx index ff658ed7..a3fc4e9f 100644 --- a/frontend/src/components/MessageInput.test.tsx +++ b/frontend/src/components/MessageInput.test.tsx @@ -8,7 +8,12 @@ vi.mock('react-i18next', () => ({ })); // The upload modal pulls in the whole ingest UI; the composer never opens it here. vi.mock('../upload/Upload', () => ({ default: () => null })); +vi.mock('../connectors/SignInAgainNotice', () => ({ + default: () => null, + useSignInAgain: () => ({ reconnect: vi.fn(), modals: null }), +})); +import connectorsReducer from '../connectors/connectorsSlice'; import notificationsReducer from '../notifications/notificationsSlice'; import { prefSlice } from '../preferences/preferenceSlice'; import type { RootState } from '../store'; @@ -28,6 +33,7 @@ const makeStore = () => preference: prefSlice.reducer, upload: uploadReducer, notifications: notificationsReducer, + connectors: connectorsReducer, }, }); diff --git a/frontend/src/components/MessageInput.tools.test.tsx b/frontend/src/components/MessageInput.tools.test.tsx new file mode 100644 index 00000000..64ee456f --- /dev/null +++ b/frontend/src/components/MessageInput.tools.test.tsx @@ -0,0 +1,174 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +import type { MultiSelectPopoverItem } from './MultiSelectPopover'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); +vi.mock('../upload/Upload', () => ({ default: () => null })); +vi.mock('../connectors/SignInAgainNotice', () => ({ + default: () => null, + useSignInAgain: () => ({ reconnect: vi.fn(), modals: null }), +})); + +const mocks = vi.hoisted(() => ({ + tools: [] as unknown[], + catalog: [] as unknown[], + connections: [] as unknown[], +})); + +vi.mock('../api/services/userService', () => ({ + default: { + getUserTools: () => + Promise.resolve({ + ok: true, + json: () => Promise.resolve({ success: true, tools: mocks.tools }), + }), + }, +})); +vi.mock('../api/services/connectorsService', () => ({ + default: { + getCatalog: () => + Promise.resolve({ success: true, connectors: mocks.catalog }), + listConnections: () => + Promise.resolve({ success: true, connections: mocks.connections }), + }, +})); + +// The Tools picker renders each item's group; a button opens it. +vi.mock('./message-input', async (importOriginal) => ({ + ...(await importOriginal()), + ToolsTrigger: ({ + items, + onOpenChange, + }: { + items: MultiSelectPopoverItem[]; + onOpenChange: (open: boolean) => void; + }) => ( +
+
+ ), +})); + +import connectorsReducer from '../connectors/connectorsSlice'; +import notificationsReducer from '../notifications/notificationsSlice'; +import { prefSlice } from '../preferences/preferenceSlice'; +import uploadReducer from '../upload/uploadSlice'; +import MessageInput from './MessageInput'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const makeStore = () => + configureStore({ + reducer: { + preference: prefSlice.reducer, + upload: uploadReducer, + notifications: notificationsReducer, + connectors: connectorsReducer, + }, + }); + +describe('MessageInput tools picker', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + localStorage.clear(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const openPicker = async () => { + await act(async () => { + root.render( + + undefined} + loading={false} + showSourceButton={false} + showToolButton + autoFocus={false} + /> + , + ); + }); + await act(async () => { + container + .querySelector('[data-testid="open-tools"]')! + .click(); + }); + for (let i = 0; i < 4; i += 1) { + await act(async () => { + await Promise.resolve(); + }); + } + }; + + const groups = () => + Object.fromEntries( + Array.from( + container.querySelectorAll('[data-testid="tool-item"]'), + ).map((item) => [item.textContent, item.dataset.group]), + ); + + // A teammate's connection is never in the caller's list; the tool still + // sits under its service rather than with the built-in or custom tools. + it("groups a teammate's connected tool under its service", async () => { + mocks.catalog = [ + { + key: 'linear', + name: 'Linear', + icon: 'linear', + publisher: 'preset', + tool_templates: ['mcp_tool'], + mcp_url: 'https://mcp.linear.app/mcp', + }, + ]; + mocks.tools = [ + { + id: 'mem', + name: 'memory', + displayName: 'Memory', + default: true, + status: true, + config: {}, + }, + { + id: 'lin', + name: 'mcp_tool', + displayName: 'Linear', + customName: 'Team Linear', + connection_id: 'owner-conn', + status: false, + in_chat: false, + access: 'viewer', + allowed_actions: ['use', 'use_in_own'], + config: { server_url: 'https://mcp.linear.app/mcp' }, + }, + ]; + await openPicker(); + expect(groups()).toEqual({ + Memory: 'settings.tools.groupBuiltIn', + 'Team Linear': 'Linear', + }); + }); +}); diff --git a/frontend/src/components/MessageInput.tsx b/frontend/src/components/MessageInput.tsx index d1f295a1..267302c7 100644 --- a/frontend/src/components/MessageInput.tsx +++ b/frontend/src/components/MessageInput.tsx @@ -35,13 +35,24 @@ import { selectToken, setSelectedDocs, } from '../preferences/preferenceSlice'; -import type { RootState } from '../store'; +import type { AppDispatch, RootState } from '../store'; import Upload from '../upload/Upload'; import { isTouchDevice } from '../utils/browserUtils'; import { Button } from './ui/button'; import { IconButton } from './ui/icon-button'; import { type MultiSelectPopoverItem } from './MultiSelectPopover'; import ToolIcon from './ToolIcon'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { + connectionNeedsSignIn, + loadConnectors, + selectConnections, + selectConnectorCatalog, +} from '../connectors/connectorsSlice'; +import { toolServiceOf } from '../connectors/toolService'; +import SignInAgainNotice, { + useSignInAgain, +} from '../connectors/SignInAgainNotice'; import { AttachFileButton, AttachmentChipList, @@ -340,6 +351,8 @@ export default function MessageInput({ const [isSourcesPopupOpen, setIsSourcesPopupOpen] = useState(false); const [isToolsPopupOpen, setIsToolsPopupOpen] = useState(false); const [userTools, setUserTools] = useState([]); + const connections = useSelector(selectConnections); + const catalog = useSelector(selectConnectorCatalog); const [toolsLoading, setToolsLoading] = useState(false); const [uploadModalState, setUploadModalState] = useState('INACTIVE'); @@ -1562,22 +1575,77 @@ export default function MessageInput({ .finally(() => setToolsLoading(false)); }, [token]); + // Launched from the Tools picker; the modals live here because the picker + // closes when a sign-in opens. + const signInAgain = useSignInAgain({ onConnected: fetchUserTools }); + useEffect(() => { - if (isToolsPopupOpen) fetchUserTools(); + if (isToolsPopupOpen) { + fetchUserTools(); + (dispatch as AppDispatch)(loadConnectors({ token })); + } }, [isToolsPopupOpen, fetchUserTools]); - const toolItems: MultiSelectPopoverItem[] = userTools.map((tool) => ({ - id: tool.id, - label: tool.customName || tool.displayName, - icon: , - description: - !isOwner(tool) && tool.shared_via - ? t('settings.tools.sharedBy', { - interpolation: { escapeValue: false }, - team: tool.shared_via, - }) - : undefined, - })); + // Tools from a connected service sit under that service; the rest are + // built in. Groups only appear once some tool comes from a connection. + // A teammate's connection is never in the caller's list, so the tool's own + // connection id decides, and the service is named from the catalog. + const toolService = (tool: UserToolType) => + toolServiceOf(tool, connections, catalog); + const anyConnectedTool = userTools.some((tool) => !!tool.connection_id); + // Same groups as the agent builder: built in, one per service, then custom + // tools (an API tool, an MCP server with no connection). + const isCustomTool = (tool: UserToolType) => + !tool.connection_id && + (tool.name === 'api_tool' || tool.name === 'mcp_tool'); + const toolRank = (tool: UserToolType) => + tool.connection_id ? 1 : isCustomTool(tool) ? 2 : 0; + const toolItems: MultiSelectPopoverItem[] = [...userTools] + .sort((a, b) => toolRank(a) - toolRank(b)) + .map((tool) => { + const service = toolService(tool); + const connection = service?.connection; + return { + id: tool.id, + label: tool.customName || tool.displayName, + icon: service?.icon ? ( + + ) : ( + + ), + group: anyConnectedTool + ? (service?.name ?? + (isCustomTool(tool) + ? t('agents.form.toolsPopup.groupCustom') + : t('settings.tools.groupBuiltIn'))) + : undefined, + // Shared-by line; the sign-in warning below wins when both apply. + description: + !isOwner(tool) && tool.shared_via + ? t('settings.tools.sharedBy', { + interpolation: { escapeValue: false }, + team: tool.shared_via, + }) + : undefined, + descriptionNode: connectionNeedsSignIn(connection) ? ( +

+ {t('settings.connectors.health.signInAgain')} +

+ ) : undefined, + }; + }); + // Each broken connection once, with the tool it would re-sign (an MCP + // preset keeps its tool rather than gaining a second one). + const brokenConnections = connections + .filter(connectionNeedsSignIn) + .flatMap((connection) => { + const tools = userTools.filter( + (tool) => tool.connection_id === connection.id, + ); + if (tools.length === 0) return []; + const mcpTool = tools.find((tool) => tool.name === 'mcp_tool'); + return [{ connection, mcpToolId: mcpTool?.id }]; + }); const selectedToolIds = userTools .filter((tool) => toolInChat(tool)) @@ -1739,6 +1807,7 @@ export default function MessageInput({ return (
+ {signInAgain.modals} {/* react-dropzone input (for drag/drop) */} connection, + )} + onReconnect={(connection) => { + setIsToolsPopupOpen(false); + signInAgain.reconnect( + connection, + brokenConnections.find( + (entry) => entry.connection.id === connection.id, + )?.mcpToolId, + ); + }} + /> + } /> )} {ENABLE_VOICE_INPUT && sttAvailable && ( diff --git a/frontend/src/components/SourcesPopoverFooter.test.tsx b/frontend/src/components/SourcesPopoverFooter.test.tsx index 8e5f6aae..5e7db7b5 100644 --- a/frontend/src/components/SourcesPopoverFooter.test.tsx +++ b/frontend/src/components/SourcesPopoverFooter.test.tsx @@ -53,8 +53,11 @@ describe('SourcesPopoverFooter', () => { it('renders the sources link as an inline link Button', async () => { await renderFooter(); - const link = container.querySelector('a')!; - expect(link.getAttribute('href')).toBe('/settings/sources'); + const [link, connect] = Array.from(container.querySelectorAll('a')); + expect(link.getAttribute('href')).toBe('/settings/knowledge'); + expect(connect.getAttribute('href')).toBe( + '/settings/connectors?capability=sync', + ); expect(link.dataset.slot).toBe('button'); expect(link.className).toContain('text-primary'); expect(link.className).toContain('text-sm'); diff --git a/frontend/src/components/SourcesPopoverFooter.tsx b/frontend/src/components/SourcesPopoverFooter.tsx index 0c7a3dcb..441de6f1 100644 --- a/frontend/src/components/SourcesPopoverFooter.tsx +++ b/frontend/src/components/SourcesPopoverFooter.tsx @@ -10,7 +10,10 @@ type SourcesPopoverFooterProps = { onUploadClick: () => void; }; -/** Shared footer for source pickers: a link to the sources page and an upload shortcut. */ +/** + * Shared footer for source pickers: links to the sources page and to the + * connectors that can sync content, and an upload shortcut. + */ export default function SourcesPopoverFooter({ onNavigate, onUploadClick, @@ -21,12 +24,20 @@ export default function SourcesPopoverFooter({ // One row when it fits (link left, upload right); on a narrow sheet or a // long locale the button wraps under the link.
- +
+ + +
} diff --git a/frontend/src/components/message-input/ToolsTrigger.tsx b/frontend/src/components/message-input/ToolsTrigger.tsx index c81e1870..77ece91d 100644 --- a/frontend/src/components/message-input/ToolsTrigger.tsx +++ b/frontend/src/components/message-input/ToolsTrigger.tsx @@ -1,5 +1,7 @@ import { ArrowRight, Wrench } from 'lucide-react'; +import type { ReactNode } from 'react'; import { useTranslation } from 'react-i18next'; +import { Link } from 'react-router-dom'; import { MultiSelectPopover, @@ -14,6 +16,8 @@ type ToolsTriggerProps = { selectedIds: string[]; onToggle: (id: string) => void; loading: boolean; + /** Shown above the links, e.g. connections that need signing in again. */ + notice?: ReactNode; }; export default function ToolsTrigger({ @@ -23,6 +27,7 @@ export default function ToolsTrigger({ selectedIds, onToggle, loading, + notice, }: ToolsTriggerProps) { const { t } = useTranslation(); @@ -38,12 +43,23 @@ export default function ToolsTrigger({ emptyMessage={t('settings.tools.noToolsFound')} loading={loading} footer={ - +
+ {notice} +
+ + +
+
} trigger={ + + )} + + ); +} + +/** + * One action's parameters, each left to the AI or fixed to a value that is + * sent on every call and never shown to the AI (a Telegram chat, a project + * key). Letting the AI decide saves at once; a fixed value saves with Save. + */ +export default function ActionParameters({ + parameters, + readOnly = false, + onSave, +}: { + parameters: ActionParameter[]; + readOnly?: boolean; + /** Saves `{name: value}`; resolves false when the save failed. */ + onSave: (changes: Record) => Promise; +}) { + const { t } = useTranslation(); + return ( +
+
    + {parameters.map((parameter) => ( + onSave({ [parameter.name]: value })} + /> + ))} +
+

+ {t('settings.connectors.parameters.hint')} +

+
+ ); +} diff --git a/frontend/src/connectors/ConnectWizard.test.tsx b/frontend/src/connectors/ConnectWizard.test.tsx new file mode 100644 index 00000000..8e14fbb1 --- /dev/null +++ b/frontend/src/connectors/ConnectWizard.test.tsx @@ -0,0 +1,1015 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter, Route, Routes } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'count' in opts ? `${key}:${opts.count}` : key, + }), +})); + +const service = vi.hoisted(() => ({ + createConnection: vi.fn(), + setup: vi.fn(), + reconnect: vi.fn(), + getConnection: vi.fn(), + getCatalog: vi.fn(), + listConnections: vi.fn(), + setToolPermissions: vi.fn(), + pickerToken: vi.fn(), + renameConnection: vi.fn(), +})); +vi.mock('../api/services/connectorsService', () => ({ default: service })); + +const mcpApi = vi.hoisted(() => ({ + testMCPConnection: vi.fn(), + saveMCPServer: vi.fn(), + getConfig: vi.fn(), +})); +vi.mock('../api/services/userService', async (importOriginal) => { + const original = await importOriginal<{ default: object }>(); + return { default: { ...original.default, ...mcpApi } }; +}); + +// The OAuth popup is covered by ConnectorAuth's own tests; here it only has +// to report a finished sign-in. +vi.mock('../components/ConnectorAuth', () => ({ + useConnectorAuth: + ({ + onSuccess, + }: { + onSuccess: (data: { connection_id: string; user_email: string }) => void; + }) => + () => + onSuccess({ connection_id: 'conn-drive', user_email: 'a@example.com' }), +})); +vi.mock('../components/FilePicker', () => ({ + FilePicker: ({ + onSelectionChange, + onFirstPickName, + }: { + onSelectionChange: (files: string[], folders?: string[]) => void; + onFirstPickName?: (name: string) => void; + }) => ( + + ), +})); + +vi.mock('./RepoPicker', () => ({ + default: ({ onChange }: { onChange: (name: string) => void }) => ( + + ), +})); + +vi.mock('./LinearPicker', async (importOriginal) => { + const original = await importOriginal(); + return { + ...original, + default: ({ + value, + onChange, + }: { + value: import('./LinearPicker').LinearSelection; + onChange: (selection: import('./LinearPicker').LinearSelection) => void; + }) => ( + + ), + }; +}); + +import actionToastReducer, { + selectActionToast, +} from '../notifications/actionToastSlice'; +import notificationsReducer, { + sseEventReceived, +} from '../notifications/notificationsSlice'; +import { + DEFAULT_RETRIEVAL_OPTIONS, + optionsToConfig, +} from '../settings/components/RetrievalOptions'; +import connectorsReducer from './connectorsSlice'; +import ConnectWizard from './ConnectWizard'; +import type { ConnectorDefinition } from './types'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const base: ConnectorDefinition = { + key: 'telegram', + name: 'Telegram', + description: 'Send messages.', + icon: 'tool_telegram', + category: 'messaging', + auth_kind: 'api_key', + capabilities: ['write'], + credential_fields: [ + { key: 'token', label: 'Bot token', secret: true, required: true }, + ], + setup_fields: [], + sync_ingestor: null, + default_sync_frequency: 'weekly', + tool_templates: ['telegram'], + setup: { tools: 'auto', sync: 'off' }, + mcp_url: null, + publisher: 'built_in', + docs_url: null, + oauth_scopes: [], + available: true, + disabled: false, + needs_setup: false, + missing_settings: [], + connected_count: 0, + connection_count: 0, + status: null, + state: 'available', + credential_policy: 'choose', +}; + +const drive: ConnectorDefinition = { + ...base, + key: 'google_drive', + name: 'Google Drive', + icon: 'drive', + category: 'files', + auth_kind: 'oauth', + capabilities: ['sync'], + credential_fields: [], + sync_ingestor: 'google_drive', + tool_templates: [], + setup: { tools: 'off', sync: 'ask' }, +}; + +const github: ConnectorDefinition = { + ...base, + key: 'github', + name: 'GitHub', + icon: 'github', + category: 'dev', + auth_kind: 'api_key', + capabilities: ['sync', 'read'], + credential_fields: [ + { + key: 'access_token', + label: 'Personal access token', + secret: true, + required: true, + }, + ], + setup_fields: [ + { key: 'repo_url', label: 'Repository', secret: false, required: true }, + ], + sync_ingestor: 'github', + tool_templates: ['mcp_tool'], + setup: { tools: 'ask', sync: 'ask' }, + mcp_url: 'https://api.githubcopilot.com/mcp/readonly', + sign_in_methods: ['api_key'], +}; + +const TELEGRAM_TOOL = { + id: 'tool-1', + name: 'telegram', + display_name: 'Telegram', + status: true, + credential_mode: 'owner', + actions: [ + { + name: 'telegram_send_message', + description: 'Send', + access: 'write', + permission: 'ask', + }, + ], +}; + +describe('ConnectWizard', () => { + let root: Root; + let container: HTMLDivElement; + + beforeEach(() => { + Object.values(service).forEach((fn) => fn.mockReset()); + service.getCatalog.mockResolvedValue({ success: true, connectors: [] }); + service.listConnections.mockResolvedValue({ + success: true, + connections: [], + }); + mcpApi.getConfig.mockResolvedValue({ json: async () => ({}) }); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + connector: ConnectorDefinition, + onClose = vi.fn(), + props: Partial[0]> = {}, + ) => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + notifications: notificationsReducer, + actionToast: actionToastReducer, + preference: (state = { token: null, selectedDocs: [] }) => state, + conversation: (state = {}) => state, + }, + }); + await act(async () => { + root.render( + + + + + } + /> + NEW_CHAT
} /> + + + , + ); + }); + return store; + }; + + const click = async (text: string) => { + const button = Array.from( + document.body.querySelectorAll('button'), + ).find((b) => b.textContent?.trim() === text); + expect(button, `button ${text}`).toBeDefined(); + await act(async () => button!.click()); + }; + + const knowledgeSwitch = () => + document.body.querySelector('[id^="knowledge-"]'); + + const typeInto = async (input: HTMLInputElement, value: string) => { + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, value); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + }; + + it('connects an API-key service and creates its tools with no further questions', async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-1' }, + }); + service.setup.mockResolvedValue({ + success: true, + tools: [TELEGRAM_TOOL], + sources: [], + }); + await render(base); + const input = document.body.querySelector( + 'input[type="password"]', + )!; + await typeInto(input, '123:abc'); + await click('settings.connectors.status.connect'); + expect(service.createConnection).toHaveBeenCalledWith( + { connector_key: 'telegram', credentials: { token: '123:abc' } }, + null, + ); + expect(service.setup).toHaveBeenCalledWith( + 'conn-1', + { create_tools: true }, + null, + ); + // Straight to the summary: no sync step for a tool-only service. + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.toolsHeading:1', + ); + }); + + it('names the account when the user gives it a name', async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-1' }, + }); + service.renameConnection.mockResolvedValue({ success: true }); + service.setup.mockResolvedValue({ success: true, tools: [], sources: [] }); + await render(base); + await typeInto( + document.body.querySelector('input[type="password"]')!, + '123:abc', + ); + await typeInto( + document.body.querySelector('#connect-account-name')!, + ' Alerts bot ', + ); + await click('settings.connectors.status.connect'); + expect(service.renameConnection).toHaveBeenCalledWith( + 'conn-1', + 'Alerts bot', + null, + ); + }); + + it('leaves an unnamed account alone', async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-1' }, + }); + service.setup.mockResolvedValue({ success: true, tools: [], sources: [] }); + await render(base); + await typeInto( + document.body.querySelector('input[type="password"]')!, + '123:abc', + ); + await click('settings.connectors.status.connect'); + expect(service.renameConnection).not.toHaveBeenCalled(); + }); + + it('names a signed-in account after the sign-in', async () => { + service.renameConnection.mockResolvedValue({ success: true }); + await render(drive); + await typeInto( + document.body.querySelector('#connect-account-name')!, + 'Work', + ); + await click('settings.connectors.wizard.signIn'); + expect(service.renameConnection).toHaveBeenCalledWith( + 'conn-drive', + 'Work', + null, + ); + }); + + it('shows the default-key refusal in the form', async () => { + service.createConnection.mockResolvedValue({ + success: false, + code: 'encryption_key_default', + }); + await render(base); + await typeInto( + document.body.querySelector('input[type="password"]')!, + 'x', + ); + await click('settings.connectors.status.connect'); + expect(document.body.textContent).toContain( + 'settings.connectors.error.defaultKey', + ); + }); + + it('lets a content service skip choosing what to sync', async () => { + await render(drive, vi.fn(), { purpose: 'knowledge' }); + await click('settings.connectors.wizard.signIn'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSync', + ); + await click('settings.connectors.wizard.skip'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + expect(service.setup).not.toHaveBeenCalled(); + }); + + it('syncs the picked folder named after it', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [{ id: 'src-1', name: 'Handbook' }], + }); + await render(drive, vi.fn(), { purpose: 'knowledge' }); + await click('settings.connectors.wizard.signIn'); + await click('pick-folder'); + await click('modals.uploadDoc.train'); + const [id, body, , key] = service.setup.mock.calls[0]; + expect(id).toBe('conn-drive'); + expect(body.sync).toEqual({ + items: { file_ids: [], folder_ids: ['folder-1'] }, + frequency: 'weekly', + name: 'Handbook', + config: optionsToConfig(DEFAULT_RETRIEVAL_OPTIONS), + }); + expect(typeof key).toBe('string'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneSummary', + ); + }); + + describe('Sync into Knowledge', () => { + const button = (text: string) => + Array.from( + document.body.querySelectorAll('button'), + ).find((b) => b.textContent?.trim() === text); + + it('is off on a plain connect: the account is made, nothing syncs', async () => { + await render(drive); + await click('settings.connectors.wizard.signIn'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSetUp', + ); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.syncToKnowledge', + ); + expect(knowledgeSwitch()!.getAttribute('aria-checked')).toBe('false'); + // The picker, name, frequency and retrieval settings stay folded away. + expect(document.body.textContent).not.toContain('pick-folder'); + expect(document.body.textContent).not.toContain( + 'settings.connectors.wizard.syncFrequency', + ); + expect(document.body.textContent).not.toContain( + 'settings.connectors.wizard.retrievalSettings', + ); + // One way on: nothing to skip. + expect(button('settings.connectors.wizard.skip')).toBeUndefined(); + await click('settings.connectors.wizard.continue'); + expect(service.setup).not.toHaveBeenCalled(); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + // The summary says nothing syncs yet, and where to start later. + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.syncLater', + ); + }); + + it('shows the picker once switched on', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [{ id: 'src-1', name: 'Handbook' }], + }); + await render(drive); + await click('settings.connectors.wizard.signIn'); + await act(async () => knowledgeSwitch()!.click()); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSync', + ); + expect(button('modals.uploadDoc.train')?.disabled).toBe(true); + await click('pick-folder'); + await click('modals.uploadDoc.train'); + expect(service.setup.mock.calls[0][1].sync.items).toEqual({ + file_ids: [], + folder_ids: ['folder-1'], + }); + expect(document.body.textContent).not.toContain( + 'settings.connectors.wizard.syncLater', + ); + }); + + it('drops the picked items when switched back off', async () => { + await render(drive, vi.fn(), { purpose: 'knowledge' }); + await click('settings.connectors.wizard.signIn'); + expect(knowledgeSwitch()!.getAttribute('aria-checked')).toBe('true'); + await click('pick-folder'); + await act(async () => knowledgeSwitch()!.click()); + await act(async () => knowledgeSwitch()!.click()); + // The picker starts over, so nothing is left to add. + expect(button('modals.uploadDoc.train')?.disabled).toBe(true); + }); + + it('sends the advanced retrieval settings with the sync', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [], + }); + await render(drive, vi.fn(), { purpose: 'knowledge' }); + await click('settings.connectors.wizard.signIn'); + const toggle = button('settings.connectors.wizard.retrievalSettings')!; + expect(toggle.getAttribute('aria-expanded')).toBe('false'); + await act(async () => toggle.click()); + expect(toggle.getAttribute('aria-expanded')).toBe('true'); + await typeInto( + document.body.querySelector('#retrieval-chunks')!, + '9', + ); + await click('pick-folder'); + await click('modals.uploadDoc.train'); + expect(service.setup.mock.calls[0][1].sync.config).toEqual( + optionsToConfig({ + ...DEFAULT_RETRIEVAL_OPTIONS, + retrieval: { ...DEFAULT_RETRIEVAL_OPTIONS.retrieval, chunks: 9 }, + }), + ); + }); + + it('blocks the sync while the prescreen settings do not add up', async () => { + await render(drive, vi.fn(), { purpose: 'knowledge' }); + await click('settings.connectors.wizard.signIn'); + await click('pick-folder'); + await click('settings.connectors.wizard.retrievalSettings'); + await act(async () => + document.body + .querySelector('#retrieval-prescreen')! + .click(), + ); + // More chunks than prescreen candidates. + await typeInto( + document.body.querySelector('#retrieval-chunks')!, + '50', + ); + expect(button('modals.uploadDoc.train')?.disabled).toBe(true); + }); + + it('is not asked about when syncing more from an account', async () => { + await render(drive, vi.fn(), { + mode: 'sync', + connectionId: 'conn-drive', + }); + expect(knowledgeSwitch()).toBeNull(); + expect(document.body.textContent).toContain('pick-folder'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSync', + ); + }); + + it('is not offered by a service that does not sync', async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-1' }, + }); + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [], + }); + await render(base, vi.fn(), { purpose: 'knowledge' }); + await typeInto( + document.body.querySelector( + 'input[type="password"]', + )!, + 't', + ); + await click('settings.connectors.status.connect'); + expect(knowledgeSwitch()).toBeNull(); + expect(document.body.textContent).not.toContain( + 'settings.connectors.wizard.syncLater', + ); + }); + }); + + it('opens a new chat from Try it in chat', async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-1' }, + }); + service.setup.mockResolvedValue({ success: true, tools: [], sources: [] }); + const onClose = vi.fn(); + await render(base, onClose); + await typeInto( + document.body.querySelector('input[type="password"]')!, + 't', + ); + await click('settings.connectors.status.connect'); + await click('settings.connectors.wizard.tryInChat'); + expect(onClose).toHaveBeenCalled(); + expect(document.body.textContent).toContain('NEW_CHAT'); + }); + + it('signs in to an MCP preset with one button and shows its tools', async () => { + const notion: ConnectorDefinition = { + ...base, + key: 'mcp:notion', + name: 'Notion', + icon: 'notion', + auth_kind: 'mcp_oauth', + credential_fields: [], + tool_templates: ['mcp_tool'], + mcp_url: 'https://mcp.notion.com/mcp', + oauth_scopes: [], + publisher: 'preset', + }; + const popup = { closed: false, close: vi.fn(), location: { href: '' } }; + const open = vi.spyOn(window, 'open').mockReturnValue(popup as never); + mcpApi.testMCPConnection.mockResolvedValue({ + json: async () => ({ requires_oauth: true, task_id: 'task-1' }), + }); + mcpApi.saveMCPServer.mockResolvedValue({ + ok: true, + json: async () => ({ success: true, id: 'tool-9' }), + }); + service.listConnections.mockResolvedValue({ + success: true, + connections: [ + { id: 'conn-9', connector_key: 'mcp:notion', updated_at: '2026-09-28' }, + ], + }); + service.getConnection.mockResolvedValue({ + success: true, + connection: { tools: [{ ...TELEGRAM_TOOL, display_name: 'Notion' }] }, + }); + const store = await render(notion); + // No server URL or auth form: one sign-in button. + expect(document.body.querySelector('input')).toBeNull(); + await click('settings.connectors.wizard.signIn'); + // The pop-up opens inside the click, then follows the worker. + expect(open).toHaveBeenCalledWith( + 'about:blank', + 'mcpOAuth', + expect.any(String), + ); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.waiting', + ); + await act(async () => { + store.dispatch( + sseEventReceived({ + id: 'ev-1', + type: 'mcp.oauth.awaiting_redirect', + scope: { kind: 'mcp_oauth', id: 'task-1' }, + payload: { authorization_url: 'https://notion.example/authorize' }, + }), + ); + }); + expect(popup.location.href).toBe('https://notion.example/authorize'); + await act(async () => { + store.dispatch( + sseEventReceived({ + id: 'ev-2', + type: 'mcp.oauth.completed', + scope: { kind: 'mcp_oauth', id: 'task-1' }, + payload: { tools: [] }, + }), + ); + }); + expect(mcpApi.saveMCPServer.mock.calls[0][0]).toMatchObject({ + displayName: 'Notion', + config: { + server_url: 'https://mcp.notion.com/mcp', + auth_type: 'oauth', + oauth_task_id: 'task-1', + }, + }); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.toolsHeading:1', + ); + open.mockRestore(); + }); + + describe('GitHub', () => { + const connectWithToken = async () => { + service.createConnection.mockResolvedValue({ + success: true, + connection: { id: 'conn-gh' }, + }); + await typeInto( + document.body.querySelector( + 'input[type="password"]', + )!, + 'github_pat_abc', + ); + await click('settings.connectors.status.connect'); + }; + + it('connects with a token when no GitHub App is set up', async () => { + await render(github); + // One way in: no method switch, a token field and how to make one. + expect(document.body.querySelector('[role="radiogroup"]')).toBeNull(); + expect(document.body.textContent).toContain( + 'settings.connectors.github.tokenHint', + ); + await connectWithToken(); + expect(service.createConnection).toHaveBeenCalledWith( + { + connector_key: 'github', + credentials: { access_token: 'github_pat_abc' }, + }, + null, + ); + // Tools are asked about, not created on sign-in. + expect(service.setup).not.toHaveBeenCalled(); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSetUp', + ); + }); + + it('offers Sign in with GitHub first, and a token instead', async () => { + await render({ ...github, sign_in_methods: ['oauth', 'api_key'] }); + expect(document.body.querySelector('input[type="password"]')).toBeNull(); + await click('settings.connectors.wizard.methodToken'); + expect( + document.body.querySelector('input[type="password"]'), + ).not.toBeNull(); + await click('settings.connectors.wizard.methodOauth'); + await click('settings.connectors.wizard.signIn'); + // The OAuth pop-up (mocked) reported the connection. + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.chooseWhatToSetUp', + ); + }); + + it('says when GitHub refused the token', async () => { + service.createConnection.mockResolvedValue({ + success: false, + code: 'invalid_credentials', + }); + await render(github); + await typeInto( + document.body.querySelector( + 'input[type="password"]', + )!, + 'nope', + ); + await click('settings.connectors.status.connect'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.credentialsRejected', + ); + }); + + it('adds the read-only tools and syncs the picked repository in one step', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [{ ...TELEGRAM_TOOL, name: 'mcp_tool', display_name: 'GitHub' }], + sources: [{ id: 'src-1', name: 'octocat/private' }], + }); + await render(github, vi.fn(), { purpose: 'knowledge' }); + await connectWithToken(); + await click('pick-repo'); + await click('settings.connectors.wizard.continue'); + const [id, body] = service.setup.mock.calls[0]; + expect(id).toBe('conn-gh'); + expect(body).toEqual({ + create_tools: true, + sync: { + items: { repo_url: 'octocat/private' }, + frequency: 'weekly', + name: 'octocat/private', + config: optionsToConfig(DEFAULT_RETRIEVAL_OPTIONS), + }, + }); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.toolsHeading:1', + ); + }); + + it('can add only the tools', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [], + }); + await render(github); + await connectWithToken(); + await click('settings.connectors.wizard.continue'); + expect(service.setup.mock.calls[0][1]).toEqual({ create_tools: true }); + }); + + it('can sync without tools', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [], + }); + await render(github, vi.fn(), { purpose: 'knowledge' }); + await connectWithToken(); + const toolSwitch = + document.body.querySelector('[role="switch"]')!; + expect(toolSwitch.getAttribute('aria-checked')).toBe('true'); + await act(async () => toolSwitch.click()); + await click('pick-repo'); + await click('settings.connectors.wizard.continue'); + expect(service.setup.mock.calls[0][1].create_tools).toBe(false); + expect(service.setup.mock.calls[0][1].sync.items).toEqual({ + repo_url: 'octocat/private', + }); + }); + + const writesSwitch = () => + document.body.querySelector('#tools-github-writes'); + + it('offers changes as a second choice, read only by default', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [], + }); + await render({ ...github, writes_opt_in: true, writes_allowed: true }); + await connectWithToken(); + expect(document.body.textContent).toContain( + 'settings.connectors.github.writes', + ); + expect(writesSwitch()!.getAttribute('aria-checked')).toBe('false'); + await act(async () => writesSwitch()!.click()); + await click('settings.connectors.wizard.continue'); + expect(service.setup.mock.calls[0][1]).toEqual({ + create_tools: true, + allow_writes: true, + }); + }); + + it('drops the choice with the tools', async () => { + await render({ ...github, writes_opt_in: true, writes_allowed: true }); + await connectWithToken(); + await act(async () => + document.body + .querySelector('#tools-github')! + .click(), + ); + expect(writesSwitch()).toBeNull(); + }); + + it('hides the choice when an admin turned changes off', async () => { + await render({ ...github, writes_opt_in: true, writes_allowed: false }); + await connectWithToken(); + expect(writesSwitch()).toBeNull(); + expect(document.body.textContent).not.toContain( + 'settings.connectors.github.writes', + ); + }); + + it('shows why the tools could not be added', async () => { + service.setup.mockResolvedValue({ + success: false, + code: 'tools_unavailable', + }); + await render(github); + await connectWithToken(); + await click('settings.connectors.wizard.continue'); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.toolsUnavailable', + ); + }); + }); + + describe('Linear', () => { + const linear: ConnectorDefinition = { + ...base, + key: 'mcp:linear', + name: 'Linear', + icon: 'linear', + category: 'projects', + auth_kind: 'mcp_oauth', + capabilities: ['sync', 'read', 'write'], + credential_fields: [], + sync_ingestor: 'linear', + tool_templates: ['mcp_tool'], + setup: { tools: 'auto', sync: 'ask' }, + mcp_url: 'https://mcp.linear.app/mcp', + publisher: 'preset', + }; + + const signIn = async (props = {}, onClose = vi.fn()) => { + const popup = { closed: false, close: vi.fn(), location: { href: '' } }; + const open = vi.spyOn(window, 'open').mockReturnValue(popup as never); + mcpApi.testMCPConnection.mockResolvedValue({ + json: async () => ({ requires_oauth: true, task_id: 'task-1' }), + }); + mcpApi.saveMCPServer.mockResolvedValue({ + ok: true, + json: async () => ({ success: true, id: 'tool-9' }), + }); + service.listConnections.mockResolvedValue({ + success: true, + connections: [ + { + id: 'conn-lin', + connector_key: 'mcp:linear', + updated_at: '2026-09-28', + }, + ], + }); + service.getConnection.mockResolvedValue({ + success: true, + connection: { tools: [{ ...TELEGRAM_TOOL, display_name: 'Linear' }] }, + }); + const store = await render(linear, onClose, props); + await click('settings.connectors.wizard.signIn'); + await act(async () => { + store.dispatch( + sseEventReceived({ + id: 'ev-2', + type: 'mcp.oauth.completed', + scope: { kind: 'mcp_oauth', id: 'task-1' }, + payload: { tools: [] }, + }), + ); + }); + open.mockRestore(); + return store; + }; + + // The tools come first on the one screen after signing in. + const toolsFirst = () => { + const text = document.body.textContent ?? ''; + const tools = text.indexOf('settings.connectors.wizard.toolsHeading:1'); + const knowledge = text.indexOf( + 'settings.connectors.wizard.syncToKnowledge', + ); + return tools >= 0 && (knowledge < 0 || tools < knowledge); + }; + + it('signs in once, then shows its tools and picks teams to sync on one screen', async () => { + service.setup.mockResolvedValue({ + success: true, + tools: [], + sources: [{ id: 'src-1', name: 'Linear · Engineering' }], + }); + const onClose = vi.fn(); + const store = await signIn({ purpose: 'knowledge' }, onClose); + expect(toolsFirst()).toBe(true); + await click('pick-team'); + await click('modals.uploadDoc.train'); + const [id, body] = service.setup.mock.calls[0]; + expect(id).toBe('conn-lin'); + expect(body).toEqual({ + create_tools: false, + sync: { + items: { + teams: [{ id: 't1', key: 'ENG', name: 'Engineering' }], + projects: [], + include_comments: true, + include_documents: false, + }, + frequency: 'weekly', + name: 'Linear · Engineering', + config: optionsToConfig(DEFAULT_RETRIEVAL_OPTIONS), + }, + }); + // Nothing more to show: the modal closes and says what syncs. + expect(onClose).toHaveBeenCalled(); + expect( + selectActionToast( + store.getState() as Parameters[0], + )?.variant, + ).toBe('success'); + }); + + it('can skip syncing and keep only the tools', async () => { + const onClose = vi.fn(); + await signIn({ purpose: 'knowledge' }, onClose); + const train = Array.from( + document.body.querySelectorAll('button'), + ).find((b) => b.textContent?.trim() === 'modals.uploadDoc.train'); + // Nothing picked yet: nothing to sync. + expect(train?.disabled).toBe(true); + await click('settings.connectors.wizard.skip'); + expect(service.setup).not.toHaveBeenCalled(); + expect(onClose).toHaveBeenCalled(); + }); + + it('asks about Knowledge under its tools on a plain connect', async () => { + const onClose = vi.fn(); + await signIn({}, onClose); + expect(toolsFirst()).toBe(true); + expect(knowledgeSwitch()!.getAttribute('aria-checked')).toBe('false'); + expect(document.body.textContent).not.toContain('pick-team'); + await click('settings.connectors.wizard.done'); + expect(service.setup).not.toHaveBeenCalled(); + expect(onClose).toHaveBeenCalled(); + }); + + it('goes straight to the summary after signing in again', async () => { + await signIn({ mode: 'reconnect', connectionId: 'conn-lin' }); + expect(document.body.textContent).toContain( + 'settings.connectors.wizard.doneTitle', + ); + }); + + it('opens on the picker to sync more from the drawer', async () => { + await render(linear, vi.fn(), { + mode: 'sync', + connectionId: 'conn-lin', + }); + expect(document.body.textContent).toContain('pick-team'); + // Syncing is why it opened: no question about it. + expect(knowledgeSwitch()).toBeNull(); + }); + }); +}); diff --git a/frontend/src/connectors/ConnectWizard.tsx b/frontend/src/connectors/ConnectWizard.tsx new file mode 100644 index 00000000..54f6a41a --- /dev/null +++ b/frontend/src/connectors/ConnectWizard.tsx @@ -0,0 +1,945 @@ +import { CircleAlert, ExternalLink } from 'lucide-react'; +import { nanoid } from '@reduxjs/toolkit'; +import { useEffect, useMemo, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; +import { useNavigate } from 'react-router-dom'; + +import { envVar } from '@/env'; +import { baseURL } from '../api/client'; +import connectorsService from '../api/services/connectorsService'; +import userService from '../api/services/userService'; +import { useConnectorAuth } from '../components/ConnectorAuth'; +import { FilePicker } from '../components/FilePicker'; +import GoogleDrivePicker from '../components/GoogleDrivePicker'; +import { + Accordion, + AccordionContent, + AccordionItem, + AccordionTrigger, +} from '../components/ui/accordion'; +import { Alert, AlertDescription } from '../components/ui/alert'; +import { Button } from '../components/ui/button'; +import { FormField } from '../components/ui/form-field'; +import { Input } from '../components/ui/input'; +import { Modal, ModalActions } from '../components/ui/modal'; +import { SectionHeader } from '../components/ui/section-header'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '../components/ui/select'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import { ToggleGroup, ToggleGroupItem } from '../components/ui/toggle-group'; +import { + setConversation, + updateConversationId, +} from '../conversation/conversationSlice'; +import { + selectToken, + setSelectedAgent, + setSelectedDocs, +} from '../preferences/preferenceSlice'; +import RetrievalOptions, { + DEFAULT_RETRIEVAL_OPTIONS, + isPrescreenConfigValid, + optionsToConfig, + type RetrievalOptionsValue, +} from '../settings/components/RetrievalOptions'; +import useRetrievalAvailability from '../settings/components/useRetrievalAvailability'; +import { showActionToast } from '../notifications/actionToastSlice'; +import type { AppDispatch } from '../store'; +import { formatCount } from '../utils/dateTimeUtils'; +import { ACCOUNT_NAME_MAX } from './accounts'; +import ConnectorIcon from './ConnectorIcon'; +import CredentialForm, { credentialsComplete } from './CredentialForm'; +import { loadConnectors, selectConnections } from './connectorsSlice'; +import { connectorDescription, connectorName, isKeyHint } from './i18n'; +import LinearPicker, { + EMPTY_LINEAR_SELECTION, + linearSourceName, + linearSyncItems, + type LinearSelection, +} from './LinearPicker'; +import RepoPicker from './RepoPicker'; +import ToolPermissions from './ToolPermissions'; +import useMcpOAuth, { type McpOAuthConfig } from './useMcpOAuth'; +import type { + ConnectionTool, + ConnectorAuthKind, + ConnectorDefinition, +} from './types'; + +export type WizardMode = 'connect' | 'reconnect' | 'sync' | 'done'; +/** + * Why the wizard opened. `knowledge` (Add knowledge, Knowledge's Connect a + * service) starts a first connect with Sync into Knowledge on; `tools`, or + * none, leaves it off for the user to choose. + */ +export type LaunchPurpose = 'knowledge' | 'tools'; + +const FREQUENCIES = ['never', 'daily', 'weekly', 'monthly'] as const; +const PICKER_CONNECTORS = new Set([ + 'google_drive', + 'share_point', + 'confluence', +]); +// Where a GitHub user makes a fine-grained token (Contents and Metadata: read; +// Issues and Pull requests: read and write when agents may make changes). +const GITHUB_TOKEN_URL = + 'https://github.com/settings/personal-access-tokens/new'; + +type CreatedSource = { id: string; name: string }; + +/** + * The one connect flow every entry point opens: sign in, choose what to set + * up, then a summary with Try it in chat. A content connector asks whether to + * sync into Knowledge (on when opened for Knowledge) and, if so, what to sync + * and with which retrieval settings. Tool connectors create their tools on + * sign-in, writes needing approval, so they go from the credentials straight + * to the summary. + */ +export default function ConnectWizard({ + connector, + mode = 'connect', + connectionId: initialConnectionId = null, + mcpToolId, + purpose, + onClose, + onFinished, +}: { + connector: ConnectorDefinition; + mode?: WizardMode; + connectionId?: string | null; + /** Reconnecting an MCP preset: the tool to update rather than add. */ + mcpToolId?: string; + purpose?: LaunchPurpose; + onClose: () => void; + onFinished?: () => void; +}) { + const { t } = useTranslation(); + const dispatch = useDispatch(); + const navigate = useNavigate(); + const token = useSelector(selectToken); + const connections = useSelector(selectConnections); + const name = connectorName(t, connector); + + const [step, setStep] = useState<'signin' | 'setup' | 'done'>( + mode === 'sync' ? 'setup' : mode === 'done' ? 'done' : 'signin', + ); + const [connectionId, setConnectionId] = useState( + initialConnectionId, + ); + const [credentials, setCredentials] = useState>({}); + const [accountName, setAccountName] = useState(''); + const [setupValues, setSetupValues] = useState>({}); + const [pending, setPending] = useState(false); + const [error, setError] = useState(''); + const [tools, setTools] = useState([]); + const [sources, setSources] = useState([]); + const [selectedFiles, setSelectedFiles] = useState([]); + const [selectedFolders, setSelectedFolders] = useState([]); + const [selectedRepo, setSelectedRepo] = useState(null); + const [linearSelection, setLinearSelection] = useState( + EMPTY_LINEAR_SELECTION, + ); + // A connector that asks about its tools (GitHub) offers them switched on. + const [toolsOn, setToolsOn] = useState(true); + // Changes (GitHub's issues, comments, pull requests) are an opt-in on top. + const [writesOn, setWritesOn] = useState(false); + const [chosenMethod, setChosenMethod] = useState( + null, + ); + const [sourceName, setSourceName] = useState(''); + const [nameTouched, setNameTouched] = useState(false); + const [frequency, setFrequency] = useState( + connector.default_sync_frequency || 'weekly', + ); + // One key per wizard: a double click on Add source queues one ingest. + const [idempotencyKey] = useState(() => nanoid()); + + const connection = connections.find((c) => c.id === connectionId); + const canSync = connector.setup.sync !== 'off' && !!connector.sync_ingestor; + // A first connect asks whether to sync into Knowledge; Sync more is only + // about syncing, so it never asks. + const askSync = canSync && mode === 'connect'; + const [syncOn, setSyncOn] = useState(purpose === 'knowledge'); + const syncing = canSync && (mode === 'sync' || syncOn); + const [retrievalOptions, setRetrievalOptions] = + useState(DEFAULT_RETRIEVAL_OPTIONS); + const { graphRAGAvailable, hybridAvailable, availableModels } = + useRetrievalAvailability(token, step === 'setup' && syncing); + // Tools the user opts into while choosing what to sync, not on sign-in. + const offerTools = + connector.setup.tools === 'ask' && + connector.tool_templates.length > 0 && + mode !== 'sync'; + // GitHub signs in two ways: Sign in with GitHub (when an admin set up the + // GitHub App) or a token. Reconnecting keeps the connection's own way. + const methods = connector.sign_in_methods?.length + ? connector.sign_in_methods + : [connector.auth_kind]; + const method: ConnectorAuthKind = + mode === 'reconnect' && connection?.auth_kind + ? connection.auth_kind + : (chosenMethod ?? methods[0]); + const usesOAuth = method === 'oauth'; + + // A finished MCP save or a reconnect lands here with only the connection id. + useEffect(() => { + if (mode !== 'done' || !connectionId) return; + connectorsService.getConnection(connectionId, token).then((data) => { + if (data?.success) setTools(data.connection.tools ?? []); + }); + }, [mode, connectionId, token]); + + const refresh = () => dispatch(loadConnectors({ token })); + + // A name tells two accounts of one service apart ("Alerts bot"). It is + // set once the account exists, so it works the same for keys and sign-ins. + const canName = + mode === 'connect' && + (connector.auth_kind === 'api_key' || connector.auth_kind === 'oauth'); + + const afterSignIn = async (id: string) => { + setConnectionId(id); + const name = accountName.trim(); + if (canName && name) { + try { + await connectorsService.renameConnection(id, name, token); + } catch { + // The account works without a name; it can be named in the drawer. + } + } + refresh(); + if (mode === 'reconnect') { + setStep('done'); + return; + } + if (connector.setup.tools === 'auto' && connector.tool_templates.length) { + const setup = await connectorsService.setup( + id, + { create_tools: true }, + token, + ); + if (setup?.success) setTools(setup.tools ?? []); + } + setStep(canSync || offerTools ? 'setup' : 'done'); + }; + + // MCP presets (Notion, Linear…) sign in over MCP OAuth: no URL or auth + // form, just "Sign in to Notion". The tool is saved on success, and the + // done step shows what it can do. + const isMcpPreset = + connector.auth_kind === 'mcp_oauth' && !!connector.mcp_url; + // An MCP preset that also syncs (Linear) has its tools from the sign-in: + // one screen shows them first, then asks about Knowledge, and finishing + // it closes the wizard with no separate summary. + const toolsFirst = isMcpPreset && mode === 'connect' && tools.length > 0; + const mcp = useMcpOAuth(); + const mcpConfig = (): McpOAuthConfig => ({ + server_url: connector.mcp_url ?? '', + auth_type: 'oauth', + oauth_scopes: connector.oauth_scopes ?? [], + timeout: 30, + redirect_uri: `${baseURL.replace(/\/$/, '')}/api/mcp_server/callback`, + }); + + const saveMcp = async (config: McpOAuthConfig, taskId: string | null) => { + setPending(true); + try { + const response = await userService.saveMCPServer( + { + displayName: name, + config: { ...config, oauth_task_id: taskId ?? '' }, + status: true, + ...(mcpToolId && { id: mcpToolId }), + }, + token, + ); + const result = await response.json(); + if (!response.ok || !result.success) throw new Error(result.error); + const list = await connectorsService.listConnections(token); + const saved = ( + (list?.connections ?? []) as { + id: string; + connector_key: string; + updated_at: string | null; + }[] + ) + .filter((c) => c.connector_key === connector.key) + .sort((a, b) => + (b.updated_at ?? '').localeCompare(a.updated_at ?? ''), + )[0]; + refresh(); + if (saved) { + setConnectionId(saved.id); + const detail = await connectorsService.getConnection(saved.id, token); + if (detail?.success) setTools(detail.connection.tools ?? []); + } + // One sign-in also syncs (Linear): choosing what to sync comes next. + setStep(canSync && saved && mode === 'connect' ? 'setup' : 'done'); + } catch (err) { + setError( + (err instanceof Error && err.message) || + t('settings.tools.mcp.errors.saveFailed'), + ); + } finally { + setPending(false); + } + }; + + const startMcpSignIn = () => { + setError(''); + const config = mcpConfig(); + mcp.start(config, { + onDone: ({ taskId }) => saveMcp(config, taskId), + onError: (message) => + setError(message || t('settings.tools.mcp.errors.oauthFailed')), + }); + }; + + const startSignIn = useConnectorAuth({ + provider: connector.key, + connectionId: + mode === 'reconnect' ? (connectionId ?? undefined) : undefined, + onSuccess: (data) => afterSignIn(data.connection_id), + onError: setError, + }); + + const submitCredentials = async () => { + setPending(true); + setError(''); + try { + const data = + mode === 'reconnect' && connectionId + ? await connectorsService.reconnect( + connectionId, + { credentials }, + token, + ) + : await connectorsService.createConnection( + { connector_key: connector.key, credentials }, + token, + ); + if (!data?.success) { + setError( + data?.code === 'encryption_key_default' + ? t('settings.connectors.error.defaultKey') + : data?.code === 'invalid_credentials' + ? t('settings.connectors.wizard.credentialsRejected', { + name, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.wizard.connectFailed'), + ); + return; + } + await afterSignIn(data.connection.id); + } catch { + setError(t('settings.connectors.wizard.connectFailed')); + } finally { + setPending(false); + } + }; + + const prefillName = (picked: string) => { + if (!nameTouched && picked) setSourceName((current) => current || picked); + }; + + const isRepoPicker = connector.key === 'github'; + const isLinearPicker = connector.sync_ingestor === 'linear'; + const syncItems = (): Record => + PICKER_CONNECTORS.has(connector.key) + ? { file_ids: selectedFiles, folder_ids: selectedFolders } + : isRepoPicker + ? { repo_url: selectedRepo } + : isLinearPicker + ? linearSyncItems(linearSelection) + : setupValues; + + const hasSyncSelection = PICKER_CONNECTORS.has(connector.key) + ? selectedFiles.length + selectedFolders.length > 0 + : isRepoPicker + ? !!selectedRepo + : isLinearPicker + ? linearSelection.teams.length + linearSelection.projects.length > 0 + : credentialsComplete(connector.setup_fields, setupValues); + const wantsTools = offerTools && toolsOn; + const offerWrites = wantsTools && !!connector.writes_allowed; + const wantsWrites = offerWrites && writesOn; + const syncReady = syncing && hasSyncSelection; + // Off, there is nothing to pick: the button only finishes connecting. + // On, something must be picked (or the tools wanted), and an incoherent + // prescreen config blocks it as in Upload; the backend would refuse it. + const canAddSource = + !syncing || + ((hasSyncSelection || wantsTools) && + isPrescreenConfigValid(retrievalOptions)); + + const toggleSync = (on: boolean) => { + setSyncOn(on); + // The file pickers start over when shown again; drop what they reported. + if (!on) { + setSelectedFiles([]); + setSelectedFolders([]); + } + }; + + const addSource = async () => { + if (!connectionId) return; + if (!wantsTools && !syncReady) { + // Connected, nothing more to set up. + if (toolsFirst) finish(); + else setStep('done'); + return; + } + setPending(true); + setError(''); + try { + const data = await connectorsService.setup( + connectionId, + { + create_tools: wantsTools, + ...(wantsWrites && { allow_writes: true }), + ...(syncReady && { + sync: { + items: syncItems(), + frequency, + name: sourceName.trim() || undefined, + config: optionsToConfig(retrievalOptions), + }, + }), + }, + token, + idempotencyKey, + ); + if (!data?.success) { + setError( + data?.code === 'tools_unavailable' + ? t('settings.connectors.wizard.toolsUnavailable', { + name, + interpolation: { escapeValue: false }, + }) + : data?.code === 'writes_forbidden' + ? t('settings.connectors.github.writesForbidden') + : data?.error || t('settings.connectors.wizard.syncFailed'), + ); + return; + } + if (wantsTools) setTools(data.tools ?? []); + setSources(data.sources ?? []); + refresh(); + if (toolsFirst) { + const count = (data.sources ?? []).length; + dispatch( + showActionToast({ + variant: 'success', + message: t('settings.connectors.wizard.doneSources', { + sources: t('settings.connectors.wizard.sourcesCount', { + count, + formatted: formatCount(count), + }), + }), + }), + ); + finish(); + return; + } + setStep('done'); + } catch { + setError(t('settings.connectors.wizard.syncFailed')); + } finally { + setPending(false); + } + }; + + const tryInChat = () => { + dispatch(setConversation([])); + dispatch(updateConversationId({ query: { conversationId: null } })); + dispatch(setSelectedAgent(null)); + if (sources.length > 0) { + dispatch( + setSelectedDocs( + sources.map((source) => ({ + id: source.id, + name: source.name, + date: new Date().toISOString(), + model: '', + type: 'connector:file', + connectionId, + })), + ), + ); + } + onFinished?.(); + onClose(); + navigate('/c/new'); + }; + + const finish = () => { + onFinished?.(); + onClose(); + }; + + const toolCount = tools.length; + const summary = useMemo(() => { + const account = connection?.account_label ?? ''; + const accountLine = + connection?.auth_kind === 'api_key' && isKeyHint(account) + ? t('settings.connectors.wizard.connectedWithKey', { + hint: account, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.wizard.doneSummaryNone', { + account, + interpolation: { escapeValue: false }, + }); + const sourcesText = t('settings.connectors.wizard.sourcesCount', { + count: sources.length, + formatted: formatCount(sources.length), + }); + const toolsText = t('settings.connectors.wizard.toolsCount', { + count: toolCount, + formatted: formatCount(toolCount), + }); + const countsLine = + sources.length && toolCount + ? t('settings.connectors.wizard.doneCounts', { + sources: sourcesText, + tools: toolsText, + }) + : sources.length + ? t('settings.connectors.wizard.doneSources', { + sources: sourcesText, + }) + : toolCount + ? t('settings.connectors.wizard.doneTools', { tools: toolsText }) + : ''; + // A new account that syncs nothing yet: say where syncing starts later. + const syncLine = + mode === 'connect' && canSync && sources.length === 0 + ? t('settings.connectors.wizard.syncLater', { + name, + interpolation: { escapeValue: false }, + }) + : ''; + return [accountLine, countsLine, syncLine].filter(Boolean).join(' '); + }, [connection, sources, toolCount, mode, canSync, name, t]); + + const renderSignIn = () => ( +
+
+ + + +

+ {connectorDescription(t, connector)} +

+
+ {error && ( + + + {error} + + )} + {isMcpPreset && mcp.pending && ( +

+ {t('settings.connectors.wizard.waiting', { + name, + interpolation: { escapeValue: false }, + })} +

+ )} + {isMcpPreset && mcp.blockedUrl && ( + + )} + {mode !== 'reconnect' && methods.length > 1 && ( + + value && setChosenMethod(value as ConnectorAuthKind) + } + aria-label={t('settings.connectors.wizard.methodLabel')} + > + {methods.map((value) => ( + + {value === 'oauth' + ? t('settings.connectors.wizard.methodOauth', { + name, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.wizard.methodToken')} + + ))} + + )} + {usesOAuth || isMcpPreset ? null : ( + <> + {connector.key === 'github' && ( +
+

+ {t('settings.connectors.github.tokenHint')} +

+ +
+ )} + + + )} + {canName && ( + + setAccountName(e.target.value)} + /> + + )} +
+ ); + + const renderSetup = () => ( +
+ {error && ( + + + {error} + + )} + {toolsFirst && connectionId && ( +
+ + {tools.map((tool) => ( + + ))} +
+ )} + {(offerTools || askSync) && ( + + {offerTools && ( + + setToolsOn(checked === true)} + /> + + )} + {offerWrites && ( + + setWritesOn(checked === true)} + /> + + )} + {askSync && ( + + toggleSync(checked === true)} + /> + + )} + + )} + {syncing && ( + <> + {isRepoPicker && connectionId ? ( + { + setSelectedRepo(fullName); + if (!nameTouched) setSourceName(fullName); + }} + /> + ) : isLinearPicker && connectionId ? ( + { + setLinearSelection(selection); + if (!nameTouched) setSourceName(linearSourceName(selection)); + }} + /> + ) : connector.key === 'google_drive' && + envVar('VITE_GOOGLE_CLIENT_ID') ? ( + { + setSelectedFiles(fileIds); + setSelectedFolders(folderIds); + }} + /> + ) : PICKER_CONNECTORS.has(connector.key) ? ( + { + setSelectedFiles(fileIds); + setSelectedFolders(folderIds); + }} + /> + ) : ( + { + setSetupValues(values); + const first = Object.values(values).find(Boolean); + if (first) prefillName(first); + }} + /> + )} +
+ { + setNameTouched(true); + setSourceName(e.target.value); + }} + /> + + + +
+ + + )} +
+ ); + + const renderDone = () => ( +
+

{summary}

+ {toolCount > 0 && connectionId && ( +
+ + + + {t('settings.connectors.wizard.toolsHeading', { + count: toolCount, + formatted: formatCount(toolCount), + })} + + +
+ {tools.map((tool) => ( + + ))} +
+ + + +
+ )} +
+ ); + + const title = + step === 'signin' + ? mode === 'reconnect' + ? t('settings.connectors.wizard.reconnectTitle', { + name, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.wizard.connectTitle', { + name, + interpolation: { escapeValue: false }, + }) + : step === 'setup' + ? toolsFirst + ? t('settings.connectors.wizard.doneTitle', { + name, + interpolation: { escapeValue: false }, + }) + : offerTools || !syncing + ? t('settings.connectors.wizard.chooseWhatToSetUp') + : t('settings.connectors.wizard.chooseWhatToSync') + : t('settings.connectors.wizard.doneTitle', { + name, + interpolation: { escapeValue: false }, + }); + + const footer = + step === 'signin' ? ( + usesOAuth || isMcpPreset ? ( + { + mcp.cancel(); + onClose(); + }} + submitLabel={t('settings.connectors.wizard.signIn', { + name, + interpolation: { escapeValue: false }, + })} + onSubmit={isMcpPreset ? startMcpSignIn : startSignIn} + pending={isMcpPreset && (mcp.pending || pending)} + /> + ) : ( + + ) + ) : step === 'setup' && !offerTools && !syncing ? ( + // Only the Knowledge question, answered no: Skip would do the same. + + ) : step === 'setup' ? ( + + mode === 'sync' ? onClose() : toolsFirst ? finish() : setStep('done') + } + submitLabel={ + offerTools + ? t('settings.connectors.wizard.continue') + : t('modals.uploadDoc.train') + } + onSubmit={addSource} + pending={pending} + disabled={!canAddSource} + /> + ) : ( + + ); + + return ( + !open && (step === 'done' ? finish() : onClose())} + title={title} + size={step === 'setup' ? 'xl' : 'lg'} + mobileVariant="sheet" + footer={footer} + > + {step === 'signin' && renderSignIn()} + {step === 'setup' && renderSetup()} + {step === 'done' && renderDone()} + + ); +} diff --git a/frontend/src/connectors/ConnectionDrawer.test.tsx b/frontend/src/connectors/ConnectionDrawer.test.tsx new file mode 100644 index 00000000..04aa9dad --- /dev/null +++ b/frontend/src/connectors/ConnectionDrawer.test.tsx @@ -0,0 +1,452 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); + +const connectors = vi.hoisted(() => ({ + getConnection: vi.fn(), + getCatalog: vi.fn(async () => ({ success: true, connectors: [] })), + listConnections: vi.fn(async () => ({ success: true, connections: [] })), + setup: vi.fn(), + renameConnection: vi.fn(), + setWrites: vi.fn(), +})); +vi.mock('../api/services/connectorsService', () => ({ default: connectors })); + +const users = vi.hoisted(() => ({ + updateToolStatus: vi.fn(), + syncConnector: vi.fn(), + syncSource: vi.fn(), +})); +vi.mock('../api/services/userService', () => ({ default: users })); + +import actionToastReducer, { + selectActionToast, +} from '../notifications/actionToastSlice'; +import ConnectionDrawer from './ConnectionDrawer'; +import connectorsReducer from './connectorsSlice'; +import type { ConnectorDefinition } from './types'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const DRIVE = { + key: 'google_drive', + name: 'Google Drive', + description: 'Sync files.', + icon: 'drive', + publisher: 'built_in', + auth_kind: 'oauth', + capabilities: ['sync'], + setup: { tools: 'off', sync: 'ask' }, + sync_ingestor: 'google_drive', + available: true, + needs_setup: false, + missing_settings: [], +} as unknown as ConnectorDefinition; + +const DETAIL = { + id: 'conn-1', + connector_key: 'google_drive', + auth_kind: 'oauth', + account_label: 'lena@example.com', + status: 'connected', + last_error: null, + server_url: null, + sources: [ + { + id: 'src-1', + name: 'Handbook', + type: 'connector:file', + last_sync: null, + sync_frequency: 'weekly', + sync_state: 'active', + }, + ], + tools: [ + { + id: 'tool-1', + name: 'mcp_tool', + display_name: 'Drive search', + status: true, + credential_mode: 'owner', + actions: [], + }, + ], +}; + +describe('ConnectionDrawer', () => { + let container: HTMLDivElement; + let root: Root; + let store: ReturnType; + const makeStore = (extra: Record[] = []) => + configureStore({ + reducer: { + connectors: connectorsReducer, + actionToast: actionToastReducer, + preference: (state = { token: null }) => state, + }, + preloadedState: { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [DRIVE], + connections: [ + { + id: 'conn-1', + connector_key: 'google_drive', + status: 'connected', + }, + { id: 'conn-gh', connector_key: 'github', status: 'connected' }, + ...extra, + ], + }, + preference: { token: null }, + }, + } as Parameters[0]); + + beforeEach(() => { + Object.values(users).forEach((fn) => fn.mockReset()); + connectors.getConnection.mockResolvedValue({ + success: true, + connection: DETAIL, + }); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + connector: ConnectorDefinition = DRIVE, + options: { + extra?: Record[]; + initialConnectionId?: string; + } = {}, + ) => { + store = makeStore(options.extra); + await act(async () => { + root.render( + + + , + ); + }); + }; + + const button = (text: string) => + Array.from( + document.body.querySelectorAll('button'), + ).find((b) => b.textContent === text)!; + + it('shows one account at a time, switched from a dropdown', async () => { + connectors.getConnection.mockImplementation(async (id: string) => ({ + success: true, + connection: + id === 'conn-2' + ? { ...DETAIL, id: 'conn-2', account_name: 'Max' } + : { ...DETAIL, account_name: 'Lena' }, + })); + await render(DRIVE, { + extra: [ + { id: 'conn-2', connector_key: 'google_drive', status: 'connected' }, + ], + initialConnectionId: 'conn-2', + }); + const text = () => document.body.textContent ?? ''; + // The account the panel was opened for, and only that one. + expect(text()).toContain('Max'); + expect(text()).not.toContain('Lena'); + + const trigger = document.body.querySelector( + '[aria-label="settings.connectors.detail.accountPicker"]', + )!; + await act(async () => { + trigger.dispatchEvent( + new PointerEvent('pointerdown', { + bubbles: true, + button: 0, + pointerType: 'mouse', + }), + ); + }); + const other = Array.from( + document.body.querySelectorAll('[role="option"]'), + ).find((o) => o.textContent?.includes('Lena'))!; + await act(async () => other.click()); + expect(text()).toContain('Lena'); + expect(text()).not.toContain('Max'); + }); + + it('has no account dropdown with a single account', async () => { + await render(); + expect( + document.body.querySelector( + '[aria-label="settings.connectors.detail.accountPicker"]', + ), + ).toBeNull(); + }); + + // The switch is the owner's "In my chats", the same one as on the tool's + // card, not an on/off for everyone the tool is shared with. + it('labels the tool switch "In my chats"', async () => { + await render(); + const toggle = document.body.querySelector( + '[aria-label="settings.tools.useInMyChatsAria"]', + )!; + expect(toggle.getAttribute('role')).toBe('switch'); + const label = document.body.querySelector( + `label[for="${toggle.id}"]`, + ); + expect(label?.textContent).toBe('settings.tools.inMyChats'); + }); + + it('turns a tool on and off from its connection page', async () => { + users.updateToolStatus.mockResolvedValue({ ok: true }); + await render(); + const toggle = document.body.querySelector( + '[aria-label="settings.tools.useInMyChatsAria"]', + )!; + expect(toggle.getAttribute('aria-checked')).toBe('true'); + await act(async () => toggle.click()); + expect(users.updateToolStatus).toHaveBeenCalledWith( + { id: 'tool-1', status: false }, + null, + ); + expect(toggle.getAttribute('aria-checked')).toBe('false'); + }); + + it('puts the switch back when the change fails', async () => { + users.updateToolStatus.mockResolvedValue({ ok: false }); + await render(); + const toggle = document.body.querySelector( + '[aria-label="settings.tools.useInMyChatsAria"]', + )!; + await act(async () => toggle.click()); + expect(toggle.getAttribute('aria-checked')).toBe('true'); + expect( + selectActionToast( + store.getState() as Parameters[0], + )?.variant, + ).toBe('destructive'); + }); + + it('syncs a source now through its connection', async () => { + users.syncConnector.mockResolvedValue({ + json: async () => ({ success: true }), + }); + await render(); + await act(async () => button('settings.connectors.detail.syncNow').click()); + expect(users.syncConnector).toHaveBeenCalledWith('src-1', null); + expect(users.syncSource).not.toHaveBeenCalled(); + expect( + selectActionToast( + store.getState() as Parameters[0], + )?.variant, + ).toBe('success'); + }); + + it('says in plain words that an expired sign-in needs redoing', async () => { + connectors.getConnection.mockResolvedValue({ + success: true, + connection: { + ...DETAIL, + status: 'reconnect_needed', + last_error: 'invalid_grant: token expired', + }, + }); + await render(); + const text = document.body.textContent ?? ''; + expect(text).toContain('settings.connectors.detail.expired'); + expect(text).toContain('settings.connectors.detail.account'); + expect(text).not.toContain('settings.connectors.detail.connectedAs'); + // The provider's message stays available on hover, not as the sentence. + expect(text).not.toContain('invalid_grant'); + expect( + document.body.querySelector('[title="invalid_grant: token expired"]'), + ).not.toBeNull(); + }); + + const openAccountMenu = async () => { + const trigger = document.body.querySelector( + '[aria-label="settings.connectors.detail.accountMenu"]', + )!; + await act(async () => { + trigger.dispatchEvent( + new PointerEvent('pointerdown', { bubbles: true, button: 0 }), + ); + trigger.click(); + }); + }; + + it('shows the name an account was given, with the account under it', async () => { + connectors.getConnection.mockResolvedValue({ + success: true, + connection: { ...DETAIL, account_name: 'Work' }, + }); + await render(); + const text = document.body.textContent ?? ''; + expect(text).toContain('Work'); + expect(text).toContain('settings.connectors.detail.connectedAs'); + }); + + it('renames an account from its menu', async () => { + connectors.renameConnection.mockResolvedValue({ + success: true, + connection: { ...DETAIL, account_name: 'Work' }, + }); + await render(); + await openAccountMenu(); + const item = Array.from( + document.body.querySelectorAll('[role="menuitem"]'), + ).find((el) => el.textContent === 'settings.connectors.detail.rename')!; + await act(async () => item.click()); + const input = + document.body.querySelector('#rename-account')!; + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, 'Work'); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + await act(async () => button('settings.connectors.rename.save').click()); + expect(connectors.renameConnection).toHaveBeenCalledWith( + 'conn-1', + 'Work', + null, + ); + }); + + it('labels synced content as Knowledge from this connection', async () => { + await render(); + expect(document.body.textContent).toContain( + 'settings.connectors.detail.sources', + ); + expect(document.body.textContent).not.toContain( + 'settings.connectors.publisher', + ); + }); + + describe('GitHub', () => { + const GITHUB = { + ...DRIVE, + key: 'github', + name: 'GitHub', + icon: 'github', + auth_kind: 'api_key', + capabilities: ['sync', 'read'], + setup: { tools: 'ask', sync: 'ask' }, + sync_ingestor: 'github', + tool_templates: ['mcp_tool'], + } as unknown as ConnectorDefinition; + const TOKEN_DETAIL = { + ...DETAIL, + connector_key: 'github', + auth_kind: 'api_key', + account_label: 'octocat', + sources: [], + tools: [], + }; + + beforeEach(() => { + connectors.setup.mockReset(); + connectors.setWrites.mockReset(); + connectors.getConnection.mockResolvedValue({ + success: true, + connection: TOKEN_DETAIL, + }); + }); + + it('names a token connection by its account, not as a key hint', async () => { + await render(GITHUB); + expect(document.body.textContent).toContain( + 'settings.connectors.detail.connectedAs', + ); + expect(document.body.textContent).not.toContain( + 'settings.connectors.detail.keyEnding', + ); + }); + + const GITHUB_TOOL = { + id: 'tool-gh', + name: 'mcp_tool', + display_name: 'GitHub', + status: true, + credential_mode: 'owner', + actions: [], + }; + const writesSwitch = () => + document.body.querySelector('#writes-conn-1'); + + it('switches a connection between reading and making changes', async () => { + connectors.getConnection.mockResolvedValue({ + success: true, + connection: { ...TOKEN_DETAIL, tools: [GITHUB_TOOL], writes: false }, + }); + connectors.setWrites.mockResolvedValue({ success: true, writes: true }); + await render({ ...GITHUB, writes_allowed: true }); + expect(writesSwitch()!.getAttribute('aria-checked')).toBe('false'); + const loads = connectors.getConnection.mock.calls.length; + await act(async () => writesSwitch()!.click()); + expect(connectors.setWrites).toHaveBeenCalledWith('conn-1', true, null); + // The page reloads the connection, with the new endpoint's actions. + expect(connectors.getConnection.mock.calls.length).toBeGreaterThan(loads); + }); + + it('says when switching failed', async () => { + connectors.getConnection.mockResolvedValue({ + success: true, + connection: { ...TOKEN_DETAIL, tools: [GITHUB_TOOL], writes: false }, + }); + connectors.setWrites.mockResolvedValue({ + success: false, + code: 'writes_forbidden', + }); + await render({ ...GITHUB, writes_allowed: true }); + await act(async () => writesSwitch()!.click()); + const toast = selectActionToast( + store.getState() as Parameters[0], + ); + expect(toast?.variant).toBe('destructive'); + expect(toast?.message).toBe('settings.connectors.github.writesForbidden'); + }); + + it('hides the switch when an admin turned changes off', async () => { + connectors.getConnection.mockResolvedValue({ + success: true, + connection: { ...TOKEN_DETAIL, tools: [GITHUB_TOOL], writes: false }, + }); + await render({ ...GITHUB, writes_allowed: false }); + expect(writesSwitch()).toBeNull(); + }); + + it('adds the tools a connection skipped during setup', async () => { + connectors.setup.mockResolvedValue({ success: true, tools: [] }); + await render(GITHUB); + await act(async () => + button('settings.connectors.detail.addTools').click(), + ); + expect(connectors.setup).toHaveBeenCalledWith( + 'conn-1', + { create_tools: true }, + null, + ); + }); + }); +}); diff --git a/frontend/src/connectors/ConnectionDrawer.tsx b/frontend/src/connectors/ConnectionDrawer.tsx new file mode 100644 index 00000000..4c74938b --- /dev/null +++ b/frontend/src/connectors/ConnectionDrawer.tsx @@ -0,0 +1,1210 @@ +import { + CircleAlert, + Pencil, + Plus, + RefreshCw, + RotateCw, + Trash2, + TriangleAlert, + Unplug, +} from 'lucide-react'; +import { useCallback, useEffect, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; + +import connectorsService from '../api/services/connectorsService'; +import userService from '../api/services/userService'; +import { Alert, AlertDescription } from '../components/ui/alert'; +import { Badge } from '../components/ui/badge'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { ActionMenu, type MenuOption } from '../components/ui/dropdown-menu'; +import { EmptyState } from '../components/ui/empty-state'; +import { FormField } from '../components/ui/form-field'; +import { Input } from '../components/ui/input'; +import { Label } from '../components/ui/label'; +import { ListRow, ListRows } from '../components/ui/list-row'; +import { LoadingState } from '../components/ui/loading-state'; +import { Modal, ModalActions } from '../components/ui/modal'; +import { SectionHeader } from '../components/ui/section-header'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '../components/ui/select'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import { + Sheet, + SheetContent, + SheetDescription, + SheetTitle, +} from '../components/ui/sheet'; +import { ToggleGroup, ToggleGroupItem } from '../components/ui/toggle-group'; +import ConfirmationModal from '../modals/ConfirmationModal'; +import { showActionToast } from '../notifications/actionToastSlice'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; +import { formatCount, formatDateTime } from '../utils/dateTimeUtils'; +import { ACCOUNT_NAME_MAX } from './accounts'; +import { CapabilityBadges } from './ConnectorCard'; +import ConnectorIcon from './ConnectorIcon'; +import ConnectorSetupNotice from './ConnectorSetupNotice'; +import { + connectionNeedsSignIn, + loadConnectors, + selectConnections, +} from './connectorsSlice'; +import { connectorDescription, connectorName, isKeyHint } from './i18n'; +import ToolPermissions from './ToolPermissions'; +import type { + ConnectionDetail, + ConnectionSource, + ConnectionTool, + ConnectionStatus, + ConnectorDefinition, +} from './types'; +import type { LaunchOptions } from './useConnectorLauncher'; + +const STATUS_VARIANT: Record< + ConnectionStatus, + 'success' | 'warning' | 'neutral' | 'destructive' +> = { + connected: 'success', + reconnect_needed: 'warning', + disconnected: 'neutral', + error: 'destructive', + pending: 'neutral', +}; + +/** + * "Connected as …", or the key hint for pasted credentials (a GitHub token + * is named after its account instead). A connection that is not working + * names its account without claiming it is connected. + */ +function useAccountTitle() { + const { t } = useTranslation(); + return (detail: ConnectionDetail) => + detail.auth_kind === 'api_key' && isKeyHint(detail.account_label) + ? t('settings.connectors.detail.keyEnding', { + hint: detail.account_label, + interpolation: { escapeValue: false }, + }) + : t( + detail.status === 'connected' + ? 'settings.connectors.detail.connectedAs' + : 'settings.connectors.detail.account', + { + account: detail.account_label, + interpolation: { escapeValue: false }, + }, + ); +} + +/** What is wrong, in plain words rather than the provider's message. */ +function useProblem() { + const { t } = useTranslation(); + return (detail: ConnectionDetail): string | null => + detail.status === 'reconnect_needed' + ? t('settings.connectors.detail.expired') + : detail.status === 'error' + ? t('settings.connectors.detail.broken') + : detail.last_error; +} + +function RemoveConnectionModal({ + detail, + name, + onClose, + onRemoved, +}: { + detail: ConnectionDetail; + name: string; + onClose: () => void; + onRemoved: () => void; +}) { + const { t } = useTranslation(); + const token = useSelector(selectToken); + const [sources, setSources] = useState<'keep' | 'delete'>('keep'); + const [tools, setTools] = useState<'keep' | 'delete'>('delete'); + const [pending, setPending] = useState(false); + const [failed, setFailed] = useState(false); + + const remove = () => { + setPending(true); + setFailed(false); + connectorsService + .remove(detail.id, { sources, tools }, token) + .then((data) => { + if (!data?.success) throw new Error('remove failed'); + onRemoved(); + }) + .catch(() => setFailed(true)) + .finally(() => setPending(false)); + }; + + return ( + !open && onClose()} + title={t('settings.connectors.remove.title', { + name, + interpolation: { escapeValue: false }, + })} + description={t('settings.connectors.remove.description')} + footer={ + + } + > +
+ {failed && ( + + + + {t('settings.connectors.remove.failed')} + + + )} + {detail.sources.length > 0 && ( + + + value && setSources(value as 'keep' | 'delete') + } + > + + {t('settings.connectors.remove.keepSources')} + + + {t('settings.connectors.remove.deleteSources')} + + + + )} + {detail.tools.length > 0 && ( + + + value && setTools(value as 'keep' | 'delete') + } + > + + {t('settings.connectors.remove.deleteTools')} + + + {t('settings.connectors.remove.keepTools')} + + + + )} +
+ + ); +} + +/** Names an account ("Alerts bot"), so two accounts of a service read apart. */ +function RenameAccountModal({ + detail, + onClose, + onRenamed, +}: { + detail: ConnectionDetail; + onClose: () => void; + onRenamed: () => void; +}) { + const { t } = useTranslation(); + const token = useSelector(selectToken); + const [name, setName] = useState(detail.account_name ?? ''); + const [pending, setPending] = useState(false); + const [failed, setFailed] = useState(false); + + const save = () => { + setPending(true); + setFailed(false); + connectorsService + .renameConnection(detail.id, name.trim(), token) + .then((data) => { + if (!data?.success) throw new Error('rename failed'); + onRenamed(); + }) + .catch(() => setFailed(true)) + .finally(() => setPending(false)); + }; + + return ( + !open && onClose()} + title={t('settings.connectors.rename.title')} + description={t('settings.connectors.rename.description')} + footer={ + + } + > +
+ {failed && ( + + + + {t('settings.connectors.rename.failed')} + + + )} + setName(e.target.value)} + onKeyDown={(e) => { + if ( + e.key === 'Enter' && + name.trim() !== (detail.account_name ?? '') + ) + save(); + }} + /> +
+ + ); +} + +/** + * A connection tool's "In my chats" switch: the owner's own preference, the + * same one as on the tool's card. Teammates the tool is shared with keep + * their own, so it never turns the tool off for them. + */ +function ToolSwitch({ + tool, + onToggle, +}: { + tool: ConnectionTool; + onToggle: (toolId: string, on: boolean) => Promise; +}) { + const { t } = useTranslation(); + const [on, setOn] = useState(tool.status); + useEffect(() => setOn(tool.status), [tool.status]); + const id = `connection-tool-in-chats-${tool.id}`; + return ( + + + { + const next = checked === true; + setOn(next); + if (!(await onToggle(tool.id, next))) setOn(!next); + }} + /> + + ); +} + +/** + * Lets agents make changes through a connection (GitHub's issues, comments + * and pull requests) or only read. Switching re-reads the tool's actions. + */ +function WritesSwitch({ + detail, + onSwitch, +}: { + detail: ConnectionDetail; + onSwitch: (detail: ConnectionDetail, allow: boolean) => Promise; +}) { + const { t } = useTranslation(); + const [pending, setPending] = useState(false); + const id = `writes-${detail.id}`; + return ( + + + { + setPending(true); + onSwitch(detail, checked === true).finally(() => setPending(false)); + }} + /> + + + ); +} + +/** + * Which of several accounts of a service the panel shows. The rest of the + * panel is that one account: its knowledge and its tools. + */ +function AccountPicker({ + accounts, + value, + onChange, +}: { + accounts: ConnectionDetail[]; + value: string; + onChange: (id: string) => void; +}) { + const { t } = useTranslation(); + const accountTitle = useAccountTitle(); + return ( + + ); +} + +/** + * The account to show from ``accounts``: the one picked, else one that needs + * signing in again (so a problem is not hidden behind another account), else + * the first. + */ +function shownAccount( + accounts: ConnectionDetail[], + picked: string | undefined, +): ConnectionDetail | undefined { + return ( + accounts.find((account) => account.id === picked) ?? + accounts.find(connectionNeedsSignIn) ?? + accounts[0] + ); +} + +function AccountSection({ + connector, + detail, + onReconnect, + onDisconnect, + onRemove, + onRename, + onSyncMore, + onRefreshTools, + onToggleTool, + onSyncNow, + onAddTools, + onSwitchWrites, +}: { + connector: ConnectorDefinition; + detail: ConnectionDetail; + onReconnect: (detail: ConnectionDetail) => void; + onDisconnect: (detail: ConnectionDetail) => void; + onRemove: (detail: ConnectionDetail) => void; + onRename: (detail: ConnectionDetail) => void; + onSyncMore: (detail: ConnectionDetail) => void; + onRefreshTools: (detail: ConnectionDetail) => Promise; + /** Turns a tool of this connection on or off for agents and chat. */ + onToggleTool: (toolId: string, on: boolean) => Promise; + onSyncNow: (source: ConnectionSource) => void; + /** Creates the tools a connection skipped while it was set up (GitHub's). */ + onAddTools: (detail: ConnectionDetail) => Promise; + /** Lets its tool make changes or only read (GitHub's). */ + onSwitchWrites: (detail: ConnectionDetail, allow: boolean) => Promise; +}) { + const { t } = useTranslation(); + const accountTitle = useAccountTitle(); + const problem = useProblem(); + const [refreshing, setRefreshing] = useState(false); + const [addingTools, setAddingTools] = useState(false); + const canAddTools = + connector.setup.tools === 'ask' && + (connector.tool_templates?.length ?? 0) > 0 && + connector.publisher !== 'custom' && + detail.tools.length === 0 && + detail.status === 'connected'; + const canSync = connector.setup.sync !== 'off' && !!connector.sync_ingestor; + const isMcp = detail.tools.some((tool) => tool.name === 'mcp_tool'); + // Hidden once an admin forbids changes; its tool then only reads anyway. + const canSwitchWrites = + !!connector.writes_allowed && + detail.writes !== null && + detail.writes !== undefined; + const menu: MenuOption[] = [ + { + icon: Pencil, + label: t('settings.connectors.detail.rename'), + onClick: () => onRename(detail), + }, + ]; + if (detail.status !== 'disconnected') { + menu.push({ + icon: Unplug, + label: t('settings.connectors.detail.disconnect'), + onClick: () => onDisconnect(detail), + variant: 'destructive', + }); + } + menu.push({ + icon: Trash2, + label: t('settings.connectors.detail.remove'), + onClick: () => onRemove(detail), + variant: 'destructive', + }); + + // An expired or failing sign-in gets its own box under the account, with + // room to say what happened at any width. + const broken = + detail.status === 'reconnect_needed' || detail.status === 'error'; + + return ( +
+
+ + + {detail.last_error} + ) : undefined + } + trailing={ +
+ + {t(`settings.connectors.connectionStatus.${detail.status}`)} + + {!broken && detail.status !== 'connected' && ( + + )} + +
+ } + /> + + + {broken && ( + + + {/* The provider's own message stays on hover, for debugging. */} + + {problem(detail)} + +
+ +
+ + )} +
+ {canSync && ( +
+ onSyncMore(detail)} + > + + {t('settings.connectors.detail.syncMore')} + + ) : undefined + } + /> + {detail.sources.length === 0 ? ( + + ) : ( + + + {detail.sources.map((source) => ( + + {t('settings.connectors.detail.paused')} + + ) : ( + + ) + } + /> + ))} + + + )} +
+ )} + {canAddTools && ( +
+ { + setAddingTools(true); + onAddTools(detail).finally(() => setAddingTools(false)); + }} + > + + {t('settings.connectors.detail.addTools')} + + } + /> + +
+ )} + {detail.tools.length > 0 && ( +
+ { + setRefreshing(true); + onRefreshTools(detail).finally(() => setRefreshing(false)); + }} + > + + {t('settings.connectors.detail.refreshTools')} + + ) : undefined + } + /> + {detail.tools.map((tool) => ( +
+ } + /> + {canSwitchWrites && tool.name === 'mcp_tool' && ( + + )} + +
+ ))} +
+ )} +
+ ); +} + +/** + * Everything about one connector: its accounts, the sources each syncs and + * the tools each provides, with their permissions. Opens from a Connectors + * page card. + */ +export default function ConnectionDrawer({ + connector, + parts = [], + initialConnectionId, + onClose, + onConnect, +}: { + connector: ConnectorDefinition | null; + /** The same service offered another way, shown here (Jira & Confluence). */ + parts?: ConnectorDefinition[]; + /** The account to show first, e.g. the one behind the tool it opened from. */ + initialConnectionId?: string; + onClose: () => void; + onConnect: (connector: ConnectorDefinition, options?: LaunchOptions) => void; +}) { + const { t } = useTranslation(); + const dispatch = useDispatch(); + const token = useSelector(selectToken); + const connections = useSelector(selectConnections); + const [details, setDetails] = useState([]); + const [loading, setLoading] = useState(false); + const [failed, setFailed] = useState(false); + const [reloadKey, setReloadKey] = useState(0); + const [toDisconnect, setToDisconnect] = useState( + null, + ); + const [toRemove, setToRemove] = useState(null); + const [toRename, setToRename] = useState(null); + // The account shown per service (the connector and each of its parts). + const [picked, setPicked] = useState>({}); + useEffect(() => { + setPicked({}); + }, [connector?.key, initialConnectionId]); + const pickedIn = (key: string, accounts: ConnectionDetail[]) => + picked[key] ?? + (accounts.some((account) => account.id === initialConnectionId) + ? initialConnectionId + : undefined); + + const keys = [connector?.key, ...parts.map((part) => part.key)]; + const accountIds = connections + .filter((connection) => keys.includes(connection.connector_key)) + .map((connection) => `${connection.id}:${connection.status}`) + .join(','); + + useEffect(() => { + if (!connector) return; + const ids = accountIds + ? accountIds.split(',').map((entry) => entry.split(':')[0]) + : []; + let cancelled = false; + setLoading(true); + setFailed(false); + Promise.all(ids.map((id) => connectorsService.getConnection(id, token))) + .then((responses) => { + if (cancelled) return; + if (responses.some((response) => !response?.success)) { + setFailed(true); + return; + } + setDetails(responses.map((response) => response.connection)); + }) + .catch(() => !cancelled && setFailed(true)) + .finally(() => !cancelled && setLoading(false)); + return () => { + cancelled = true; + }; + }, [connector, accountIds, token, reloadKey]); + + const refresh = useCallback(() => { + dispatch(loadConnectors({ token })); + setReloadKey((key) => key + 1); + }, [dispatch, token]); + + const confirmDisconnect = () => { + if (!toDisconnect) return; + connectorsService + .disconnect(toDisconnect.id, token) + .then((data) => { + if (!data?.success) throw new Error('disconnect failed'); + }) + .catch(() => + dispatch( + showActionToast({ + variant: 'destructive', + message: t('settings.connectors.disconnect.failed'), + }), + ), + ) + .finally(() => { + setToDisconnect(null); + refresh(); + }); + }; + + const reconnect = (detail: ConnectionDetail) => { + if (!connector) return; + // A part's account (Jira & Confluence under Confluence) reconnects + // through its own connector. + const target = + parts.find((part) => part.key === detail.connector_key) ?? connector; + const mcpTool = detail.tools.find((tool) => tool.name === 'mcp_tool'); + onConnect(target, { + mode: 'reconnect', + connectionId: detail.id, + mcpServer: + mcpTool && detail.server_url + ? { + id: mcpTool.id, + displayName: mcpTool.display_name, + server_url: detail.server_url, + auth_type: detail.auth_kind === 'mcp_oauth' ? 'oauth' : 'none', + } + : undefined, + }); + }; + + const toggleTool = async (toolId: string, on: boolean) => { + try { + const response = await userService.updateToolStatus( + { id: toolId, status: on }, + token, + ); + if (!response.ok) throw new Error('toggle failed'); + return true; + } catch { + dispatch( + showActionToast({ + variant: 'destructive', + message: t('settings.connectors.detail.toolSwitchFailed'), + }), + ); + return false; + } + }; + + const syncNow = async (source: ConnectionSource) => { + try { + // Drive, SharePoint and Confluence sources sync through the connector + // endpoint; S3 and Reddit through the remote-source one. + const response = source.type?.startsWith('connector') + ? await userService.syncConnector(source.id, token) + : await userService.syncSource({ source_id: source.id }, token); + const data = await response.json(); + if (!data?.success) throw new Error('sync failed'); + dispatch( + showActionToast({ + variant: 'success', + message: t('settings.connectors.detail.syncStarted', { + name: source.name, + interpolation: { escapeValue: false }, + }), + }), + ); + } catch { + dispatch( + showActionToast({ + variant: 'destructive', + message: t('settings.connectors.detail.syncFailed'), + }), + ); + } + }; + + const addTools = async (detail: ConnectionDetail) => { + const data = await connectorsService + .setup(detail.id, { create_tools: true }, token) + .catch(() => null); + if (!data?.success) { + dispatch( + showActionToast({ + variant: 'destructive', + message: + data?.code === 'tools_unavailable' + ? t('settings.connectors.wizard.toolsUnavailable', { + name: connector ? connectorName(t, connector) : detail.name, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.detail.addToolsFailed'), + }), + ); + } + refresh(); + }; + + const switchWrites = async (detail: ConnectionDetail, allow: boolean) => { + const data = await connectorsService + .setWrites(detail.id, allow, token) + .catch(() => null); + if (!data?.success) { + dispatch( + showActionToast({ + variant: 'destructive', + message: + data?.code === 'writes_forbidden' + ? t('settings.connectors.github.writesForbidden') + : data?.code === 'tools_unavailable' + ? t('settings.connectors.wizard.toolsUnavailable', { + name: connector ? connectorName(t, connector) : detail.name, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.github.writesFailed'), + }), + ); + } + refresh(); + }; + + const refreshTools = async (detail: ConnectionDetail) => { + const data = await connectorsService.refreshTools(detail.id, token); + dispatch( + showActionToast( + data?.success + ? { + variant: 'success', + message: t('settings.connectors.detail.refreshed', { + added: formatCount(data.added?.length ?? 0), + removed: formatCount(data.removed?.length ?? 0), + }), + } + : { + variant: 'destructive', + message: t('settings.connectors.detail.refreshFailed'), + }, + ), + ); + refresh(); + }; + + if (!connector) return null; + const name = connectorName(t, connector); + const ownDetails = details.filter( + (detail) => detail.connector_key === connector.key, + ); + + return ( + <> + !open && onClose()}> + +
+ {/* pr-12 keeps the header clear of the close X. */} +
+ + + + {name} +
+ + {connectorDescription(t, connector)} + + + + {connector.publisher === 'custom' && ( + + + + {t('settings.connectors.unverified')} + + + )} + + {connector.needs_setup && ( + + )} + +
+ 0 ? ( + + ) : undefined + } + /> + {loading && details.length === 0 ? ( + + ) : failed ? ( + setReloadKey((key) => key + 1)} + > + {t('retry')} + + } + /> + ) : ownDetails.length === 0 ? ( + onConnect(connector)} + > + {t('settings.connectors.status.connect')} + + ) : undefined + } + /> + ) : ( +
+ {ownDetails.length > 1 && ( + + setPicked((state) => ({ + ...state, + [connector.key]: id, + })) + } + /> + )} + {[ + shownAccount( + ownDetails, + pickedIn(connector.key, ownDetails), + )!, + ].map((detail) => ( + + onConnect(connector, { + mode: 'sync', + connectionId: d.id, + }) + } + onRefreshTools={refreshTools} + onToggleTool={toggleTool} + onSyncNow={syncNow} + onAddTools={addTools} + onSwitchWrites={switchWrites} + /> + ))} +
+ )} +
+ + {parts.map((part) => { + const partDetails = details.filter( + (detail) => detail.connector_key === part.key, + ); + return ( +
+ onConnect(part)} + > + {t('settings.connectors.status.connect')} + + ) : undefined + } + /> + + {partDetails.length > 1 && ( + + setPicked((state) => ({ ...state, [part.key]: id })) + } + /> + )} + {partDetails + .filter( + (detail) => + detail.id === + shownAccount( + partDetails, + pickedIn(part.key, partDetails), + )?.id, + ) + .map((detail) => ( + + onConnect(part, { mode: 'sync', connectionId: d.id }) + } + onRefreshTools={refreshTools} + onToggleTool={toggleTool} + onSyncNow={syncNow} + onAddTools={addTools} + onSwitchWrites={switchWrites} + /> + ))} +
+ ); + })} +
+ + + state === 'INACTIVE' && setToDisconnect(null)} + handleSubmit={confirmDisconnect} + submitLabel={t('settings.connectors.detail.disconnect')} + variant="destructive" + /> + {toRename && ( + setToRename(null)} + onRenamed={() => { + setToRename(null); + refresh(); + }} + /> + )} + {toRemove && ( + setToRemove(null)} + onRemoved={() => { + setToRemove(null); + refresh(); + }} + /> + )} + + ); +} diff --git a/frontend/src/connectors/ConnectionHealthDot.test.tsx b/frontend/src/connectors/ConnectionHealthDot.test.tsx new file mode 100644 index 00000000..7b950f63 --- /dev/null +++ b/frontend/src/connectors/ConnectionHealthDot.test.tsx @@ -0,0 +1,67 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); + +import ConnectionHealthDot from './ConnectionHealthDot'; +import connectorsReducer from './connectorsSlice'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +describe('ConnectionHealthDot', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (statuses: string[]) => { + const store = configureStore({ + reducer: { connectors: connectorsReducer }, + preloadedState: { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [], + connections: statuses.map((status, i) => ({ id: `c${i}`, status })), + }, + }, + } as Parameters[0]); + await act(async () => { + root.render( + + + , + ); + }); + }; + + it('shows while a connection needs signing in again', async () => { + await render(['connected', 'reconnect_needed']); + const dot = container.querySelector( + '[data-testid="connection-health-dot"]', + ); + expect(dot?.getAttribute('aria-label')).toBe( + 'settings.connectors.health.navDot', + ); + }); + + it('stays away when every connection works', async () => { + await render(['connected', 'disconnected']); + expect(container.innerHTML).toBe(''); + }); +}); diff --git a/frontend/src/connectors/ConnectionHealthDot.tsx b/frontend/src/connectors/ConnectionHealthDot.tsx new file mode 100644 index 00000000..9b83179e --- /dev/null +++ b/frontend/src/connectors/ConnectionHealthDot.tsx @@ -0,0 +1,23 @@ +import { useTranslation } from 'react-i18next'; +import { useSelector } from 'react-redux'; + +import { selectConnectionsNeedAttention } from './connectorsSlice'; + +/** + * A warning dot on the nav entries that lead to Connectors while a + * connection needs signing in again. Unlike the reconnect toast it does not + * expire, so the signal stays until the connection is fixed. + */ +export default function ConnectionHealthDot() { + const { t } = useTranslation(); + const needsAttention = useSelector(selectConnectionsNeedAttention); + if (!needsAttention) return null; + return ( + + ); +} diff --git a/frontend/src/connectors/ConnectorCard.tsx b/frontend/src/connectors/ConnectorCard.tsx new file mode 100644 index 00000000..30c14b12 --- /dev/null +++ b/frontend/src/connectors/ConnectorCard.tsx @@ -0,0 +1,125 @@ +import { Plus } from 'lucide-react'; +import { useTranslation } from 'react-i18next'; + +import { Badge } from '../components/ui/badge'; +import { + Card, + CardDescription, + CardFooter, + CardTitle, +} from '../components/ui/card'; +import { formatCount } from '../utils/dateTimeUtils'; +import ConnectorIcon from './ConnectorIcon'; +import { connectorDescription, connectorName } from './i18n'; +import type { Capability, ConnectorDefinition } from './types'; + +export function CapabilityBadges({ + capabilities, +}: { + capabilities: Capability[]; +}) { + const { t } = useTranslation(); + return ( +
+ {capabilities.map((capability) => ( + // What it does, in plain words; categories, not states, so neutral and + // the status hues stay free for the state badge. + + {t(`settings.connectors.capabilityPlain.${capability}`)} + + ))} +
+ ); +} + +/** The state a catalog card ends on: a badge, or the Connect call to action. */ +export function ConnectorStateBadge({ + connector, +}: { + connector: ConnectorDefinition; +}) { + const { t } = useTranslation(); + switch (connector.state) { + case 'connected': + return ( + + {connector.connected_count > 1 + ? t('settings.connectors.status.connectedCount', { + count: connector.connected_count, + formatted: formatCount(connector.connected_count), + }) + : t('settings.connectors.status.connected')} + + ); + case 'reconnect': + return ( + + {t('settings.connectors.status.reconnect')} + + ); + case 'needs_setup': + return ( + + {t('settings.connectors.status.needsAdminSetup')} + + ); + case 'disabled': + return ( + + {t('settings.connectors.status.disabledByAdmin')} + + ); + default: + return ( + + + ); + } +} + +/** + * A catalog tile. The whole card is the one action: it connects an + * available service and opens the connection details for everything else. + */ +export default function ConnectorCard({ + connector, + onOpen, +}: { + connector: ConnectorDefinition; + onOpen: (connector: ConnectorDefinition) => void; +}) { + const { t } = useTranslation(); + const name = connectorName(t, connector); + return ( + + + + ); +} diff --git a/frontend/src/connectors/ConnectorIcon.tsx b/frontend/src/connectors/ConnectorIcon.tsx new file mode 100644 index 00000000..da2020a9 --- /dev/null +++ b/frontend/src/connectors/ConnectorIcon.tsx @@ -0,0 +1,70 @@ +import { Plug } from 'lucide-react'; +import * as React from 'react'; + +import { cn } from '@/lib/utils'; +import ConfluenceIcon from '../assets/confluence.svg?react'; +import DriveIcon from '../assets/drive.svg?react'; +import RedditIcon from '../assets/reddit.svg?react'; +import S3Icon from '../assets/s3.svg?react'; +import SharePointIcon from '../assets/sharepoint.svg?react'; + +type SvgComponent = React.FC>; + +// Preset logos live in assets/connectors; built-in tool services reuse the +// tool icons. Every asset draws in currentColor unless it is a multi-colour +// brand mark (Brave), so the icon follows the theme through `text-*`. +const presetModules = import.meta.glob('../assets/connectors/*.svg', { + query: '?react', + import: 'default', + eager: true, +}) as Record; +const toolModules = import.meta.glob('../assets/toolIcons/tool_*.svg', { + query: '?react', + import: 'default', + eager: true, +}) as Record; + +const ICONS: Record = { + drive: DriveIcon, + sharepoint: SharePointIcon, + confluence: ConfluenceIcon, + s3: S3Icon, + reddit: RedditIcon, +}; +for (const [path, Component] of Object.entries(presetModules)) { + const match = path.match(/connectors\/(.+)\.svg$/); + if (match) ICONS[match[1]] = Component; +} +for (const [path, Component] of Object.entries(toolModules)) { + const match = path.match(/(tool_.+)\.svg$/); + if (match) ICONS[match[1]] = Component; +} + +type ConnectorIconProps = { + /** Catalog `icon` key, e.g. `drive`, `tool_telegram` or `notion`. */ + icon: string; + className?: string; + /** Accessible name. Omit when the connector's name is next to the icon. */ + title?: string; +}; + +/** + * A connector's logo. Unknown keys (a custom MCP server) fall back to a plug, + * the same glyph the Connectors page uses in the navigation. + */ +export default function ConnectorIcon({ + icon, + className, + title, +}: ConnectorIconProps) { + const Icon = ICONS[icon]; + const a11y = title + ? { role: 'img' as const, 'aria-label': title } + : { 'aria-hidden': true as const }; + if (!Icon) { + return ( + + ); + } + return ; +} diff --git a/frontend/src/connectors/ConnectorSetupNotice.tsx b/frontend/src/connectors/ConnectorSetupNotice.tsx new file mode 100644 index 00000000..56fa90f4 --- /dev/null +++ b/frontend/src/connectors/ConnectorSetupNotice.tsx @@ -0,0 +1,57 @@ +import { ExternalLink, TriangleAlert } from 'lucide-react'; +import { useTranslation } from 'react-i18next'; + +import { Alert, AlertDescription, AlertTitle } from '../components/ui/alert'; +import { Button } from '../components/ui/button'; +import type { ConnectorDefinition } from './types'; + +/** + * "Needs admin setup" for a connector that still lacks server settings: + * the missing setting names for an admin, "ask your admin" for everyone + * else, and the setup guide when there is one. + */ +export default function ConnectorSetupNotice({ + connector, +}: { + connector: Pick; +}) { + const { t } = useTranslation(); + return ( + + + {t('settings.connectors.status.needsAdminSetup')} + +
+ {connector.missing_settings.length > 0 ? ( + <> + {t('settings.connectors.setupSettings')} + + {connector.missing_settings.join(', ')} + + + ) : ( + {t('settings.connectors.askAdmin')} + )} + {connector.docs_url && ( + + )} +
+ + + ); +} diff --git a/frontend/src/connectors/CredentialForm.test.tsx b/frontend/src/connectors/CredentialForm.test.tsx new file mode 100644 index 00000000..d74f6772 --- /dev/null +++ b/frontend/src/connectors/CredentialForm.test.tsx @@ -0,0 +1,61 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: { defaultValue?: string }) => + opts?.defaultValue ?? key, + }), +})); + +import CredentialForm from './CredentialForm'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +describe('CredentialForm', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + it('shows a field hint under the field and links it to the input', async () => { + await act(async () => { + root.render( + undefined} + />, + ); + }); + const input = container.querySelector( + '#connect-telegram-chat_id', + )!; + expect(input.type).toBe('text'); + const hintId = input.getAttribute('aria-describedby')!; + expect(document.getElementById(hintId)?.textContent).toBe( + 'Messages go to this chat.', + ); + expect(container.textContent).toContain('Default chat ID'); + }); +}); diff --git a/frontend/src/connectors/CredentialForm.tsx b/frontend/src/connectors/CredentialForm.tsx new file mode 100644 index 00000000..d568e4e3 --- /dev/null +++ b/frontend/src/connectors/CredentialForm.tsx @@ -0,0 +1,85 @@ +import type { ChangeEvent } from 'react'; +import { useTranslation } from 'react-i18next'; + +import { FormField } from '../components/ui/form-field'; +import { Input } from '../components/ui/input'; +import type { CredentialField } from './types'; + +/** + * The short form an API-key connector asks for, generated from its catalog + * `credential_fields` (or `setup_fields`). Secrets are masked. Labels come + * from `settings.connectors.fields._`, then + * `settings.connectors.fields.`, then the catalog label; a field's hint + * from `settings.connectors.fieldHints._`, then the catalog. + */ +export default function CredentialForm({ + connectorKey, + fields, + values, + onChange, + idPrefix, + labelSurface = 'card', +}: { + /** Picks the connector's own label for a shared key (Telegram's "Bot token"). */ + connectorKey: string; + fields: CredentialField[]; + values: Record; + onChange: (values: Record) => void; + idPrefix: string; + labelSurface?: 'card' | 'background' | 'muted'; +}) { + const { t } = useTranslation(); + return ( +
+ {fields.map((field) => { + const id = `${idPrefix}-${field.key}`; + const label = t( + `settings.connectors.fields.${connectorKey}_${field.key}`, + { + defaultValue: t(`settings.connectors.fields.${field.key}`, { + defaultValue: field.label, + }), + }, + ); + const inputProps = { + id, + type: field.secret ? 'password' : 'text', + autoComplete: field.secret ? 'new-password' : 'off', + value: values[field.key] ?? '', + onChange: (e: ChangeEvent) => + onChange({ ...values, [field.key]: e.target.value }), + }; + if (!field.hint) + return ( + + ); + return ( + + + + ); + })} +
+ ); +} + +/** Whether every required field has a value. */ +export const credentialsComplete = ( + fields: CredentialField[], + values: Record, +) => fields.every((field) => !field.required || !!values[field.key]?.trim()); diff --git a/frontend/src/connectors/LinearPicker.test.tsx b/frontend/src/connectors/LinearPicker.test.tsx new file mode 100644 index 00000000..8171ae4c --- /dev/null +++ b/frontend/src/connectors/LinearPicker.test.tsx @@ -0,0 +1,187 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'count' in opts ? `${key}:${opts.count}` : key, + }), +})); + +const service = vi.hoisted(() => ({ linearWorkspace: vi.fn() })); +vi.mock('../api/services/connectorsService', () => ({ default: service })); + +import LinearPicker, { + EMPTY_LINEAR_SELECTION, + linearSourceName, + type LinearSelection, +} from './LinearPicker'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const WORKSPACE = { + success: true, + teams: [ + { id: 't1', key: 'ENG', name: 'Engineering' }, + { id: 't2', key: 'DES', name: 'Design' }, + ], + projects: [ + { id: 'p1', name: 'Launch', state: 'Started', teams: ['Engineering'] }, + ], +}; + +describe('LinearPicker', () => { + let root: Root; + let container: HTMLDivElement; + + beforeEach(() => { + service.linearWorkspace.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + value: LinearSelection = EMPTY_LINEAR_SELECTION, + onChange = vi.fn(), + ) => { + await act(async () => { + root.render( + , + ); + }); + return onChange; + }; + + const checkboxes = () => + Array.from( + container.querySelectorAll('[role="checkbox"]'), + ); + const switches = () => + Array.from( + container.querySelectorAll('[role="switch"]'), + ); + + it('lists the teams and projects to pick', async () => { + service.linearWorkspace.mockResolvedValue(WORKSPACE); + await render(); + expect(service.linearWorkspace).toHaveBeenCalledWith('conn-1', null); + expect(container.textContent).toContain('Engineering'); + expect(container.textContent).toContain('ENG'); + expect(container.textContent).toContain('Launch'); + expect(checkboxes()).toHaveLength(3); + }); + + it('picks a team with what the source needs to name it', async () => { + service.linearWorkspace.mockResolvedValue(WORKSPACE); + const onChange = await render(); + await act(async () => checkboxes()[0].click()); + expect(onChange).toHaveBeenCalledWith({ + ...EMPTY_LINEAR_SELECTION, + teams: [{ id: 't1', key: 'ENG', name: 'Engineering' }], + }); + }); + + it('unpicks a picked project', async () => { + service.linearWorkspace.mockResolvedValue(WORKSPACE); + const picked = { + ...EMPTY_LINEAR_SELECTION, + projects: [{ id: 'p1', name: 'Launch' }], + }; + const onChange = await render(picked); + const project = checkboxes()[2]; + expect(project.getAttribute('aria-checked')).toBe('true'); + await act(async () => project.click()); + expect(onChange).toHaveBeenCalledWith({ + ...picked, + projects: [], + }); + }); + + it('keeps comments on by default and offers documents once a project is picked', async () => { + service.linearWorkspace.mockResolvedValue(WORKSPACE); + const onChange = await render(); + const [comments, documents] = switches(); + expect(comments.getAttribute('aria-checked')).toBe('true'); + expect(documents.disabled).toBe(true); + await act(async () => comments.click()); + expect(onChange).toHaveBeenCalledWith({ + ...EMPTY_LINEAR_SELECTION, + includeComments: false, + }); + await render({ + ...EMPTY_LINEAR_SELECTION, + projects: [{ id: 'p1', name: 'Launch' }], + }); + expect(switches()[1].disabled).toBe(false); + }); + + it('filters by the search text', async () => { + service.linearWorkspace.mockResolvedValue({ + ...WORKSPACE, + teams: Array.from({ length: 9 }, (_, n) => ({ + id: `t${n}`, + key: `T${n}`, + name: n === 4 ? 'Platform' : `Team ${n}`, + })), + }); + await render(); + const input = container.querySelector('input')!; + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, 'platf'); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + expect(checkboxes()).toHaveLength(1); + expect(container.textContent).toContain('Platform'); + }); + + it('asks to reconnect when the sign-in stopped working', async () => { + service.linearWorkspace.mockResolvedValue({ + success: false, + code: 'reconnect', + }); + await render(); + expect(container.textContent).toContain( + 'settings.connectors.detail.expired', + ); + }); + + it('offers a retry when Linear did not answer', async () => { + service.linearWorkspace.mockResolvedValueOnce({ success: false }); + await render(); + expect(container.textContent).toContain( + 'settings.connectors.linear.loadFailed', + ); + service.linearWorkspace.mockResolvedValueOnce(WORKSPACE); + const retry = Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === 'retry', + )!; + await act(async () => retry.click()); + expect(checkboxes()).toHaveLength(3); + }); + + it('names a source after what it syncs', () => { + expect(linearSourceName(EMPTY_LINEAR_SELECTION)).toBe(''); + expect( + linearSourceName({ + ...EMPTY_LINEAR_SELECTION, + teams: [{ id: 't1', key: 'ENG', name: 'Engineering' }], + projects: [{ id: 'p1', name: 'Launch' }], + }), + ).toBe('Linear · Engineering, Launch'); + }); +}); diff --git a/frontend/src/connectors/LinearPicker.tsx b/frontend/src/connectors/LinearPicker.tsx new file mode 100644 index 00000000..9b2c0c7e --- /dev/null +++ b/frontend/src/connectors/LinearPicker.tsx @@ -0,0 +1,295 @@ +import { + useCallback, + useEffect, + useRef, + useState, + type ReactNode, +} from 'react'; +import { useTranslation } from 'react-i18next'; + +import connectorsService from '../api/services/connectorsService'; +import SearchInput from '../components/SearchInput'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { Checkbox } from '../components/ui/checkbox'; +import { EmptyState } from '../components/ui/empty-state'; +import { ListRow, ListRows } from '../components/ui/list-row'; +import { LoadingState } from '../components/ui/loading-state'; +import { SectionHeader } from '../components/ui/section-header'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import type { LinearProject, LinearTeam } from './types'; + +/** What a Linear source syncs: the picked teams' and projects' issues. */ +export type LinearSelection = { + teams: LinearTeam[]; + projects: Pick[]; + includeComments: boolean; + /** The picked projects' Linear documents. */ + includeDocuments: boolean; +}; + +export const EMPTY_LINEAR_SELECTION: LinearSelection = { + teams: [], + projects: [], + includeComments: true, + includeDocuments: false, +}; + +/** `Linear · Engineering, Launch`: a source's name from what it syncs. */ +export const linearSourceName = (selection: LinearSelection) => { + const names = [...selection.teams, ...selection.projects].map((p) => p.name); + return names.length ? `Linear · ${names.join(', ')}` : ''; +}; + +/** The wizard's choice as the setup endpoint takes it. */ +export const linearSyncItems = (selection: LinearSelection) => ({ + teams: selection.teams, + projects: selection.projects, + include_comments: selection.includeComments, + include_documents: selection.includeDocuments && !!selection.projects.length, +}); + +type LoadError = 'reconnect' | 'failed' | null; + +// More rows than this and a search field helps. +const SEARCH_FROM = 8; + +/** + * Pick the Linear teams and projects whose issues a source syncs, and + * whether it brings comments and the projects' documents. The lists come + * from Linear's MCP server, read with the connection's sign-in. + */ +export default function LinearPicker({ + connectionId, + token, + value, + onChange, +}: { + connectionId: string; + token: string | null; + value: LinearSelection; + onChange: (selection: LinearSelection) => void; +}) { + const { t } = useTranslation(); + const [teams, setTeams] = useState([]); + const [projects, setProjects] = useState([]); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + const [query, setQuery] = useState(''); + const requestRef = useRef(0); + + const load = useCallback(async () => { + const request = ++requestRef.current; + setLoading(true); + setError(null); + try { + const data = await connectorsService.linearWorkspace(connectionId, token); + if (request !== requestRef.current) return; + if (!data?.success) { + setError(data?.code === 'reconnect' ? 'reconnect' : 'failed'); + return; + } + setTeams(data.teams ?? []); + setProjects(data.projects ?? []); + } catch { + if (request === requestRef.current) setError('failed'); + } finally { + if (request === requestRef.current) setLoading(false); + } + }, [connectionId, token]); + + useEffect(() => { + load(); + }, [load]); + + // A few hundred rows at most, so filtering on each render is cheap. + const needle = query.trim().toLowerCase(); + const matches = (...texts: string[]) => + !needle || texts.some((text) => text.toLowerCase().includes(needle)); + const visibleTeams = teams.filter((team) => matches(team.name, team.key)); + const visibleProjects = projects.filter((project) => matches(project.name)); + + const pickedTeams = new Set(value.teams.map((team) => team.id)); + const pickedProjects = new Set(value.projects.map((project) => project.id)); + + const toggleTeam = (team: LinearTeam, on: boolean) => + onChange({ + ...value, + teams: on + ? [...value.teams, { id: team.id, key: team.key, name: team.name }] + : value.teams.filter((picked) => picked.id !== team.id), + }); + const toggleProject = (project: LinearProject, on: boolean) => + onChange({ + ...value, + projects: on + ? [...value.projects, { id: project.id, name: project.name }] + : value.projects.filter((picked) => picked.id !== project.id), + }); + + if (loading && teams.length + projects.length === 0) + return ; + if (error) { + return ( + + {t('retry')} + + ) : undefined + } + /> + ); + } + if (teams.length + projects.length === 0) { + return ( + + ); + } + + const row = ( + id: string, + checked: boolean, + title: string, + description: string, + onToggle: (on: boolean) => void, + ) => ( + onToggle(state === true)} + /> + } + title={title} + description={description || undefined} + > + + ); + + const list = (children: ReactNode) => ( + + {children} + + ); + + return ( +
+ {teams.length + projects.length > SEARCH_FROM && ( + setQuery(e.target.value)} + /> + )} + {visibleTeams.length + visibleProjects.length === 0 ? ( + + ) : ( + <> + {visibleTeams.length > 0 && ( +
+ + {list( + visibleTeams.map((team) => + row( + team.id, + pickedTeams.has(team.id), + team.name, + team.key, + (on) => toggleTeam(team, on), + ), + ), + )} +
+ )} + {visibleProjects.length > 0 && ( +
+ + {list( + visibleProjects.map((project) => + row( + project.id, + pickedProjects.has(project.id), + project.name, + [project.state, project.teams.join(', ')] + .filter(Boolean) + .join(' · '), + (on) => toggleProject(project, on), + ), + ), + )} +
+ )} + + )} + + + + onChange({ ...value, includeComments: checked === true }) + } + /> + + + 0} + disabled={value.projects.length === 0} + onCheckedChange={(checked) => + onChange({ ...value, includeDocuments: checked === true }) + } + /> + + +
+ ); +} diff --git a/frontend/src/connectors/RepoPicker.test.tsx b/frontend/src/connectors/RepoPicker.test.tsx new file mode 100644 index 00000000..39f9d33c --- /dev/null +++ b/frontend/src/connectors/RepoPicker.test.tsx @@ -0,0 +1,164 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'count' in opts ? `${key}:${opts.count}` : key, + }), +})); + +const service = vi.hoisted(() => ({ repositories: vi.fn() })); +vi.mock('../api/services/connectorsService', () => ({ default: service })); + +const install = vi.hoisted(() => ({ + options: null as null | { install?: boolean; onSuccess: () => void }, + start: vi.fn(), +})); +vi.mock('../components/ConnectorAuth', () => ({ + useConnectorAuth: (options: { install?: boolean; onSuccess: () => void }) => { + install.options = options; + return install.start; + }, +})); + +import RepoPicker from './RepoPicker'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const REPOS = [ + { + full_name: 'octocat/private', + private: true, + description: 'Secret stuff', + default_branch: 'main', + updated_at: '2026-09-01T00:00:00Z', + html_url: 'https://github.com/octocat/private', + }, + { + full_name: 'octocat/hello-world', + private: false, + description: '', + default_branch: 'main', + updated_at: '2026-08-01T00:00:00Z', + html_url: 'https://github.com/octocat/hello-world', + }, +]; + +describe('RepoPicker', () => { + let root: Root; + let container: HTMLDivElement; + + beforeEach(() => { + service.repositories.mockReset(); + install.start.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (value: string | null, onChange = vi.fn()) => { + await act(async () => { + root.render( + , + ); + }); + return onChange; + }; + + const radios = () => + Array.from(container.querySelectorAll('[role="radio"]')); + + it('lists the repositories and picks one', async () => { + service.repositories.mockResolvedValue({ + success: true, + repositories: REPOS, + install_url: null, + }); + const onChange = await render(null); + expect(service.repositories).toHaveBeenCalledWith('conn-1', null); + expect(radios().map((r) => r.textContent)).toEqual([ + expect.stringContaining('octocat/private'), + expect.stringContaining('octocat/hello-world'), + ]); + await act(async () => radios()[1].click()); + expect(onChange).toHaveBeenCalledWith('octocat/hello-world'); + }); + + it('marks the picked repository', async () => { + service.repositories.mockResolvedValue({ + success: true, + repositories: REPOS, + install_url: null, + }); + await render('octocat/private'); + expect(radios()[0].getAttribute('aria-checked')).toBe('true'); + expect(radios()[1].getAttribute('aria-checked')).toBe('false'); + }); + + it('filters by the search text', async () => { + service.repositories.mockResolvedValue({ + success: true, + repositories: REPOS, + install_url: null, + }); + await render(null); + const input = container.querySelector('input')!; + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, 'hello'); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + expect(radios()).toHaveLength(1); + expect(radios()[0].textContent).toContain('octocat/hello-world'); + }); + + it('sends a GitHub App sign-in to choose repositories, then reloads', async () => { + service.repositories.mockResolvedValueOnce({ + success: true, + repositories: [], + install_url: 'https://github.com/apps/docsgpt/installations/new', + }); + await render(null); + expect(container.textContent).toContain( + 'settings.connectors.github.noAppRepositories', + ); + const choose = Array.from(container.querySelectorAll('button')).find((b) => + b.textContent?.includes('settings.connectors.github.chooseRepositories'), + )!; + await act(async () => choose.click()); + expect(install.start).toHaveBeenCalled(); + expect(install.options?.install).toBe(true); + service.repositories.mockResolvedValueOnce({ + success: true, + repositories: REPOS, + install_url: 'https://github.com/apps/docsgpt/installations/new', + }); + await act(async () => install.options!.onSuccess()); + expect(radios()).toHaveLength(2); + }); + + it('asks to reconnect when the token stopped working', async () => { + service.repositories.mockResolvedValue({ + success: false, + code: 'reconnect', + }); + await render(null); + expect(container.textContent).toContain( + 'settings.connectors.detail.expired', + ); + }); +}); diff --git a/frontend/src/connectors/RepoPicker.tsx b/frontend/src/connectors/RepoPicker.tsx new file mode 100644 index 00000000..5f290437 --- /dev/null +++ b/frontend/src/connectors/RepoPicker.tsx @@ -0,0 +1,216 @@ +import { BookMarked, Check, ExternalLink, Lock } from 'lucide-react'; +import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { useTranslation } from 'react-i18next'; + +import connectorsService from '../api/services/connectorsService'; +import { useConnectorAuth } from '../components/ConnectorAuth'; +import SearchInput from '../components/SearchInput'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { EmptyState } from '../components/ui/empty-state'; +import { ListRow, ListRows } from '../components/ui/list-row'; +import { LoadingState } from '../components/ui/loading-state'; +import { formatDateTime } from '../utils/dateTimeUtils'; +import type { GitHubRepository } from './types'; + +type LoadError = 'reconnect' | 'failed' | null; + +/** + * Pick one repository a GitHub connection can read. A token lists what it + * was granted; a GitHub App sign-in lists its installations' repositories, + * and more are chosen on GitHub (the app's installation page), after which + * the list reloads. + */ +export default function RepoPicker({ + connectionId, + token, + value, + onChange, +}: { + connectionId: string; + token: string | null; + /** The picked repository's `owner/name`. */ + value: string | null; + onChange: (fullName: string) => void; +}) { + const { t } = useTranslation(); + const [repositories, setRepositories] = useState([]); + const [installUrl, setInstallUrl] = useState(null); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + const [query, setQuery] = useState(''); + // Only the latest load may land: a reload after choosing repositories + // must not be overwritten by a slower earlier one. + const requestRef = useRef(0); + + const load = useCallback(async () => { + const request = ++requestRef.current; + setLoading(true); + setError(null); + try { + const data = await connectorsService.repositories(connectionId, token); + if (request !== requestRef.current) return; + if (!data?.success) { + setError(data?.code === 'reconnect' ? 'reconnect' : 'failed'); + return; + } + setRepositories(data.repositories ?? []); + setInstallUrl(data.install_url ?? null); + } catch { + if (request === requestRef.current) setError('failed'); + } finally { + if (request === requestRef.current) setLoading(false); + } + }, [connectionId, token]); + + useEffect(() => { + load(); + }, [load]); + + // The popup reports success when the app asks for authorization during + // installation; otherwise the user closes it. Reload either way. + const chooseRepositories = useConnectorAuth({ + provider: 'github', + connectionId, + install: true, + onSuccess: () => load(), + onError: () => load(), + }); + + const visible = useMemo(() => { + const needle = query.trim().toLowerCase(); + if (!needle) return repositories; + return repositories.filter( + (repo) => + repo.full_name.toLowerCase().includes(needle) || + repo.description.toLowerCase().includes(needle), + ); + }, [repositories, query]); + + const chooseButton = installUrl ? ( + + ) : null; + + if (loading && repositories.length === 0) + return ; + if (error) { + return ( + + {t('retry')} + + ) : undefined + } + /> + ); + } + if (repositories.length === 0) { + return ( + + ); + } + + return ( +
+ setQuery(e.target.value)} + /> + {visible.length === 0 ? ( + + ) : ( + +
+ + {visible.map((repo) => { + const picked = repo.full_name === value; + return ( + + {repo.private ? ( + + ) : ( + + )} + + } + title={repo.full_name} + description={ + repo.description || + (repo.updated_at + ? t('settings.connectors.github.updated', { + date: formatDateTime(repo.updated_at), + interpolation: { escapeValue: false }, + }) + : undefined) + } + trailing={ + picked ? ( + + ) : undefined + } + > +
+ + )} + {chooseButton} +
+ ); +} diff --git a/frontend/src/connectors/SignInAgainNotice.test.tsx b/frontend/src/connectors/SignInAgainNotice.test.tsx new file mode 100644 index 00000000..3053f664 --- /dev/null +++ b/frontend/src/connectors/SignInAgainNotice.test.tsx @@ -0,0 +1,155 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter, Route, Routes, useLocation } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts?.name ? `${key}:${opts.name}` : key, + }), +})); + +const launch = vi.hoisted(() => vi.fn()); +vi.mock('./useConnectorLauncher', () => ({ + default: () => ({ launch, modals: null }), +})); + +import connectorsReducer from './connectorsSlice'; +import SignInAgainNotice, { useSignInAgain } from './SignInAgainNotice'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const LINEAR = { + key: 'mcp:linear', + name: 'Linear', + auth_kind: 'mcp_oauth', + mcp_url: 'https://mcp.linear.app/mcp', + available: true, +}; +const CUSTOM = { key: 'custom_mcp', name: 'MCP server', auth_kind: 'mcp' }; + +function Where() { + return ( +
+ {useLocation().pathname + useLocation().search} +
+ ); +} + +function Harness({ + connectorKey, + mcpToolId, +}: { + connectorKey: string; + mcpToolId?: string; +}) { + const { reconnect, modals } = useSignInAgain(); + return ( + <> + reconnect(connection, mcpToolId)} + /> + {modals} + + ); +} + +describe('SignInAgainNotice', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + launch.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (connectorKey: string, mcpToolId?: string) => { + const store = configureStore({ + reducer: { connectors: connectorsReducer }, + preloadedState: { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [LINEAR, CUSTOM], + connections: [], + }, + }, + } as Parameters[0]); + await act(async () => { + root.render( + + + + + } + /> + } /> + + + , + ); + }); + }; + + const reconnectButton = () => + Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === 'settings.connectors.status.reconnect', + )!; + + it('names the connection that needs signing in again', async () => { + await render('mcp:linear'); + expect(container.textContent).toContain( + 'settings.connectors.health.pickerNotice:Linear', + ); + }); + + it('reconnects a service in place, keeping its MCP tool', async () => { + await render('mcp:linear', 'tool-lin'); + await act(async () => reconnectButton().click()); + expect(launch).toHaveBeenCalledWith(LINEAR, { + mode: 'reconnect', + connectionId: 'conn-2', + mcpServer: { id: 'tool-lin' }, + }); + }); + + it('opens the connector page when the sign-in needs its form', async () => { + await render('custom_mcp'); + await act(async () => reconnectButton().click()); + expect(launch).not.toHaveBeenCalled(); + expect(container.querySelector('[data-testid="where"]')?.textContent).toBe( + '/settings/connectors?connector=custom_mcp', + ); + }); + + it('renders nothing when every connection works', async () => { + const store = configureStore({ + reducer: { connectors: connectorsReducer }, + }); + await act(async () => { + root.render( + + + , + ); + }); + expect(container.innerHTML).toBe(''); + }); +}); diff --git a/frontend/src/connectors/SignInAgainNotice.tsx b/frontend/src/connectors/SignInAgainNotice.tsx new file mode 100644 index 00000000..5f0fea7a --- /dev/null +++ b/frontend/src/connectors/SignInAgainNotice.tsx @@ -0,0 +1,89 @@ +import { TriangleAlert } from 'lucide-react'; +import { useCallback } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useSelector } from 'react-redux'; +import { useNavigate } from 'react-router-dom'; + +import { Alert, AlertDescription } from '../components/ui/alert'; +import { Button } from '../components/ui/button'; +import { selectConnectorCatalog } from './connectorsSlice'; +import { reconnectsInPlace } from './launchRules'; +import useConnectorLauncher from './useConnectorLauncher'; + +type ConnectionRef = { id: string; connector_key: string; name: string }; + +/** + * Reconnects a connection from wherever it shows up broken: in place when the + * wizard can redo its sign-in, otherwise on its connector page. Render + * `modals` somewhere that stays mounted (not inside a closing popover). + */ +export function useSignInAgain({ + onConnected, +}: { onConnected?: () => void } = {}) { + const catalog = useSelector(selectConnectorCatalog); + const navigate = useNavigate(); + const { launch, modals } = useConnectorLauncher({ onConnected }); + + const reconnect = useCallback( + (connection: Omit, mcpToolId?: string) => { + const connector = catalog.find((c) => c.key === connection.connector_key); + if (connector && reconnectsInPlace(connector)) { + launch(connector, { + mode: 'reconnect', + connectionId: connection.id, + // An MCP preset re-signs its existing tool rather than adding one. + mcpServer: mcpToolId ? { id: mcpToolId } : undefined, + }); + return; + } + navigate( + `/settings/connectors?connector=${encodeURIComponent(connection.connector_key)}`, + ); + }, + [catalog, launch, navigate], + ); + + return { reconnect, modals }; +} + +/** + * One line per connection that needs signing in again, each with Reconnect. + * Sits in the tool pickers so a broken service is visible where it is used. + */ +export default function SignInAgainNotice({ + connections, + onReconnect, +}: { + connections: ConnectionRef[]; + onReconnect: (connection: ConnectionRef) => void; +}) { + const { t } = useTranslation(); + if (connections.length === 0) return null; + return ( +
+ {connections.map((connection) => ( + + + + + {t('settings.connectors.health.pickerNotice', { + name: connection.name, + interpolation: { escapeValue: false }, + })} + + + + + ))} +
+ ); +} diff --git a/frontend/src/connectors/ToolPermissions.test.tsx b/frontend/src/connectors/ToolPermissions.test.tsx new file mode 100644 index 00000000..f7b85471 --- /dev/null +++ b/frontend/src/connectors/ToolPermissions.test.tsx @@ -0,0 +1,272 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts?.count !== undefined ? `${key}:${opts.count}` : key, + }), +})); + +const connectors = vi.hoisted(() => ({ + setToolPermissions: vi.fn(), + setToolParameters: vi.fn(), +})); +vi.mock('../api/services/connectorsService', () => ({ default: connectors })); + +import actionToastReducer from '../notifications/actionToastSlice'; +import ToolPermissions from './ToolPermissions'; +import type { ActionParameter, ConnectionTool } from './types'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const action = ( + name: string, + access: 'read' | 'write', + permission: 'always' | 'ask' | 'off', + parameters: ActionParameter[] = [], +) => ({ name, access, permission, description: `Does ${name}.`, parameters }); + +const param = ( + name: string, + fixed: boolean, + value: ActionParameter['value'] = null, +): ActionParameter => ({ + name, + description: `The ${name}.`, + type: 'string', + required: false, + fixed, + value, +}); + +const tool = (actions: ReturnType[]): ConnectionTool => + ({ + id: 'tool-1', + name: 'mcp_tool', + display_name: 'Linear', + status: true, + credential_mode: 'owner', + actions, + }) as ConnectionTool; + +const MANY_READS = Array.from({ length: 8 }, (_, i) => + action(`get_issue_${i}`, 'read', 'always'), +); + +describe('ToolPermissions', () => { + let container: HTMLDivElement; + let root: Root; + const store = () => + configureStore({ + reducer: { + actionToast: actionToastReducer, + preference: (state = { token: null }) => state, + }, + }); + + beforeEach(() => { + connectors.setToolPermissions.mockReset(); + connectors.setToolPermissions.mockImplementation(async () => ({ + success: true, + tool: tool([]), + })); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (value: ConnectionTool) => { + const s = store(); + await act(async () => { + root.render( + + + , + ); + }); + }; + + const group = (access: 'read' | 'write') => + container.querySelector(`[data-access="${access}"]`)!; + const choice = (access: 'read' | 'write', permission: string) => + group(access).querySelector( + `[data-permission="${permission}"]`, + )!; + + it('sets every read at once', async () => { + await render(tool([...MANY_READS, action('create_issue', 'write', 'ask')])); + await act(async () => choice('read', 'ask').click()); + expect(connectors.setToolPermissions).toHaveBeenCalledWith( + 'conn-1', + 'tool-1', + Object.fromEntries(MANY_READS.map((a) => [a.name, 'ask'])), + null, + ); + expect(choice('read', 'ask').getAttribute('data-state')).toBe('on'); + // Writes are untouched. + expect(choice('write', 'ask').getAttribute('data-state')).toBe('on'); + }); + + it('shows no group choice when its actions differ', async () => { + await render( + tool([action('a', 'write', 'ask'), action('b', 'write', 'always')]), + ); + for (const permission of ['always', 'ask', 'off']) + expect(choice('write', permission).getAttribute('data-state')).toBe( + 'off', + ); + }); + + it('keeps a long list folded, then lists actions in words with what they do', async () => { + await render(tool(MANY_READS)); + expect(container.textContent).not.toContain('Get issue 0'); + const customize = Array.from(container.querySelectorAll('button')).find( + (b) => + b.textContent?.startsWith('settings.connectors.permission.customize'), + )!; + await act(async () => customize.click()); + expect(container.textContent).toContain('Get issue 0'); + expect(container.textContent).toContain('Does get_issue_0.'); + }); + + it('follows the tool when its actions change', async () => { + await render(tool([action('old_one', 'read', 'always')])); + await render(tool([action('new_one', 'read', 'always')])); + expect(container.textContent).toContain('New one'); + expect(container.textContent).not.toContain('Old one'); + }); + + const button = (text: string) => + Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === text, + )!; + const typeInto = (input: HTMLInputElement, value: string) => { + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + setter.call(input, value); + input.dispatchEvent(new Event('input', { bubbles: true })); + }; + + it('marks an action that has fixed values', async () => { + await render( + tool([ + action('telegram_send_message', 'write', 'ask', [ + param('text', false), + param('chat_id', true, '-100'), + ]), + ]), + ); + expect(container.textContent).toContain( + 'settings.connectors.parameters.fixedCount:1', + ); + }); + + it('fixes a parameter to a value the AI cannot change', async () => { + const send = action('telegram_send_message', 'write', 'ask', [ + param('text', false), + param('chat_id', false), + ]); + connectors.setToolParameters.mockResolvedValue({ + success: true, + tool: tool([ + { + ...send, + parameters: [param('text', false), param('chat_id', true, '-100')], + }, + ]), + }); + await render(tool([send])); + await act(async () => + button('settings.connectors.parameters.show').click(), + ); + const row = container.querySelector( + '[data-parameter="chat_id"]', + )!; + await act(async () => + row.querySelector('[data-mode="fixed"]')!.click(), + ); + const input = row.querySelector('input')!; + await act(async () => typeInto(input, '-100')); + await act(async () => + button('settings.connectors.parameters.save').click(), + ); + expect(connectors.setToolParameters).toHaveBeenCalledWith( + 'conn-1', + 'tool-1', + 'telegram_send_message', + { chat_id: '-100' }, + null, + ); + expect(container.textContent).toContain( + 'settings.connectors.parameters.fixedCount:1', + ); + }); + + it('lets the AI decide a fixed parameter again', async () => { + connectors.setToolParameters.mockResolvedValue({ + success: true, + tool: tool([]), + }); + await render( + tool([ + action('telegram_send_message', 'write', 'ask', [ + param('chat_id', true, '-100'), + ]), + ]), + ); + await act(async () => + button('settings.connectors.parameters.show').click(), + ); + const row = container.querySelector( + '[data-parameter="chat_id"]', + )!; + expect(row.querySelector('input')!.value).toBe('-100'); + await act(async () => + row.querySelector('[data-mode="ai"]')!.click(), + ); + expect(connectors.setToolParameters).toHaveBeenCalledWith( + 'conn-1', + 'tool-1', + 'telegram_send_message', + { chat_id: null }, + null, + ); + }); + + it('shows a chat the account sets as set there, not as a choice', async () => { + await render( + tool([ + action('telegram_send_message', 'write', 'ask', [ + { ...param('chat_id', true, '-100'), set_by: 'account' }, + ]), + ]), + ); + await act(async () => + button('settings.connectors.parameters.show').click(), + ); + const row = container.querySelector( + '[data-parameter="chat_id"]', + )!; + expect(row.textContent).toContain( + 'settings.connectors.parameters.fromAccount', + ); + expect(row.querySelector('[data-mode]')).toBeNull(); + }); + + it('says what Ask first means outside chat', async () => { + await render(tool([action('create_issue', 'write', 'ask')])); + expect(container.textContent).toContain( + 'settings.connectors.permission.hint', + ); + }); +}); diff --git a/frontend/src/connectors/ToolPermissions.tsx b/frontend/src/connectors/ToolPermissions.tsx new file mode 100644 index 00000000..84242ff2 --- /dev/null +++ b/frontend/src/connectors/ToolPermissions.tsx @@ -0,0 +1,325 @@ +import { useEffect, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; + +import connectorsService from '../api/services/connectorsService'; +import { Badge } from '../components/ui/badge'; +import { Button } from '../components/ui/button'; +import { Card } from '../components/ui/card'; +import { SectionHeader } from '../components/ui/section-header'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '../components/ui/select'; +import { ToggleGroup, ToggleGroupItem } from '../components/ui/toggle-group'; +import { showActionToast } from '../notifications/actionToastSlice'; +import { selectToken } from '../preferences/preferenceSlice'; +import { formatCount } from '../utils/dateTimeUtils'; +import ActionParameters, { type ParameterValue } from './ActionParameters'; +import { actionTitle } from './i18n'; +import type { + ActionPermission, + ConnectionTool, + ConnectionToolAction, +} from './types'; + +const PERMISSIONS: ActionPermission[] = ['always', 'ask', 'off']; + +/** Longer groups start folded to their one group choice. */ +const FOLD_AFTER = 5; + +/** + * One action under Customize: its name, what it does and its permission, + * with its parameters one click away and a count of the values fixed there. + */ +function ActionRow({ + action, + readOnly, + onPermission, + onParameters, +}: { + action: ConnectionToolAction; + readOnly: boolean; + onPermission: (permission: ActionPermission) => void; + onParameters: (changes: Record) => Promise; +}) { + const { t } = useTranslation(); + const [showParameters, setShowParameters] = useState(false); + const parameters = action.parameters ?? []; + const fixedCount = parameters.filter((parameter) => parameter.fixed).length; + return ( +
  • +
    +
    +
    +

    + {actionTitle(action.name)} +

    + {fixedCount > 0 && ( + + {t('settings.connectors.parameters.fixedCount', { + count: fixedCount, + formatted: formatCount(fixedCount), + })} + + )} +
    + {action.description && ( +

    + {action.description} +

    + )} + {parameters.length > 0 && ( + + )} +
    + +
    + {showParameters && ( + + )} +
    • + ); +} + +/** + * One tool's actions in two groups, what it looks up and what it does, each + * set at once to Allow / Ask first / Off; single actions can differ under + * Customize, where each action also opens its parameters to fix values the + * AI must always use. Changes save immediately; a failed save puts the + * previous choices back and says so. + */ +export default function ToolPermissions({ + connectionId, + tool, + onChange, + readOnly = false, + variant = 'outline', +}: { + connectionId: string; + tool: ConnectionTool; + onChange?: (tool: ConnectionTool) => void; + readOnly?: boolean; + /** The panel surface: `outline` in a modal, `subtle` in the drawer. */ + variant?: 'outline' | 'subtle'; +}) { + const { t } = useTranslation(); + const dispatch = useDispatch(); + const token = useSelector(selectToken); + const [actions, setActions] = useState(tool.actions); + const [unfolded, setUnfolded] = useState>({}); + // A refreshed tool brings its new list of actions. + useEffect(() => setActions(tool.actions), [tool.actions]); + + const setPermissions = (changes: Record) => { + const previous = actions; + setActions( + actions.map((action) => + changes[action.name] + ? { ...action, permission: changes[action.name] } + : action, + ), + ); + connectorsService + .setToolPermissions(connectionId, tool.id, changes, token) + .then((data) => { + if (!data?.success) throw new Error('save failed'); + onChange?.(data.tool); + }) + .catch(() => { + setActions(previous); + dispatch( + showActionToast({ + variant: 'destructive', + message: t('settings.connectors.permission.saveFailed'), + }), + ); + }); + }; + + const setParameters = async ( + actionName: string, + changes: Record, + ) => { + try { + const data = await connectorsService.setToolParameters( + connectionId, + tool.id, + actionName, + changes, + token, + ); + if (!data?.success) throw new Error('save failed'); + setActions(data.tool.actions); + onChange?.(data.tool); + return true; + } catch { + dispatch( + showActionToast({ + variant: 'destructive', + message: t('settings.connectors.parameters.saveFailed'), + }), + ); + return false; + } + }; + + const groups = (['read', 'write'] as const) + .map((access) => ({ + access, + actions: actions.filter((action) => action.access === access), + })) + .filter((group) => group.actions.length > 0); + + return ( + + {groups.map((group) => { + const title = t(`settings.connectors.capabilityPlain.${group.access}`); + const first = group.actions[0].permission; + // One choice for the group when its actions agree; none when mixed. + const common = group.actions.every( + (action) => action.permission === first, + ) + ? first + : ''; + const foldable = group.actions.length > FOLD_AFTER; + const open = !foldable || unfolded[group.access]; + return ( +
    +
    + +
    + + value && + setPermissions( + Object.fromEntries( + group.actions.map((action) => [ + action.name, + value as ActionPermission, + ]), + ), + ) + } + > + {PERMISSIONS.map((permission) => ( + + {t(`settings.connectors.permission.${permission}`)} + + ))} + +
    +
    + {foldable && ( + + )} + {open && ( +
      + {group.actions.map((action) => ( + + setPermissions({ [action.name]: permission }) + } + onParameters={(changes) => + setParameters(action.name, changes) + } + /> + ))} +
    + )} +
    + ); + })} +

    + {t('settings.connectors.permission.hint')} +

    +     + ); +} diff --git a/frontend/src/connectors/accounts.ts b/frontend/src/connectors/accounts.ts new file mode 100644 index 00000000..eebae4fc --- /dev/null +++ b/frontend/src/connectors/accounts.ts @@ -0,0 +1,2 @@ +/** The longest account name the server keeps (`ACCOUNT_NAME_MAX`). */ +export const ACCOUNT_NAME_MAX = 80; diff --git a/frontend/src/connectors/catalogCards.ts b/frontend/src/connectors/catalogCards.ts new file mode 100644 index 00000000..c1694968 --- /dev/null +++ b/frontend/src/connectors/catalogCards.ts @@ -0,0 +1,89 @@ +import type { ConnectorDefinition } from './types'; + +/** + * Connected and needing attention first, then what can be connected, then + * what an admin still has to set up. + */ +export const STATE_ORDER: Record = { + reconnect: 0, + connected: 1, + available: 2, + custom: 3, + needs_setup: 4, + disabled: 5, +}; + +/** Sorts catalog cards by `STATE_ORDER`, keeping catalog order within one. */ +export const byState = (a: ConnectorDefinition, b: ConnectorDefinition) => + STATE_ORDER[a.state] - STATE_ORDER[b.state]; + +/** The parts listed under a parent card (`part_of` names the parent). */ +export const partsOf = (catalog: ConnectorDefinition[], key: string) => + catalog.filter((c) => c.part_of === key); + +/** + * One service offered two ways (Confluence sync and the Jira & Confluence + * MCP actions) is one card. A part is shown on its own only when its + * parent is not listed (not set up, or turned off). + */ +export const isShownUnderParent = ( + catalog: ConnectorDefinition[], + connector: ConnectorDefinition, +) => !!connector.part_of && catalog.some((c) => c.key === connector.part_of); + +/** + * A parent card with its parts folded in: every capability of either, the + * accounts of both, and the state that needs the reader most. + */ +export const mergeParts = ( + catalog: ConnectorDefinition[], + connector: ConnectorDefinition, +): ConnectorDefinition => { + const parts = partsOf(catalog, connector.key); + if (parts.length === 0) return connector; + const all = [connector, ...parts]; + const state = all.some((c) => c.state === 'reconnect') + ? 'reconnect' + : all.some((c) => c.state === 'connected') + ? 'connected' + : connector.state; + return { + ...connector, + capabilities: Array.from(new Set(all.flatMap((c) => c.capabilities))), + connection_count: all.reduce((n, c) => n + c.connection_count, 0), + connected_count: all.reduce((n, c) => n + c.connected_count, 0), + state, + }; +}; + +/** The catalog as the Connectors page lists it: one card per service. */ +export const catalogCards = (catalog: ConnectorDefinition[]) => + catalog + .filter((connector) => !isShownUnderParent(catalog, connector)) + .map((connector) => mergeParts(catalog, connector)); + +/** + * The services a user can connect to sync into Knowledge, in the + * Connectors page's order: each card with the catalog entry that syncs + * (the card itself, or its sync part), and only where that can be + * connected now. + */ +export const syncTargets = (catalog: ConnectorDefinition[]) => + catalogCards(catalog) + .filter((card) => card.capabilities?.includes('sync')) + .map((card) => ({ + card, + // The card's own entry, not the merged card, which has its parts' + // capabilities too. + target: [ + ...catalog.filter((c) => c.key === card.key), + ...partsOf(catalog, card.key), + ].find((c) => c.available && c.capabilities?.includes('sync')), + })) + .filter( + ( + entry, + ): entry is { card: ConnectorDefinition; target: ConnectorDefinition } => + !!entry.target, + ) + .sort((a, b) => byState(a.card, b.card)); diff --git a/frontend/src/connectors/connectorsSlice.test.ts b/frontend/src/connectors/connectorsSlice.test.ts new file mode 100644 index 00000000..6ca27c5d --- /dev/null +++ b/frontend/src/connectors/connectorsSlice.test.ts @@ -0,0 +1,59 @@ +import reducer, { + loadConnectors, + type ConnectorsState, +} from './connectorsSlice'; +import type { Connection } from './types'; + +const arg = { token: null }; +const connection = (id: string) => ({ id }) as unknown as Connection; +const payload = (ids: string[]) => ({ + catalog: [], + connections: ids.map(connection), +}); + +const pending = (state: ConnectorsState | undefined, requestId: string) => + reducer(state, loadConnectors.pending(requestId, arg)); + +describe('connectorsSlice loadConnectors', () => { + it('keeps the newer response when an older one arrives last', () => { + let state = pending(undefined, 'old'); + state = pending(state, 'new'); + state = reducer( + state, + loadConnectors.fulfilled(payload(['after-connect']), 'new', arg), + ); + state = reducer( + state, + loadConnectors.fulfilled(payload(['before-connect']), 'old', arg), + ); + expect(state.connections.map((c) => c.id)).toEqual(['after-connect']); + expect(state.loading).toBe(false); + }); + + it('stays loading while the latest request is still running', () => { + let state = pending(undefined, 'old'); + state = pending(state, 'new'); + state = reducer( + state, + loadConnectors.fulfilled(payload(['stale']), 'old', arg), + ); + expect(state.loading).toBe(true); + expect(state.connections).toEqual([]); + }); + + it('ignores the failure of an older request', () => { + let state = pending(undefined, 'old'); + state = pending(state, 'new'); + state = reducer( + state, + loadConnectors.rejected(new Error('boom'), 'old', arg), + ); + expect(state.failed).toBe(false); + state = reducer( + state, + loadConnectors.rejected(new Error('boom'), 'new', arg), + ); + expect(state.failed).toBe(true); + expect(state.loading).toBe(false); + }); +}); diff --git a/frontend/src/connectors/connectorsSlice.ts b/frontend/src/connectors/connectorsSlice.ts new file mode 100644 index 00000000..08547d34 --- /dev/null +++ b/frontend/src/connectors/connectorsSlice.ts @@ -0,0 +1,117 @@ +import { + createAsyncThunk, + createSlice, + type PayloadAction, +} from '@reduxjs/toolkit'; + +import connectorsService from '../api/services/connectorsService'; +import type { Connection, ConnectorDefinition } from './types'; + +export type ConnectorsState = { + /** False when the backend predates connectors (`/api/config`). */ + enabled: boolean; + catalog: ConnectorDefinition[]; + connections: Connection[]; + loading: boolean; + loaded: boolean; + failed: boolean; + /** + * The `loadConnectors` call started last. Loads overlap (several screens + * refresh at once), and only its answer is applied: an older one could + * hold connections from before a connect. + */ + latestRequestId?: string | null; +}; + +const initialState: ConnectorsState = { + enabled: true, + catalog: [], + connections: [], + loading: false, + loaded: false, + failed: false, + latestRequestId: null, +}; + +/** + * Load the catalog and the caller's connections together. Every screen that + * shows a connector's state (the Connectors page, Add Source, Add Tool, the + * composer pickers) reads this one copy, so a connect anywhere updates all. + */ +export const loadConnectors = createAsyncThunk< + { catalog: ConnectorDefinition[]; connections: Connection[] }, + { token: string | null } +>('connectors/load', async ({ token }) => { + const [catalog, connections] = await Promise.all([ + connectorsService.getCatalog(token), + connectorsService.listConnections(token), + ]); + if (!catalog?.success || !connections?.success) { + throw new Error('Failed to load connectors'); + } + return { + catalog: catalog.connectors ?? [], + connections: connections.connections ?? [], + }; +}); + +const connectorsSlice = createSlice({ + name: 'connectors', + initialState, + reducers: { + setConnectorsEnabled(state, action: PayloadAction) { + state.enabled = action.payload; + }, + }, + extraReducers: (builder) => { + builder + .addCase(loadConnectors.pending, (state, action) => { + state.latestRequestId = action.meta.requestId; + state.loading = true; + state.failed = false; + }) + .addCase(loadConnectors.fulfilled, (state, action) => { + if (action.meta.requestId !== state.latestRequestId) return; + state.latestRequestId = null; + state.loading = false; + state.loaded = true; + state.catalog = action.payload.catalog; + state.connections = action.payload.connections; + }) + .addCase(loadConnectors.rejected, (state, action) => { + if (action.meta.requestId !== state.latestRequestId) return; + state.latestRequestId = null; + state.loading = false; + state.failed = true; + }); + }, +}); + +export const { setConnectorsEnabled } = connectorsSlice.actions; + +type RootLike = { connectors: ConnectorsState }; + +export const selectConnectorsEnabled = (state: RootLike) => + state.connectors?.enabled !== false; + +export const selectConnectorCatalog = (state: RootLike) => + state.connectors.catalog; +export const selectConnections = (state: RootLike) => + state.connectors.connections; +/** Whether a connection needs the user to sign in again (expired or failing). */ +export const connectionNeedsSignIn = ( + connection: { status: string } | null | undefined, +) => + connection?.status === 'reconnect_needed' || connection?.status === 'error'; + +/** A connection that needs the user (signing in again, or failing). */ +export const selectConnectionsNeedAttention = (state: RootLike) => + (state.connectors?.connections ?? []).some(connectionNeedsSignIn); +export const selectConnectorsLoading = (state: RootLike) => + state.connectors.loading; +export const selectConnectorsLoaded = (state: RootLike) => + state.connectors.loaded; +export const selectConnectorsFailed = (state: RootLike) => + state.connectors.failed; + +export default connectorsSlice.reducer; diff --git a/frontend/src/connectors/i18n.ts b/frontend/src/connectors/i18n.ts new file mode 100644 index 00000000..5ca4aed4 --- /dev/null +++ b/frontend/src/connectors/i18n.ts @@ -0,0 +1,68 @@ +import type { TFunction } from 'i18next'; + +import type { ConnectorDefinition } from './types'; + +/** + * Locale key segment for a catalog key. Preset keys look like `mcp:notion`, + * and i18next reads `:` as a namespace separator, so it becomes `_`. + */ +export const connectorLocaleKey = (key: string) => key.replace(/[:.]/g, '_'); + +/** The card's one-line description, translated when the locale has it. */ +export const connectorDescription = ( + t: TFunction, + connector: Pick, +) => + t(`settings.connectors.descriptions.${connectorLocaleKey(connector.key)}`, { + defaultValue: connector.description, + }); + +/** + * Whether an API-key connection's account label is a hint of the key + * (`…abcd`) rather than an account name (GitHub names a token connection + * after its login). + */ +export const isKeyHint = (label: string | null | undefined) => + !!label && label.startsWith('…'); + +/** An action's name in words: `get_triage_responsibility` → "Get triage responsibility". */ +export const actionTitle = (name: string) => { + const words = name.replace(/[_-]+/g, ' ').trim(); + return words.charAt(0).toUpperCase() + words.slice(1); +}; + +/** The connector's name; only the two custom kinds have translated names. */ +export const connectorName = ( + t: TFunction, + connector: Pick, +) => + connector.publisher === 'custom' + ? t(`settings.connectors.custom.${connectorLocaleKey(connector.key)}`, { + defaultValue: connector.name, + }) + : connector.name; + +const BUILT_IN_ICONS: Record = { + google_drive: 'drive', + share_point: 'sharepoint', + confluence: 'confluence', + s3: 's3', + reddit: 'reddit', + brave: 'tool_brave', + telegram: 'tool_telegram', + ntfy: 'tool_ntfy', + postgres: 'tool_postgres', + github: 'github', +}; + +/** + * The catalog icon for a connector key, without the catalog: chat chips and + * citations render before (or without) the Connectors data. Presets are + * `mcp:` with a `` logo; a custom server falls back to a plug. + */ +export const connectorIconKey = (key: string | null | undefined) => { + if (!key) return 'plug'; + if (BUILT_IN_ICONS[key]) return BUILT_IN_ICONS[key]; + if (key.startsWith('mcp:')) return key.slice(4); + return 'plug'; +}; diff --git a/frontend/src/connectors/launchRules.ts b/frontend/src/connectors/launchRules.ts new file mode 100644 index 00000000..753ca965 --- /dev/null +++ b/frontend/src/connectors/launchRules.ts @@ -0,0 +1,11 @@ +import type { ConnectorDefinition } from './types'; + +/** An MCP preset that signs in over OAuth: it goes through the wizard. */ +export const isMcpPreset = (connector: ConnectorDefinition) => + connector.auth_kind === 'mcp_oauth' && !!connector.mcp_url; + +/** Connectors whose sign-in the wizard redoes in place, with no form. */ +export const reconnectsInPlace = (connector: ConnectorDefinition) => + connector.auth_kind === 'oauth' || + connector.auth_kind === 'api_key' || + isMcpPreset(connector); diff --git a/frontend/src/connectors/toolService.test.ts b/frontend/src/connectors/toolService.test.ts new file mode 100644 index 00000000..a376766e --- /dev/null +++ b/frontend/src/connectors/toolService.test.ts @@ -0,0 +1,127 @@ +import { toolServiceOf } from './toolService'; +import type { Connection, ConnectorDefinition } from './types'; + +const connection = (overrides: Partial): Connection => + ({ + id: 'conn-1', + connector_key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + account_label: '…abcd', + status: 'connected', + ...overrides, + }) as Connection; + +const connector = ( + overrides: Partial, +): ConnectorDefinition => + ({ + key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + publisher: 'built_in', + tool_templates: ['telegram'], + mcp_url: null, + credential_policy: 'choose', + ...overrides, + }) as ConnectorDefinition; + +const CATALOG = [ + connector({}), + connector({ + key: 'linear', + name: 'Linear', + icon: 'linear', + publisher: 'preset', + tool_templates: ['mcp_tool'], + mcp_url: 'https://mcp.linear.app/mcp', + }), + connector({ + key: 'custom_mcp', + name: 'MCP server', + icon: 'tool_mcp_tool', + publisher: 'custom', + tool_templates: ['mcp_tool'], + }), + connector({ + key: 'custom_openapi', + name: 'OpenAPI / REST', + icon: 'tool_api_tool', + publisher: 'custom', + tool_templates: ['api_tool'], + }), +]; + +describe('toolServiceOf', () => { + it('is undefined for a tool with no connection', () => { + expect( + toolServiceOf( + { name: 'api_tool', displayName: 'My API', connection_id: null }, + [], + CATALOG, + ), + ).toBeUndefined(); + }); + + it("uses the caller's own connection when it is loaded", () => { + const own = connection({}); + expect( + toolServiceOf( + { name: 'telegram', displayName: 'Telegram', connection_id: 'conn-1' }, + [own], + CATALOG, + ), + ).toEqual({ + name: 'Telegram', + icon: 'tool_telegram', + connection: own, + connector: CATALOG[0], + }); + }); + + it("names a teammate's built-in service tool after its connector", () => { + const service = toolServiceOf( + { + name: 'telegram', + displayName: 'Telegram · Alerts bot', + connection_id: 'owner-conn', + }, + [], + CATALOG, + ); + expect(service).toEqual({ + name: 'Telegram', + icon: 'tool_telegram', + connector: CATALOG[0], + }); + }); + + it("matches a teammate's MCP preset by its server", () => { + const service = toolServiceOf( + { + name: 'mcp_tool', + displayName: 'Linear', + connection_id: 'owner-conn', + config: { server_url: 'https://mcp.linear.app/sse' }, + }, + [], + CATALOG, + ); + expect(service?.name).toBe('Linear'); + expect(service?.icon).toBe('linear'); + }); + + it("groups a teammate's custom server under its own name, without an account", () => { + const service = toolServiceOf( + { + name: 'mcp_tool', + displayName: 'Carrier Rates · ops@x', + connection_id: 'owner-conn', + config: { server_url: 'https://rates.example.com/mcp' }, + }, + [], + CATALOG, + ); + expect(service).toEqual({ name: 'Carrier Rates', icon: null }); + }); +}); diff --git a/frontend/src/connectors/toolService.ts b/frontend/src/connectors/toolService.ts new file mode 100644 index 00000000..898ebbf8 --- /dev/null +++ b/frontend/src/connectors/toolService.ts @@ -0,0 +1,99 @@ +import type { Connection, ConnectorDefinition } from './types'; + +/** The service a connected tool comes from, as the pickers group it. */ +export type ToolService = { + name: string; + /** A `ConnectorIcon` name; null draws the tool's own icon. */ + icon: string | null; + /** Set only when the connection is the caller's own. */ + connection?: Connection; + connector?: ConnectorDefinition; +}; + +type ToolLike = { + name: string; + displayName?: string | null; + customName?: string | null; + connection_id?: string | null; + config?: unknown; +}; + +// Templates any server can use (an MCP server, an OpenAPI spec): a tool +// made from one belongs to its connection, not to a connector. Mirrors +// `_GENERIC_TOOL_TEMPLATES` in `docsgpt/connectors/catalog.py`. +const GENERIC_TEMPLATES = new Set(['mcp_tool', 'api_tool']); + +// `scheme://host`, the part of an MCP URL a connection is keyed on. +const baseUrl = (url: unknown): string => { + if (typeof url !== 'string' || !url) return ''; + try { + const parsed = new URL(url); + return `${parsed.protocol}//${parsed.host}`; + } catch { + return ''; + } +}; + +// The catalog connector a tool was made from, found from the tool alone: a +// built-in service by its tool template, an MCP preset by its server. +const connectorForTool = ( + tool: ToolLike, + catalog: ConnectorDefinition[], +): ConnectorDefinition | undefined => { + if (!GENERIC_TEMPLATES.has(tool.name)) { + return catalog.find( + (c) => + c.publisher === 'built_in' && c.tool_templates?.includes(tool.name), + ); + } + if (tool.name !== 'mcp_tool') return undefined; + const server = baseUrl( + (tool.config as { server_url?: unknown } | undefined)?.server_url, + ); + if (!server) return undefined; + return catalog.find( + (c) => + c.publisher !== 'custom' && !!c.mcp_url && baseUrl(c.mcp_url) === server, + ); +}; + +/** + * The service a tool with a connection belongs to, for grouping it in the + * tool pickers. The caller's own connection names it; a teammate's + * connection is never in the caller's list, so a shared tool is named from + * the catalog, or failing that from its own name without the " · account" + * the server adds. Undefined for a tool with no connection. + * + * Args: + * tool: The tool, as `/api/get_tools` returns it. + * connections: The caller's own connections. + * catalog: The connector catalog. + * + * Returns: + * The group name and icon, or undefined. + */ +export function toolServiceOf( + tool: ToolLike, + connections: Connection[], + catalog: ConnectorDefinition[], +): ToolService | undefined { + if (!tool.connection_id) return undefined; + const connection = connections.find((c) => c.id === tool.connection_id); + if (connection) { + const connector = catalog.find((c) => c.key === connection.connector_key); + return { + name: connection.name, + icon: connection.icon, + connection, + ...(connector && { connector }), + }; + } + const connector = connectorForTool(tool, catalog); + if (connector) { + return { name: connector.name, icon: connector.icon, connector }; + } + const own = (tool.displayName || tool.customName || tool.name).split( + ' · ', + )[0]; + return { name: own, icon: null }; +} diff --git a/frontend/src/connectors/types.ts b/frontend/src/connectors/types.ts new file mode 100644 index 00000000..4727480c --- /dev/null +++ b/frontend/src/connectors/types.ts @@ -0,0 +1,178 @@ +export type ConnectorCategory = + | 'files' + | 'knowledge' + | 'projects' + | 'dev' + | 'business' + | 'messaging' + | 'database' + | 'search' + | 'custom'; + +export type Capability = 'sync' | 'read' | 'write'; + +export type ConnectorAuthKind = + 'oauth' | 'mcp_oauth' | 'api_key' | 'none' | 'mcp'; + +export type ConnectionStatus = + 'connected' | 'reconnect_needed' | 'disconnected' | 'error' | 'pending'; + +/** What a catalog card shows, worked out on the server. */ +export type ConnectorCardState = + | 'available' + | 'connected' + | 'reconnect' + | 'needs_setup' + | 'disabled' + | 'custom'; + +export type CredentialMode = 'owner' | 'member'; + +export type CredentialField = { + key: string; + label: string; + secret: boolean; + required: boolean; + /** A tool parameter this field sets for every call (Telegram's chat). */ + parameter?: string | null; + /** English help under the field; translated when the locale has it. */ + hint?: string | null; +}; + +export type ConnectorDefinition = { + key: string; + name: string; + description: string; + icon: string; + category: ConnectorCategory; + auth_kind: ConnectorAuthKind; + capabilities: Capability[]; + credential_fields: CredentialField[]; + setup_fields: CredentialField[]; + sync_ingestor: string | null; + default_sync_frequency: string; + tool_templates: string[]; + setup: { tools: 'auto' | 'ask' | 'off'; sync: 'auto' | 'ask' | 'off' }; + mcp_url: string | null; + /** Its tool can also make changes, when a connection opts in (GitHub). */ + writes_opt_in?: boolean; + publisher: 'built_in' | 'preset' | 'custom'; + docs_url: string | null; + oauth_scopes: string[]; + /** Shown under this connector's card (one service offered two ways). */ + part_of?: string | null; + /** + * How a user can connect, preferred first. GitHub offers `oauth` (Sign in + * with GitHub, once an admin sets up the GitHub App) before `api_key`. + */ + sign_in_methods?: ConnectorAuthKind[]; + available: boolean; + disabled: boolean; + needs_setup: boolean; + missing_settings: string[]; + connected_count: number; + connection_count: number; + status: ConnectionStatus | null; + state: ConnectorCardState; + credential_policy: 'choose' | CredentialMode; + /** A connection may opt into changes: offered, and no admin forbade it. */ + writes_allowed?: boolean; +}; + +export type Connection = { + id: string; + connector_key: string; + name: string; + display_name: string | null; + icon: string; + account_label: string; + /** What the owner calls the account; `account_label` stays its identity. */ + account_name?: string | null; + auth_kind: ConnectorAuthKind | null; + status: ConnectionStatus; + server_url: string | null; + last_error: string | null; + created_at: string | null; + updated_at: string | null; + last_used_at: string | null; + source_count: number; + tool_count: number; +}; + +export type ActionAccess = 'read' | 'write'; +export type ActionPermission = 'always' | 'ask' | 'off'; + +export type ActionParameter = { + name: string; + description: string; + type: string; + required: boolean; + /** Sent with `value` on every call; the AI never sees it. */ + fixed: boolean; + value: string | number | boolean | null; + /** Who fixed it: the tool's own setting, or the account (a default chat). */ + set_by?: 'tool' | 'account' | null; +}; + +export type ConnectionToolAction = { + name: string; + description: string; + access: ActionAccess; + permission: ActionPermission; + parameters?: ActionParameter[]; +}; + +export type ConnectionTool = { + id: string; + name: string; + display_name: string; + status: boolean; + credential_mode: CredentialMode; + actions: ConnectionToolAction[]; +}; + +export type ConnectionSource = { + id: string; + name: string; + type: string; + last_sync: string | null; + sync_frequency: string; + sync_state: 'active' | 'paused_reconnect'; +}; + +/** A repository a GitHub connection can read, for the sync picker. */ +export type GitHubRepository = { + full_name: string; + private: boolean; + description: string; + default_branch: string; + updated_at: string | null; + html_url: string; +}; + +/** A Linear team a Linear connection can sync, for the sync picker. */ +export type LinearTeam = { + id: string; + /** The issue prefix, `ENG` in `ENG-123`. */ + key: string; + name: string; +}; + +/** A Linear project a Linear connection can sync, for the sync picker. */ +export type LinearProject = { + id: string; + name: string; + state: string; + /** Names of the teams it belongs to. */ + teams: string[]; +}; + +export type ConnectionDetail = Connection & { + sources: ConnectionSource[]; + tools: ConnectionTool[]; + /** + * Whether its tool can make changes (GitHub's write endpoint); null when + * the connector offers no such choice or the tool is not added yet. + */ + writes?: boolean | null; +}; diff --git a/frontend/src/connectors/useConnectorLauncher.test.tsx b/frontend/src/connectors/useConnectorLauncher.test.tsx new file mode 100644 index 00000000..2f7bf505 --- /dev/null +++ b/frontend/src/connectors/useConnectorLauncher.test.tsx @@ -0,0 +1,124 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter } from 'react-router-dom'; + +// The wizard is tested on its own; here only what it is opened with matters. +vi.mock('./ConnectWizard', () => ({ + default: ({ + mode, + connectionId, + purpose, + }: { + mode: string; + connectionId?: string; + purpose?: string; + }) => ( +
    + {`${mode}:${connectionId ?? ''}`} +
    + ), +})); +vi.mock('../modals/MCPServerModal', () => ({ default: () => null })); + +import connectorsReducer from './connectorsSlice'; +import type { ConnectorDefinition } from './types'; +import useConnectorLauncher, { + type LaunchOptions, +} from './useConnectorLauncher'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const LINEAR = { + key: 'mcp:linear', + name: 'Linear', + auth_kind: 'mcp_oauth', + mcp_url: 'https://mcp.linear.app/mcp', + sync_ingestor: 'linear', +} as unknown as ConnectorDefinition; + +const DRIVE = { + key: 'google_drive', + name: 'Google Drive', + auth_kind: 'oauth', + mcp_url: null, + sync_ingestor: 'google_drive', +} as unknown as ConnectorDefinition; + +let launchRef: ((c: ConnectorDefinition, o?: LaunchOptions) => void) | null = + null; + +function Harness() { + const { launch, modals } = useConnectorLauncher(); + launchRef = launch; + return <>{modals}; +} + +describe('useConnectorLauncher', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(async () => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + }); + await act(async () => { + root.render( + + + + + , + ); + }); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const wizard = () => + container.querySelector('[data-testid="wizard"]')?.textContent; + + it('opens an MCP preset that syncs straight at picking what to sync', async () => { + await act(async () => + launchRef!(LINEAR, { mode: 'sync', connectionId: 'c1' }), + ); + expect(wizard()).toBe('sync:c1'); + }); + + it('opens an MCP preset at signing in to connect or reconnect', async () => { + await act(async () => launchRef!(LINEAR)); + expect(wizard()).toBe('connect:'); + await act(async () => + launchRef!(LINEAR, { mode: 'reconnect', connectionId: 'c1' }), + ); + expect(wizard()).toBe('reconnect:c1'); + }); + + const purpose = () => + container + .querySelector('[data-testid="wizard"]') + ?.getAttribute('data-purpose'); + + it('tells the wizard it was opened for Knowledge', async () => { + await act(async () => launchRef!(LINEAR, { purpose: 'knowledge' })); + expect(purpose()).toBe('knowledge'); + await act(async () => launchRef!(DRIVE, { purpose: 'knowledge' })); + expect(wizard()).toBe('connect:'); + expect(purpose()).toBe('knowledge'); + }); + + it('leaves a plain connect without a purpose', async () => { + await act(async () => launchRef!(DRIVE)); + expect(purpose()).toBe(''); + }); +}); diff --git a/frontend/src/connectors/useConnectorLauncher.tsx b/frontend/src/connectors/useConnectorLauncher.tsx new file mode 100644 index 00000000..f0aa305f --- /dev/null +++ b/frontend/src/connectors/useConnectorLauncher.tsx @@ -0,0 +1,175 @@ +import { useCallback, useState, type ReactNode } from 'react'; +import { useDispatch, useSelector } from 'react-redux'; +import { useNavigate } from 'react-router-dom'; + +import connectorsService from '../api/services/connectorsService'; +import MCPServerModal from '../modals/MCPServerModal'; +import type { ActiveState } from '../models/misc'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; +import ConnectWizard, { + type LaunchPurpose, + type WizardMode, +} from './ConnectWizard'; +import { loadConnectors } from './connectorsSlice'; +import { isMcpPreset } from './launchRules'; +import type { ConnectorDefinition } from './types'; + +export type LaunchOptions = { + /** `connect` a new account, `reconnect` one, or `sync` more from one. */ + mode?: Exclude; + connectionId?: string; + /** An existing MCP tool to reconnect through the MCP server form. */ + mcpServer?: Record; + /** + * Why the wizard opens. `knowledge` (Add knowledge, Knowledge's Connect a + * service) starts with Sync into Knowledge on; otherwise it starts off. + */ + purpose?: LaunchPurpose; +}; + +type Active = + | { + kind: 'wizard'; + connector: ConnectorDefinition; + mode: WizardMode; + connectionId?: string; + mcpToolId?: string; + purpose?: LaunchPurpose; + } + | { + kind: 'mcp'; + connector: ConnectorDefinition; + server?: Record; + } + | null; + +const isMcp = (connector: ConnectorDefinition) => + connector.auth_kind === 'mcp' || connector.auth_kind === 'mcp_oauth'; + +/** + * The one way to start connecting a catalog entry, used by every entry point + * (the Connectors page and drawer, Add Source, Add Tool, the chat's Connect + * card). Returns `launch` and the modals it drives; render `modals` once. + */ +export default function useConnectorLauncher({ + onConnected, +}: { onConnected?: () => void } = {}) { + const dispatch = useDispatch(); + const navigate = useNavigate(); + const token = useSelector(selectToken); + const [active, setActive] = useState(null); + + const refresh = useCallback(() => { + dispatch(loadConnectors({ token })); + onConnected?.(); + }, [dispatch, token, onConnected]); + + const launch = useCallback( + async (connector: ConnectorDefinition, options: LaunchOptions = {}) => { + if (isMcpPreset(connector)) { + setActive({ + kind: 'wizard', + connector, + // Sync more from an account skips signing in, like any connector. + mode: options.mode ?? 'connect', + connectionId: options.connectionId, + mcpToolId: + typeof options.mcpServer?.id === 'string' + ? options.mcpServer.id + : undefined, + purpose: options.purpose, + }); + return; + } + if (isMcp(connector)) { + setActive({ + kind: 'mcp', + connector, + server: + options.mcpServer ?? + (connector.mcp_url + ? { + displayName: connector.name, + server_url: connector.mcp_url, + auth_type: + connector.auth_kind === 'mcp_oauth' ? 'oauth' : 'none', + oauth_scopes: connector.oauth_scopes.join(', '), + preset: true, + } + : undefined), + }); + return; + } + if (connector.key === 'custom_openapi') { + // The API tool screen on the Tools page imports the spec; the tool + // is only created when it is saved there. + navigate('/settings/tools', { state: { newApiTool: true } }); + return; + } + setActive({ + kind: 'wizard', + connector, + mode: options.mode ?? 'connect', + connectionId: options.connectionId, + purpose: options.purpose, + }); + }, + [navigate], + ); + + const afterMcpSave = async () => { + if (active?.kind !== 'mcp') return; + const connector = active.connector; + const list = await connectorsService.listConnections(token); + refresh(); + const connections = (list?.connections ?? []) as { + id: string; + connector_key: string; + updated_at: string | null; + }[]; + // The newest connection for this connector is the one just saved. + const saved = connections + .filter((c) => c.connector_key === connector.key) + .sort((a, b) => + (b.updated_at ?? '').localeCompare(a.updated_at ?? ''), + )[0]; + setActive( + saved + ? { kind: 'wizard', connector, mode: 'done', connectionId: saved.id } + : null, + ); + }; + + const closeMcp = (state: ActiveState) => { + if (state === 'INACTIVE') setActive(null); + }; + + const modals: ReactNode = ( + <> + {active?.kind === 'wizard' && ( + { + setActive(null); + refresh(); + }} + /> + )} + {active?.kind === 'mcp' && ( + + )} + + ); + + return { launch, modals }; +} diff --git a/frontend/src/connectors/useMcpOAuth.ts b/frontend/src/connectors/useMcpOAuth.ts new file mode 100644 index 00000000..705ef9ce --- /dev/null +++ b/frontend/src/connectors/useMcpOAuth.ts @@ -0,0 +1,125 @@ +import { useCallback, useEffect, useRef, useState } from 'react'; +import { useSelector } from 'react-redux'; + +import userService from '../api/services/userService'; +import { selectRecentEvents } from '../notifications/notificationsSlice'; +import { selectToken } from '../preferences/preferenceSlice'; + +export type McpOAuthConfig = { + server_url: string; + auth_type: 'oauth'; + oauth_scopes: string[]; + timeout: number; + redirect_uri: string; +}; + +type Handlers = { + /** Signed in: `taskId` is the OAuth task to save with (null when the + * server already had a valid sign-in). */ + onDone: (result: { taskId: string | null }) => void; + onError: (message: string) => void; +}; + +/** + * Sign in to an MCP server over OAuth from a click. The pop-up opens blank + * inside the click (so the browser allows it) and follows the worker's + * `mcp.oauth.*` events: pointed at the provider on `awaiting_redirect`, + * closed on `completed` or `failed`. `blockedUrl` is set when the browser + * blocked the pop-up anyway, so the caller can offer the link. + */ +export default function useMcpOAuth() { + const token = useSelector(selectToken); + const events = useSelector(selectRecentEvents); + const [taskId, setTaskId] = useState(null); + const [pending, setPending] = useState(false); + const [blockedUrl, setBlockedUrl] = useState(null); + const popupRef = useRef(null); + const handlersRef = useRef(null); + const handledRef = useRef>(new Set()); + + const closePopup = () => { + if (popupRef.current && !popupRef.current.closed) popupRef.current.close(); + popupRef.current = null; + }; + + const finish = useCallback(() => { + closePopup(); + setTaskId(null); + setPending(false); + handlersRef.current = null; + }, []); + + useEffect(() => finish, [finish]); + + const start = useCallback( + async (config: McpOAuthConfig, handlers: Handlers) => { + handlersRef.current = handlers; + handledRef.current = new Set(); + setBlockedUrl(null); + setPending(true); + closePopup(); + popupRef.current = window.open( + 'about:blank', + 'mcpOAuth', + 'width=600,height=700', + ); + try { + const response = await userService.testMCPConnection({ config }, token); + const result = await response.json(); + if (result.requires_oauth && result.task_id) { + setTaskId(result.task_id); + return; + } + const done = handlersRef.current; + finish(); + if (result.success) done?.onDone({ taskId: null }); + else done?.onError(result.message || result.error || ''); + } catch { + const done = handlersRef.current; + finish(); + done?.onError(''); + } + }, + [token, finish], + ); + + useEffect(() => { + if (!taskId) return; + // Newest first in the slice; walk oldest first so a redirect buffered + // before its completion still opens the provider. + for (let i = events.length - 1; i >= 0; i--) { + const event = events[i]; + if (event.scope?.id !== taskId || !event.id) continue; + if (handledRef.current.has(event.id)) continue; + handledRef.current.add(event.id); + const payload = (event.payload ?? {}) as Record; + if (event.type === 'mcp.oauth.awaiting_redirect') { + const url = payload.authorization_url as string | undefined; + if (!url) continue; + if (popupRef.current && !popupRef.current.closed) { + popupRef.current.location.href = url; + } else { + popupRef.current = window.open( + url, + 'mcpOAuth', + 'width=600,height=700', + ); + if (!popupRef.current) setBlockedUrl(url); + } + } else if (event.type === 'mcp.oauth.completed') { + const done = handlersRef.current; + const id = taskId; + finish(); + done?.onDone({ taskId: id }); + return; + } else if (event.type === 'mcp.oauth.failed') { + const done = handlersRef.current; + finish(); + done?.onError(String(payload.error ?? '')); + return; + } + } + }, [events, taskId, finish]); + + return { start, cancel: finish, pending, blockedUrl }; +} diff --git a/frontend/src/conversation/AnswerFlow.tsx b/frontend/src/conversation/AnswerFlow.tsx index 9e134617..7e208297 100644 --- a/frontend/src/conversation/AnswerFlow.tsx +++ b/frontend/src/conversation/AnswerFlow.tsx @@ -4,6 +4,8 @@ import { useTranslation } from 'react-i18next'; import SchedulerToolCallCard from '../agents/schedules/SchedulerToolCallCard'; import CopyButton from '../components/CopyButton'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { connectorIconKey } from '../connectors/i18n'; import ToolIcon from '../components/ToolIcon'; import { Button } from '../components/ui/button'; import { usePacedText } from '../hooks'; @@ -15,7 +17,7 @@ import { AnswerSegment, getAnswerSegments } from './answerSegments'; import MarkdownAnswer from './MarkdownAnswer'; import { type SandboxArtifact } from './sandboxLinks'; import StreamingStatusLine from './StreamingStatusLine'; -import { ToolCallsType } from './types'; +import { shownArguments, ToolCallsType } from './types'; import { isWikiWriteCall } from './wikiToolCall'; import { cn } from '@/lib/utils'; @@ -270,10 +272,17 @@ function InlineToolCallChip({ isLive ? 'animate-pulse' : '', )} > - + {toolCall.connector_key ? ( + + ) : ( + + )}

    - {JSON.stringify(toolCall.arguments ?? {}, null, 2)} + {JSON.stringify(shownArguments(toolCall), null, 2)}

    ({ + useTranslation: () => ({ + t: (key: string, opts?: { name?: string }) => + opts?.name ? `${key}:${opts.name}` : key, + }), +})); + +vi.mock('../api/services/connectorsService', () => ({ + default: { + getCatalog: vi.fn().mockResolvedValue({ success: true, connectors: [] }), + listConnections: vi + .fn() + .mockResolvedValue({ success: true, connections: [] }), + }, +})); + +const launch = vi.fn(); +const launcherOptions = vi.hoisted(() => ({ + current: {} as { onConnected?: () => void }, +})); +vi.mock('../connectors/useConnectorLauncher', () => ({ + default: (options: { onConnected?: () => void } = {}) => { + launcherOptions.current = options; + return { launch, modals: null }; + }, +})); + +import connectorsReducer from '../connectors/connectorsSlice'; +import ConnectToolCallBar from './ConnectToolCallBar'; +import type { ToolCallsType } from './types'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const call = (status: string): ToolCallsType => ({ + tool_name: 'mcp_tool', + action_name: 'search_pages', + call_id: 'call-1', + arguments: {}, + status: 'awaiting_approval', + connection_required: { + connector_key: 'mcp:notion', + connector_name: 'Notion', + status, + }, +}); + +describe('ConnectToolCallBar', () => { + let root: Root; + let container: HTMLDivElement; + + beforeEach(() => { + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + toolCall: ToolCallsType, + onToolAction = vi.fn(), + connectors?: Record, + ) => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + preloadedState: connectors + ? { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [], + connections: [], + ...connectors, + }, + preference: { token: null }, + } + : undefined, + } as Parameters[0]); + await act(async () => { + root.render( + + + + + } + /> + DRAWER
    } /> + + + , + ); + }); + return onToolAction; + }; + + const button = (text: string) => + Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === text, + )!; + + it('a cancelled connect does not offer Continue', async () => { + await render(call('missing'), vi.fn(), { catalog: [] }); + // The launcher reports every close, cancelled or not. + await act(async () => launcherOptions.current.onConnected?.()); + expect(button('conversation.toolApproval.continue')).toBeUndefined(); + }); + + it('offers Continue once an account for the service is connected', async () => { + await render(call('missing'), vi.fn(), { + connections: [ + { id: 'c9', connector_key: 'mcp:notion', status: 'connected' }, + ], + }); + expect(button('conversation.toolApproval.continue')).toBeDefined(); + }); + + it('names the service it needs', async () => { + await render(call('missing')); + expect(container.textContent).toContain( + 'conversation.toolApproval.connectTitle:Notion', + ); + expect(button('conversation.toolApproval.connect:Notion')).toBeDefined(); + }); + + it('skipping denies the pending call', async () => { + const onToolAction = await render(call('missing')); + await act(async () => button('conversation.toolApproval.skip').click()); + expect(onToolAction).toHaveBeenCalledWith('call-1', 'denied'); + }); + + const TELEGRAM = { + key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + auth_kind: 'api_key', + }; + const ownCall = (status: string): ToolCallsType => ({ + ...call(status), + connection_required: { + connector_key: 'telegram', + connector_name: 'Telegram', + status, + connection_id: 'conn-1', + owner_account: false, + }, + }); + + it('reconnects your own account right here', async () => { + launch.mockClear(); + await render(ownCall('reconnect_needed'), vi.fn(), { + catalog: [TELEGRAM], + connections: [{ id: 'conn-1', status: 'reconnect_needed' }], + }); + await act(async () => + button('conversation.toolApproval.connect:Telegram').click(), + ); + expect(launch).toHaveBeenCalledWith(TELEGRAM, { + mode: 'reconnect', + connectionId: 'conn-1', + }); + expect(document.body.textContent).not.toContain('DRAWER'); + }); + + it('offers Continue once the connection works again', async () => { + await render(ownCall('reconnect_needed'), vi.fn(), { + catalog: [TELEGRAM], + connections: [{ id: 'conn-1', status: 'connected' }], + }); + expect(button('conversation.toolApproval.continue')).toBeDefined(); + }); + + it('an account that needs signing in again is healed from its drawer', async () => { + await render(call('reconnect_needed')); + await act(async () => + button('conversation.toolApproval.connect:Notion').click(), + ); + expect(document.body.textContent).toContain('DRAWER'); + }); +}); diff --git a/frontend/src/conversation/ConnectToolCallBar.tsx b/frontend/src/conversation/ConnectToolCallBar.tsx new file mode 100644 index 00000000..5d8bf3f7 --- /dev/null +++ b/frontend/src/conversation/ConnectToolCallBar.tsx @@ -0,0 +1,133 @@ +import { Plug } from 'lucide-react'; +import { useEffect } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; +import { useNavigate } from 'react-router-dom'; + +import { Button } from '../components/ui/button'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { + loadConnectors, + selectConnections, + selectConnectorCatalog, + selectConnectorsLoaded, +} from '../connectors/connectorsSlice'; +import { reconnectsInPlace } from '../connectors/launchRules'; +import useConnectorLauncher from '../connectors/useConnectorLauncher'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; +import type { ToolCallsType } from './types'; + +/** + * The approval card's "Connect to continue" variant: a paused call to a tool + * whose account needs signing in. Connecting keeps the call pending; Continue + * resumes it with the new connection, Skip denies it. + */ +export default function ConnectToolCallBar({ + toolCall, + onToolAction, +}: { + toolCall: ToolCallsType; + onToolAction?: (callId: string, decision: 'approved' | 'denied') => void; +}) { + const { t } = useTranslation(); + const dispatch = useDispatch(); + const navigate = useNavigate(); + const token = useSelector(selectToken); + const catalog = useSelector(selectConnectorCatalog); + const loaded = useSelector(selectConnectorsLoaded); + const connections = useSelector(selectConnections); + // The launcher refreshes the store whenever its dialog closes, cancelled or + // not, so readiness comes from the connections themselves. + const { launch, modals } = useConnectorLauncher(); + const required = toolCall.connection_required; + const connector = catalog.find((c) => c.key === required?.connector_key); + // A missing account is ready once one for the service works; an account + // that needed signing in again, once it works again. + const ready = required?.connection_id + ? connections.some( + (c) => c.id === required.connection_id && c.status === 'connected', + ) + : connections.some( + (c) => + c.connector_key === required?.connector_key && + c.status === 'connected', + ); + const name = + required?.connector_name || + connector?.name || + t('conversation.toolApproval.thisService'); + + useEffect(() => { + if (!loaded) dispatch(loadConnectors({ token })); + }, [loaded, dispatch, token]); + + const connect = () => { + // No account yet: connect one. The caller's own account that needs + // signing in again is reconnected right here (every tool and source on + // it heals). Custom MCP servers and anyone else's account go to the + // drawer. + if (connector && required?.status === 'missing') { + launch(connector); + return; + } + const inPlace = !!connector && reconnectsInPlace(connector); + if (connector && required?.connection_id && inPlace) { + launch(connector, { + mode: 'reconnect', + connectionId: required.connection_id, + }); + return; + } + navigate( + `/settings/connectors${connector ? `?connector=${encodeURIComponent(connector.key)}` : ''}`, + ); + }; + + return ( +
    +
    + {connector ? ( + + ) : ( + + )} + + {t('conversation.toolApproval.connectTitle', { + name, + interpolation: { escapeValue: false }, + })} + +
    +
    + {ready ? ( + + ) : ( + + )} + +
    + {modals} +
    + ); +} diff --git a/frontend/src/conversation/ConversationBubble.test.tsx b/frontend/src/conversation/ConversationBubble.test.tsx index 6ae7edd2..bc515c19 100644 --- a/frontend/src/conversation/ConversationBubble.test.tsx +++ b/frontend/src/conversation/ConversationBubble.test.tsx @@ -77,6 +77,30 @@ describe('ConversationBubble', () => { (b) => b.textContent?.trim() === text, ) as HTMLButtonElement; + it('shows the fixed values a call will send, not what the model asked', async () => { + await render( + {}} + />, + ); + expect(container.textContent).toContain('"chat_id":"111"'); + expect(container.textContent).not.toContain('666'); + const details = container.querySelector( + `button[aria-label="${tr('conversation.toolApproval.details')}"]`, + ) as HTMLButtonElement; + await act(async () => details.click()); + expect(container.querySelector('pre')?.textContent).toContain('"111"'); + expect(container.textContent).not.toContain('666'); + }); + it('renders the approval bar on Button variants and disables Approve while a deny reason is typed', async () => { await render( void; thought?: string; - sources?: { title: string; text: string; link: string }[]; + sources?: { + title: string; + text: string; + link: string; + connector_key?: string | null; + connector_name?: string | null; + }[]; toolCalls?: ToolCallsType[]; /** Arrival order of the answer's parts; drives inline rendering. */ segments?: AnswerSegment[]; @@ -368,6 +377,23 @@ const ConversationBubble = forwardRef<

    + ) : source.connector_name ? ( +
    + +

    + {t('conversation.sources.fromConnectorTitle', { + name: source.connector_name, + title: source.title, + interpolation: { escapeValue: false }, + })} +

    +
    ) : (
    @@ -473,10 +499,17 @@ const ConversationBubble = forwardRef< onOpenArtifact={onOpenArtifact} renderApproval={(toolCall: ToolCallsType) => (
    - + {toolCall.connection_required ? ( + + ) : ( + + )}
    )} renderWikiWrite={(toolCall: ToolCallsType, isLive: boolean) => ( @@ -653,7 +686,13 @@ function onActivateKey( } type AllSourcesProps = { - sources: { title: string; text: string; link?: string }[]; + sources: { + title: string; + text: string; + link?: string; + connector_key?: string | null; + connector_name?: string | null; + }[]; }; function AllSources(sources: AllSourcesProps) { @@ -706,6 +745,20 @@ function AllSources(sources: AllSourcesProps) { )}

    + {source.connector_name && ( +

    + + + {t('conversation.sources.fromConnector', { + name: source.connector_name, + interpolation: { escapeValue: false }, + })} + +

    + )}

    {source.text}

    @@ -737,7 +790,7 @@ function ToolCallApprovalBar({ 0, toolCall.action_name.lastIndexOf('_'), ); - const argPreview = JSON.stringify(toolCall.arguments); + const argPreview = JSON.stringify(shownArguments(toolCall)); const truncated = argPreview.length > 60 ? argPreview.slice(0, 57) + '...' : argPreview; @@ -846,7 +899,7 @@ function ToolCallApprovalBar({ 
    -                {JSON.stringify(toolCall.arguments, null, 2)}
+                {JSON.stringify(shownArguments(toolCall), null, 2)}
    diff --git a/frontend/src/conversation/conversationModels.ts b/frontend/src/conversation/conversationModels.ts index e8ba8713..fb0f2360 100644 --- a/frontend/src/conversation/conversationModels.ts +++ b/frontend/src/conversation/conversationModels.ts @@ -61,7 +61,14 @@ export interface Query { conversationId?: string | null; title?: string | null; thought?: string; - sources?: { title: string; text: string; link: string }[]; + sources?: { + title: string; + text: string; + link: string; + // A chunk synced from a connection names the service, never the account. + connector_key?: string | null; + connector_name?: string | null; + }[]; tool_calls?: ToolCallsType[]; // Arrival-ordered layout of the fields above, so reasoning and tool calls // render where they happened. Live-stream only; absent on reload, where diff --git a/frontend/src/conversation/types/index.ts b/frontend/src/conversation/types/index.ts index 730ca286..e119bf9c 100644 --- a/frontend/src/conversation/types/index.ts +++ b/frontend/src/conversation/types/index.ts @@ -3,6 +3,9 @@ export type ToolCallsType = { action_name: string; call_id: string; arguments: Record; + // What the call sends once fixed values replace what the model asked for; + // absent when they are the same. The chat shows this. + sent_arguments?: Record; result?: Record; error?: string; status?: @@ -22,4 +25,24 @@ export type ToolCallsType = { // Remote-device tool calls carry the device id so the approval UI can // offer a "don't ask again" sticky-pattern action without a lookup. device_id?: string; + // A connection-backed tool whose account needs signing in pauses on a + // Connect card instead of an approval. Never carries an account or secret. + connection_required?: { + connector_key: string | null; + connector_name: string | null; + status: 'missing' | 'reconnect_needed' | 'disconnected' | 'error' | string; + /** The caller's own connection, reconnected in place. */ + connection_id?: string; + /** The tool runs on its owner's account, not the caller's. */ + owner_account?: boolean; + }; + // Which connection a tool call used, for the connector's logo and name on + // its chip (never an account or a secret). + connector_key?: string | null; + connector_name?: string | null; + access?: 'read' | 'write' | null; }; + +/** The arguments to show for a call: what it sends, not what the model asked. */ +export const shownArguments = (toolCall: ToolCallsType): Record => + toolCall.sent_arguments ?? toolCall.arguments ?? {}; diff --git a/frontend/src/events/dispatchEvent.ts b/frontend/src/events/dispatchEvent.ts index 3d8fc151..bcb0dc01 100644 --- a/frontend/src/events/dispatchEvent.ts +++ b/frontend/src/events/dispatchEvent.ts @@ -44,6 +44,9 @@ const KNOWN_TYPES: ReadonlySet = new Set([ 'graph.extract.progress', 'graph.extract.completed', 'graph.extract.failed', + // A connection's sign-in stopped working (connectors/service.py); + // consumed by ConnectionHealthToast via selectRecentEvents. + 'connection.reconnect_needed', ]); /** diff --git a/frontend/src/hooks/useDataInitializer.ts b/frontend/src/hooks/useDataInitializer.ts index 8cb7a3cb..e56c002a 100644 --- a/frontend/src/hooks/useDataInitializer.ts +++ b/frontend/src/hooks/useDataInitializer.ts @@ -2,6 +2,12 @@ import { useEffect } from 'react'; import { useDispatch, useSelector } from 'react-redux'; import userService from '../api/services/userService'; +import { + loadConnectors, + selectConnectorsEnabled, + setConnectorsEnabled, +} from '../connectors/connectorsSlice'; +import { claimLegacySessionTokens } from '../utils/providerUtils'; import { getDocs, getConversations, @@ -15,6 +21,7 @@ import { setSourceDocs, setSpeechAvailability, } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; /** * useDataInitializer Hook @@ -29,7 +36,7 @@ import { * @param isAuthLoading - */ export default function useDataInitializer(isAuthLoading: boolean) { - const dispatch = useDispatch(); + const dispatch = useDispatch(); const token = useSelector(selectToken); const conversations = useSelector(selectConversations); @@ -45,10 +52,27 @@ export default function useDataInitializer(isAuthLoading: boolean) { stt: config?.stt_available !== false, }), ); + // A backend from before connectors has no flag: hide the page. + dispatch(setConnectorsEnabled(config?.connectors_enabled === true)); }) .catch(() => undefined); }, [dispatch]); + // Connections load once at start so the nav can flag one that needs + // signing in again before any connectors page is opened. + const connectorsEnabled = useSelector(selectConnectorsEnabled); + useEffect(() => { + if (isAuthLoading || !connectorsEnabled) return; + dispatch(loadConnectors({ token })); + }, [isAuthLoading, connectorsEnabled, token, dispatch]); + + // Connector sign-ins used to leave a session token in localStorage. Link + // each one to its server-side connection once, then forget it. + useEffect(() => { + if (isAuthLoading) return; + claimLegacySessionTokens(token); + }, [isAuthLoading, token]); + // Initialize documents useEffect(() => { // Skip if auth is still loading diff --git a/frontend/src/locale/connectors.test.ts b/frontend/src/locale/connectors.test.ts new file mode 100644 index 00000000..7dbafc8c --- /dev/null +++ b/frontend/src/locale/connectors.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from 'vitest'; + +import de from './de.json'; +import en from './en.json'; +import es from './es.json'; +import jp from './jp.json'; +import ru from './ru.json'; +import zhTW from './zh-TW.json'; +import zh from './zh.json'; + +type Tree = { [key: string]: string | Tree }; + +const PLURAL_SUFFIX = /_(zero|one|two|few|many|other)$/; + +const flatten = (tree: Tree, prefix = ''): string[] => + Object.entries(tree).flatMap(([key, value]) => + typeof value === 'string' + ? [prefix + key.replace(PLURAL_SUFFIX, '')] + : flatten(value, `${prefix}${key}.`), + ); + +const values = (tree: Tree): string[] => + Object.values(tree).flatMap((value) => + typeof value === 'string' ? [value] : values(value), + ); + +const block = (locale: object, path: string): Tree => + path + .split('.') + .reduce((node, key) => (node[key] as Tree) ?? {}, locale as Tree); + +const keysOf = (locale: object, path: string): string[] => + Array.from(new Set(flatten(block(locale, path)))).sort(); + +const LOCALES = { es, de, jp, ru, zh, zhTW }; + +// Every block the Connectors work adds strings to. +const BLOCKS = ['settings.connectors']; + +describe('connectors locale blocks', () => { + it.each(Object.entries(LOCALES))( + '%s has the same connector keys as en', + (_name, locale) => { + for (const path of BLOCKS) { + expect(keysOf(locale, path)).toEqual(keysOf(en, path)); + } + }, + ); + + it.each(Object.entries(LOCALES))( + '%s is translated, not an English copy', + (_name, locale) => { + const source = block(en, 'settings.connectors'); + const target = block(locale, 'settings.connectors'); + expect(target.subtitle).not.toBe(source.subtitle); + expect((target.status as Tree).connect).not.toBe( + (source.status as Tree).connect, + ); + }, + ); + + it.each(Object.entries({ en, ...LOCALES }))( + '%s has no em dash in connector strings', + (_name, locale) => { + for (const path of BLOCKS) { + for (const value of values(block(locale, path))) { + expect(value).not.toContain('—'); + } + } + }, + ); +}); diff --git a/frontend/src/locale/de.json b/frontend/src/locale/de.json index 63bfedc4..cdb2fe19 100644 --- a/frontend/src/locale/de.json +++ b/frontend/src/locale/de.json @@ -10,12 +10,6 @@ "loading": "Wird geladen...", "retry": "Erneut versuchen", "cancel": "Abbrechen", - "common": { - "close": "Schließen", - "viewOnlyNotice": "Du kannst dies ansehen, aber deine Rolle kann es nicht ändern.", - "credentialsLockedNotice": "Nur der Eigentümer kann die Zugangsdaten dieses Tools ändern.", - "savedSecretHint": "Gespeichert. Leer lassen, um es zu behalten." - }, "help": "Hilfe", "emailUs": "E-Mail senden", "documentation": "Dokumentation", @@ -106,16 +100,16 @@ }, "sources": { "title": "Hier kannst du alle verfügbaren Quelldateien verwalten, die dir zur Verfügung stehen und die du hochgeladen hast.", - "subtitle": "Dokumente und Wissensquellen hochladen und verwalten, die deine Antworten antreiben", - "label": "Quellen", + "subtitle": "Inhalte, die DocsGPT durchsuchen und zitieren kann: Uploads, Websites, Wikis und aus deinen Verbindungen synchronisierte Inhalte.", + "label": "Wissen", "name": "Quellenname", "date": "Vektor-Datum", "type": "Typ", "tokenUsage": "Token-Verbrauch", - "noData": "Keine vorhandenen Quellen", + "noData": "Noch kein Wissen", "searchPlaceholder": "Suchen...", "addNew": "Neu hinzufügen", - "addSource": "Quelle hinzufügen", + "addSource": "Wissen hinzufügen", "addChunk": "Chunk hinzufügen", "preLoaded": "Vorgeladen", "private": "Privat", @@ -185,7 +179,17 @@ "pagesTitle": "Seiten", "loadFailed": "Die Wiki-Seiten konnten nicht geladen werden.", "pageLoadFailed": "Diese Seite konnte nicht geladen werden.", - "editTitle": "Seite bearbeiten" + "editTitle": "Seite bearbeiten", + "settings": { + "action": "Wiki-Einstellungen", + "title": "Wiki-Einstellungen", + "outsideEdits": { + "label": "API- und Widget-Nutzern erlauben, dieses Wiki zu bearbeiten", + "description": "Wenn das aus ist, können Personen, die einen Agenten über seinen API-Schlüssel oder sein Widget nutzen, dieses Wiki lesen, aber nicht ändern. Besucher über einen öffentlichen Link können nur Wikis bearbeiten, die sie selbst bearbeiten dürfen, und werden vor jeder Änderung gefragt. Du und die Bearbeiter des Wikis können es immer bearbeiten." + }, + "loadError": "Die Wiki-Einstellungen konnten nicht geladen werden.", + "saveError": "Die Änderung konnte nicht gespeichert werden. Bitte versuche es erneut." + } }, "graphrag": { "badge": "GraphRAG", @@ -303,7 +307,7 @@ "chunks": "Chunks", "noChunks": "Keine Chunks gefunden", "noChunksAlt": "Keine Chunks gefunden", - "goToSources": "Zu den Quellen", + "goToSources": "Zum Wissen", "uploadNew": "Neu hochladen", "noResults": "Keine Ergebnisse gefunden", "fileName": "Name", @@ -446,6 +450,9 @@ "editChunkDescription": "{{file}} · Chunk {{n}} · {{tokens}} Tokens", "previousChunk": "Vorheriger Chunk", "nextChunk": "Nächster Chunk", + "paused": "Sync pausiert. Verbinde {{name}} neu, um fortzufahren.", + "emptyHint": "Lade Dateien hoch, füge eine Website hinzu oder verbinde einen Dienst, um seine Inhalte zu synchronisieren.", + "connectService": "Dienst verbinden", "viewConfig": "Quelleneinstellungen ansehen", "errors": { "forbidden": "Dazu hast du für diese Quelle keine Berechtigung.", @@ -676,10 +683,6 @@ "createTeamError": "Team konnte nicht erstellt werden.", "noTeams": "Noch keine Teams.", "noDescription": "Keine Beschreibung", - "memberCount_one": "{{formatted}} Mitglied", - "memberCount_other": "{{formatted}} Mitglieder", - "sharedCount_one": "{{formatted}} geteilt", - "sharedCount_other": "{{formatted}} geteilt", "loadError": "Teams konnten nicht geladen werden.", "roleAdmin": "Admin", "roleMember": "Mitglied", @@ -759,8 +762,46 @@ "teams": "Teams", "people": "Personen", "editors": "Bearbeiter" + }, + "uses": { + "title": "Was dieser Agent nutzt", + "intro": "Alle, die diesen Agenten nutzen, erhalten diese Elemente, jeweils mit dem angezeigten Zugriff.", + "stopped": "Gestoppt", + "writesOff": "Nicht über API", + "writesNote": "„Erlauben“ bei einem Connector gilt für Chats. Mit „Nicht über API“ markierte Änderungen bleiben für API-, Widget- und Link-Nutzer aus, bis du sie unter Zugangsdaten erlaubst.", + "writesNoteEditor": "„Erlauben“ bei einem Connector gilt für Chats. Mit „Nicht über API“ markierte Änderungen bleiben für API-, Widget- und Link-Nutzer aus, bis der Eigentümer sie unter Zugangsdaten erlaubt.", + "access": { + "you": "Dein Zugriff", + "person": "Zugriff von {{person}}", + "other": "Zugriff einer anderen Person", + "owner": "Zugriff des Eigentümers", + "member": "Das eigene {{service}}-Konto jeder Person (API und Widget: deins)", + "memberNoService": "Das eigene Konto jeder Person (API und Widget: deins)", + "yourAccount": "Dein {{service}}-Konto", + "yourAccountNoService": "Dein Konto", + "personAccount": "{{service}}-Konto von {{person}}", + "personAccountNoService": "Konto von {{person}}", + "memberShared": "Das eigene {{service}}-Konto jeder Person (API und Widget: das des Eigentümers)", + "memberSharedNoService": "Das eigene Konto jeder Person (API und Widget: das des Eigentümers)", + "otherAccount": "{{service}}-Konto einer anderen Person", + "otherAccountNoService": "Konto einer anderen Person", + "yourCredentials": "Deine gespeicherten Zugangsdaten", + "personCredentials": "Gespeicherte Zugangsdaten von {{person}}", + "otherCredentials": "Gespeicherte Zugangsdaten einer anderen Person" + }, + "writesSomeOff": "Nicht alles über API", + "adminOff": "Änderungen von einem Admin deaktiviert", + "writesNoteApi": "„Erlauben“ bei einem Connector gilt für Chats. Mit „Nicht über API“ markierte Änderungen bleiben für API- und Widget-Nutzer aus, bis du sie unter Zugangsdaten erlaubst.", + "writesNoteApiEditor": "„Erlauben“ bei einem Connector gilt für Chats. Mit „Nicht über API“ markierte Änderungen bleiben für API- und Widget-Nutzer aus, bis der Eigentümer sie unter Zugangsdaten erlaubt.", + "writesNoteMemberTail": "Bei Werkzeugen, bei denen jede Person ihr eigenes Konto nutzt, ändern Link-Nutzer mit ihrem eigenen Konto.", + "openAccessDetails": "Zugangsdaten öffnen", + "accessDetailsNeedsKey": "Die Zugangsdaten listen diese Änderungen auf, sobald der Agent einen API-Schlüssel hat." } }, + "memberCount_one": "{{formatted}} Mitglied", + "memberCount_other": "{{formatted}} Mitglieder", + "sharedCount_one": "{{formatted}} geteilt", + "sharedCount_other": "{{formatted}} geteilt", "accessChangeError": "Zugriff konnte nicht geändert werden.", "accessSettings": { "title": "Zugriffseinstellungen", @@ -808,7 +849,8 @@ }, "viewers_can_use_in_agents": { "label": "Betrachter dürfen es in eigenen Agenten nutzen", - "description": "Es läuft mit deinen Anmeldedaten." + "description": "Es läuft mit deinen Anmeldedaten.", + "descriptionMember": "Jede Person nutzt ihr eigenes Konto." } }, "prompt": { @@ -930,11 +972,11 @@ }, "tools": { "label": "Werkzeuge", - "subtitle": "Werkzeuge und Integrationen durchsuchen, verbinden und konfigurieren, die deine Agenten antreiben", + "subtitle": "Aktionen, die deine Agenten ausführen können. Integrierte Werkzeuge findest du hier. Werkzeuge eines verbundenen Dienstes sind unter diesem Dienst gruppiert.", "searchPlaceholder": "Werkzeuge suchen...", "addTool": "Werkzeug hinzufügen", "noToolsFound": "Keine Werkzeuge gefunden", - "selectToolSetup": "Wähle ein Werkzeug zur Einrichtung", + "selectToolSetup": "Werkzeug hinzufügen", "settingsIconAlt": "Einstellungssymbol", "configureToolAria": "{{toolName}} konfigurieren", "toggleToolAria": "{{toolName}} umschalten", @@ -946,8 +988,8 @@ "builtIn": "Integriert", "authStatus": { "connected": "Verbunden", - "needsAuth": "Authentifizierung nötig", - "configured": "Konfiguriert" + "needsAuth": "Neu verbinden", + "configured": "Verbunden" }, "deleteWarning": "Bist du sicher, dass du das Werkzeug \"{{toolName}}\" löschen möchtest?", "unsavedChanges": "Du hast ungespeicherte Änderungen, die verloren gehen, wenn du ohne Speichern verlässt.", @@ -1063,6 +1105,11 @@ }, "headerValuePlaceholder": "z. B. application/json", "toolIconTitle": "{{name}}-Symbol", + "groupBuiltIn": "Integriert", + "groupService": "Aus einem Dienst", + "browseConnectors": "Alle Konnektoren ansehen", + "viaConnection": "über {{name}}", + "off": "Aus", "view": "Ansehen", "inMyChats": "In meinen Chats", "useInMyChatsAria": "{{toolName}} in meinen Chats verwenden", @@ -1298,13 +1345,320 @@ "subtitle": "Das neue Geheimnis für \"{{name}}\" ist bereit. Das alte funktioniert nicht mehr." } } + }, + "connectors": { + "label": "Konnektoren", + "subtitle": "Verbinde DocsGPT mit den Diensten, die dein Team nutzt. Eine Verbindung kann Inhalte ins Wissen synchronisieren und Agenten Werkzeuge geben.", + "search": "Konnektoren suchen...", + "addCustom": "Eigenes Werkzeug hinzufügen", + "custom": { + "custom_mcp": "MCP-Server", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "Konnektoren nach Kategorie filtern", + "categories": { + "all": "Alle", + "connected": "Verbunden", + "files": "Dateien & Speicher", + "knowledge": "Dokus & Wikis", + "dev": "Entwicklerwerkzeuge", + "messaging": "Nachrichten", + "database": "Datenbanken", + "search": "Suche & Web", + "custom": "Eigene", + "projects": "Projekte & Tickets", + "business": "Zahlungen & Business" + }, + "capability": { + "sync": "Sync", + "read": "Lesen", + "write": "Schreiben" + }, + "status": { + "connect": "Verbinden", + "connected": "Verbunden", + "connectedCount": "{{formatted}} verbunden", + "reconnect": "Neu verbinden", + "needsAdminSetup": "Admin-Einrichtung nötig", + "disabledByAdmin": "Vom Admin deaktiviert", + "custom": "Eigener" + }, + "connectionStatus": { + "connected": "Verbunden", + "reconnect_needed": "Neu verbinden nötig", + "disconnected": "Getrennt", + "error": "Fehler", + "pending": "Wartet auf Anmeldung" + }, + "publisher": { + "built_in": "Integriert", + "preset": "Vorlage", + "custom": "Eigener" + }, + "descriptions": { + "google_drive": "Docs, Tabellen und PDFs ins Wissen synchronisieren.", + "share_point": "Dateien aus SharePoint-Sites und OneDrive ins Wissen synchronisieren.", + "confluence": "Confluence-Bereiche und -Seiten ins Wissen synchronisieren.", + "s3": "Dokumente aus einem S3-Bucket ins Wissen synchronisieren.", + "reddit": "Reddit-Beiträge zu deinen Suchen ins Wissen synchronisieren.", + "brave": "Web und Bilder mit der Brave Search API durchsuchen.", + "telegram": "Nachrichten und Bilder an einen Telegram-Chat senden.", + "ntfy": "Push-Benachrichtigungen über einen ntfy-Server senden.", + "postgres": "Schema lesen und SQL auf einer Postgres-Datenbank ausführen.", + "custom_mcp": "Einen beliebigen Remote-Server für das Model Context Protocol verbinden.", + "custom_openapi": "Eine OpenAPI-Spezifikation importieren und ihre Endpunkte als Werkzeuge aufrufen.", + "mcp_notion": "Notion-Seiten und -Datenbanken durchsuchen, lesen und aktualisieren.", + "mcp_linear": "Issues und Dokumente in Wissen synchronisieren und Agenten Issues finden, anlegen und aktualisieren lassen.", + "mcp_atlassian": "Jira-Issues und Confluence-Seiten durchsuchen und aktualisieren.", + "mcp_sentry": "Sentry-Issues, -Events und -Releases nachschlagen.", + "mcp_asana": "Asana-Aufgaben und -Projekte finden und aktualisieren.", + "mcp_stripe": "Kunden, Zahlungen und Abonnements in Stripe nachschlagen.", + "github": "Repositorys in Wissen synchronisieren und Agenten Code, Issues und Pull Requests lesen lassen." + }, + "empty": "Noch nichts verbunden. Verbinde einen Dienst, um seine Inhalte zu synchronisieren oder Agenten neue Werkzeuge zu geben.", + "noMatches": "Keine Konnektoren passen zu deiner Suche.", + "loadFailed": "Konnektoren konnten nicht geladen werden.", + "askAdmin": "Bitte einen Admin, dies einzurichten.", + "setupSettings": "Setze diese Servereinstellungen und starte DocsGPT neu:", + "setupGuide": "Einrichtungsanleitung", + "unverified": "Dieser Server wurde nicht von DocsGPT geprüft. Verbinde nur Server, denen du vertraust.", + "detail": { + "accounts": "Konten", + "connectedAs": "Verbunden als {{account}}", + "connectAnother": "Weiteres Konto verbinden", + "sources": "Wissen aus dieser Verbindung", + "tools": "Werkzeuge aus dieser Verbindung", + "noAccounts": "Noch kein Konto verbunden.", + "failed": "Diese Verbindung konnte nicht geladen werden.", + "lastSync": "Synchronisiert {{date}} · {{frequency}}", + "paused": "Sync pausiert", + "toolOn": "An", + "toolOff": "Aus", + "accountMenu": "Kontoaktionen", + "disconnect": "Trennen", + "keyEnding": "Schlüssel endet auf {{hint}}", + "syncMore": "Weitere Inhalte synchronisieren", + "refreshTools": "Werkzeuge aktualisieren", + "noSources": "Aus dieser Verbindung wurde noch nichts synchronisiert.", + "remove": "Entfernen", + "refreshed": "Werkzeuge aktualisiert: {{added}} hinzugefügt, {{removed}} entfernt.", + "refreshFailed": "Die Werkzeuge konnten nicht aktualisiert werden.", + "syncNow": "Jetzt synchronisieren", + "syncStarted": "{{name}} wird synchronisiert…", + "syncFailed": "Die Synchronisierung konnte nicht gestartet werden. Versuche es erneut.", + "toolSwitchFailed": "Das Werkzeug konnte nicht geändert werden. Versuche es erneut.", + "account": "Konto: {{account}}", + "expired": "Die Anmeldung ist abgelaufen. Verbinde neu, um diese Verbindung weiter zu nutzen.", + "broken": "Diese Verbindung funktioniert nicht mehr. Verbinde neu, um es erneut zu versuchen.", + "addTools": "Werkzeuge hinzufügen", + "noTools": "Noch keine Werkzeuge aus dieser Verbindung.", + "addToolsFailed": "Die Werkzeuge konnten nicht hinzugefügt werden. Versuche es erneut.", + "rename": "Umbenennen", + "accountPicker": "Konto auswählen" + }, + "permission": { + "always": "Erlauben", + "ask": "Vorher fragen", + "off": "Aus", + "label": "Berechtigung für {{action}}", + "saveFailed": "Die Berechtigung konnte nicht gespeichert werden. Versuche es erneut.", + "groupLabel": "Berechtigung für alles unter {{group}}", + "fold": "Aktionen ausblenden", + "hint": "„Erlauben“ und „Vorher fragen“ gelten im Chat; bei „Vorher fragen“ fragt DocsGPT dich, bevor die Aktion ausgeführt wird. Änderungen über die API, das Widget oder den öffentlichen Link eines Agenten erlaubst du pro Agent unter seinen Zugangsdaten.", + "actionCount_one": "{{formatted}} Aktion", + "actionCount_other": "{{formatted}} Aktionen", + "customize_one": "Aktion einzeln festlegen", + "customize_other": "Alle {{formatted}} einzeln festlegen" + }, + "disconnect": { + "title": "{{name}} trennen?", + "body_zero": "Werkzeuge aus dieser Verbindung funktionieren erst wieder, wenn du neu verbindest.", + "body_one": "Das synchronisierte Element bleibt im Wissen, wird aber nicht mehr aktualisiert. Werkzeuge dieser Verbindung funktionieren erst nach dem Neuverbinden wieder.", + "body_other": "Die {{formatted}} synchronisierten Elemente bleiben im Wissen, werden aber nicht mehr aktualisiert. Werkzeuge dieser Verbindung funktionieren erst nach dem Neuverbinden wieder.", + "failed": "Trennen fehlgeschlagen. Versuche es erneut." + }, + "fields": { + "token": "API-Schlüssel", + "aws_access_key_id": "Zugriffsschlüssel-ID", + "aws_secret_access_key": "Geheimer Zugriffsschlüssel", + "region": "Region", + "endpoint_url": "Eigene Endpunkt-URL", + "client_id": "Client-ID", + "client_secret": "Client-Secret", + "user_agent": "User-Agent", + "bucket": "Bucket", + "prefix": "Pfadpräfix", + "search_queries": "Suchanfragen", + "number_posts": "Anzahl der Beiträge", + "telegram_token": "Bot-Token", + "ntfy_token": "Zugriffstoken", + "postgres_token": "Verbindungszeichenfolge", + "brave_token": "API-Schlüssel", + "github_access_token": "Persönliches Zugriffstoken", + "repo_url": "Repository", + "telegram_chat_id": "Standard-Chat-ID" + }, + "wizard": { + "connectTitle": "{{name}} verbinden", + "reconnectTitle": "{{name}} neu verbinden", + "signIn": "Bei {{name}} anmelden", + "chooseWhatToSync": "Auswählen, was synchronisiert wird", + "skip": "Vorerst überspringen", + "syncFrequency": "Sync", + "doneTitle": "{{name}} ist verbunden", + "doneSummary": "Verbunden als {{account}}. {{sources}} werden ins Wissen synchronisiert, {{tools}} bereit.", + "doneSummaryNone": "Verbunden als {{account}}.", + "sourcesCount_one": "{{formatted}} Element", + "sourcesCount_other": "{{formatted}} Elemente", + "toolsCount_one": "{{formatted}} Werkzeug", + "toolsCount_other": "{{formatted}} Werkzeuge", + "toolsHeading": "Werkzeuge ({{formatted}})", + "tryInChat": "Im Chat ausprobieren", + "done": "Fertig", + "connectFailed": "Verbindung fehlgeschlagen. Prüfe die Angaben und versuche es erneut.", + "syncFailed": "Die Synchronisierung konnte nicht gestartet werden. Versuche es erneut.", + "connectedWithKey": "Verbunden mit dem Schlüssel, der auf {{hint}} endet.", + "doneCounts": "{{sources}} werden ins Wissen synchronisiert, {{tools}} bereit.", + "doneSources": "{{sources}} werden ins Wissen synchronisiert.", + "doneTools": "{{tools}} bereit.", + "waiting": "Schließe die Anmeldung bei {{name}} im geöffneten Fenster ab.", + "openSignIn": "Anmeldung bei {{name}} öffnen", + "chooseWhatToSetUp": "Auswählen, was eingerichtet wird", + "continue": "Weiter", + "methodLabel": "Art der Verbindung", + "methodOauth": "Mit {{name}} anmelden", + "methodToken": "Zugriffstoken verwenden", + "credentialsRejected": "{{name}} hat diese Zugangsdaten nicht akzeptiert. Prüfe sie und versuche es erneut.", + "toolsUnavailable": "Die Werkzeuge von {{name}} waren nicht erreichbar. Versuche es erneut oder schalte sie aus, um fortzufahren.", + "addTools": "Agenten {{name}} nutzen lassen", + "addToolsDescription": { + "read": "Agenten können Dinge nachschlagen. Sie können nichts ändern.", + "readWrite": "Agenten können Dinge nachschlagen und ändern. Änderungen brauchen deine Freigabe." + }, + "accountName": "Konto benennen", + "accountNameHint": "Optional. Unterscheidet deine Konten, für dich und die KI, z. B. Alarm-Bot.", + "syncToKnowledge": "Ins Wissen synchronisieren", + "syncToKnowledgeDescription": "Wähle Inhalte aus {{name}}, die Antworten durchsuchen. Bleibt es aus, wird nur verbunden; synchronisieren kannst du später unter Konnektoren.", + "syncLater": "Noch wird nichts ins Wissen synchronisiert. Um Inhalte hinzuzufügen, öffne {{name}} unter Konnektoren und wähle Weitere Inhalte synchronisieren.", + "retrievalSettings": "Erweiterte Abrufeinstellungen" + }, + "remove": { + "title": "{{name}} entfernen?", + "description": "Die gespeicherte Anmeldung wird gelöscht. Wähle, was mit den Inhalten dieser Verbindung passiert.", + "sourcesLabel_one": "Das Element, das sie ins Wissen synchronisiert hat", + "sourcesLabel_other": "Die {{formatted}} Elemente, die sie ins Wissen synchronisiert hat", + "keepSources": "Inhalt behalten (Sync stoppen)", + "deleteSources": "Löschen", + "toolsLabel_one": "Ihr Werkzeug", + "toolsLabel_other": "Ihre {{formatted}} Werkzeuge", + "deleteTools": "Löschen", + "keepTools": "Behalten, ausgeschaltet", + "failed": "Die Verbindung konnte nicht entfernt werden. Versuche es erneut." + }, + "health": { + "reconnect": "Verbinde {{name}} neu, um weiter zu synchronisieren.", + "reconnectSources_one": "{{name}} neu verbinden: {{formatted}} Element wird nicht mehr ins Wissen synchronisiert", + "reconnectSources_other": "{{name}} neu verbinden: {{formatted}} Elemente werden nicht mehr ins Wissen synchronisiert", + "reconnectTools": "{{name}} neu verbinden, damit Agenten es wieder nutzen können", + "reconnectBoth_one": "{{name}} neu verbinden: {{formatted}} Element synchronisiert nicht mehr und Agenten können es nicht nutzen", + "reconnectBoth_other": "{{name}} neu verbinden: {{formatted}} Elemente synchronisieren nicht mehr und Agenten können es nicht nutzen", + "navDot": "Eine Verbindung braucht Aufmerksamkeit", + "signInAgain": "Erneut anmelden", + "pickerNotice": "Bei {{name}} musst du dich erneut anmelden." + }, + "error": { + "decrypt": "Die gespeicherten Zugangsdaten konnten nicht entschlüsselt werden. Verbinde neu, um fortzufahren.", + "defaultKey": "Ein Admin muss ENCRYPTION_SECRET_KEY setzen, bevor Dienste verbunden werden können." + }, + "sharing": { + "owner": "Alle nutzen dein Konto", + "member": "Jede Person verbindet ihr eigenes Konto", + "ownerShort": "Dein Konto", + "memberShort": "Jeweils eigenes", + "ownerShortShared": "Konto des Eigentümers" + }, + "share": { + "heading": "Mit wessen Konto es läuft", + "ownerWarning": "Teammitglieder handeln in {{name}} als {{account}}.", + "ownerWarningShared": "Teammitglieder handeln mit dem {{name}}-Konto des Eigentümers.", + "confirmWrite": "Ich verstehe, dass Teammitglieder mit meinem Konto Aktionen ausführen können.", + "confirmWriteShared": "Ich verstehe, dass Teammitglieder mit dem {{name}}-Konto des Eigentümers Aktionen ausführen können.", + "memberNote": "Jede Person nutzt ihr eigenes {{name}}-Konto und verbindet es, wenn sie dieses Werkzeug zum ersten Mal nutzt. Über den API-Schlüssel oder das Widget eines Agenten wird das Konto des Agent-Eigentümers genutzt.", + "forced": "Ein Admin hat das für alle Freigaben dieses Konnektors festgelegt.", + "ownerChooses": "Nur der Eigentümer kann das ändern.", + "saveFailed": "Die Änderung konnte nicht gespeichert werden. Versuche es erneut." + }, + "createFailed": "Der Konnektor konnte nicht hinzugefügt werden. Versuche es erneut.", + "capabilityFilter": { + "sync": "Angezeigt werden Konnektoren, die Inhalte ins Wissen synchronisieren.", + "tools": "Angezeigt werden Konnektoren, die Agenten Werkzeuge geben.", + "showAll": "Alle anzeigen" + }, + "capabilityPlain": { + "sync": "Synchronisiert ins Wissen", + "read": "Schlägt nach", + "write": "Führt Aktionen aus" + }, + "github": { + "tokenHint": "Erstelle ein feingranulares Token mit Lesezugriff auf „Contents“ und „Metadata“ für die Repositorys, die du nutzen möchtest. Damit Agenten Änderungen vornehmen können, gib ihm außerdem Lese- und Schreibzugriff auf „Issues“ und „Pull requests“.", + "writes": "Agenten auch Änderungen erlauben (Issues, Kommentare, Pull Requests)", + "writesDescription": "Änderungen fragen dich standardmäßig zuerst. Dein Token oder die GitHub App braucht Lese- und Schreibzugriff auf „Issues“ und „Pull requests“.", + "writesForbidden": "Ein Admin hat Änderungen über GitHub ausgeschaltet.", + "writesFailed": "Es konnte nicht geändert werden, was Agenten auf GitHub dürfen.", + "createToken": "Token auf GitHub erstellen", + "chooseRepositories": "Repositorys auf GitHub auswählen", + "noAppRepositories": "DocsGPT sieht noch keine Repositorys. Wähle auf GitHub aus, welche es lesen darf.", + "noTokenRepositories": "Dieses Token kann keine Repositorys lesen. Gib ihm auf GitHub Zugriff und verbinde dann neu.", + "searchRepositories": "Repositorys durchsuchen", + "repositories": "Repositorys", + "updated": "Aktualisiert {{date}}", + "loadFailed": "Deine Repositorys konnten nicht geladen werden." + }, + "linear": { + "teams": "Teams", + "teamsHint": "Die Issues dieser Teams synchronisieren.", + "projects": "Projekte", + "projectsHint": "Die Issues dieser Projekte synchronisieren.", + "search": "Teams und Projekte durchsuchen", + "noMatches": "Keine Teams oder Projekte passen zu deiner Suche.", + "includeComments": "Kommentare einbeziehen", + "includeCommentsDescription": "Jedem Issue seine Kommentare hinzufügen.", + "includeDocuments": "Projektdokumente einbeziehen", + "includeDocumentsDescription": "Die Linear-Dokumente der ausgewählten Projekte synchronisieren.", + "loadFailed": "Deine Linear-Teams und -Projekte konnten nicht geladen werden.", + "empty": "Dieses Linear-Konto hat keine Teams oder Projekte zum Synchronisieren." + }, + "parameters": { + "show": "Parameter", + "hide": "Parameter ausblenden", + "fixedCount_one": "{{formatted}} festgelegt", + "fixedCount_other": "{{formatted}} festgelegt", + "ai": "KI entscheidet", + "fixed": "Immer verwenden", + "choiceLabel": "Wie {{parameter}} ausgefüllt wird", + "valueLabel": "Wert für {{parameter}}", + "save": "Speichern", + "saveFailed": "Der Wert konnte nicht gespeichert werden. Versuche es erneut.", + "hint": "Ein Wert mit „Immer verwenden“ wird bei jedem Aufruf gesendet. Die KI sieht ihn nicht und kann ihn nicht ändern.", + "fromAccount": "In diesem Konto festgelegt: {{value}}" + }, + "fieldHints": { + "telegram_chat_id": "Optional. Nachrichten gehen an diesen Chat, und die KI kann keinen anderen wählen. Füge den Bot zum Chat hinzu, sende ihm eine Nachricht und finde die ID des Chats unter https://api.telegram.org/bot/getUpdates." + }, + "rename": { + "title": "Konto umbenennen", + "description": "Der Name unterscheidet dieses Konto von deinen anderen Konten desselben Dienstes, für dich und die KI. Lass ihn leer, um das Konto selbst anzuzeigen.", + "label": "Kontoname", + "save": "Speichern", + "failed": "Das Konto konnte nicht umbenannt werden. Versuche es erneut." + } } }, "modals": { "uploadDoc": { - "label": "Neues Dokument hochladen", + "label": "Wissen hinzufügen", "select": "Wähle, wie du dein Dokument zu DocsGPT hochladen möchtest", - "selectSource": "Wähle die Art, wie du deine Quelle hinzufügen möchtest", + "selectSource": "Wissen hinzufügen", "selectedFiles": "Ausgewählte Dateien", "noFilesSelected": "Keine Dateien ausgewählt", "file": "Vom Gerät hochladen", @@ -1319,19 +1673,11 @@ "filesRejected": "Nicht hinzugefügt (größer als 25 MB oder kein unterstützter Dateityp): {{files}}", "uploadedFiles": "Hochgeladene Dateien", "cancel": "Abbrechen", - "train": "Trainieren", + "train": "Zum Wissen hinzufügen", "create": "Erstellen", "link": "Link", "urlLink": "URL-Link", "repoUrl": "Repository-URL", - "reddit": { - "id": "Client-ID", - "secret": "Client-Secret", - "agent": "User-Agent", - "searchQueries": "Suchanfragen", - "numberOfPosts": "Anzahl der Beiträge", - "addQuery": "Anfrage hinzufügen" - }, "drag": { "title": "Anhänge hier ablegen", "description": "Loslassen, um deine Anhänge hochzuladen" @@ -1353,8 +1699,6 @@ "clear": "Löschen", "ingestionFailed": "Aufnahme fehlgeschlagen." }, - "showAdvanced": "Erweiterte Optionen anzeigen", - "hideAdvanced": "Erweiterte Optionen ausblenden", "ingestors": { "local_file": { "label": "Datei hochladen", @@ -1372,26 +1716,6 @@ "label": "GitHub", "heading": "Inhalt von GitHub hinzufügen" }, - "reddit": { - "label": "Reddit", - "heading": "Inhalt von Reddit hinzufügen" - }, - "google_drive": { - "label": "Google Drive", - "heading": "Von Google Drive hochladen" - }, - "s3": { - "label": "Amazon S3", - "heading": "Inhalt von Amazon S3 hinzufügen" - }, - "share_point": { - "label": "SharePoint", - "heading": "Von SharePoint hochladen" - }, - "confluence": { - "label": "Confluence", - "heading": "Von Confluence hochladen" - }, "wiki": { "label": "Neues Wiki", "heading": "Lebendiges Wiki erstellen" @@ -1445,13 +1769,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "AWS-Zugriffsschlüssel-ID", - "awsSecretAccessKey": "Geheimer AWS-Zugriffsschlüssel", - "bucket": "Bucket-Name", - "prefix": "Pfadpräfix (optional)", - "region": "AWS-Region", - "endpointUrl": "Benutzerdefinierte Endpunkt-URL (optional)", "initialContent": "Anfangsinhalt (optional)" + }, + "github": { + "privateHint": "Privates Repository? Verbinde dein GitHub-Konto, um es zu synchronisieren.", + "connectedHint": "Private Repositorys werden über deine GitHub-Verbindung ({{account}}) synchronisiert.", + "connect": "GitHub verbinden", + "pickRepository": "Repository auswählen" + }, + "connectData": { + "title": "Deine Daten verbinden", + "description": "{{services}} synchronisieren", + "more": "mehr", + "intro": "Wähle einen Dienst, melde dich an und wähle dann, was in Wissen synchronisiert wird.", + "connectedAs": "Verbunden als {{account}}", + "browseAll": "Alle Connectors ansehen" } }, "createAPIKey": { @@ -1488,8 +1820,8 @@ "configTool": { "title": "Werkzeug-Konfiguration", "type": "Typ", - "apiKeyLabel": "API-Schlüssel / OAuth", - "apiKeyPlaceholder": "API-Schlüssel / OAuth eingeben", + "apiKeyLabel": "API-Schlüssel", + "apiKeyPlaceholder": "API-Schlüssel eingeben", "addButton": "Werkzeug hinzufügen", "closeButton": "Schließen", "customNamePlaceholder": "Benutzerdefinierten Namen eingeben (optional)", @@ -1567,6 +1899,24 @@ "learnMore": "Mehr erfahren", "resetKey": "Schlüssel zurücksetzen", "resetKeyConfirm": "Möchten Sie den API-Schlüssel wirklich zurücksetzen? Der aktuelle Schlüssel funktioniert sofort nicht mehr und diese Aktion kann nicht rückgängig gemacht werden.", + "apiWrites": { + "title": "Änderungen anderer in deinem Namen", + "description": "Über den API-Schlüssel, das Widget oder den öffentlichen Link des Agenten kann niemand eine Aktion für dich bestätigen. Änderungen mit deinen verbundenen Konten und gespeicherten Zugangsdaten sind daher blockiert, außer du erlaubst sie hier.", + "saveFailed": "Speichern fehlgeschlagen. Versuche es erneut.", + "summaryNone": "Noch nichts erlaubt", + "summaryTools_one": "{{tools}} kann Änderungen vornehmen", + "summaryTools_other": "{{tools}} können Änderungen vornehmen", + "summaryCount": "{{allowed}} von {{formatted}} erlaubt", + "toolCount": "Änderungen über API, Widget und öffentliche Links · {{allowed}} von {{formatted}} erlaubt", + "toolLabel": "Änderungen über API, Widget und öffentliche Links für {{tool}} erlauben", + "choice": { + "off": "Aus", + "all": "Alle" + }, + "customize_one": "Änderung einzeln festlegen", + "customize_other": "Alle {{formatted}} einzeln festlegen", + "fold": "Änderungen ausblenden" + }, "actionFailed": "Das hat nicht funktioniert. Bitte versuche es erneut.", "apiKeyAfterPublish": "Veröffentliche den Agenten, um seinen API-Schlüssel zu erstellen." }, @@ -1661,14 +2011,18 @@ }, "sources": { "title": "Quellen", - "text": "Wähle deine Quellen", + "text": "Wissen auswählen", "link": "Quellen-Link", "view_more_one": "{{count}} weitere Quelle", "view_more_other": "{{count}} weitere Quellen", - "noSourcesAvailable": "Keine Quellen verfügbar", - "selectedCount": "{{count}} Quellen ausgewählt", - "groupOwn": "Deine Quellen", - "groupTeam": "Mit dem Team geteilt" + "noSourcesAvailable": "Noch kein Wissen", + "selectedCount": "{{count}} ausgewählt", + "groupOwn": "Deine", + "groupTeam": "Mit dem Team geteilt", + "connectMore": "Weitere verbinden", + "fromConnector": "Aus {{name}}", + "fromConnectorTitle": "Aus {{name}}: {{title}}", + "knowledge": "Wissen" }, "attachments": { "attach": "Anhängen", @@ -1709,7 +2063,9 @@ "accessingMemory": "Greife auf das Gedächtnis zu…", "runningCode": "Führe Code aus…", "creatingArtifact": "Erstelle ein Artefakt…", - "usingTool": "Verwende {{tool}}…" + "usingTool": "Verwende {{tool}}…", + "searchingConnector": "Durchsuche {{name}}...", + "usingConnector": "Nutze {{name}}..." }, "toolChip": { "searchingWeb": "Im Web gesucht · „{{query}}“", @@ -1720,7 +2076,10 @@ "accessingMemory": "Auf Gedächtnis zugegriffen", "runningCode": "Code ausgeführt", "creatingArtifact": "Artefakt erstellt", - "usingTool": "{{tool}} verwendet" + "usingTool": "{{tool}} verwendet", + "searchedConnector": "{{name}} durchsucht", + "readConnector": "Aus {{name}} gelesen", + "usedConnector": "{{name}} genutzt: {{action}}" }, "inlineSteps": { "arguments": "Argumente", @@ -1770,7 +2129,12 @@ "approveAlways": "Genehmigen, nicht mehr fragen", "deny": "Ablehnen", "details": "Details", - "denyReasonPlaceholder": "Optionaler Grund für die Ablehnung…" + "denyReasonPlaceholder": "Optionaler Grund für die Ablehnung…", + "connectTitle": "Dieses Werkzeug braucht dein {{name}}-Konto.", + "connect": "{{name}} verbinden", + "continue": "Weiter", + "skip": "Überspringen", + "thisService": "diesen Dienst" }, "research": { "title": "Recherche", @@ -1786,7 +2150,6 @@ "agents": { "title": "Agenten", "edit": "Bearbeiten", - "view": "Ansehen", "card": { "pin": "Agent anheften", "unpin": "Agent lösen", @@ -1880,7 +2243,7 @@ "placeholders": { "agentName": "Agentenname", "describeAgent": "Beschreibe deinen Agenten", - "selectSources": "Quellen auswählen", + "selectSources": "Wissen auswählen", "chunksPerQuery": "Chunks pro Anfrage", "selectType": "Typ auswählen", "selectTools": "Werkzeuge auswählen", @@ -1890,20 +2253,23 @@ "enterRequestLimit": "Anfrage-Limit eingeben" }, "sourcePopup": { - "title": "Quellen auswählen", - "searchPlaceholder": "Quellen suchen...", - "noOptionsMessage": "Keine Quellen verfügbar", + "title": "Wissen auswählen", + "searchPlaceholder": "Wissen durchsuchen...", + "noOptionsMessage": "Noch kein Wissen", "groupOwn": "Deine Quellen", "groupTeam": "Mit dem Team geteilt", - "noSourceHint": "Optional. Ohne Quelle antwortet der Agent nur anhand des Modells und seiner Werkzeuge." + "noSourceHint": "Optional. Ohne Wissen antwortet der Agent nur mit dem Modell und seinen Werkzeugen." }, "toolsPopup": { "title": "Werkzeuge auswählen", "searchPlaceholder": "Werkzeuge suchen...", "noOptionsMessage": "Keine Werkzeuge verfügbar", "groupBuiltin": "Integriert", - "groupDefault": "Standard", - "groupCustom": "Benutzerdefiniert" + "groupDefault": "Integriert", + "groupCustom": "Benutzerdefiniert", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "An diesem Agenten", + "attachedHint": "Von jemand anderem hinzugefügt. Du kannst es entfernen." }, "modelsPopup": { "title": "Modelle auswählen", @@ -1923,7 +2289,7 @@ "defaultModel": "Standardmodell", "name": "Name", "description": "Beschreibung", - "sources": "Quellen", + "sources": "Wissen", "avatar": "Avatar" }, "advanced": { @@ -2012,11 +2378,69 @@ "published": "Veröffentlicht" }, "sponsors": { - "attachNote": "Tools, Quellen und Prompts, die du aus deiner eigenen Bibliothek hinzufügst, laufen für alle, die diesen Agenten nutzen, mit deinem Zugriff.", + "attachNote": "Tools, Quellen und Prompts, die du hinzufügst und die der Eigentümer nicht nutzen kann, laufen für alle, die diesen Agenten nutzen, mit deinem Zugriff. Du musst sie besitzen oder bearbeiten können und wirst um Bestätigung gebeten.", "publicLinkNote": "Dieser Agent hat einen öffentlichen Link, daher kann jeder mit dem Link Antworten daraus erhalten.", "addedBy": "Hinzugefügt von {{person}}: {{names}}", - "unavailable": "Nicht aktiv: {{names}}. {{person}} hat sie hinzugefügt und hat keinen Zugriff mehr. Entferne sie oder wähle welche, die der Eigentümer nutzen kann.", - "unknownItem": "Unbenanntes Element" + "addedByOther": "Hinzugefügt von einer anderen Person: {{names}}", + "unknownItem": "Unbenanntes Element", + "notAllowed": "Du kannst {{names}} diesem Agenten nicht hinzufügen. Der Eigentümer kann sie nicht nutzen, und du kannst nur teilen, was du besitzt oder bearbeiten kannst.", + "takeOver": "{{name}} mit meinem Zugriff ausführen", + "takeOverPending": "{{name}} läuft mit deinem Zugriff, sobald du speicherst.", + "undoTakeOver": "Rückgängig", + "confirmationOutdated": "Dieser Agent hat sich geändert, seit du bestätigt hast, was mit deinem Zugriff läuft. Prüfe die Hinweise und speichere erneut." + }, + "resourceStates": { + "title": "Einige Tools, Quellen oder Prompts laufen nicht", + "reason": { + "deleted": "{{name}} wurde gelöscht.", + "ownerLostAccessYou": "Du kannst {{name}} nicht mehr verwenden.", + "ownerLostAccess": "Der Eigentümer des Agenten kann {{name}} nicht mehr verwenden.", + "sponsorCannotEditAgent": "{{name}} lief mit dem Zugriff von {{person}}, und diese Person kann den Agenten nicht mehr bearbeiten.", + "sponsorCannotEditAgentOther": "{{name}} lief mit dem Zugriff einer anderen Person, und diese Person kann den Agenten nicht mehr bearbeiten.", + "sponsorCannotEditItem": "{{name}} lief mit dem Zugriff von {{person}}, und diese Person kann es nicht mehr bearbeiten.", + "sponsorCannotEditItemOther": "{{name}} lief mit dem Zugriff einer anderen Person, und diese Person kann es nicht mehr bearbeiten.", + "connectionNeedsReconnect": "Für {{name}} muss das {{service}}-Konto erneut angemeldet werden.", + "connectionRemoved": "Das {{service}}-Konto, das {{name}} verwendet hat, wurde entfernt.", + "connectorDisabled": "Ein Admin hat {{service}} deaktiviert, daher kann {{name}} nicht laufen.", + "unknown": "{{name}} läuft nicht." + }, + "ask": { + "shareAgain": "Bitte {{person}}, es erneut zu teilen.", + "signInAgain": "Bitte {{person}}, sich erneut anzumelden.", + "connectAgain": "Bitte {{person}}, das Konto erneut zu verbinden.", + "admin": "Bitte einen Admin, es wieder zu aktivieren.", + "shareAgainOwner": "Bitte den Eigentümer, es erneut zu teilen.", + "signInAgainOwner": "Bitte den Eigentümer, sich erneut anzumelden.", + "connectAgainOwner": "Bitte den Eigentümer, das Konto erneut zu verbinden." + }, + "promptFallback": "Bis dahin verwendet der Agent den Standard-Prompt.", + "serviceFallback": "Dienst", + "remove": "Entfernen", + "removeLabel": "{{name}} entfernen", + "reconnectLabel": "{{name}} neu verbinden", + "unnamed": { + "tool": "Tool {{id}}", + "source": "Quelle {{id}}", + "prompt": "Prompt {{id}}" + }, + "chip": "Läuft nicht: {{count}}" + }, + "sponsorConfirm": { + "title": "Mit deinem Zugriff ausführen?", + "description": "Der Eigentümer des Agenten kann diese nicht nutzen. Wenn du sie hinzufügst, laufen sie für alle, die diesen Agenten nutzen, mit deinem Zugriff.", + "audienceTitle": "Wer sie über diesen Agenten nutzen kann", + "audienceTeams": "Mitglieder von {{teams}}", + "audienceApiKey": "Alle mit dem API-Schlüssel des Agenten, auch über sein Website-Widget", + "audiencePublicLink": "Alle mit dem öffentlichen Link des Agenten", + "audienceWebhook": "Alles, was den Webhook des Agenten aufruft", + "audienceEditors": "Alle, die diesen Agenten öffnen können", + "stopNote": "Sie laufen nicht mehr, wenn du den Bearbeitungszugriff auf sie oder auf diesen Agenten verlierst. Dann bleiben sie angehängt, tun aber nichts, bis jemand, der sie bearbeiten kann, sie übernimmt oder sie entfernt werden.", + "confirm": "Mit meinem Zugriff hinzufügen", + "types": { + "tool": "Tool", + "source": "Quelle", + "prompt": "Prompt" + } } }, "logs": { @@ -2308,7 +2732,7 @@ "outputVariable": "Ausgabevariable", "outputVariablePlaceholder": "Variablenname für die Ausgabe", "streamToUser": "Ausgabe an Nutzer streamen", - "sources": "Quellen", + "sources": "Wissen", "documents": "Dokumente", "documentsHint": "Dokumente, die diesem Agenten aus Uploads oder vorgelagerten Knoten übergeben werden.", "filePassing": "Dateiübergabe", @@ -2347,7 +2771,8 @@ "detailsTitle": "Workflow-Details", "detailsDescription": "Name, Beschreibung und Bild, die auf der Agentenkarte angezeigt werden.", "unsavedChanges": "Nicht gespeicherte Änderungen", - "editDetailsMenu": "Details bearbeiten" + "editDetailsMenu": "Details bearbeiten", + "attachedOption": "{{name}} (von jemand anderem hinzugefügt)" }, "validation": { "nameRequired": "Workflow-Name ist erforderlich", @@ -2453,6 +2878,7 @@ "workflowTitle": "Workflow-Agent", "workflowDescription": "Entwirf komplexe mehrstufige Workflows mit verschiedenen Modellen, bedingter Logik und Zustandsverwaltung" }, + "view": "Ansehen", "deleteFailed": "Der Agent konnte nicht gelöscht werden. Bitte versuche es erneut." }, "components": { @@ -2521,7 +2947,8 @@ "shared": "Geteilt", "connectTo": "Mit {{provider}} verbinden", "sessionExpiredFor": "Sitzung abgelaufen. Bitte verbinde dich erneut mit {{provider}}.", - "selectFilesFrom": "Dateien aus {{provider}} auswählen" + "selectFilesFrom": "Dateien aus {{provider}} auswählen", + "account": "Konto" }, "actionButtons": { "openNewChat": "Neuen Chat öffnen", @@ -2579,5 +3006,15 @@ }, "prompts": { "textAriaLabel": "Prompt-Text" + }, + "connectHero": { + "title": "Daten verbinden", + "body": "Chatte mit {{names}} und mehr." + }, + "common": { + "close": "Schließen", + "viewOnlyNotice": "Du kannst dies ansehen, aber deine Rolle kann es nicht ändern.", + "credentialsLockedNotice": "Nur der Eigentümer kann die Zugangsdaten dieses Tools ändern.", + "savedSecretHint": "Gespeichert. Leer lassen, um es zu behalten." } } diff --git a/frontend/src/locale/en.json b/frontend/src/locale/en.json index 3968ff98..20dc1c3f 100644 --- a/frontend/src/locale/en.json +++ b/frontend/src/locale/en.json @@ -10,12 +10,6 @@ "loading": "Loading...", "retry": "Retry", "cancel": "Cancel", - "common": { - "close": "Close", - "viewOnlyNotice": "You can view this, but your role can't change it.", - "credentialsLockedNotice": "Only the owner can change this tool's credentials.", - "savedSecretHint": "Saved. Leave empty to keep it." - }, "errorBoundary": { "message": "Something went wrong displaying this content.", "tryAgain": "Try again" @@ -110,16 +104,16 @@ }, "sources": { "title": "Here you can manage all of the source file that are available to you and those you have uploaded.", - "subtitle": "Upload and manage the documents and knowledge sources used to power your responses", - "label": "Sources", + "subtitle": "Content DocsGPT can search and cite: uploads, websites, wikis and content synced from your connections.", + "label": "Knowledge", "name": "Source Name", "date": "Vector Date", "type": "Type", "tokenUsage": "Token Usage", - "noData": "No existing Sources", + "noData": "No knowledge yet", "searchPlaceholder": "Search...", "addNew": "Add New", - "addSource": "Add Source", + "addSource": "Add knowledge", "addChunk": "Add Chunk", "preLoaded": "Pre-loaded", "private": "Private", @@ -190,7 +184,17 @@ "pagesTitle": "Pages", "loadFailed": "Could not load the wiki pages.", "pageLoadFailed": "Could not load this page.", - "editTitle": "Edit page" + "editTitle": "Edit page", + "settings": { + "action": "Wiki settings", + "title": "Wiki settings", + "outsideEdits": { + "label": "Let API and widget users edit this wiki", + "description": "When this is off, people who use an agent through its API key or widget can read this wiki but not change it. Public-link visitors can only edit wikis they can edit themselves, and are asked before each edit. You and the wiki's editors can always edit it." + }, + "loadError": "Could not load the wiki settings.", + "saveError": "Could not save the change. Please try again." + } }, "graphrag": { "badge": "GraphRAG", @@ -308,7 +312,7 @@ "chunks": "Chunks", "noChunks": "No chunks found", "noChunksAlt": "No chunks found", - "goToSources": "Go to Sources", + "goToSources": "Go to Knowledge", "uploadNew": "Upload new", "noResults": "No results found", "fileName": "Name", @@ -451,6 +455,9 @@ "editChunkDescription": "{{file}} · chunk {{n}} · {{tokens}} tokens", "previousChunk": "Previous chunk", "nextChunk": "Next chunk", + "paused": "Sync paused. Reconnect {{name}} to resume.", + "emptyHint": "Upload files, add a website, or connect a service to sync its content.", + "connectService": "Connect a service", "viewConfig": "View source settings", "errors": { "forbidden": "You don't have permission to do that on this source.", @@ -682,10 +689,6 @@ "createTeamError": "Could not create team.", "noTeams": "No teams yet.", "noDescription": "No description", - "memberCount_one": "{{formatted}} member", - "memberCount_other": "{{formatted}} members", - "sharedCount_one": "{{formatted}} shared", - "sharedCount_other": "{{formatted}} shared", "loadError": "Failed to load teams.", "roleAdmin": "admin", "roleMember": "member", @@ -765,8 +768,46 @@ "teams": "Teams", "people": "People", "editors": "Editors" + }, + "uses": { + "title": "What this agent uses", + "intro": "Everyone who uses this agent gets these, each with the access shown.", + "stopped": "Stopped", + "writesOff": "Not via API", + "writesNote": "“Allow” on a connector applies to chats. Changes marked “Not via API” stay off for API, widget and public-link users until you allow them in Access Details.", + "writesNoteEditor": "“Allow” on a connector applies to chats. Changes marked “Not via API” stay off for API, widget and public-link users until the owner allows them in Access Details.", + "access": { + "you": "Your access", + "person": "{{person}}'s access", + "other": "Someone else's access", + "owner": "The owner's access", + "member": "Each person's own {{service}} account (API and widget: yours)", + "memberNoService": "Each person's own account (API and widget: yours)", + "yourAccount": "Your {{service}} account", + "yourAccountNoService": "Your account", + "personAccount": "{{person}}'s {{service}} account", + "personAccountNoService": "{{person}}'s account", + "memberShared": "Each person's own {{service}} account (API and widget: the owner's)", + "memberSharedNoService": "Each person's own account (API and widget: the owner's)", + "otherAccount": "Someone else's {{service}} account", + "otherAccountNoService": "Someone else's account", + "yourCredentials": "Your saved credentials", + "personCredentials": "{{person}}'s saved credentials", + "otherCredentials": "Someone else's saved credentials" + }, + "writesSomeOff": "Not all via API", + "adminOff": "Changes off by an admin", + "writesNoteApi": "“Allow” on a connector applies to chats. Changes marked “Not via API” stay off for API and widget users until you allow them in Access Details.", + "writesNoteApiEditor": "“Allow” on a connector applies to chats. Changes marked “Not via API” stay off for API and widget users until the owner allows them in Access Details.", + "writesNoteMemberTail": "On tools where each person uses their own account, public-link users make changes with their own account.", + "openAccessDetails": "Open Access Details", + "accessDetailsNeedsKey": "Access Details lists these changes once the agent has an API key." } }, + "memberCount_one": "{{formatted}} member", + "memberCount_other": "{{formatted}} members", + "sharedCount_one": "{{formatted}} shared", + "sharedCount_other": "{{formatted}} shared", "accessChangeError": "Could not change access.", "accessSettings": { "title": "Access settings", @@ -814,7 +855,8 @@ }, "viewers_can_use_in_agents": { "label": "Viewers can use it in their own agents", - "description": "It runs with your credentials." + "description": "It runs with your credentials.", + "descriptionMember": "Each person uses their own account." } }, "prompt": { @@ -936,11 +978,11 @@ }, "tools": { "label": "Tools", - "subtitle": "Browse, connect, and configure the tools and integrations that power your agents", + "subtitle": "Actions your agents can take. Built-in tools live here. Tools from a connected service are grouped under that service.", "searchPlaceholder": "Search tools...", "addTool": "Add Tool", "noToolsFound": "No tools found", - "selectToolSetup": "Select a tool to set up", + "selectToolSetup": "Add a tool", "settingsIconAlt": "Settings icon", "configureToolAria": "Configure {{toolName}}", "toggleToolAria": "Toggle {{toolName}}", @@ -952,8 +994,8 @@ "builtIn": "Built-in", "authStatus": { "connected": "Connected", - "needsAuth": "Needs Auth", - "configured": "Configured" + "needsAuth": "Reconnect", + "configured": "Connected" }, "deleteWarning": "Are you sure you want to delete the tool \"{{toolName}}\"?", "unsavedChanges": "You have unsaved changes that will be lost if you leave without saving.", @@ -1069,6 +1111,11 @@ }, "headerValuePlaceholder": "e.g., application/json", "toolIconTitle": "{{name}} icon", + "groupBuiltIn": "Built in", + "groupService": "From a service", + "browseConnectors": "Browse all connectors", + "viaConnection": "via {{name}}", + "off": "Off", "view": "View", "inMyChats": "In my chats", "useInMyChatsAria": "Use {{toolName}} in my chats", @@ -1304,13 +1351,320 @@ "subtitle": "The new secret for \"{{name}}\" is ready. The old one no longer works." } } + }, + "connectors": { + "label": "Connectors", + "subtitle": "Connect DocsGPT to the services your team uses. A connection can sync content into Knowledge and give agents tools.", + "search": "Search connectors...", + "addCustom": "Add custom tool", + "custom": { + "custom_mcp": "MCP server", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "Filter connectors by category", + "categories": { + "all": "All", + "connected": "Connected", + "files": "Files & storage", + "knowledge": "Docs & wikis", + "dev": "Developer tools", + "messaging": "Messaging", + "database": "Databases", + "search": "Search & web", + "custom": "Custom", + "projects": "Projects & issues", + "business": "Payments & business" + }, + "capability": { + "sync": "Sync", + "read": "Read", + "write": "Write" + }, + "status": { + "connect": "Connect", + "connected": "Connected", + "connectedCount": "{{formatted}} connected", + "reconnect": "Reconnect", + "needsAdminSetup": "Needs admin setup", + "disabledByAdmin": "Disabled by admin", + "custom": "Custom" + }, + "connectionStatus": { + "connected": "Connected", + "reconnect_needed": "Reconnect needed", + "disconnected": "Disconnected", + "error": "Error", + "pending": "Waiting for sign-in" + }, + "publisher": { + "built_in": "Built in", + "preset": "Preset", + "custom": "Custom" + }, + "descriptions": { + "google_drive": "Sync Docs, Sheets and PDFs into Knowledge.", + "share_point": "Sync files from SharePoint sites and OneDrive into Knowledge.", + "confluence": "Sync Confluence spaces and pages into Knowledge.", + "s3": "Sync documents from an S3 bucket into Knowledge.", + "reddit": "Sync Reddit posts that match your searches into Knowledge.", + "brave": "Search the web and images with the Brave Search API.", + "telegram": "Send messages and images to a Telegram chat.", + "ntfy": "Send push notifications through an ntfy server.", + "postgres": "Read the schema and run SQL against a Postgres database.", + "custom_mcp": "Connect any remote Model Context Protocol server.", + "custom_openapi": "Import an OpenAPI spec and call its endpoints as tools.", + "mcp_notion": "Search, read and update Notion pages and databases.", + "mcp_linear": "Sync issues and documents into Knowledge, and let agents find, create and update issues.", + "mcp_atlassian": "Search and update Jira issues and Confluence pages.", + "mcp_sentry": "Look up Sentry issues, events and releases.", + "mcp_asana": "Find and update Asana tasks and projects.", + "mcp_stripe": "Look up customers, payments and subscriptions in Stripe.", + "github": "Sync repositories into Knowledge and let agents read code, issues and pull requests." + }, + "empty": "Nothing connected yet. Connect a service to sync its content or give agents new tools.", + "noMatches": "No connectors match your search.", + "loadFailed": "Could not load connectors.", + "askAdmin": "Ask an admin to set this up.", + "setupSettings": "Set these server settings, then restart DocsGPT:", + "setupGuide": "Setup guide", + "unverified": "This server is not reviewed by DocsGPT. Only connect servers you trust.", + "detail": { + "accounts": "Accounts", + "connectedAs": "Connected as {{account}}", + "connectAnother": "Connect another account", + "sources": "Knowledge from this connection", + "tools": "Tools from this connection", + "noAccounts": "No account connected yet.", + "failed": "Could not load this connection.", + "lastSync": "Synced {{date}} · {{frequency}}", + "paused": "Sync paused", + "toolOn": "On", + "toolOff": "Off", + "accountMenu": "Account actions", + "disconnect": "Disconnect", + "keyEnding": "Key ending {{hint}}", + "syncMore": "Sync more content", + "refreshTools": "Refresh tools", + "noSources": "Nothing synced from this connection yet.", + "remove": "Remove", + "refreshed": "Tools refreshed: {{added}} added, {{removed}} removed.", + "refreshFailed": "Could not refresh the tools.", + "syncNow": "Sync now", + "syncStarted": "Syncing {{name}}…", + "syncFailed": "Could not start the sync. Try again.", + "toolSwitchFailed": "Could not change the tool. Try again.", + "account": "Account: {{account}}", + "expired": "The sign-in expired. Reconnect to keep using this connection.", + "broken": "This connection stopped working. Reconnect to try again.", + "addTools": "Add tools", + "noTools": "No tools from this connection yet.", + "addToolsFailed": "Could not add the tools. Try again.", + "rename": "Rename", + "accountPicker": "Choose an account" + }, + "permission": { + "always": "Allow", + "ask": "Ask first", + "off": "Off", + "label": "Permission for {{action}}", + "saveFailed": "Could not save the permission. Try again.", + "groupLabel": "Permission for all of {{group}}", + "fold": "Hide actions", + "hint": "“Allow” and “Ask first” apply in chat; with “Ask first”, DocsGPT asks you before running the action. Changes through an agent's API, widget or public link are allowed per agent, in its Access Details.", + "actionCount_one": "{{formatted}} action", + "actionCount_other": "{{formatted}} actions", + "customize_one": "Customize the action", + "customize_other": "Customize each of {{formatted}}" + }, + "disconnect": { + "title": "Disconnect {{name}}?", + "body_zero": "Tools from this connection stop working until you reconnect.", + "body_one": "The item it synced stays in Knowledge but stops updating. Tools from this connection stop working until you reconnect.", + "body_other": "The {{formatted}} items it synced stay in Knowledge but stop updating. Tools from this connection stop working until you reconnect.", + "failed": "Could not disconnect. Try again." + }, + "fields": { + "token": "API key", + "aws_access_key_id": "Access key ID", + "aws_secret_access_key": "Secret access key", + "region": "Region", + "endpoint_url": "Custom endpoint URL", + "client_id": "Client ID", + "client_secret": "Client secret", + "user_agent": "User agent", + "bucket": "Bucket", + "prefix": "Path prefix", + "search_queries": "Search queries", + "number_posts": "Number of posts", + "telegram_token": "Bot token", + "ntfy_token": "Access token", + "postgres_token": "Connection string", + "brave_token": "API key", + "github_access_token": "Personal access token", + "repo_url": "Repository", + "telegram_chat_id": "Default chat ID" + }, + "wizard": { + "connectTitle": "Connect {{name}}", + "reconnectTitle": "Reconnect {{name}}", + "signIn": "Sign in to {{name}}", + "chooseWhatToSync": "Choose what to sync", + "skip": "Skip for now", + "syncFrequency": "Sync", + "doneTitle": "{{name}} is connected", + "doneSummary": "Connected as {{account}}. {{sources}} syncing into Knowledge, {{tools}} ready.", + "doneSummaryNone": "Connected as {{account}}.", + "sourcesCount_one": "{{formatted}} item", + "sourcesCount_other": "{{formatted}} items", + "toolsCount_one": "{{formatted}} tool", + "toolsCount_other": "{{formatted}} tools", + "toolsHeading": "Tools ({{formatted}})", + "tryInChat": "Try it in chat", + "done": "Done", + "connectFailed": "Could not connect. Check the details and try again.", + "syncFailed": "Could not start the sync. Try again.", + "connectedWithKey": "Connected with the key ending {{hint}}.", + "doneCounts": "{{sources}} syncing into Knowledge, {{tools}} ready.", + "doneSources": "{{sources}} syncing into Knowledge.", + "doneTools": "{{tools}} ready.", + "waiting": "Finish signing in to {{name}} in the window that opened.", + "openSignIn": "Open the {{name}} sign-in", + "chooseWhatToSetUp": "Choose what to set up", + "continue": "Continue", + "methodLabel": "How to connect", + "methodOauth": "Sign in with {{name}}", + "methodToken": "Use an access token", + "credentialsRejected": "{{name}} did not accept these credentials. Check them and try again.", + "toolsUnavailable": "{{name}}'s tools could not be reached. Try again, or turn them off to continue.", + "addTools": "Let agents use {{name}}", + "addToolsDescription": { + "read": "Agents can look things up. They cannot change anything.", + "readWrite": "Agents can look things up and make changes. Changes need your approval." + }, + "accountName": "Name this account", + "accountNameHint": "Optional. Tells your accounts apart, for you and the AI, e.g. Alerts bot.", + "syncToKnowledge": "Sync into Knowledge", + "syncToKnowledgeDescription": "Pick content from {{name}} for answers to search. Leave it off to only connect; you can sync later from Connectors.", + "syncLater": "Nothing syncs into Knowledge yet. To add content, open {{name}} in Connectors and choose Sync more content.", + "retrievalSettings": "Advanced retrieval settings" + }, + "remove": { + "title": "Remove {{name}}?", + "description": "The stored sign-in is deleted. Choose what happens to what this connection feeds.", + "sourcesLabel_one": "The item it synced into Knowledge", + "sourcesLabel_other": "The {{formatted}} items it synced into Knowledge", + "keepSources": "Keep content (stop syncing)", + "deleteSources": "Delete", + "toolsLabel_one": "Its tool", + "toolsLabel_other": "Its {{formatted}} tools", + "deleteTools": "Delete", + "keepTools": "Keep, turned off", + "failed": "Could not remove the connection. Try again." + }, + "health": { + "reconnect": "Reconnect {{name}} to keep syncing.", + "reconnectSources_one": "Reconnect {{name}}: {{formatted}} item stopped syncing into Knowledge", + "reconnectSources_other": "Reconnect {{name}}: {{formatted}} items stopped syncing into Knowledge", + "reconnectTools": "Reconnect {{name}} so agents can use it again", + "reconnectBoth_one": "Reconnect {{name}}: {{formatted}} item stopped syncing and agents can't use it", + "reconnectBoth_other": "Reconnect {{name}}: {{formatted}} items stopped syncing and agents can't use it", + "navDot": "A connection needs attention", + "signInAgain": "Sign in again", + "pickerNotice": "{{name}} needs you to sign in again." + }, + "error": { + "decrypt": "Stored credentials could not be decrypted. Reconnect to continue.", + "defaultKey": "An admin must set ENCRYPTION_SECRET_KEY before services can be connected." + }, + "sharing": { + "owner": "Everyone uses your account", + "member": "Each person connects their own account", + "ownerShort": "Your account", + "memberShort": "Each person's own", + "ownerShortShared": "The owner's account" + }, + "share": { + "heading": "Whose account it runs with", + "ownerWarning": "Team members will act as {{account}} on {{name}}.", + "ownerWarningShared": "Team members will act as the owner's {{name}} account.", + "confirmWrite": "I understand teammates can take actions with my account.", + "confirmWriteShared": "I understand teammates can take actions with the owner's {{name}} account.", + "memberNote": "Each person uses their own {{name}} account, and connects it the first time they use this tool. Through an agent's API key or widget, it uses the agent owner's account.", + "forced": "An admin chose this for every share of this connector.", + "ownerChooses": "Only the owner can change this.", + "saveFailed": "Could not save the change. Try again." + }, + "createFailed": "Could not add the connector. Try again.", + "capabilityFilter": { + "sync": "Showing connectors that sync content into Knowledge.", + "tools": "Showing connectors that give agents tools.", + "showAll": "Show all" + }, + "capabilityPlain": { + "sync": "Syncs into Knowledge", + "read": "Looks things up", + "write": "Takes actions" + }, + "github": { + "tokenHint": "Create a fine-grained token with read access to Contents and Metadata on the repositories you want to use. For agents to make changes, also give it read and write access to Issues and Pull requests.", + "writes": "Also let agents make changes (issues, comments, pull requests)", + "writesDescription": "Changes ask you first by default. Your token, or the GitHub App, needs read and write access to Issues and Pull requests.", + "writesForbidden": "An admin turned off changes through GitHub.", + "writesFailed": "Could not change what agents may do on GitHub.", + "createToken": "Create a token on GitHub", + "chooseRepositories": "Choose repositories on GitHub", + "noAppRepositories": "DocsGPT cannot see any repositories yet. Choose which ones it may read on GitHub.", + "noTokenRepositories": "This token cannot read any repositories. Give it access on GitHub, then reconnect.", + "searchRepositories": "Search repositories", + "repositories": "Repositories", + "updated": "Updated {{date}}", + "loadFailed": "Could not load your repositories." + }, + "linear": { + "teams": "Teams", + "teamsHint": "Sync the issues of these teams.", + "projects": "Projects", + "projectsHint": "Sync the issues of these projects.", + "search": "Search teams and projects", + "noMatches": "No teams or projects match your search.", + "includeComments": "Include comments", + "includeCommentsDescription": "Add each issue's comments to it.", + "includeDocuments": "Include project documents", + "includeDocumentsDescription": "Sync the Linear documents of the projects you pick.", + "loadFailed": "Could not load your Linear teams and projects.", + "empty": "This Linear account has no teams or projects to sync." + }, + "parameters": { + "show": "Parameters", + "hide": "Hide parameters", + "fixedCount_one": "{{formatted}} fixed", + "fixedCount_other": "{{formatted}} fixed", + "ai": "Let AI decide", + "fixed": "Always use", + "choiceLabel": "How {{parameter}} is filled in", + "valueLabel": "Value for {{parameter}}", + "save": "Save", + "saveFailed": "Could not save the value. Try again.", + "hint": "A value set to Always use is sent on every call. The AI never sees it and cannot change it.", + "fromAccount": "Set on this account: {{value}}" + }, + "fieldHints": { + "telegram_chat_id": "Optional. Messages go to this chat, and the AI cannot pick another. Add the bot to the chat, send it a message, then find the chat's id in https://api.telegram.org/bot/getUpdates." + }, + "rename": { + "title": "Rename account", + "description": "The name tells this account apart from your other accounts of the same service, for you and the AI. Leave it empty to show the account itself.", + "label": "Account name", + "save": "Save", + "failed": "Could not rename the account. Try again." + } } }, "modals": { "uploadDoc": { - "label": "Upload new document", + "label": "Add knowledge", "select": "Choose how to upload your document to DocsGPT", - "selectSource": "Select the way to add your source", + "selectSource": "Add knowledge", "selectedFiles": "Selected Files", "noFilesSelected": "No files selected", "file": "Upload from device", @@ -1325,19 +1679,11 @@ "filesRejected": "Not added (over 25 MB or not a supported type): {{files}}", "uploadedFiles": "Uploaded Files", "cancel": "Cancel", - "train": "Train", + "train": "Add to Knowledge", "create": "Create", "link": "Link", "urlLink": "URL Link", "repoUrl": "Repository URL", - "reddit": { - "id": "Client ID", - "secret": "Client Secret", - "agent": "User agent", - "searchQueries": "Search queries", - "numberOfPosts": "Number of posts", - "addQuery": "Add Query" - }, "drag": { "title": "Drop attachments here", "description": "Release to upload your attachments" @@ -1359,8 +1705,6 @@ "clear": "Clear", "ingestionFailed": "Ingestion failed." }, - "showAdvanced": "Show advanced options", - "hideAdvanced": "Hide advanced options", "ingestors": { "local_file": { "label": "Upload File", @@ -1378,26 +1722,6 @@ "label": "GitHub", "heading": "Add content from GitHub" }, - "reddit": { - "label": "Reddit", - "heading": "Add content from Reddit" - }, - "google_drive": { - "label": "Google Drive", - "heading": "Upload from Google Drive" - }, - "s3": { - "label": "Amazon S3", - "heading": "Add content from Amazon S3" - }, - "share_point": { - "label": "SharePoint", - "heading": "Upload from SharePoint" - }, - "confluence": { - "label": "Confluence", - "heading": "Upload from Confluence" - }, "wiki": { "label": "New wiki", "heading": "Create a living wiki" @@ -1451,13 +1775,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "AWS Access Key ID", - "awsSecretAccessKey": "AWS Secret Access Key", - "bucket": "Bucket Name", - "prefix": "Path Prefix (optional)", - "region": "AWS Region", - "endpointUrl": "Custom Endpoint URL (optional)", "initialContent": "Initial content (optional)" + }, + "github": { + "privateHint": "Private repository? Connect your GitHub account to sync it.", + "connectedHint": "Private repositories sync from your GitHub connection ({{account}}).", + "connect": "Connect GitHub", + "pickRepository": "Pick a repository" + }, + "connectData": { + "title": "Connect your data", + "description": "Sync {{services}}", + "more": "more", + "intro": "Pick a service, sign in, then choose what to sync into Knowledge.", + "connectedAs": "Connected as {{account}}", + "browseAll": "Browse all connectors" } }, "createAPIKey": { @@ -1494,8 +1826,8 @@ "configTool": { "title": "Tool Config", "type": "Type", - "apiKeyLabel": "API Key / OAuth", - "apiKeyPlaceholder": "Enter API Key / OAuth", + "apiKeyLabel": "API key", + "apiKeyPlaceholder": "Enter API key", "addButton": "Add Tool", "closeButton": "Close", "customNamePlaceholder": "Enter custom name (optional)", @@ -1573,6 +1905,24 @@ "learnMore": "Learn more", "resetKey": "Reset key", "resetKeyConfirm": "Are you sure you want to reset the API key? The current key will stop working immediately and this action cannot be undone.", + "apiWrites": { + "title": "Changes others can make as you", + "description": "Nobody can approve an action for you through the API key, the widget or the agent's public link, so changes made with your connected accounts and saved credentials are blocked unless you allow them here.", + "saveFailed": "Could not save. Try again.", + "summaryNone": "None allowed yet", + "summaryTools_one": "{{tools}} can make changes", + "summaryTools_other": "{{tools}} can make changes", + "summaryCount": "{{allowed}} of {{formatted}} allowed", + "toolCount": "Changes from API, widget and public links · {{allowed}} of {{formatted}} allowed", + "toolLabel": "Allow changes from API, widget and public links on {{tool}}", + "choice": { + "off": "Off", + "all": "All" + }, + "customize_one": "Customize the change", + "customize_other": "Customize each of {{formatted}}", + "fold": "Hide changes" + }, "actionFailed": "That didn't work. Please try again.", "apiKeyAfterPublish": "Publish the agent to create its API key." }, @@ -1667,14 +2017,18 @@ }, "sources": { "title": "Sources", - "text": "Choose Your Sources", + "text": "Choose knowledge", "link": "Source link", "view_more_one": "{{count}} more source", "view_more_other": "{{count}} more sources", - "noSourcesAvailable": "No sources available", - "selectedCount": "{{count}} sources selected", - "groupOwn": "Your sources", - "groupTeam": "Shared with team" + "noSourcesAvailable": "No knowledge yet", + "selectedCount": "{{count}} selected", + "groupOwn": "Yours", + "groupTeam": "Shared with team", + "connectMore": "Connect more", + "fromConnector": "From {{name}}", + "fromConnectorTitle": "From {{name}}: {{title}}", + "knowledge": "Knowledge" }, "attachments": { "attach": "Attach", @@ -1720,7 +2074,9 @@ "accessingMemory": "Accessing memory…", "runningCode": "Running code…", "creatingArtifact": "Creating an artifact…", - "usingTool": "Using {{tool}}…" + "usingTool": "Using {{tool}}…", + "searchingConnector": "Searching {{name}}...", + "usingConnector": "Using {{name}}..." }, "toolChip": { "searchingWeb": "Searched the web · “{{query}}”", @@ -1731,7 +2087,10 @@ "accessingMemory": "Accessed memory", "runningCode": "Ran code", "creatingArtifact": "Created an artifact", - "usingTool": "Used {{tool}}" + "usingTool": "Used {{tool}}", + "searchedConnector": "Searched {{name}}", + "readConnector": "Read from {{name}}", + "usedConnector": "Used {{name}}: {{action}}" }, "inlineSteps": { "arguments": "Arguments", @@ -1786,7 +2145,12 @@ "approveAlways": "Approve, don't ask again", "deny": "Deny", "details": "Details", - "denyReasonPlaceholder": "Optional reason for denying…" + "denyReasonPlaceholder": "Optional reason for denying…", + "connectTitle": "This tool needs your {{name}} account.", + "connect": "Connect {{name}}", + "continue": "Continue", + "skip": "Skip", + "thisService": "this service" }, "research": { "title": "Research", @@ -1802,7 +2166,6 @@ "agents": { "title": "Agents", "edit": "Edit", - "view": "View", "card": { "pin": "Pin agent", "unpin": "Unpin agent", @@ -1896,7 +2259,7 @@ "placeholders": { "agentName": "Agent name", "describeAgent": "Describe your agent", - "selectSources": "Select sources", + "selectSources": "Select knowledge", "chunksPerQuery": "Chunks per query", "selectType": "Select type", "selectTools": "Select tools", @@ -1906,20 +2269,23 @@ "enterRequestLimit": "Enter request limit" }, "sourcePopup": { - "title": "Select Sources", - "searchPlaceholder": "Search sources...", - "noOptionsMessage": "No sources available", + "title": "Select knowledge", + "searchPlaceholder": "Search knowledge...", + "noOptionsMessage": "No knowledge yet", "groupOwn": "Your sources", "groupTeam": "Shared with team", - "noSourceHint": "Optional. Without a source the agent answers from the model and its tools only." + "noSourceHint": "Optional. Without knowledge the agent answers from the model and its tools only." }, "toolsPopup": { "title": "Select Tools", "searchPlaceholder": "Search tools...", "noOptionsMessage": "No tools available", "groupBuiltin": "Built-in", - "groupDefault": "Default", - "groupCustom": "Custom" + "groupDefault": "Built-in", + "groupCustom": "Custom", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "On this agent", + "attachedHint": "Added by someone else. You can remove it." }, "modelsPopup": { "title": "Select Models", @@ -1939,7 +2305,7 @@ "defaultModel": "Default Model", "name": "Name", "description": "Description", - "sources": "Sources", + "sources": "Knowledge", "avatar": "Avatar" }, "advanced": { @@ -2030,11 +2396,69 @@ "published": "Published" }, "sponsors": { - "attachNote": "Tools, sources and prompts you add from your own library run with your access for everyone who uses this agent.", + "attachNote": "Tools, sources and prompts you add that the owner can't use run with your access for everyone who uses this agent. You need to own them or be able to edit them, and you'll be asked to confirm.", "publicLinkNote": "This agent has a public link, so anyone with the link can get answers from them.", "addedBy": "Added by {{person}}: {{names}}", - "unavailable": "Not running: {{names}}. {{person}} added them and no longer has access. Remove them or choose ones the owner can use.", - "unknownItem": "Unnamed item" + "addedByOther": "Added by someone else: {{names}}", + "unknownItem": "Unnamed item", + "notAllowed": "You can't add {{names}} to this agent. The owner can't use them, and you can only share what you own or can edit.", + "takeOver": "Run {{name}} with my access", + "takeOverPending": "{{name}} will run with your access when you save.", + "undoTakeOver": "Undo", + "confirmationOutdated": "This agent changed since you confirmed what runs with your access. Check the notices and save again." + }, + "resourceStates": { + "title": "Some tools, sources or prompts aren't running", + "reason": { + "deleted": "{{name}} was deleted.", + "ownerLostAccessYou": "You can no longer use {{name}}.", + "ownerLostAccess": "The agent's owner can no longer use {{name}}.", + "sponsorCannotEditAgent": "{{name}} ran with {{person}}'s access, and they can no longer edit this agent.", + "sponsorCannotEditAgentOther": "{{name}} ran with someone else's access, and they can no longer edit this agent.", + "sponsorCannotEditItem": "{{name}} ran with {{person}}'s access, and they can no longer edit it.", + "sponsorCannotEditItemOther": "{{name}} ran with someone else's access, and they can no longer edit it.", + "connectionNeedsReconnect": "{{name}} needs its {{service}} account signed in again.", + "connectionRemoved": "The {{service}} account {{name}} used was removed.", + "connectorDisabled": "An admin turned off {{service}}, so {{name}} can't run.", + "unknown": "{{name}} isn't running." + }, + "ask": { + "shareAgain": "Ask {{person}} to share it again.", + "signInAgain": "Ask {{person}} to sign in again.", + "connectAgain": "Ask {{person}} to connect the account again.", + "admin": "Ask an admin to turn it back on.", + "shareAgainOwner": "Ask its owner to share it again.", + "signInAgainOwner": "Ask its owner to sign in again.", + "connectAgainOwner": "Ask its owner to connect the account again." + }, + "promptFallback": "The agent uses the default prompt until then.", + "serviceFallback": "the service", + "remove": "Remove", + "removeLabel": "Remove {{name}}", + "reconnectLabel": "Reconnect {{name}}", + "unnamed": { + "tool": "Tool {{id}}", + "source": "Source {{id}}", + "prompt": "Prompt {{id}}" + }, + "chip": "Not running: {{count}}" + }, + "sponsorConfirm": { + "title": "Run with your access?", + "description": "The agent's owner can't use these. If you add them, they run with your access for everyone who uses this agent.", + "audienceTitle": "Who can use them through this agent", + "audienceTeams": "Members of {{teams}}", + "audienceApiKey": "Anyone with the agent's API key, including its website widget", + "audiencePublicLink": "Anyone with the agent's public link", + "audienceWebhook": "Anything that calls the agent's webhook", + "audienceEditors": "Everyone who can open this agent", + "stopNote": "They stop running if you lose edit access to them or to this agent. They then stay attached but do nothing until someone who can edit them takes them over, or they are removed.", + "confirm": "Add with my access", + "types": { + "tool": "Tool", + "source": "Source", + "prompt": "Prompt" + } } }, "logs": { @@ -2343,7 +2767,7 @@ "outputVariable": "Output Variable", "outputVariablePlaceholder": "Variable name for output", "streamToUser": "Stream output to user", - "sources": "Sources", + "sources": "Knowledge", "documents": "Documents", "documentsHint": "Documents passed to this agent from uploads or upstream nodes.", "filePassing": "File passing", @@ -2382,7 +2806,8 @@ "detailsTitle": "Workflow details", "detailsDescription": "Name, description and image shown on the agent card.", "unsavedChanges": "Unsaved changes", - "editDetailsMenu": "Edit details" + "editDetailsMenu": "Edit details", + "attachedOption": "{{name}} (added by someone else)" }, "validation": { "nameRequired": "Workflow name is required", @@ -2485,6 +2910,7 @@ "workflowTitle": "Workflow Agent", "workflowDescription": "Design complex multi-step workflows with different models, conditional logic, and state management" }, + "view": "View", "deleteFailed": "Could not delete the agent. Please try again." }, "components": { @@ -2553,7 +2979,8 @@ "shared": "Shared", "connectTo": "Connect to {{provider}}", "sessionExpiredFor": "Session expired. Please reconnect to {{provider}}.", - "selectFilesFrom": "Select Files from {{provider}}" + "selectFilesFrom": "Select Files from {{provider}}", + "account": "Account" }, "actionButtons": { "openNewChat": "Open New Chat", @@ -2620,7 +3047,8 @@ "admins": "Admins", "usage": "Usage", "quotas": "Quotas", - "audit": "Activity" + "audit": "Activity", + "connectors": "Connectors" }, "quotas": { "unpriced": { @@ -2629,5 +3057,15 @@ "model": "{{model}} ({{tokens}} tokens)" } } + }, + "connectHero": { + "title": "Connect your data", + "body": "Chat with {{names}} and more." + }, + "common": { + "close": "Close", + "viewOnlyNotice": "You can view this, but your role can't change it.", + "credentialsLockedNotice": "Only the owner can change this tool's credentials.", + "savedSecretHint": "Saved. Leave empty to keep it." } } diff --git a/frontend/src/locale/es.json b/frontend/src/locale/es.json index 3b06f31e..65e4d40d 100644 --- a/frontend/src/locale/es.json +++ b/frontend/src/locale/es.json @@ -10,12 +10,6 @@ "loading": "Cargando...", "retry": "Reintentar", "cancel": "Cancelar", - "common": { - "close": "Cerrar", - "viewOnlyNotice": "Puedes ver esto, pero tu rol no puede cambiarlo.", - "credentialsLockedNotice": "Solo el propietario puede cambiar las credenciales de esta herramienta.", - "savedSecretHint": "Guardado. Déjalo vacío para conservarlo." - }, "help": "Asistencia", "emailUs": "Envíanos un correo", "documentation": "Documentación", @@ -106,16 +100,16 @@ }, "sources": { "title": "Aquí puedes gestionar todos los archivos fuente que están disponibles para ti y los que has subido.", - "subtitle": "Sube y gestiona los documentos y fuentes de conocimiento que impulsan tus respuestas", - "label": "Fuentes", + "subtitle": "Contenido que DocsGPT puede buscar y citar: archivos subidos, sitios web, wikis y contenido sincronizado desde tus conexiones.", + "label": "Conocimiento", "name": "Nombre de la Fuente", "date": "Fecha de Vector", "type": "Tipo", "tokenUsage": "Uso de Tokens", - "noData": "No hay fuentes existentes", + "noData": "Aún no hay conocimiento", "searchPlaceholder": "Buscar...", "addNew": "Agregar Nuevo", - "addSource": "Agregar Fuente", + "addSource": "Añadir conocimiento", "addChunk": "Agregar Fragmento", "preLoaded": "Precargado", "private": "Privado", @@ -185,7 +179,17 @@ "pagesTitle": "Páginas", "loadFailed": "No se pudieron cargar las páginas del wiki.", "pageLoadFailed": "No se pudo cargar esta página.", - "editTitle": "Editar página" + "editTitle": "Editar página", + "settings": { + "action": "Configuración del wiki", + "title": "Configuración del wiki", + "outsideEdits": { + "label": "Permitir que los usuarios de la API y del widget editen este wiki", + "description": "Si está desactivado, quienes usan un agente mediante su clave de API o su widget pueden leer este wiki, pero no cambiarlo. Los visitantes de un enlace público solo pueden editar los wikis que ellos mismos pueden editar, y se les pregunta antes de cada cambio. Tú y los editores del wiki siempre pueden editarlo." + }, + "loadError": "No se pudo cargar la configuración del wiki.", + "saveError": "No se pudo guardar el cambio. Inténtalo de nuevo." + } }, "graphrag": { "badge": "GraphRAG", @@ -303,7 +307,7 @@ "chunks": "Fragmentos", "noChunks": "No se encontraron fragmentos", "noChunksAlt": "No se encontraron fragmentos", - "goToSources": "Ir a Fuentes", + "goToSources": "Ir a Conocimiento", "uploadNew": "Subir nuevo", "noResults": "No se encontraron resultados", "fileName": "Nombre", @@ -446,6 +450,9 @@ "editChunkDescription": "{{file}} · fragmento {{n}} · {{tokens}} tokens", "previousChunk": "Fragmento anterior", "nextChunk": "Fragmento siguiente", + "paused": "Sincronización en pausa. Reconecta {{name}} para reanudarla.", + "emptyHint": "Sube archivos, añade un sitio web o conecta un servicio para sincronizar su contenido.", + "connectService": "Conectar un servicio", "viewConfig": "Ver ajustes de la fuente", "errors": { "forbidden": "No tienes permiso para hacer eso en esta fuente.", @@ -676,10 +683,6 @@ "createTeamError": "No se pudo crear el equipo.", "noTeams": "Aún no hay equipos.", "noDescription": "Sin descripción", - "memberCount_one": "{{formatted}} miembro", - "memberCount_other": "{{formatted}} miembros", - "sharedCount_one": "{{formatted}} compartido", - "sharedCount_other": "{{formatted}} compartidos", "loadError": "No se pudieron cargar los equipos.", "roleAdmin": "admin", "roleMember": "miembro", @@ -759,8 +762,46 @@ "teams": "Equipos", "people": "Personas", "editors": "Editores" + }, + "uses": { + "title": "Qué usa este agente", + "intro": "Todos los que usan este agente reciben estos elementos, cada uno con el acceso indicado.", + "stopped": "Detenido", + "writesOff": "No por API", + "writesNote": "«Permitir» en un conector se aplica a los chats. Los cambios marcados «No por API» siguen desactivados para los usuarios de la API, el widget y el enlace público hasta que los permitas en Access Details.", + "writesNoteEditor": "«Permitir» en un conector se aplica a los chats. Los cambios marcados «No por API» siguen desactivados para los usuarios de la API, el widget y el enlace público hasta que el propietario los permita en Access Details.", + "access": { + "you": "Tu acceso", + "person": "Acceso de {{person}}", + "other": "Acceso de otra persona", + "owner": "Acceso del propietario", + "member": "La propia cuenta de {{service}} de cada persona (API y widget: la tuya)", + "memberNoService": "La propia cuenta de cada persona (API y widget: la tuya)", + "yourAccount": "Tu cuenta de {{service}}", + "yourAccountNoService": "Tu cuenta", + "personAccount": "Cuenta de {{service}} de {{person}}", + "personAccountNoService": "Cuenta de {{person}}", + "memberShared": "La propia cuenta de {{service}} de cada persona (API y widget: la del propietario)", + "memberSharedNoService": "La propia cuenta de cada persona (API y widget: la del propietario)", + "otherAccount": "Cuenta de {{service}} de otra persona", + "otherAccountNoService": "Cuenta de otra persona", + "yourCredentials": "Tus credenciales guardadas", + "personCredentials": "Credenciales guardadas de {{person}}", + "otherCredentials": "Credenciales guardadas de otra persona" + }, + "writesSomeOff": "No todo por API", + "adminOff": "Cambios desactivados por un administrador", + "writesNoteApi": "«Permitir» en un conector se aplica a los chats. Los cambios marcados «No por API» siguen desactivados para los usuarios de la API y el widget hasta que los permitas en Access Details.", + "writesNoteApiEditor": "«Permitir» en un conector se aplica a los chats. Los cambios marcados «No por API» siguen desactivados para los usuarios de la API y el widget hasta que el propietario los permita en Access Details.", + "writesNoteMemberTail": "En las herramientas donde cada persona usa su propia cuenta, los usuarios del enlace público hacen cambios con su propia cuenta.", + "openAccessDetails": "Abrir Access Details", + "accessDetailsNeedsKey": "Access Details muestra estos cambios cuando el agente tiene una clave de API." } }, + "memberCount_one": "{{formatted}} miembro", + "memberCount_other": "{{formatted}} miembros", + "sharedCount_one": "{{formatted}} compartido", + "sharedCount_other": "{{formatted}} compartidos", "accessChangeError": "No se pudo cambiar el acceso.", "accessSettings": { "title": "Ajustes de acceso", @@ -808,7 +849,8 @@ }, "viewers_can_use_in_agents": { "label": "Los lectores pueden usarla en sus propios agentes", - "description": "Se ejecuta con tus credenciales." + "description": "Se ejecuta con tus credenciales.", + "descriptionMember": "Cada persona usa su propia cuenta." } }, "prompt": { @@ -930,11 +972,11 @@ }, "tools": { "label": "Herramientas", - "subtitle": "Explora, conecta y configura las herramientas e integraciones que impulsan a tus agentes", + "subtitle": "Acciones que pueden realizar tus agentes. Aquí están las herramientas integradas. Las herramientas de un servicio conectado se agrupan bajo ese servicio.", "searchPlaceholder": "Buscar...", "addTool": "Agregar Herramienta", "noToolsFound": "No se encontraron herramientas", - "selectToolSetup": "Seleccione una herramienta para configurar", + "selectToolSetup": "Añadir una herramienta", "settingsIconAlt": "Icono de configuración", "configureToolAria": "Configurar {{toolName}}", "toggleToolAria": "Alternar {{toolName}}", @@ -946,8 +988,8 @@ "builtIn": "Integrada", "authStatus": { "connected": "Conectado", - "needsAuth": "Requiere autenticación", - "configured": "Configurado" + "needsAuth": "Reconectar", + "configured": "Conectado" }, "deleteWarning": "¿Estás seguro de que deseas eliminar la herramienta \"{{toolName}}\"?", "unsavedChanges": "Tienes cambios sin guardar que se perderán si sales sin guardar.", @@ -1063,6 +1105,11 @@ }, "headerValuePlaceholder": "p. ej., application/json", "toolIconTitle": "Icono de {{name}}", + "groupBuiltIn": "Integradas", + "groupService": "De un servicio", + "browseConnectors": "Ver todos los conectores", + "viaConnection": "mediante {{name}}", + "off": "Desactivada", "view": "Ver", "inMyChats": "En mis chats", "useInMyChatsAria": "Usar {{toolName}} en mis chats", @@ -1298,13 +1345,320 @@ "subtitle": "El nuevo secreto de \"{{name}}\" está listo. El anterior ya no funciona." } } + }, + "connectors": { + "label": "Conectores", + "subtitle": "Conecta DocsGPT a los servicios que usa tu equipo. Una conexión puede sincronizar contenido en Conocimiento y dar herramientas a los agentes.", + "search": "Buscar conectores...", + "addCustom": "Añadir herramienta propia", + "custom": { + "custom_mcp": "Servidor MCP", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "Filtrar conectores por categoría", + "categories": { + "all": "Todos", + "connected": "Conectados", + "files": "Archivos y almacenamiento", + "knowledge": "Documentos y wikis", + "dev": "Herramientas de desarrollo", + "messaging": "Mensajería", + "database": "Bases de datos", + "search": "Búsqueda y web", + "custom": "Personalizados", + "projects": "Proyectos e incidencias", + "business": "Pagos y negocio" + }, + "capability": { + "sync": "Sincronizar", + "read": "Leer", + "write": "Escribir" + }, + "status": { + "connect": "Conectar", + "connected": "Conectado", + "connectedCount": "{{formatted}} conectadas", + "reconnect": "Reconectar", + "needsAdminSetup": "Requiere configuración del administrador", + "disabledByAdmin": "Desactivado por el administrador", + "custom": "Personalizado" + }, + "connectionStatus": { + "connected": "Conectado", + "reconnect_needed": "Hay que reconectar", + "disconnected": "Desconectado", + "error": "Error", + "pending": "Esperando el inicio de sesión" + }, + "publisher": { + "built_in": "Integrado", + "preset": "Predefinido", + "custom": "Personalizado" + }, + "descriptions": { + "google_drive": "Sincroniza Docs, Hojas de cálculo y PDF en Conocimiento.", + "share_point": "Sincroniza archivos de sitios de SharePoint y OneDrive en Conocimiento.", + "confluence": "Sincroniza espacios y páginas de Confluence en Conocimiento.", + "s3": "Sincroniza documentos de un bucket de S3 en Conocimiento.", + "reddit": "Sincroniza publicaciones de Reddit que coinciden con tus búsquedas en Conocimiento.", + "brave": "Busca en la web e imágenes con la API de Brave Search.", + "telegram": "Envía mensajes e imágenes a un chat de Telegram.", + "ntfy": "Envía notificaciones push mediante un servidor ntfy.", + "postgres": "Lee el esquema y ejecuta SQL en una base de datos Postgres.", + "custom_mcp": "Conecta cualquier servidor remoto de Model Context Protocol.", + "custom_openapi": "Importa una especificación OpenAPI y usa sus endpoints como herramientas.", + "mcp_notion": "Busca, lee y actualiza páginas y bases de datos de Notion.", + "mcp_linear": "Sincroniza incidencias y documentos con Conocimiento y deja que los agentes busquen, creen y actualicen incidencias.", + "mcp_atlassian": "Busca y actualiza incidencias de Jira y páginas de Confluence.", + "mcp_sentry": "Consulta incidencias, eventos y versiones de Sentry.", + "mcp_asana": "Encuentra y actualiza tareas y proyectos de Asana.", + "mcp_stripe": "Consulta clientes, pagos y suscripciones en Stripe.", + "github": "Sincroniza repositorios con Conocimiento y deja que los agentes lean código, issues y pull requests." + }, + "empty": "Aún no hay nada conectado. Conecta un servicio para sincronizar su contenido o dar nuevas herramientas a los agentes.", + "noMatches": "Ningún conector coincide con tu búsqueda.", + "loadFailed": "No se pudieron cargar los conectores.", + "askAdmin": "Pide a un administrador que lo configure.", + "setupSettings": "Define estos ajustes del servidor y reinicia DocsGPT:", + "setupGuide": "Guía de configuración", + "unverified": "DocsGPT no ha revisado este servidor. Conecta solo servidores de confianza.", + "detail": { + "accounts": "Cuentas", + "connectedAs": "Conectado como {{account}}", + "connectAnother": "Conectar otra cuenta", + "sources": "Conocimiento de esta conexión", + "tools": "Herramientas de esta conexión", + "noAccounts": "Aún no hay ninguna cuenta conectada.", + "failed": "No se pudo cargar esta conexión.", + "lastSync": "Sincronizado {{date}} · {{frequency}}", + "paused": "Sincronización en pausa", + "toolOn": "Activada", + "toolOff": "Desactivada", + "accountMenu": "Acciones de la cuenta", + "disconnect": "Desconectar", + "keyEnding": "Clave que termina en {{hint}}", + "syncMore": "Sincronizar más contenido", + "refreshTools": "Actualizar herramientas", + "noSources": "Aún no se ha sincronizado nada desde esta conexión.", + "remove": "Eliminar", + "refreshed": "Herramientas actualizadas: {{added}} añadidas, {{removed}} eliminadas.", + "refreshFailed": "No se pudieron actualizar las herramientas.", + "syncNow": "Sincronizar ahora", + "syncStarted": "Sincronizando {{name}}…", + "syncFailed": "No se pudo iniciar la sincronización. Inténtalo de nuevo.", + "toolSwitchFailed": "No se pudo cambiar la herramienta. Inténtalo de nuevo.", + "account": "Cuenta: {{account}}", + "expired": "El inicio de sesión caducó. Reconecta para seguir usando esta conexión.", + "broken": "Esta conexión dejó de funcionar. Reconecta para intentarlo de nuevo.", + "addTools": "Añadir herramientas", + "noTools": "Todavía no hay herramientas de esta conexión.", + "addToolsFailed": "No se pudieron añadir las herramientas. Inténtalo de nuevo.", + "rename": "Cambiar nombre", + "accountPicker": "Elegir una cuenta" + }, + "permission": { + "always": "Permitir", + "ask": "Preguntar antes", + "off": "Desactivado", + "label": "Permiso para {{action}}", + "saveFailed": "No se pudo guardar el permiso. Inténtalo de nuevo.", + "groupLabel": "Permiso para todo en {{group}}", + "fold": "Ocultar acciones", + "hint": "«Permitir» y «Preguntar antes» se aplican en el chat; con «Preguntar antes», DocsGPT te pregunta antes de ejecutar la acción. Los cambios por la API, el widget o el enlace público de un agente se permiten por agente, en su Access Details.", + "actionCount_one": "{{formatted}} acción", + "actionCount_other": "{{formatted}} acciones", + "customize_one": "Ajustar la acción", + "customize_other": "Ajustar cada una de {{formatted}}" + }, + "disconnect": { + "title": "¿Desconectar {{name}}?", + "body_zero": "Las herramientas de esta conexión dejarán de funcionar hasta que vuelvas a conectarla.", + "body_one": "El elemento que sincronizó sigue en Conocimiento, pero deja de actualizarse. Las herramientas de esta conexión dejan de funcionar hasta que reconectes.", + "body_other": "Los {{formatted}} elementos que sincronizó siguen en Conocimiento, pero dejan de actualizarse. Las herramientas de esta conexión dejan de funcionar hasta que reconectes.", + "failed": "No se pudo desconectar. Inténtalo de nuevo." + }, + "fields": { + "token": "Clave de API", + "aws_access_key_id": "ID de clave de acceso", + "aws_secret_access_key": "Clave de acceso secreta", + "region": "Región", + "endpoint_url": "URL de endpoint personalizada", + "client_id": "ID de cliente", + "client_secret": "Secreto de cliente", + "user_agent": "User agent", + "bucket": "Bucket", + "prefix": "Prefijo de ruta", + "search_queries": "Consultas de búsqueda", + "number_posts": "Número de publicaciones", + "telegram_token": "Token del bot", + "ntfy_token": "Token de acceso", + "postgres_token": "Cadena de conexión", + "brave_token": "Clave de API", + "github_access_token": "Token de acceso personal", + "repo_url": "Repositorio", + "telegram_chat_id": "ID del chat predeterminado" + }, + "wizard": { + "connectTitle": "Conectar {{name}}", + "reconnectTitle": "Reconectar {{name}}", + "signIn": "Iniciar sesión en {{name}}", + "chooseWhatToSync": "Elige qué sincronizar", + "skip": "Omitir por ahora", + "syncFrequency": "Sincronizar", + "doneTitle": "{{name}} está conectado", + "doneSummary": "Conectado como {{account}}. {{sources}} sincronizándose en Conocimiento, {{tools}} listas.", + "doneSummaryNone": "Conectado como {{account}}.", + "sourcesCount_one": "{{formatted}} elemento", + "sourcesCount_other": "{{formatted}} elementos", + "toolsCount_one": "{{formatted}} herramienta", + "toolsCount_other": "{{formatted}} herramientas", + "toolsHeading": "Herramientas ({{formatted}})", + "tryInChat": "Probar en el chat", + "done": "Listo", + "connectFailed": "No se pudo conectar. Revisa los datos e inténtalo de nuevo.", + "syncFailed": "No se pudo iniciar la sincronización. Inténtalo de nuevo.", + "connectedWithKey": "Conectado con la clave que termina en {{hint}}.", + "doneCounts": "{{sources}} sincronizándose en Conocimiento, {{tools}} listas.", + "doneSources": "{{sources}} sincronizándose en Conocimiento.", + "doneTools": "{{tools}} listas.", + "waiting": "Termina de iniciar sesión en {{name}} en la ventana que se abrió.", + "openSignIn": "Abrir el inicio de sesión de {{name}}", + "chooseWhatToSetUp": "Elige qué configurar", + "continue": "Continuar", + "methodLabel": "Cómo conectar", + "methodOauth": "Iniciar sesión con {{name}}", + "methodToken": "Usar un token de acceso", + "credentialsRejected": "{{name}} no aceptó estas credenciales. Revísalas e inténtalo de nuevo.", + "toolsUnavailable": "No se pudo acceder a las herramientas de {{name}}. Inténtalo de nuevo o desactívalas para continuar.", + "addTools": "Permitir que los agentes usen {{name}}", + "addToolsDescription": { + "read": "Los agentes pueden consultar información. No pueden cambiar nada.", + "readWrite": "Los agentes pueden consultar información y hacer cambios. Los cambios necesitan tu aprobación." + }, + "accountName": "Nombre de esta cuenta", + "accountNameHint": "Opcional. Distingue tus cuentas, para ti y para la IA; por ejemplo, Bot de alertas.", + "syncToKnowledge": "Sincronizar en Conocimiento", + "syncToKnowledgeDescription": "Elige contenido de {{name}} para que las respuestas lo busquen. Desactivado, solo se conecta; puedes sincronizar más tarde desde Conectores.", + "syncLater": "Aún no se sincroniza nada en Conocimiento. Para añadir contenido, abre {{name}} en Conectores y elige Sincronizar más contenido.", + "retrievalSettings": "Configuración avanzada de recuperación" + }, + "remove": { + "title": "¿Eliminar {{name}}?", + "description": "Se borra el inicio de sesión guardado. Elige qué pasa con lo que alimenta esta conexión.", + "sourcesLabel_one": "El elemento que sincronizó en Conocimiento", + "sourcesLabel_other": "Los {{formatted}} elementos que sincronizó en Conocimiento", + "keepSources": "Conservar el contenido (dejar de sincronizar)", + "deleteSources": "Eliminar", + "toolsLabel_one": "Su herramienta", + "toolsLabel_other": "Sus {{formatted}} herramientas", + "deleteTools": "Eliminar", + "keepTools": "Conservar, desactivadas", + "failed": "No se pudo eliminar la conexión. Inténtalo de nuevo." + }, + "health": { + "reconnect": "Reconecta {{name}} para seguir sincronizando.", + "reconnectSources_one": "Reconecta {{name}}: {{formatted}} elemento dejó de sincronizarse en Conocimiento", + "reconnectSources_other": "Reconecta {{name}}: {{formatted}} elementos dejaron de sincronizarse en Conocimiento", + "reconnectTools": "Vuelve a conectar {{name}} para que los agentes puedan usarlo de nuevo", + "reconnectBoth_one": "Reconecta {{name}}: {{formatted}} elemento dejó de sincronizarse y los agentes no pueden usarlo", + "reconnectBoth_other": "Reconecta {{name}}: {{formatted}} elementos dejaron de sincronizarse y los agentes no pueden usarlo", + "navDot": "Una conexión necesita atención", + "signInAgain": "Vuelve a iniciar sesión", + "pickerNotice": "{{name}} necesita que vuelvas a iniciar sesión." + }, + "error": { + "decrypt": "No se pudieron descifrar las credenciales guardadas. Reconecta para continuar.", + "defaultKey": "Un administrador debe definir ENCRYPTION_SECRET_KEY antes de poder conectar servicios." + }, + "sharing": { + "owner": "Todos usan tu cuenta", + "member": "Cada persona conecta su propia cuenta", + "ownerShort": "Tu cuenta", + "memberShort": "La de cada persona", + "ownerShortShared": "La cuenta del propietario" + }, + "share": { + "heading": "Con qué cuenta se ejecuta", + "ownerWarning": "Los miembros del equipo actuarán como {{account}} en {{name}}.", + "ownerWarningShared": "Los miembros del equipo actuarán con la cuenta de {{name}} del propietario.", + "confirmWrite": "Entiendo que los miembros del equipo pueden realizar acciones con mi cuenta.", + "confirmWriteShared": "Entiendo que los miembros del equipo pueden realizar acciones con la cuenta de {{name}} del propietario.", + "memberNote": "Cada persona usa su propia cuenta de {{name}} y la conecta la primera vez que usa esta herramienta. A través de la clave de API o el widget de un agente, se usa la cuenta del propietario del agente.", + "forced": "Un administrador eligió esto para todos los recursos compartidos de este conector.", + "ownerChooses": "Solo el propietario puede cambiar esto.", + "saveFailed": "No se pudo guardar el cambio. Inténtalo de nuevo." + }, + "createFailed": "No se pudo añadir el conector. Inténtalo de nuevo.", + "capabilityFilter": { + "sync": "Se muestran los conectores que sincronizan contenido en Conocimiento.", + "tools": "Se muestran los conectores que dan herramientas a los agentes.", + "showAll": "Mostrar todos" + }, + "capabilityPlain": { + "sync": "Sincroniza en Conocimiento", + "read": "Consulta datos", + "write": "Realiza acciones" + }, + "github": { + "tokenHint": "Crea un token detallado con acceso de lectura a Contents y Metadata en los repositorios que quieras usar. Para que los agentes hagan cambios, dale también acceso de lectura y escritura a Issues y Pull requests.", + "writes": "Permitir también que los agentes hagan cambios (issues, comentarios, pull requests)", + "writesDescription": "Por defecto, cada cambio te pide aprobación primero. Tu token, o la GitHub App, necesita acceso de lectura y escritura a Issues y Pull requests.", + "writesForbidden": "Un administrador desactivó los cambios a través de GitHub.", + "writesFailed": "No se pudo cambiar lo que los agentes pueden hacer en GitHub.", + "createToken": "Crear un token en GitHub", + "chooseRepositories": "Elegir repositorios en GitHub", + "noAppRepositories": "DocsGPT aún no ve ningún repositorio. Elige en GitHub cuáles puede leer.", + "noTokenRepositories": "Este token no puede leer ningún repositorio. Dale acceso en GitHub y vuelve a conectar.", + "searchRepositories": "Buscar repositorios", + "repositories": "Repositorios", + "updated": "Actualizado {{date}}", + "loadFailed": "No se pudieron cargar tus repositorios." + }, + "linear": { + "teams": "Equipos", + "teamsHint": "Sincroniza las incidencias de estos equipos.", + "projects": "Proyectos", + "projectsHint": "Sincroniza las incidencias de estos proyectos.", + "search": "Buscar equipos y proyectos", + "noMatches": "Ningún equipo ni proyecto coincide con tu búsqueda.", + "includeComments": "Incluir comentarios", + "includeCommentsDescription": "Añade a cada incidencia sus comentarios.", + "includeDocuments": "Incluir documentos de proyecto", + "includeDocumentsDescription": "Sincroniza los documentos de Linear de los proyectos que elijas.", + "loadFailed": "No se pudieron cargar tus equipos y proyectos de Linear.", + "empty": "Esta cuenta de Linear no tiene equipos ni proyectos que sincronizar." + }, + "parameters": { + "show": "Parámetros", + "hide": "Ocultar parámetros", + "fixedCount_one": "{{formatted}} fijo", + "fixedCount_other": "{{formatted}} fijos", + "ai": "Que decida la IA", + "fixed": "Usar siempre", + "choiceLabel": "Cómo se rellena {{parameter}}", + "valueLabel": "Valor de {{parameter}}", + "save": "Guardar", + "saveFailed": "No se pudo guardar el valor. Inténtalo de nuevo.", + "hint": "Un valor con «Usar siempre» se envía en cada llamada. La IA nunca lo ve ni puede cambiarlo.", + "fromAccount": "Definido en esta cuenta: {{value}}" + }, + "fieldHints": { + "telegram_chat_id": "Opcional. Los mensajes van a este chat y la IA no puede elegir otro. Añade el bot al chat, envíale un mensaje y busca el id del chat en https://api.telegram.org/bot/getUpdates." + }, + "rename": { + "title": "Cambiar el nombre de la cuenta", + "description": "El nombre distingue esta cuenta de tus otras cuentas del mismo servicio, para ti y para la IA. Déjalo vacío para mostrar la cuenta en sí.", + "label": "Nombre de la cuenta", + "save": "Guardar", + "failed": "No se pudo cambiar el nombre de la cuenta. Inténtalo de nuevo." + } } }, "modals": { "uploadDoc": { - "label": "Subir nuevo documento", + "label": "Añadir conocimiento", "select": "Elige cómo cargar tu documento en DocsGPT", - "selectSource": "Selecciona la forma de agregar tu fuente", + "selectSource": "Añadir conocimiento", "selectedFiles": "Archivos Seleccionados", "noFilesSelected": "No hay archivos seleccionados", "file": "Subir desde el dispositivo", @@ -1319,19 +1673,11 @@ "filesRejected": "No se añadieron (más de 25 MB o tipo no admitido): {{files}}", "uploadedFiles": "Archivos Subidos", "cancel": "Cancelar", - "train": "Entrenar", + "train": "Añadir al conocimiento", "create": "Crear", "link": "Enlace", "urlLink": "Enlace URL", "repoUrl": "URL del Repositorio", - "reddit": { - "id": "ID del Cliente", - "secret": "Secreto del Cliente", - "agent": "Agente de usuario", - "searchQueries": "Consultas de búsqueda", - "numberOfPosts": "Número de publicaciones", - "addQuery": "Agregar Consulta" - }, "drag": { "title": "Suelta los archivos adjuntos aquí", "description": "Suelta para subir tus archivos adjuntos" @@ -1353,8 +1699,6 @@ "clear": "Limpiar", "ingestionFailed": "La ingesta falló." }, - "showAdvanced": "Mostrar opciones avanzadas", - "hideAdvanced": "Ocultar opciones avanzadas", "ingestors": { "local_file": { "label": "Subir archivo", @@ -1372,26 +1716,6 @@ "label": "GitHub", "heading": "Agregar contenido desde GitHub" }, - "reddit": { - "label": "Reddit", - "heading": "Agregar contenido desde Reddit" - }, - "google_drive": { - "label": "Google Drive", - "heading": "Subir desde Google Drive" - }, - "s3": { - "label": "Amazon S3", - "heading": "Agregar contenido desde Amazon S3" - }, - "share_point": { - "label": "SharePoint", - "heading": "Subir desde SharePoint" - }, - "confluence": { - "label": "Confluence", - "heading": "Subir desde Confluence" - }, "wiki": { "label": "Nueva wiki", "heading": "Crear una wiki viva" @@ -1445,13 +1769,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "ID de clave de acceso de AWS", - "awsSecretAccessKey": "Clave de acceso secreta de AWS", - "bucket": "Nombre del bucket", - "prefix": "Prefijo de ruta (opcional)", - "region": "Región de AWS", - "endpointUrl": "URL de endpoint personalizada (opcional)", "initialContent": "Contenido inicial (opcional)" + }, + "github": { + "privateHint": "¿Repositorio privado? Conecta tu cuenta de GitHub para sincronizarlo.", + "connectedHint": "Los repositorios privados se sincronizan desde tu conexión de GitHub ({{account}}).", + "connect": "Conectar GitHub", + "pickRepository": "Elegir un repositorio" + }, + "connectData": { + "title": "Conecta tus datos", + "description": "Sincroniza {{services}}", + "more": "más", + "intro": "Elige un servicio, inicia sesión y luego elige qué sincronizar en Conocimiento.", + "connectedAs": "Conectado como {{account}}", + "browseAll": "Ver todos los conectores" } }, "createAPIKey": { @@ -1488,8 +1820,8 @@ "configTool": { "title": "Configuración de la Herramienta", "type": "Tipo", - "apiKeyLabel": "Clave API / OAuth", - "apiKeyPlaceholder": "Ingrese la Clave API / OAuth", + "apiKeyLabel": "Clave de API", + "apiKeyPlaceholder": "Introduce la clave de API", "addButton": "Agregar Herramienta", "closeButton": "Cerrar", "customNamePlaceholder": "Enter custom name (optional)", @@ -1567,6 +1899,24 @@ "learnMore": "Learn more", "resetKey": "Restablecer clave", "resetKeyConfirm": "¿Seguro que quieres restablecer la clave de API? La clave actual dejará de funcionar de inmediato y esta acción no se puede deshacer.", + "apiWrites": { + "title": "Cambios de otros en tu nombre", + "description": "Nadie puede aprobar una acción por ti a través de la clave de API, el widget o el enlace público del agente, así que los cambios con tus cuentas conectadas y credenciales guardadas se bloquean salvo que los permitas aquí.", + "saveFailed": "No se pudo guardar. Inténtalo de nuevo.", + "summaryNone": "Aún no hay nada permitido", + "summaryTools_one": "{{tools}} puede hacer cambios", + "summaryTools_other": "{{tools}} pueden hacer cambios", + "summaryCount": "{{allowed}} de {{formatted}} permitidos", + "toolCount": "Cambios por API, widget y enlaces públicos · {{allowed}} de {{formatted}} permitidos", + "toolLabel": "Permitir cambios por API, widget y enlaces públicos en {{tool}}", + "choice": { + "off": "Desactivado", + "all": "Todos" + }, + "customize_one": "Ajustar el cambio", + "customize_other": "Ajustar cada uno de {{formatted}}", + "fold": "Ocultar cambios" + }, "actionFailed": "No funcionó. Inténtalo de nuevo.", "apiKeyAfterPublish": "Publica el agente para crear su clave de API." }, @@ -1661,14 +2011,18 @@ }, "sources": { "title": "Fuentes", - "text": "Elegir tus fuentes", + "text": "Elegir conocimiento", "link": "Enlace fuente", "view_more_one": "Ver {{count}} fuente más", "view_more_other": "Ver {{count}} fuentes más", - "noSourcesAvailable": "No hay fuentes disponibles", - "selectedCount": "{{count}} fuentes seleccionadas", - "groupOwn": "Tus fuentes", - "groupTeam": "Compartidas con el equipo" + "noSourcesAvailable": "Aún no hay conocimiento", + "selectedCount": "{{count}} seleccionados", + "groupOwn": "Tuyos", + "groupTeam": "Compartidas con el equipo", + "connectMore": "Conectar más", + "fromConnector": "De {{name}}", + "fromConnectorTitle": "De {{name}}: {{title}}", + "knowledge": "Conocimiento" }, "attachments": { "attach": "Adjuntar", @@ -1709,7 +2063,9 @@ "accessingMemory": "Accediendo a la memoria…", "runningCode": "Ejecutando código…", "creatingArtifact": "Creando un artefacto…", - "usingTool": "Usando {{tool}}…" + "usingTool": "Usando {{tool}}…", + "searchingConnector": "Buscando en {{name}}...", + "usingConnector": "Usando {{name}}..." }, "toolChip": { "searchingWeb": "Buscado en la web · “{{query}}”", @@ -1720,7 +2076,10 @@ "accessingMemory": "Memoria consultada", "runningCode": "Código ejecutado", "creatingArtifact": "Artefacto creado", - "usingTool": "Usado {{tool}}" + "usingTool": "Usado {{tool}}", + "searchedConnector": "Buscó en {{name}}", + "readConnector": "Leyó de {{name}}", + "usedConnector": "Usó {{name}}: {{action}}" }, "inlineSteps": { "arguments": "Argumentos", @@ -1770,7 +2129,12 @@ "approveAlways": "Aprobar y no volver a preguntar", "deny": "Denegar", "details": "Detalles", - "denyReasonPlaceholder": "Motivo del rechazo (opcional)…" + "denyReasonPlaceholder": "Motivo del rechazo (opcional)…", + "connectTitle": "Esta herramienta necesita tu cuenta de {{name}}.", + "connect": "Conectar {{name}}", + "continue": "Continuar", + "skip": "Omitir", + "thisService": "este servicio" }, "research": { "title": "Investigación", @@ -1786,7 +2150,6 @@ "agents": { "title": "Agentes", "edit": "Editar", - "view": "Ver", "card": { "pin": "Fijar agente", "unpin": "Dejar de fijar agente", @@ -1880,7 +2243,7 @@ "placeholders": { "agentName": "Nombre del agente", "describeAgent": "Describe tu agente", - "selectSources": "Seleccionar fuentes", + "selectSources": "Seleccionar conocimiento", "chunksPerQuery": "Fragmentos por consulta", "selectType": "Seleccionar tipo", "selectTools": "Seleccionar herramientas", @@ -1890,20 +2253,23 @@ "enterRequestLimit": "Ingresar límite de solicitudes" }, "sourcePopup": { - "title": "Seleccionar Fuentes", - "searchPlaceholder": "Buscar fuentes...", - "noOptionsMessage": "No hay fuentes disponibles", + "title": "Seleccionar conocimiento", + "searchPlaceholder": "Buscar conocimiento...", + "noOptionsMessage": "Aún no hay conocimiento", "groupOwn": "Tus fuentes", "groupTeam": "Compartidas con el equipo", - "noSourceHint": "Opcional. Sin una fuente, el agente responde solo con el modelo y sus herramientas." + "noSourceHint": "Opcional. Sin conocimiento, el agente responde solo con el modelo y sus herramientas." }, "toolsPopup": { "title": "Seleccionar Herramientas", "searchPlaceholder": "Buscar herramientas...", "noOptionsMessage": "No hay herramientas disponibles", "groupBuiltin": "Integradas", - "groupDefault": "Predeterminadas", - "groupCustom": "Personalizadas" + "groupDefault": "Integradas", + "groupCustom": "Personalizadas", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "En este agente", + "attachedHint": "Lo añadió otra persona. Puedes quitarlo." }, "modelsPopup": { "title": "Seleccionar Modelos", @@ -1923,7 +2289,7 @@ "defaultModel": "Modelo Predeterminado", "name": "Nombre", "description": "Descripción", - "sources": "Fuentes", + "sources": "Conocimiento", "avatar": "Avatar" }, "advanced": { @@ -2012,11 +2378,69 @@ "published": "Publicado" }, "sponsors": { - "attachNote": "Las herramientas, fuentes y prompts que añadas desde tu propia biblioteca se ejecutan con tu acceso para todos los que usen este agente.", + "attachNote": "Las herramientas, fuentes y prompts que añadas y que el propietario no pueda usar se ejecutan con tu acceso para todos los que usen este agente. Debes ser su propietario o poder editarlos, y se te pedirá que lo confirmes.", "publicLinkNote": "Este agente tiene un enlace público, así que cualquiera con el enlace puede obtener respuestas de ellos.", "addedBy": "Añadido por {{person}}: {{names}}", - "unavailable": "Sin ejecutarse: {{names}}. {{person}} los añadió y ya no tiene acceso. Quítalos o elige otros que el propietario pueda usar.", - "unknownItem": "Elemento sin nombre" + "addedByOther": "Añadido por otra persona: {{names}}", + "unknownItem": "Elemento sin nombre", + "notAllowed": "No puedes añadir {{names}} a este agente. El propietario no puede usarlos y solo puedes compartir lo que es tuyo o puedes editar.", + "takeOver": "Ejecutar {{name}} con mi acceso", + "takeOverPending": "{{name}} se ejecutará con tu acceso cuando guardes.", + "undoTakeOver": "Deshacer", + "confirmationOutdated": "Este agente cambió desde que confirmaste lo que se ejecuta con tu acceso. Revisa los avisos y vuelve a guardar." + }, + "resourceStates": { + "title": "Algunas herramientas, fuentes o prompts no se están ejecutando", + "reason": { + "deleted": "{{name}} se eliminó.", + "ownerLostAccessYou": "Ya no puedes usar {{name}}.", + "ownerLostAccess": "El propietario del agente ya no puede usar {{name}}.", + "sponsorCannotEditAgent": "{{name}} se ejecutaba con el acceso de {{person}}, que ya no puede editar este agente.", + "sponsorCannotEditAgentOther": "{{name}} se ejecutaba con el acceso de otra persona, que ya no puede editar este agente.", + "sponsorCannotEditItem": "{{name}} se ejecutaba con el acceso de {{person}}, que ya no puede editarlo.", + "sponsorCannotEditItemOther": "{{name}} se ejecutaba con el acceso de otra persona, que ya no puede editarlo.", + "connectionNeedsReconnect": "{{name}} necesita que su cuenta de {{service}} vuelva a iniciar sesión.", + "connectionRemoved": "Se eliminó la cuenta de {{service}} que usaba {{name}}.", + "connectorDisabled": "Un administrador desactivó {{service}}, así que {{name}} no puede ejecutarse.", + "unknown": "{{name}} no se está ejecutando." + }, + "ask": { + "shareAgain": "Pide a {{person}} que lo vuelva a compartir.", + "signInAgain": "Pide a {{person}} que vuelva a iniciar sesión.", + "connectAgain": "Pide a {{person}} que vuelva a conectar la cuenta.", + "admin": "Pide a un administrador que lo vuelva a activar.", + "shareAgainOwner": "Pide a su propietario que lo vuelva a compartir.", + "signInAgainOwner": "Pide a su propietario que vuelva a iniciar sesión.", + "connectAgainOwner": "Pide a su propietario que vuelva a conectar la cuenta." + }, + "promptFallback": "Mientras tanto, el agente usa el prompt predeterminado.", + "serviceFallback": "el servicio", + "remove": "Quitar", + "removeLabel": "Quitar {{name}}", + "reconnectLabel": "Reconectar {{name}}", + "unnamed": { + "tool": "Herramienta {{id}}", + "source": "Fuente {{id}}", + "prompt": "Prompt {{id}}" + }, + "chip": "Sin ejecutar: {{count}}" + }, + "sponsorConfirm": { + "title": "¿Ejecutar con tu acceso?", + "description": "El propietario del agente no puede usarlos. Si los añades, se ejecutan con tu acceso para todos los que usen este agente.", + "audienceTitle": "Quién puede usarlos a través de este agente", + "audienceTeams": "Miembros de {{teams}}", + "audienceApiKey": "Cualquiera con la clave API del agente, incluido su widget web", + "audiencePublicLink": "Cualquiera con el enlace público del agente", + "audienceWebhook": "Todo lo que llame al webhook del agente", + "audienceEditors": "Todos los que pueden abrir este agente", + "stopNote": "Dejan de ejecutarse si pierdes el acceso de edición a ellos o a este agente. Entonces siguen adjuntos pero no hacen nada hasta que alguien que pueda editarlos se haga cargo de ellos o se quiten.", + "confirm": "Añadir con mi acceso", + "types": { + "tool": "Herramienta", + "source": "Fuente", + "prompt": "Prompt" + } } }, "logs": { @@ -2308,7 +2732,7 @@ "outputVariable": "Variable de salida", "outputVariablePlaceholder": "Nombre de la variable de salida", "streamToUser": "Transmitir la salida al usuario", - "sources": "Fuentes", + "sources": "Conocimiento", "documents": "Documentos", "documentsHint": "Documentos que se pasan a este agente desde cargas o nodos anteriores.", "filePassing": "Paso de archivos", @@ -2347,7 +2771,8 @@ "detailsTitle": "Detalles del flujo de trabajo", "detailsDescription": "Nombre, descripción e imagen que se muestran en la tarjeta del agente.", "unsavedChanges": "Cambios sin guardar", - "editDetailsMenu": "Editar detalles" + "editDetailsMenu": "Editar detalles", + "attachedOption": "{{name}} (añadido por otra persona)" }, "validation": { "nameRequired": "El nombre del flujo de trabajo es obligatorio", @@ -2453,6 +2878,7 @@ "workflowTitle": "Agente de flujo de trabajo", "workflowDescription": "Diseña flujos de trabajo complejos de varios pasos con distintos modelos, lógica condicional y gestión de estado" }, + "view": "Ver", "deleteFailed": "No se pudo eliminar el agente. Inténtalo de nuevo." }, "components": { @@ -2521,7 +2947,8 @@ "shared": "Compartido", "connectTo": "Conectar a {{provider}}", "sessionExpiredFor": "Sesión expirada. Por favor, reconecte a {{provider}}.", - "selectFilesFrom": "Seleccionar archivos de {{provider}}" + "selectFilesFrom": "Seleccionar archivos de {{provider}}", + "account": "Cuenta" }, "actionButtons": { "openNewChat": "Abrir nuevo chat", @@ -2579,5 +3006,15 @@ }, "prompts": { "textAriaLabel": "Texto del prompt" + }, + "connectHero": { + "title": "Conecta tus datos", + "body": "Chatea con {{names}} y más." + }, + "common": { + "close": "Cerrar", + "viewOnlyNotice": "Puedes ver esto, pero tu rol no puede cambiarlo.", + "credentialsLockedNotice": "Solo el propietario puede cambiar las credenciales de esta herramienta.", + "savedSecretHint": "Guardado. Déjalo vacío para conservarlo." } } diff --git a/frontend/src/locale/jp.json b/frontend/src/locale/jp.json index 94dd1bbf..504f2460 100644 --- a/frontend/src/locale/jp.json +++ b/frontend/src/locale/jp.json @@ -10,12 +10,6 @@ "loading": "読み込み中...", "retry": "再試行", "cancel": "キャンセル", - "common": { - "close": "閉じる", - "viewOnlyNotice": "表示はできますが、あなたのロールでは変更できません。", - "credentialsLockedNotice": "このツールの認証情報を変更できるのはオーナーだけです。", - "savedSecretHint": "保存済みです。空のままにすると保持されます。" - }, "help": "ヘルプ", "emailUs": "メールを送る", "documentation": "ドキュメント", @@ -106,16 +100,16 @@ }, "sources": { "title": "ここでは、利用可能なすべてのソースファイルとアップロードしたファイルを管理できます。", - "subtitle": "レスポンスを機能させるドキュメントとナレッジソースをアップロードして管理", - "label": "ソース", + "subtitle": "DocsGPT が検索・引用できるコンテンツ：アップロード、ウェブサイト、Wiki、接続から同期したコンテンツ。", + "label": "ナレッジ", "name": "ソース名", "date": "ベクトル日付", "type": "タイプ", "tokenUsage": "トークン使用量", - "noData": "既存のソースがありません", + "noData": "ナレッジはまだありません", "searchPlaceholder": "検索...", "addNew": "新規追加", - "addSource": "ソースを追加", + "addSource": "ナレッジを追加", "addChunk": "チャンクを追加", "preLoaded": "プリロード済み", "private": "プライベート", @@ -176,7 +170,6 @@ } }, "badge": "リビングWiki", - "byline_one": "{{pages}} ページ · {{tokens}} トークン", "byline_other": "{{pages}} ページ · {{tokens}} トークン", "explainer": "エージェントは作業しながらこれらのページを読み、書き換えます。", "explainerEditable": "エージェントは作業しながらこれらのページを読み、書き換えます。このソースを使うエージェントは誰でも編集できます。", @@ -185,7 +178,18 @@ "pagesTitle": "ページ", "loadFailed": "Wikiページを読み込めませんでした。", "pageLoadFailed": "このページを読み込めませんでした。", - "editTitle": "ページを編集" + "editTitle": "ページを編集", + "settings": { + "action": "Wiki設定", + "title": "Wiki設定", + "outsideEdits": { + "label": "APIとウィジェットの利用者にこのWikiの編集を許可", + "description": "オフの場合、APIキーやウィジェットからエージェントを利用する人はこのWikiを読めますが、変更はできません。公開リンクの訪問者は自分で編集できるWikiだけを編集でき、変更のたびに確認されます。あなたとWikiの編集者はいつでも編集できます。" + }, + "loadError": "Wiki設定を読み込めませんでした。", + "saveError": "変更を保存できませんでした。もう一度お試しください。" + }, + "byline_one": "{{pages}} ページ · {{tokens}} トークン" }, "graphrag": { "badge": "GraphRAG", @@ -235,7 +239,6 @@ "byConnections": "（接続数順）", "typeFilter": "種類で絞り込む", "otherTypes": "その他: {{types}}", - "otherTypesMore_one": "他{{formatted}}件", "otherTypesMore_other": "他{{formatted}}件", "untyped": "種類なし", "loadFailed": "グラフを読み込めませんでした。", @@ -247,7 +250,6 @@ "showMore": "もっと見る", "showLess": "表示を減らす", "relationships": "関係", - "relationshipsCapped_one": "{{total}}件の関係のうち、強い順に{{shown}}件を表示しています。", "relationshipsCapped_other": "{{total}}件の関係のうち、強い順に{{shown}}件を表示しています。", "noRelationships": "このエンティティには関係がありません。", "relatedTo": "関連", @@ -256,18 +258,20 @@ "sourceChunks": "ソースチャンク", "showInGraph": "グラフで表示", "allTypes": "すべての種類", - "chunkCount_one": "{{formatted}}件のチャンク", "chunkCount_other": "{{formatted}}件のチャンク", - "entityCount_one": "{{formatted}}件のエンティティ", "entityCount_other": "{{formatted}}件のエンティティ", - "relationshipCount_one": "{{formatted}}件の関係", "relationshipCount_other": "{{formatted}}件の関係", "chunk": "チャンク", "chunkMeta": "{{file}} · {{tokens}} トークン", "chunkTokens": "{{tokens}} トークン", "openInFiles": "ファイルで開く", "editChunk": "チャンクを編集", - "chunkSaved": "チャンクを保存しました" + "chunkSaved": "チャンクを保存しました", + "otherTypesMore_one": "他{{formatted}}件", + "relationshipsCapped_one": "{{total}}件の関係のうち、強い順に{{shown}}件を表示しています。", + "chunkCount_one": "{{formatted}}件のチャンク", + "entityCount_one": "{{formatted}}件のエンティティ", + "relationshipCount_one": "{{formatted}}件の関係" } }, "testRetrieval": { @@ -302,7 +306,7 @@ "chunks": "チャンク", "noChunks": "チャンクが見つかりません", "noChunksAlt": "チャンクが見つかりません", - "goToSources": "ソースへ移動", + "goToSources": "ナレッジへ移動", "uploadNew": "新規アップロード", "noResults": "結果が見つかりません", "fileName": "名前", @@ -426,11 +430,9 @@ "chunkTitleHint": "回答がこのチャンクを引用するときの名前です。", "files": "ファイル", "filesLoadError": "ファイルを読み込めませんでした", - "filesByline_one": "{{files}} ファイル · {{tokens}} トークン", "filesByline_other": "{{files}} ファイル · {{tokens}} トークン", "filterFiles": "ファイルを絞り込む", "searchChunks": "チャンクを検索", - "chunkCount_one": "{{formatted}} 件のチャンク", "chunkCount_other": "{{formatted}} 件のチャンク", "editor": { "write": "編集", @@ -445,6 +447,11 @@ "editChunkDescription": "{{file}} · チャンク {{n}} · {{tokens}} トークン", "previousChunk": "前のチャンク", "nextChunk": "次のチャンク", + "paused": "同期を一時停止中です。再開するには {{name}} を再接続してください。", + "emptyHint": "ファイルをアップロードするか、ウェブサイトを追加するか、サービスを接続してコンテンツを同期しましょう。", + "connectService": "サービスを接続", + "filesByline_one": "{{files}} ファイル · {{tokens}} トークン", + "chunkCount_one": "{{formatted}} 件のチャンク", "viewConfig": "ソース設定を表示", "errors": { "forbidden": "このソースでその操作を行う権限がありません。", @@ -675,10 +682,6 @@ "createTeamError": "チームを作成できませんでした。", "noTeams": "チームはまだありません。", "noDescription": "説明なし", - "memberCount_one": "{{formatted}}人のメンバー", - "memberCount_other": "{{formatted}}人のメンバー", - "sharedCount_one": "{{formatted}}件の共有", - "sharedCount_other": "{{formatted}}件の共有", "loadError": "チームの読み込みに失敗しました。", "roleAdmin": "管理者", "roleMember": "メンバー", @@ -758,8 +761,46 @@ "teams": "チーム", "people": "ユーザー", "editors": "編集者" + }, + "uses": { + "title": "このエージェントが使うもの", + "intro": "このエージェントを使うすべての人が、表示されたアクセスでこれらを使います。", + "stopped": "停止中", + "writesOff": "API 経由は不可", + "writesNote": "コネクタの「許可」はチャットに適用されます。「API 経由は不可」と示された変更は、Access Details で許可するまで、API、ウィジェット、公開リンクの利用者には行えません。", + "writesNoteEditor": "コネクタの「許可」はチャットに適用されます。「API 経由は不可」と示された変更は、オーナーが Access Details で許可するまで、API、ウィジェット、公開リンクの利用者には行えません。", + "access": { + "you": "あなたのアクセス", + "person": "{{person}} のアクセス", + "other": "他の人のアクセス", + "owner": "オーナーのアクセス", + "member": "各自の {{service}} アカウント（API とウィジェット：あなたのもの）", + "memberNoService": "各自のアカウント（API とウィジェット：あなたのもの）", + "yourAccount": "あなたの {{service}} アカウント", + "yourAccountNoService": "あなたのアカウント", + "personAccount": "{{person}} の {{service}} アカウント", + "personAccountNoService": "{{person}} のアカウント", + "memberShared": "各自の {{service}} アカウント（API とウィジェット：オーナーのもの）", + "memberSharedNoService": "各自のアカウント（API とウィジェット：オーナーのもの）", + "otherAccount": "他の人の {{service}} アカウント", + "otherAccountNoService": "他の人のアカウント", + "yourCredentials": "あなたの保存済み認証情報", + "personCredentials": "{{person}} の保存済み認証情報", + "otherCredentials": "他の人の保存済み認証情報" + }, + "writesSomeOff": "一部は API 経由不可", + "adminOff": "管理者が変更をオフにしています", + "writesNoteApi": "コネクタの「許可」はチャットに適用されます。「API 経由は不可」と示された変更は、Access Details で許可するまで、API とウィジェットの利用者には行えません。", + "writesNoteApiEditor": "コネクタの「許可」はチャットに適用されます。「API 経由は不可」と示された変更は、オーナーが Access Details で許可するまで、API とウィジェットの利用者には行えません。", + "writesNoteMemberTail": "各自のアカウントを使うツールでは、公開リンクの利用者は自分のアカウントで変更します。", + "openAccessDetails": "Access Details を開く", + "accessDetailsNeedsKey": "エージェントに API キーがあると、Access Details にこれらの変更が表示されます。" } }, + "memberCount_one": "{{formatted}}人のメンバー", + "memberCount_other": "{{formatted}}人のメンバー", + "sharedCount_one": "{{formatted}}件の共有", + "sharedCount_other": "{{formatted}}件の共有", "accessChangeError": "アクセス権を変更できませんでした。", "accessSettings": { "title": "アクセス設定", @@ -807,7 +848,8 @@ }, "viewers_can_use_in_agents": { "label": "閲覧者が自分のエージェントで使える", - "description": "あなたの認証情報で実行されます。" + "description": "あなたの認証情報で実行されます。", + "descriptionMember": "各自が自分のアカウントを使います。" } }, "prompt": { @@ -929,11 +971,11 @@ }, "tools": { "label": "ツール", - "subtitle": "エージェントを強化するツールやインテグレーションを検索、接続、設定", + "subtitle": "エージェントが実行できる操作です。組み込みツールはここにあります。接続したサービスのツールは、そのサービスの下にまとめて表示されます。", "searchPlaceholder": "検索...", "addTool": "ツールを追加", "noToolsFound": "ツールが見つかりません", - "selectToolSetup": "設定するツールを選択してください", + "selectToolSetup": "ツールを追加", "settingsIconAlt": "設定アイコン", "configureToolAria": "{{toolName}}を設定", "toggleToolAria": "{{toolName}}を切り替え", @@ -945,8 +987,8 @@ "builtIn": "ビルトイン", "authStatus": { "connected": "接続済み", - "needsAuth": "要認証", - "configured": "設定済み" + "needsAuth": "再接続", + "configured": "接続済み" }, "deleteWarning": "ツール \"{{toolName}}\" を削除してもよろしいですか？", "unsavedChanges": "保存されていない変更があります。保存せずに離れると失われます。", @@ -1062,6 +1104,11 @@ }, "headerValuePlaceholder": "例: application/json", "toolIconTitle": "{{name}}のアイコン", + "groupBuiltIn": "組み込み", + "groupService": "サービスから", + "browseConnectors": "すべてのコネクタを見る", + "viaConnection": "{{name}} 経由", + "off": "オフ", "view": "表示", "inMyChats": "自分のチャットで使用", "useInMyChatsAria": "{{toolName}} を自分のチャットで使用", @@ -1297,13 +1344,310 @@ "subtitle": "「{{name}}」の新しいシークレットの準備ができました。以前のものは使用できません。" } } + }, + "connectors": { + "label": "コネクタ", + "subtitle": "DocsGPT をチームで使っているサービスに接続します。接続ごとに、コンテンツをナレッジへ同期したり、エージェントにツールを提供したりできます。", + "search": "コネクタを検索...", + "addCustom": "カスタムツールを追加", + "custom": { + "custom_mcp": "MCP サーバー", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "カテゴリでコネクタを絞り込む", + "categories": { + "all": "すべて", + "connected": "接続済み", + "files": "ファイルとストレージ", + "knowledge": "ドキュメントと Wiki", + "dev": "開発者ツール", + "messaging": "メッセージ", + "database": "データベース", + "search": "検索とウェブ", + "custom": "カスタム", + "projects": "プロジェクトと課題", + "business": "決済とビジネス" + }, + "capability": { + "sync": "同期", + "read": "読み取り", + "write": "書き込み" + }, + "status": { + "connect": "接続", + "connected": "接続済み", + "connectedCount": "{{formatted}} 件接続済み", + "reconnect": "再接続", + "needsAdminSetup": "管理者の設定が必要", + "disabledByAdmin": "管理者により無効", + "custom": "カスタム" + }, + "connectionStatus": { + "connected": "接続済み", + "reconnect_needed": "再接続が必要", + "disconnected": "切断済み", + "error": "エラー", + "pending": "サインイン待ち" + }, + "publisher": { + "built_in": "組み込み", + "preset": "プリセット", + "custom": "カスタム" + }, + "descriptions": { + "google_drive": "ドキュメント、スプレッドシート、PDF をナレッジに同期します。", + "share_point": "SharePoint サイトと OneDrive のファイルをナレッジに同期します。", + "confluence": "Confluence のスペースとページをナレッジに同期します。", + "s3": "S3 バケットのドキュメントをナレッジに同期します。", + "reddit": "検索条件に一致する Reddit の投稿をナレッジに同期します。", + "brave": "Brave Search API でウェブと画像を検索します。", + "telegram": "Telegram のチャットにメッセージや画像を送信します。", + "ntfy": "ntfy サーバー経由でプッシュ通知を送信します。", + "postgres": "Postgres データベースのスキーマを読み取り、SQL を実行します。", + "custom_mcp": "任意のリモート Model Context Protocol サーバーに接続します。", + "custom_openapi": "OpenAPI 仕様をインポートし、そのエンドポイントをツールとして呼び出します。", + "mcp_notion": "Notion のページやデータベースを検索、閲覧、更新します。", + "mcp_linear": "課題とドキュメントをナレッジに同期し、エージェントが課題を検索、作成、更新できるようにします。", + "mcp_atlassian": "Jira の課題と Confluence のページを検索、更新します。", + "mcp_sentry": "Sentry の課題、イベント、リリースを調べます。", + "mcp_asana": "Asana のタスクやプロジェクトを検索、更新します。", + "mcp_stripe": "Stripe の顧客、支払い、サブスクリプションを調べます。", + "github": "リポジトリをナレッジに同期し、エージェントがコード、Issue、プルリクエストを読めるようにします。" + }, + "empty": "まだ何も接続されていません。サービスを接続すると、コンテンツを同期したり、エージェントに新しいツールを提供したりできます。", + "noMatches": "検索に一致するコネクタはありません。", + "loadFailed": "コネクタを読み込めませんでした。", + "askAdmin": "管理者に設定を依頼してください。", + "setupSettings": "次のサーバー設定を行い、DocsGPT を再起動してください：", + "setupGuide": "設定ガイド", + "unverified": "このサーバーは DocsGPT による確認を受けていません。信頼できるサーバーにのみ接続してください。", + "detail": { + "accounts": "アカウント", + "connectedAs": "{{account}} として接続中", + "connectAnother": "別のアカウントを接続", + "sources": "この接続からのナレッジ", + "tools": "この接続のツール", + "noAccounts": "まだアカウントが接続されていません。", + "failed": "この接続を読み込めませんでした。", + "lastSync": "{{date}} に同期 · {{frequency}}", + "paused": "同期を一時停止中", + "toolOn": "オン", + "toolOff": "オフ", + "accountMenu": "アカウントの操作", + "disconnect": "切断", + "keyEnding": "末尾が {{hint}} のキー", + "syncMore": "さらに同期する", + "refreshTools": "ツールを更新", + "noSources": "この接続からはまだ何も同期されていません。", + "remove": "削除", + "refreshed": "ツールを更新しました: {{added}} 件追加、{{removed}} 件削除。", + "refreshFailed": "ツールを更新できませんでした。", + "syncNow": "今すぐ同期", + "syncStarted": "{{name}} を同期しています…", + "syncFailed": "同期を開始できませんでした。もう一度お試しください。", + "toolSwitchFailed": "ツールを変更できませんでした。もう一度お試しください。", + "account": "アカウント: {{account}}", + "expired": "サインインの有効期限が切れました。この接続を使い続けるには再接続してください。", + "broken": "この接続は動作しなくなりました。再接続してもう一度お試しください。", + "addTools": "ツールを追加", + "noTools": "この接続のツールはまだありません。", + "addToolsFailed": "ツールを追加できませんでした。もう一度お試しください。", + "rename": "名前を変更", + "accountPicker": "アカウントを選択" + }, + "permission": { + "always": "許可", + "ask": "事前に確認", + "off": "オフ", + "label": "{{action}} の権限", + "saveFailed": "権限を保存できませんでした。もう一度お試しください。", + "groupLabel": "{{group}} すべての権限", + "fold": "アクションを隠す", + "hint": "「許可」と「事前に確認」はチャットに適用されます。「事前に確認」では、DocsGPT がアクションの実行前に確認します。エージェントの API、ウィジェット、公開リンクからの変更は、エージェントごとに Access Details で許可します。", + "actionCount_other": "{{formatted}} 件のアクション", + "customize_other": "{{formatted}} 件を個別に設定" + }, + "disconnect": { + "title": "{{name}} を切断しますか？", + "body_zero": "この接続のツールは、再接続するまで使えなくなります。", + "body_other": "同期した {{formatted}} 件はナレッジに残りますが、更新されなくなります。この接続のツールは再接続するまで動作しません。", + "failed": "切断できませんでした。もう一度お試しください。" + }, + "fields": { + "token": "API キー", + "aws_access_key_id": "アクセスキー ID", + "aws_secret_access_key": "シークレットアクセスキー", + "region": "リージョン", + "endpoint_url": "カスタムエンドポイント URL", + "client_id": "クライアント ID", + "client_secret": "クライアントシークレット", + "user_agent": "ユーザーエージェント", + "bucket": "バケット", + "prefix": "パスのプレフィックス", + "search_queries": "検索クエリ", + "number_posts": "投稿数", + "telegram_token": "ボットトークン", + "ntfy_token": "アクセストークン", + "postgres_token": "接続文字列", + "brave_token": "API キー", + "github_access_token": "個人用アクセストークン", + "repo_url": "リポジトリ", + "telegram_chat_id": "既定のチャット ID" + }, + "wizard": { + "connectTitle": "{{name}} に接続", + "reconnectTitle": "{{name}} に再接続", + "signIn": "{{name}} にサインイン", + "chooseWhatToSync": "同期する内容を選択", + "skip": "今はスキップ", + "syncFrequency": "同期", + "doneTitle": "{{name}} に接続しました", + "doneSummary": "{{account}} として接続しました。{{sources}} をナレッジに同期中、{{tools}} が利用可能です。", + "doneSummaryNone": "{{account}} として接続しました。", + "sourcesCount_other": "{{formatted}} 件", + "toolsCount_other": "{{formatted}} 個のツール", + "toolsHeading": "ツール ({{formatted}})", + "tryInChat": "チャットで試す", + "done": "完了", + "connectFailed": "接続できませんでした。入力内容を確認して、もう一度お試しください。", + "syncFailed": "同期を開始できませんでした。もう一度お試しください。", + "connectedWithKey": "末尾が {{hint}} のキーで接続しました。", + "doneCounts": "{{sources}} をナレッジに同期中、{{tools}} が利用可能です。", + "doneSources": "{{sources}} をナレッジに同期中です。", + "doneTools": "{{tools}}を利用できます。", + "waiting": "開いたウィンドウで {{name}} へのサインインを完了してください。", + "openSignIn": "{{name}} のサインインを開く", + "chooseWhatToSetUp": "設定する内容を選択", + "continue": "続行", + "methodLabel": "接続方法", + "methodOauth": "{{name}} でサインイン", + "methodToken": "アクセストークンを使用", + "credentialsRejected": "{{name}} がこの認証情報を受け付けませんでした。確認してもう一度お試しください。", + "toolsUnavailable": "{{name}} のツールに接続できませんでした。もう一度試すか、オフにして続行してください。", + "addTools": "エージェントに {{name}} の使用を許可", + "addToolsDescription": { + "read": "エージェントは情報を参照できます。変更はできません。", + "readWrite": "エージェントは情報の参照と変更ができます。変更にはあなたの承認が必要です。" + }, + "accountName": "このアカウントの名前", + "accountNameHint": "任意。あなたと AI がアカウントを区別できるようにします。例: 通知ボット", + "syncToKnowledge": "ナレッジに同期", + "syncToKnowledgeDescription": "{{name}} から回答で検索するコンテンツを選びます。オフのままなら接続のみ行い、後でコネクタから同期できます。", + "syncLater": "まだナレッジには何も同期されていません。コンテンツを追加するには、コネクタで {{name}} を開き、「さらに同期する」を選択してください。", + "retrievalSettings": "検索の詳細設定" + }, + "remove": { + "title": "{{name}} を削除しますか？", + "description": "保存されたサインイン情報は削除されます。この接続から提供されているものをどうするか選んでください。", + "sourcesLabel_other": "ナレッジに同期した {{formatted}} 件", + "keepSources": "コンテンツを保持（同期を停止）", + "deleteSources": "削除", + "toolsLabel_other": "{{formatted}} 個のツール", + "deleteTools": "削除", + "keepTools": "保持してオフにする", + "failed": "接続を削除できませんでした。もう一度お試しください。" + }, + "health": { + "reconnect": "同期を続けるには {{name}} を再接続してください。", + "reconnectSources_other": "{{name}} を再接続してください: {{formatted}} 件がナレッジへの同期を停止しています", + "reconnectTools": "エージェントが再び使えるよう {{name}} を再接続してください", + "reconnectBoth_other": "{{name}} を再接続してください: {{formatted}} 件の同期が停止し、エージェントも使えません", + "navDot": "接続に対応が必要です", + "signInAgain": "再度サインイン", + "pickerNotice": "{{name}} に再度サインインしてください。" + }, + "error": { + "decrypt": "保存された認証情報を復号できませんでした。続行するには再接続してください。", + "defaultKey": "サービスを接続する前に、管理者が ENCRYPTION_SECRET_KEY を設定する必要があります。" + }, + "sharing": { + "owner": "全員があなたのアカウントを使う", + "member": "各自が自分のアカウントを接続する", + "ownerShort": "あなたのアカウント", + "memberShort": "各自のもの", + "ownerShortShared": "オーナーのアカウント" + }, + "share": { + "heading": "実行に使うアカウント", + "ownerWarning": "チームメンバーは {{name}} で {{account}} として操作します。", + "ownerWarningShared": "チームメンバーは所有者の {{name}} アカウントで操作します。", + "confirmWrite": "チームメンバーが自分のアカウントで操作を行えることを理解しました。", + "confirmWriteShared": "チームメンバーが所有者の {{name}} アカウントで操作を行えることを理解しました。", + "memberNote": "各自が自分の {{name}} アカウントを使い、このツールを初めて使うときに接続します。エージェントの API キーやウィジェット経由では、エージェントのオーナーのアカウントが使われます。", + "forced": "このコネクタのすべての共有に対して管理者が設定しています。", + "ownerChooses": "これを変更できるのは所有者だけです。", + "saveFailed": "変更を保存できませんでした。もう一度お試しください。" + }, + "createFailed": "コネクタを追加できませんでした。もう一度お試しください。", + "capabilityFilter": { + "sync": "ナレッジにコンテンツを同期するコネクタを表示しています。", + "tools": "エージェントにツールを提供するコネクタを表示しています。", + "showAll": "すべて表示" + }, + "capabilityPlain": { + "sync": "ナレッジに同期", + "read": "情報を調べる", + "write": "操作を行う" + }, + "github": { + "tokenHint": "使用するリポジトリに対して Contents と Metadata の読み取り権限を持つ fine-grained トークンを作成してください。エージェントに変更を許可する場合は、Issues と Pull requests の読み取りと書き込みの権限も付与してください。", + "writes": "エージェントによる変更も許可する（Issue、コメント、プルリクエスト）", + "writesDescription": "変更は既定で先にあなたの承認を求めます。トークンまたは GitHub App に Issues と Pull requests の読み取りと書き込みの権限が必要です。", + "writesForbidden": "管理者が GitHub での変更をオフにしています。", + "writesFailed": "GitHub でエージェントに許可する操作を変更できませんでした。", + "createToken": "GitHub でトークンを作成", + "chooseRepositories": "GitHub でリポジトリを選択", + "noAppRepositories": "DocsGPT からはまだリポジトリが見えません。読み取りを許可するリポジトリを GitHub で選択してください。", + "noTokenRepositories": "このトークンで読み取れるリポジトリがありません。GitHub でアクセス権を付与してから再接続してください。", + "searchRepositories": "リポジトリを検索", + "repositories": "リポジトリ", + "updated": "{{date}} に更新", + "loadFailed": "リポジトリを読み込めませんでした。" + }, + "linear": { + "teams": "チーム", + "teamsHint": "これらのチームの課題を同期します。", + "projects": "プロジェクト", + "projectsHint": "これらのプロジェクトの課題を同期します。", + "search": "チームとプロジェクトを検索", + "noMatches": "検索に一致するチームやプロジェクトはありません。", + "includeComments": "コメントを含める", + "includeCommentsDescription": "各課題にそのコメントを追加します。", + "includeDocuments": "プロジェクトのドキュメントを含める", + "includeDocumentsDescription": "選択したプロジェクトの Linear ドキュメントを同期します。", + "loadFailed": "Linear のチームとプロジェクトを読み込めませんでした。", + "empty": "この Linear アカウントには同期できるチームやプロジェクトがありません。" + }, + "parameters": { + "show": "パラメーター", + "hide": "パラメーターを隠す", + "fixedCount_other": "{{formatted}} 件固定", + "ai": "AI に任せる", + "fixed": "常に使用", + "choiceLabel": "{{parameter}} の入力方法", + "valueLabel": "{{parameter}} の値", + "save": "保存", + "saveFailed": "値を保存できませんでした。もう一度お試しください。", + "hint": "「常に使用」の値は毎回の呼び出しで送信されます。AI はこの値を見ることも変更することもできません。", + "fromAccount": "このアカウントで設定済み: {{value}}" + }, + "fieldHints": { + "telegram_chat_id": "任意。メッセージはこのチャットに送られ、AI は別のチャットを選べません。ボットをチャットに追加してメッセージを送り、https://api.telegram.org/bot/getUpdates でチャットの id を確認してください。" + }, + "rename": { + "title": "アカウント名を変更", + "description": "この名前で、同じサービスの他のアカウントと区別できます（あなたと AI の両方）。空欄にするとアカウントそのものが表示されます。", + "label": "アカウント名", + "save": "保存", + "failed": "アカウント名を変更できませんでした。もう一度お試しください。" + } } }, "modals": { "uploadDoc": { - "label": "新しい文書をアップロードする", + "label": "ナレッジを追加", "select": "ドキュメントを DocsGPT にアップロードする方法を選択します", - "selectSource": "ソースを追加する方法を選択してください", + "selectSource": "ナレッジを追加", "selectedFiles": "選択されたファイル", "noFilesSelected": "ファイルが選択されていません", "file": "デバイスからアップロード", @@ -1318,19 +1662,11 @@ "filesRejected": "追加されませんでした（25 MB を超えているか、未対応の形式です）: {{files}}", "uploadedFiles": "アップロードされたファイル", "cancel": "キャンセル", - "train": "トレーニング", + "train": "ナレッジに追加", "create": "作成", "link": "リンク", "urlLink": "URLリンク", "repoUrl": "リポジトリURL", - "reddit": { - "id": "クライアントID", - "secret": "クライアントシークレット", - "agent": "ユーザーエージェント", - "searchQueries": "検索クエリ", - "numberOfPosts": "投稿数", - "addQuery": "クエリを追加" - }, "drag": { "title": "添付ファイルをここにドロップ", "description": "リリースして添付ファイルをアップロード" @@ -1352,8 +1688,6 @@ "clear": "クリア", "ingestionFailed": "取り込みに失敗しました。" }, - "showAdvanced": "詳細オプションを表示", - "hideAdvanced": "詳細オプションを非表示", "ingestors": { "local_file": { "label": "ファイルをアップロード", @@ -1371,26 +1705,6 @@ "label": "GitHub", "heading": "GitHubからコンテンツを追加" }, - "reddit": { - "label": "Reddit", - "heading": "Redditからコンテンツを追加" - }, - "google_drive": { - "label": "Google Drive", - "heading": "Google Driveからアップロード" - }, - "s3": { - "label": "Amazon S3", - "heading": "Amazon S3からコンテンツを追加" - }, - "share_point": { - "label": "SharePoint", - "heading": "SharePointからアップロード" - }, - "confluence": { - "label": "Confluence", - "heading": "Confluenceからアップロード" - }, "wiki": { "label": "新しいWiki", "heading": "リビングWikiを作成" @@ -1444,13 +1758,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "AWS アクセスキー ID", - "awsSecretAccessKey": "AWS シークレットアクセスキー", - "bucket": "バケット名", - "prefix": "パスプレフィックス（任意）", - "region": "AWS リージョン", - "endpointUrl": "カスタムエンドポイント URL（任意）", "initialContent": "初期コンテンツ（任意）" + }, + "github": { + "privateHint": "プライベートリポジトリですか？同期するには GitHub アカウントを接続してください。", + "connectedHint": "プライベートリポジトリは GitHub 接続（{{account}}）から同期されます。", + "connect": "GitHub を接続", + "pickRepository": "リポジトリを選択" + }, + "connectData": { + "title": "データを接続", + "description": "{{services}} を同期", + "more": "その他", + "intro": "サービスを選んでサインインし、ナレッジに同期する内容を選びます。", + "connectedAs": "{{account}} として接続中", + "browseAll": "すべてのコネクタを見る" } }, "createAPIKey": { @@ -1487,8 +1809,8 @@ "configTool": { "title": "ツール設定", "type": "タイプ", - "apiKeyLabel": "APIキー / OAuth", - "apiKeyPlaceholder": "APIキー / OAuthを入力してください", + "apiKeyLabel": "API キー", + "apiKeyPlaceholder": "API キーを入力", "addButton": "ツールを追加", "closeButton": "閉じる", "customNamePlaceholder": "Enter custom name (optional)", @@ -1566,6 +1888,24 @@ "learnMore": "Learn more", "resetKey": "キーをリセット", "resetKeyConfirm": "APIキーをリセットしてもよろしいですか？現在のキーは直ちに無効になり、この操作は元に戻せません。", + "apiWrites": { + "title": "他の人があなたとして行える変更", + "description": "API キー、ウィジェット、エージェントの公開リンクでは誰もあなたの代わりに操作を承認できないため、接続済みアカウントや保存済みの認証情報を使う変更は、ここで許可しない限りブロックされます。", + "saveFailed": "保存できませんでした。もう一度お試しください。", + "summaryNone": "まだ何も許可されていません", + "summaryTools_one": "{{tools}} は変更を実行できます", + "summaryTools_other": "{{tools}} は変更を実行できます", + "summaryCount": "{{formatted}} 件中 {{allowed}} 件を許可", + "toolCount": "API、ウィジェット、公開リンクからの変更 · {{formatted}} 件中 {{allowed}} 件を許可", + "toolLabel": "{{tool}} で API、ウィジェット、公開リンクからの変更を許可", + "choice": { + "off": "オフ", + "all": "すべて" + }, + "customize_one": "変更を個別に設定", + "customize_other": "{{formatted}} 件を個別に設定", + "fold": "変更を隠す" + }, "actionFailed": "うまくいきませんでした。もう一度お試しください。", "apiKeyAfterPublish": "APIキーを作成するには、エージェントを公開してください。" }, @@ -1659,13 +1999,17 @@ }, "sources": { "title": "ソース", - "text": "ソーステキスト", + "text": "ナレッジを選択", "link": "ソースリンク", "view_more": "さらに{{count}}個のソース", - "noSourcesAvailable": "利用可能なソースがありません", - "selectedCount": "{{count}} 件のソースを選択中", - "groupOwn": "自分のソース", - "groupTeam": "チームと共有" + "noSourcesAvailable": "ナレッジはまだありません", + "selectedCount": "{{count}} 件選択中", + "groupOwn": "あなたのもの", + "groupTeam": "チームと共有", + "connectMore": "さらに接続", + "fromConnector": "{{name}} から", + "fromConnectorTitle": "{{name}} から: {{title}}", + "knowledge": "ナレッジ" }, "attachments": { "attach": "添付", @@ -1706,7 +2050,9 @@ "accessingMemory": "メモリにアクセス中…", "runningCode": "コードを実行中…", "creatingArtifact": "アーティファクトを作成中…", - "usingTool": "{{tool}} を使用中…" + "usingTool": "{{tool}} を使用中…", + "searchingConnector": "{{name}} を検索中...", + "usingConnector": "{{name}} を使用中..." }, "toolChip": { "searchingWeb": "ウェブ検索済み · 「{{query}}」", @@ -1717,7 +2063,10 @@ "accessingMemory": "メモリにアクセス済み", "runningCode": "コード実行済み", "creatingArtifact": "アーティファクト作成済み", - "usingTool": "{{tool}} を使用済み" + "usingTool": "{{tool}} を使用済み", + "searchedConnector": "{{name}} を検索しました", + "readConnector": "{{name}} から読み取りました", + "usedConnector": "{{name}} を使用: {{action}}" }, "inlineSteps": { "arguments": "引数", @@ -1767,7 +2116,12 @@ "approveAlways": "承認し、今後は確認しない", "deny": "拒否", "details": "詳細", - "denyReasonPlaceholder": "拒否の理由（任意）…" + "denyReasonPlaceholder": "拒否の理由（任意）…", + "connectTitle": "このツールには {{name}} アカウントが必要です。", + "connect": "{{name}} に接続", + "continue": "続行", + "skip": "スキップ", + "thisService": "このサービス" }, "research": { "title": "リサーチ", @@ -1782,7 +2136,6 @@ "agents": { "title": "エージェント", "edit": "編集", - "view": "表示", "card": { "pin": "エージェントをピン留め", "unpin": "エージェントのピン留めを解除", @@ -1876,7 +2229,7 @@ "placeholders": { "agentName": "エージェント名", "describeAgent": "エージェントを説明してください", - "selectSources": "ソースを選択", + "selectSources": "ナレッジを選択", "chunksPerQuery": "クエリごとのチャンク数", "selectType": "タイプを選択", "selectTools": "ツールを選択", @@ -1886,20 +2239,23 @@ "enterRequestLimit": "リクエスト制限を入力" }, "sourcePopup": { - "title": "ソースを選択", - "searchPlaceholder": "ソースを検索...", - "noOptionsMessage": "利用可能なソースがありません", + "title": "ナレッジを選択", + "searchPlaceholder": "ナレッジを検索...", + "noOptionsMessage": "ナレッジはまだありません", "groupOwn": "自分のソース", "groupTeam": "チームと共有", - "noSourceHint": "任意。ソースがない場合、エージェントはモデルとツールのみで回答します。" + "noSourceHint": "任意です。ナレッジがない場合、エージェントはモデルとツールだけで回答します。" }, "toolsPopup": { "title": "ツールを選択", "searchPlaceholder": "ツールを検索...", "noOptionsMessage": "利用可能なツールがありません", "groupBuiltin": "ビルトイン", - "groupDefault": "デフォルト", - "groupCustom": "カスタム" + "groupDefault": "組み込み", + "groupCustom": "カスタム", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "このエージェントに追加済み", + "attachedHint": "他のユーザーが追加しました。削除できます。" }, "modelsPopup": { "title": "モデルを選択", @@ -1919,7 +2275,7 @@ "defaultModel": "デフォルトモデル", "name": "名前", "description": "説明", - "sources": "ソース", + "sources": "ナレッジ", "avatar": "アバター" }, "advanced": { @@ -2007,11 +2363,69 @@ "published": "公開済み" }, "sponsors": { - "attachNote": "自分のライブラリから追加したツール、ソース、プロンプトは、このエージェントを使うすべての人に対してあなたのアクセス権で実行されます。", + "attachNote": "オーナーが使用できないツール、ソース、プロンプトを追加すると、このエージェントを使うすべての人に対してあなたのアクセス権で実行されます。自分が所有しているか編集できるものに限られ、追加時に確認を求められます。", "publicLinkNote": "このエージェントには公開リンクがあるため、リンクを知っている人は誰でもそれらから回答を得られます。", "addedBy": "{{person}} が追加: {{names}}", - "unavailable": "実行されていません: {{names}}。追加した {{person}} はアクセス権を失いました。削除するか、オーナーが使用できるものを選んでください。", - "unknownItem": "名前のない項目" + "addedByOther": "他の人が追加: {{names}}", + "unknownItem": "名前のない項目", + "notAllowed": "{{names}} をこのエージェントに追加できません。オーナーはそれらを使用できず、共有できるのは自分が所有しているか編集できるものだけです。", + "takeOver": "{{name}} を自分のアクセス権で実行", + "takeOverPending": "保存すると、{{name}} はあなたのアクセス権で実行されます。", + "undoTakeOver": "元に戻す", + "confirmationOutdated": "あなたのアクセス権で実行するものを確認した後に、このエージェントが変更されました。通知を確認して、もう一度保存してください。" + }, + "resourceStates": { + "title": "一部のツール、ソース、プロンプトが実行されていません", + "reason": { + "deleted": "{{name}} は削除されました。", + "ownerLostAccessYou": "{{name}} を使用できなくなりました。", + "ownerLostAccess": "エージェントの所有者は {{name}} を使用できなくなりました。", + "sponsorCannotEditAgent": "{{name}} は {{person}} のアクセス権で実行されていましたが、この人はこのエージェントを編集できなくなりました。", + "sponsorCannotEditAgentOther": "{{name}} は他の人のアクセス権で実行されていましたが、この人はこのエージェントを編集できなくなりました。", + "sponsorCannotEditItem": "{{name}} は {{person}} のアクセス権で実行されていましたが、この人はそれを編集できなくなりました。", + "sponsorCannotEditItemOther": "{{name}} は他の人のアクセス権で実行されていましたが、この人はそれを編集できなくなりました。", + "connectionNeedsReconnect": "{{name}} の {{service}} アカウントに再度サインインする必要があります。", + "connectionRemoved": "{{name}} が使っていた {{service}} アカウントは削除されました。", + "connectorDisabled": "管理者が {{service}} をオフにしたため、{{name}} は実行できません。", + "unknown": "{{name}} は実行されていません。" + }, + "ask": { + "shareAgain": "{{person}} に再度共有するよう依頼してください。", + "signInAgain": "{{person}} に再度サインインするよう依頼してください。", + "connectAgain": "{{person}} にアカウントを再接続するよう依頼してください。", + "admin": "管理者にオンに戻すよう依頼してください。", + "shareAgainOwner": "所有者に再度共有するよう依頼してください。", + "signInAgainOwner": "所有者に再度サインインするよう依頼してください。", + "connectAgainOwner": "所有者にアカウントを再接続するよう依頼してください。" + }, + "promptFallback": "それまでエージェントはデフォルトのプロンプトを使用します。", + "serviceFallback": "サービス", + "remove": "削除", + "removeLabel": "{{name}} を削除", + "reconnectLabel": "{{name}} を再接続", + "unnamed": { + "tool": "ツール {{id}}", + "source": "ソース {{id}}", + "prompt": "プロンプト {{id}}" + }, + "chip": "実行されていません: {{count}}" + }, + "sponsorConfirm": { + "title": "あなたのアクセス権で実行しますか？", + "description": "エージェントのオーナーはこれらを使用できません。追加すると、このエージェントを使うすべての人に対してあなたのアクセス権で実行されます。", + "audienceTitle": "このエージェントを通じて使用できる人", + "audienceTeams": "{{teams}} のメンバー", + "audienceApiKey": "エージェントの API キーを持つすべての人（ウェブサイトウィジェットを含む）", + "audiencePublicLink": "エージェントの公開リンクを知っているすべての人", + "audienceWebhook": "エージェントの Webhook を呼び出すすべてのもの", + "audienceEditors": "このエージェントを開けるすべての人", + "stopNote": "それらやこのエージェントの編集権限を失うと、実行されなくなります。その後は、編集できる人が引き継ぐか削除されるまで、追加されたまま何もしません。", + "confirm": "自分のアクセス権で追加", + "types": { + "tool": "ツール", + "source": "ソース", + "prompt": "プロンプト" + } } }, "logs": { @@ -2303,7 +2717,7 @@ "outputVariable": "出力変数", "outputVariablePlaceholder": "出力の変数名", "streamToUser": "出力をユーザーにストリーミング", - "sources": "ソース", + "sources": "ナレッジ", "documents": "ドキュメント", "documentsHint": "アップロードまたは上流ノードからこのエージェントに渡されるドキュメント。", "filePassing": "ファイルの受け渡し", @@ -2342,7 +2756,8 @@ "detailsTitle": "ワークフローの詳細", "detailsDescription": "エージェントカードに表示される名前、説明、画像です。", "unsavedChanges": "未保存の変更", - "editDetailsMenu": "詳細を編集" + "editDetailsMenu": "詳細を編集", + "attachedOption": "{{name}}（他のユーザーが追加）" }, "validation": { "nameRequired": "ワークフロー名は必須です", @@ -2448,6 +2863,7 @@ "workflowTitle": "ワークフローエージェント", "workflowDescription": "複数のモデル、条件ロジック、状態管理を使った複雑なマルチステップのワークフローを設計します" }, + "view": "表示", "deleteFailed": "エージェントを削除できませんでした。もう一度お試しください。" }, "components": { @@ -2473,9 +2889,9 @@ "empty": "ToDo はまだありません" }, "note": { - "lines_one": "{{count}} 行", "lines_other": "{{count}} 行", - "empty": "空のメモ" + "empty": "空のメモ", + "lines_one": "{{count}} 行" }, "notFound": "アーティファクトが見つかりません", "loadFailed": "アーティファクトを読み込めませんでした", @@ -2516,7 +2932,8 @@ "shared": "共有", "connectTo": "{{provider}}に接続", "sessionExpiredFor": "セッションが期限切れです。{{provider}}に再接続してください。", - "selectFilesFrom": "{{provider}}からファイルを選択" + "selectFilesFrom": "{{provider}}からファイルを選択", + "account": "アカウント" }, "actionButtons": { "openNewChat": "新しいチャットを開く", @@ -2574,5 +2991,15 @@ }, "prompts": { "textAriaLabel": "プロンプトテキスト" + }, + "connectHero": { + "title": "データを接続", + "body": "{{names}} などとチャットできます。" + }, + "common": { + "close": "閉じる", + "viewOnlyNotice": "表示はできますが、あなたのロールでは変更できません。", + "credentialsLockedNotice": "このツールの認証情報を変更できるのはオーナーだけです。", + "savedSecretHint": "保存済みです。空のままにすると保持されます。" } } diff --git a/frontend/src/locale/ru.json b/frontend/src/locale/ru.json index 8d1a61e2..9eddd91e 100644 --- a/frontend/src/locale/ru.json +++ b/frontend/src/locale/ru.json @@ -10,12 +10,6 @@ "loading": "Загрузка...", "retry": "Повторить", "cancel": "Отмена", - "common": { - "close": "Закрыть", - "viewOnlyNotice": "Вы можете это просматривать, но ваша роль не позволяет это изменить.", - "credentialsLockedNotice": "Только владелец может изменить учётные данные этого инструмента.", - "savedSecretHint": "Сохранено. Оставьте поле пустым, чтобы сохранить значение." - }, "help": "Помощь", "emailUs": "Напишите нам", "documentation": "Документация", @@ -106,16 +100,16 @@ }, "sources": { "title": "Здесь вы можете управлять всеми исходными файлами, которые доступны вам и которые вы загрузили.", - "subtitle": "Загружайте и управляйте документами и источниками знаний, которые обеспечивают ваши ответы", - "label": "Источники", + "subtitle": "Контент, по которому DocsGPT может искать и который может цитировать: загрузки, сайты, вики и контент, синхронизированный из ваших подключений.", + "label": "Знания", "name": "Название источника", "date": "Дата вектора", "type": "Тип", "tokenUsage": "Использование токена", - "noData": "Нет существующих источников", + "noData": "Знаний пока нет", "searchPlaceholder": "Поиск...", "addNew": "добавить новый", - "addSource": "Добавить источник", + "addSource": "Добавить знания", "addChunk": "Добавить фрагмент", "preLoaded": "Предзагруженный", "private": "Частный", @@ -191,7 +185,17 @@ "pagesTitle": "Страницы", "loadFailed": "Не удалось загрузить страницы вики.", "pageLoadFailed": "Не удалось загрузить эту страницу.", - "editTitle": "Редактировать страницу" + "editTitle": "Редактировать страницу", + "settings": { + "action": "Настройки вики", + "title": "Настройки вики", + "outsideEdits": { + "label": "Разрешить пользователям API и виджета редактировать эту вики", + "description": "Если выключено, те, кто пользуется агентом через его API-ключ или виджет, могут читать эту вики, но не изменять её. Посетители по публичной ссылке могут редактировать только те вики, которые могут редактировать сами, и перед каждым изменением у них спрашивают подтверждение. Вы и редакторы вики всегда могут её редактировать." + }, + "loadError": "Не удалось загрузить настройки вики.", + "saveError": "Не удалось сохранить изменение. Попробуйте ещё раз." + } }, "graphrag": { "badge": "GraphRAG", @@ -327,7 +331,7 @@ "chunks": "Фрагменты", "noChunks": "Фрагменты не найдены", "noChunksAlt": "Фрагменты не найдены", - "goToSources": "Перейти к источникам", + "goToSources": "К знаниям", "uploadNew": "Загрузить новый", "noResults": "Результаты не найдены", "fileName": "Имя", @@ -474,6 +478,9 @@ "editChunkDescription": "{{file}} · фрагмент {{n}} · токенов: {{tokens}}", "previousChunk": "Предыдущий фрагмент", "nextChunk": "Следующий фрагмент", + "paused": "Синхронизация приостановлена. Переподключите {{name}}, чтобы продолжить.", + "emptyHint": "Загрузите файлы, добавьте сайт или подключите сервис, чтобы синхронизировать его содержимое.", + "connectService": "Подключить сервис", "viewConfig": "Посмотреть настройки источника", "errors": { "forbidden": "У вас нет прав на это действие с этим источником.", @@ -718,14 +725,6 @@ "createTeamError": "Не удалось создать команду.", "noTeams": "Команд пока нет.", "noDescription": "Без описания", - "memberCount_one": "{{formatted}} участник", - "memberCount_few": "{{formatted}} участника", - "memberCount_many": "{{formatted}} участников", - "memberCount_other": "{{formatted}} участника", - "sharedCount_one": "{{formatted}} общий ресурс", - "sharedCount_few": "{{formatted}} общих ресурса", - "sharedCount_many": "{{formatted}} общих ресурсов", - "sharedCount_other": "{{formatted}} общих ресурса", "loadError": "Не удалось загрузить команды.", "roleAdmin": "администратор", "roleMember": "участник", @@ -807,8 +806,50 @@ "teams": "Команды", "people": "Люди", "editors": "Редакторы" + }, + "uses": { + "title": "Что использует этот агент", + "intro": "Все, кто пользуется этим агентом, получают это, каждое — с указанным доступом.", + "stopped": "Остановлено", + "writesOff": "Не через API", + "writesNote": "«Разрешать» у коннектора действует в чатах. Изменения с отметкой «Не через API» остаются выключены для пользователей API, виджета и публичной ссылки, пока вы не разрешите их в Access Details.", + "writesNoteEditor": "«Разрешать» у коннектора действует в чатах. Изменения с отметкой «Не через API» остаются выключены для пользователей API, виджета и публичной ссылки, пока владелец не разрешит их в Access Details.", + "access": { + "you": "Ваш доступ", + "person": "Доступ {{person}}", + "other": "Доступ другого пользователя", + "owner": "Доступ владельца", + "member": "Собственный аккаунт {{service}} у каждого (API и виджет: ваш)", + "memberNoService": "Собственный аккаунт у каждого (API и виджет: ваш)", + "yourAccount": "Ваш аккаунт {{service}}", + "yourAccountNoService": "Ваш аккаунт", + "personAccount": "Аккаунт {{service}} пользователя {{person}}", + "personAccountNoService": "Аккаунт пользователя {{person}}", + "memberShared": "Собственный аккаунт {{service}} у каждого (API и виджет: владельца)", + "memberSharedNoService": "Собственный аккаунт у каждого (API и виджет: владельца)", + "otherAccount": "Аккаунт {{service}} другого пользователя", + "otherAccountNoService": "Аккаунт другого пользователя", + "yourCredentials": "Ваши сохранённые учётные данные", + "personCredentials": "Сохранённые учётные данные {{person}}", + "otherCredentials": "Сохранённые учётные данные другого пользователя" + }, + "writesSomeOff": "Не всё через API", + "adminOff": "Изменения выключены администратором", + "writesNoteApi": "«Разрешать» у коннектора действует в чатах. Изменения с отметкой «Не через API» остаются выключены для пользователей API и виджета, пока вы не разрешите их в Access Details.", + "writesNoteApiEditor": "«Разрешать» у коннектора действует в чатах. Изменения с отметкой «Не через API» остаются выключены для пользователей API и виджета, пока владелец не разрешит их в Access Details.", + "writesNoteMemberTail": "В инструментах, где у каждого свой аккаунт, пользователи публичной ссылки вносят изменения через свой аккаунт.", + "openAccessDetails": "Открыть Access Details", + "accessDetailsNeedsKey": "Access Details покажет эти изменения, когда у агента появится API-ключ." } }, + "memberCount_one": "{{formatted}} участник", + "memberCount_few": "{{formatted}} участника", + "memberCount_many": "{{formatted}} участников", + "memberCount_other": "{{formatted}} участника", + "sharedCount_one": "{{formatted}} общий ресурс", + "sharedCount_few": "{{formatted}} общих ресурса", + "sharedCount_many": "{{formatted}} общих ресурсов", + "sharedCount_other": "{{formatted}} общих ресурса", "accessChangeError": "Не удалось изменить доступ.", "accessSettings": { "title": "Настройки доступа", @@ -856,7 +897,8 @@ }, "viewers_can_use_in_agents": { "label": "Читатели могут использовать его в своих агентах", - "description": "Он работает с вашими учётными данными." + "description": "Он работает с вашими учётными данными.", + "descriptionMember": "Каждый использует свой аккаунт." } }, "prompt": { @@ -980,11 +1022,11 @@ }, "tools": { "label": "Инструменты", - "subtitle": "Просматривайте, подключайте и настраивайте инструменты и интеграции для ваших агентов", + "subtitle": "Действия, которые могут выполнять ваши агенты. Здесь находятся встроенные инструменты. Инструменты подключённого сервиса сгруппированы под этим сервисом.", "searchPlaceholder": "Поиск...", "addTool": "Добавить инструмент", "noToolsFound": "Инструменты не найдены", - "selectToolSetup": "Выберите инструмент для настройки", + "selectToolSetup": "Добавить инструмент", "settingsIconAlt": "Значок настроек", "configureToolAria": "Настроить {{toolName}}", "toggleToolAria": "Переключить {{toolName}}", @@ -996,8 +1038,8 @@ "builtIn": "Встроенный", "authStatus": { "connected": "Подключено", - "needsAuth": "Требуется авторизация", - "configured": "Настроено" + "needsAuth": "Переподключить", + "configured": "Подключено" }, "deleteWarning": "Вы уверены, что хотите удалить инструмент \"{{toolName}}\"?", "unsavedChanges": "У вас есть несохраненные изменения, которые будут потеряны, если вы уйдете без сохранения.", @@ -1113,6 +1155,11 @@ }, "headerValuePlaceholder": "например, application/json", "toolIconTitle": "Значок {{name}}", + "groupBuiltIn": "Встроенные", + "groupService": "Из сервиса", + "browseConnectors": "Все коннекторы", + "viaConnection": "через {{name}}", + "off": "Выкл.", "view": "Просмотр", "inMyChats": "В моих чатах", "useInMyChatsAria": "Использовать {{toolName}} в моих чатах", @@ -1362,13 +1409,340 @@ "subtitle": "Новый секрет для «{{name}}» готов. Старый больше не работает." } } + }, + "connectors": { + "label": "Коннекторы", + "subtitle": "Подключите DocsGPT к сервисам, которыми пользуется ваша команда. Подключение может синхронизировать содержимое в знания и давать агентам инструменты.", + "search": "Поиск коннекторов...", + "addCustom": "Добавить свой инструмент", + "custom": { + "custom_mcp": "MCP-сервер", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "Фильтр коннекторов по категории", + "categories": { + "all": "Все", + "connected": "Подключённые", + "files": "Файлы и хранилища", + "knowledge": "Документы и вики", + "dev": "Инструменты разработчика", + "messaging": "Сообщения", + "database": "Базы данных", + "search": "Поиск и веб", + "custom": "Свои", + "projects": "Проекты и задачи", + "business": "Платежи и бизнес" + }, + "capability": { + "sync": "Синхронизация", + "read": "Чтение", + "write": "Запись" + }, + "status": { + "connect": "Подключить", + "connected": "Подключено", + "connectedCount": "Подключено: {{formatted}}", + "reconnect": "Переподключить", + "needsAdminSetup": "Нужна настройка администратором", + "disabledByAdmin": "Отключено администратором", + "custom": "Свой" + }, + "connectionStatus": { + "connected": "Подключено", + "reconnect_needed": "Нужно переподключить", + "disconnected": "Отключено", + "error": "Ошибка", + "pending": "Ожидание входа" + }, + "publisher": { + "built_in": "Встроенный", + "preset": "Готовый", + "custom": "Свой" + }, + "descriptions": { + "google_drive": "Синхронизирует Документы, Таблицы и PDF в знания.", + "share_point": "Синхронизирует файлы с сайтов SharePoint и из OneDrive в знания.", + "confluence": "Синхронизирует пространства и страницы Confluence в знания.", + "s3": "Синхронизирует документы из бакета S3 в знания.", + "reddit": "Синхронизирует публикации Reddit по вашим запросам в знания.", + "brave": "Поиск в интернете и по картинкам через Brave Search API.", + "telegram": "Отправляет сообщения и изображения в чат Telegram.", + "ntfy": "Отправляет push-уведомления через сервер ntfy.", + "postgres": "Читает схему и выполняет SQL в базе данных Postgres.", + "custom_mcp": "Подключает любой удалённый сервер Model Context Protocol.", + "custom_openapi": "Импортирует спецификацию OpenAPI и вызывает её эндпоинты как инструменты.", + "mcp_notion": "Поиск, чтение и обновление страниц и баз данных Notion.", + "mcp_linear": "Синхронизируйте задачи и документы в базу знаний и дайте агентам находить, создавать и обновлять задачи.", + "mcp_atlassian": "Поиск и обновление задач Jira и страниц Confluence.", + "mcp_sentry": "Просмотр проблем, событий и релизов Sentry.", + "mcp_asana": "Поиск и обновление задач и проектов Asana.", + "mcp_stripe": "Просмотр клиентов, платежей и подписок в Stripe.", + "github": "Синхронизируйте репозитории в базу знаний и дайте агентам читать код, issues и pull requests." + }, + "empty": "Пока ничего не подключено. Подключите сервис, чтобы синхронизировать его контент или дать агентам новые инструменты.", + "noMatches": "Нет коннекторов, подходящих под запрос.", + "loadFailed": "Не удалось загрузить коннекторы.", + "askAdmin": "Попросите администратора настроить это.", + "setupSettings": "Задайте эти настройки сервера и перезапустите DocsGPT:", + "setupGuide": "Инструкция по настройке", + "unverified": "DocsGPT не проверял этот сервер. Подключайте только серверы, которым доверяете.", + "detail": { + "accounts": "Аккаунты", + "connectedAs": "Подключено как {{account}}", + "connectAnother": "Подключить другой аккаунт", + "sources": "Знания из этого подключения", + "tools": "Инструменты из этого подключения", + "noAccounts": "Аккаунт ещё не подключён.", + "failed": "Не удалось загрузить это подключение.", + "lastSync": "Синхронизировано {{date}} · {{frequency}}", + "paused": "Синхронизация приостановлена", + "toolOn": "Вкл.", + "toolOff": "Выкл.", + "accountMenu": "Действия с аккаунтом", + "disconnect": "Отключить", + "keyEnding": "Ключ, оканчивающийся на {{hint}}", + "syncMore": "Синхронизировать ещё", + "refreshTools": "Обновить инструменты", + "noSources": "Из этого подключения пока ничего не синхронизировано.", + "remove": "Удалить", + "refreshed": "Инструменты обновлены: добавлено {{added}}, удалено {{removed}}.", + "refreshFailed": "Не удалось обновить инструменты.", + "syncNow": "Синхронизировать", + "syncStarted": "Синхронизация {{name}}…", + "syncFailed": "Не удалось запустить синхронизацию. Попробуйте ещё раз.", + "toolSwitchFailed": "Не удалось изменить инструмент. Попробуйте ещё раз.", + "account": "Аккаунт: {{account}}", + "expired": "Срок входа истёк. Переподключите, чтобы продолжить пользоваться этим подключением.", + "broken": "Это подключение перестало работать. Переподключите, чтобы попробовать снова.", + "addTools": "Добавить инструменты", + "noTools": "У этого подключения пока нет инструментов.", + "addToolsFailed": "Не удалось добавить инструменты. Попробуйте ещё раз.", + "rename": "Переименовать", + "accountPicker": "Выберите аккаунт" + }, + "permission": { + "always": "Разрешать", + "ask": "Спрашивать", + "off": "Выкл.", + "label": "Разрешение для {{action}}", + "saveFailed": "Не удалось сохранить разрешение. Попробуйте ещё раз.", + "groupLabel": "Разрешение для всего в группе «{{group}}»", + "fold": "Скрыть действия", + "hint": "«Разрешать» и «Спрашивать» действуют в чате: при «Спрашивать» DocsGPT спросит вас перед выполнением действия. Изменения через API, виджет или публичную ссылку агента разрешаются для каждого агента отдельно, в его Access Details.", + "actionCount_one": "{{formatted}} действие", + "actionCount_few": "{{formatted}} действия", + "actionCount_many": "{{formatted}} действий", + "actionCount_other": "{{formatted}} действия", + "customize_one": "Настроить действие", + "customize_few": "Настроить каждое из {{formatted}}", + "customize_many": "Настроить каждое из {{formatted}}", + "customize_other": "Настроить каждое из {{formatted}}" + }, + "disconnect": { + "title": "Отключить {{name}}?", + "body_zero": "Инструменты из этого подключения перестанут работать, пока вы не переподключитесь.", + "body_one": "Синхронизированный элемент останется в знаниях, но перестанет обновляться. Инструменты этого подключения не будут работать, пока вы не переподключитесь.", + "body_few": "{{formatted}} синхронизированных элемента останутся в знаниях, но перестанут обновляться. Инструменты этого подключения не будут работать, пока вы не переподключитесь.", + "body_many": "{{formatted}} синхронизированных элементов останутся в знаниях, но перестанут обновляться. Инструменты этого подключения не будут работать, пока вы не переподключитесь.", + "body_other": "{{formatted}} синхронизированных элемента останутся в знаниях, но перестанут обновляться. Инструменты этого подключения не будут работать, пока вы не переподключитесь.", + "failed": "Не удалось отключить. Попробуйте ещё раз." + }, + "fields": { + "token": "API-ключ", + "aws_access_key_id": "ID ключа доступа", + "aws_secret_access_key": "Секретный ключ доступа", + "region": "Регион", + "endpoint_url": "Свой URL эндпоинта", + "client_id": "ID клиента", + "client_secret": "Секрет клиента", + "user_agent": "User agent", + "bucket": "Бакет", + "prefix": "Префикс пути", + "search_queries": "Поисковые запросы", + "number_posts": "Количество постов", + "telegram_token": "Токен бота", + "ntfy_token": "Токен доступа", + "postgres_token": "Строка подключения", + "brave_token": "API-ключ", + "github_access_token": "Персональный токен доступа", + "repo_url": "Репозиторий", + "telegram_chat_id": "ID чата по умолчанию" + }, + "wizard": { + "connectTitle": "Подключить {{name}}", + "reconnectTitle": "Переподключить {{name}}", + "signIn": "Войти в {{name}}", + "chooseWhatToSync": "Выберите, что синхронизировать", + "skip": "Пока пропустить", + "syncFrequency": "Синхронизация", + "doneTitle": "{{name}} подключён", + "doneSummary": "Подключено как {{account}}. Синхронизируется в знания: {{sources}}; готово: {{tools}}.", + "doneSummaryNone": "Подключено как {{account}}.", + "sourcesCount_one": "{{formatted}} элемент", + "sourcesCount_few": "{{formatted}} элемента", + "sourcesCount_many": "{{formatted}} элементов", + "sourcesCount_other": "{{formatted}} элемента", + "toolsCount_one": "{{formatted}} инструмент", + "toolsCount_few": "{{formatted}} инструмента", + "toolsCount_many": "{{formatted}} инструментов", + "toolsCount_other": "{{formatted}} инструмента", + "toolsHeading": "Инструменты ({{formatted}})", + "tryInChat": "Попробовать в чате", + "done": "Готово", + "connectFailed": "Не удалось подключиться. Проверьте данные и попробуйте ещё раз.", + "syncFailed": "Не удалось запустить синхронизацию. Попробуйте ещё раз.", + "connectedWithKey": "Подключено ключом, оканчивающимся на {{hint}}.", + "doneCounts": "Синхронизируется в знания: {{sources}}; готово: {{tools}}.", + "doneSources": "Синхронизируется в знания: {{sources}}.", + "doneTools": "Готово: {{tools}}.", + "waiting": "Завершите вход в {{name}} в открывшемся окне.", + "openSignIn": "Открыть вход в {{name}}", + "chooseWhatToSetUp": "Выберите, что настроить", + "continue": "Продолжить", + "methodLabel": "Способ подключения", + "methodOauth": "Войти через {{name}}", + "methodToken": "Использовать токен доступа", + "credentialsRejected": "{{name}} не принял эти учётные данные. Проверьте их и попробуйте ещё раз.", + "toolsUnavailable": "Инструменты {{name}} недоступны. Попробуйте ещё раз или отключите их, чтобы продолжить.", + "addTools": "Разрешить агентам использовать {{name}}", + "addToolsDescription": { + "read": "Агенты могут находить информацию. Изменять ничего не могут.", + "readWrite": "Агенты могут находить информацию и вносить изменения. Изменения требуют вашего одобрения." + }, + "accountName": "Название аккаунта", + "accountNameHint": "Необязательно. Помогает вам и ИИ различать аккаунты, например «Бот оповещений».", + "syncToKnowledge": "Синхронизировать в знания", + "syncToKnowledgeDescription": "Выберите содержимое {{name}}, по которому будут искаться ответы. Если выключено, сервис только подключается; синхронизировать можно позже в разделе «Коннекторы».", + "syncLater": "В знания пока ничего не синхронизируется. Чтобы добавить содержимое, откройте {{name}} в разделе «Коннекторы» и выберите «Синхронизировать ещё».", + "retrievalSettings": "Расширенные настройки поиска" + }, + "remove": { + "title": "Удалить {{name}}?", + "description": "Сохранённый вход будет удалён. Выберите, что станет с тем, что даёт это подключение.", + "sourcesLabel_one": "Элемент, синхронизированный в знания", + "sourcesLabel_few": "{{formatted}} элемента, синхронизированных в знания", + "sourcesLabel_many": "{{formatted}} элементов, синхронизированных в знания", + "sourcesLabel_other": "{{formatted}} элемента, синхронизированных в знания", + "keepSources": "Оставить контент (остановить синхронизацию)", + "deleteSources": "Удалить", + "toolsLabel_one": "Его инструмент", + "toolsLabel_few": "Его {{formatted}} инструмента", + "toolsLabel_many": "Его {{formatted}} инструментов", + "toolsLabel_other": "Его {{formatted}} инструмента", + "deleteTools": "Удалить", + "keepTools": "Оставить выключенными", + "failed": "Не удалось удалить подключение. Попробуйте ещё раз." + }, + "health": { + "reconnect": "Переподключите {{name}}, чтобы синхронизация продолжилась.", + "reconnectSources_one": "Переподключите {{name}}: {{formatted}} элемент больше не синхронизируется в знания", + "reconnectSources_few": "Переподключите {{name}}: {{formatted}} элемента больше не синхронизируются в знания", + "reconnectSources_many": "Переподключите {{name}}: {{formatted}} элементов больше не синхронизируются в знания", + "reconnectSources_other": "Переподключите {{name}}: {{formatted}} элемента больше не синхронизируются в знания", + "reconnectTools": "Переподключите {{name}}, чтобы агенты снова могли его использовать", + "reconnectBoth_one": "Переподключите {{name}}: {{formatted}} элемент не синхронизируется, агенты не могут его использовать", + "reconnectBoth_few": "Переподключите {{name}}: {{formatted}} элемента не синхронизируются, агенты не могут его использовать", + "reconnectBoth_many": "Переподключите {{name}}: {{formatted}} элементов не синхронизируются, агенты не могут его использовать", + "reconnectBoth_other": "Переподключите {{name}}: {{formatted}} элемента не синхронизируются, агенты не могут его использовать", + "navDot": "Подключению требуется внимание", + "signInAgain": "Войдите снова", + "pickerNotice": "{{name}}: нужно войти снова." + }, + "error": { + "decrypt": "Не удалось расшифровать сохранённые учётные данные. Переподключитесь, чтобы продолжить.", + "defaultKey": "Администратор должен задать ENCRYPTION_SECRET_KEY, прежде чем можно будет подключать сервисы." + }, + "sharing": { + "owner": "Все используют ваш аккаунт", + "member": "Каждый подключает свой аккаунт", + "ownerShort": "Ваш аккаунт", + "memberShort": "У каждого свой", + "ownerShortShared": "Аккаунт владельца" + }, + "share": { + "heading": "С чьим аккаунтом он работает", + "ownerWarning": "Участники команды будут действовать в {{name}} как {{account}}.", + "ownerWarningShared": "Участники команды будут действовать от имени аккаунта {{name}} владельца.", + "confirmWrite": "Я понимаю, что участники команды могут выполнять действия от имени моего аккаунта.", + "confirmWriteShared": "Я понимаю, что участники команды могут выполнять действия от имени аккаунта {{name}} владельца.", + "memberNote": "Каждый использует свой аккаунт {{name}} и подключает его при первом использовании этого инструмента. Через API-ключ или виджет агента используется аккаунт владельца агента.", + "forced": "Администратор задал это для всех общих доступов к этому коннектору.", + "ownerChooses": "Изменить это может только владелец.", + "saveFailed": "Не удалось сохранить изменение. Попробуйте ещё раз." + }, + "createFailed": "Не удалось добавить коннектор. Попробуйте ещё раз.", + "capabilityFilter": { + "sync": "Показаны коннекторы, которые синхронизируют содержимое в знания.", + "tools": "Показаны коннекторы, которые дают агентам инструменты.", + "showAll": "Показать все" + }, + "capabilityPlain": { + "sync": "Синхронизирует в знания", + "read": "Ищет информацию", + "write": "Выполняет действия" + }, + "github": { + "tokenHint": "Создайте детализированный токен с доступом на чтение к Contents и Metadata для нужных репозиториев. Чтобы агенты могли вносить изменения, дайте ему также доступ на чтение и запись к Issues и Pull requests.", + "writes": "Разрешить агентам также вносить изменения (задачи, комментарии, пул-реквесты)", + "writesDescription": "По умолчанию каждое изменение сначала требует вашего одобрения. Токену или GitHub App нужен доступ на чтение и запись к Issues и Pull requests.", + "writesForbidden": "Администратор отключил изменения через GitHub.", + "writesFailed": "Не удалось изменить, что агентам можно делать в GitHub.", + "createToken": "Создать токен на GitHub", + "chooseRepositories": "Выбрать репозитории на GitHub", + "noAppRepositories": "DocsGPT пока не видит ни одного репозитория. Выберите на GitHub, какие ему можно читать.", + "noTokenRepositories": "Этот токен не может читать ни одного репозитория. Дайте ему доступ на GitHub и переподключите.", + "searchRepositories": "Поиск репозиториев", + "repositories": "Репозитории", + "updated": "Обновлён {{date}}", + "loadFailed": "Не удалось загрузить ваши репозитории." + }, + "linear": { + "teams": "Команды", + "teamsHint": "Синхронизировать задачи этих команд.", + "projects": "Проекты", + "projectsHint": "Синхронизировать задачи этих проектов.", + "search": "Поиск команд и проектов", + "noMatches": "Нет команд или проектов, подходящих под запрос.", + "includeComments": "Включать комментарии", + "includeCommentsDescription": "Добавлять к каждой задаче её комментарии.", + "includeDocuments": "Включать документы проектов", + "includeDocumentsDescription": "Синхронизировать документы Linear выбранных проектов.", + "loadFailed": "Не удалось загрузить ваши команды и проекты Linear.", + "empty": "В этом аккаунте Linear нет команд или проектов для синхронизации." + }, + "parameters": { + "show": "Параметры", + "hide": "Скрыть параметры", + "fixedCount_one": "{{formatted}} задан", + "fixedCount_few": "{{formatted}} задано", + "fixedCount_many": "{{formatted}} задано", + "fixedCount_other": "{{formatted}} задано", + "ai": "Решает ИИ", + "fixed": "Всегда использовать", + "choiceLabel": "Как заполняется {{parameter}}", + "valueLabel": "Значение {{parameter}}", + "save": "Сохранить", + "saveFailed": "Не удалось сохранить значение. Попробуйте ещё раз.", + "hint": "Значение «Всегда использовать» отправляется при каждом вызове. ИИ его не видит и не может изменить.", + "fromAccount": "Задано в этом аккаунте: {{value}}" + }, + "fieldHints": { + "telegram_chat_id": "Необязательно. Сообщения уходят в этот чат, и ИИ не может выбрать другой. Добавьте бота в чат, отправьте ему сообщение и найдите id чата на https://api.telegram.org/bot/getUpdates." + }, + "rename": { + "title": "Переименовать аккаунт", + "description": "Название отличает этот аккаунт от других ваших аккаунтов того же сервиса, для вас и для ИИ. Оставьте поле пустым, чтобы показывать сам аккаунт.", + "label": "Название аккаунта", + "save": "Сохранить", + "failed": "Не удалось переименовать аккаунт. Попробуйте ещё раз." + } } }, "modals": { "uploadDoc": { - "label": "Загрузить новый документ", + "label": "Добавить знания", "select": "Выберите способ загрузки документа в DocsGPT", - "selectSource": "Выберите способ добавления источника", + "selectSource": "Добавить знания", "selectedFiles": "Выбранные файлы", "noFilesSelected": "Файлы не выбраны", "file": "Загрузить с устройства", @@ -1383,19 +1757,11 @@ "filesRejected": "Не добавлены (больше 25 МБ или неподдерживаемый формат): {{files}}", "uploadedFiles": "Загруженные файлы", "cancel": "Отмена", - "train": "Тренировка", + "train": "Добавить в знания", "create": "Создать", "link": "Ссылка", "urlLink": "URL ссылка", "repoUrl": "URL репозитория", - "reddit": { - "id": "ID клиента", - "secret": "Секрет клиента", - "agent": "Пользовательский агент", - "searchQueries": "Поисковые запросы", - "numberOfPosts": "Количество сообщений", - "addQuery": "Добавить запрос" - }, "drag": { "title": "Перетащите вложения сюда", "description": "Отпустите, чтобы загрузить ваши вложения" @@ -1417,8 +1783,6 @@ "clear": "Очистить", "ingestionFailed": "Не удалось загрузить данные." }, - "showAdvanced": "Показать расширенные настройки", - "hideAdvanced": "Скрыть расширенные настройки", "ingestors": { "local_file": { "label": "Загрузить файл", @@ -1436,26 +1800,6 @@ "label": "GitHub", "heading": "Добавить контент из GitHub" }, - "reddit": { - "label": "Reddit", - "heading": "Добавить контент из Reddit" - }, - "google_drive": { - "label": "Google Drive", - "heading": "Загрузить из Google Drive" - }, - "s3": { - "label": "Amazon S3", - "heading": "Добавить контент из Amazon S3" - }, - "share_point": { - "label": "SharePoint", - "heading": "Загрузить из SharePoint" - }, - "confluence": { - "label": "Confluence", - "heading": "Загрузить из Confluence" - }, "wiki": { "label": "Новая вики", "heading": "Создать живую вики" @@ -1509,13 +1853,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "Идентификатор ключа доступа AWS", - "awsSecretAccessKey": "Секретный ключ доступа AWS", - "bucket": "Имя бакета", - "prefix": "Префикс пути (необязательно)", - "region": "Регион AWS", - "endpointUrl": "Пользовательский URL эндпоинта (необязательно)", "initialContent": "Исходное содержимое (необязательно)" + }, + "github": { + "privateHint": "Приватный репозиторий? Подключите аккаунт GitHub, чтобы синхронизировать его.", + "connectedHint": "Приватные репозитории синхронизируются через ваше подключение GitHub ({{account}}).", + "connect": "Подключить GitHub", + "pickRepository": "Выбрать репозиторий" + }, + "connectData": { + "title": "Подключите свои данные", + "description": "Синхронизируйте {{services}}", + "more": "другие сервисы", + "intro": "Выберите сервис, войдите в аккаунт и выберите, что синхронизировать в знания.", + "connectedAs": "Подключено как {{account}}", + "browseAll": "Все коннекторы" } }, "createAPIKey": { @@ -1552,8 +1904,8 @@ "configTool": { "title": "Настройка инструмента", "type": "Тип", - "apiKeyLabel": "API ключ / OAuth", - "apiKeyPlaceholder": "Введите API ключ / OAuth", + "apiKeyLabel": "API-ключ", + "apiKeyPlaceholder": "Введите API-ключ", "addButton": "Добавить инструмент", "closeButton": "Закрыть", "customNamePlaceholder": "Enter custom name (optional)", @@ -1631,6 +1983,28 @@ "learnMore": "Learn more", "resetKey": "Сбросить ключ", "resetKeyConfirm": "Вы уверены, что хотите сбросить API-ключ? Текущий ключ немедленно перестанет работать, и это действие нельзя отменить.", + "apiWrites": { + "title": "Изменения от вашего имени", + "description": "Через API-ключ, виджет или публичную ссылку агента никто не может подтвердить действие за вас, поэтому изменения с вашими подключёнными аккаунтами и сохранёнными учётными данными заблокированы, если вы не разрешите их здесь.", + "saveFailed": "Не удалось сохранить. Попробуйте ещё раз.", + "summaryNone": "Пока ничего не разрешено", + "summaryTools_one": "{{tools}} может вносить изменения", + "summaryTools_few": "{{tools}} могут вносить изменения", + "summaryTools_many": "{{tools}} могут вносить изменения", + "summaryTools_other": "{{tools}} могут вносить изменения", + "summaryCount": "разрешено {{allowed}} из {{formatted}}", + "toolCount": "Изменения через API, виджет и публичные ссылки · разрешено {{allowed}} из {{formatted}}", + "toolLabel": "Разрешить изменения через API, виджет и публичные ссылки для {{tool}}", + "choice": { + "off": "Выкл.", + "all": "Все" + }, + "customize_one": "Настроить изменение", + "customize_few": "Настроить каждое из {{formatted}}", + "customize_many": "Настроить каждое из {{formatted}}", + "customize_other": "Настроить каждое из {{formatted}}", + "fold": "Скрыть изменения" + }, "actionFailed": "Не получилось. Попробуйте ещё раз.", "apiKeyAfterPublish": "Опубликуйте агента, чтобы создать его API-ключ." }, @@ -1727,16 +2101,20 @@ }, "sources": { "title": "Источники", - "text": "Выберите ваши источники", + "text": "Выберите знания", "link": "Ссылка на источник", "view_more_one": "ещё {{count}} источник", "view_more_few": "ещё {{count}} источника", "view_more_many": "ещё {{count}} источников", "view_more_other": "ещё {{count}} источников", - "noSourcesAvailable": "Нет доступных источников", - "selectedCount": "Выбрано источников: {{count}}", - "groupOwn": "Ваши источники", - "groupTeam": "Общие с командой" + "noSourcesAvailable": "Знаний пока нет", + "selectedCount": "Выбрано: {{count}}", + "groupOwn": "Ваши", + "groupTeam": "Общие с командой", + "connectMore": "Подключить ещё", + "fromConnector": "Из {{name}}", + "fromConnectorTitle": "Из {{name}}: {{title}}", + "knowledge": "Знания" }, "attachments": { "attach": "Прикрепить", @@ -1777,7 +2155,9 @@ "accessingMemory": "Обращаюсь к памяти…", "runningCode": "Выполняю код…", "creatingArtifact": "Создаю артефакт…", - "usingTool": "Использую {{tool}}…" + "usingTool": "Использую {{tool}}…", + "searchingConnector": "Поиск в {{name}}...", + "usingConnector": "Использую {{name}}..." }, "toolChip": { "searchingWeb": "Поиск в интернете · «{{query}}»", @@ -1788,7 +2168,10 @@ "accessingMemory": "Обращение к памяти выполнено", "runningCode": "Код выполнен", "creatingArtifact": "Артефакт создан", - "usingTool": "Использован {{tool}}" + "usingTool": "Использован {{tool}}", + "searchedConnector": "Поиск в {{name}}", + "readConnector": "Прочитано из {{name}}", + "usedConnector": "Использован {{name}}: {{action}}" }, "inlineSteps": { "arguments": "Аргументы", @@ -1838,7 +2221,12 @@ "approveAlways": "Одобрить и больше не спрашивать", "deny": "Отклонить", "details": "Подробности", - "denyReasonPlaceholder": "Причина отказа (необязательно)…" + "denyReasonPlaceholder": "Причина отказа (необязательно)…", + "connectTitle": "Этому инструменту нужен ваш аккаунт {{name}}.", + "connect": "Подключить {{name}}", + "continue": "Продолжить", + "skip": "Пропустить", + "thisService": "этот сервис" }, "research": { "title": "Исследование", @@ -1856,7 +2244,6 @@ "agents": { "title": "Агенты", "edit": "Редактировать", - "view": "Просмотр", "card": { "pin": "Закрепить агента", "unpin": "Открепить агента", @@ -1950,7 +2337,7 @@ "placeholders": { "agentName": "Имя агента", "describeAgent": "Опишите вашего агента", - "selectSources": "Выберите источники", + "selectSources": "Выберите знания", "chunksPerQuery": "Фрагментов на запрос", "selectType": "Выберите тип", "selectTools": "Выберите инструменты", @@ -1960,20 +2347,23 @@ "enterRequestLimit": "Введите лимит запросов" }, "sourcePopup": { - "title": "Выберите Источники", - "searchPlaceholder": "Поиск источников...", - "noOptionsMessage": "Нет доступных источников", + "title": "Выберите знания", + "searchPlaceholder": "Поиск по знаниям...", + "noOptionsMessage": "Знаний пока нет", "groupOwn": "Ваши источники", "groupTeam": "Общие с командой", - "noSourceHint": "Необязательно. Без источника агент отвечает только на основе модели и своих инструментов." + "noSourceHint": "Необязательно. Без знаний агент отвечает только с помощью модели и своих инструментов." }, "toolsPopup": { "title": "Выберите Инструменты", "searchPlaceholder": "Поиск инструментов...", "noOptionsMessage": "Нет доступных инструментов", "groupBuiltin": "Встроенные", - "groupDefault": "По умолчанию", - "groupCustom": "Пользовательские" + "groupDefault": "Встроенные", + "groupCustom": "Пользовательские", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "В этом агенте", + "attachedHint": "Добавлено другим пользователем. Вы можете удалить это." }, "modelsPopup": { "title": "Выберите Модели", @@ -1993,7 +2383,7 @@ "defaultModel": "Модель по умолчанию", "name": "Имя", "description": "Описание", - "sources": "Источники", + "sources": "Знания", "avatar": "Аватар" }, "advanced": { @@ -2084,11 +2474,69 @@ "published": "Опубликован" }, "sponsors": { - "attachNote": "Инструменты, источники и промпты, которые вы добавляете из своей библиотеки, работают с вашим доступом для всех, кто пользуется этим агентом.", + "attachNote": "Инструменты, источники и промпты, которые вы добавляете и которые владелец не может использовать, работают с вашим доступом для всех, кто пользуется этим агентом. Вы должны быть их владельцем или иметь право их редактировать, и вас попросят это подтвердить.", "publicLinkNote": "У этого агента есть публичная ссылка, поэтому любой, у кого она есть, может получать ответы на их основе.", "addedBy": "Добавил(а) {{person}}: {{names}}", - "unavailable": "Не работают: {{names}}. {{person}} добавил(а) их и больше не имеет доступа. Удалите их или выберите те, которые может использовать владелец.", - "unknownItem": "Элемент без названия" + "addedByOther": "Добавил(а) другой пользователь: {{names}}", + "unknownItem": "Элемент без названия", + "notAllowed": "Вы не можете добавить {{names}} к этому агенту. Владелец не может их использовать, а делиться можно только тем, чем вы владеете или что можете редактировать.", + "takeOver": "Запускать {{name}} с моим доступом", + "takeOverPending": "{{name}} будет работать с вашим доступом после сохранения.", + "undoTakeOver": "Отменить", + "confirmationOutdated": "Агент изменился после того, как вы подтвердили, что работает с вашим доступом. Проверьте уведомления и сохраните снова." + }, + "resourceStates": { + "title": "Некоторые инструменты, источники или промпты не работают", + "reason": { + "deleted": "{{name}} удалён.", + "ownerLostAccessYou": "Вы больше не можете использовать {{name}}.", + "ownerLostAccess": "Владелец агента больше не может использовать {{name}}.", + "sponsorCannotEditAgent": "{{name}} работал с доступом {{person}}, а этот человек больше не может редактировать агента.", + "sponsorCannotEditAgentOther": "{{name}} работал с доступом другого пользователя, а этот человек больше не может редактировать агента.", + "sponsorCannotEditItem": "{{name}} работал с доступом {{person}}, а этот человек больше не может его редактировать.", + "sponsorCannotEditItemOther": "{{name}} работал с доступом другого пользователя, а этот человек больше не может его редактировать.", + "connectionNeedsReconnect": "Для {{name}} нужно снова войти в аккаунт {{service}}.", + "connectionRemoved": "Аккаунт {{service}}, который использовал {{name}}, удалён.", + "connectorDisabled": "Администратор отключил {{service}}, поэтому {{name}} не может работать.", + "unknown": "{{name}} не работает." + }, + "ask": { + "shareAgain": "Попросите {{person}} снова поделиться им.", + "signInAgain": "Попросите {{person}} снова войти в аккаунт.", + "connectAgain": "Попросите {{person}} снова подключить аккаунт.", + "admin": "Попросите администратора снова включить его.", + "shareAgainOwner": "Попросите владельца снова поделиться им.", + "signInAgainOwner": "Попросите владельца снова войти в аккаунт.", + "connectAgainOwner": "Попросите владельца снова подключить аккаунт." + }, + "promptFallback": "До тех пор агент использует промпт по умолчанию.", + "serviceFallback": "сервис", + "remove": "Убрать", + "removeLabel": "Убрать {{name}}", + "reconnectLabel": "Переподключить {{name}}", + "unnamed": { + "tool": "Инструмент {{id}}", + "source": "Источник {{id}}", + "prompt": "Промпт {{id}}" + }, + "chip": "Не работает: {{count}}" + }, + "sponsorConfirm": { + "title": "Запускать с вашим доступом?", + "description": "Владелец агента не может их использовать. Если вы их добавите, они будут работать с вашим доступом для всех, кто пользуется этим агентом.", + "audienceTitle": "Кто сможет использовать их через этого агента", + "audienceTeams": "Участники {{teams}}", + "audienceApiKey": "Все, у кого есть API-ключ агента, включая виджет на сайте", + "audiencePublicLink": "Все, у кого есть публичная ссылка на агента", + "audienceWebhook": "Всё, что вызывает вебхук агента", + "audienceEditors": "Все, кто может открыть этого агента", + "stopNote": "Они перестанут работать, если вы потеряете право редактировать их или этого агента. Тогда они остаются подключёнными, но ничего не делают, пока кто-то с правом редактирования не возьмёт их на себя или их не удалят.", + "confirm": "Добавить с моим доступом", + "types": { + "tool": "Инструмент", + "source": "Источник", + "prompt": "Промпт" + } } }, "logs": { @@ -2386,7 +2834,7 @@ "outputVariable": "Выходная переменная", "outputVariablePlaceholder": "Имя переменной для вывода", "streamToUser": "Передавать вывод пользователю потоком", - "sources": "Источники", + "sources": "Знания", "documents": "Документы", "documentsHint": "Документы, передаваемые этому агенту из загрузок или предыдущих узлов.", "filePassing": "Передача файлов", @@ -2425,7 +2873,8 @@ "detailsTitle": "Сведения о рабочем процессе", "detailsDescription": "Название, описание и изображение, которые показываются на карточке агента.", "unsavedChanges": "Несохранённые изменения", - "editDetailsMenu": "Изменить сведения" + "editDetailsMenu": "Изменить сведения", + "attachedOption": "{{name}} (добавлено другим пользователем)" }, "validation": { "nameRequired": "Укажите название рабочего процесса", @@ -2537,6 +2986,7 @@ "workflowTitle": "Агент рабочего процесса", "workflowDescription": "Создавайте сложные многошаговые рабочие процессы с разными моделями, условной логикой и управлением состоянием" }, + "view": "Просмотр", "deleteFailed": "Не удалось удалить агента. Попробуйте ещё раз." }, "components": { @@ -2609,7 +3059,8 @@ "shared": "Общие", "connectTo": "Подключиться к {{provider}}", "sessionExpiredFor": "Сеанс истек. Пожалуйста, переподключитесь к {{provider}}.", - "selectFilesFrom": "Выберите файлы из {{provider}}" + "selectFilesFrom": "Выберите файлы из {{provider}}", + "account": "Аккаунт" }, "actionButtons": { "openNewChat": "Открыть новый чат", @@ -2667,5 +3118,15 @@ }, "prompts": { "textAriaLabel": "Текст подсказки" + }, + "connectHero": { + "title": "Подключите данные", + "body": "Общайтесь с {{names}} и не только." + }, + "common": { + "close": "Закрыть", + "viewOnlyNotice": "Вы можете это просматривать, но ваша роль не позволяет это изменить.", + "credentialsLockedNotice": "Только владелец может изменить учётные данные этого инструмента.", + "savedSecretHint": "Сохранено. Оставьте поле пустым, чтобы сохранить значение." } } diff --git a/frontend/src/locale/zh-TW.json b/frontend/src/locale/zh-TW.json index ddbe792e..82fe518c 100644 --- a/frontend/src/locale/zh-TW.json +++ b/frontend/src/locale/zh-TW.json @@ -10,12 +10,6 @@ "loading": "載入中...", "retry": "重試", "cancel": "取消", - "common": { - "close": "關閉", - "viewOnlyNotice": "你可以檢視此內容，但你的角色無法變更它。", - "credentialsLockedNotice": "只有擁有者可以變更此工具的憑證。", - "savedSecretHint": "已儲存。留空即可保留。" - }, "help": "幫助", "emailUs": "給我們發電郵", "documentation": "文件", @@ -106,16 +100,16 @@ }, "sources": { "title": "在這裡您可以管理所有可用的來源檔案以及您上傳的檔案。", - "subtitle": "上傳並管理您的回答所依賴的文件和知識來源", - "label": "來源", + "subtitle": "DocsGPT 可以搜尋和引用的內容：上傳的檔案、網站、Wiki，以及從連線同步的內容。", + "label": "知識庫", "name": "來源名稱", "date": "向量日期", "type": "類型", "tokenUsage": "Token 使用量", - "noData": "沒有現有的來源", + "noData": "還沒有知識", "searchPlaceholder": "搜尋...", "addNew": "新增文件", - "addSource": "新增來源", + "addSource": "新增知識", "addChunk": "新增區塊", "preLoaded": "預載入", "private": "私人", @@ -176,7 +170,6 @@ } }, "badge": "動態 Wiki", - "byline_one": "{{pages}} 個頁面 · {{tokens}} 個 token", "byline_other": "{{pages}} 個頁面 · {{tokens}} 個 token", "explainer": "代理在工作時會閱讀並改寫這些頁面。", "explainerEditable": "代理在工作時會閱讀並改寫這些頁面；任何使用此來源的代理都可以編輯它。", @@ -185,7 +178,18 @@ "pagesTitle": "頁面", "loadFailed": "無法載入 Wiki 頁面。", "pageLoadFailed": "無法載入此頁面。", - "editTitle": "編輯頁面" + "editTitle": "編輯頁面", + "settings": { + "action": "Wiki 設定", + "title": "Wiki 設定", + "outsideEdits": { + "label": "允許 API 和小工具使用者編輯此 Wiki", + "description": "關閉時，透過代理的 API 金鑰或小工具使用它的人可以閱讀此 Wiki，但無法變更。公開連結的訪客只能編輯他們自己有權編輯的 Wiki，且每次變更前都會被詢問。你和此 Wiki 的編輯者隨時都能編輯它。" + }, + "loadError": "無法載入 Wiki 設定。", + "saveError": "無法儲存變更。請再試一次。" + }, + "byline_one": "{{pages}} 個頁面 · {{tokens}} 個 token" }, "graphrag": { "badge": "GraphRAG", @@ -235,7 +239,6 @@ "byConnections": "（依連結數）", "typeFilter": "依類型篩選", "otherTypes": "其他：{{types}}", - "otherTypesMore_one": "另外 {{formatted}} 個", "otherTypesMore_other": "另外 {{formatted}} 個", "untyped": "無類型", "loadFailed": "無法載入圖譜。", @@ -247,7 +250,6 @@ "showMore": "顯示更多", "showLess": "收合", "relationships": "關係", - "relationshipsCapped_one": "顯示 {{total}} 個關係中最強的 {{shown}} 個。", "relationshipsCapped_other": "顯示 {{total}} 個關係中最強的 {{shown}} 個。", "noRelationships": "此實體沒有關係。", "relatedTo": "相關", @@ -256,18 +258,20 @@ "sourceChunks": "來源區塊", "showInGraph": "在圖譜中顯示", "allTypes": "所有類型", - "chunkCount_one": "{{formatted}} 個區塊", "chunkCount_other": "{{formatted}} 個區塊", - "entityCount_one": "{{formatted}} 個實體", "entityCount_other": "{{formatted}} 個實體", - "relationshipCount_one": "{{formatted}} 個關係", "relationshipCount_other": "{{formatted}} 個關係", "chunk": "區塊", "chunkMeta": "{{file}} · {{tokens}} Token", "chunkTokens": "{{tokens}} Token", "openInFiles": "在檔案中開啟", "editChunk": "編輯區塊", - "chunkSaved": "區塊已儲存" + "chunkSaved": "區塊已儲存", + "otherTypesMore_one": "另外 {{formatted}} 個", + "relationshipsCapped_one": "顯示 {{total}} 個關係中最強的 {{shown}} 個。", + "chunkCount_one": "{{formatted}} 個區塊", + "entityCount_one": "{{formatted}} 個實體", + "relationshipCount_one": "{{formatted}} 個關係" } }, "testRetrieval": { @@ -302,7 +306,7 @@ "chunks": "文本塊", "noChunks": "未找到文本塊", "noChunksAlt": "未找到文本塊", - "goToSources": "前往來源", + "goToSources": "前往知識庫", "uploadNew": "上傳新文件", "noResults": "未找到結果", "fileName": "名稱", @@ -426,11 +430,9 @@ "chunkTitleHint": "回答引用此文本塊時使用的名稱。", "files": "檔案", "filesLoadError": "無法載入檔案", - "filesByline_one": "{{files}} 個檔案 · {{tokens}} Token", "filesByline_other": "{{files}} 個檔案 · {{tokens}} Token", "filterFiles": "篩選檔案", "searchChunks": "搜尋文本塊", - "chunkCount_one": "{{formatted}} 個文本塊", "chunkCount_other": "{{formatted}} 個文本塊", "editor": { "write": "編輯", @@ -445,6 +447,11 @@ "editChunkDescription": "{{file}} · 第 {{n}} 個文本塊 · {{tokens}} Token", "previousChunk": "上一個文本塊", "nextChunk": "下一個文本塊", + "paused": "同步已暫停。重新連線 {{name}} 以繼續。", + "emptyHint": "上傳檔案、新增網站，或連接一個服務來同步它的內容。", + "connectService": "連接服務", + "filesByline_one": "{{files}} 個檔案 · {{tokens}} Token", + "chunkCount_one": "{{formatted}} 個文本塊", "viewConfig": "檢視來源設定", "errors": { "forbidden": "你沒有權限對此來源執行該操作。", @@ -675,10 +682,6 @@ "createTeamError": "無法建立團隊。", "noTeams": "尚無團隊。", "noDescription": "無描述", - "memberCount_one": "{{formatted}} 位成員", - "memberCount_other": "{{formatted}} 位成員", - "sharedCount_one": "已分享 {{formatted}} 項", - "sharedCount_other": "已分享 {{formatted}} 項", "loadError": "載入團隊失敗。", "roleAdmin": "管理員", "roleMember": "成員", @@ -758,8 +761,46 @@ "teams": "團隊", "people": "人員", "editors": "編輯者" + }, + "uses": { + "title": "此代理使用的內容", + "intro": "使用此代理的每個人都會用到這些內容，各自依所示的存取權限執行。", + "stopped": "已停止", + "writesOff": "無法透過 API", + "writesNote": "連接器上的「允許」適用於聊天。標示為「無法透過 API」的變更，在你於 Access Details 中允許之前，API、小工具和公開連結的使用者無法進行。", + "writesNoteEditor": "連接器上的「允許」適用於聊天。標示為「無法透過 API」的變更，在擁有者於 Access Details 中允許之前，API、小工具和公開連結的使用者無法進行。", + "access": { + "you": "你的存取權限", + "person": "{{person}} 的存取權限", + "other": "其他人的存取權限", + "owner": "擁有者的存取權限", + "member": "每個人自己的 {{service}} 帳號（API 和小工具：你的）", + "memberNoService": "每個人自己的帳號（API 和小工具：你的）", + "yourAccount": "你的 {{service}} 帳號", + "yourAccountNoService": "你的帳號", + "personAccount": "{{person}} 的 {{service}} 帳號", + "personAccountNoService": "{{person}} 的帳號", + "memberShared": "每個人自己的 {{service}} 帳號（API 和小工具：擁有者的）", + "memberSharedNoService": "每個人自己的帳號（API 和小工具：擁有者的）", + "otherAccount": "其他人的 {{service}} 帳號", + "otherAccountNoService": "其他人的帳號", + "yourCredentials": "你儲存的憑證", + "personCredentials": "{{person}} 儲存的憑證", + "otherCredentials": "其他人儲存的憑證" + }, + "writesSomeOff": "部分無法透過 API", + "adminOff": "管理員已關閉變更", + "writesNoteApi": "連接器上的「允許」適用於聊天。標示為「無法透過 API」的變更，在你於 Access Details 中允許之前，API 和小工具的使用者無法進行。", + "writesNoteApiEditor": "連接器上的「允許」適用於聊天。標示為「無法透過 API」的變更，在擁有者於 Access Details 中允許之前，API 和小工具的使用者無法進行。", + "writesNoteMemberTail": "在每個人使用自己帳號的工具上，公開連結的使用者以自己的帳號進行變更。", + "openAccessDetails": "開啟 Access Details", + "accessDetailsNeedsKey": "代理有 API 金鑰後，Access Details 中會列出這些變更。" } }, + "memberCount_one": "{{formatted}} 位成員", + "memberCount_other": "{{formatted}} 位成員", + "sharedCount_one": "已分享 {{formatted}} 項", + "sharedCount_other": "已分享 {{formatted}} 項", "accessChangeError": "無法變更存取權。", "accessSettings": { "title": "存取設定", @@ -807,7 +848,8 @@ }, "viewers_can_use_in_agents": { "label": "檢視者可以在自己的代理中使用", - "description": "它會以你的憑證執行。" + "description": "它會以你的憑證執行。", + "descriptionMember": "每個人使用自己的帳號。" } }, "prompt": { @@ -929,11 +971,11 @@ }, "tools": { "label": "工具", - "subtitle": "瀏覽、連接並設定驅動您的智能體的工具和整合", + "subtitle": "代理可以執行的操作。內建工具在這裡。已連線服務提供的工具歸在該服務之下。", "searchPlaceholder": "搜尋工具...", "addTool": "新增工具", "noToolsFound": "找不到工具", - "selectToolSetup": "選擇要設定的工具", + "selectToolSetup": "新增工具", "settingsIconAlt": "設定圖示", "configureToolAria": "設定 {{toolName}}", "toggleToolAria": "切換 {{toolName}}", @@ -945,8 +987,8 @@ "builtIn": "內建", "authStatus": { "connected": "已連線", - "needsAuth": "需要驗證", - "configured": "已設定" + "needsAuth": "重新連線", + "configured": "已連線" }, "deleteWarning": "您確定要刪除工具 \"{{toolName}}\" 嗎？", "unsavedChanges": "您有未儲存的變更，如果不儲存就離開將會遺失。", @@ -1062,6 +1104,11 @@ }, "headerValuePlaceholder": "例如：application/json", "toolIconTitle": "{{name}} 圖示", + "groupBuiltIn": "內建", + "groupService": "來自服務", + "browseConnectors": "瀏覽所有連接器", + "viaConnection": "透過 {{name}}", + "off": "已關閉", "view": "檢視", "inMyChats": "在我的聊天中", "useInMyChatsAria": "在我的聊天中使用 {{toolName}}", @@ -1297,13 +1344,310 @@ "subtitle": "「{{name}}」的新密鑰已就緒，舊密鑰已失效。" } } + }, + "connectors": { + "label": "連接器", + "subtitle": "將 DocsGPT 連接到團隊使用的服務。一個連線可以把內容同步到知識庫，也可以為代理提供工具。", + "search": "搜尋連接器...", + "addCustom": "新增自訂工具", + "custom": { + "custom_mcp": "MCP 伺服器", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "依類別篩選連接器", + "categories": { + "all": "全部", + "connected": "已連線", + "files": "檔案與儲存", + "knowledge": "文件與 Wiki", + "dev": "開發者工具", + "messaging": "訊息", + "database": "資料庫", + "search": "搜尋與網頁", + "custom": "自訂", + "projects": "專案與事項", + "business": "付款與商務" + }, + "capability": { + "sync": "同步", + "read": "讀取", + "write": "寫入" + }, + "status": { + "connect": "連線", + "connected": "已連線", + "connectedCount": "已連線 {{formatted}} 個", + "reconnect": "重新連線", + "needsAdminSetup": "需要管理員設定", + "disabledByAdmin": "已被管理員停用", + "custom": "自訂" + }, + "connectionStatus": { + "connected": "已連線", + "reconnect_needed": "需要重新連線", + "disconnected": "已中斷連線", + "error": "錯誤", + "pending": "等待登入" + }, + "publisher": { + "built_in": "內建", + "preset": "預設", + "custom": "自訂" + }, + "descriptions": { + "google_drive": "將文件、試算表和 PDF 同步到知識庫。", + "share_point": "將 SharePoint 網站和 OneDrive 中的檔案同步到知識庫。", + "confluence": "將 Confluence 空間和頁面同步到知識庫。", + "s3": "將 S3 儲存貯體中的文件同步到知識庫。", + "reddit": "將符合你搜尋條件的 Reddit 貼文同步到知識庫。", + "brave": "使用 Brave Search API 搜尋網頁和圖片。", + "telegram": "傳送訊息和圖片到 Telegram 聊天。", + "ntfy": "透過 ntfy 伺服器傳送推播通知。", + "postgres": "讀取 Postgres 資料庫的結構並執行 SQL。", + "custom_mcp": "連接任何遠端 Model Context Protocol 伺服器。", + "custom_openapi": "匯入 OpenAPI 規格，並將其端點作為工具呼叫。", + "mcp_notion": "搜尋、閱讀和更新 Notion 頁面與資料庫。", + "mcp_linear": "將議題和文件同步到知識庫，並讓代理尋找、建立和更新議題。", + "mcp_atlassian": "搜尋和更新 Jira 議題與 Confluence 頁面。", + "mcp_sentry": "查看 Sentry 問題、事件和發行版本。", + "mcp_asana": "尋找和更新 Asana 任務與專案。", + "mcp_stripe": "查看 Stripe 中的客戶、付款和訂閱。", + "github": "將儲存庫同步到知識庫，並讓代理讀取程式碼、Issue 和提取要求。" + }, + "empty": "尚未連線任何服務。連線一個服務以同步其內容，或為代理提供新工具。", + "noMatches": "沒有符合搜尋的連接器。", + "loadFailed": "無法載入連接器。", + "askAdmin": "請管理員完成設定。", + "setupSettings": "設定以下伺服器設定，然後重新啟動 DocsGPT：", + "setupGuide": "設定指南", + "unverified": "此伺服器未經 DocsGPT 審查。請只連接你信任的伺服器。", + "detail": { + "accounts": "帳號", + "connectedAs": "已以 {{account}} 連線", + "connectAnother": "連線其他帳號", + "sources": "來自此連線的知識", + "tools": "來自此連線的工具", + "noAccounts": "尚未連線帳號。", + "failed": "無法載入此連線。", + "lastSync": "已於 {{date}} 同步 · {{frequency}}", + "paused": "同步已暫停", + "toolOn": "開啟", + "toolOff": "關閉", + "accountMenu": "帳號操作", + "disconnect": "中斷連線", + "keyEnding": "結尾為 {{hint}} 的金鑰", + "syncMore": "同步更多內容", + "refreshTools": "重新整理工具", + "noSources": "尚未從此連線同步任何內容。", + "remove": "移除", + "refreshed": "工具已重新整理：新增 {{added}} 個，移除 {{removed}} 個。", + "refreshFailed": "無法重新整理工具。", + "syncNow": "立即同步", + "syncStarted": "正在同步 {{name}}…", + "syncFailed": "無法開始同步。請再試一次。", + "toolSwitchFailed": "無法變更工具。請再試一次。", + "account": "帳號：{{account}}", + "expired": "登入已過期。重新連線以繼續使用此連線。", + "broken": "此連線已停止運作。重新連線以再試一次。", + "addTools": "新增工具", + "noTools": "此連線還沒有工具。", + "addToolsFailed": "無法新增工具。請再試一次。", + "rename": "重新命名", + "accountPicker": "選擇帳號" + }, + "permission": { + "always": "允許", + "ask": "先詢問", + "off": "關閉", + "label": "{{action}} 的權限", + "saveFailed": "無法儲存權限。請再試一次。", + "groupLabel": "{{group}} 的全部權限", + "fold": "隱藏動作", + "hint": "「允許」和「先詢問」適用於聊天：選擇「先詢問」時，DocsGPT 會在執行動作前詢問你。透過代理的 API、小工具或公開連結進行的變更，需在該代理的 Access Details 中逐一代理允許。", + "actionCount_other": "{{formatted}} 個動作", + "customize_other": "分別設定 {{formatted}} 個" + }, + "disconnect": { + "title": "要中斷 {{name}} 的連線嗎？", + "body_zero": "在重新連線之前，此連線的工具將無法使用。", + "body_other": "已同步的 {{formatted}} 項會保留在知識庫中，但不再更新。此連線的工具在重新連線前無法使用。", + "failed": "無法中斷連線。請再試一次。" + }, + "fields": { + "token": "API 金鑰", + "aws_access_key_id": "存取金鑰 ID", + "aws_secret_access_key": "秘密存取金鑰", + "region": "區域", + "endpoint_url": "自訂端點 URL", + "client_id": "用戶端 ID", + "client_secret": "用戶端密鑰", + "user_agent": "User agent", + "bucket": "儲存貯體", + "prefix": "路徑前綴", + "search_queries": "搜尋詞", + "number_posts": "貼文數量", + "telegram_token": "機器人權杖", + "ntfy_token": "存取權杖", + "postgres_token": "連線字串", + "brave_token": "API 金鑰", + "github_access_token": "個人存取權杖", + "repo_url": "儲存庫", + "telegram_chat_id": "預設聊天 ID" + }, + "wizard": { + "connectTitle": "連線 {{name}}", + "reconnectTitle": "重新連線 {{name}}", + "signIn": "登入 {{name}}", + "chooseWhatToSync": "選擇要同步的內容", + "skip": "暫時略過", + "syncFrequency": "同步", + "doneTitle": "{{name}} 已連線", + "doneSummary": "已以 {{account}} 身分連線。{{sources}} 正在同步到知識庫，{{tools}} 已就緒。", + "doneSummaryNone": "已以 {{account}} 連線。", + "sourcesCount_other": "{{formatted}} 項", + "toolsCount_other": "{{formatted}} 個工具", + "toolsHeading": "工具（{{formatted}}）", + "tryInChat": "在聊天中試試", + "done": "完成", + "connectFailed": "無法連線。請檢查資料後再試一次。", + "syncFailed": "無法開始同步。請再試一次。", + "connectedWithKey": "已使用結尾為 {{hint}} 的金鑰連線。", + "doneCounts": "{{sources}} 正在同步到知識庫，{{tools}} 已就緒。", + "doneSources": "{{sources}} 正在同步到知識庫。", + "doneTools": "{{tools}}已就緒。", + "waiting": "請在開啟的視窗中完成 {{name}} 登入。", + "openSignIn": "開啟 {{name}} 登入", + "chooseWhatToSetUp": "選擇要設定的內容", + "continue": "繼續", + "methodLabel": "連線方式", + "methodOauth": "使用 {{name}} 登入", + "methodToken": "使用存取權杖", + "credentialsRejected": "{{name}} 未接受這些憑證。請檢查後再試一次。", + "toolsUnavailable": "無法存取 {{name}} 的工具。請再試一次，或將其關閉以繼續。", + "addTools": "允許代理使用 {{name}}", + "addToolsDescription": { + "read": "代理可以查詢資訊，但無法做任何變更。", + "readWrite": "代理可以查詢資訊並進行變更。變更需要你的核准。" + }, + "accountName": "為此帳戶命名", + "accountNameHint": "選填。讓你和 AI 區分你的帳戶，例如「警示機器人」。", + "syncToKnowledge": "同步到知識庫", + "syncToKnowledgeDescription": "從 {{name}} 選擇要在回答中搜尋的內容。關閉時僅建立連線，之後可在連接器中同步。", + "syncLater": "尚未同步任何內容到知識庫。若要新增內容，請在連接器中開啟 {{name}}，然後選擇「同步更多內容」。", + "retrievalSettings": "進階檢索設定" + }, + "remove": { + "title": "要移除 {{name}} 嗎？", + "description": "儲存的登入資訊將被刪除。請選擇如何處理此連線提供的內容。", + "sourcesLabel_other": "它同步到知識庫的 {{formatted}} 項", + "keepSources": "保留內容（停止同步）", + "deleteSources": "刪除", + "toolsLabel_other": "它的 {{formatted}} 個工具", + "deleteTools": "刪除", + "keepTools": "保留並關閉", + "failed": "無法移除此連線。請再試一次。" + }, + "health": { + "reconnect": "重新連線 {{name}} 以繼續同步。", + "reconnectSources_other": "重新連線 {{name}}：{{formatted}} 項已停止同步到知識庫", + "reconnectTools": "請重新連線 {{name}}，讓代理可以再次使用它", + "reconnectBoth_other": "重新連線 {{name}}：{{formatted}} 項已停止同步，代理也無法使用它", + "navDot": "有連線需要處理", + "signInAgain": "重新登入", + "pickerNotice": "{{name}} 需要你重新登入。" + }, + "error": { + "decrypt": "無法解密已儲存的憑證。請重新連線以繼續。", + "defaultKey": "管理員必須先設定 ENCRYPTION_SECRET_KEY，才能連線服務。" + }, + "sharing": { + "owner": "所有人使用你的帳號", + "member": "每個人連線自己的帳號", + "ownerShort": "你的帳號", + "memberShort": "各自的帳號", + "ownerShortShared": "擁有者的帳號" + }, + "share": { + "heading": "使用誰的帳號執行", + "ownerWarning": "團隊成員將在 {{name}} 中以 {{account}} 的身分操作。", + "ownerWarningShared": "團隊成員將以擁有者的 {{name}} 帳號操作。", + "confirmWrite": "我了解團隊成員可以使用我的帳號執行操作。", + "confirmWriteShared": "我了解團隊成員可以使用擁有者的 {{name}} 帳號執行操作。", + "memberNote": "每個人使用自己的 {{name}} 帳號，並在首次使用此工具時連線。透過代理的 API 金鑰或小工具使用時，會使用代理擁有者的帳號。", + "forced": "管理員已為此連接器的所有共用設定此項。", + "ownerChooses": "只有擁有者可以變更此項。", + "saveFailed": "無法儲存變更。請再試一次。" + }, + "createFailed": "無法新增連接器。請再試一次。", + "capabilityFilter": { + "sync": "正在顯示將內容同步到知識庫的連接器。", + "tools": "正在顯示為代理提供工具的連接器。", + "showAll": "顯示全部" + }, + "capabilityPlain": { + "sync": "同步到知識庫", + "read": "查詢資訊", + "write": "執行操作" + }, + "github": { + "tokenHint": "為要使用的儲存庫建立細部權杖，並授予 Contents 和 Metadata 的讀取權限。如要讓代理進行變更，還需授予 Issues 和 Pull requests 的讀寫權限。", + "writes": "也允許代理進行變更（議題、留言、提取要求）", + "writesDescription": "預設情況下，每次變更都會先徵求你的同意。你的權杖或 GitHub App 需要 Issues 和 Pull requests 的讀寫權限。", + "writesForbidden": "管理員已關閉透過 GitHub 進行變更。", + "writesFailed": "無法變更代理在 GitHub 上可執行的操作。", + "createToken": "在 GitHub 上建立權杖", + "chooseRepositories": "在 GitHub 上選擇儲存庫", + "noAppRepositories": "DocsGPT 還看不到任何儲存庫。請在 GitHub 上選擇允許它讀取的儲存庫。", + "noTokenRepositories": "此權杖無法讀取任何儲存庫。請在 GitHub 上授予存取權，然後重新連線。", + "searchRepositories": "搜尋儲存庫", + "repositories": "儲存庫", + "updated": "更新於 {{date}}", + "loadFailed": "無法載入你的儲存庫。" + }, + "linear": { + "teams": "團隊", + "teamsHint": "同步這些團隊的議題。", + "projects": "專案", + "projectsHint": "同步這些專案的議題。", + "search": "搜尋團隊和專案", + "noMatches": "沒有符合搜尋的團隊或專案。", + "includeComments": "包含留言", + "includeCommentsDescription": "將每個議題的留言一併加入。", + "includeDocuments": "包含專案文件", + "includeDocumentsDescription": "同步所選專案的 Linear 文件。", + "loadFailed": "無法載入你的 Linear 團隊和專案。", + "empty": "此 Linear 帳戶沒有可同步的團隊或專案。" + }, + "parameters": { + "show": "參數", + "hide": "隱藏參數", + "fixedCount_other": "已固定 {{formatted}} 個", + "ai": "由 AI 決定", + "fixed": "一律使用", + "choiceLabel": "{{parameter}} 的填寫方式", + "valueLabel": "{{parameter}} 的值", + "save": "儲存", + "saveFailed": "無法儲存此值。請再試一次。", + "hint": "設為「一律使用」的值會在每次呼叫時送出。AI 看不到它，也無法變更。", + "fromAccount": "已在此帳戶中設定：{{value}}" + }, + "fieldHints": { + "telegram_chat_id": "選填。訊息會傳送到此聊天，AI 無法選擇其他聊天。將機器人加入聊天並傳送一則訊息給它，然後在 https://api.telegram.org/bot/getUpdates 中找到該聊天的 id。" + }, + "rename": { + "title": "重新命名帳戶", + "description": "此名稱可讓你和 AI 將此帳戶與同一服務的其他帳戶區分開來。留空則顯示帳戶本身。", + "label": "帳戶名稱", + "save": "儲存", + "failed": "無法重新命名帳戶。請再試一次。" + } } }, "modals": { "uploadDoc": { - "label": "上傳新文件", + "label": "新增知識", "select": "選擇如何將文件上傳到 DocsGPT", - "selectSource": "選擇新增來源的方式", + "selectSource": "新增知識", "selectedFiles": "已選擇的檔案", "noFilesSelected": "未選擇檔案", "file": "從檔案", @@ -1318,19 +1662,11 @@ "filesRejected": "未新增（超過 25 MB 或不支援的類型）：{{files}}", "uploadedFiles": "已上傳檔案", "cancel": "取消", - "train": "訓練", + "train": "新增到知識庫", "create": "建立", "link": "連結", "urlLink": "URL 連結", "repoUrl": "儲存庫 URL", - "reddit": { - "id": "客戶端ID", - "secret": "客戶端密鑰", - "agent": "使用者代理", - "searchQueries": "搜尋查詢", - "numberOfPosts": "貼文數量", - "addQuery": "新增查詢" - }, "drag": { "title": "將附件拖放到此處", "description": "釋放以上傳您的附件" @@ -1352,8 +1688,6 @@ "clear": "清除", "ingestionFailed": "匯入失敗。" }, - "showAdvanced": "顯示進階選項", - "hideAdvanced": "隱藏進階選項", "ingestors": { "local_file": { "label": "上傳檔案", @@ -1371,26 +1705,6 @@ "label": "GitHub", "heading": "從GitHub新增內容" }, - "reddit": { - "label": "Reddit", - "heading": "從Reddit新增內容" - }, - "google_drive": { - "label": "Google Drive", - "heading": "從Google Drive上傳" - }, - "s3": { - "label": "Amazon S3", - "heading": "從Amazon S3新增內容" - }, - "share_point": { - "label": "SharePoint", - "heading": "從SharePoint上傳" - }, - "confluence": { - "label": "Confluence", - "heading": "從Confluence上傳" - }, "wiki": { "label": "新 Wiki", "heading": "建立動態 Wiki" @@ -1444,13 +1758,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "AWS 存取金鑰 ID", - "awsSecretAccessKey": "AWS 秘密存取金鑰", - "bucket": "儲存貯體名稱", - "prefix": "路徑前綴（選填）", - "region": "AWS 區域", - "endpointUrl": "自訂端點 URL（選填）", "initialContent": "初始內容（選填）" + }, + "github": { + "privateHint": "私人儲存庫？連結你的 GitHub 帳號即可同步。", + "connectedHint": "私人儲存庫透過你的 GitHub 連線（{{account}}）同步。", + "connect": "連結 GitHub", + "pickRepository": "選擇儲存庫" + }, + "connectData": { + "title": "連接你的資料", + "description": "同步 {{services}}", + "more": "更多服務", + "intro": "選擇一個服務並登入，然後選擇要同步到知識庫的內容。", + "connectedAs": "已以 {{account}} 連線", + "browseAll": "瀏覽所有連接器" } }, "createAPIKey": { @@ -1487,8 +1809,8 @@ "configTool": { "title": "工具設定", "type": "類型", - "apiKeyLabel": "API 金鑰 / OAuth", - "apiKeyPlaceholder": "輸入 API 金鑰 / OAuth", + "apiKeyLabel": "API 金鑰", + "apiKeyPlaceholder": "輸入 API 金鑰", "addButton": "新增工具", "closeButton": "關閉", "customNamePlaceholder": "Enter custom name (optional)", @@ -1566,6 +1888,24 @@ "learnMore": "Learn more", "resetKey": "重設金鑰", "resetKeyConfirm": "確定要重設 API 金鑰嗎？目前的金鑰將立即停止運作，此操作無法復原。", + "apiWrites": { + "title": "他人以你的身分進行的變更", + "description": "透過 API 金鑰、小工具或代理的公開連結，無人能替你核准操作，因此除非你在此允許，否則使用你的已連線帳號和已儲存憑證的變更都會被封鎖。", + "saveFailed": "無法儲存。請再試一次。", + "summaryNone": "尚未允許任何變更", + "summaryTools_one": "{{tools}} 可以進行變更", + "summaryTools_other": "{{tools}} 可以進行變更", + "summaryCount": "已允許 {{formatted}} 項中的 {{allowed}} 項", + "toolCount": "透過 API、小工具和公開連結的變更 · 已允許 {{formatted}} 項中的 {{allowed}} 項", + "toolLabel": "允許 {{tool}} 透過 API、小工具和公開連結進行變更", + "choice": { + "off": "關閉", + "all": "全部" + }, + "customize_one": "分別設定 {{formatted}} 項", + "customize_other": "分別設定 {{formatted}} 項", + "fold": "隱藏變更" + }, "actionFailed": "操作未成功，請再試一次。", "apiKeyAfterPublish": "發布此代理後即可建立其 API 金鑰。" }, @@ -1659,13 +1999,17 @@ }, "sources": { "title": "來源", - "text": "來源文字", + "text": "選擇知識", "link": "來源連結", "view_more": "查看更多 {{count}} 個來源", - "noSourcesAvailable": "沒有可用的來源", - "selectedCount": "已選擇 {{count}} 個來源", - "groupOwn": "我的來源", - "groupTeam": "團隊共享" + "noSourcesAvailable": "還沒有知識", + "selectedCount": "已選 {{count}} 個", + "groupOwn": "你的", + "groupTeam": "團隊共享", + "connectMore": "連線更多", + "fromConnector": "來自 {{name}}", + "fromConnectorTitle": "來自 {{name}}：{{title}}", + "knowledge": "知識庫" }, "attachments": { "attach": "附件", @@ -1706,7 +2050,9 @@ "accessingMemory": "正在存取記憶…", "runningCode": "正在執行程式碼…", "creatingArtifact": "正在建立 Artifact…", - "usingTool": "正在使用 {{tool}}…" + "usingTool": "正在使用 {{tool}}…", + "searchingConnector": "正在搜尋 {{name}}...", + "usingConnector": "正在使用 {{name}}..." }, "toolChip": { "searchingWeb": "已搜尋網路 · 「{{query}}」", @@ -1717,7 +2063,10 @@ "accessingMemory": "已存取記憶", "runningCode": "已執行程式碼", "creatingArtifact": "已建立 Artifact", - "usingTool": "已使用 {{tool}}" + "usingTool": "已使用 {{tool}}", + "searchedConnector": "已搜尋 {{name}}", + "readConnector": "已從 {{name}} 讀取", + "usedConnector": "已使用 {{name}}：{{action}}" }, "inlineSteps": { "arguments": "參數", @@ -1767,7 +2116,12 @@ "approveAlways": "核准，不再詢問", "deny": "拒絕", "details": "詳細資料", - "denyReasonPlaceholder": "拒絕原因（選填）…" + "denyReasonPlaceholder": "拒絕原因（選填）…", + "connectTitle": "此工具需要你的 {{name}} 帳號。", + "connect": "連線 {{name}}", + "continue": "繼續", + "skip": "略過", + "thisService": "此服務" }, "research": { "title": "研究", @@ -1782,7 +2136,6 @@ "agents": { "title": "代理", "edit": "編輯", - "view": "檢視", "card": { "pin": "釘選代理", "unpin": "取消釘選代理", @@ -1876,7 +2229,7 @@ "placeholders": { "agentName": "代理名稱", "describeAgent": "描述您的代理", - "selectSources": "選擇來源", + "selectSources": "選擇知識", "chunksPerQuery": "每次查詢的區塊數", "selectType": "選擇類型", "selectTools": "選擇工具", @@ -1886,20 +2239,23 @@ "enterRequestLimit": "輸入請求限制" }, "sourcePopup": { - "title": "選擇來源", - "searchPlaceholder": "搜尋來源...", - "noOptionsMessage": "沒有可用的來源", + "title": "選擇知識", + "searchPlaceholder": "搜尋知識...", + "noOptionsMessage": "還沒有知識", "groupOwn": "我的來源", "groupTeam": "團隊共享", - "noSourceHint": "可選。未選擇來源時，代理僅根據模型和工具作答。" + "noSourceHint": "選填。沒有知識時，代理只用模型和它的工具回答。" }, "toolsPopup": { "title": "選擇工具", "searchPlaceholder": "搜尋工具...", "noOptionsMessage": "沒有可用的工具", "groupBuiltin": "內建", - "groupDefault": "預設", - "groupCustom": "自訂" + "groupDefault": "內建", + "groupCustom": "自訂", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "已在此智慧代理上", + "attachedHint": "由其他人新增。你可以將其移除。" }, "modelsPopup": { "title": "選擇模型", @@ -1919,7 +2275,7 @@ "defaultModel": "預設模型", "name": "名稱", "description": "描述", - "sources": "來源", + "sources": "知識庫", "avatar": "頭像" }, "advanced": { @@ -2007,11 +2363,69 @@ "published": "已發佈" }, "sponsors": { - "attachNote": "你從自己的資料庫新增的工具、來源和提示詞，會以你的存取權限為所有使用此智慧代理的人執行。", + "attachNote": "你新增的、擁有者無法使用的工具、來源和提示詞，會以你的存取權限為所有使用此智慧代理的人執行。你必須擁有它們或能夠編輯它們，並且需要確認。", "publicLinkNote": "此智慧代理有公開連結，任何擁有該連結的人都可以從中取得回答。", "addedBy": "由 {{person}} 新增：{{names}}", - "unavailable": "未執行：{{names}}。新增它們的 {{person}} 已失去存取權限。請移除它們，或選擇擁有者可以使用的項目。", - "unknownItem": "未命名項目" + "addedByOther": "由其他人新增：{{names}}", + "unknownItem": "未命名項目", + "notAllowed": "無法將 {{names}} 新增到此智慧代理。擁有者無法使用它們，而你只能分享自己擁有或可以編輯的項目。", + "takeOver": "以我的存取權限執行 {{name}}", + "takeOverPending": "儲存後，{{name}} 將以你的存取權限執行。", + "undoTakeOver": "復原", + "confirmationOutdated": "在你確認以你的存取權限執行的內容之後，此智慧代理已變更。請查看提示並重新儲存。" + }, + "resourceStates": { + "title": "部分工具、來源或提示詞未在執行", + "reason": { + "deleted": "{{name}} 已被刪除。", + "ownerLostAccessYou": "你已無法使用 {{name}}。", + "ownerLostAccess": "代理的擁有者已無法使用 {{name}}。", + "sponsorCannotEditAgent": "{{name}} 先前以 {{person}} 的存取權限執行，而對方已無法編輯此代理。", + "sponsorCannotEditAgentOther": "{{name}} 先前以其他人的存取權限執行，而對方已無法編輯此代理。", + "sponsorCannotEditItem": "{{name}} 先前以 {{person}} 的存取權限執行，而對方已無法編輯它。", + "sponsorCannotEditItemOther": "{{name}} 先前以其他人的存取權限執行，而對方已無法編輯它。", + "connectionNeedsReconnect": "{{name}} 的 {{service}} 帳戶需要重新登入。", + "connectionRemoved": "{{name}} 使用的 {{service}} 帳戶已被移除。", + "connectorDisabled": "管理員已關閉 {{service}}，因此 {{name}} 無法執行。", + "unknown": "{{name}} 未在執行。" + }, + "ask": { + "shareAgain": "請 {{person}} 重新分享。", + "signInAgain": "請 {{person}} 重新登入。", + "connectAgain": "請 {{person}} 重新連接該帳戶。", + "admin": "請管理員重新開啟。", + "shareAgainOwner": "請其擁有者重新分享。", + "signInAgainOwner": "請其擁有者重新登入。", + "connectAgainOwner": "請其擁有者重新連接該帳戶。" + }, + "promptFallback": "在此之前，代理會使用預設提示詞。", + "serviceFallback": "該服務", + "remove": "移除", + "removeLabel": "移除 {{name}}", + "reconnectLabel": "重新連接 {{name}}", + "unnamed": { + "tool": "工具 {{id}}", + "source": "來源 {{id}}", + "prompt": "提示詞 {{id}}" + }, + "chip": "未在執行：{{count}}" + }, + "sponsorConfirm": { + "title": "以你的存取權限執行？", + "description": "智慧代理的擁有者無法使用這些項目。如果新增它們，它們會以你的存取權限為所有使用此智慧代理的人執行。", + "audienceTitle": "誰可以透過此智慧代理使用它們", + "audienceTeams": "{{teams}} 的成員", + "audienceApiKey": "任何擁有此智慧代理 API 金鑰的人，包括其網站小工具", + "audiencePublicLink": "任何擁有此智慧代理公開連結的人", + "audienceWebhook": "任何呼叫此智慧代理 Webhook 的程式", + "audienceEditors": "所有可以開啟此智慧代理的人", + "stopNote": "如果你失去對它們或此智慧代理的編輯權限，它們將停止執行。之後它們仍附加在智慧代理上但不起作用，直到有編輯權限的人接管它們或將其移除。", + "confirm": "以我的存取權限新增", + "types": { + "tool": "工具", + "source": "來源", + "prompt": "提示詞" + } } }, "logs": { @@ -2303,7 +2717,7 @@ "outputVariable": "輸出變數", "outputVariablePlaceholder": "輸出的變數名稱", "streamToUser": "將輸出串流給使用者", - "sources": "來源", + "sources": "知識庫", "documents": "文件", "documentsHint": "從上傳或上游節點傳給此代理的文件。", "filePassing": "檔案傳遞", @@ -2342,7 +2756,8 @@ "detailsTitle": "工作流程詳細資料", "detailsDescription": "顯示在代理卡片上的名稱、說明和圖片。", "unsavedChanges": "未儲存的變更", - "editDetailsMenu": "編輯詳細資料" + "editDetailsMenu": "編輯詳細資料", + "attachedOption": "{{name}}（由其他人新增）" }, "validation": { "nameRequired": "工作流程名稱為必填", @@ -2448,6 +2863,7 @@ "workflowTitle": "工作流程代理", "workflowDescription": "設計包含不同模型、條件邏輯和狀態管理的複雜多步驟工作流程" }, + "view": "檢視", "deleteFailed": "無法刪除此代理，請再試一次。" }, "components": { @@ -2473,9 +2889,9 @@ "empty": "尚無待辦事項" }, "note": { - "lines_one": "{{count}} 行", "lines_other": "{{count}} 行", - "empty": "空白筆記" + "empty": "空白筆記", + "lines_one": "{{count}} 行" }, "notFound": "找不到 Artifact", "loadFailed": "無法載入 Artifact", @@ -2516,7 +2932,8 @@ "shared": "共用", "connectTo": "連接到 {{provider}}", "sessionExpiredFor": "工作階段已過期。請重新連接到 {{provider}}。", - "selectFilesFrom": "從 {{provider}} 選擇檔案" + "selectFilesFrom": "從 {{provider}} 選擇檔案", + "account": "帳號" }, "actionButtons": { "openNewChat": "開啟新聊天", @@ -2574,5 +2991,15 @@ }, "prompts": { "textAriaLabel": "提示文字" + }, + "connectHero": { + "title": "連線你的資料", + "body": "與 {{names}} 等進行對話。" + }, + "common": { + "close": "關閉", + "viewOnlyNotice": "你可以檢視此內容，但你的角色無法變更它。", + "credentialsLockedNotice": "只有擁有者可以變更此工具的憑證。", + "savedSecretHint": "已儲存。留空即可保留。" } } diff --git a/frontend/src/locale/zh.json b/frontend/src/locale/zh.json index 2bdc5001..860fba48 100644 --- a/frontend/src/locale/zh.json +++ b/frontend/src/locale/zh.json @@ -10,12 +10,6 @@ "loading": "加载中...", "retry": "重试", "cancel": "取消", - "common": { - "close": "关闭", - "viewOnlyNotice": "你可以查看此内容，但你的角色无法更改它。", - "credentialsLockedNotice": "只有所有者可以更改此工具的凭据。", - "savedSecretHint": "已保存。留空即可保留。" - }, "help": "帮助", "emailUs": "给我们发邮件", "documentation": "文档", @@ -106,16 +100,16 @@ }, "sources": { "title": "在这里您可以管理所有可用的源文件以及您上传的文件。", - "subtitle": "上传并管理您的回答所依赖的文档和知识来源", - "label": "来源", + "subtitle": "DocsGPT 可以搜索和引用的内容：上传的文件、网站、Wiki，以及从连接同步的内容。", + "label": "知识库", "name": "来源名称", "date": "向量日期", "type": "类型", "tokenUsage": "令牌使用", - "noData": "没有现有的来源", + "noData": "还没有知识", "searchPlaceholder": "搜索...", "addNew": "添加新文档", - "addSource": "添加来源", + "addSource": "添加知识", "addChunk": "添加块", "preLoaded": "预加载", "private": "私有", @@ -176,7 +170,6 @@ } }, "badge": "动态 Wiki", - "byline_one": "{{pages}} 个页面 · {{tokens}} 个 token", "byline_other": "{{pages}} 个页面 · {{tokens}} 个 token", "explainer": "代理在工作时会阅读并改写这些页面。", "explainerEditable": "代理在工作时会阅读并改写这些页面；任何使用此来源的代理都可以编辑它。", @@ -185,7 +178,18 @@ "pagesTitle": "页面", "loadFailed": "无法加载 Wiki 页面。", "pageLoadFailed": "无法加载此页面。", - "editTitle": "编辑页面" + "editTitle": "编辑页面", + "settings": { + "action": "Wiki 设置", + "title": "Wiki 设置", + "outsideEdits": { + "label": "允许 API 和小组件用户编辑此 Wiki", + "description": "关闭时，通过智能体的 API 密钥或小组件使用它的人可以阅读此 Wiki，但不能更改。公开链接的访问者只能编辑他们自己有权编辑的 Wiki，并且每次更改前都会被询问。你和此 Wiki 的编辑者始终可以编辑它。" + }, + "loadError": "无法加载 Wiki 设置。", + "saveError": "无法保存更改。请重试。" + }, + "byline_one": "{{pages}} 个页面 · {{tokens}} 个 token" }, "graphrag": { "badge": "GraphRAG", @@ -235,7 +239,6 @@ "byConnections": "（按连接数）", "typeFilter": "按类型筛选", "otherTypes": "其他：{{types}}", - "otherTypesMore_one": "另外 {{formatted}} 个", "otherTypesMore_other": "另外 {{formatted}} 个", "untyped": "无类型", "loadFailed": "无法加载图谱。", @@ -247,7 +250,6 @@ "showMore": "显示更多", "showLess": "收起", "relationships": "关系", - "relationshipsCapped_one": "显示 {{total}} 个关系中最强的 {{shown}} 个。", "relationshipsCapped_other": "显示 {{total}} 个关系中最强的 {{shown}} 个。", "noRelationships": "此实体没有关系。", "relatedTo": "相关", @@ -256,18 +258,20 @@ "sourceChunks": "来源分块", "showInGraph": "在图谱中显示", "allTypes": "所有类型", - "chunkCount_one": "{{formatted}} 个分块", "chunkCount_other": "{{formatted}} 个分块", - "entityCount_one": "{{formatted}} 个实体", "entityCount_other": "{{formatted}} 个实体", - "relationshipCount_one": "{{formatted}} 个关系", "relationshipCount_other": "{{formatted}} 个关系", "chunk": "分块", "chunkMeta": "{{file}} · {{tokens}} 个令牌", "chunkTokens": "{{tokens}} 个令牌", "openInFiles": "在文件中打开", "editChunk": "编辑分块", - "chunkSaved": "分块已保存" + "chunkSaved": "分块已保存", + "otherTypesMore_one": "另外 {{formatted}} 个", + "relationshipsCapped_one": "显示 {{total}} 个关系中最强的 {{shown}} 个。", + "chunkCount_one": "{{formatted}} 个分块", + "entityCount_one": "{{formatted}} 个实体", + "relationshipCount_one": "{{formatted}} 个关系" } }, "testRetrieval": { @@ -302,7 +306,7 @@ "chunks": "文本块", "noChunks": "未找到文本块", "noChunksAlt": "未找到文本块", - "goToSources": "前往来源", + "goToSources": "前往知识库", "uploadNew": "上传新文档", "noResults": "未找到结果", "fileName": "名称", @@ -426,11 +430,9 @@ "chunkTitleHint": "回答引用此文本块时使用的名称。", "files": "文件", "filesLoadError": "无法加载文件", - "filesByline_one": "{{files}} 个文件 · {{tokens}} 个令牌", "filesByline_other": "{{files}} 个文件 · {{tokens}} 个令牌", "filterFiles": "筛选文件", "searchChunks": "搜索文本块", - "chunkCount_one": "{{formatted}} 个文本块", "chunkCount_other": "{{formatted}} 个文本块", "editor": { "write": "编辑", @@ -445,6 +447,11 @@ "editChunkDescription": "{{file}} · 第 {{n}} 个文本块 · {{tokens}} 个令牌", "previousChunk": "上一个文本块", "nextChunk": "下一个文本块", + "paused": "同步已暂停。重新连接 {{name}} 以继续。", + "emptyHint": "上传文件、添加网站，或连接一个服务来同步它的内容。", + "connectService": "连接服务", + "filesByline_one": "{{files}} 个文件 · {{tokens}} 个令牌", + "chunkCount_one": "{{formatted}} 个文本块", "viewConfig": "查看来源设置", "errors": { "forbidden": "你无权对此来源执行该操作。", @@ -675,10 +682,6 @@ "createTeamError": "无法创建团队。", "noTeams": "暂无团队。", "noDescription": "无描述", - "memberCount_one": "{{formatted}} 名成员", - "memberCount_other": "{{formatted}} 名成员", - "sharedCount_one": "{{formatted}} 项共享", - "sharedCount_other": "{{formatted}} 项共享", "loadError": "加载团队失败。", "roleAdmin": "管理员", "roleMember": "成员", @@ -758,8 +761,46 @@ "teams": "团队", "people": "人员", "editors": "编辑者" + }, + "uses": { + "title": "此智能体使用的内容", + "intro": "使用此智能体的每个人都会用到这些内容，各自按所示的访问权限运行。", + "stopped": "已停止", + "writesOff": "不可通过 API", + "writesNote": "连接器上的“允许”适用于聊天。标记为“不可通过 API”的更改，在你于 Access Details 中允许之前，API、小组件和公开链接的用户无法进行。", + "writesNoteEditor": "连接器上的“允许”适用于聊天。标记为“不可通过 API”的更改，在所有者于 Access Details 中允许之前，API、小组件和公开链接的用户无法进行。", + "access": { + "you": "你的访问权限", + "person": "{{person}} 的访问权限", + "other": "其他人的访问权限", + "owner": "所有者的访问权限", + "member": "每个人自己的 {{service}} 账号（API 和小组件：你的）", + "memberNoService": "每个人自己的账号（API 和小组件：你的）", + "yourAccount": "你的 {{service}} 账号", + "yourAccountNoService": "你的账号", + "personAccount": "{{person}} 的 {{service}} 账号", + "personAccountNoService": "{{person}} 的账号", + "memberShared": "每个人自己的 {{service}} 账号（API 和小组件：所有者的）", + "memberSharedNoService": "每个人自己的账号（API 和小组件：所有者的）", + "otherAccount": "其他人的 {{service}} 账号", + "otherAccountNoService": "其他人的账号", + "yourCredentials": "你保存的凭据", + "personCredentials": "{{person}} 保存的凭据", + "otherCredentials": "其他人保存的凭据" + }, + "writesSomeOff": "部分不可通过 API", + "adminOff": "管理员已关闭更改", + "writesNoteApi": "连接器上的“允许”适用于聊天。标记为“不可通过 API”的更改，在你于 Access Details 中允许之前，API 和小组件的用户无法进行。", + "writesNoteApiEditor": "连接器上的“允许”适用于聊天。标记为“不可通过 API”的更改，在所有者于 Access Details 中允许之前，API 和小组件的用户无法进行。", + "writesNoteMemberTail": "在每个人使用自己账号的工具上，公开链接的用户用自己的账号进行更改。", + "openAccessDetails": "打开 Access Details", + "accessDetailsNeedsKey": "智能体有 API 密钥后，Access Details 中会列出这些更改。" } }, + "memberCount_one": "{{formatted}} 名成员", + "memberCount_other": "{{formatted}} 名成员", + "sharedCount_one": "{{formatted}} 项共享", + "sharedCount_other": "{{formatted}} 项共享", "accessChangeError": "无法更改访问权限。", "accessSettings": { "title": "访问设置", @@ -807,7 +848,8 @@ }, "viewers_can_use_in_agents": { "label": "查看者可以在自己的代理中使用", - "description": "它使用你的凭据运行。" + "description": "它使用你的凭据运行。", + "descriptionMember": "每个人使用自己的账号。" } }, "prompt": { @@ -929,11 +971,11 @@ }, "tools": { "label": "工具", - "subtitle": "浏览、连接并配置驱动您的智能体的工具和集成", + "subtitle": "智能体可以执行的操作。内置工具在这里。已连接服务提供的工具归在该服务下。", "searchPlaceholder": "搜索工具...", "addTool": "添加工具", "noToolsFound": "未找到工具", - "selectToolSetup": "选择要设置的工具", + "selectToolSetup": "添加工具", "settingsIconAlt": "设置图标", "configureToolAria": "配置 {{toolName}}", "toggleToolAria": "切换 {{toolName}}", @@ -945,8 +987,8 @@ "builtIn": "内置", "authStatus": { "connected": "已连接", - "needsAuth": "需要认证", - "configured": "已配置" + "needsAuth": "重新连接", + "configured": "已连接" }, "deleteWarning": "您确定要删除工具 \"{{toolName}}\" 吗？", "unsavedChanges": "您有未保存的更改，如果不保存就离开将会丢失。", @@ -1062,6 +1104,11 @@ }, "headerValuePlaceholder": "例如：application/json", "toolIconTitle": "{{name}} 图标", + "groupBuiltIn": "内置", + "groupService": "来自服务", + "browseConnectors": "浏览所有连接器", + "viaConnection": "通过 {{name}}", + "off": "已关闭", "view": "查看", "inMyChats": "在我的聊天中", "useInMyChatsAria": "在我的聊天中使用 {{toolName}}", @@ -1297,13 +1344,310 @@ "subtitle": "“{{name}}”的新密钥已就绪，旧密钥已失效。" } } + }, + "connectors": { + "label": "连接器", + "subtitle": "将 DocsGPT 连接到团队使用的服务。一个连接可以把内容同步到知识库，也可以为智能体提供工具。", + "search": "搜索连接器...", + "addCustom": "添加自定义工具", + "custom": { + "custom_mcp": "MCP 服务器", + "custom_openapi": "OpenAPI / REST" + }, + "categoriesLabel": "按类别筛选连接器", + "categories": { + "all": "全部", + "connected": "已连接", + "files": "文件与存储", + "knowledge": "文档与 Wiki", + "dev": "开发者工具", + "messaging": "消息", + "database": "数据库", + "search": "搜索与网页", + "custom": "自定义", + "projects": "项目与事务", + "business": "支付与业务" + }, + "capability": { + "sync": "同步", + "read": "读取", + "write": "写入" + }, + "status": { + "connect": "连接", + "connected": "已连接", + "connectedCount": "已连接 {{formatted}} 个", + "reconnect": "重新连接", + "needsAdminSetup": "需要管理员设置", + "disabledByAdmin": "已被管理员停用", + "custom": "自定义" + }, + "connectionStatus": { + "connected": "已连接", + "reconnect_needed": "需要重新连接", + "disconnected": "已断开", + "error": "错误", + "pending": "等待登录" + }, + "publisher": { + "built_in": "内置", + "preset": "预设", + "custom": "自定义" + }, + "descriptions": { + "google_drive": "将文档、表格和 PDF 同步到知识库。", + "share_point": "将 SharePoint 站点和 OneDrive 中的文件同步到知识库。", + "confluence": "将 Confluence 空间和页面同步到知识库。", + "s3": "将 S3 存储桶中的文档同步到知识库。", + "reddit": "将符合你搜索条件的 Reddit 帖子同步到知识库。", + "brave": "使用 Brave Search API 搜索网页和图片。", + "telegram": "向 Telegram 聊天发送消息和图片。", + "ntfy": "通过 ntfy 服务器发送推送通知。", + "postgres": "读取 Postgres 数据库的结构并执行 SQL。", + "custom_mcp": "连接任意远程 Model Context Protocol 服务器。", + "custom_openapi": "导入 OpenAPI 规范，并将其端点作为工具调用。", + "mcp_notion": "搜索、阅读和更新 Notion 页面与数据库。", + "mcp_linear": "将问题和文档同步到知识库，并让智能体查找、创建和更新问题。", + "mcp_atlassian": "搜索和更新 Jira 问题与 Confluence 页面。", + "mcp_sentry": "查看 Sentry 问题、事件和发布版本。", + "mcp_asana": "查找和更新 Asana 任务与项目。", + "mcp_stripe": "查看 Stripe 中的客户、付款和订阅。", + "github": "将仓库同步到知识库，并让智能体读取代码、Issue 和拉取请求。" + }, + "empty": "尚未连接任何服务。连接一个服务以同步其内容，或为智能体提供新工具。", + "noMatches": "没有与搜索匹配的连接器。", + "loadFailed": "无法加载连接器。", + "askAdmin": "请让管理员完成设置。", + "setupSettings": "设置以下服务器配置，然后重启 DocsGPT：", + "setupGuide": "设置指南", + "unverified": "此服务器未经 DocsGPT 审核。请只连接你信任的服务器。", + "detail": { + "accounts": "账号", + "connectedAs": "已作为 {{account}} 连接", + "connectAnother": "连接其他账号", + "sources": "来自此连接的知识", + "tools": "来自此连接的工具", + "noAccounts": "尚未连接账号。", + "failed": "无法加载此连接。", + "lastSync": "已于 {{date}} 同步 · {{frequency}}", + "paused": "同步已暂停", + "toolOn": "开启", + "toolOff": "关闭", + "accountMenu": "账号操作", + "disconnect": "断开连接", + "keyEnding": "以 {{hint}} 结尾的密钥", + "syncMore": "同步更多内容", + "refreshTools": "刷新工具", + "noSources": "尚未从此连接同步任何内容。", + "remove": "移除", + "refreshed": "工具已刷新：新增 {{added}} 个，移除 {{removed}} 个。", + "refreshFailed": "无法刷新工具。", + "syncNow": "立即同步", + "syncStarted": "正在同步 {{name}}…", + "syncFailed": "无法开始同步。请重试。", + "toolSwitchFailed": "无法更改工具。请重试。", + "account": "账号：{{account}}", + "expired": "登录已过期。重新连接以继续使用此连接。", + "broken": "此连接已停止工作。重新连接以再试一次。", + "addTools": "添加工具", + "noTools": "此连接还没有工具。", + "addToolsFailed": "无法添加工具。请重试。", + "rename": "重命名", + "accountPicker": "选择账号" + }, + "permission": { + "always": "允许", + "ask": "先询问", + "off": "关闭", + "label": "{{action}} 的权限", + "saveFailed": "无法保存权限。请重试。", + "groupLabel": "{{group}} 的全部权限", + "fold": "隐藏操作", + "hint": "“允许”和“先询问”适用于聊天：选择“先询问”时，DocsGPT 会在执行操作前询问你。通过智能体的 API、小组件或公开链接进行的更改，需在该智能体的 Access Details 中逐个智能体允许。", + "actionCount_other": "{{formatted}} 个操作", + "customize_other": "分别设置 {{formatted}} 个" + }, + "disconnect": { + "title": "断开 {{name}}？", + "body_zero": "在重新连接之前，此连接的工具将无法使用。", + "body_other": "已同步的 {{formatted}} 项会保留在知识库中，但不再更新。此连接的工具在重新连接前无法使用。", + "failed": "无法断开连接。请重试。" + }, + "fields": { + "token": "API 密钥", + "aws_access_key_id": "访问密钥 ID", + "aws_secret_access_key": "秘密访问密钥", + "region": "区域", + "endpoint_url": "自定义端点 URL", + "client_id": "客户端 ID", + "client_secret": "客户端密钥", + "user_agent": "User agent", + "bucket": "存储桶", + "prefix": "路径前缀", + "search_queries": "搜索词", + "number_posts": "帖子数量", + "telegram_token": "机器人令牌", + "ntfy_token": "访问令牌", + "postgres_token": "连接字符串", + "brave_token": "API 密钥", + "github_access_token": "个人访问令牌", + "repo_url": "仓库", + "telegram_chat_id": "默认聊天 ID" + }, + "wizard": { + "connectTitle": "连接 {{name}}", + "reconnectTitle": "重新连接 {{name}}", + "signIn": "登录 {{name}}", + "chooseWhatToSync": "选择要同步的内容", + "skip": "暂时跳过", + "syncFrequency": "同步", + "doneTitle": "{{name}} 已连接", + "doneSummary": "已以 {{account}} 身份连接。{{sources}} 正在同步到知识库，{{tools}} 已就绪。", + "doneSummaryNone": "已作为 {{account}} 连接。", + "sourcesCount_other": "{{formatted}} 项", + "toolsCount_other": "{{formatted}} 个工具", + "toolsHeading": "工具（{{formatted}}）", + "tryInChat": "在聊天中试试", + "done": "完成", + "connectFailed": "无法连接。请检查信息后重试。", + "syncFailed": "无法开始同步。请重试。", + "connectedWithKey": "已使用以 {{hint}} 结尾的密钥连接。", + "doneCounts": "{{sources}} 正在同步到知识库，{{tools}} 已就绪。", + "doneSources": "{{sources}} 正在同步到知识库。", + "doneTools": "{{tools}}已就绪。", + "waiting": "请在打开的窗口中完成 {{name}} 登录。", + "openSignIn": "打开 {{name}} 登录", + "chooseWhatToSetUp": "选择要设置的内容", + "continue": "继续", + "methodLabel": "连接方式", + "methodOauth": "使用 {{name}} 登录", + "methodToken": "使用访问令牌", + "credentialsRejected": "{{name}} 未接受这些凭据。请检查后重试。", + "toolsUnavailable": "无法访问 {{name}} 的工具。请重试，或将其关闭以继续。", + "addTools": "允许智能体使用 {{name}}", + "addToolsDescription": { + "read": "智能体可以查询信息，但不能做任何更改。", + "readWrite": "智能体可以查询信息并进行更改。更改需要你的批准。" + }, + "accountName": "为此账户命名", + "accountNameHint": "可选。让你和 AI 区分你的账户，例如“告警机器人”。", + "syncToKnowledge": "同步到知识库", + "syncToKnowledgeDescription": "从 {{name}} 选择要在回答中搜索的内容。关闭时仅建立连接，之后可在连接器中同步。", + "syncLater": "尚未同步任何内容到知识库。要添加内容，请在连接器中打开 {{name}}，然后选择“同步更多内容”。", + "retrievalSettings": "高级检索设置" + }, + "remove": { + "title": "移除 {{name}}？", + "description": "保存的登录信息将被删除。请选择如何处理此连接提供的内容。", + "sourcesLabel_other": "它同步到知识库的 {{formatted}} 项", + "keepSources": "保留内容（停止同步）", + "deleteSources": "删除", + "toolsLabel_other": "它的 {{formatted}} 个工具", + "deleteTools": "删除", + "keepTools": "保留并关闭", + "failed": "无法移除此连接。请重试。" + }, + "health": { + "reconnect": "重新连接 {{name}} 以继续同步。", + "reconnectSources_other": "重新连接 {{name}}：{{formatted}} 项已停止同步到知识库", + "reconnectTools": "请重新连接 {{name}}，让智能体可以再次使用它", + "reconnectBoth_other": "重新连接 {{name}}：{{formatted}} 项已停止同步，智能体也无法使用它", + "navDot": "有连接需要处理", + "signInAgain": "重新登录", + "pickerNotice": "{{name}} 需要你重新登录。" + }, + "error": { + "decrypt": "无法解密保存的凭据。请重新连接以继续。", + "defaultKey": "管理员必须先设置 ENCRYPTION_SECRET_KEY，才能连接服务。" + }, + "sharing": { + "owner": "所有人使用你的账号", + "member": "每个人连接自己的账号", + "ownerShort": "你的账号", + "memberShort": "各自的账号", + "ownerShortShared": "所有者的账号" + }, + "share": { + "heading": "使用谁的账号运行", + "ownerWarning": "团队成员将在 {{name}} 中以 {{account}} 的身份操作。", + "ownerWarningShared": "团队成员将以所有者的 {{name}} 账号操作。", + "confirmWrite": "我了解团队成员可以使用我的账号执行操作。", + "confirmWriteShared": "我了解团队成员可以使用所有者的 {{name}} 账号执行操作。", + "memberNote": "每个人使用自己的 {{name}} 账号，并在首次使用此工具时连接。通过智能体的 API 密钥或小组件使用时，使用智能体所有者的账号。", + "forced": "管理员已为此连接器的所有共享设定了此项。", + "ownerChooses": "只有所有者可以更改此项。", + "saveFailed": "无法保存更改。请重试。" + }, + "createFailed": "无法添加连接器。请重试。", + "capabilityFilter": { + "sync": "正在显示将内容同步到知识库的连接器。", + "tools": "正在显示为智能体提供工具的连接器。", + "showAll": "显示全部" + }, + "capabilityPlain": { + "sync": "同步到知识库", + "read": "查找信息", + "write": "执行操作" + }, + "github": { + "tokenHint": "为要使用的仓库创建一个细粒度令牌，并授予 Contents 和 Metadata 的读取权限。如需让智能体进行更改，还需授予 Issues 和 Pull requests 的读写权限。", + "writes": "也允许智能体进行更改（议题、评论、拉取请求）", + "writesDescription": "默认情况下，每次更改都会先征求你的同意。你的令牌或 GitHub App 需要 Issues 和 Pull requests 的读写权限。", + "writesForbidden": "管理员已关闭通过 GitHub 进行更改。", + "writesFailed": "无法更改智能体在 GitHub 上可执行的操作。", + "createToken": "在 GitHub 上创建令牌", + "chooseRepositories": "在 GitHub 上选择仓库", + "noAppRepositories": "DocsGPT 还看不到任何仓库。请在 GitHub 上选择允许它读取的仓库。", + "noTokenRepositories": "此令牌无法读取任何仓库。请在 GitHub 上为其授予访问权限，然后重新连接。", + "searchRepositories": "搜索仓库", + "repositories": "仓库", + "updated": "更新于 {{date}}", + "loadFailed": "无法加载你的仓库。" + }, + "linear": { + "teams": "团队", + "teamsHint": "同步这些团队的问题。", + "projects": "项目", + "projectsHint": "同步这些项目的问题。", + "search": "搜索团队和项目", + "noMatches": "没有与搜索匹配的团队或项目。", + "includeComments": "包含评论", + "includeCommentsDescription": "将每个问题的评论一并加入。", + "includeDocuments": "包含项目文档", + "includeDocumentsDescription": "同步所选项目的 Linear 文档。", + "loadFailed": "无法加载你的 Linear 团队和项目。", + "empty": "此 Linear 账户没有可同步的团队或项目。" + }, + "parameters": { + "show": "参数", + "hide": "隐藏参数", + "fixedCount_other": "已固定 {{formatted}} 个", + "ai": "由 AI 决定", + "fixed": "始终使用", + "choiceLabel": "{{parameter}} 的填写方式", + "valueLabel": "{{parameter}} 的值", + "save": "保存", + "saveFailed": "无法保存该值。请重试。", + "hint": "设为“始终使用”的值会在每次调用时发送。AI 看不到它，也无法更改。", + "fromAccount": "已在此账户中设置：{{value}}" + }, + "fieldHints": { + "telegram_chat_id": "可选。消息会发送到此聊天，AI 无法选择其他聊天。将机器人加入聊天并给它发一条消息，然后在 https://api.telegram.org/bot/getUpdates 中找到该聊天的 id。" + }, + "rename": { + "title": "重命名账户", + "description": "此名称可让你和 AI 将此账户与同一服务的其他账户区分开。留空则显示账户本身。", + "label": "账户名称", + "save": "保存", + "failed": "无法重命名账户。请重试。" + } } }, "modals": { "uploadDoc": { - "label": "上传新文档", + "label": "添加知识", "select": "选择如何将文档上传到 DocsGPT", - "selectSource": "选择添加源的方式", + "selectSource": "添加知识", "selectedFiles": "已选择的文件", "noFilesSelected": "未选择文件", "file": "从设备上传", @@ -1318,19 +1662,11 @@ "filesRejected": "未添加（超过 25 MB 或不支持的类型）：{{files}}", "uploadedFiles": "已上传文件", "cancel": "取消", - "train": "训练", + "train": "添加到知识库", "create": "创建", "link": "链接", "urlLink": "URL 链接", "repoUrl": "存储库 URL", - "reddit": { - "id": "客户端ID", - "secret": "客户端密钥", - "agent": "用户代理", - "searchQueries": "搜索查询", - "numberOfPosts": "帖子数量", - "addQuery": "添加查询" - }, "drag": { "title": "将附件拖放到此处", "description": "释放以上传您的附件" @@ -1352,8 +1688,6 @@ "clear": "清除", "ingestionFailed": "导入失败。" }, - "showAdvanced": "显示高级选项", - "hideAdvanced": "隐藏高级选项", "ingestors": { "local_file": { "label": "上传文件", @@ -1371,26 +1705,6 @@ "label": "GitHub", "heading": "从GitHub添加内容" }, - "reddit": { - "label": "Reddit", - "heading": "从Reddit添加内容" - }, - "google_drive": { - "label": "Google Drive", - "heading": "从Google Drive上传" - }, - "s3": { - "label": "Amazon S3", - "heading": "从Amazon S3添加内容" - }, - "share_point": { - "label": "SharePoint", - "heading": "从SharePoint上传" - }, - "confluence": { - "label": "Confluence", - "heading": "从Confluence上传" - }, "wiki": { "label": "新建 Wiki", "heading": "创建动态 Wiki" @@ -1444,13 +1758,21 @@ }, "fields": { "url": "URL", - "awsAccessKeyId": "AWS 访问密钥 ID", - "awsSecretAccessKey": "AWS 秘密访问密钥", - "bucket": "存储桶名称", - "prefix": "路径前缀（可选）", - "region": "AWS 区域", - "endpointUrl": "自定义端点 URL（可选）", "initialContent": "初始内容（可选）" + }, + "github": { + "privateHint": "私有仓库？连接你的 GitHub 账号即可同步。", + "connectedHint": "私有仓库通过你的 GitHub 连接（{{account}}）同步。", + "connect": "连接 GitHub", + "pickRepository": "选择仓库" + }, + "connectData": { + "title": "连接你的数据", + "description": "同步 {{services}}", + "more": "更多服务", + "intro": "选择一个服务并登录，然后选择要同步到知识库的内容。", + "connectedAs": "已作为 {{account}} 连接", + "browseAll": "浏览所有连接器" } }, "createAPIKey": { @@ -1487,8 +1809,8 @@ "configTool": { "title": "工具配置", "type": "类型", - "apiKeyLabel": "API 密钥 / OAuth", - "apiKeyPlaceholder": "输入 API 密钥 / OAuth", + "apiKeyLabel": "API 密钥", + "apiKeyPlaceholder": "输入 API 密钥", "addButton": "添加工具", "closeButton": "关闭", "customNamePlaceholder": "Enter custom name (optional)", @@ -1566,6 +1888,22 @@ "learnMore": "Learn more", "resetKey": "重置密钥", "resetKeyConfirm": "确定要重置 API 密钥吗？当前密钥将立即停止工作，此操作无法撤销。", + "apiWrites": { + "title": "他人以你的身份进行的更改", + "description": "通过 API 密钥、小组件或智能体的公开链接，无人能替你批准操作，因此除非你在此允许，否则使用你的已连接账号和已保存凭据的更改都会被阻止。", + "saveFailed": "无法保存。请重试。", + "summaryNone": "尚未允许任何更改", + "summaryTools_other": "{{tools}} 可以进行更改", + "summaryCount": "已允许 {{formatted}} 项中的 {{allowed}} 项", + "toolCount": "通过 API、小组件和公开链接的更改 · 已允许 {{formatted}} 项中的 {{allowed}} 项", + "toolLabel": "允许 {{tool}} 通过 API、小组件和公开链接进行更改", + "choice": { + "off": "关闭", + "all": "全部" + }, + "customize_other": "分别设置 {{formatted}} 项", + "fold": "隐藏更改" + }, "actionFailed": "操作未成功，请重试。", "apiKeyAfterPublish": "发布该代理后即可创建其 API 密钥。" }, @@ -1659,13 +1997,17 @@ }, "sources": { "title": "来源", - "text": "来源文本", + "text": "选择知识", "link": "来源链接", "view_more": "还有{{count}}个来源", - "noSourcesAvailable": "没有可用的来源", - "selectedCount": "已选择 {{count}} 个来源", - "groupOwn": "我的来源", - "groupTeam": "团队共享" + "noSourcesAvailable": "还没有知识", + "selectedCount": "已选 {{count}} 个", + "groupOwn": "你的", + "groupTeam": "团队共享", + "connectMore": "连接更多", + "fromConnector": "来自 {{name}}", + "fromConnectorTitle": "来自 {{name}}：{{title}}", + "knowledge": "知识库" }, "attachments": { "attach": "附件", @@ -1706,7 +2048,9 @@ "accessingMemory": "正在访问记忆…", "runningCode": "正在运行代码…", "creatingArtifact": "正在创建工件…", - "usingTool": "正在使用 {{tool}}…" + "usingTool": "正在使用 {{tool}}…", + "searchingConnector": "正在搜索 {{name}}...", + "usingConnector": "正在使用 {{name}}..." }, "toolChip": { "searchingWeb": "已搜索网络 · “{{query}}”", @@ -1717,7 +2061,10 @@ "accessingMemory": "已访问记忆", "runningCode": "已运行代码", "creatingArtifact": "已创建工件", - "usingTool": "已使用 {{tool}}" + "usingTool": "已使用 {{tool}}", + "searchedConnector": "已搜索 {{name}}", + "readConnector": "已从 {{name}} 读取", + "usedConnector": "已使用 {{name}}：{{action}}" }, "inlineSteps": { "arguments": "参数", @@ -1767,7 +2114,12 @@ "approveAlways": "批准，不再询问", "deny": "拒绝", "details": "详情", - "denyReasonPlaceholder": "拒绝原因（可选）…" + "denyReasonPlaceholder": "拒绝原因（可选）…", + "connectTitle": "此工具需要你的 {{name}} 账号。", + "connect": "连接 {{name}}", + "continue": "继续", + "skip": "跳过", + "thisService": "此服务" }, "research": { "title": "研究", @@ -1782,7 +2134,6 @@ "agents": { "title": "代理", "edit": "编辑", - "view": "查看", "card": { "pin": "固定代理", "unpin": "取消固定代理", @@ -1876,7 +2227,7 @@ "placeholders": { "agentName": "代理名称", "describeAgent": "描述您的代理", - "selectSources": "选择来源", + "selectSources": "选择知识", "chunksPerQuery": "每次查询的块数", "selectType": "选择类型", "selectTools": "选择工具", @@ -1886,20 +2237,23 @@ "enterRequestLimit": "输入请求限制" }, "sourcePopup": { - "title": "选择来源", - "searchPlaceholder": "搜索来源...", - "noOptionsMessage": "没有可用的来源", + "title": "选择知识", + "searchPlaceholder": "搜索知识...", + "noOptionsMessage": "还没有知识", "groupOwn": "我的来源", "groupTeam": "团队共享", - "noSourceHint": "可选。未选择来源时，智能体仅根据模型和工具作答。" + "noSourceHint": "可选。没有知识时，智能体只用模型和它的工具回答。" }, "toolsPopup": { "title": "选择工具", "searchPlaceholder": "搜索工具...", "noOptionsMessage": "没有可用的工具", "groupBuiltin": "内置", - "groupDefault": "默认", - "groupCustom": "自定义" + "groupDefault": "内置", + "groupCustom": "自定义", + "groupConnection": "{{name}} · {{account}}", + "groupAttached": "已在此智能体上", + "attachedHint": "由其他人添加。你可以将其移除。" }, "modelsPopup": { "title": "选择模型", @@ -1919,7 +2273,7 @@ "defaultModel": "默认模型", "name": "名称", "description": "描述", - "sources": "来源", + "sources": "知识库", "avatar": "头像" }, "advanced": { @@ -2007,11 +2361,69 @@ "published": "已发布" }, "sponsors": { - "attachNote": "你从自己的资料库添加的工具、来源和提示词，会以你的访问权限为所有使用此智能体的人运行。", + "attachNote": "你添加的、所有者无法使用的工具、来源和提示词，会以你的访问权限为所有使用此智能体的人运行。你必须拥有它们或能够编辑它们，并且需要确认。", "publicLinkNote": "此智能体有公开链接，任何拥有该链接的人都可以从中获得回答。", "addedBy": "由 {{person}} 添加：{{names}}", - "unavailable": "未运行：{{names}}。添加它们的 {{person}} 已失去访问权限。请移除它们，或选择所有者可以使用的项目。", - "unknownItem": "未命名项目" + "addedByOther": "由其他人添加：{{names}}", + "unknownItem": "未命名项目", + "notAllowed": "无法将 {{names}} 添加到此智能体。所有者无法使用它们，而你只能共享自己拥有或可以编辑的项目。", + "takeOver": "以我的访问权限运行 {{name}}", + "takeOverPending": "保存后，{{name}} 将以你的访问权限运行。", + "undoTakeOver": "撤销", + "confirmationOutdated": "在你确认以你的访问权限运行的内容之后，此智能体已更改。请查看提示并重新保存。" + }, + "resourceStates": { + "title": "部分工具、来源或提示词未在运行", + "reason": { + "deleted": "{{name}} 已被删除。", + "ownerLostAccessYou": "你已无法使用 {{name}}。", + "ownerLostAccess": "智能体的所有者已无法使用 {{name}}。", + "sponsorCannotEditAgent": "{{name}} 此前以 {{person}} 的访问权限运行，而其已无法编辑此智能体。", + "sponsorCannotEditAgentOther": "{{name}} 此前以其他人的访问权限运行，而其已无法编辑此智能体。", + "sponsorCannotEditItem": "{{name}} 此前以 {{person}} 的访问权限运行，而其已无法编辑它。", + "sponsorCannotEditItemOther": "{{name}} 此前以其他人的访问权限运行，而其已无法编辑它。", + "connectionNeedsReconnect": "{{name}} 的 {{service}} 账户需要重新登录。", + "connectionRemoved": "{{name}} 使用的 {{service}} 账户已被移除。", + "connectorDisabled": "管理员已关闭 {{service}}，因此 {{name}} 无法运行。", + "unknown": "{{name}} 未在运行。" + }, + "ask": { + "shareAgain": "请让 {{person}} 重新共享。", + "signInAgain": "请让 {{person}} 重新登录。", + "connectAgain": "请让 {{person}} 重新连接该账户。", + "admin": "请让管理员重新开启。", + "shareAgainOwner": "请让其所有者重新共享。", + "signInAgainOwner": "请让其所有者重新登录。", + "connectAgainOwner": "请让其所有者重新连接该账户。" + }, + "promptFallback": "在此之前，智能体使用默认提示词。", + "serviceFallback": "该服务", + "remove": "移除", + "removeLabel": "移除 {{name}}", + "reconnectLabel": "重新连接 {{name}}", + "unnamed": { + "tool": "工具 {{id}}", + "source": "来源 {{id}}", + "prompt": "提示词 {{id}}" + }, + "chip": "未在运行：{{count}}" + }, + "sponsorConfirm": { + "title": "以你的访问权限运行？", + "description": "智能体的所有者无法使用这些项目。如果添加它们，它们会以你的访问权限为所有使用此智能体的人运行。", + "audienceTitle": "谁可以通过此智能体使用它们", + "audienceTeams": "{{teams}} 的成员", + "audienceApiKey": "任何拥有此智能体 API 密钥的人，包括其网站小组件", + "audiencePublicLink": "任何拥有此智能体公开链接的人", + "audienceWebhook": "任何调用此智能体 Webhook 的程序", + "audienceEditors": "所有可以打开此智能体的人", + "stopNote": "如果你失去对它们或此智能体的编辑权限，它们将停止运行。之后它们仍附加在智能体上但不起作用，直到有编辑权限的人接管它们或将其移除。", + "confirm": "以我的访问权限添加", + "types": { + "tool": "工具", + "source": "来源", + "prompt": "提示词" + } } }, "logs": { @@ -2303,7 +2715,7 @@ "outputVariable": "输出变量", "outputVariablePlaceholder": "输出的变量名", "streamToUser": "将输出流式传输给用户", - "sources": "来源", + "sources": "知识库", "documents": "文档", "documentsHint": "从上传或上游节点传给此智能体的文档。", "filePassing": "文件传递", @@ -2342,7 +2754,8 @@ "detailsTitle": "工作流详情", "detailsDescription": "显示在代理卡片上的名称、描述和图片。", "unsavedChanges": "未保存的更改", - "editDetailsMenu": "编辑详情" + "editDetailsMenu": "编辑详情", + "attachedOption": "{{name}}（由其他人添加）" }, "validation": { "nameRequired": "工作流名称为必填项", @@ -2448,6 +2861,7 @@ "workflowTitle": "工作流智能体", "workflowDescription": "设计包含不同模型、条件逻辑和状态管理的复杂多步骤工作流" }, + "view": "查看", "deleteFailed": "无法删除该代理，请重试。" }, "components": { @@ -2473,9 +2887,9 @@ "empty": "暂无待办事项" }, "note": { - "lines_one": "{{count}} 行", "lines_other": "{{count}} 行", - "empty": "空笔记" + "empty": "空笔记", + "lines_one": "{{count}} 行" }, "notFound": "未找到工件", "loadFailed": "无法加载工件", @@ -2516,7 +2930,8 @@ "shared": "共享", "connectTo": "连接到 {{provider}}", "sessionExpiredFor": "会话已过期。请重新连接到 {{provider}}。", - "selectFilesFrom": "从 {{provider}} 选择文件" + "selectFilesFrom": "从 {{provider}} 选择文件", + "account": "账号" }, "actionButtons": { "openNewChat": "打开新聊天", @@ -2574,5 +2989,15 @@ }, "prompts": { "textAriaLabel": "提示文本" + }, + "connectHero": { + "title": "连接你的数据", + "body": "与 {{names}} 等进行对话。" + }, + "common": { + "close": "关闭", + "viewOnlyNotice": "你可以查看此内容，但你的角色无法更改它。", + "credentialsLockedNotice": "只有所有者可以更改此工具的凭据。", + "savedSecretHint": "已保存。留空即可保留。" } } diff --git a/frontend/src/modals/AddToolModal.test.tsx b/frontend/src/modals/AddToolModal.test.tsx new file mode 100644 index 00000000..d7250e3c --- /dev/null +++ b/frontend/src/modals/AddToolModal.test.tsx @@ -0,0 +1,177 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter, Route, Routes } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); + +vi.mock('../api/services/userService', () => ({ + default: { + getAvailableTools: vi.fn(async () => ({ + json: async () => ({ + data: [ + { + name: 'memory', + displayName: 'Memory', + description: 'Remembers', + configRequirements: {}, + actions: [], + group: 'built_in', + }, + ], + }), + })), + }, +})); + +vi.mock('../api/services/connectorsService', () => ({ + default: { + getCatalog: vi.fn(async () => ({ + success: true, + connectors: server.catalog, + })), + listConnections: vi.fn(async () => ({ success: true, connections: [] })), + }, +})); + +const launch = vi.hoisted(() => vi.fn()); +vi.mock('../connectors/useConnectorLauncher', () => ({ + default: () => ({ launch, modals: null }), +})); + +import connectorsReducer from '../connectors/connectorsSlice'; +import AddToolModal from './AddToolModal'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const entry = (overrides: Record) => ({ + description: '', + category: 'dev', + publisher: 'built_in', + capabilities: ['read', 'write'], + available: true, + connection_count: 0, + connected_count: 0, + state: 'available', + ...overrides, +}); + +const server = vi.hoisted(() => ({ catalog: [] as unknown[] })); + +describe('AddToolModal', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + launch.mockClear(); + server.catalog = [ + entry({ + key: 'mcp:notion', + name: 'Notion', + icon: 'notion', + publisher: 'preset', + }), + entry({ + key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + connection_count: 1, + connected_count: 1, + state: 'connected', + }), + entry({ + key: 'google_drive', + name: 'Google Drive', + icon: 'drive', + capabilities: ['sync'], + }), + entry({ + key: 'custom_mcp', + name: 'MCP server', + icon: 'mcp', + publisher: 'custom', + }), + ]; + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async () => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + }); + const setModalState = vi.fn(); + await act(async () => { + root.render( + + + + + } + /> + DRAWER
    } /> + + + , + ); + }); + // The loader holds its skeleton for 250 ms after the tools arrive. + await act(async () => { + await new Promise((resolve) => setTimeout(resolve, 300)); + }); + return setModalState; + }; + + const service = (key: string) => + document.body.querySelector( + `[data-testid="add-tool-service-${key}"]`, + ); + + it('lists services, presets included, before built-in tools', async () => { + await render(); + const text = document.body.textContent ?? ''; + const services = text.indexOf('settings.tools.groupService'); + expect(services).toBeGreaterThan(-1); + expect(services).toBeLessThan(text.indexOf('settings.tools.groupBuiltIn')); + expect(service('mcp:notion')).not.toBeNull(); + // Sync-only and custom connectors are not tools here. + expect(service('google_drive')).toBeNull(); + expect(service('custom_mcp')).toBeNull(); + }); + + it('connects a service that is not connected yet', async () => { + await render(); + await act(async () => service('mcp:notion')!.click()); + expect(launch).toHaveBeenCalledWith( + expect.objectContaining({ key: 'mcp:notion' }), + ); + }); + + it('opens a connected service to manage its tools', async () => { + await render(); + await act(async () => service('telegram')!.click()); + expect(launch).not.toHaveBeenCalled(); + expect(document.body.textContent).toContain('DRAWER'); + }); +}); diff --git a/frontend/src/modals/AddToolModal.tsx b/frontend/src/modals/AddToolModal.tsx index 81ae3771..10d20c16 100644 --- a/frontend/src/modals/AddToolModal.tsx +++ b/frontend/src/modals/AddToolModal.tsx @@ -1,18 +1,36 @@ +import { ArrowRight } from 'lucide-react'; import React from 'react'; import { useTranslation } from 'react-i18next'; -import { useSelector } from 'react-redux'; +import { useDispatch, useSelector } from 'react-redux'; +import { Link, useNavigate } from 'react-router-dom'; import userService from '../api/services/userService'; import SkeletonLoader from '../components/SkeletonLoader'; import ToolIcon from '../components/ToolIcon'; -import { Card, CardDescription, CardTitle } from '../components/ui/card'; -import { Modal } from '../components/ui/modal'; +import { Button } from '../components/ui/button'; +import { + Card, + CardDescription, + CardFooter, + CardTitle, +} from '../components/ui/card'; +import { Modal, ModalActions } from '../components/ui/modal'; +import { SectionHeader } from '../components/ui/section-header'; import { useLoaderState } from '../hooks'; +import { + loadConnectors, + selectConnectorCatalog, +} from '../connectors/connectorsSlice'; +import { ConnectorStateBadge } from '../connectors/ConnectorCard'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { connectorDescription, connectorName } from '../connectors/i18n'; +import type { ConnectorDefinition } from '../connectors/types'; +import useConnectorLauncher from '../connectors/useConnectorLauncher'; import PairDeviceModal from '../settings/PairDeviceModal'; +import type { AppDispatch } from '../store'; import { ActiveState } from '../models/misc'; import { selectToken } from '../preferences/preferenceSlice'; import ConfigToolModal from './ConfigToolModal'; -import MCPServerModal from './MCPServerModal'; import { AvailableToolType } from './types'; export default function AddToolModal({ @@ -31,7 +49,13 @@ export default function AddToolModal({ onDevicePaired?: (deviceId: string) => void; }) { const { t } = useTranslation(); + const navigate = useNavigate(); const token = useSelector(selectToken); + const dispatch = useDispatch(); + const catalog = useSelector(selectConnectorCatalog); + const { launch, modals: connectModals } = useConnectorLauncher({ + onConnected: getUserTools, + }); const [availableTools, setAvailableTools] = React.useState< AvailableToolType[] >([]); @@ -39,8 +63,6 @@ export default function AddToolModal({ React.useState(null); const [configModalState, setConfigModalState] = React.useState('INACTIVE'); - const [mcpModalState, setMcpModalState] = - React.useState('INACTIVE'); const [pairModalState, setPairModalState] = React.useState('INACTIVE'); const [loading, setLoading] = useLoaderState(false); @@ -53,11 +75,39 @@ export default function AddToolModal({ return res.json(); }) .then((data) => { - setAvailableTools(data.data); + setAvailableTools( + (data.data as AvailableToolType[]).filter( + (tool) => tool.group !== 'custom', + ), + ); setLoading(false); }); }; + // Services come from the connector catalog (tool connectors and MCP + // presets): they are added by connecting the service, first in the list. + const services = catalog.filter( + (connector) => + connector.publisher !== 'custom' && + connector.available && + connector.capabilities.some((c) => c === 'read' || c === 'write'), + ); + const builtIn = availableTools.filter( + (tool) => (tool.group ?? 'built_in') === 'built_in', + ); + + const openService = (connector: ConnectorDefinition) => { + setModalState('INACTIVE'); + // Already connected: its tools exist; open its drawer to manage them. + if (connector.connection_count > 0) { + navigate( + `/settings/connectors?connector=${encodeURIComponent(connector.key)}`, + ); + return; + } + launch(connector); + }; + const handleAddTool = (tool: AvailableToolType) => { // ``remote_device`` is created server-side via the pairing redeem // endpoint, not the standard create_tool path. @@ -66,6 +116,15 @@ export default function AddToolModal({ setPairModalState('ACTIVE'); return; } + // A service's tool is added by connecting the service: the key goes on + // a connection and the tool is created with write actions needing + // approval. + const connector = catalog.find((c) => c.key === tool.connector_key); + if (tool.group === 'service' && connector) { + setModalState('INACTIVE'); + launch(connector); + return; + } if (Object.keys(tool.configRequirements).length === 0) { userService .createTool( @@ -96,9 +155,6 @@ export default function AddToolModal({ .catch((error) => { console.error('Failed to create tool:', error); }); - } else if (tool.name === 'mcp_tool') { - setModalState('INACTIVE'); - setMcpModalState('ACTIVE'); } else { setModalState('INACTIVE'); setConfigModalState('ACTIVE'); @@ -106,14 +162,12 @@ export default function AddToolModal({ }; React.useEffect(() => { - if (modalState === 'ACTIVE') getAvailableTools(); + if (modalState === 'ACTIVE') { + getAvailableTools(); + dispatch(loadConnectors({ token })); + } }, [modalState]); - const handleMcpServerAdded = () => { - getUserTools(); - setMcpModalState('INACTIVE'); - }; - return ( <> !o && setModalState('INACTIVE')} title={t('settings.tools.selectToolSetup')} size="xl" + footer={ + + setModalState('INACTIVE')} + > + {t('settings.tools.browseConnectors')} + + + + } + cancelLabel={t('cancel')} + onCancel={() => setModalState('INACTIVE')} + submitLabel={t('settings.connectors.addCustom')} + onSubmit={() => { + setModalState('INACTIVE'); + navigate('/settings/connectors?filter=custom'); + }} + /> + } > -
    -
    -
    - {loading ? ( -
    - -
    - ) : ( -
    - {availableTools.map((tool, index) => ( - - - - ))} -
    - )} +
    + {loading ? ( +
    +
    -
    + ) : ( + <> + {services.length > 0 && ( +
    + +
    + {services.map((connector) => ( + + + + ))} +
    +
    + )} + {builtIn.length > 0 && ( +
    + +
    + {builtIn.map((tool) => ( + + + + ))} +
    +
    + )} + + )}
    - + {connectModals} ({ })); vi.mock('../api/services/userService', () => ({ default: mocks })); +// Tested on its own; it needs the store and the tool list. Here it only has +// to show up (or not). +vi.mock('../agents/ApiWriteAllowlist', () => ({ + default: ({ defaultOpen }: { defaultOpen?: boolean }) => ( +
    + ), +})); vi.mock('./ConfirmationModal', () => ({ default: ({ @@ -56,7 +63,7 @@ describe('AgentDetailsModal', () => { document.body.innerHTML = ''; }); - const render = async (agent: Partial) => { + const render = async (agent: Partial, openApiWrites?: boolean) => { await act(async () => { root.render( { mode="edit" modalState="ACTIVE" setModalState={() => undefined} + openApiWrites={openApiWrites} />, ); }); @@ -97,6 +105,40 @@ describe('AgentDetailsModal', () => { ); }); + // The allowlist acts on the owner's connected accounts; the server keeps + // it unchanged for anyone else, so only the owner sees it. + describe('API write allowlist', () => { + const allowlist = () => + document.querySelector('[data-testid="api-write-allowlist"]'); + + it('shows it to the owner of an agent with a key', async () => { + await render({ status: 'published', key: 'k-1', access: 'owner' }); + expect(allowlist()).not.toBeNull(); + }); + + it('starts it folded, or open when sent to allow changes', async () => { + await render({ status: 'published', key: 'k-1', access: 'owner' }); + expect(allowlist()?.getAttribute('data-open')).toBe('false'); + await render({ status: 'published', key: 'k-1', access: 'owner' }, true); + expect(allowlist()?.getAttribute('data-open')).toBe('true'); + }); + + it('hides it from an editor, even with a key', async () => { + await render({ + status: 'published', + key: 'k-1', + access: 'editor', + allowed_actions: ['view', 'edit', 'manage_access_details'], + }); + expect(allowlist()).toBeNull(); + }); + + it('hides it until the agent has a key', async () => { + await render({ status: 'published', access: 'owner' }); + expect(allowlist()).toBeNull(); + }); + }); + it('shows a refused public link in an alert', async () => { mocks.shareAgent.mockReturnValue( respond({ success: false, message: 'Not allowed' }, false), diff --git a/frontend/src/modals/AgentDetailsModal.tsx b/frontend/src/modals/AgentDetailsModal.tsx index 6a150a4a..e8bc6e57 100644 --- a/frontend/src/modals/AgentDetailsModal.tsx +++ b/frontend/src/modals/AgentDetailsModal.tsx @@ -4,7 +4,8 @@ import { useEffect, useState } from 'react'; import { useTranslation } from 'react-i18next'; import { useSelector } from 'react-redux'; -import { Agent } from '../agents/types'; +import ApiWriteAllowlist from '../agents/ApiWriteAllowlist'; +import { Agent, type AgentConfig } from '../agents/types'; import userService from '../api/services/userService'; import CopyButton from '../components/CopyButton'; import { Alert, AlertDescription } from '../components/ui/alert'; @@ -13,6 +14,7 @@ import { Modal } from '../components/ui/modal'; import { SectionHeader } from '../components/ui/section-header'; import { ActiveState } from '../models/misc'; import { selectToken } from '../preferences/preferenceSlice'; +import { isOwner } from '../utils/accessUtils'; import ConfirmationModal from './ConfirmationModal'; const baseURL = envVar('VITE_BASE_URL'); @@ -35,6 +37,12 @@ type AgentDetailsModalProps = { modalState: ActiveState; setModalState: (state: ActiveState) => void; onKeyRegenerated?: (key: string) => void; + /** The agent's saved config after its API-key write allowlist changed. */ + onConfigChange?: (config: AgentConfig) => void; + /** The agent's last saved config, which an allowlist change saves onto. */ + getSavedConfig?: () => AgentConfig | undefined; + /** Open with the API write allowlist unfolded (sent here to allow changes). */ + openApiWrites?: boolean; }; export default function AgentDetailsModal({ @@ -43,6 +51,9 @@ export default function AgentDetailsModal({ modalState, setModalState, onKeyRegenerated, + onConfigChange, + getSavedConfig, + openApiWrites = false, }: AgentDetailsModalProps) { const { t } = useTranslation(); const token = useSelector(selectToken); @@ -267,6 +278,16 @@ export default function AgentDetailsModal({ {t('modals.agentDetails.generate')} )} + {/* The allowlist acts on the owner's connected accounts, so + only the owner changes it (the server keeps it otherwise). */} + {apiKey && isOwner(agent) && ( + + )}
    diff --git a/frontend/src/modals/ConfirmationModal.tsx b/frontend/src/modals/ConfirmationModal.tsx index 4e986a03..1cfbdfe6 100644 --- a/frontend/src/modals/ConfirmationModal.tsx +++ b/frontend/src/modals/ConfirmationModal.tsx @@ -5,6 +5,7 @@ import { ActiveState } from '../models/misc'; export default function ConfirmationModal({ message, + description, modalState, setModalState, submitLabel, @@ -14,6 +15,8 @@ export default function ConfirmationModal({ variant = 'default', }: { message: string; + /** A muted line under the title that says what the action does. */ + description?: string; modalState: ActiveState; setModalState: (state: ActiveState) => void; submitLabel: string; @@ -41,6 +44,7 @@ export default function ConfirmationModal({ if (!open) setModalState('INACTIVE'); }} title={message} + description={description} footer={
    ); - case 'oauth': - return ( - - - handleInputChange('oauth_scopes', e.target.value) - } - placeholder="read, write" - disabled={oauthOwnerOnly} - /> - - ); default: return null; } @@ -617,11 +604,16 @@ export default function MCPServerModal({ } }} title={ - server + server?.id ? t('settings.tools.mcp.reconnectServer', { defaultValue: 'Reconnect Server', }) - : t('settings.tools.mcp.addServer') + : server?.preset + ? t('settings.connectors.wizard.connectTitle', { + name: server.displayName, + interpolation: { escapeValue: false }, + }) + : t('settings.tools.mcp.addServer') } description={ isShared @@ -662,6 +654,14 @@ export default function MCPServerModal({ } >
    + {!server?.preset && ( + + + )} {isShared && ( @@ -733,29 +733,60 @@ export default function MCPServerModal({ {renderAuthFields()} - + {formData.auth_type === 'oauth' && ( + + + handleInputChange('oauth_scopes', e.target.value) + } + placeholder="read, write" + disabled={oauthOwnerOnly} + /> + + )} + + { + const value = e.target.value; + if (value === '') { + handleInputChange('timeout', ''); + } else { + const numValue = parseInt(value); + if (!isNaN(numValue) && numValue >= 1) { + handleInputChange('timeout', numValue); + } + } + }} + placeholder="30" + min={1} + max={300} + /> + + + ) : null} + {testResult && ( diff --git a/frontend/src/modals/types/index.ts b/frontend/src/modals/types/index.ts index 2922c00c..d6e66b6e 100644 --- a/frontend/src/modals/types/index.ts +++ b/frontend/src/modals/types/index.ts @@ -24,4 +24,7 @@ export type AvailableToolType = { description: string; parameters: object; }[]; + /** `built_in`, `service` (a connector's tool) or `custom` (MCP / OpenAPI). */ + group?: 'built_in' | 'service' | 'custom'; + connector_key?: string | null; }; diff --git a/frontend/src/models/misc.ts b/frontend/src/models/misc.ts index d043280d..71fe2f8f 100644 --- a/frontend/src/models/misc.ts +++ b/frontend/src/models/misc.ts @@ -85,6 +85,8 @@ export type Doc = { // Access level when shared via a team: 'viewer' (read-only) or 'editor' // (full write). Null/absent for sources the caller owns. team_access?: 'viewer' | 'editor' | null; + // The connection a synced source comes from (Google Drive account, S3 keys). + connectionId?: string | null; // The caller's role and what it allows (sources API); gate UI with // `can(doc, action)` from utils/accessUtils. access?: 'owner' | 'editor' | 'viewer' | null; diff --git a/frontend/src/navigation/SectionIndexPage.tsx b/frontend/src/navigation/SectionIndexPage.tsx index c04d58c1..dacccc36 100644 --- a/frontend/src/navigation/SectionIndexPage.tsx +++ b/frontend/src/navigation/SectionIndexPage.tsx @@ -6,6 +6,7 @@ import { Link } from 'react-router-dom'; import { ListRow, ListRows } from '@/components/ui/list-row'; import { selectIsAdmin } from '@/preferences/preferenceSlice'; +import { selectConnectorsEnabled } from '../connectors/connectorsSlice'; import { getVisibleGroups, type Section } from './sections'; /** @@ -17,7 +18,11 @@ import { getVisibleGroups, type Section } from './sections'; export default function SectionIndexPage({ section }: { section: Section }) { const { t } = useTranslation(); const isAdmin = useSelector(selectIsAdmin); - const groups = getVisibleGroups(section, { isAdmin }); + const connectorsEnabled = useSelector(selectConnectorsEnabled); + const groups = getVisibleGroups(section, { + isAdmin, + features: { connectors: connectorsEnabled }, + }); return (
    diff --git a/frontend/src/navigation/SectionNav.tsx b/frontend/src/navigation/SectionNav.tsx index 1b2ef85a..98792f83 100644 --- a/frontend/src/navigation/SectionNav.tsx +++ b/frontend/src/navigation/SectionNav.tsx @@ -1,9 +1,12 @@ import { ArrowLeft } from 'lucide-react'; import { useTranslation } from 'react-i18next'; +import { useSelector } from 'react-redux'; import { Link } from 'react-router-dom'; import { Button } from '@/components/ui/button'; +import ConnectionHealthDot from '../connectors/ConnectionHealthDot'; +import { selectConnectorsEnabled } from '../connectors/connectorsSlice'; import { getVisibleGroups, type Section, type SectionItem } from './sections'; import { useSidebarLevel } from './SidebarLevelProvider'; @@ -35,7 +38,11 @@ export default function SectionNav({ }: SectionNavProps) { const { t } = useTranslation(); const { goToLevel } = useSidebarLevel(); - const groups = getVisibleGroups(section, { isAdmin }); + const connectorsEnabled = useSelector(selectConnectorsEnabled); + const groups = getVisibleGroups(section, { + isAdmin, + features: { connectors: connectorsEnabled }, + }); const sectionTitle = section.title ?? t(section.titleKey); const renderItem = (item: SectionItem) => { @@ -63,6 +70,7 @@ export default function SectionNav({ > {t(item.labelKey)} + {item.path === '/settings/connectors' && } ); diff --git a/frontend/src/navigation/SectionPills.tsx b/frontend/src/navigation/SectionPills.tsx index 4ecb590e..71567d46 100644 --- a/frontend/src/navigation/SectionPills.tsx +++ b/frontend/src/navigation/SectionPills.tsx @@ -6,6 +6,7 @@ import { Button } from '@/components/ui/button'; import { cn } from '@/lib/utils'; import { selectIsAdmin } from '@/preferences/preferenceSlice'; +import { selectConnectorsEnabled } from '../connectors/connectorsSlice'; import { getVisibleGroups } from './sections'; import { useSectionContext } from './useSectionContext'; @@ -24,11 +25,13 @@ export default function SectionPills({ className }: { className?: string }) { const { t } = useTranslation(); const { section, item } = useSectionContext(); const isAdmin = useSelector(selectIsAdmin); + const connectorsEnabled = useSelector(selectConnectorsEnabled); if (!section) return null; - const items = getVisibleGroups(section, { isAdmin }).flatMap( - (group) => group.items, - ); + const items = getVisibleGroups(section, { + isAdmin, + features: { connectors: connectorsEnabled }, + }).flatMap((group) => group.items); if (items.length < 2) return null; return ( diff --git a/frontend/src/navigation/sections.ts b/frontend/src/navigation/sections.ts index c74eff01..8b2de97d 100644 --- a/frontend/src/navigation/sections.ts +++ b/frontend/src/navigation/sections.ts @@ -11,6 +11,7 @@ import { LayoutDashboard, LayoutGrid, LayoutTemplate, + Plug, ScrollText, Settings2, ShieldCheck, @@ -43,6 +44,8 @@ export type SectionItem = { adminOnly?: boolean; /** Jumps to a different section rather than navigating within this one. */ leavesSection?: boolean; + /** Hidden when the backend reports this feature off (`/api/config`). */ + feature?: 'connectors'; }; /** Items sharing a heading in the nav. */ @@ -113,10 +116,17 @@ export const SETTINGS_SECTION: Section = { items: [ { key: 'sources', - path: '/settings/sources', + path: '/settings/knowledge', labelKey: 'settings.sources.label', icon: Database, }, + { + key: 'connectors', + path: '/settings/connectors', + labelKey: 'settings.connectors.label', + icon: Plug, + feature: 'connectors', + }, { key: 'tools', path: '/settings/tools', @@ -212,6 +222,13 @@ export const ADMIN_SECTION: Section = { labelKey: 'admin.tabs.quotas', icon: Gauge, }, + { + key: 'connectors', + path: '/admin/connectors', + labelKey: 'admin.tabs.connectors', + icon: Plug, + feature: 'connectors', + }, { key: 'audit', path: '/admin/audit', @@ -362,12 +379,22 @@ export function getSectionItems( /** Groups with admin-only entries removed, dropping any group left empty. */ export function getVisibleGroups( section: Section, - { isAdmin = true }: { isAdmin?: boolean } = {}, + { + isAdmin = true, + features = {}, + }: { + isAdmin?: boolean; + features?: Partial, boolean>>; + } = {}, ): SectionGroup[] { return section.groups .map((group) => ({ ...group, - items: group.items.filter((item) => !item.adminOnly || isAdmin), + items: group.items.filter( + (item) => + (!item.adminOnly || isAdmin) && + (!item.feature || features[item.feature] !== false), + ), })) .filter((group) => group.items.length > 0); } diff --git a/frontend/src/notifications/ConnectionHealthToast.test.tsx b/frontend/src/notifications/ConnectionHealthToast.test.tsx new file mode 100644 index 00000000..8e7cd934 --- /dev/null +++ b/frontend/src/notifications/ConnectionHealthToast.test.tsx @@ -0,0 +1,138 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { MemoryRouter } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: Record) => + opts && 'formatted' in opts ? `${key}:${opts.formatted}` : key, + }), +})); + +const server = vi.hoisted(() => ({ + catalog: [] as unknown[], + connections: [] as unknown[], +})); +vi.mock('../api/services/connectorsService', () => ({ + default: { + getCatalog: vi.fn(async () => ({ + success: true, + connectors: server.catalog, + })), + listConnections: vi.fn(async () => ({ + success: true, + connections: server.connections, + })), + }, +})); + +const launch = vi.fn(); +vi.mock('../connectors/useConnectorLauncher', () => ({ + default: () => ({ launch, modals: null }), +})); + +import connectorsReducer from '../connectors/connectorsSlice'; +import ConnectionHealthToast from './ConnectionHealthToast'; +import notificationsReducer, { sseEventReceived } from './notificationsSlice'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const DRIVE = { key: 'google_drive', name: 'Google Drive', auth_kind: 'oauth' }; + +describe('ConnectionHealthToast', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + localStorage.clear(); + launch.mockClear(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async ( + payload: Record, + connectionStatus = 'reconnect_needed', + ) => { + server.catalog = [DRIVE]; + server.connections = [{ id: 'conn-1', status: connectionStatus }]; + const store = configureStore({ + reducer: { + notifications: notificationsReducer, + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + preloadedState: { + connectors: { + enabled: true, + loading: false, + loaded: true, + failed: false, + catalog: [DRIVE], + connections: [{ id: 'conn-1', status: connectionStatus }], + }, + preference: { token: null }, + }, + } as Parameters[0]); + store.dispatch( + sseEventReceived({ + id: 'e1', + type: 'connection.reconnect_needed', + ts: new Date().toISOString(), + scope: { kind: 'connection', id: 'conn-1' }, + payload: { + connection_id: 'conn-1', + connector_key: 'google_drive', + name: 'Google Drive', + ...payload, + }, + }), + ); + await act(async () => { + root.render( + + + + + , + ); + }); + }; + + const title = () => + container.querySelector('[data-slot="toast-title"]')?.textContent; + + it('says which sources are paused', async () => { + await render({ source_count: 3, tool_count: 0 }); + expect(title()).toBe('settings.connectors.health.reconnectSources:3'); + }); + + it('says agents cannot use a tool-only connection', async () => { + await render({ source_count: 0, tool_count: 2 }); + expect(title()).toBe('settings.connectors.health.reconnectTools:0'); + }); + + it('reconnects in place and stays until it works', async () => { + await render({ source_count: 1, tool_count: 1 }); + const link = container.querySelector('a')!; + await act(async () => link.click()); + expect(launch).toHaveBeenCalledWith(DRIVE, { + mode: 'reconnect', + connectionId: 'conn-1', + }); + expect(title()).toBe('settings.connectors.health.reconnectBoth:1'); + }); + + it('closes itself once the connection works again', async () => { + await render({ source_count: 1 }, 'connected'); + expect(container.querySelector('[data-slot="toast"]')).toBeNull(); + }); +}); diff --git a/frontend/src/notifications/ConnectionHealthToast.tsx b/frontend/src/notifications/ConnectionHealthToast.tsx new file mode 100644 index 00000000..2fa05ff7 --- /dev/null +++ b/frontend/src/notifications/ConnectionHealthToast.tsx @@ -0,0 +1,159 @@ +import { X } from 'lucide-react'; +import { useCallback, useEffect, useMemo } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; +import { Link } from 'react-router-dom'; + +import { Button } from '../components/ui/button'; +import { + Toast, + ToastActions, + ToastFooter, + ToastHeader, + ToastTitle, +} from '../components/ui/toast'; +import { + loadConnectors, + selectConnections, + selectConnectorCatalog, + selectConnectorsLoaded, +} from '../connectors/connectorsSlice'; +import { reconnectsInPlace } from '../connectors/launchRules'; +import useConnectorLauncher from '../connectors/useConnectorLauncher'; +import { formatCount } from '../utils/dateTimeUtils'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; +import { + dismissShareNotification, + selectDismissedShareNotifications, + selectRecentEvents, + type SSEEvent, +} from './notificationsSlice'; + +// Backlog replay re-delivers up to a day of events; an old one is noise. +const MAX_AGE_MS = 24 * 60 * 60 * 1000; +const MAX_VISIBLE = 2; + +/** + * "Reconnect Google Drive: 3 sources paused": a connection whose sign-in + * stopped working (``connection.reconnect_needed``). Reconnect signs in + * again right here; the toast closes itself once the connection works, or + * when dismissed. Shares the team notifications' persisted dismissals, so a + * reload does not pop it again. + */ +export default function ConnectionHealthToast() { + const dispatch = useDispatch(); + const { t } = useTranslation(); + const token = useSelector(selectToken); + const events = useSelector(selectRecentEvents); + const dismissed = useSelector(selectDismissedShareNotifications); + const dismissedSet = useMemo(() => new Set(dismissed), [dismissed]); + const catalog = useSelector(selectConnectorCatalog); + const connections = useSelector(selectConnections); + const connectionsLoaded = useSelector(selectConnectorsLoaded); + const { launch, modals } = useConnectorLauncher(); + + const onDismiss = useCallback( + (id: string) => dispatch(dismissShareNotification(id)), + [dispatch], + ); + + const now = Date.now(); + const visible: SSEEvent[] = []; + const seenConnections = new Set(); + for (const event of events) { + if (event.type !== 'connection.reconnect_needed') continue; + if (!event.id || dismissedSet.has(event.id)) continue; + if (event.ts) { + const age = now - Date.parse(event.ts); + if (Number.isFinite(age) && age > MAX_AGE_MS) continue; + } + const connectionId = String(event.scope?.id ?? event.id); + if (seenConnections.has(connectionId)) continue; + // Reconnected since (here or anywhere): nothing to say any more. + const current = connections.find((c) => c.id === connectionId); + if (connectionsLoaded && current?.status === 'connected') continue; + seenConnections.add(connectionId); + visible.push(event); + if (visible.length >= MAX_VISIBLE) break; + } + + // Refresh the badges on the Connectors, Sources and Tools pages. + const newest = visible[0]?.id; + useEffect(() => { + if (newest) dispatch(loadConnectors({ token })); + }, [newest, dispatch, token]); + + if (visible.length === 0) return <>{modals}; + + const title = (payload: Record) => { + const name = String(payload.name ?? ''); + const sources = Number(payload.source_count ?? 0); + const tools = Number(payload.tool_count ?? 0); + const values = { + name, + count: sources, + formatted: formatCount(sources), + interpolation: { escapeValue: false }, + }; + if (sources && tools) + return t('settings.connectors.health.reconnectBoth', values); + if (sources) + return t('settings.connectors.health.reconnectSources', values); + if (tools) return t('settings.connectors.health.reconnectTools', values); + return t('settings.connectors.health.reconnect', values); + }; + + const reconnect = (payload: Record) => { + const connectionId = String(payload.connection_id ?? ''); + const connector = catalog.find((c) => c.key === payload.connector_key); + const inPlace = !!connector && reconnectsInPlace(connector); + if (connector && connectionId && inPlace) { + launch(connector, { mode: 'reconnect', connectionId }); + return true; + } + return false; + }; + + return ( + <> + {visible.map((event) => { + const payload = (event.payload ?? {}) as Record; + const key = String(payload.connector_key ?? ''); + return ( + + + {title(payload)} + + + + + + {/* Sign in again right here; custom MCP servers reconnect from the + connector's drawer. The toast stays until it works. */} + + + + ); + })} + {modals} + + ); +} diff --git a/frontend/src/settings/Connectors.test.tsx b/frontend/src/settings/Connectors.test.tsx new file mode 100644 index 00000000..b8623122 --- /dev/null +++ b/frontend/src/settings/Connectors.test.tsx @@ -0,0 +1,402 @@ +import { configureStore } from '@reduxjs/toolkit'; +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; +import { Provider } from 'react-redux'; +import { Link, MemoryRouter } from 'react-router-dom'; + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ + t: (key: string, opts?: { defaultValue?: string; count?: number }) => + opts?.count !== undefined ? `${key}:${opts.count}` : key, + }), +})); + +const service = vi.hoisted(() => ({ + getCatalog: vi.fn(), + listConnections: vi.fn(), + getConnection: vi.fn(), + disconnect: vi.fn(), +})); +vi.mock('../api/services/connectorsService', () => ({ default: service })); + +// The wizard is tested on its own; here only what it is opened with matters. +const launch = vi.hoisted(() => vi.fn()); +vi.mock('../connectors/useConnectorLauncher', () => ({ + default: () => ({ launch, modals: null }), +})); + +import connectorsReducer from '../connectors/connectorsSlice'; +import type { ConnectorDefinition } from '../connectors/types'; +import Connectors from './Connectors'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const definition = ( + overrides: Partial, +): ConnectorDefinition => ({ + key: 'telegram', + name: 'Telegram', + description: 'Send messages.', + icon: 'tool_telegram', + category: 'messaging', + auth_kind: 'api_key', + capabilities: ['write'], + credential_fields: [], + setup_fields: [], + sync_ingestor: null, + default_sync_frequency: 'weekly', + tool_templates: ['telegram'], + setup: { tools: 'auto', sync: 'off' }, + mcp_url: null, + publisher: 'built_in', + docs_url: null, + oauth_scopes: [], + available: true, + disabled: false, + needs_setup: false, + missing_settings: [], + connected_count: 0, + connection_count: 0, + status: null, + state: 'available', + credential_policy: 'choose', + ...overrides, +}); + +const CATALOG = [ + definition({}), + definition({ + key: 'google_drive', + name: 'Google Drive', + category: 'files', + capabilities: ['sync'], + state: 'connected', + connected_count: 2, + connection_count: 2, + status: 'connected', + }), + definition({ + key: 'confluence', + name: 'Confluence', + category: 'knowledge', + capabilities: ['sync'], + state: 'reconnect', + connection_count: 1, + status: 'reconnect_needed', + }), + definition({ + key: 'share_point', + name: 'SharePoint', + category: 'files', + state: 'needs_setup', + available: false, + needs_setup: true, + }), + definition({ + key: 'brave', + name: 'Brave Search', + category: 'search', + state: 'disabled', + available: false, + disabled: true, + }), + definition({ + key: 'custom_mcp', + name: 'MCP server', + category: 'custom', + publisher: 'custom', + auth_kind: 'mcp', + state: 'custom', + }), +]; + +describe('Connectors page', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: CATALOG, + }); + service.listConnections.mockResolvedValue({ + success: true, + connections: [], + }); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + }); + + const render = async (path = '/settings/connectors') => { + const store = configureStore({ + reducer: { + connectors: connectorsReducer, + preference: (state = { token: null }) => state, + }, + }); + await act(async () => { + root.render( + + + + + + , + ); + }); + }; + + const card = (key: string) => + container.querySelector( + `[data-testid="connector-card-${key}"]`, + ); + + it('renders every card state', async () => { + await render(); + expect(card('telegram')!.textContent).toContain( + 'settings.connectors.status.connect', + ); + expect(card('google_drive')!.textContent).toContain( + 'settings.connectors.status.connectedCount:2', + ); + expect(card('confluence')!.textContent).toContain( + 'settings.connectors.status.reconnect', + ); + expect(card('share_point')!.textContent).toContain( + 'settings.connectors.status.needsAdminSetup', + ); + expect(card('brave')!.textContent).toContain( + 'settings.connectors.status.disabledByAdmin', + ); + expect(card('brave')!.disabled).toBe(true); + // No publisher jargon ("Preset", "Built in") on cards. + expect(card('custom_mcp')!.textContent).not.toContain( + 'settings.connectors.publisher', + ); + }); + + it('sorts connections that need attention first', async () => { + await render(); + const keys = Array.from( + container.querySelectorAll('[data-testid^="connector-card-"]'), + ).map((el) => el.getAttribute('data-testid')); + expect(keys[0]).toBe('connector-card-confluence'); + expect(keys[1]).toBe('connector-card-google_drive'); + expect(keys[keys.length - 1]).toBe('connector-card-brave'); + }); + + it('shows capability chips', async () => { + await render(); + expect(card('google_drive')!.textContent).toContain( + 'settings.connectors.capabilityPlain.sync', + ); + expect(card('telegram')!.textContent).toContain( + 'settings.connectors.capabilityPlain.write', + ); + }); + + it('filters to connected services', async () => { + await render('/settings/connectors?filter=connected'); + expect(card('google_drive')).not.toBeNull(); + expect(card('confluence')).not.toBeNull(); + expect(card('telegram')).toBeNull(); + }); + + it('shows the empty state when nothing is connected', async () => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [definition({})], + }); + await render('/settings/connectors?filter=connected'); + expect(container.textContent).toContain('settings.connectors.empty'); + }); + + it('says when the composer narrowed the list, and clears it', async () => { + await render('/settings/connectors?capability=sync'); + expect(container.textContent).toContain( + 'settings.connectors.capabilityFilter.sync', + ); + expect(card('telegram')).toBeNull(); + const showAll = Array.from(container.querySelectorAll('button')).find( + (button) => + button.textContent === 'settings.connectors.capabilityFilter.showAll', + )!; + await act(async () => showAll.click()); + expect(container.textContent).not.toContain( + 'settings.connectors.capabilityFilter.sync', + ); + expect(card('telegram')).not.toBeNull(); + }); + + it('only offers categories that have connectors', async () => { + await render(); + const pills = Array.from( + container.querySelectorAll('[data-slot="toggle-group-item"]'), + ).map((item) => item.textContent); + expect(pills).toContain('settings.connectors.categories.all'); + expect(pills).toContain('settings.connectors.categories.files'); + expect(pills).not.toContain('settings.connectors.categories.database'); + }); + + it('offers no category that the capability filter would leave empty', async () => { + await render('/settings/connectors?capability=sync'); + const pills = Array.from( + container.querySelectorAll('[data-slot="toggle-group-item"]'), + ).map((item) => item.textContent); + expect(pills).toContain('settings.connectors.categories.files'); + // Telegram (messaging) cannot sync. + expect(pills).not.toContain('settings.connectors.categories.messaging'); + }); + + describe('opened for Knowledge', () => { + const S3 = definition({ + key: 's3', + name: 'Amazon S3', + category: 'files', + capabilities: ['sync'], + sync_ingestor: 's3', + tool_templates: [], + setup: { tools: 'off', sync: 'ask' }, + }); + + beforeEach(() => { + launch.mockClear(); + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [...CATALOG, S3], + }); + }); + + it('connects with syncing switched on from a sync-only list', async () => { + await render('/settings/connectors?capability=sync'); + await act(async () => card('s3')!.click()); + expect(launch).toHaveBeenCalledWith( + expect.objectContaining({ key: 's3' }), + { purpose: 'knowledge' }, + ); + }); + + it('leaves the choice off on a plain visit', async () => { + await render(); + await act(async () => card('s3')!.click()); + expect(launch).toHaveBeenCalledWith( + expect.objectContaining({ key: 's3' }), + {}, + ); + }); + + it("carries it through the connector's page", async () => { + await render('/settings/connectors?capability=sync'); + await act(async () => card('google_drive')!.click()); + // No account of the user's own yet: the page offers Connect. + const connect = Array.from( + document.body.querySelectorAll( + '[role="dialog"] button', + ), + ).find( + (b) => b.textContent?.trim() === 'settings.connectors.status.connect', + )!; + await act(async () => connect.click()); + expect(launch).toHaveBeenCalledWith( + expect.objectContaining({ key: 'google_drive' }), + { purpose: 'knowledge' }, + ); + }); + }); + + const ATLASSIAN = definition({ + key: 'mcp:atlassian', + name: 'Jira & Confluence', + description: 'Search and update Jira issues.', + category: 'dev', + auth_kind: 'mcp_oauth', + capabilities: ['read', 'write'], + publisher: 'preset', + part_of: 'confluence', + }); + + it('shows one Confluence card that also does what its MCP part does', async () => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [...CATALOG, ATLASSIAN], + }); + await render(); + expect(card('mcp:atlassian')).toBeNull(); + const confluence = card('confluence')!; + expect(confluence.textContent).toContain( + 'settings.connectors.capabilityPlain.sync', + ); + expect(confluence.textContent).toContain( + 'settings.connectors.capabilityPlain.write', + ); + }); + + it("finds the merged card by its part's name", async () => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [...CATALOG, ATLASSIAN], + }); + await render(); + const input = container.querySelector( + '#connector-search-input', + )!; + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, 'jira'); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + expect(card('confluence')).not.toBeNull(); + }); + + it('shows the part on its own when its parent is not listed', async () => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [...CATALOG.filter((c) => c.key !== 'confluence'), ATLASSIAN], + }); + await render(); + expect(card('mcp:atlassian')).not.toBeNull(); + }); + + it("opens the merged card's page with a section for its part", async () => { + service.getCatalog.mockResolvedValue({ + success: true, + connectors: [...CATALOG, ATLASSIAN], + }); + await render(); + await act(async () => card('confluence')!.click()); + expect(document.body.textContent).toContain('Jira & Confluence'); + expect(document.body.textContent).toContain( + 'settings.connectors.descriptions.mcp_atlassian', + ); + }); + + it('opens the page a ?connector= link names while already on the list', async () => { + await render(); + expect(document.body.querySelector('[role="dialog"]')).toBeNull(); + await act(async () => + container + .querySelector('[data-testid="reconnect-link"]')! + .click(), + ); + const dialog = document.body.querySelector('[role="dialog"]'); + expect(dialog?.textContent).toContain('Google Drive'); + }); + + it('shows a retry when the catalog fails to load', async () => { + service.getCatalog.mockResolvedValue({ success: false }); + await render(); + expect(container.textContent).toContain('settings.connectors.loadFailed'); + }); +}); diff --git a/frontend/src/settings/Connectors.tsx b/frontend/src/settings/Connectors.tsx new file mode 100644 index 00000000..dc075613 --- /dev/null +++ b/frontend/src/settings/Connectors.tsx @@ -0,0 +1,322 @@ +import { ChevronDown, Plus } from 'lucide-react'; +import { useEffect, useMemo, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useDispatch, useSelector } from 'react-redux'; +import { useSearchParams } from 'react-router-dom'; + +import PageToolbar from '../components/PageToolbar'; +import SearchInput from '../components/SearchInput'; +import SkeletonLoader from '../components/SkeletonLoader'; +import { Button } from '../components/ui/button'; +import { + DropdownMenu, + DropdownMenuContent, + DropdownMenuItem, + DropdownMenuTrigger, +} from '../components/ui/dropdown-menu'; +import { EmptyState } from '../components/ui/empty-state'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '../components/ui/select'; +import { ToggleGroup, ToggleGroupItem } from '../components/ui/toggle-group'; +import ConnectionDrawer from '../connectors/ConnectionDrawer'; +import ConnectorCard from '../connectors/ConnectorCard'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { + byState, + catalogCards, + partsOf as partsOfCatalog, +} from '../connectors/catalogCards'; +import { + loadConnectors, + selectConnectorCatalog, + selectConnectorsFailed, + selectConnectorsLoaded, +} from '../connectors/connectorsSlice'; +import { connectorDescription, connectorName } from '../connectors/i18n'; +import type { ConnectorDefinition } from '../connectors/types'; +import useConnectorLauncher, { + type LaunchOptions, +} from '../connectors/useConnectorLauncher'; +import { selectToken } from '../preferences/preferenceSlice'; +import type { AppDispatch } from '../store'; + +const FILTERS = [ + 'all', + 'connected', + 'files', + 'knowledge', + 'projects', + 'dev', + 'business', + 'messaging', + 'database', + 'search', + 'custom', +] as const; +type Filter = (typeof FILTERS)[number]; + +const isConnected = (connector: ConnectorDefinition) => + connector.connection_count > 0; + +export default function Connectors() { + const { t } = useTranslation(); + const dispatch = useDispatch(); + const token = useSelector(selectToken); + const catalog = useSelector(selectConnectorCatalog); + const loaded = useSelector(selectConnectorsLoaded); + const failed = useSelector(selectConnectorsFailed); + const [searchParams, setSearchParams] = useSearchParams(); + const [search, setSearch] = useState(''); + const initialFilter = searchParams.get('filter') as Filter | null; + const [filter, setFilter] = useState( + initialFilter && FILTERS.includes(initialFilter) ? initialFilter : 'all', + ); + const connectorParam = searchParams.get('connector'); + const [openKey, setOpenKey] = useState(connectorParam); + // A `?connector=` link followed while this page is already open (the + // global Reconnect toast) changes only the search: open what it names. + const [seenConnectorParam, setSeenConnectorParam] = useState(connectorParam); + if (connectorParam !== seenConnectorParam) { + setSeenConnectorParam(connectorParam); + if (connectorParam) setOpenKey(connectorParam); + } + const { launch, modals } = useConnectorLauncher(); + + useEffect(() => { + dispatch(loadConnectors({ token })); + }, [dispatch, token]); + + const custom = catalog.filter((c) => c.publisher === 'custom'); + // A `?connector=` link to a part (a Reconnect that can't happen in place) + // opens its parent's page. + const openTarget = catalog.find((c) => c.key === openKey); + const openConnector = + (openTarget?.part_of && + catalog.find((c) => c.key === openTarget.part_of)) || + openTarget || + null; + + const partsOf = (key: string) => partsOfCatalog(catalog, key); + + // "Connect more" in the composer opens the connectors that can do what the + // picker is for: sync content, or give tools. + const capability = searchParams.get('capability'); + const clearCapability = () => { + searchParams.delete('capability'); + setSearchParams(searchParams, { replace: true }); + }; + // Listed for syncing (Knowledge's Connect a service, Add knowledge's Browse + // all connectors, the sources picker's Connect more): a connect starts with + // Sync into Knowledge on. + const withPurpose = (options: LaunchOptions = {}): LaunchOptions => + capability === 'sync' ? { ...options, purpose: 'knowledge' } : options; + + // Only categories that have something in them once the composer's + // capability filter applies (hidden connectors can empty one too), so no + // pill leads to an empty page. + const cards = catalogCards(catalog); + const withCapability = cards.filter((connector) => + capability === 'sync' + ? connector.capabilities.includes('sync') + : capability === 'tools' + ? connector.capabilities.some((c) => c !== 'sync') + : true, + ); + const filters = FILTERS.filter( + (key) => + key === 'all' || + key === filter || + (key === 'connected' + ? withCapability.some(isConnected) + : withCapability.some((connector) => connector.category === key)), + ); + + const visible = useMemo(() => { + const query = search.trim().toLowerCase(); + return withCapability + .filter((connector) => { + if (filter === 'connected') return isConnected(connector); + if (filter !== 'all') return connector.category === filter; + return true; + }) + .filter( + (connector) => + !query || + [connector, ...partsOf(connector.key)].some( + (c) => + connectorName(t, c).toLowerCase().includes(query) || + connectorDescription(t, c).toLowerCase().includes(query), + ), + ) + .sort(byState); + }, [withCapability, filter, search, t]); + + const open = (connector: ConnectorDefinition) => { + const hasParts = partsOf(connector.key).length > 0; + if ( + !hasParts && + (connector.state === 'available' || connector.state === 'custom') + ) { + launch(connector, withPurpose()); + return; + } + setOpenKey(connector.key); + }; + + const closeDrawer = () => { + setOpenKey(null); + if (searchParams.has('connector')) { + searchParams.delete('connector'); + setSearchParams(searchParams, { replace: true }); + } + }; + + return ( +
    + setSearch(e.target.value)} + /> + } + action={ + + + + + + {custom.map((connector) => ( + launch(connector)} + > + + {connectorName(t, connector)} + + ))} + + + } + > +
    + {/* Nine pills wrap to four lines on a phone: a Select there. */} + value && setFilter(value as Filter)} + aria-label={t('settings.connectors.categoriesLabel')} + className="hidden sm:flex" + > + {filters.map((key) => ( + + {t(`settings.connectors.categories.${key}`)} + + ))} + +
    + +
    + {(capability === 'sync' || capability === 'tools') && ( +

    + {t(`settings.connectors.capabilityFilter.${capability}`)}{' '} + +

    + )} +
    +     + + {!loaded && !failed ? ( +
    + +
    + ) : failed && catalog.length === 0 ? ( + dispatch(loadConnectors({ token }))} + > + {t('retry')} + + } + /> + ) : visible.length === 0 ? ( + filter === 'connected' && !search ? ( + + ) : ( + + ) + ) : ( +
    + {visible.map((connector) => ( + + ))} +
    + )} + + { + closeDrawer(); + launch(connector, withPurpose(options)); + }} + /> + {modals} +
    + ); +} diff --git a/frontend/src/settings/Sources.test.tsx b/frontend/src/settings/Sources.test.tsx index fcef47c7..b2249849 100644 --- a/frontend/src/settings/Sources.test.tsx +++ b/frontend/src/settings/Sources.test.tsx @@ -1,23 +1,28 @@ import { act, useState } from 'react'; import { createRoot, type Root } from 'react-dom/client'; +import { MemoryRouter, useLocation } from 'react-router-dom'; -const { dispatch, service, view } = vi.hoisted(() => ({ - // The heavy children: each view reports the canEdit it was given. - view: - (testId: string) => - ({ canEdit }: { canEdit?: boolean }) => ( -
    - ), - dispatch: vi.fn(), - service: { - getConfig: vi.fn(), - manageSync: vi.fn(), - syncSource: vi.fn(), - syncConnector: vi.fn(), - reingestSource: vi.fn(), - getDirectoryStructure: vi.fn(), - }, -})); +const { dispatch, service, view, connectors, uploadProps, reconnect } = + vi.hoisted(() => ({ + reconnect: vi.fn(), + uploadProps: vi.fn(), + connectors: { connections: [] as Record[] }, + // The heavy children: each view reports the canEdit it was given. + view: + (testId: string) => + ({ canEdit }: { canEdit?: boolean }) => ( +
    + ), + dispatch: vi.fn(), + service: { + getConfig: vi.fn(), + manageSync: vi.fn(), + syncSource: vi.fn(), + syncConnector: vi.fn(), + reingestSource: vi.fn(), + getDirectoryStructure: vi.fn(), + }, + })); vi.mock('react-i18next', () => ({ useTranslation: () => ({ t: (key: string) => key }), @@ -30,6 +35,7 @@ vi.mock('react-redux', () => ({ preference: { token: null }, upload: { tasks: [] }, graphBuild: { builds: {} }, + connectors: { connections: connectors.connections, loaded: true }, }), })); @@ -60,9 +66,22 @@ vi.mock('../components/graph/GraphSourceView', () => ({ vi.mock('./SourceConfigModal', () => ({ default: () => null })); vi.mock('./TestRetrievalModal', () => ({ default: () => null })); vi.mock('./ConvertToWikiModal', () => ({ default: () => null })); +vi.mock('./WikiSettingsModal', () => ({ + default: ({ document }: { document: { name: string } }) => ( +
    {document.name}
    + ), +})); vi.mock('./EnableGraphRAGModal', () => ({ default: () => null })); vi.mock('../teams/ShareToTeamModal', () => ({ default: () => null })); -vi.mock('../upload/Upload', () => ({ default: () => null })); +vi.mock('../connectors/SignInAgainNotice', () => ({ + useSignInAgain: () => ({ reconnect, modals: null }), +})); +vi.mock('../upload/Upload', () => ({ + default: (props: unknown) => { + uploadProps(props); + return null; + }, +})); import type { Doc } from '../models/misc'; import Sources from './Sources'; @@ -100,13 +119,21 @@ describe('Sources access', () => { document.body.innerHTML = ''; }); + function Where() { + const location = useLocation(); + return
    {location.pathname + location.search}
    ; + } + const render = async (document: Doc) => { await act(async () => { root.render( - , + + + + , ); }); }; @@ -161,6 +188,97 @@ describe('Sources access', () => { ]); }); + // The connection is managed on the Connectors page; a synced source's + // menu keeps only what acts on the source itself. + it('a synced source has no Manage connection item', async () => { + connectors.connections = [ + { + id: 'conn-1', + connector_key: 'google_drive', + name: 'Google Drive', + icon: 'drive', + status: 'connected', + account_label: 'alex@example.com', + }, + ]; + await render( + doc({ + access: 'owner', + allowed_actions: [...OWNER, 'use', 'view_config'], + connectionId: 'conn-1', + } as Partial), + ); + const items = await menuItems(); + connectors.connections = []; + expect(items).not.toContain('settings.connectors.manageConnection'); + expect(items).toContain('settings.sources.editConfig'); + expect(items).toContain('convTile.delete'); + }); + + // A source whose connection needs signing in again says so on its tile + // and signs in again from there, without opening the source. + it('reconnects a paused synced source from its tile', async () => { + connectors.connections = [ + { + id: 'conn-1', + connector_key: 'google_drive', + name: 'Google Drive', + icon: 'drive', + status: 'reconnect_needed', + account_label: 'alex@example.com', + }, + ]; + reconnect.mockClear(); + await render(doc({ connectionId: 'conn-1' } as Partial)); + connectors.connections = []; + const button = Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === 'settings.connectors.status.reconnect', + )!; + expect(button).toBeDefined(); + await act(async () => button.click()); + expect(reconnect).toHaveBeenCalledWith( + expect.objectContaining({ id: 'conn-1', connector_key: 'google_drive' }), + ); + // Only the reconnect: the source view stays closed. + expect(container.querySelector('[data-testid="chunks"]')).toBeNull(); + }); + + it('offers no Reconnect on a synced source that is running', async () => { + connectors.connections = [ + { + id: 'conn-1', + connector_key: 'google_drive', + name: 'Google Drive', + status: 'connected', + }, + ]; + await render(doc({ connectionId: 'conn-1' } as Partial)); + connectors.connections = []; + expect( + Array.from(container.querySelectorAll('button')).some( + (b) => b.textContent === 'settings.connectors.status.reconnect', + ), + ).toBe(false); + }); + + // Leaving Knowledge loses nothing, so its Add knowledge may browse the + // whole Connectors page; the other openers keep the list in the dialog. + it('lets Add knowledge browse the syncing connectors from Knowledge', async () => { + await render(doc()); + uploadProps.mockClear(); + const add = Array.from(container.querySelectorAll('button')).find( + (b) => b.textContent === 'settings.sources.addSource', + )!; + await act(async () => add.click()); + const props = uploadProps.mock.calls.at(-1)![0] as { + onBrowseConnectors?: () => void; + }; + await act(async () => props.onBrowseConnectors!()); + expect(container.querySelector('[data-testid="where"]')?.textContent).toBe( + '/settings/connectors?capability=sync', + ); + }); + it('a source with no access fields is the caller’s own', async () => { await render(doc()); const items = await menuItems(); @@ -227,6 +345,41 @@ describe('Sources access', () => { ]); }); + it('a wiki owner gets Wiki settings; editors and viewers do not', async () => { + const wiki = { type: 'wiki', config: { kind: 'wiki' } }; + await render( + doc({ ...wiki, access: 'owner', allowed_actions: [...OWNER, 'use'] }), + ); + expect(await menuItems()).toContain( + 'settings.sources.wiki.settings.action', + ); + await clickItem('settings.sources.wiki.settings.action'); + expect( + document.querySelector('[data-testid="wiki-settings"]')?.textContent, + ).toBe('Contracts'); + + for (const allowed of [EDITOR, VIEWER]) { + await act(async () => root.unmount()); + root = createRoot(container); + document.body + .querySelectorAll('[role="menu"]') + .forEach((m) => m.remove()); + await render( + doc({ ...wiki, access: 'editor', allowed_actions: allowed }), + ); + expect(await menuItems()).not.toContain( + 'settings.sources.wiki.settings.action', + ); + } + }); + + it('a classic source has no Wiki settings', async () => { + await render(doc({ access: 'owner', allowed_actions: [...OWNER, 'use'] })); + expect(await menuItems()).not.toContain( + 'settings.sources.wiki.settings.action', + ); + }); + it('sync and reingest are editor actions', async () => { const synced = { syncFrequency: 'daily', ingestStatus: 'failed' as const }; await render(doc({ ...synced, access: 'editor', allowed_actions: EDITOR })); diff --git a/frontend/src/settings/Sources.tsx b/frontend/src/settings/Sources.tsx index cdbd0773..c3b836c6 100644 --- a/frontend/src/settings/Sources.tsx +++ b/frontend/src/settings/Sources.tsx @@ -14,6 +14,7 @@ import { import { useCallback, useEffect, useState } from 'react'; import { useTranslation } from 'react-i18next'; import { useDispatch, useSelector } from 'react-redux'; +import { useNavigate } from 'react-router-dom'; import userService from '../api/services/userService'; import modelService from '../api/services/modelService'; @@ -28,6 +29,11 @@ import { Card, CardFooter, CardTitle } from '../components/ui/card'; import { ActionMenu, type MenuOption } from '../components/ui/dropdown-menu'; import { EmptyState } from '../components/ui/empty-state'; import { Pagination } from '../components/ui/pagination'; +import { + Tooltip, + TooltipContent, + TooltipTrigger, +} from '../components/ui/tooltip'; import { useDebouncedValue, useLoaderState } from '../hooks'; import ConfirmationModal from '../modals/ConfirmationModal'; import { ActiveState, Doc, DocumentsProps } from '../models/misc'; @@ -51,6 +57,14 @@ import { can } from '../utils/accessUtils'; import { formatDate } from '../utils/dateTimeUtils'; import FileTree from '../components/FileTree'; import ConnectorTree from '../components/ConnectorTree'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { useSignInAgain } from '../connectors/SignInAgainNotice'; +import { + loadConnectors, + selectConnections, + selectConnectorsLoaded, +} from '../connectors/connectorsSlice'; +import type { AppDispatch } from '../store'; import Chunks from '../components/Chunks'; import WikiViewer from '../components/WikiViewer'; import GraphSourceView from '../components/graph/GraphSourceView'; @@ -59,6 +73,7 @@ import EnableGraphRAGModal from './EnableGraphRAGModal'; import { clearGraphBuild, selectGraphBuilds } from './graphBuildSlice'; import SourceConfigModal from './SourceConfigModal'; import TestRetrievalModal from './TestRetrievalModal'; +import WikiSettingsModal from './WikiSettingsModal'; const formatTokens = (tokens: number): string => { const roundToTwoDecimals = (num: number): string => { @@ -81,9 +96,18 @@ export default function Sources({ handleDeleteDocument, }: DocumentsProps) { const { t } = useTranslation(); - const dispatch = useDispatch(); + const navigate = useNavigate(); + const dispatch = useDispatch(); const token = useSelector(selectToken); const uploadTasks = useSelector(selectUploadTasks); + const connections = useSelector(selectConnections); + // Signing in again reloads the connections, which lifts the pause. + const { reconnect, modals: signInModals } = useSignInAgain(); + const connectorsLoaded = useSelector(selectConnectorsLoaded); + + useEffect(() => { + if (!connectorsLoaded) dispatch(loadConnectors({ token })); + }, [connectorsLoaded, dispatch, token]); const [searchTerm, setSearchTerm] = useState(''); const debouncedSearchTerm = useDebouncedValue(searchTerm, 500); @@ -108,6 +132,8 @@ export default function Sources({ ]; const [documentToView, setDocumentToView] = useState(); const [documentToShare, setDocumentToShare] = useState(null); + const [documentForWikiSettings, setDocumentForWikiSettings] = + useState(null); const [documentToConfigure, setDocumentToConfigure] = useState( null, ); @@ -233,26 +259,6 @@ export default function Sources({ }); }; - const getConnectorProvider = async (doc: Doc): Promise => { - if (doc.provider) { - return doc.provider; - } - if (!doc.id) { - return null; - } - try { - const directoryResponse = await userService.getDirectoryStructure( - doc.id, - token, - ); - const directoryData = await directoryResponse.json(); - return directoryData?.provider ?? null; - } catch (error) { - console.error('Error fetching connector provider:', error); - return null; - } - }; - const handleSyncNow = async (doc: Doc) => { if (!doc.id) { return; @@ -261,13 +267,8 @@ export default function Sources({ try { let response: Response; if (doc.type?.startsWith('connector')) { - const provider = await getConnectorProvider(doc); - if (!provider) { - console.error('Sync now failed: provider not found'); - showActionError(syncFailed); - return; - } - response = await userService.syncConnector(doc.id, provider, token); + // The server finds the connector from the source itself. + response = await userService.syncConnector(doc.id, token); } else { response = await userService.syncSource({ source_id: doc.id }, token); } @@ -425,6 +426,18 @@ export default function Sources({ }); } + // A wiki's own settings are the owner's (manage_settings). + if (document.id && isWiki && can(document, 'manage_settings')) { + actions.push({ + icon: SlidersHorizontal, + label: t('settings.sources.wiki.settings.action'), + onClick: () => { + setDocumentForWikiSettings(document); + }, + variant: 'default', + }); + } + if (document.id) { actions.push({ icon: Search, @@ -647,11 +660,49 @@ export default function Sources({
    ) : !currentDocuments?.length ? ( - + searchTerm ? ( + + ) : ( + // Nothing yet: the two ways in, side by side. + + + +
    + } + /> + ) ) : (
    {currentDocuments.map((document, index) => { const docId = document.id ? document.id.toString() : ''; + const connection = document.connectionId + ? connections.find((c) => c.id === document.connectionId) + : undefined; + const paused = + connection?.status === 'reconnect_needed' || + connection?.status === 'disconnected'; return (
    @@ -695,6 +746,39 @@ export default function Sources({
    + {connection && paused && ( +
    + + + + {t('settings.connectors.detail.paused')} + + + + {t('settings.sources.paused', { + name: connection.name, + interpolation: { escapeValue: false }, + })} + + + {/* The reader's own connection (only theirs are + loaded): sign in again right here, without + opening the source. */} + +
    + )} {document.ingestStatus === 'failed' && ( {t('settings.sources.ingestFailed')} @@ -735,6 +819,23 @@ export default function Sources({ ); })()} + {connection && ( + + + + {t('settings.tools.viaConnection', { + name: connection.name, + interpolation: { escapeValue: false }, + })} + + + )} {document.date ? formatDate(document.date) : ''} @@ -785,9 +886,14 @@ export default function Sources({ onSuccessfulUpload={() => refreshDocs(undefined, currentPage, rowsPerPage) } + onBrowseConnectors={() => + navigate('/settings/connectors?capability=sync') + } /> )} + {signInModals} + {deleteModalState === 'ACTIVE' && documentToDelete && ( + {documentForWikiSettings && ( + setDocumentForWikiSettings(null)} + /> + )} + { diff --git a/frontend/src/settings/Teams.test.tsx b/frontend/src/settings/Teams.test.tsx index 218403a7..e64b5214 100644 --- a/frontend/src/settings/Teams.test.tsx +++ b/frontend/src/settings/Teams.test.tsx @@ -1,6 +1,6 @@ import { act } from 'react'; import { createRoot, type Root } from 'react-dom/client'; -import { MemoryRouter } from 'react-router-dom'; +import { MemoryRouter, useLocation } from 'react-router-dom'; // A JWT whose payload is {"sub":"me"}. const TOKEN = `x.${btoa(JSON.stringify({ sub: 'me' }))}.y`; @@ -183,6 +183,10 @@ describe('Teams page', () => { document.body.innerHTML = ''; }); + function Where() { + return
    {useLocation().pathname}
    ; + } + const render = async () => { act(() => { root.render( @@ -190,6 +194,7 @@ describe('Teams page', () => { initialEntries={[{ pathname: '/teams', state: { openTeamId: 't1' } }]} > + , ); }); @@ -362,6 +367,27 @@ describe('Teams page', () => { ); }); + it('opens a shared source on the Knowledge page', async () => { + listGrants.mockResolvedValue({ + team_role: 'team_member', + grants: [grant({ caller: VIEWER })], + }); + await render(); + await openDrawer(); + const open = Array.from( + body().querySelectorAll( + '[data-slot="sheet-content"] button', + ), + ).find((b) => + b.textContent?.startsWith('settings.teams.drawer.open(type='), + )!; + act(() => open.click()); + await flush(); + expect(body().querySelector('[data-testid="where"]')?.textContent).toBe( + '/settings/knowledge', + ); + }); + it('sends target_user_id when removing a per-member grant', async () => { listGrants.mockResolvedValue({ team_role: 'team_admin', diff --git a/frontend/src/settings/Teams.tsx b/frontend/src/settings/Teams.tsx index ddc8f203..da48633d 100644 --- a/frontend/src/settings/Teams.tsx +++ b/frontend/src/settings/Teams.tsx @@ -737,7 +737,7 @@ export default function Teams() { return agent ? agentEditPathFor(agent) : agentEditPath(r.id); } case 'source': - return '/settings/sources'; + return '/settings/knowledge'; case 'tool': return '/settings/tools'; case 'prompt': diff --git a/frontend/src/settings/ToolConfig.test.tsx b/frontend/src/settings/ToolConfig.test.tsx index e9fb5756..4a5467b8 100644 --- a/frontend/src/settings/ToolConfig.test.tsx +++ b/frontend/src/settings/ToolConfig.test.tsx @@ -19,9 +19,11 @@ vi.mock('../modals/AddActionModal', () => ({ default: () => null })); vi.mock('../modals/ImportSpecModal', () => ({ default: () => null })); const updateTool = vi.fn(); +const createTool = vi.fn(); vi.mock('../api/services/userService', () => ({ default: { updateTool: (...args: unknown[]) => updateTool(...args), + createTool: (...args: unknown[]) => createTool(...args), deleteTool: () => Promise.resolve(), }, })); @@ -183,6 +185,28 @@ describe('ToolConfig', () => { expect(alert?.className).toContain('text-destructive'); }); + it('shows a save the server refused as failed', async () => { + updateTool.mockResolvedValue({ ok: false, status: 400 }); + await render({ ...userTool, customName: '' }); + const name = container.querySelector( + 'input[placeholder="settings.tools.customNamePlaceholder"]', + ); + await act(async () => { + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )?.set; + setter?.call(name, 'Renamed'); + name?.dispatchEvent(new Event('input', { bubbles: true })); + }); + await act(async () => { + buttonByText('settings.tools.save')?.click(); + }); + expect( + container.querySelector('[role="alert"]')?.textContent, + ).toBe('settings.tools.saveFailed'); + }); + describe('a shared MCP server', () => { const mcpTool = (authType: string) => ({ @@ -451,6 +475,16 @@ describe('ToolConfig', () => { expect(cancel?.dataset.shape).toBe('pill'); }); + it('creates a draft OpenAPI tool on its first save, not before', async () => { + createTool.mockResolvedValue({ ok: true }); + updateTool.mockClear(); + await render({ ...apiTool, id: '' } as APIToolType); + expect(createTool).not.toHaveBeenCalled(); + await act(async () => buttonByText('settings.tools.save')!.click()); + expect(updateTool).not.toHaveBeenCalled(); + expect(createTool.mock.calls[0][0]).toMatchObject({ name: 'api_tool' }); + }); + describe('access', () => { const viewer = { access: 'viewer', allowed_actions: ['use'] }; const editorNoCreds = { @@ -533,6 +567,30 @@ describe('ToolConfig', () => { expect(credential && disabled(credential)).toBe(true); }); + it('locks fixed values for anyone but the owner', async () => { + // Even an editor who may change credentials: the server refuses a fixed + // value from anyone but the owner. + await render({ + ...userTool, + access: 'editor', + allowed_actions: ['edit', 'edit_credentials', 'use', 'use_in_own'], + } as UserToolType); + await act(async () => { + ( + container.querySelector('[class*="cursor-pointer"]') as HTMLElement + ).click(); + }); + const filled = container.querySelector( + '[aria-label="settings.tools.filledByLLM"]', + ); + expect(filled?.hasAttribute('disabled')).toBe(true); + const tableInputs = Array.from( + container.querySelectorAll('table input[data-slot]'), + ); + // The description stays editable; the value is the fixed one. + expect(tableInputs.at(-1)?.disabled).toBe(true); + }); + it('says why the credentials are locked for an editor without edit_credentials', async () => { await render({ ...configTool, ...editorNoCreds } as UserToolType); const note = container.querySelector('[data-slot="alert"]'); @@ -551,6 +609,48 @@ describe('ToolConfig', () => { ); }); + // The connection's secret is the owner's alone: the server refuses any + // credential change from anyone else, so the form must not offer one. + it("locks a connected tool's credentials for an editor allowed to change credentials", async () => { + updateTool.mockResolvedValue({ ok: true }); + await render({ + ...configTool, + connection_id: 'owner-conn', + access: 'editor', + allowed_actions: ['edit', 'edit_credentials', 'use'], + } as UserToolType); + const note = container.querySelector('[data-slot="alert"]'); + expect(note?.textContent).toBe('common.credentialsLockedNotice'); + const credential = container.querySelector( + 'input[type="password"]', + ); + expect(credential && disabled(credential)).toBe(true); + await act(async () => { + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )?.set; + setter?.call(nameInput(), 'Renamed'); + nameInput().dispatchEvent(new Event('input', { bubbles: true })); + }); + await act(async () => { + buttonByText('settings.tools.save')?.click(); + }); + expect(updateTool).toHaveBeenCalledTimes(1); + expect(updateTool.mock.calls[0][0]).not.toHaveProperty('config'); + }); + + it("keeps a connected tool's credentials open to its owner", async () => { + await render({ + ...configTool, + connection_id: 'my-conn', + access: 'owner', + } as UserToolType); + expect(container.textContent).not.toContain( + 'common.credentialsLockedNotice', + ); + }); + it("disables an API tool's URL and header values without edit_credentials", async () => { await render({ ...apiTool, ...editorNoCreds } as APIToolType); await expandFirstAction(); diff --git a/frontend/src/settings/ToolConfig.tsx b/frontend/src/settings/ToolConfig.tsx index d9eeba9a..76cf5a33 100644 --- a/frontend/src/settings/ToolConfig.tsx +++ b/frontend/src/settings/ToolConfig.tsx @@ -40,7 +40,7 @@ import ImportSpecModal from '../modals/ImportSpecModal'; import { ActiveState } from '../models/misc'; import { selectToken } from '../preferences/preferenceSlice'; import { getMethodBadgeVariant } from '../utils/httpMethodColors'; -import { can } from '../utils/accessUtils'; +import { can, isOwner } from '../utils/accessUtils'; import { isSharedOAuthMcp } from '../utils/toolUtils'; import { areObjectsEqual } from '../utils/objectUtils'; import { cn, focusRing } from '@/lib/utils'; @@ -58,11 +58,14 @@ const BODY_TYPE_HINT_KEYS: Record = { /** * What the caller may change on the open tool (`utils/accessUtils` `can`): * `canEdit` covers the name and the actions, `canEditCredentials` the - * secrets, URLs and header / query values. + * secrets, URLs and header / query values, and `canFixValues` whether a + * parameter is filled by the AI or fixed, and its fixed value (the owner's + * alone: the server refuses anyone else). */ const ToolAccessContext = React.createContext({ canEdit: true, canEditCredentials: true, + canFixValues: true, }); /** Maps a body content type to its hint's locale key suffix (JSON by default). */ @@ -125,12 +128,18 @@ export default function ToolConfig({ // A shared OAuth server's connection stays with its owner (the backend // refuses it), so its fields lock like credentials the caller can't change. const sharedOAuth = isSharedOAuthMcp(tool); - const canEditCredentials = can(tool, 'edit_credentials') && !sharedOAuth; + // A connected tool's secret lives on the owner's connection, which the + // server lets only its owner change, whatever the owner's switch says. + const ownerOnlyConnection = + 'connection_id' in tool && !!tool.connection_id && !isOwner(tool); + const canEditCredentials = + can(tool, 'edit_credentials') && !sharedOAuth && !ownerOnlyConnection; // Neither: the tool opens as a read-only view with no Save. const readOnly = !canEdit && !canEditCredentials; + const canFixValues = isOwner(tool); const access = React.useMemo( - () => ({ canEdit, canEditCredentials }), - [canEdit, canEditCredentials], + () => ({ canEdit, canEditCredentials, canFixValues }), + [canEdit, canEditCredentials, canFixValues], ); const toggleUserActionExpand = (index: number) => { @@ -269,24 +278,25 @@ export default function ToolConfig({ }); }; - /** Sends the edit; a non-2xx response throws so the caller shows it. */ - const saveTool = async (configToSave: { [key: string]: any }) => { - const response = await userService.updateTool( - { - id: tool.id, - name: tool.name, - displayName: tool.displayName, - customName: customName, - description: tool.description, - // Locked config isn't sent, so a rename or action edit still saves. - ...((canEditCredentials || tool.name === 'api_tool') && { - config: configToSave, - }), - actions: 'actions' in tool ? tool.actions : [], - status: tool.status, - }, - token, - ); + // Saves the tool; a draft without an id (a new OpenAPI tool) is created + // on its first save, so leaving without saving leaves nothing behind. A + // non-2xx response throws so the caller shows it. + const persistTool = async (configToSave: Record) => { + const payload = { + name: tool.name, + displayName: tool.displayName, + customName: customName, + description: tool.description, + // Locked config isn't sent, so a rename or action edit still saves. + ...((canEditCredentials || tool.name === 'api_tool') && { + config: configToSave, + }), + actions: 'actions' in tool ? tool.actions : [], + status: tool.status, + }; + const response = tool.id + ? await userService.updateTool({ id: tool.id, ...payload }, token) + : await userService.createTool(payload, token); if (!response?.ok) throw new Error('Failed to save tool'); }; @@ -298,7 +308,7 @@ export default function ToolConfig({ setSaveError(''); try { - await saveTool(configToSave); + await persistTool(configToSave); setInitialState({ customName, configValues: { ...configValues }, @@ -314,12 +324,6 @@ export default function ToolConfig({ } }; - const handleDelete = () => { - userService.deleteTool({ id: tool.id }, token).then(() => { - handleGoBack(); - }); - }; - const handleAddNewAction = (actionName: string) => { const toolCopy = tool as APIToolType; @@ -396,7 +400,8 @@ export default function ToolConfig({ size="sm" shape="pill" onClick={handleSaveChanges} - disabled={!hasUnsavedChanges} + // A draft (no id yet) is saved to create it. + disabled={!hasUnsavedChanges && !!tool.id} loading={saving} > {t('settings.tools.save')} @@ -680,6 +685,7 @@ export default function ToolConfig({ { setTool({ @@ -820,7 +829,7 @@ export default function ToolConfig({ setSaveError(''); try { - await saveTool(configToSave); + await persistTool(configToSave); setShowUnsavedModal(false); handleGoBack(); } catch { @@ -1292,7 +1301,8 @@ function APIActionTable({ ) => void; }) { const { t } = useTranslation(); - const { canEditCredentials } = React.useContext(ToolAccessContext); + const { canEditCredentials, canFixValues } = + React.useContext(ToolAccessContext); const idPrefix = React.useId(); const [action, setAction] = React.useState(apiAction); @@ -1556,6 +1566,7 @@ function APIActionTable({ id={`${idPrefix}-${section}-${index}-filled-by-llm`} aria-label={t('settings.tools.filledByLLM')} checked={param.filled_by_llm} + disabled={!canFixValues} onCheckedChange={(checked) => handlePropertyChange( section, @@ -1587,6 +1598,7 @@ function APIActionTable({ value={param.value} disabled={ param.filled_by_llm || + !canFixValues || (section === 'query_params' && !canEditCredentials) } onChange={(e) => diff --git a/frontend/src/settings/Tools.test.tsx b/frontend/src/settings/Tools.test.tsx index 8421017e..f08feace 100644 --- a/frontend/src/settings/Tools.test.tsx +++ b/frontend/src/settings/Tools.test.tsx @@ -1,9 +1,22 @@ import { act, useState } from 'react'; import { createRoot, type Root } from 'react-dom/client'; +import { MemoryRouter, Route, Routes, useLocation } from 'react-router-dom'; +// The store's slices the page reads; each test sets the connections it needs. +const reduxState = { + preference: { token: 'token' }, + connectors: { + enabled: true, + catalog: [] as unknown[], + connections: [] as unknown[], + loaded: true, + loading: false, + failed: false, + }, +}; const dispatch = vi.fn(); vi.mock('react-redux', () => ({ - useSelector: () => 'token', + useSelector: (selector: (state: unknown) => unknown) => selector(reduxState), useDispatch: () => dispatch, })); @@ -56,9 +69,20 @@ vi.mock('../modals/MCPServerModal', () => ({ return null; }, })); -vi.mock('../teams/ShareToTeamModal', () => ({ default: () => null })); +const shareModalProps = vi.fn(); +vi.mock('../teams/ShareToTeamModal', () => ({ + default: (props: unknown) => { + shareModalProps(props); + return null; + }, +})); vi.mock('../api/services/devicesService', () => ({ default: {} })); +const reconnect = vi.fn(); +vi.mock('../connectors/SignInAgainNotice', () => ({ + useSignInAgain: () => ({ reconnect, modals: null }), +})); + const getUserTools = vi.fn(); const updateToolStatus = vi.fn(); vi.mock('../api/services/userService', () => ({ @@ -127,11 +151,30 @@ const viewerTool = { const jsonResponse = (body: unknown, ok = true, status = 200) => Promise.resolve({ ok, status, json: () => Promise.resolve(body) }); +function Where() { + const location = useLocation(); + return
    {location.pathname + location.search}
    ; +} + +const renderTools = (root: Root) => + act(async () => { + root.render( + + + } /> + } /> + + , + ); + }); + describe('Tools', () => { let container: HTMLDivElement; let root: Root; beforeEach(() => { + reduxState.connectors.catalog = []; + reduxState.connectors.connections = []; dispatch.mockReset(); getUserTools.mockReset(); updateToolStatus.mockReset(); @@ -149,9 +192,7 @@ describe('Tools', () => { const render = async (tools: unknown[]) => { getUserTools.mockImplementation(() => jsonResponse({ tools })); - await act(async () => { - root.render(); - }); + await renderTools(root); }; const card = (name: string) => @@ -225,39 +266,25 @@ describe('Tools', () => { }); }); - it('labels the switch "In my chats" and binds it to in_chat', async () => { + // The caller's own "In my chats" switch stays on the tile, named for + // screen readers only: the tile shows no label text beside it. + it('keeps the In my chats switch, bound to in_chat, without its label text', async () => { await render([ownTool, editorTool]); const sw = switchOf('ed'); expect(sw.getAttribute('aria-checked')).toBe('false'); expect(switchOf('own').getAttribute('aria-checked')).toBe('true'); - const label = card('ed').querySelector( - `label[for="${sw.id}"]`, - ); - expect(label?.textContent).toBe('settings.tools.inMyChats'); expect(sw.getAttribute('aria-label')).toBe( 'settings.tools.useInMyChatsAria:{"toolName":"ed"}', ); + expect(card('ed').querySelector(`label[for="${sw.id}"]`)).toBeNull(); + expect(container.textContent).not.toContain('settings.tools.inMyChats'); }); // The tool can't be in the caller's chats at all (the composer picker // hides it too), so there is no state to show: no switch, no bare grey one. - it('hides the switch and its label for a shared tool without use_in_own', async () => { + it('hides the switch for a shared tool without use_in_own', async () => { await render([viewerTool]); expect(card('vw').querySelector('[role="switch"]')).toBeNull(); - expect(card('vw').textContent).not.toContain('settings.tools.inMyChats'); - }); - - it('shows the role on a shared tile as a neutral Users Badge', async () => { - await render([ownTool, editorTool, viewerTool]); - const badge = (id: string) => - card(id).querySelector('[data-testid="role-badge"]'); - expect(badge('own')).toBeNull(); - expect(badge('ed')?.dataset.variant).toBe('neutral'); - expect(badge('ed')?.textContent).toBe('teamAccess.editor'); - expect(badge('vw')?.textContent).toBe('teamAccess.viewer'); - expect( - badge('ed')?.querySelector('svg')?.getAttribute('class'), - ).not.toContain('size-3'); }); it('reverts the switch and shows an error toast when the update fails', async () => { @@ -280,9 +307,348 @@ describe('Tools', () => { it('keeps the new value when the update succeeds', async () => { await render([editorTool]); + // Mounting loads the connectors; only a toast would come after it. + dispatch.mockClear(); updateToolStatus.mockImplementation(() => jsonResponse({ success: true })); await act(async () => switchOf('ed').click()); expect(switchOf('ed').getAttribute('aria-checked')).toBe('true'); expect(dispatch).not.toHaveBeenCalled(); }); + + it('shows the role on a shared tile as a neutral Users Badge', async () => { + await render([ownTool, editorTool, viewerTool]); + const badge = (id: string) => + card(id).querySelector('[data-testid="role-badge"]'); + expect(badge('own')).toBeNull(); + expect(badge('ed')?.dataset.variant).toBe('neutral'); + expect(badge('ed')?.textContent).toBe('teamAccess.editor'); + expect(badge('vw')?.textContent).toBe('teamAccess.viewer'); + expect( + badge('ed')?.querySelector('svg')?.getAttribute('class'), + ).not.toContain('size-3'); + }); +}); + +describe('Tools page connections', () => { + let container: HTMLDivElement; + let root: Root; + + const TOOLS = [ + { + id: 'tg', + name: 'telegram', + displayName: 'Telegram · Alerts bot', + customName: 'Telegram · Alerts bot', + description: 'Send messages', + status: true, + config: {}, + actions: [], + connection_id: 'conn-1', + }, + { + id: 'api', + name: 'api_tool', + displayName: 'API Tool', + customName: 'My API', + description: 'Calls an API', + status: true, + config: {}, + actions: [], + connection_id: null, + }, + { + id: 'lin', + name: 'mcp_tool', + displayName: 'Linear', + customName: '', + description: 'MCP Server: https://mcp.linear.app/mcp', + status: true, + config: {}, + actions: [], + connection_id: 'conn-2', + }, + ]; + + beforeEach(() => { + reduxState.connectors.catalog = [ + { + key: 'telegram', + name: 'Telegram', + description: 'Send messages to a chat.', + publisher: 'built_in', + available: true, + capabilities: ['write'], + }, + { + key: 'mcp:linear', + name: 'Linear', + description: 'Issues and projects.', + publisher: 'preset', + available: true, + capabilities: ['write'], + }, + ]; + reduxState.connectors.connections = [ + { + id: 'conn-1', + connector_key: 'telegram', + name: 'Telegram', + icon: 'tool_telegram', + status: 'connected', + account_label: '…abcd', + account_name: 'Alerts bot', + }, + { + id: 'conn-2', + connector_key: 'mcp:linear', + name: 'Linear', + icon: 'linear', + status: 'reconnect_needed', + account_label: 'Linear', + }, + ]; + dispatch.mockReset(); + getUserTools.mockReset(); + getUserTools.mockImplementation(() => jsonResponse({ tools: TOOLS })); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(() => { + act(() => root.unmount()); + container.remove(); + }); + + const card = (title: string) => + Array.from(container.querySelectorAll('h2')) + .find((h) => h.textContent === title)! + .closest('[data-slot="card"]')!; + const menuItem = (title: string, label: string) => + Array.from( + card(title).querySelectorAll( + '[data-testid="menu"] button', + ), + ).find((b) => b.textContent === label); + + // A connected tool is managed on the Connectors page; its card keeps + // what acts on the tool itself. + it('gives the owner of a connected tool Share and Delete, no connection item', async () => { + await renderTools(root); + expect( + Array.from( + card('Telegram').querySelectorAll('[data-testid="menu"] button'), + ).map((b) => b.textContent), + ).toEqual(['settings.tools.shareWithTeam', 'settings.tools.delete']); + expect( + menuItem('Telegram', 'settings.connectors.manageConnection'), + ).toBeUndefined(); + }); + + it('shows each connected account as its own tool, the account on its footer', async () => { + await renderTools(root); + const telegram = card('Telegram'); + // Which account this card is beside its own switch, and the catalog's + // plain description. + const footer = telegram.querySelector('[data-slot="card-footer"]')!; + expect(footer.textContent).toBe('Alerts bot'); + expect(footer.querySelector('[role="switch"]')).not.toBeNull(); + expect(telegram.textContent).toContain( + 'settings.connectors.descriptions.telegram', + ); + }); + + it('names an account by its login, not as a key ending', async () => { + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + ...TOOLS, + { + id: 'gh', + name: 'mcp_tool', + displayName: 'GitHub', + customName: '', + description: '', + status: true, + config: {}, + actions: [], + connection_id: 'conn-gh', + }, + ], + }), + ); + reduxState.connectors.connections = [ + ...reduxState.connectors.connections, + { + id: 'conn-gh', + connector_key: 'github', + name: 'GitHub', + icon: 'github', + status: 'connected', + auth_kind: 'api_key', + account_label: 'dartpain', + }, + ]; + await renderTools(root); + const footer = card('GitHub').querySelector('[data-slot="card-footer"]')!; + expect(footer.textContent).toBe('dartpain'); + }); + + it('says a connected tool needs signing in again, with no raw MCP reconnect', async () => { + await renderTools(root); + expect(card('Linear').textContent).toContain( + 'settings.connectors.health.signInAgain', + ); + expect(card('Linear').textContent).not.toContain('MCP Server:'); + expect(menuItem('Linear', 'settings.tools.reconnect')).toBeUndefined(); + }); + + // The owner signs in again right from the card; an MCP preset re-signs + // its existing tool. + it('offers the owner Sign in again on a connected tool that needs it', async () => { + reconnect.mockClear(); + await renderTools(root); + expect( + menuItem('Telegram', 'settings.connectors.health.signInAgain'), + ).toBeUndefined(); + await act(async () => + menuItem('Linear', 'settings.connectors.health.signInAgain')!.click(), + ); + expect(reconnect).toHaveBeenCalledWith( + expect.objectContaining({ id: 'conn-2', connector_key: 'mcp:linear' }), + 'lin', + ); + }); + + it("offers no Sign in again on a teammate's connected tool", async () => { + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + { + ...TOOLS[2], + customName: 'Linear (shared)', + connection_id: 'owner-conn', + access: 'editor', + ownership: 'team', + allowed_actions: ['edit', 'use', 'use_in_own'], + }, + ], + }), + ); + await renderTools(root); + expect( + menuItem('Linear (shared)', 'settings.connectors.health.signInAgain'), + ).toBeUndefined(); + }); + + it('keeps Edit for a tool that is not from a connection', async () => { + await renderTools(root); + expect(menuItem('My API', 'settings.tools.edit')).toBeDefined(); + expect(card('My API').querySelector('[role="switch"]')).not.toBeNull(); + }); + + it("gives a viewer of a teammate's connected tool View", async () => { + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + { + ...TOOLS[0], + customName: 'Telegram (shared)', + connection_id: 'owner-conn', + access: 'viewer', + ownership: 'team', + team_access: 'viewer', + allowed_actions: ['use', 'use_in_own'], + }, + ], + }), + ); + await renderTools(root); + expect(menuItem('Telegram (shared)', 'settings.tools.view')).toBeDefined(); + expect( + menuItem('Telegram (shared)', 'settings.tools.reconnect'), + ).toBeUndefined(); + }); + + // The connections load after the tools; the owner must not see the raw + // MCP Reconnect for a connected tool in between. + it("offers no Reconnect for the owner's connected MCP tool before connections load", async () => { + reduxState.connectors.connections = []; + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + { + ...TOOLS[2], + customName: 'Linear (mine)', + config: { auth_type: 'bearer' }, + access: 'owner', + }, + ], + }), + ); + await renderTools(root); + expect( + menuItem('Linear (mine)', 'settings.tools.reconnect'), + ).toBeUndefined(); + // Nor the tool editor: the owner manages it on the Connectors page. + expect(menuItem('Linear (mine)', 'settings.tools.edit')).toBeUndefined(); + }); + + it("shows an editor who may share the owner's account, locked, with the write confirmation", async () => { + shareModalProps.mockReset(); + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + { + ...TOOLS[0], + customName: 'Telegram (shared)', + connection_id: 'owner-conn', + credential_mode: 'owner', + actions: [{ name: 'send', access: 'write' }], + access: 'editor', + ownership: 'team', + team_access: 'editor', + allowed_actions: ['edit', 'share', 'use', 'use_in_own'], + }, + ], + }), + ); + await renderTools(root); + await act(async () => + menuItem('Telegram (shared)', 'settings.tools.shareWithTeam')!.click(), + ); + const props = shareModalProps.mock.calls.at(-1)?.[0] as { + credentials?: Record; + }; + expect(props.credentials).toMatchObject({ + toolId: 'tg', + connectorName: 'Telegram', + mode: 'owner', + hasWrites: true, + readOnly: true, + }); + }); + + it("offers no Reconnect for a teammate's connected MCP tool", async () => { + getUserTools.mockImplementation(() => + jsonResponse({ + tools: [ + { + ...TOOLS[2], + customName: 'Linear (shared)', + connection_id: 'owner-conn', + config: { auth_type: 'bearer' }, + access: 'editor', + ownership: 'team', + allowed_actions: ['edit', 'edit_credentials', 'use', 'use_in_own'], + }, + ], + }), + ); + await renderTools(root); + expect( + menuItem('Linear (shared)', 'settings.tools.reconnect'), + ).toBeUndefined(); + expect(menuItem('Linear (shared)', 'settings.tools.edit')).toBeDefined(); + }); }); diff --git a/frontend/src/settings/Tools.tsx b/frontend/src/settings/Tools.tsx index 05dad368..0999ca1f 100644 --- a/frontend/src/settings/Tools.tsx +++ b/frontend/src/settings/Tools.tsx @@ -2,6 +2,7 @@ import { Eye, Pencil, RefreshCw, Trash2, Users } from 'lucide-react'; import React from 'react'; import { useTranslation } from 'react-i18next'; import { useDispatch, useSelector } from 'react-redux'; +import { useLocation, useNavigate } from 'react-router-dom'; import devicesService from '../api/services/devicesService'; import userService from '../api/services/userService'; @@ -12,19 +13,38 @@ import SkeletonLoader from '../components/SkeletonLoader'; import ToolIcon from '../components/ToolIcon'; import { Badge } from '../components/ui/badge'; import { Button } from '../components/ui/button'; -import { Card, CardDescription, CardTitle } from '../components/ui/card'; -import { Label } from '../components/ui/label'; -import { Switch } from '../components/ui/switch'; +import { + Card, + CardDescription, + CardFooter, + CardTitle, +} from '../components/ui/card'; import { ActionMenu, type MenuOption } from '../components/ui/dropdown-menu'; +import { Switch } from '../components/ui/switch'; import { EmptyState } from '../components/ui/empty-state'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { + connectionNeedsSignIn, + loadConnectors, + selectConnections, + selectConnectorCatalog, +} from '../connectors/connectorsSlice'; +import { connectorDescription, isKeyHint } from '../connectors/i18n'; +import type { Connection } from '../connectors/types'; +import { useSignInAgain } from '../connectors/SignInAgainNotice'; +import { toolServiceOf } from '../connectors/toolService'; import { useLoaderState } from '../hooks'; +import type { AvailableToolType } from '../modals/types'; import AddToolModal from '../modals/AddToolModal'; import ConfirmationModal from '../modals/ConfirmationModal'; import MCPServerModal from '../modals/MCPServerModal'; import { ActiveState } from '../models/misc'; import { showActionToast } from '../notifications/actionToastSlice'; import { selectToken } from '../preferences/preferenceSlice'; -import ShareToTeamModal from '../teams/ShareToTeamModal'; +import type { AppDispatch } from '../store'; +import ShareToTeamModal, { + type ShareCredentials, +} from '../teams/ShareToTeamModal'; import { can, isOwner, roleOf } from '../utils/accessUtils'; import { canAddToolToOwn, @@ -38,7 +58,11 @@ import { APIToolType, UserToolType } from './types'; export default function Tools() { const { t } = useTranslation(); const token = useSelector(selectToken); - const dispatch = useDispatch(); + const dispatch = useDispatch(); + const connections = useSelector(selectConnections); + const catalog = useSelector(selectConnectorCatalog); + const location = useLocation(); + const navigate = useNavigate(); const [searchTerm, setSearchTerm] = React.useState(''); const [addToolModalState, setAddToolModalState] = @@ -63,6 +87,33 @@ export default function Tools() { [toolId: string]: string; }>({}); + // A connection-backed tool shares its connection's account or asks each + // member to connect their own; the share dialog shows that choice. Whose + // account it is stays the owner's choice: an editor the owner lets share + // sees it locked, and still confirms before sharing a tool that can act. + const shareCredentials = ( + tool: UserToolType, + ): ShareCredentials | undefined => { + if (!tool.connection_id) return undefined; + const owner = isOwner(tool); + const service = toolServiceOf(tool, connections, catalog); + // The owner's own connection names the account; until it loads there + // is nothing to show them. + if (!service || (owner && !service.connection)) return undefined; + const policy = service.connector?.credential_policy; + return { + toolId: tool.id, + connectorName: service.name, + account: owner ? (service.connection?.account_label ?? '') : '', + mode: tool.credential_mode === 'member' ? 'member' : 'owner', + forcedMode: policy === 'owner' || policy === 'member' ? policy : null, + hasWrites: (tool.actions ?? []).some( + (action) => action.access === 'write', + ), + readOnly: !owner, + }; + }; + const handleDeleteTool = (tool: UserToolType) => { setToolToDelete(tool); setDeleteModalState('ACTIVE'); @@ -129,24 +180,45 @@ export default function Tools() { const getMenuOptions = (tool: UserToolType): MenuOption[] => { const canEdit = can(tool, 'edit') || can(tool, 'edit_credentials'); - const options: MenuOption[] = [ - canEdit - ? { - icon: Pencil, - label: t('settings.tools.edit'), - onClick: () => handleSettingsClick(tool), - variant: 'default', - } - : { - icon: Eye, - label: t('settings.tools.view'), - onClick: () => handleSettingsClick(tool), - variant: 'default', - }, - ]; - // A shared OAuth server's sign-in is the owner's to redo. + const options: MenuOption[] = []; + // The caller's own connection that needs signing in again (the badge + // says so) is signed in again from here, in place where it can be. + const connection = connectionOf(tool); + if (connection && connectionNeedsSignIn(connection)) + options.push({ + icon: RefreshCw, + label: t('settings.connectors.health.signInAgain'), + onClick: () => + reconnect(connection, tool.name === 'mcp_tool' ? tool.id : undefined), + variant: 'default', + }); + // The owner's connected tool (its connection is theirs) is managed on + // the Connectors page, so it has no editor here; a teammate's opens the + // tool editor like any other shared tool. + if (!(tool.connection_id && isOwner(tool))) + options.push( + canEdit + ? { + icon: Pencil, + label: t('settings.tools.edit'), + onClick: () => setSelectedTool(tool), + variant: 'default', + } + : { + icon: Eye, + label: t('settings.tools.view'), + onClick: () => setSelectedTool(tool), + variant: 'default', + }, + ); + // A connected server signs in again through its connection (Sign in + // again above, for the caller's own); only an MCP tool without one keeps + // the server form. The tool's own connection id decides, for everyone: a + // teammate never sees the owner's connection, and the owner's loads + // after the tools. A shared OAuth server's sign-in is the owner's to redo. if ( tool.name === 'mcp_tool' && + !tool.connection_id && can(tool, 'edit_credentials') && !isSharedOAuthMcp(tool) ) { @@ -188,6 +260,10 @@ export default function Tools() { .catch(() => {}); }, [token]); + const { reconnect, modals: signInModals } = useSignInAgain({ + onConnected: () => getUserTools(), + }); + const getUserTools = () => { setLoading(true); userService @@ -247,9 +323,24 @@ export default function Tools() { }); }; - const handleSettingsClick = (tool: UserToolType) => { - setSelectedTool(tool); - }; + // The caller's own connection behind a connected tool (only their own + // connections are loaded): the card names its account. + const connectionOf = (tool: UserToolType) => + tool.connection_id + ? connections.find((c) => c.id === tool.connection_id) + : undefined; + // What tells two accounts of one service apart on their cards: the name + // the owner gave it, else what identifies it. + const accountLine = (connection: Connection) => + connection.account_name + ? connection.account_name + : connection.auth_kind === 'api_key' && + isKeyHint(connection.account_label) + ? t('settings.connectors.detail.keyEnding', { + hint: connection.account_label, + interpolation: { escapeValue: false }, + }) + : connection.account_label; const handleGoBack = () => { setSelectedTool(null); @@ -294,7 +385,46 @@ export default function Tools() { React.useEffect(() => { getUserTools(); fetchMcpStatuses(); + dispatch(loadConnectors({ token })); }, []); + + // The Connectors page opens a new OpenAPI tool here as an unsaved draft: + // the spec import opens straight away and the tool is created on save. + const routeState = location.state as { + openToolId?: string; + newApiTool?: boolean; + } | null; + const openToolId = routeState?.openToolId; + const newApiTool = routeState?.newApiTool; + React.useEffect(() => { + if (!openToolId) return; + handleToolAdded(openToolId); + navigate(location.pathname, { replace: true, state: null }); + }, [openToolId]); + React.useEffect(() => { + if (!newApiTool) return; + navigate(location.pathname, { replace: true, state: null }); + userService + .getAvailableTools(token) + .then((res) => res.json()) + .then((data) => { + const template = (data.data as AvailableToolType[] | undefined)?.find( + (candidate) => candidate.name === 'api_tool', + ); + if (!template) return; + setSelectedTool({ + id: '', + name: template.name, + displayName: template.displayName, + customName: '', + description: template.description, + config: {}, + actions: template.actions, + status: true, + } as unknown as UserToolType); + }) + .catch(() => undefined); + }, [newApiTool]); return (
    {selectedTool ? ( @@ -356,99 +486,148 @@ export default function Tools() { ) : (
    - {filtered.map((tool, index) => ( - - {!tool.default && ( - - )} -
    -
    - { + const connection = connectionOf(tool); + const connector = connection + ? catalog.find( + (c) => c.key === connection.connector_key, + ) + : undefined; + // A catalog service reads as the catalog describes it; + // a custom server keeps the description it came with. + // The account is on the card's own line, so the title + // drops the " · account" the server adds to tell + // accounts apart in pickers. + const fullName = tool.customName || tool.displayName; + const title = + connection && + fullName.startsWith(`${connection.name} · `) + ? connection.name + : fullName; + const description = + connector && connector.publisher !== 'custom' + ? connectorDescription(t, connector) + : tool.description; + return ( + + {!tool.default && ( + - {tool.default && ( - - {t('settings.tools.builtIn')} - - )} - {tool.name === 'mcp_tool' && - mcpStatuses[tool.id] && ( - - {mcpStatuses[tool.id] === 'connected' - ? t('settings.tools.authStatus.connected') - : mcpStatuses[tool.id] === 'needs_auth' - ? t('settings.tools.authStatus.needsAuth') - : t( - 'settings.tools.authStatus.configured', - )} + )} +
    +
    + + {tool.default && ( + + {t('settings.tools.builtIn')} )} - + {connectionNeedsSignIn(connection) && ( + + {t('settings.connectors.health.signInAgain')} + + )} + {tool.name === 'mcp_tool' && + !connection && + mcpStatuses[tool.id] && ( + + {mcpStatuses[tool.id] === 'connected' + ? t('settings.tools.authStatus.connected') + : mcpStatuses[tool.id] === 'needs_auth' + ? t( + 'settings.tools.authStatus.needsAuth', + ) + : t( + 'settings.tools.authStatus.configured', + )} + + )} + +
    +
    + + {title} + + + {description} + +
    -
    - - {tool.customName || tool.displayName} - - - {tool.description} - -
    -
    - {/* A shared tool without use_in_own can't be in the - caller's chats at all, so there is no switch. */} - {canAddToolToOwn(tool) && ( -
    - - - updateToolStatus(tool.id, checked) - } - id={`toolToggle-${index}`} - aria-label={t('settings.tools.useInMyChatsAria', { - interpolation: { escapeValue: false }, - toolName: tool.customName || tool.displayName, - })} - /> -
    - )} - - ))} + {/* Which account this is (each account of a + service is its own tool) and the caller's own + "In my chats" switch, named for screen readers + only, share the meta row. A shared tool without + use_in_own can't be in the caller's chats at + all, so it has no switch. */} + {(connection || canAddToolToOwn(tool)) && ( + + {connection && ( + + + + {accountLine(connection)} + + + )} + {canAddToolToOwn(tool) && ( + + updateToolStatus(tool.id, checked) + } + aria-label={t( + 'settings.tools.useInMyChatsAria', + { + interpolation: { escapeValue: false }, + toolName: + tool.customName || tool.displayName, + }, + )} + /> + )} + + )} + + ); + })}
    ); })() @@ -474,6 +653,7 @@ export default function Tools() { submitLabel={t('settings.tools.delete')} variant="destructive" /> + {signInModals} setToolToShare(null)} + credentials={shareCredentials(toolToShare)} + onClose={() => { + setToolToShare(null); + getUserTools(); + }} /> )}
    diff --git a/frontend/src/settings/WikiSettingsModal.test.tsx b/frontend/src/settings/WikiSettingsModal.test.tsx new file mode 100644 index 00000000..01263b8b --- /dev/null +++ b/frontend/src/settings/WikiSettingsModal.test.tsx @@ -0,0 +1,124 @@ +import { act } from 'react'; +import { createRoot, type Root } from 'react-dom/client'; + +const { service } = vi.hoisted(() => ({ + service: { getWikiSettings: vi.fn(), updateWikiSettings: vi.fn() }, +})); + +vi.mock('react-i18next', () => ({ + useTranslation: () => ({ t: (key: string) => key }), +})); + +vi.mock('react-redux', () => ({ + useSelector: () => 'token', +})); + +vi.mock('../api/services/userService', () => ({ default: service })); + +import type { Doc } from '../models/misc'; +import WikiSettingsModal from './WikiSettingsModal'; + +Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + +const wiki: Doc = { + id: 'wiki-1', + name: 'Handbook', + date: '', + model: '', + type: 'wiki', + config: { kind: 'wiki' }, +}; + +const reply = (status: number, body: unknown) => + Promise.resolve({ ok: status < 400, status, json: async () => body }); + +const OWNER = ['use', 'edit', 'manage_settings']; + +describe('WikiSettingsModal', () => { + let container: HTMLDivElement; + let root: Root; + + beforeEach(() => { + Object.values(service).forEach((fn) => fn.mockReset()); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(async () => { + await act(async () => root.unmount()); + container.remove(); + document.body.innerHTML = ''; + }); + + const render = async () => { + await act(async () => { + root.render(); + }); + }; + + const theSwitch = () => + document.body.querySelector('[role="switch"]'); + + it('shows the stored value, off by default', async () => { + service.getWikiSettings.mockReturnValue( + reply(200, { allow_outside_edits: false, allowed_actions: OWNER }), + ); + await render(); + expect(service.getWikiSettings).toHaveBeenCalledWith('wiki-1', 'token'); + expect(document.body.textContent).toContain( + 'settings.sources.wiki.settings.outsideEdits.label', + ); + expect(theSwitch()?.getAttribute('aria-checked')).toBe('false'); + expect(theSwitch()?.disabled).toBe(false); + }); + + it('the owner turns it on', async () => { + service.getWikiSettings.mockReturnValue( + reply(200, { allow_outside_edits: false, allowed_actions: OWNER }), + ); + service.updateWikiSettings.mockReturnValue( + reply(200, { allow_outside_edits: true, allowed_actions: OWNER }), + ); + await render(); + await act(async () => theSwitch()!.click()); + expect(service.updateWikiSettings).toHaveBeenCalledWith( + 'wiki-1', + { allow_outside_edits: true }, + 'token', + ); + expect(theSwitch()?.getAttribute('aria-checked')).toBe('true'); + }); + + it('flips back and says so when the save fails', async () => { + service.getWikiSettings.mockReturnValue( + reply(200, { allow_outside_edits: false, allowed_actions: OWNER }), + ); + service.updateWikiSettings.mockReturnValue(reply(403, { success: false })); + await render(); + await act(async () => theSwitch()!.click()); + expect(theSwitch()?.getAttribute('aria-checked')).toBe('false'); + expect(document.body.textContent).toContain( + 'settings.sources.wiki.settings.saveError', + ); + }); + + it('is read-only without manage_settings', async () => { + service.getWikiSettings.mockReturnValue( + reply(200, { allow_outside_edits: true, allowed_actions: ['use'] }), + ); + await render(); + expect(theSwitch()?.getAttribute('aria-checked')).toBe('true'); + expect(theSwitch()?.disabled).toBe(true); + expect(document.body.textContent).toContain('common.viewOnlyNotice'); + }); + + it('says when the settings could not load', async () => { + service.getWikiSettings.mockReturnValue(reply(500, { success: false })); + await render(); + expect(theSwitch()).toBeNull(); + expect(document.body.textContent).toContain( + 'settings.sources.wiki.settings.loadError', + ); + }); +}); diff --git a/frontend/src/settings/WikiSettingsModal.tsx b/frontend/src/settings/WikiSettingsModal.tsx new file mode 100644 index 00000000..ec1f397c --- /dev/null +++ b/frontend/src/settings/WikiSettingsModal.tsx @@ -0,0 +1,150 @@ +import { CircleAlert } from 'lucide-react'; +import { useEffect, useId, useRef, useState } from 'react'; +import { useTranslation } from 'react-i18next'; +import { useSelector } from 'react-redux'; + +import userService from '../api/services/userService'; +import ViewOnlyNotice from '../components/ViewOnlyNotice'; +import { Alert, AlertDescription } from '../components/ui/alert'; +import { LoadingState } from '../components/ui/loading-state'; +import { Modal } from '../components/ui/modal'; +import { SettingRow, SettingRows } from '../components/ui/setting-row'; +import { Switch } from '../components/ui/switch'; +import { Doc } from '../models/misc'; +import { selectToken } from '../preferences/preferenceSlice'; +import { can } from '../utils/accessUtils'; + +type WikiSettings = { + allow_outside_edits: boolean; + allowed_actions?: string[]; +}; + +interface WikiSettingsModalProps { + document: Doc; + onClose: () => void; +} + +/** + * A wiki's owner settings. Today one switch: whether people using an agent + * through its API key or widget may edit the wiki. It saves at + * once and flips back when the server refuses. + */ +export default function WikiSettingsModal({ + document, + onClose, +}: WikiSettingsModalProps) { + const { t } = useTranslation(); + const token = useSelector(selectToken); + const switchId = useId(); + + const [settings, setSettings] = useState(null); + const [loadFailed, setLoadFailed] = useState(false); + const [saving, setSaving] = useState(false); + const [saveFailed, setSaveFailed] = useState(false); + + const sourceId = document.id ?? ''; + // Read through a ref: a token refresh is the same user and must not reload. + const tokenRef = useRef(token); + tokenRef.current = token; + + useEffect(() => { + let cancelled = false; + setSettings(null); + setLoadFailed(false); + userService + .getWikiSettings(sourceId, tokenRef.current) + .then(async (response) => { + if (!response.ok) throw new Error(`HTTP ${response.status}`); + const data = (await response.json()) as WikiSettings; + if (!cancelled) setSettings(data); + }) + .catch(() => { + if (!cancelled) setLoadFailed(true); + }); + return () => { + cancelled = true; + }; + }, [sourceId]); + + const canManage = can(settings, 'manage_settings'); + + const toggle = async (value: boolean) => { + if (!settings) return; + const previous = settings; + setSettings({ ...previous, allow_outside_edits: value }); + setSaving(true); + setSaveFailed(false); + try { + const response = await userService.updateWikiSettings( + sourceId, + { allow_outside_edits: value }, + token, + ); + if (!response.ok) throw new Error(`HTTP ${response.status}`); + const data = (await response.json()) as WikiSettings; + setSettings(data); + } catch { + setSettings(previous); + setSaveFailed(true); + } finally { + setSaving(false); + } + }; + + const body = () => { + if (loadFailed) { + return ( + + + + {t('settings.sources.wiki.settings.loadError')} + + + ); + } + if (!settings) return ; + return ( +
    + {!canManage && } + {saveFailed && ( + + + + {t('settings.sources.wiki.settings.saveError')} + + + )} + + + + + +
    + ); + }; + + return ( + { + if (!open) onClose(); + }} + title={t('settings.sources.wiki.settings.title')} + description={document.name} + > + {body()} + + ); +} diff --git a/frontend/src/settings/components/RetrievalOptions.tsx b/frontend/src/settings/components/RetrievalOptions.tsx index 054c850d..5a7d52d9 100644 --- a/frontend/src/settings/components/RetrievalOptions.tsx +++ b/frontend/src/settings/components/RetrievalOptions.tsx @@ -319,6 +319,9 @@ type RetrievalOptionsProps = { // the retrieval tester, where those knobs cannot affect the result and // showing them would imply they do. queryOnly?: boolean; + // The collapsible toggle's label, where "Advanced settings" alone would be + // unclear (the connect wizard). Defaults to the shared title. + title?: string; }; /** @@ -335,6 +338,7 @@ export default function RetrievalOptions({ graphRAGAvailable = false, availableModels = [], queryOnly = false, + title, }: RetrievalOptionsProps) { const { t } = useTranslation(); const [open, setOpen] = useState(false); @@ -859,6 +863,7 @@ export default function RetrievalOptions({ variant="link" size="sm" onClick={() => setOpen((o) => !o)} + aria-expanded={expanded} className="-ml-3 w-fit justify-start" > - {tr('title')} + {title ?? tr('title')}
    ([]); + + useEffect(() => { + if (!enabled) return; + let cancelled = false; + userService + .getConfig() + .then((response) => response.json()) + .then((config) => { + if (!cancelled) { + setGraphRAGAvailable(!!config?.graphrag_available); + setHybridAvailable(!!config?.hybrid_available); + } + }) + .catch(() => undefined); + return () => { + cancelled = true; + }; + }, [enabled]); + + // Models back the graphrag extraction-model picker; only fetched when the + // instance supports graphrag. + useEffect(() => { + if (!enabled || !graphRAGAvailable) return; + let cancelled = false; + modelService + .getModels(token) + .then((response) => (response.ok ? response.json() : null)) + .then((data) => { + if (!cancelled && data) + setAvailableModels(modelService.transformModels(data.models || [])); + }) + .catch(() => undefined); + return () => { + cancelled = true; + }; + }, [enabled, graphRAGAvailable, token]); + + return { graphRAGAvailable, hybridAvailable, availableModels }; +} diff --git a/frontend/src/settings/index.tsx b/frontend/src/settings/index.tsx index 8798edd4..0209f2d9 100644 --- a/frontend/src/settings/index.tsx +++ b/frontend/src/settings/index.tsx @@ -17,6 +17,7 @@ import { setSourceDocs, } from '../preferences/preferenceSlice'; import Analytics from './Analytics'; +import Connectors from './Connectors'; import CustomModels from './CustomModels'; import General from './General'; import Logs from './Logs'; @@ -87,8 +88,10 @@ export default function Settings() { } /> } /> + {/* Sources are called Knowledge now; old links keep working. */} + } /> } /> } /> + } /> } /> ); } + +/** `/settings/sources` from before the rename, query string kept. */ +function SourcesRedirect() { + const { search, hash } = useLocation(); + return ( + + ); +} diff --git a/frontend/src/settings/types/index.ts b/frontend/src/settings/types/index.ts index 072ed6d8..449b8cf2 100644 --- a/frontend/src/settings/types/index.ts +++ b/frontend/src/settings/types/index.ts @@ -172,6 +172,10 @@ export type UserToolType = { // Access level when shared via a team: 'viewer' (use) or 'editor' (edit // actions; secrets stay owner-only). Null/absent for tools the caller owns. team_access?: 'viewer' | 'editor' | null; + // The connection whose account or credentials the tool runs with. + connection_id?: string | null; + // Whether team members use the owner's account or their own. + credential_mode?: 'owner' | 'member'; // Caller's role and what it may do (`utils/accessUtils` `can`). access?: Access | null; allowed_actions?: string[]; @@ -205,6 +209,8 @@ export type UserToolType = { }; active: boolean; require_approval?: boolean; + // Read or write, set on tools that come from a connection. + access?: 'read' | 'write'; }[]; }; diff --git a/frontend/src/store.ts b/frontend/src/store.ts index 45841869..c2c987f7 100644 --- a/frontend/src/store.ts +++ b/frontend/src/store.ts @@ -1,6 +1,7 @@ import { configureStore } from '@reduxjs/toolkit'; import agentPreviewReducer from './agents/agentPreviewSlice'; +import connectorsReducer from './connectors/connectorsSlice'; import schedulesReducer from './agents/schedules/schedulesSlice'; import workflowPreviewReducer from './agents/workflow/workflowPreviewSlice'; import { @@ -75,6 +76,7 @@ const store = configureStore({ schedules: schedulesReducer, teams: teamsReducer, graphBuild: graphBuildReducer, + connectors: connectorsReducer, }, middleware: (getDefaultMiddleware) => getDefaultMiddleware().concat( diff --git a/frontend/src/teams/ShareToTeamModal.test.tsx b/frontend/src/teams/ShareToTeamModal.test.tsx index e2c4fb77..1ef86cd0 100644 --- a/frontend/src/teams/ShareToTeamModal.test.tsx +++ b/frontend/src/teams/ShareToTeamModal.test.tsx @@ -47,13 +47,27 @@ vi.mock('../api/services/teamsService', () => ({ }, })); +const setCredentialMode = vi.fn(); +vi.mock('../api/services/connectorsService', () => ({ + default: { + setCredentialMode: (...a: unknown[]) => setCredentialMode(...a), + }, +})); + +// The agent section loads the agent itself; its own tests cover it. +vi.mock('../agents/components/AgentUsesSection', () => ({ + default: ({ agentId }: { agentId: string }) => ( +
    {agentId}
    + ), +})); + // Mark formatted counts so a raw number in the UI shows up in a test. vi.mock('../utils/dateTimeUtils', async (importOriginal) => ({ ...(await importOriginal()), formatCount: (value: number) => `#${value}`, })); -import ShareToTeamModal from './ShareToTeamModal'; +import ShareToTeamModal, { type ShareCredentials } from './ShareToTeamModal'; Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); @@ -131,6 +145,13 @@ describe('ShareToTeamModal', () => { await flush(); }; + it('lists what the agent uses', async () => { + await render(); + expect( + body().querySelector('[data-testid="agent-uses"]')?.textContent, + ).toBe('a1'); + }); + describe('access settings', () => { it('shows a collapsed Access settings toggle to the owner', async () => { await render(); @@ -361,3 +382,231 @@ describe('ShareToTeamModal', () => { ); }); }); + +describe('ShareToTeamModal credentials', () => { + let container: HTMLDivElement; + let root: Root; + + const credentials = ( + overrides: Partial = {}, + ): ShareCredentials => ({ + toolId: 'tool-1', + connectorName: 'Linear', + account: 'lena@meridian.example', + mode: 'owner', + hasWrites: false, + ...overrides, + }); + + beforeEach(() => { + listResourceShares.mockReset().mockResolvedValue({ shares: [] }); + getResourceSettings.mockReset().mockResolvedValue(settingsResponse()); + listMembers.mockReset().mockResolvedValue({ members: [] }); + setCredentialMode.mockReset(); + container = document.createElement('div'); + document.body.appendChild(container); + root = createRoot(container); + }); + + afterEach(() => { + act(() => root.unmount()); + container.remove(); + document.body.innerHTML = ''; + }); + + const render = async (creds?: ShareCredentials) => { + act(() => { + root.render( + undefined} + />, + ); + }); + await flush(); + }; + + // A compact segmented choice, not a pair of picker tiles. + const toggle = (value: 'owner' | 'member') => + Array.from( + body().querySelectorAll( + '[data-slot="toggle-group-item"]', + ), + ).find((item) => + item.textContent?.startsWith(`settings.connectors.sharing.${value}Short`), + )!; + const picker = () => + body().querySelector('[role="combobox"]')!; + + it('shows nothing about accounts for a resource without a connection', async () => { + await render(); + expect(text()).not.toContain('settings.connectors.share.heading'); + }); + + it('has no agent resource list for a tool', async () => { + await render(credentials()); + expect(body().querySelector('[data-testid="agent-uses"]')).toBeNull(); + }); + + it('says whose account members use on a tool that only reads', async () => { + await render(credentials()); + expect(toggle('owner').textContent).toBe( + 'settings.connectors.sharing.ownerShort', + ); + expect(body().querySelector('[role="note"]')).toBeNull(); + expect(body().querySelector('[data-slot="option-card"]')).toBeNull(); + expect(toggle('owner').getAttribute('aria-checked')).toBe('true'); + expect(text()).toContain( + 'settings.connectors.share.ownerWarning(account=lena@meridian.example', + ); + }); + + it('asks only for the confirmation on a tool that can act, with no warning box', async () => { + await render(credentials({ hasWrites: true })); + expect(body().querySelector('[role="note"]')).toBeNull(); + expect(text()).not.toContain('settings.connectors.share.ownerWarning'); + expect(body().querySelector('#share-confirm-writes')).not.toBeNull(); + }); + + it('blocks sharing an owner-mode tool with writes until confirmed', async () => { + await render(credentials({ hasWrites: true })); + expect(picker().disabled).toBe(true); + const box = body().querySelector( + '#share-confirm-writes', + )!; + await act(async () => box.click()); + expect(picker().disabled).toBe(false); + }); + + it('saves a mode change and rolls back when the save fails', async () => { + setCredentialMode.mockResolvedValue({ success: false }); + await render(credentials({ hasWrites: true })); + await act(async () => toggle('member').click()); + await flush(); + expect(setCredentialMode).toHaveBeenCalledWith('tool-1', 'member', 'tok'); + expect(toggle('owner').getAttribute('aria-checked')).toBe('true'); + expect(text()).toContain('settings.connectors.share.saveFailed'); + }); + + it('locks the other mode when an admin forces one', async () => { + await render(credentials({ forcedMode: 'member' })); + expect(toggle('member').getAttribute('aria-checked')).toBe('true'); + expect(toggle('owner').disabled).toBe(true); + expect(text()).toContain('settings.connectors.share.forced'); + }); + + it('follows the mode once the connection loads after opening', async () => { + await render(); + await render(credentials({ hasWrites: true, mode: 'member' })); + expect(toggle('member').getAttribute('aria-checked')).toBe('true'); + // Member mode runs on each member's own account: nothing to confirm. + expect(body().querySelector('#share-confirm-writes')).toBeNull(); + await render(credentials({ hasWrites: true, forcedMode: 'owner' })); + expect(toggle('owner').getAttribute('aria-checked')).toBe('true'); + expect(toggle('member').disabled).toBe(true); + }); + + describe('access settings of a connected tool', () => { + const toolSettings = (values: Record = {}) => ({ + success: true, + resource_type: 'tool', + resource_id: 'tool-1', + settings: [ + { key: 'editors_can_change_credentials', default: true }, + { key: 'editors_can_share', default: false }, + { key: 'viewers_can_use_in_agents', default: true }, + ].map((s) => ({ ...s, value: values[s.key] ?? s.default })), + access: 'owner', + allowed_actions: OWNER_ACTIONS, + }); + const openSettings = () => + act(() => buttonByText('settings.teams.accessSettings.title')!.click()); + + beforeEach(() => { + getResourceSettings.mockResolvedValue(toolSettings()); + }); + + // The connection's secret is the owner's alone, so the switch would + // promise editors something the server refuses. + it('has no switch for editors to change credentials', async () => { + await render(credentials()); + openSettings(); + expect( + body().querySelector('#share-setting-editors_can_change_credentials'), + ).toBeNull(); + expect( + body().querySelector('#share-setting-viewers_can_use_in_agents'), + ).not.toBeNull(); + }); + + it('keeps the switch for a tool with no connection', async () => { + await render(); + openSettings(); + expect( + body().querySelector('#share-setting-editors_can_change_credentials'), + ).not.toBeNull(); + }); + + it('tells editors they cannot change credentials', async () => { + await render(credentials()); + const hint = body().querySelector('[data-testid="share-editor-hint"]'); + expect(hint?.textContent).toContain( + 'settings.teams.editorHint.toolNoCredentials', + ); + }); + + it("says viewers' runs use each member's own account in member mode", async () => { + await render(credentials({ mode: 'member' })); + openSettings(); + expect(text()).toContain( + 'settings.teams.accessSettings.tool.viewers_can_use_in_agents.descriptionMember', + ); + }); + + it("says viewers' runs use the owner's credentials in owner mode", async () => { + await render(credentials()); + openSettings(); + expect(text()).toContain( + 'settings.teams.accessSettings.tool.viewers_can_use_in_agents.description', + ); + expect(text()).not.toContain('descriptionMember'); + }); + }); + + describe('for an editor allowed to share', () => { + const shared = (overrides: Partial = {}) => + credentials({ account: '', readOnly: true, ...overrides }); + + it("shows whose account shares use but doesn't let them change it", async () => { + await render(shared()); + // The owner's account, not "Your account". + expect(toggle('owner').textContent).toBe( + 'settings.connectors.sharing.ownerShortShared', + ); + expect(toggle('owner').getAttribute('aria-checked')).toBe('true'); + expect(toggle('owner').disabled).toBe(true); + expect(toggle('member').disabled).toBe(true); + expect(text()).toContain('settings.connectors.share.ownerChooses'); + expect(text()).toContain( + 'settings.connectors.share.ownerWarningShared(name=Linear)', + ); + }); + + it('still asks for the write confirmation before adding people', async () => { + await render(shared({ hasWrites: true })); + expect(text()).toContain( + 'settings.connectors.share.confirmWriteShared(name=Linear)', + ); + expect(picker().disabled).toBe(true); + await act(async () => + body() + .querySelector('#share-confirm-writes')! + .click(), + ); + expect(picker().disabled).toBe(false); + }); + }); +}); diff --git a/frontend/src/teams/ShareToTeamModal.tsx b/frontend/src/teams/ShareToTeamModal.tsx index 6ead1a0f..7a87b7ab 100644 --- a/frontend/src/teams/ShareToTeamModal.tsx +++ b/frontend/src/teams/ShareToTeamModal.tsx @@ -4,6 +4,8 @@ import { ChevronRight, CircleAlert, Trash2, + UserRound, + UsersRound, } from 'lucide-react'; import { useEffect, useMemo, useRef, useState } from 'react'; import { useTranslation } from 'react-i18next'; @@ -17,8 +19,12 @@ import teamsService, { ResourceType, TeamMember, } from '../api/services/teamsService'; +import connectorsService from '../api/services/connectorsService'; +import AgentUsesSection from '../agents/components/AgentUsesSection'; import SearchInput from '../components/SearchInput'; import { Alert, AlertDescription } from '../components/ui/alert'; +import { Checkbox } from '../components/ui/checkbox'; +import { Label } from '../components/ui/label'; import { Avatar } from '../components/ui/avatar'; import { Button } from '../components/ui/button'; import { EmptyState } from '../components/ui/empty-state'; @@ -61,14 +67,39 @@ import { errorMessage, resolveSettings, settingCopy, + shownSettings, } from './accessSettings'; import { loadTeams, selectTeams } from './teamsSlice'; +/** A connection-backed tool: whose account shares of it run with. */ +export type ShareCredentials = { + toolId: string; + connectorName: string; + /** The connection's account; empty when the caller isn't its owner. */ + account: string; + mode: 'owner' | 'member'; + /** Set when an admin forces one mode for every share of this connector. */ + forcedMode?: 'owner' | 'member' | null; + /** Owner-mode shares of a tool with write actions need an explicit OK. */ + hasWrites: boolean; + /** + * An editor the owner lets share: the mode is the owner's choice, shown + * but locked, and the write confirmation still applies. + */ + readOnly?: boolean; +}; + type Props = { resourceType: ResourceType; resourceId: string; resourceName?: string; + credentials?: ShareCredentials; onClose: () => void; + /** + * An agent's: opens its Access details (the API write allowlist) from + * "What this agent uses". The caller closes this dialog for it. + */ + onOpenAccessDetails?: () => void; }; // Member subs (OIDC subs) can be long; there's no display-name endpoint, so we @@ -114,7 +145,9 @@ export default function ShareToTeamModal({ resourceType, resourceId, resourceName, + credentials, onClose, + onOpenAccessDetails, }: Props) { const { t } = useTranslation(); const dispatch = useDispatch(); @@ -131,6 +164,37 @@ export default function ShareToTeamModal({ const [loadError, setLoadError] = useState(false); const [actionError, setActionError] = useState(null); + const savedCredentialMode = + credentials?.forcedMode ?? credentials?.mode ?? 'owner'; + const [credentialMode, setCredentialMode] = useState<'owner' | 'member'>( + savedCredentialMode, + ); + // The caller may open the dialog before the tool's connection has loaded: + // follow the mode (or an admin's forced one) when it arrives or changes. + const [seenCredentialMode, setSeenCredentialMode] = + useState(savedCredentialMode); + if (savedCredentialMode !== seenCredentialMode) { + setSeenCredentialMode(savedCredentialMode); + setCredentialMode(savedCredentialMode); + } + const [writesConfirmed, setWritesConfirmed] = useState(false); + const needsWriteConfirm = + !!credentials && credentialMode === 'owner' && credentials.hasWrites; + const changeCredentialMode = (mode: 'owner' | 'member') => { + if (!credentials || credentials.readOnly || mode === credentialMode) return; + const previous = credentialMode; + setCredentialMode(mode); + connectorsService + .setCredentialMode(credentials.toolId, mode, token) + .then((data) => { + if (!data?.success) throw new Error('save failed'); + }) + .catch(() => { + setCredentialMode(previous); + setActionError(t('settings.connectors.share.saveFailed')); + }); + }; + // The owner's per-resource switches and the caller's own access, from // GET /api/resource_settings. Null until loaded (or when it fails: the hint // then uses the defaults and the settings group stays hidden). @@ -221,7 +285,12 @@ export default function ShareToTeamModal({ .getResourceSettings(resourceType, resourceId, token) .then((r) => { setSettingsInfo(r); - if (anyChanged(resolveSettings(resourceType, r?.settings))) { + const shown = shownSettings( + resourceType, + resolveSettings(resourceType, r?.settings), + !!credentials, + ); + if (anyChanged(shown)) { setSettingsOpen(true); } }) @@ -231,9 +300,16 @@ export default function ShareToTeamModal({ }); }, [resourceType, resourceId]); + // A connected tool hides "Editors can change credentials": its secret is + // the owner's connection, which editors never change. const settings = useMemo( - () => resolveSettings(resourceType, settingsInfo?.settings), - [resourceType, settingsInfo], + () => + shownSettings( + resourceType, + resolveSettings(resourceType, settingsInfo?.settings), + !!credentials, + ), + [resourceType, settingsInfo, credentials], ); const canManageSettings = can(settingsInfo, 'manage_settings'); @@ -758,7 +834,12 @@ export default function ShareToTeamModal({ {settingsOpen && ( {settings.map((setting) => { - const copy = settingCopy(t, resourceType, setting.key); + const copy = settingCopy( + t, + resourceType, + setting.key, + credentials ? credentialMode : undefined, + ); const id = `share-setting-${setting.key}`; return ( {t('settings.teams.share.noTeams')}

    ) : ( <> + {credentials && ( +
    + + {/* A compact one-of-two; the line under it says what the + choice means. */} +
    + + value && changeCredentialMode(value as 'owner' | 'member') + } + > + {(['owner', 'member'] as const).map((mode) => ( + + {mode === 'owner' ? : } + {/* "Your account" to the owner; an editor sees the + owner's, like the agent's "What this agent uses". */} + {t( + mode === 'owner' && credentials.readOnly + ? 'settings.connectors.sharing.ownerShortShared' + : `settings.connectors.sharing.${mode}Short`, + )} + + ))} + +
    + {credentials.forcedMode ? ( +

    + {t('settings.connectors.share.forced')} +

    + ) : ( + credentials.readOnly && ( +

    + {t('settings.connectors.share.ownerChooses')} +

    + ) + )} + {credentialMode === 'owner' ? ( + // A tool that can act asks for the confirmation below, which + // says the same; one that only reads gets a plain line. + needsWriteConfirm ? null : ( +

    + {credentials.readOnly + ? t('settings.connectors.share.ownerWarningShared', { + name: credentials.connectorName, + interpolation: { escapeValue: false }, + }) + : t('settings.connectors.share.ownerWarning', { + account: credentials.account, + name: credentials.connectorName, + interpolation: { escapeValue: false }, + })} +

    + ) + ) : ( +

    + {t('settings.connectors.share.memberNote', { + name: credentials.connectorName, + interpolation: { escapeValue: false }, + })} +

    + )} + {needsWriteConfirm && ( +
    + + setWritesConfirmed(checked === true) + } + /> + +
    + )} +
    + )} +
    {/* Add row: type-ahead combobox + access level select. */}
    @@ -804,7 +987,9 @@ export default function ShareToTeamModal({ variant="combobox" role="combobox" aria-expanded={pickerOpen} - disabled={committing} + disabled={ + committing || (needsWriteConfirm && !writesConfirmed) + } data-placeholder="" className="min-w-0 flex-1 justify-start" > @@ -915,7 +1100,7 @@ export default function ShareToTeamModal({ data-testid="share-editor-hint" className="text-muted-foreground mt-1.5 text-xs" > - {editorHint(t, resourceType, settings)} + {editorHint(t, resourceType, settings, !!credentials)}

    @@ -957,6 +1142,17 @@ export default function ShareToTeamModal({ )} + {/* Whose access each of the agent's tools, sources and prompt runs + with, for the people it is shared with (owners and editors only: + viewers never open this dialog). */} + {resourceType === 'agent' && ( + + )} + {accessSettings}
    ); diff --git a/frontend/src/teams/accessSettings.ts b/frontend/src/teams/accessSettings.ts index 46306127..393113e9 100644 --- a/frontend/src/teams/accessSettings.ts +++ b/frontend/src/teams/accessSettings.ts @@ -62,18 +62,34 @@ export const settingValue = ( key: string, ): boolean => settings.find((s) => s.key === key)?.value ?? false; +/** + * The switches the share dialog lists. A connected tool drops "Editors can + * change credentials": its secret is on the owner's connection, which only + * the owner can change whatever the switch says. + */ +export const shownSettings = ( + type: ResourceType, + settings: ResourceSetting[], + connected = false, +): ResourceSetting[] => + type === 'tool' && connected + ? settings.filter((s) => s.key !== 'editors_can_change_credentials') + : settings; + /** Whether any switch is off its default (opens Access settings by default). */ export const anyChanged = (settings: ResourceSetting[]): boolean => settings.some((s) => s.value !== s.default); /** * What an Editor can do on this resource, as one or two sentences: a base - * sentence per type, then whether they may also share or delete it. + * sentence per type, then whether they may also share or delete it. Editors + * never change a connected tool's credentials (see `shownSettings`). */ export function editorHint( t: TFunction, type: ResourceType, settings: ResourceSetting[], + connected = false, ): string { const on = (key: string) => settingValue(settings, key); let base: string; @@ -82,9 +98,10 @@ export function editorHint( ? t('settings.teams.editorHint.agent') : t('settings.teams.editorHint.agentNoAccessDetails'); } else if (type === 'tool') { - base = on('editors_can_change_credentials') - ? t('settings.teams.editorHint.tool') - : t('settings.teams.editorHint.toolNoCredentials'); + base = + !connected && on('editors_can_change_credentials') + ? t('settings.teams.editorHint.tool') + : t('settings.teams.editorHint.toolNoCredentials'); } else { base = t(`settings.teams.editorHint.${type}`); } @@ -111,19 +128,33 @@ export function editorHint( return `${base} ${tail}`; } -/** A switch's label and description in the share dialog. */ +/** + * A switch's label and description in the share dialog. `credentialMode` is + * a connected tool's: in member mode a viewer's agent runs with the viewer's + * own account, not the owner's credentials. + */ export const settingCopy = ( t: TFunction, type: ResourceType, key: string, -): { label: string; description: string } => ({ - label: t(`settings.teams.accessSettings.${type}.${key}.label`, { - defaultValue: key, - }), - description: t(`settings.teams.accessSettings.${type}.${key}.description`, { - defaultValue: '', - }), -}); + credentialMode?: 'owner' | 'member', +): { label: string; description: string } => { + const description = + type === 'tool' && + key === 'viewers_can_use_in_agents' && + credentialMode === 'member' + ? 'descriptionMember' + : 'description'; + return { + label: t(`settings.teams.accessSettings.${type}.${key}.label`, { + defaultValue: key, + }), + description: t( + `settings.teams.accessSettings.${type}.${key}.${description}`, + { defaultValue: '' }, + ), + }; +}; /** * The read-only "What people here can do" lines: the viewer and editor diff --git a/frontend/src/upload/Upload.test.tsx b/frontend/src/upload/Upload.test.tsx index dbe5daf3..abca6440 100644 --- a/frontend/src/upload/Upload.test.tsx +++ b/frontend/src/upload/Upload.test.tsx @@ -2,11 +2,42 @@ import { act } from 'react'; import { createRoot, type Root } from 'react-dom/client'; vi.mock('react-i18next', () => ({ - useTranslation: () => ({ t: (key: string) => key }), + useTranslation: () => ({ + t: (key: string, opts?: { services?: string; account?: string }) => + opts?.services + ? `${key}(${opts.services})` + : opts?.account + ? `${key}(${opts.account})` + : key, + i18n: { language: 'en' }, + }), +})); + +const connectorsState = vi.hoisted(() => ({ + catalog: [] as Record[], + connections: [] as Record[], + enabled: true, })); vi.mock('react-redux', () => ({ - useSelector: () => null, + useSelector: (selector: (state: unknown) => unknown) => { + const state = { + preference: { token: null, selectedDocs: [] }, + connectors: { + catalog: connectorsState.catalog, + connections: connectorsState.connections, + enabled: connectorsState.enabled, + loaded: true, + loading: false, + failed: false, + }, + }; + try { + return selector(state); + } catch { + return null; + } + }, useDispatch: () => vi.fn(), useStore: () => ({ getState: () => ({}) }), })); @@ -17,6 +48,11 @@ vi.mock('../api/services/userService', () => ({ }, })); +const launch = vi.hoisted(() => vi.fn()); +vi.mock('../connectors/useConnectorLauncher', () => ({ + default: () => ({ launch, modals: null }), +})); + import Upload from './Upload'; Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); @@ -36,7 +72,8 @@ describe('Upload source-type tiles', () => { container.remove(); }); - const render = async () => { + const close = vi.fn(); + const render = async (onBrowseConnectors?: () => void) => { await act(async () => { root.render( { setModalState={vi.fn()} isOnboarding={false} renderTab={null} - close={vi.fn()} + close={close} + onBrowseConnectors={onBrowseConnectors} />, ); }); @@ -84,6 +122,328 @@ describe('Upload source-type tiles', () => { ); }); + const DRIVE = { + key: 'google_drive', + name: 'Google Drive', + state: 'available', + icon: 'drive', + sync_ingestor: 'google_drive', + capabilities: ['sync'], + available: true, + missing_settings: [], + }; + const connectTile = () => + tiles().find((tile) => + tile.textContent?.includes('modals.uploadDoc.connectData.title'), + ); + + // One list of what needs no account, then one tile that sends the user to + // connect a service; connected services sync from the Connectors page. + it('lists the no-account types and one Connect your data tile', async () => { + connectorsState.catalog = [DRIVE]; + await render(); + const labels = tiles().map((tile) => tile.textContent ?? ''); + for (const type of ['local_file', 'url', 'crawler', 'github', 'wiki']) + expect(labels.some((l) => l.includes(`ingestors.${type}.label`))).toBe( + true, + ); + for (const type of [ + 'google_drive', + 'share_point', + 'confluence', + 's3', + 'reddit', + ]) + expect(labels.some((l) => l.includes(`ingestors.${type}.label`))).toBe( + false, + ); + expect(labels.at(-1)).toContain('modals.uploadDoc.connectData.title'); + // Named after what this instance can sync. + expect(labels.at(-1)).toContain( + 'modals.uploadDoc.connectData.description(Google Drive)', + ); + expect(document.body.querySelector('h3')).toBeNull(); + connectorsState.catalog = []; + }); + + // Like the Connectors page: connected services first, and a part (a + // service's sync half) listed under its parent. GitHub has its own tile. + const CATALOG = [ + DRIVE, + { ...DRIVE, key: 'share_point', name: 'SharePoint' }, + { ...DRIVE, key: 'github', name: 'GitHub', state: 'connected' }, + { ...DRIVE, key: 's3', name: 'Amazon S3', state: 'connected' }, + { + key: 'mcp:atlassian', + name: 'Atlassian', + capabilities: ['read', 'write'], + available: true, + state: 'available', + }, + { + ...DRIVE, + key: 'confluence', + name: 'Confluence', + part_of: 'mcp:atlassian', + }, + { ...DRIVE, key: 'off', name: 'Needs setup', available: false }, + { key: 'telegram', name: 'Telegram', capabilities: ['write'] }, + ]; + + it('names three syncing services, then the rest as more', async () => { + connectorsState.catalog = CATALOG; + await render(); + expect(connectTile()!.textContent).toContain( + 'modals.uploadDoc.connectData.description(Amazon S3, Google Drive, SharePoint, and modals.uploadDoc.connectData.more)', + ); + connectorsState.catalog = []; + }); + + const serviceTiles = () => + tiles().map( + (tile) => tile.querySelector('[data-slot="card-title"]')?.textContent, + ); + + it('lists the services that sync in place of the tiles, with a way back', async () => { + connectorsState.catalog = CATALOG; + close.mockClear(); + await render(); + await act(async () => connectTile()!.click()); + expect(close).not.toHaveBeenCalled(); + expect(serviceTiles()).toEqual([ + 'GitHub', + 'Amazon S3', + 'Google Drive', + 'SharePoint', + 'Atlassian', + ]); + const back = Array.from(document.body.querySelectorAll('button')).find( + (b) => b.textContent === 'modals.uploadDoc.back', + )!; + await act(async () => back.click()); + expect(connectTile()).toBeDefined(); + connectorsState.catalog = []; + }); + + it('connects a service in place, for Knowledge', async () => { + connectorsState.catalog = CATALOG; + connectorsState.connections = [ + { + id: 'k1', + connector_key: 's3', + status: 'connected', + account_label: 'bucket-reader', + }, + ]; + launch.mockClear(); + await render(); + await act(async () => connectTile()!.click()); + const tile = (name: string) => + tiles().find( + (t) => + t.querySelector('[data-slot="card-title"]')?.textContent === name, + )!; + expect(tile('Amazon S3').textContent).toContain( + 'modals.uploadDoc.connectData.connectedAs(bucket-reader)', + ); + // An account already connected goes straight to choosing what to sync. + await act(async () => tile('Amazon S3').click()); + expect(launch).toHaveBeenLastCalledWith( + expect.objectContaining({ key: 's3' }), + { mode: 'sync', connectionId: 'k1', purpose: 'knowledge' }, + ); + connectorsState.connections = []; + connectorsState.catalog = []; + }); + + it('connects the sync part of a service listed under its parent', async () => { + connectorsState.catalog = CATALOG; + launch.mockClear(); + await render(); + await act(async () => connectTile()!.click()); + const atlassian = tiles().find( + (t) => + t.querySelector('[data-slot="card-title"]')?.textContent === + 'Atlassian', + )!; + await act(async () => atlassian.click()); + expect(launch).toHaveBeenLastCalledWith( + expect.objectContaining({ key: 'confluence' }), + { purpose: 'knowledge' }, + ); + connectorsState.catalog = []; + }); + + it('browses all connectors only when the opener offers it', async () => { + connectorsState.catalog = CATALOG; + const browseLink = () => + Array.from(document.body.querySelectorAll('button')).find( + (b) => b.textContent === 'modals.uploadDoc.connectData.browseAll', + ); + await render(); + await act(async () => connectTile()!.click()); + expect(browseLink()).toBeUndefined(); + + const browse = vi.fn(); + close.mockClear(); + await render(browse); + expect(browseLink()).toBeDefined(); + await act(async () => browseLink()!.click()); + expect(close).toHaveBeenCalled(); + expect(browse).toHaveBeenCalled(); + connectorsState.catalog = []; + }); + + it('offers no Connect tile when no service can sync', async () => { + connectorsState.catalog = [ + { key: 'telegram', capabilities: ['write'], available: true }, + { ...DRIVE, available: false }, + ]; + await render(); + expect(connectTile()).toBeUndefined(); + connectorsState.catalog = []; + }); + + it('offers no Connect tile when connectors are off', async () => { + connectorsState.catalog = [DRIVE]; + connectorsState.enabled = false; + await render(); + expect(connectTile()).toBeUndefined(); + connectorsState.enabled = true; + connectorsState.catalog = []; + }); + + describe('GitHub', () => { + const githubConnector = { + key: 'github', + icon: 'github', + sync_ingestor: 'github', + auth_kind: 'api_key', + available: true, + missing_settings: [], + }; + const openGitHub = async () => { + await render(); + const tile = tiles().find((t) => + t.textContent?.includes('modals.uploadDoc.ingestors.github.label'), + )!; + await act(async () => tile.click()); + }; + const handOver = () => + Array.from(document.body.querySelectorAll('button')).find((b) => + b.textContent?.includes('modals.uploadDoc.github.'), + ); + + afterEach(() => { + connectorsState.catalog = []; + connectorsState.connections = []; + }); + + it('stays one public-repository tile, with a way to private ones', async () => { + launch.mockClear(); + connectorsState.catalog = [githubConnector]; + await render(); + const githubTiles = tiles().filter((t) => + t.textContent?.includes('modals.uploadDoc.ingestors.github.label'), + ); + expect(githubTiles).toHaveLength(1); + await openGitHub(); + // The repository URL form still works without an account. + expect(document.body.textContent).toContain( + 'modals.uploadDoc.ingestors.github.heading', + ); + expect(document.body.textContent).toContain( + 'modals.uploadDoc.github.privateHint', + ); + await act(async () => handOver()!.click()); + expect(launch).toHaveBeenCalledWith(githubConnector, { + purpose: 'knowledge', + }); + }); + + it('goes straight to picking a repository with a connected account', async () => { + launch.mockClear(); + connectorsState.catalog = [githubConnector]; + connectorsState.connections = [ + { + id: 'gh-1', + connector_key: 'github', + status: 'connected', + account_label: 'octocat', + }, + ]; + await openGitHub(); + expect(document.body.textContent).toContain( + 'modals.uploadDoc.github.connectedHint', + ); + await act(async () => handOver()!.click()); + expect(launch).toHaveBeenCalledWith(githubConnector, { + mode: 'sync', + connectionId: 'gh-1', + purpose: 'knowledge', + }); + }); + + it('offers no hand-over when GitHub connections are off', async () => { + connectorsState.catalog = []; + await openGitHub(); + expect(handOver()).toBeUndefined(); + }); + }); + + it('sends a crawler source to the remote ingest with its URL', async () => { + const sent: { url: string; body: FormData }[] = []; + class FakeXhr { + upload = { addEventListener() {} }; + url = ''; + addEventListener() {} + open(_method: string, url: string) { + this.url = url; + } + setRequestHeader() {} + send(body: FormData) { + sent.push({ url: this.url, body }); + } + } + vi.stubGlobal('XMLHttpRequest', FakeXhr); + await render(); + const crawler = tiles().find((tile) => + tile.textContent?.includes('modals.uploadDoc.ingestors.crawler.label'), + )!; + await act(async () => crawler.click()); + const type = async (input: HTMLInputElement, value: string) => { + const setter = Object.getOwnPropertyDescriptor( + HTMLInputElement.prototype, + 'value', + )!.set!; + await act(async () => { + setter.call(input, value); + input.dispatchEvent(new Event('input', { bubbles: true })); + }); + }; + const inputs = Array.from( + document.body.querySelectorAll('input[type="text"]'), + ); + // The source's name first, then the crawler's URL field. + await type(inputs[0], 'Docs site'); + await type( + document.body.querySelector('input[name="url"]')!, + 'https://docs.example', + ); + const train = Array.from(document.body.querySelectorAll('button')).find( + (b) => b.textContent === 'modals.uploadDoc.train', + )!; + expect(train.disabled).toBe(false); + await act(async () => train.click()); + expect(sent).toHaveLength(1); + expect(sent[0].url).toContain('/api/remote'); + expect(sent[0].body.get('source')).toBe('crawler'); + expect(JSON.parse(String(sent[0].body.get('data')))).toEqual({ + url: 'https://docs.example', + }); + vi.unstubAllGlobals(); + }); + it('leaves the disabled Train button on the default variant', async () => { await render(); const crawler = tiles().find((tile) => diff --git a/frontend/src/upload/Upload.tsx b/frontend/src/upload/Upload.tsx index 1ebbcf5d..e8c070da 100644 --- a/frontend/src/upload/Upload.tsx +++ b/frontend/src/upload/Upload.tsx @@ -1,6 +1,5 @@ -import { ChevronLeft, FileText } from 'lucide-react'; +import { ArrowRight, ChevronLeft, FileText, Lock, Plug } from 'lucide-react'; import { envVar } from '@/env'; -import { cn } from '@/lib/utils'; import { useCallback, useEffect, useState } from 'react'; import { nanoid } from '@reduxjs/toolkit'; import type { FileRejection } from 'react-dropzone'; @@ -9,25 +8,26 @@ import { useDispatch, useSelector, useStore } from 'react-redux'; import type { RootState } from '../store'; import userService from '../api/services/userService'; -import modelService from '../api/services/modelService'; -import type { Model } from '../models/types'; -import { getSessionToken } from '../utils/providerUtils'; +import { Alert, AlertDescription } from '../components/ui/alert'; import { Button } from '../components/ui/button'; import { Input } from '../components/ui/input'; import { FormField as UiFormField } from '../components/ui/form-field'; -import { Label } from '../components/ui/label'; -import { - Select, - SelectContent, - SelectItem, - SelectTrigger, - SelectValue, -} from '../components/ui/select'; -import { Switch } from '../components/ui/switch'; import { Textarea } from '../components/ui/textarea'; import { Modal } from '../components/ui/modal'; -import { Separator } from '../components/ui/separator'; import { OptionCard } from '../components/ui/option-card'; +import ConnectorIcon from '../connectors/ConnectorIcon'; +import { syncTargets } from '../connectors/catalogCards'; +import useConnectorLauncher from '../connectors/useConnectorLauncher'; +import { connectorName } from '../connectors/i18n'; +import { formatCount, intlLocale } from '../utils/dateTimeUtils'; +import { + loadConnectors, + selectConnections, + selectConnectorCatalog, + selectConnectorsEnabled, + selectConnectorsLoaded, +} from '../connectors/connectorsSlice'; +import type { AppDispatch } from '../store'; import { Card } from '../components/ui/card'; import { Dropzone } from '../components/ui/dropzone'; import { ListRow, ListRows } from '../components/ui/list-row'; @@ -46,13 +46,12 @@ import { IngestorFormSchemas, getIngestorSchema, IngestorOption, + UPLOAD_AND_WEB_INGESTORS, } from '../upload/types/ingestor'; import { addUploadTask, updateUploadTask } from './uploadSlice'; import { FormField, IngestorConfig, IngestorType } from './types/ingestor'; -import { FilePicker } from '../components/FilePicker'; -import GoogleDrivePicker from '../components/GoogleDrivePicker'; import { FILE_UPLOAD_ACCEPT } from '../constants/fileUpload'; import RetrievalOptions, { DEFAULT_RETRIEVAL_OPTIONS, @@ -60,6 +59,7 @@ import RetrievalOptions, { optionsToConfig, type RetrievalOptionsValue, } from '../settings/components/RetrievalOptions'; +import useRetrievalAvailability from '../settings/components/useRetrievalAvailability'; /** Per-file limit for local uploads (25 MB), enforced by the dropzone. */ const MAX_UPLOAD_BYTES = 25000000; @@ -72,6 +72,7 @@ function Upload({ close, onSuccessfulUpload = () => undefined, selectUploadedDoc = true, + onBrowseConnectors, }: { receivedFile: File[]; setModalState: (state: ActiveState) => void; @@ -86,93 +87,46 @@ function Upload({ * uploading never repoints the conversation the user left open. */ selectUploadedDoc?: boolean; + /** + * Opens the Connectors page on the services that sync. Only a caller that + * loses nothing by leaving (the Knowledge page) passes it; the dialog + * closes first. + */ + onBrowseConnectors?: () => void; }) { const token = useSelector(selectToken); const selectedDocs = useSelector(selectSelectedDocs); + const connectorCatalog = useSelector(selectConnectorCatalog); + const connections = useSelector(selectConnections); + const connectorsLoaded = useSelector(selectConnectorsLoaded); + const connectorsEnabled = useSelector(selectConnectorsEnabled); + // Connecting a service (Connect your data, GitHub's private-repository + // hand-over) goes to the connect wizard, the one flow every entry point + // uses; this modal steps aside and closes with it. + const [handedOver, setHandedOver] = useState(false); + // Connect your data: the services that sync, in place of the tiles. + const [connecting, setConnecting] = useState(false); + const { launch, modals: connectModals } = useConnectorLauncher({ + onConnected: () => close(), + }); const [files, setfiles] = useState(receivedFile); // Names of the files the last drop turned away (over the size limit or of // an unaccepted type), shown under the dropzone. const [rejectedFiles, setRejectedFiles] = useState([]); const [activeTab, setActiveTab] = useState(true); - const [showAdvancedOptions, setShowAdvancedOptions] = useState(false); const [retrievalOptions, setRetrievalOptions] = useState(DEFAULT_RETRIEVAL_OPTIONS); - const [graphRAGAvailable, setGraphRAGAvailable] = useState(false); - const [hybridAvailable, setHybridAvailable] = useState(false); - const [availableModels, setAvailableModels] = useState([]); - - // File picker state - const [selectedFiles, setSelectedFiles] = useState([]); - const [selectedFolders, setSelectedFolders] = useState([]); - - useEffect(() => { - let cancelled = false; - userService - .getConfig() - .then((response) => response.json()) - .then((config) => { - if (!cancelled) { - setGraphRAGAvailable(!!config?.graphrag_available); - setHybridAvailable(!!config?.hybrid_available); - } - }) - .catch(() => undefined); - return () => { - cancelled = true; - }; - }, []); - - // Models back the graphrag extraction-model picker; only fetched when the - // instance supports graphrag. - useEffect(() => { - if (!graphRAGAvailable) return; - let cancelled = false; - modelService - .getModels(token) - .then((response) => (response.ok ? response.json() : null)) - .then((data) => { - if (!cancelled && data) - setAvailableModels(modelService.transformModels(data.models || [])); - }) - .catch(() => undefined); - return () => { - cancelled = true; - }; - }, [graphRAGAvailable, token]); + const { graphRAGAvailable, hybridAvailable, availableModels } = + useRetrievalAvailability(token); const renderFormFields = () => { if (!ingestor.type) return null; const ingestorSchema = getIngestorSchema(ingestor.type as IngestorType); if (!ingestorSchema) return null; - const schema: FormField[] = ingestorSchema.fields; - - const generalFields = schema.filter((field: FormField) => !field.advanced); - const advancedFields = schema.filter((field: FormField) => field.advanced); - return (
    -
    - {generalFields.map((field: FormField) => renderField(field))} -
    - - {advancedFields.length > 0 && ( -
    -
    - -
    - {advancedFields.map((field: FormField) => renderField(field))} -
    -
    -
    - )} + {ingestorSchema.fields.map((field: FormField) => renderField(field))}
    ); }; @@ -200,81 +154,6 @@ function Upload({ required={isRequired} /> ); - case 'number': - return ( - - handleIngestorChange( - field.name as keyof IngestorConfig['config'], - Number(e.target.value), - ) - } - required={isRequired} - /> - ); - case 'enum': { - const currentValue = String( - ingestor.config[field.name as keyof typeof ingestor.config] ?? '', - ); - return ( - - - - ); - } - case 'boolean': - return ( -
    - - { - handleIngestorChange( - field.name as keyof IngestorConfig['config'], - checked, - ); - }} - /> -
    - ); case 'textarea': return ( ); - case 'remote_file_picker': - return ( - { - setSelectedFiles(selectedFileIds); - setSelectedFolders(selectedFolderIds); - }} - provider={ingestor.type as unknown as string} - token={token} - initialSelectedFiles={selectedFiles} - initialSelectedFolders={selectedFolders} - /> - ); - case 'google_drive_picker': - return ( - { - setSelectedFiles(selectedFileIds); - setSelectedFolders(selectedFolderIds); - }} - token={token} - /> - ); - case 'share_point_picker': - return ( - { - setSelectedFiles(selectedFileIds); - setSelectedFolders(selectedFolderIds); - }} - provider="share_point" - token={token} - initialSelectedFiles={selectedFiles} - initialSelectedFolders={selectedFolders} - /> - ); - case 'confluence_picker': - return ( - { - setSelectedFiles(selectedFileIds); - setSelectedFolders(selectedFolderIds); - }} - provider="confluence" - token={token} - initialSelectedFiles={selectedFiles} - initialSelectedFolders={selectedFolders} - /> - ); default: return null; } @@ -422,26 +236,27 @@ function Upload({ })); const [nameTouched, setNameTouched] = useState(false); - const { t } = useTranslation(); - const dispatch = useDispatch(); + const { t, i18n } = useTranslation(); + const dispatch = useDispatch(); const store = useStore(); - const ingestorOptions: IngestorOption[] = IngestorFormSchemas.filter( - (schema) => (schema.validate ? schema.validate() : true), - ).map((schema) => ({ - label: schema.label, - value: schema.key, - icon: schema.icon, - heading: schema.heading, - })); + useEffect(() => { + if (!connectorsLoaded) dispatch(loadConnectors({ token })); + }, [connectorsLoaded, dispatch, token]); + + const ingestorOptions: IngestorOption[] = IngestorFormSchemas.map( + (schema) => ({ + label: schema.label, + value: schema.key, + icon: schema.icon, + heading: schema.heading, + }), + ); const resetUploaderState = useCallback(() => { setIngestor({ type: null, name: '', config: {} }); setfiles([]); setRejectedFiles([]); - setSelectedFiles([]); - setSelectedFolders([]); - setShowAdvancedOptions(false); setRetrievalOptions(DEFAULT_RETRIEVAL_OPTIONS); setNameTouched(false); }, []); @@ -722,57 +537,11 @@ function Upload({ JSON.stringify(optionsToConfig(retrievalOptions)), ); - const ingestorSchema = getIngestorSchema(ingestor.type as IngestorType); - if (!ingestorSchema) { - handleTaskFailure(clientTaskId); - return; - } - - const schema: FormField[] = ingestorSchema.fields; - const hasLocalFilePicker = schema.some( - (field: FormField) => field.type === 'local_file_picker', - ); - const hasRemoteFilePicker = schema.some( - (field: FormField) => field.type === 'remote_file_picker', - ); - const hasGoogleDrivePicker = schema.some( - (field: FormField) => field.type === 'google_drive_picker', - ); - const hasSharePointPicker = schema.some( - (field: FormField) => field.type === 'share_point_picker', - ); - const hasConfluencePicker = schema.some( - (field: FormField) => field.type === 'confluence_picker', - ); - - let configData: Record = { ...ingestor.config }; - - if (hasLocalFilePicker) { - files.forEach((file) => { - formData.append('file', file); - }); - } else if ( - hasRemoteFilePicker || - hasGoogleDrivePicker || - hasSharePointPicker || - hasConfluencePicker - ) { - const sessionToken = getSessionToken(ingestor.type as string); - configData = { - provider: ingestor.type as string, - session_token: sessionToken, - file_ids: selectedFiles, - folder_ids: selectedFolders, - }; - } - - formData.append('data', JSON.stringify(configData)); + formData.append('data', JSON.stringify(ingestor.config)); const apiHost: string = envVar('VITE_API_HOST'); - const endpoint = - ingestor.type === 'local_file' - ? `${apiHost}/api/upload` - : `${apiHost}/api/remote`; + // Local files go through uploadFile; everything else is fetched remotely. + const endpoint = `${apiHost}/api/remote`; const xhr = new XMLHttpRequest(); @@ -934,70 +703,14 @@ function Upload({ if (!isPrescreenConfigValid(retrievalOptions)) return true; if (!ingestor.type) return true; - const ingestorSchemaForValidation = getIngestorSchema( - ingestor.type as IngestorType, - ); - if (!ingestorSchemaForValidation) return true; - const schema: FormField[] = ingestorSchemaForValidation.fields; - const hasLocalFilePicker = schema.some( - (field: FormField) => field.type === 'local_file_picker', - ); - const hasRemoteFilePicker = schema.some( - (field: FormField) => field.type === 'remote_file_picker', - ); - const hasGoogleDrivePicker = schema.some( - (field: FormField) => field.type === 'google_drive_picker', - ); - const hasSharePointPicker = schema.some( - (field: FormField) => field.type === 'share_point_picker', - ); - const hasConfluencePicker = schema.some( - (field: FormField) => field.type === 'confluence_picker', - ); - - if (hasLocalFilePicker) { - if (files.length === 0) { - return true; - } - } else if ( - hasRemoteFilePicker || - hasGoogleDrivePicker || - hasSharePointPicker || - hasConfluencePicker - ) { - if (selectedFiles.length === 0 && selectedFolders.length === 0) { - return true; - } - } - - const ingestorSchemaForFields = getIngestorSchema( - ingestor.type as IngestorType, - ); - if (!ingestorSchemaForFields) return false; - const formFields: FormField[] = ingestorSchemaForFields.fields; - for (const field of formFields) { - if (field.required) { - // Validate only required fields - const value = - ingestor.config[field.name as keyof typeof ingestor.config]; - - if (typeof value === 'string' && !value.trim()) { - return true; - } - - if ( - typeof value === 'number' && - (value === null || value === undefined || value <= 0) - ) { - return true; - } - - if (typeof value === 'boolean' && value === undefined) { - return true; - } - } - } - return false; + const schema = getIngestorSchema(ingestor.type); + if (!schema) return true; + return schema.fields.some((field: FormField) => { + if (field.type === 'local_file_picker') return files.length === 0; + if (!field.required) return false; + const value = ingestor.config[field.name]; + return typeof value === 'string' && !value.trim(); + }); }; const handleIngestorChange = ( key: keyof IngestorConfig['config'], @@ -1039,28 +752,195 @@ function Upload({ } }; - const renderIngestorSelection = () => { - return ( + // Services that sync into Knowledge are connected here, through the + // connect wizard: one tile opens their list, named after the first few + // (GitHub has a tile of its own). + const syncServices = connectorsEnabled ? syncTargets(connectorCatalog) : []; + const namedServices = syncServices + .filter(({ card }) => card.key !== 'github') + .map(({ card }) => connectorName(t, card)); + const SERVICES_NAMED = 3; + const connectDescription = t('modals.uploadDoc.connectData.description', { + services: new Intl.ListFormat(intlLocale(i18n.language), { + type: 'conjunction', + }).format( + namedServices.length > SERVICES_NAMED + ? [ + ...namedServices.slice(0, SERVICES_NAMED), + t('modals.uploadDoc.connectData.more'), + ] + : namedServices, + ), + interpolation: { escapeValue: false }, + }); + + const connectedAccounts = (key: string) => + connections.filter( + (c) => c.connector_key === key && c.status === 'connected', + ); + + /** Which account a service's tile syncs from, when it has any. */ + const accountLine = (key: string) => { + const accounts = connectedAccounts(key); + if (accounts.length === 1) + return t('modals.uploadDoc.connectData.connectedAs', { + account: accounts[0].account_label, + interpolation: { escapeValue: false }, + }); + if (accounts.length > 1) + return t('settings.connectors.status.connectedCount', { + count: accounts.length, + formatted: formatCount(accounts.length), + }); + return undefined; + }; + + const renderConnectStep = () => ( +
    + +
    +

    + {t('modals.uploadDoc.connectData.title')} +

    +

    + {t('modals.uploadDoc.connectData.intro')} +

    +
    - {ingestorOptions.map((option) => ( + {syncServices.map(({ card, target }) => ( - } - title={t(`modals.uploadDoc.ingestors.${option.value}.label`)} - onClick={() => - handleIngestorTypeChange(option.value as IngestorType) - } + key={card.key} + icon={} + title={connectorName(t, card)} + description={accountLine(target.key)} + onClick={() => { + const account = connectedAccounts(target.key)[0]; + setHandedOver(true); + // Opened to add knowledge: syncing starts switched on, and an + // account already connected goes straight to what to sync. + launch( + target, + account + ? { + mode: 'sync', + connectionId: account.id, + purpose: 'knowledge', + } + : { purpose: 'knowledge' }, + ); + }} /> ))}
    + {onBrowseConnectors && ( + + )} +
    + ); + + const renderIngestorSelection = () => { + const options = UPLOAD_AND_WEB_INGESTORS.map((type) => + ingestorOptions.find((o) => o.value === type), + ).filter((option): option is IngestorOption => !!option); + return ( +
    + {options.map((option) => ( + + } + title={t(`modals.uploadDoc.ingestors.${option.value}.label`)} + onClick={() => handleIngestorTypeChange(option.value)} + /> + ))} + {syncServices.length > 0 && ( + } + title={t('modals.uploadDoc.connectData.title')} + description={connectDescription} + onClick={() => setConnecting(true)} + /> + )} +
    ); }; + + // The GitHub tile reads public repositories by URL with no account. A + // private one needs the user's own GitHub connection: hand over to the + // connect wizard, which picks from the account's repositories. + const githubConnector = connectorsEnabled + ? connectorCatalog.find((c) => c.key === 'github' && c.available) + : undefined; + const githubAccount = githubConnector + ? connections.find( + (c) => c.connector_key === 'github' && c.status === 'connected', + ) + : undefined; + const renderGitHubHandOver = () => + githubConnector ? ( + + + + {githubAccount + ? t('modals.uploadDoc.github.connectedHint', { + account: githubAccount.account_label, + interpolation: { escapeValue: false }, + }) + : t('modals.uploadDoc.github.privateHint')} + +
    + +
    +     + ) : null; + + if (handedOver) return <>{connectModals}; + return (
    - {!ingestor.type && ( + {!ingestor.type && !connecting && (

    {t('modals.uploadDoc.selectSource')}

    @@ -1094,7 +974,8 @@ function Upload({ {activeTab && ( <> - {!ingestor.type && renderIngestorSelection()} + {!ingestor.type && + (connecting ? renderConnectStep() : renderIngestorSelection())} {ingestor.type && (
    - )} )}
    diff --git a/frontend/src/upload/types/ingestor.ts b/frontend/src/upload/types/ingestor.ts index 1c3d1eff..07031c35 100644 --- a/frontend/src/upload/types/ingestor.ts +++ b/frontend/src/upload/types/ingestor.ts @@ -1,26 +1,10 @@ -import { envVar } from '@/env'; import CrawlerIcon from '../../assets/crawler.svg'; import FileUploadIcon from '../../assets/file_upload.svg'; import UrlIcon from '../../assets/url.svg'; import GithubIcon from '../../assets/github.svg'; -import RedditIcon from '../../assets/reddit.svg'; -import DriveIcon from '../../assets/drive.svg'; -import S3Icon from '../../assets/s3.svg'; -import SharePoint from '../../assets/sharepoint.svg'; -import ConfluenceIcon from '../../assets/confluence.svg'; import BookIcon from '../../assets/book-mono.svg'; -export type IngestorType = - | 'confluence' - | 'crawler' - | 'github' - | 'reddit' - | 'url' - | 'google_drive' - | 'local_file' - | 's3' - | 'share_point' - | 'wiki'; +export type IngestorType = 'crawler' | 'github' | 'url' | 'local_file' | 'wiki'; export interface IngestorConfig { type: IngestorType | null; @@ -35,17 +19,7 @@ export type IngestorFormData = { data: string; }; -export type FieldType = - | 'string' - | 'number' - | 'enum' - | 'boolean' - | 'textarea' - | 'local_file_picker' - | 'remote_file_picker' - | 'google_drive_picker' - | 'share_point_picker' - | 'confluence_picker'; +export type FieldType = 'string' | 'textarea' | 'local_file_picker'; export interface FormField { name: string; @@ -54,8 +28,6 @@ export interface FormField { labelKey?: string; type: FieldType; required?: boolean; - advanced?: boolean; - options?: { label: string; value: string }[]; } export interface IngestorSchema { @@ -63,10 +35,21 @@ export interface IngestorSchema { label: string; icon: string; heading: string; - validate?: () => boolean; fields: FormField[]; } +/** + * Add knowledge tiles: the source types that need no account. Services + * that sync into Knowledge are connected through the connect wizard. + */ +export const UPLOAD_AND_WEB_INGESTORS: IngestorType[] = [ + 'local_file', + 'url', + 'crawler', + 'github', + 'wiki', +]; + export const IngestorFormSchemas: IngestorSchema[] = [ { key: 'local_file', @@ -127,153 +110,6 @@ export const IngestorFormSchemas: IngestorSchema[] = [ }, ], }, - { - key: 'reddit', - label: 'Reddit', - icon: RedditIcon, - heading: 'Add content from Reddit', - fields: [ - { - name: 'client_id', - label: 'Client ID', - labelKey: 'modals.uploadDoc.reddit.id', - type: 'string', - required: true, - }, - { - name: 'client_secret', - label: 'Client Secret', - labelKey: 'modals.uploadDoc.reddit.secret', - type: 'string', - required: true, - }, - { - name: 'user_agent', - label: 'User Agent', - labelKey: 'modals.uploadDoc.reddit.agent', - type: 'string', - required: true, - }, - { - name: 'search_queries', - label: 'Search Queries', - labelKey: 'modals.uploadDoc.reddit.searchQueries', - type: 'string', - required: true, - }, - { - name: 'number_posts', - label: 'Number of Posts', - labelKey: 'modals.uploadDoc.reddit.numberOfPosts', - type: 'number', - required: true, - }, - ], - }, - { - key: 'google_drive', - label: 'Google Drive', - icon: DriveIcon, - heading: 'Upload from Google Drive', - validate: () => { - const googleClientId = envVar('VITE_GOOGLE_CLIENT_ID'); - return !!googleClientId; - }, - fields: [ - { - name: 'files', - label: 'Select Files from Google Drive', - type: 'google_drive_picker', - required: true, - }, - ], - }, - { - key: 's3', - label: 'Amazon S3', - icon: S3Icon, - heading: 'Add content from Amazon S3', - fields: [ - { - name: 'aws_access_key_id', - label: 'AWS Access Key ID', - labelKey: 'modals.uploadDoc.fields.awsAccessKeyId', - type: 'string', - required: true, - }, - { - name: 'aws_secret_access_key', - label: 'AWS Secret Access Key', - labelKey: 'modals.uploadDoc.fields.awsSecretAccessKey', - type: 'string', - required: true, - }, - { - name: 'bucket', - label: 'Bucket Name', - labelKey: 'modals.uploadDoc.fields.bucket', - type: 'string', - required: true, - }, - { - name: 'prefix', - label: 'Path Prefix (optional)', - labelKey: 'modals.uploadDoc.fields.prefix', - type: 'string', - required: false, - }, - { - name: 'region', - label: 'AWS Region', - labelKey: 'modals.uploadDoc.fields.region', - type: 'string', - required: false, - }, - { - name: 'endpoint_url', - label: 'Custom Endpoint URL (optional)', - labelKey: 'modals.uploadDoc.fields.endpointUrl', - type: 'string', - required: false, - }, - ], - }, - { - key: 'share_point', - label: 'Share Point', - icon: SharePoint, - heading: 'Upload from Share Point', - validate: () => { - const sharePointClientId = envVar('VITE_SHARE_POINT_CLIENT_ID'); - return !!sharePointClientId; - }, - fields: [ - { - name: 'files', - label: 'Select Files from Share Point', - type: 'share_point_picker', - required: true, - }, - ], - }, - { - key: 'confluence', - label: 'Confluence', - icon: ConfluenceIcon, - heading: 'Upload from Confluence', - validate: () => { - const confluenceClientId = envVar('VITE_CONFLUENCE_CLIENT_ID'); - return !!confluenceClientId; - }, - fields: [ - { - name: 'files', - label: 'Select Pages from Confluence', - type: 'confluence_picker', - required: true, - }, - ], - }, { key: 'wiki', label: 'New wiki', @@ -297,52 +133,8 @@ export const IngestorDefaultConfigs: Record< > = { crawler: { name: '', config: { url: '' } }, url: { name: '', config: { url: '' } }, - reddit: { - name: '', - config: { - client_id: '', - client_secret: '', - user_agent: '', - search_queries: '', - number_posts: 10, - }, - }, github: { name: '', config: { repo_url: '' } }, - google_drive: { - name: '', - config: { - file_ids: '', - folder_ids: '', - recursive: true, - }, - }, local_file: { name: '', config: { files: [] } }, - s3: { - name: '', - config: { - aws_access_key_id: '', - aws_secret_access_key: '', - bucket: '', - prefix: '', - region: 'us-east-1', - endpoint_url: '', - }, - }, - share_point: { - name: '', - config: { - file_ids: '', - folder_ids: '', - recursive: true, - }, - }, - confluence: { - name: '', - config: { - file_ids: '', - folder_ids: '', - }, - }, wiki: { name: '', config: { diff --git a/frontend/src/utils/providerUtils.test.ts b/frontend/src/utils/providerUtils.test.ts new file mode 100644 index 00000000..b13ddabf --- /dev/null +++ b/frontend/src/utils/providerUtils.test.ts @@ -0,0 +1,39 @@ +const claim = vi.fn(); +vi.mock('../api/services/connectorsService', () => ({ + default: { claim: (...args: unknown[]) => claim(...args) }, +})); + +import { claimLegacySessionTokens } from './providerUtils'; + +describe('claimLegacySessionTokens', () => { + beforeEach(() => { + claim.mockReset(); + localStorage.clear(); + }); + + it('claims every stored session token once and removes it', async () => { + localStorage.setItem('google_drive_session_token', 'g-token'); + localStorage.setItem('confluence_session_token', 'c-token'); + claim.mockResolvedValue({ success: true, connection_id: 'x' }); + + await claimLegacySessionTokens('jwt'); + + expect(claim).toHaveBeenCalledWith('google_drive', 'g-token', 'jwt'); + expect(claim).toHaveBeenCalledWith('confluence', 'c-token', 'jwt'); + expect(claim).toHaveBeenCalledTimes(2); + expect(localStorage.getItem('google_drive_session_token')).toBeNull(); + expect(localStorage.getItem('confluence_session_token')).toBeNull(); + }); + + it('drops the token even when the claim fails', async () => { + localStorage.setItem('share_point_session_token', 's-token'); + claim.mockRejectedValue(new Error('offline')); + await claimLegacySessionTokens(null); + expect(localStorage.getItem('share_point_session_token')).toBeNull(); + }); + + it('does nothing without stored tokens', async () => { + await claimLegacySessionTokens(null); + expect(claim).not.toHaveBeenCalled(); + }); +}); diff --git a/frontend/src/utils/providerUtils.ts b/frontend/src/utils/providerUtils.ts index 837df1c7..da4625a5 100644 --- a/frontend/src/utils/providerUtils.ts +++ b/frontend/src/utils/providerUtils.ts @@ -1,25 +1,34 @@ /** - * Utility functions for managing session tokens for different cloud service providers. - * Follows the convention: {provider}_session_token + * Connector sign-ins used to keep a random session handle per provider in + * localStorage (``_session_token``). Credentials now live only on + * the server and the browser refers to a connection by its id, so the old + * handles are claimed once (linked to the signed-in user's connection) and + * removed. */ -import userService from '../api/services/userService'; +import connectorsService from '../api/services/connectorsService'; -export const getSessionToken = (provider: string): string | null => { - return localStorage.getItem(`${provider}_session_token`); -}; +const LEGACY_PROVIDERS = ['google_drive', 'share_point', 'confluence']; -export const setSessionToken = (provider: string, token: string): void => { - localStorage.setItem(`${provider}_session_token`, token); -}; +const legacyKey = (provider: string) => `${provider}_session_token`; -export const removeSessionToken = (provider: string): void => { - localStorage.removeItem(`${provider}_session_token`); -}; - -export const validateProviderSession = async ( +export const claimLegacySessionTokens = async ( token: string | null, - provider: string, -) => { - return await userService.validateConnectorSession(provider, token); +): Promise => { + for (const provider of LEGACY_PROVIDERS) { + let value: string | null = null; + try { + value = localStorage.getItem(legacyKey(provider)); + } catch { + return; + } + if (!value) continue; + try { + await connectorsService.claim(provider, value, token); + } catch { + // The handle is useless to this frontend either way; drop it. + } finally { + localStorage.removeItem(legacyKey(provider)); + } + } }; diff --git a/frontend/src/utils/streamingStatusUtils.test.ts b/frontend/src/utils/streamingStatusUtils.test.ts index 1b30823c..5973bdc2 100644 --- a/frontend/src/utils/streamingStatusUtils.test.ts +++ b/frontend/src/utils/streamingStatusUtils.test.ts @@ -125,4 +125,64 @@ describe('getToolChipLabel', () => { ), ).toBe('conversation.toolChip.usingTool|Mcp Tool'); }); + it('names a connection-backed call after its service', () => { + const notion = { tool_name: 'mcp_tool', connector_name: 'Notion' }; + expect( + getToolChipLabel( + call({ ...notion, action_name: 'notion-search', access: 'read' }), + t, + ), + ).toBe('conversation.toolChip.searchedConnector|Notion'); + expect( + getToolChipLabel( + call({ + ...notion, + action_name: 'notion-fetch', + access: 'read', + status: 'pending', + }), + t, + ), + ).toBe('conversation.streamingStatus.usingConnector|Notion'); + expect( + getToolChipLabel( + call({ ...notion, action_name: 'notion-fetch', access: 'read' }), + t, + ), + ).toBe('conversation.toolChip.readConnector|Notion'); + }); + + it('names the action of a write without the service prefix', () => { + expect( + getToolChipLabel( + call({ + tool_name: 'mcp_tool', + connector_name: 'Linear', + action_name: 'linear_create_issue', + access: 'write', + }), + t, + ), + ).toBe('conversation.toolChip.usedConnector|Linear,create issue'); + expect( + getToolChipLabel( + call({ + tool_name: 'telegram', + connector_name: 'Telegram', + action_name: 'send_message', + access: 'write', + }), + t, + ), + ).toBe('conversation.toolChip.usedConnector|Telegram,send message'); + }); + + it('keeps built-in labels for web search even with a connector name', () => { + expect( + getToolChipLabel( + call({ connector_name: 'Brave', arguments: { query: 'docsgpt' } }), + t, + ), + ).toBe('conversation.toolChip.searchingWeb|docsgpt'); + }); }); diff --git a/frontend/src/utils/streamingStatusUtils.ts b/frontend/src/utils/streamingStatusUtils.ts index f1925447..dc0a466d 100644 --- a/frontend/src/utils/streamingStatusUtils.ts +++ b/frontend/src/utils/streamingStatusUtils.ts @@ -41,6 +41,53 @@ const NO_ESCAPE = { interpolation: { escapeValue: false } } as const; */ export type ToolActivity = { key: string; values?: Record }; +const CONNECTOR_SEARCH_WORDS = /search|query|find|list/i; + +/** + * A call to a connection-backed tool, named after its service: "Searched + * Notion", "Read from Google Drive", "Used Linear: create issue". Reads that + * look up something search; other reads read; writes name their action. + */ +function describeConnectorCall(toolCall: ToolCallsType): ToolActivity | null { + const name = toolCall.connector_name; + if (!name) return null; + const action = toolCall.action_name ?? ''; + if (toolCall.access === 'write') { + const readable = action + .replace(/^[a-z0-9]+[_-](?=[a-z])/i, (prefix) => + // "linear_create_issue" and "notion-create-pages" drop the service + // prefix; a bare verb ("search") keeps its whole name. + name.toLowerCase().startsWith(prefix.slice(0, -1).toLowerCase()) + ? '' + : prefix, + ) + .replace(/[_-]+/g, ' ') + .trim() + .toLowerCase(); + return { key: 'connectorWrite', values: { name, action: readable } }; + } + if (CONNECTOR_SEARCH_WORDS.test(action)) + return { key: 'connectorSearch', values: { name } }; + return { key: 'connectorRead', values: { name } }; +} + +// Connector activities read differently while running and once done. +const CONNECTOR_KEYS: Record< + 'streamingStatus' | 'toolChip', + Record +> = { + streamingStatus: { + connectorSearch: 'searchingConnector', + connectorRead: 'usingConnector', + connectorWrite: 'usingConnector', + }, + toolChip: { + connectorSearch: 'searchedConnector', + connectorRead: 'readConnector', + connectorWrite: 'usedConnector', + }, +}; + export function describeToolCall(toolCall: ToolCallsType): ToolActivity { const { tool_name, action_name, arguments: args } = toolCall; const query = typeof args?.query === 'string' ? args.query : undefined; @@ -63,6 +110,8 @@ export function describeToolCall(toolCall: ToolCallsType): ToolActivity { return { key: 'readingPage', values: { target } }; } } + const connector = describeConnectorCall(toolCall); + if (connector) return connector; if (action_name === 'run_code') return { key: 'runningCode' }; if (ARTIFACT_ACTIONS.has(action_name)) return { key: 'creatingArtifact' }; if (tool_name === 'internal_search') return { key: 'searchingKnowledge' }; @@ -84,7 +133,9 @@ function activityLabel( t: TFunction, ): string { const key = - activity.key === 'web' ? GENERIC_SEARCH_KEY[namespace] : activity.key; + activity.key === 'web' + ? GENERIC_SEARCH_KEY[namespace] + : (CONNECTOR_KEYS[namespace][activity.key] ?? activity.key); return t( `conversation.${namespace}.${key}`, activity.values ? { ...activity.values, ...NO_ESCAPE } : undefined, diff --git a/tests/agents/test_headless_runner_callers.py b/tests/agents/test_headless_runner_callers.py new file mode 100644 index 00000000..6305e717 --- /dev/null +++ b/tests/agents/test_headless_runner_callers.py @@ -0,0 +1,67 @@ +"""``run_agent_headless`` holds a run for an outside caller to the write allowlist. + +Webhooks, schedules set through the API, and schedules a public-link user +set all run as the owner, but nobody behind them can approve for the owner. +""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import pytest + + +def _executor_kwargs(monkeypatch, config_extra=None, agent_kwargs=None, **run_kwargs): + from docsgpt.agents import headless_runner as hr + + agent = MagicMock(name="agent") + agent.gen.return_value = iter([{"answer": "ok"}]) + agent.llm.token_usage = {} + retriever = MagicMock(search=MagicMock(return_value=[])) + captured = {} + + def _executor(*_args, **kwargs): + captured.update(kwargs) + return MagicMock(headless_denials=[]) + + monkeypatch.setattr(hr, "get_prompt", lambda _pid: "system prompt") + monkeypatch.setattr(hr.RetrieverCreator, "create_retriever", classmethod(lambda cls, *a, **kw: retriever)) + monkeypatch.setattr(hr, "ToolExecutor", _executor) + def _create_agent(cls, *_args, **kwargs): + if agent_kwargs is not None: + agent_kwargs.update(kwargs) + return agent + + monkeypatch.setattr(hr.AgentCreator, "create_agent", classmethod(_create_agent)) + monkeypatch.setattr(hr.QuotaService, "check", lambda *a, **kw: None) + config = {"user_id": "u1", "id": "agent-1", "default_model_id": "m", **(config_extra or {})} + with patch("docsgpt.core.model_utils.validate_model_id", return_value=True), \ + patch("docsgpt.core.model_utils.get_provider_from_model_id", return_value="openai"), \ + patch("docsgpt.core.model_utils.get_api_key_for_provider", return_value="k"), \ + patch("docsgpt.utils.calculate_doc_token_budget", return_value=1000): + hr.run_agent_headless(config, "do the thing", **run_kwargs) + return captured + + +@pytest.mark.unit +class TestHeadlessCallerRules: + def test_owner_run_has_no_outside_caller(self, monkeypatch): + kwargs = _executor_kwargs(monkeypatch) + assert kwargs.get("external_caller") is False + assert kwargs.get("public_link_caller") is False + + @pytest.mark.parametrize("flag", ["external_caller", "public_link_caller"]) + def test_outside_caller_carries_the_agents_allowlist(self, monkeypatch, flag): + config = {"config": {"api_write_allowlist": ["tool-1:send"]}} + kwargs = _executor_kwargs(monkeypatch, config, **{flag: True}) + assert kwargs[flag] is True + assert kwargs["api_write_allowlist"] == ["tool-1:send"] + + @pytest.mark.parametrize("flag", ["external_caller", "public_link_caller"]) + def test_outside_caller_run_gets_no_wiki_editor(self, monkeypatch, flag): + # A scheduled or webhook run has no wiki tool at all, so an outside + # caller's schedule can't edit a wiki whatever the wiki allows. + agent_kwargs = {} + _executor_kwargs(monkeypatch, agent_kwargs=agent_kwargs, **{flag: True}) + assert agent_kwargs + assert "wiki_config" not in agent_kwargs diff --git a/tests/agents/test_research_agent.py b/tests/agents/test_research_agent.py index 79b899e7..aeae1e95 100644 --- a/tests/agents/test_research_agent.py +++ b/tests/agents/test_research_agent.py @@ -1676,3 +1676,136 @@ class TestGenInnerTraceSpans: llm_spans = [s for s in trace.spans if s.kind == tracing.KIND_LLM] assert llm_spans[0].parent_id == by_name["research step 1"].id assert by_name["research step 1"].previews["query"] == "s1" + + +# ===================================================================== +# Research steps honour approval and outside-caller gates +# ===================================================================== + + +@pytest.mark.unit +class TestResearchStepPauseGates: + """A research step can't pause for anyone, so a gated call is refused, not run.""" + + def _run(self, agent_base_params, mock_llm_handler, executor, tools_dict, calls, monkeypatch): + from docsgpt.agents import tool_executor as te_mod + + # Journal writes need a database; the refusal still goes through them. + monkeypatch.setattr(te_mod, "_record_proposed", lambda *a, **kw: False) + agent = ResearchAgent(tool_executor=executor, **agent_base_params) + executor.prepare_tools_for_llm(tools_dict) + ran = [] + + def gen_execute(tools, tc, llm_class): + ran.append(tc.name) + yield {"type": "tool_call", "data": {"action_name": tc.name, "status": "pending"}} + return (f"ran {tc.name}", tc.id) + + executor.execute = gen_execute + mock_llm_handler.create_tool_message = Mock( + side_effect=lambda call, result: {"role": "tool", "tool_call_id": call.id, "content": str(result)} + ) + messages, _ = agent._execute_step_tools_with_refinement( + calls, tools_dict, [{"role": "user", "content": "q"}], executor, False + ) + results = {m["tool_call_id"]: m["content"] for m in messages if m.get("role") == "tool"} + return ran, results, messages + + def test_public_link_wiki_write_is_refused_and_view_still_runs( + self, agent_base_params, mock_llm, mock_llm_handler, mock_llm_creator, mock_llm_handler_creator, monkeypatch, + ): + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.agents.tools.wiki import add_wiki_tool + from docsgpt.llm.handlers.base import ToolCall + + tools_dict = {} + add_wiki_tool( + tools_dict, + {"source_id": "w1", "source_owner_id": "visitor", "user": "visitor", "approval_required": True}, + ) + executor = ToolExecutor(user="visitor", decoded_token={"sub": "visitor"}, public_link_caller=True) + calls = [ + ToolCall(id="c1", name="wiki_create", arguments={"path": "/x.md", "content": "x"}), + ToolCall(id="c2", name="wiki_view", arguments={"path": "/"}), + ] + ran, results, messages = self._run( + agent_base_params, mock_llm_handler, executor, tools_dict, calls, monkeypatch + ) + assert ran == ["wiki_view"] + assert "approval" in results["c1"].lower() and "not run" in results["c1"].lower() + assert results["c2"] == "ran wiki_view" + # Each call, refused or not, still gets its assistant tool_call turn. + assert [m["tool_calls"][0]["id"] for m in messages if m.get("role") == "assistant"] == ["c1", "c2"] + + def test_external_caller_owner_credential_write_is_refused_read_runs( + self, agent_base_params, mock_llm, mock_llm_handler, mock_llm_creator, mock_llm_handler_creator, monkeypatch, + ): + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.llm.handlers.base import ToolCall + + params = {"type": "object", "properties": {"text": {"type": "string"}}} + tools_dict = { + "tool-1": { + "id": "tool-1", + "name": "telegram", + "user_id": "owner", + "config": {"encrypted_credentials": "secret"}, + "actions": [ + {"name": "send_message", "description": "Send", "active": True, "parameters": params}, + {"name": "get_updates", "description": "Read", "active": True, "parameters": params}, + ], + } + } + executor = ToolExecutor(user="owner", decoded_token={"sub": "owner"}, external_caller=True) + calls = [ + ToolCall(id="c1", name="send_message", arguments={"text": "hi"}), + ToolCall(id="c2", name="get_updates", arguments={"text": "x"}), + ] + ran, results, _ = self._run(agent_base_params, mock_llm_handler, executor, tools_dict, calls, monkeypatch) + assert ran == ["get_updates"] + assert "can't take this action" in results["c1"] + assert results["c2"] == "ran get_updates" + + def test_approval_required_action_is_refused_for_the_owner_too( + self, agent_base_params, mock_llm, mock_llm_handler, mock_llm_creator, mock_llm_handler_creator, monkeypatch, + ): + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.llm.handlers.base import ToolCall + + tools_dict = { + "tool-2": { + "id": "tool-2", + "name": "api_tool_like", + "user_id": "owner", + "config": {}, + "actions": [ + {"name": "delete_all", "description": "Delete", "active": True, "require_approval": True, + "parameters": {"type": "object", "properties": {}}}, + ], + } + } + executor = ToolExecutor(user="owner", decoded_token={"sub": "owner"}) + calls = [ToolCall(id="c1", name="delete_all", arguments={})] + ran, results, _ = self._run(agent_base_params, mock_llm_handler, executor, tools_dict, calls, monkeypatch) + assert ran == [] + assert "approval" in results["c1"].lower() + + def test_headless_denial_is_recorded( + self, agent_base_params, mock_llm, mock_llm_handler, mock_llm_creator, mock_llm_handler_creator, monkeypatch, + ): + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.llm.handlers.base import ToolCall + + tools_dict = { + "tool-2": { + "id": "tool-2", "name": "x", "user_id": "owner", "config": {}, + "actions": [{"name": "delete_all", "description": "Delete", "active": True, + "require_approval": True, "parameters": {"type": "object", "properties": {}}}], + } + } + executor = ToolExecutor(user="owner", decoded_token={"sub": "owner"}, headless=True) + calls = [ToolCall(id="c1", name="delete_all", arguments={})] + ran, results, _ = self._run(agent_base_params, mock_llm_handler, executor, tools_dict, calls, monkeypatch) + assert ran == [] + assert results["c1"].startswith("Tool denied") + assert [d["action_name"] for d in executor.headless_denials] == ["delete_all"] diff --git a/tests/agents/test_telegram_tool.py b/tests/agents/test_telegram_tool.py index acc92d44..c4209b99 100644 --- a/tests/agents/test_telegram_tool.py +++ b/tests/agents/test_telegram_tool.py @@ -63,6 +63,20 @@ class TestTelegramExecuteAction: assert result["status_code"] == 403 + @patch("docsgpt.agents.tools.telegram.requests.post") + def test_sends_to_the_default_chat_when_none_is_named(self, mock_post): + mock_post.return_value = MagicMock(status_code=200) + tool = TelegramTool(config={"token": "bot123:ABC", "chat_id": "-1001"}) + tool.execute_action("telegram_send_message", text="Hello") + assert mock_post.call_args[1]["data"]["chat_id"] == "-1001" + + @patch("docsgpt.agents.tools.telegram.requests.post") + def test_without_any_chat_it_says_how_to_set_one(self, mock_post, tool): + result = tool.execute_action("telegram_send_message", text="Hello") + mock_post.assert_not_called() + assert result["status"] == "error" + assert "chat" in result["error"].lower() + @pytest.mark.unit class TestTelegramMetadata: diff --git a/tests/agents/test_tool_pins.py b/tests/agents/test_tool_pins.py new file mode 100644 index 00000000..25495d68 --- /dev/null +++ b/tests/agents/test_tool_pins.py @@ -0,0 +1,356 @@ +"""Fixed ("pinned") tool parameters: the model never sees or overrides them.""" + +from unittest.mock import Mock + +import pytest + +from docsgpt.agents.tool_executor import ToolExecutor + + +def _action(**properties): + return { + "name": "telegram_send_message", + "description": "Send a message", + "active": True, + "parameters": {"type": "object", "properties": properties}, + } + + +def _llm(description="", **extra): + return {"type": "string", "description": description, "filled_by_llm": True, "value": "", **extra} + + +def _pinned(value, type_="string"): + return {"type": type_, "description": "", "filled_by_llm": False, "value": value} + + +def _run(executor, tools_dict, call_args, monkeypatch): + monkeypatch.setattr( + "docsgpt.agents.tool_executor.ToolActionParser", + lambda _cls, **kw: Mock(parse_args=Mock(return_value=("t1", "telegram_send_message", call_args))), + ) + call = Mock() + call.name = "telegram_send_message" + call.id = "c1" + call.arguments = "{}" + gen = executor.execute(tools_dict, call, "MockLLM") + while True: + try: + next(gen) + except StopIteration as stop: + return stop.value + + +def _tools(action): + return { + "t1": { + "id": "00000000-0000-0000-0000-000000000001", + "name": "telegram", + "config": {}, + "actions": [action], + } + } + + +@pytest.mark.unit +class TestPinnedValuesAtRunTime: + def test_llm_cannot_override_a_pinned_value(self, mock_tool_manager, monkeypatch): + """A model that sends the pinned key anyway (a mistake, or a prompt + injection naming another chat) still sends to the pinned chat.""" + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm(), chat_id=_pinned("111")) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi", "chat_id": "666"}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi", chat_id="111") + + @pytest.mark.parametrize("value", [0, False]) + def test_falsy_pins_are_honoured(self, mock_tool_manager, monkeypatch, value): + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm(), limit=_pinned(value, "integer")) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi", "limit": 50}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi", limit=value) + + def test_hidden_parameter_without_a_value_is_omitted_even_if_the_llm_sends_it( + self, mock_tool_manager, monkeypatch, + ): + """An empty fixed value means "leave it out" (an OpenAPI optional + parameter); the model was never shown it, so its value is dropped.""" + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm(), chat_id=_pinned("")) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi", "chat_id": "666"}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi") + + def test_llm_filled_parameter_keeps_its_default_when_omitted(self, mock_tool_manager, monkeypatch): + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm(), chat_id=_llm(value="42")) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi"}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi", chat_id="42") + + def test_llm_filled_parameter_takes_the_llm_value(self, mock_tool_manager, monkeypatch): + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm(), chat_id=_llm(value="42")) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi", "chat_id": "7"}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi", chat_id="7") + + def test_unknown_llm_arguments_are_dropped(self, mock_tool_manager, monkeypatch): + tool = mock_tool_manager.load_tool.return_value + action = _action(text=_llm()) + _run(ToolExecutor(user="u"), _tools(action), {"text": "hi", "token": "x"}, monkeypatch) + tool.execute_action.assert_called_once_with("telegram_send_message", text="hi") + + def test_api_tool_pinned_header_is_not_overridden(self, mock_tool_manager, monkeypatch): + executor = ToolExecutor(user="u") + monkeypatch.setattr( + "docsgpt.agents.tool_executor.ToolActionParser", + lambda _cls, **kw: Mock(parse_args=Mock(return_value=("t1", "get_item", {"id": "1", "X-Tenant": "b"}))), + ) + tools_dict = { + "t1": { + "id": "00000000-0000-0000-0000-000000000001", + "name": "api_tool", + "config": { + "actions": { + "get_item": { + "name": "get_item", + "url": "https://api.example.com/items", + "method": "GET", + "active": True, + "headers": {"properties": {"X-Tenant": _pinned("a")}}, + "query_params": {"properties": {"id": _llm()}}, + "body": {"properties": {}}, + } + } + }, + } + } + call = Mock() + call.name = "get_item" + call.id = "c1" + gen = executor.execute(tools_dict, call, "MockLLM") + while True: + try: + next(gen) + except StopIteration: + break + _, kwargs = mock_tool_manager.load_tool.call_args + assert kwargs["tool_config"]["headers"] == {"X-Tenant": "a"} + assert kwargs["tool_config"]["query_params"] == {"id": "1"} + + +@pytest.mark.unit +class TestPinnedValuesInTheSchema: + def test_pinned_parameter_is_not_shown_to_the_llm(self): + executor = ToolExecutor(user="u") + action = _action(text=_llm(), chat_id=_pinned("111"), limit=_pinned(0, "integer")) + functions = executor.prepare_tools_for_llm(_tools(action)) + params = functions[0]["function"]["parameters"] + assert set(params["properties"]) == {"text"} + + def test_pinned_parameter_is_dropped_from_the_required_list(self): + executor = ToolExecutor(user="u") + action = _action(text=_llm(required=True), chat_id={**_pinned("111"), "required": True}) + params = executor.prepare_tools_for_llm(_tools(action))[0]["function"]["parameters"] + assert params["required"] == ["text"] + + +@pytest.mark.unit +class TestPinHelpers: + def test_set_pins_fixes_and_releases_parameters(self): + from docsgpt.agents.tool_pins import set_pins + + action = _action(text=_llm(), chat_id=_llm()) + pinned = set_pins(action, {"chat_id": "123"}) + assert pinned["parameters"]["properties"]["chat_id"] == { + "type": "string", "description": "", "filled_by_llm": False, "value": "123", + } + released = set_pins(pinned, {"chat_id": None}) + assert released["parameters"]["properties"]["chat_id"]["filled_by_llm"] is True + assert released["parameters"]["properties"]["chat_id"]["value"] == "" + # The input is not modified. + assert action["parameters"]["properties"]["chat_id"]["filled_by_llm"] is True + + def test_set_pins_rejects_unknown_parameters(self): + from docsgpt.agents.tool_pins import set_pins + + with pytest.raises(ValueError): + set_pins(_action(text=_llm()), {"token": "x"}) + + @pytest.mark.parametrize( + "type_, value, expected", + [ + ("integer", "5", 5), + ("integer", 0, 0), + ("number", "2.5", 2.5), + ("boolean", "false", False), + ("boolean", True, True), + ("string", 12, "12"), + ], + ) + def test_coerce_value(self, type_, value, expected): + from docsgpt.agents.tool_pins import coerce_value + + assert coerce_value({"type": type_}, value) == expected + + @pytest.mark.parametrize( + "type_, value", + [("integer", "five"), ("integer", 2.5), ("boolean", "maybe"), ("string", ""), ("string", None), + ("string", {"a": 1}), ("array", "x")], + ) + def test_coerce_value_rejects(self, type_, value): + from docsgpt.agents.tool_pins import coerce_value + + with pytest.raises(ValueError): + coerce_value({"type": type_}, value) + + def test_merge_keeps_schema_and_ignores_type_changes(self): + from docsgpt.agents.tool_pins import merge_submitted_actions + + stored = [_action(text=_llm(), chat_id=_llm())] + merged = merge_submitted_actions(stored, [{ + "name": "telegram_send_message", + "parameters": {"properties": {"chat_id": {"type": "object", "filled_by_llm": False, "value": "9"}}}, + }], may_change_pins=True) + chat_id = merged[0]["parameters"]["properties"]["chat_id"] + assert chat_id["type"] == "string" + assert chat_id["value"] == "9" and chat_id["filled_by_llm"] is False + + def test_merge_refuses_pin_changes_without_permission(self): + from docsgpt.agents.tool_pins import PinChangeRefused, merge_submitted_actions + + stored = [_action(text=_llm(), chat_id=_pinned("111"))] + with pytest.raises(PinChangeRefused): + merge_submitted_actions(stored, [{ + "name": "telegram_send_message", + "parameters": {"properties": {"chat_id": {"filled_by_llm": True}}}, + }], may_change_pins=False) + # Resending the stored values unchanged is fine. + merged = merge_submitted_actions(stored, [{ + "name": "telegram_send_message", + "active": False, + "parameters": {"properties": {"chat_id": {"filled_by_llm": False, "value": "111"}}}, + }], may_change_pins=False) + assert merged[0]["active"] is False + + def test_carry_pins(self): + from docsgpt.agents.tool_pins import carry_pins + + old = _action(q=_llm(), team=_pinned("ENG")) + fresh = _action(q={"type": "string", "filled_by_llm": True, "value": ""}, team={"type": "string"}) + carried = carry_pins(old, fresh) + assert carried["parameters"]["properties"]["team"]["value"] == "ENG" + assert carried["parameters"]["properties"]["team"]["filled_by_llm"] is False + + +@pytest.mark.unit +class TestArgumentsShownForACall: + """What the chat shows for a call is what is sent, not what the model asked.""" + + def _pause(self, executor, action, call_args, monkeypatch): + monkeypatch.setattr( + "docsgpt.agents.tool_executor.ToolActionParser", + lambda _cls, **kw: Mock(parse_args=Mock(return_value=("t1", "telegram_send_message", call_args))), + ) + call = Mock() + call.name = "telegram_send_message" + call.id = "c1" + return executor.check_pause(_tools(action), call, "MockLLM") + + def test_approval_card_marks_a_fixed_value_without_revealing_it(self, monkeypatch): + """A value the owner fixed may be a secret, and the chat is shown to + whoever runs the agent: the card says it is fixed, not what it is.""" + from docsgpt.agents.tool_pins import FIXED_MASK + + action = {**_action(text=_llm(), chat_id=_pinned("111")), "require_approval": True} + pending = self._pause(ToolExecutor(user="u"), action, {"text": "hi", "chat_id": "666"}, monkeypatch) + assert pending["pause_type"] == "awaiting_approval" + assert pending["sent_arguments"] == {"text": "hi", "chat_id": FIXED_MASK} + # What the model asked stays as it was: resuming replays it to the model. + assert pending["arguments"] == {"text": "hi", "chat_id": "666"} + + def test_no_separate_arguments_when_nothing_changes(self, monkeypatch): + action = {**_action(text=_llm()), "require_approval": True} + pending = self._pause(ToolExecutor(user="u"), action, {"text": "hi"}, monkeypatch) + assert "sent_arguments" not in pending + + def test_a_finished_call_records_what_was_sent_and_keeps_it(self, mock_tool_manager, monkeypatch): + from docsgpt.agents.tool_pins import FIXED_MASK + + executor = ToolExecutor(user="u") + action = _action(text=_llm(), chat_id=_pinned("111")) + _run(executor, _tools(action), {"text": "hi", "chat_id": "666"}, monkeypatch) + recorded = executor.tool_calls[-1] + assert recorded["sent_arguments"] == {"text": "hi", "chat_id": FIXED_MASK} + assert recorded["arguments"] == {"text": "hi", "chat_id": "666"} + # Saved with the conversation, so a reload shows the same. + (saved,) = executor.get_truncated_tool_calls() + assert saved["sent_arguments"] == {"text": "hi", "chat_id": FIXED_MASK} + + def test_a_value_the_connection_sets_is_shown(self, monkeypatch): + """Telegram's default chat is the account's own setting, not a secret.""" + from docsgpt.agents.tool_pins import sent_arguments + + action = _action(text=_llm(), chat_id=_llm()) + assert sent_arguments(action, {"text": "hi", "chat_id": "666"}, {"chat_id": "111"}) == { + "text": "hi", "chat_id": "111", + } + + def test_fixed_query_and_body_values_and_headers_stay_hidden(self): + from docsgpt.agents.tool_pins import FIXED_MASK, sent_arguments + + action = { + "headers": {"properties": {"Authorization": _pinned("Bearer secret")}}, + "query_params": {"properties": {"id": _llm(), "api_key": _pinned("sk-query")}}, + "body": {"properties": {"token": _pinned("sk-body")}}, + } + shown = sent_arguments(action, {"id": "1"}) + assert shown == {"id": "1", "api_key": FIXED_MASK, "token": FIXED_MASK} + assert "secret" not in str(shown) and "sk-" not in str(shown) + + +@pytest.mark.unit +class TestStoredValuesNeverShown: + """A value the model did not send came from the owner's stored config, + which for an api_tool query parameter is a decrypted secret.""" + + def test_a_stored_default_the_model_did_not_send_is_masked(self): + from docsgpt.agents.tool_pins import FIXED_MASK, sent_arguments + + action = { + "query_params": {"properties": {"id": _llm(), "token": _llm(value="q-secret")}}, + "body": {"properties": {"note": _llm(value="b-secret")}}, + } + shown = sent_arguments(action, {"id": "1"}) + assert shown == {"id": "1", "token": FIXED_MASK, "note": FIXED_MASK} + # What the model itself sent is its own and shows as it is. + assert sent_arguments(action, {"id": "1", "token": "mine"})["token"] == "mine" + + def test_a_restored_api_tool_secret_is_not_recorded_with_the_call(self, mock_tool_manager, monkeypatch): + from docsgpt.api.user.tools.routes import _seal_api_tool_secrets + + config = _seal_api_tool_secrets({"actions": {"get_item": { + "name": "get_item", "url": "https://api.example.com/items", "method": "GET", "active": True, + "headers": {"properties": {}}, + "query_params": {"properties": {"id": _llm(), "token": _llm(value="q-secret")}}, + "body": {"properties": {}}, + }}}, {}, "owner") + tools_dict = {"t1": {"id": "00000000-0000-0000-0000-000000000001", "user_id": "owner", + "name": "api_tool", "config": config}} + monkeypatch.setattr( + "docsgpt.agents.tool_executor.ToolActionParser", + lambda _cls, **kw: Mock(parse_args=Mock(return_value=("t1", "get_item", {"id": "1"}))), + ) + call = Mock() + call.name = "get_item" + call.id = "c1" + executor = ToolExecutor(user="u") + events = [] + gen = executor.execute(tools_dict, call, "MockLLM") + while True: + try: + events.append(next(gen)) + except StopIteration: + break + # The call still sends the value; the chat never shows it. + _, kwargs = mock_tool_manager.load_tool.call_args + assert kwargs["tool_config"]["query_params"]["token"] == "q-secret" + assert "q-secret" not in str(events) + assert "q-secret" not in str(executor.get_truncated_tool_calls()) diff --git a/tests/agents/test_workflow_agent_types.py b/tests/agents/test_workflow_agent_types.py index 18bf22d3..7f0a09ee 100644 --- a/tests/agents/test_workflow_agent_types.py +++ b/tests/agents/test_workflow_agent_types.py @@ -540,18 +540,18 @@ class TestWorkflowNodeSourceAuthorization: monkeypatch.setattr(session, "db_readonly", _conn) def test_owner_sources_survive(self, monkeypatch): - import docsgpt.api.user.team_sharing as ts + import docsgpt.api.user.resource_access as ra self._stub_db(monkeypatch) - monkeypatch.setattr(ts, "can_access", lambda *a, **k: True) + monkeypatch.setattr(ra, "can_use_ref", lambda *a, **k: True) engine = self._engine("owner") assert engine._authorized_node_sources(["s1", "s2"]) == ["s1", "s2"] def test_foreign_sources_are_dropped(self, monkeypatch): - import docsgpt.api.user.team_sharing as ts + import docsgpt.api.user.resource_access as ra self._stub_db(monkeypatch) - monkeypatch.setattr(ts, "can_access", lambda conn, k, sid, u: sid == "mine") + monkeypatch.setattr(ra, "can_use_ref", lambda conn, k, sid, u: sid == "mine") engine = self._engine("owner") assert engine._authorized_node_sources(["mine", "theirs"]) == ["mine"] @@ -563,13 +563,13 @@ class TestWorkflowNodeSourceAuthorization: assert engine._authorized_node_sources(["s1"]) == [] def test_authorization_error_fails_closed(self, monkeypatch): - import docsgpt.api.user.team_sharing as ts + import docsgpt.api.user.resource_access as ra def _boom(*a, **k): raise RuntimeError("db down") self._stub_db(monkeypatch) - monkeypatch.setattr(ts, "can_access", _boom) + monkeypatch.setattr(ra, "can_use_ref", _boom) engine = self._engine("owner") assert engine._authorized_node_sources(["s1"]) == [] diff --git a/tests/agents/tools/test_mcp_tool.py b/tests/agents/tools/test_mcp_tool.py index d418ef0a..fbef348c 100644 --- a/tests/agents/tools/test_mcp_tool.py +++ b/tests/agents/tools/test_mcp_tool.py @@ -266,6 +266,23 @@ class TestGenerateCacheKey: }) assert "apikey:none" in tool._cache_key + @pytest.mark.parametrize("auth_type, field", [("bearer", "bearer_token"), ("api_key", "api_key")]) + def test_tokens_with_a_shared_prefix_get_their_own_client(self, auth_type, field): + """Every fine-grained GitHub token starts with ``github_pat_``; two users' tokens + must never share a cached client (it carries the first user's token).""" + first = _make_tool({ + "server_url": "https://mcp.example.com", + "auth_type": auth_type, + "auth_credentials": {field: "github_pat_11AAAAAAA_alice"}, + }) + second = _make_tool({ + "server_url": "https://mcp.example.com", + "auth_type": auth_type, + "auth_credentials": {field: "github_pat_11AAAAAAA_bob"}, + }) + assert first._cache_key != second._cache_key + assert "github_pat" not in first._cache_key + # ===================================================================== # Transport Creation @@ -840,6 +857,15 @@ class TestMCPOAuthManager: assert result is True mock_redis.setex.assert_called() + def test_handle_callback_keeps_the_issuer(self): + from docsgpt.agents.tools.mcp_tool import MCPOAuthManager + + mock_redis = MagicMock() + manager = MCPOAuthManager(mock_redis) + + assert manager.handle_oauth_callback(state="s", code="c", iss="https://issuer.example") is True + mock_redis.setex.assert_any_call("mcp_oauth:iss:s", 300, "https://issuer.example") + def test_handle_callback_no_redis(self): from docsgpt.agents.tools.mcp_tool import MCPOAuthManager @@ -1010,6 +1036,10 @@ class TestDBTokenStorage: monkeypatch.setattr(session_mod, "db_session", _yield) monkeypatch.setattr(session_mod, "db_readonly", _yield) + import docsgpt.connectors.service as service_mod + + monkeypatch.setattr(service_mod, "db_session", _yield) + monkeypatch.setattr(service_mod, "db_readonly", _yield) def test_get_base_url(self): from docsgpt.agents.tools.mcp_tool import DBTokenStorage @@ -1132,7 +1162,13 @@ class TestDBTokenStorage: # ``server_url`` must NOT be duplicated inside the JSONB blob. session_data = row["session_data"] or {} assert "server_url" not in session_data - assert session_data.get("tokens", {}).get("access_token") == "at" + # Tokens live only in the encrypted envelope, never in plaintext. + assert "tokens" not in session_data + assert row["encrypted_credentials"].startswith("v2:") + from docsgpt.connectors.service import read_secrets + + assert read_secrets(row)["tokens"]["access_token"] == "at" + assert row["status"] == "connected" def test_clear_removes_row(self, monkeypatch, pg_conn): from mcp.shared.auth import OAuthToken @@ -1167,6 +1203,128 @@ class TestDBTokenStorage: is None ) + def test_stored_tokens_remember_when_they_expire(self, monkeypatch, pg_conn): + import time + + from mcp.shared.auth import OAuthToken + + from docsgpt.agents.tools.mcp_tool import DBTokenStorage + + self._patch_db(monkeypatch, pg_conn) + storage = DBTokenStorage(server_url="https://mcp.expiry.example.com/mcp", user_id="user-expiry") + loop = asyncio.new_event_loop() + try: + before = time.time() + loop.run_until_complete(storage.set_tokens(OAuthToken( + access_token="at", token_type="Bearer", expires_in=3600, refresh_token="rt", + ))) + # A later process reads the tokens back with when they expire. + reader = DBTokenStorage(server_url="https://mcp.expiry.example.com/mcp", user_id="user-expiry") + tokens = loop.run_until_complete(reader.get_tokens()) + finally: + loop.close() + assert tokens.access_token == "at" + assert before + 3600 <= reader.expires_at <= time.time() + 3600 + + def test_tokens_without_a_lifetime_have_no_expiry(self, monkeypatch, pg_conn): + from mcp.shared.auth import OAuthToken + + from docsgpt.agents.tools.mcp_tool import DBTokenStorage + + self._patch_db(monkeypatch, pg_conn) + storage = DBTokenStorage(server_url="https://mcp.forever.example.com/mcp", user_id="user-forever") + loop = asyncio.new_event_loop() + try: + loop.run_until_complete(storage.set_tokens(OAuthToken(access_token="at", token_type="Bearer"))) + loop.run_until_complete(storage.get_tokens()) + finally: + loop.close() + assert storage.expires_at is None + + +# ===================================================================== +# Renewing a stored sign-in +# ===================================================================== + + +class _StoredTokens: + """A token storage holding one sign-in, with or without its expiry.""" + + def __init__(self, tokens, expires_at=None): + self.tokens = tokens + self.expires_at = None + self._expires_at = expires_at + + async def get_tokens(self): + self.expires_at = self._expires_at + return self.tokens + + async def get_client_info(self): + return None + + +@pytest.mark.unit +class TestStoredSignInRenewal: + """A stored MCP sign-in is renewed with its refresh token once it expires. + + The SDK only knows when a token expires if it obtained it in this + process; a worker that loads it from the connection must be told, or it + sends the expired token, gets a 401 and asks the user to sign in again. + """ + + @staticmethod + def _oauth(storage): + from docsgpt.agents.tools.mcp_tool import NonInteractiveOAuth + + oauth = NonInteractiveOAuth( + mcp_url="https://mcp.example.com/mcp", + redirect_uri="https://docsgpt.example.com/api/mcp_server/callback", + user_id="alice", + connection_id="c1", + ) + oauth.context.storage = storage + return oauth + + @staticmethod + def _token(**extra): + from mcp.shared.auth import OAuthToken + + return OAuthToken(access_token="at", token_type="Bearer", **extra) + + def test_expired_stored_token_is_due_for_renewal(self): + import time + + oauth = self._oauth(_StoredTokens(self._token(refresh_token="rt", expires_in=3600), time.time() - 5)) + asyncio.run(oauth._initialize()) + assert oauth.context.is_token_valid() is False + + def test_unexpired_stored_token_is_used_as_it_is(self): + import time + + expires_at = time.time() + 600 + oauth = self._oauth(_StoredTokens(self._token(refresh_token="rt", expires_in=3600), expires_at)) + asyncio.run(oauth._initialize()) + assert oauth.context.token_expiry_time == expires_at + assert oauth.context.is_token_valid() is True + + def test_token_saved_before_expiries_were_kept_is_renewed_once(self): + # Its age is unknown, so it is treated as expired while it can be renewed. + oauth = self._oauth(_StoredTokens(self._token(refresh_token="rt", expires_in=3600))) + asyncio.run(oauth._initialize()) + assert oauth.context.is_token_valid() is False + + def test_token_that_cannot_be_renewed_is_still_tried(self): + oauth = self._oauth(_StoredTokens(self._token(expires_in=3600))) + asyncio.run(oauth._initialize()) + assert oauth.context.is_token_valid() is True + + def test_a_lost_sign_in_raises_a_typed_error(self): + from docsgpt.agents.tools.mcp_tool import MCPReauthorizationRequired + + oauth = self._oauth(_StoredTokens(None)) + with pytest.raises(MCPReauthorizationRequired, match="OAuth session expired"): + asyncio.run(oauth.redirect_handler("https://mcp.example.com/authorize?state=x")) + # ===================================================================== # NonInteractiveOAuth @@ -1267,6 +1425,30 @@ class TestResolveRedirectUriExtended: }) assert tool.redirect_uri == "https://custom.redirect/callback" + def test_setting_wins_over_the_redirect_the_page_sends(self, monkeypatch): + from docsgpt.core.settings import settings + + # A page opened on a plain-HTTP address sends its own origin; the + # operator's HTTPS callback is the one servers accept. + monkeypatch.setattr(settings, "MCP_OAUTH_REDIRECT_URI", "https://docs.example.com/api/mcp_server/callback") + tool = _make_tool({ + "server_url": "https://mcp.example.com", + "auth_type": "none", + "redirect_uri": "http://10.0.0.5:7091/api/mcp_server/callback", + }) + assert tool.redirect_uri == "https://docs.example.com/api/mcp_server/callback" + + def test_page_redirect_used_without_the_setting(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "MCP_OAUTH_REDIRECT_URI", None, raising=False) + tool = _make_tool({ + "server_url": "https://mcp.example.com", + "auth_type": "none", + "redirect_uri": "https://app.example.com/api/mcp_server/callback/", + }) + assert tool.redirect_uri == "https://app.example.com/api/mcp_server/callback" + def test_connector_redirect_base_uri_setting(self, monkeypatch): from docsgpt.core.settings import settings @@ -1992,8 +2174,11 @@ class TestDocsGPTOAuthExtended: mock_db.__getitem__ = MagicMock(return_value=mock_collection) mock_redis = MagicMock() - # First get returns the code - mock_redis.get.return_value = b"auth_code_123" + stored = { + "mcp_oauth:code:mystate": b"auth_code_123", + "mcp_oauth:iss:mystate": b"https://auth.example.com", + } + mock_redis.get.side_effect = stored.get oauth = DocsGPTOAuth( mcp_url="https://mcp.example.com/api", @@ -2009,9 +2194,11 @@ class TestDocsGPTOAuthExtended: loop = asyncio.new_event_loop() try: - code, state = loop.run_until_complete(oauth.callback_handler()) - assert code == "auth_code_123" - assert state == "mystate" + # The MCP SDK reads ``.code``, ``.state`` and the RFC 9207 ``.iss``. + result = loop.run_until_complete(oauth.callback_handler()) + assert result.code == "auth_code_123" + assert result.state == "mystate" + assert result.iss == "https://auth.example.com" finally: loop.close() diff --git a/tests/agents/tools/test_wiki.py b/tests/agents/tools/test_wiki.py index a830b5be..ce02008c 100644 --- a/tests/agents/tools/test_wiki.py +++ b/tests/agents/tools/test_wiki.py @@ -640,3 +640,169 @@ class TestBuildAgentGating: assert cfg is not None # v1 binds the first writable wiki source; the extra is skipped. assert cfg["source_id"] == "wiki-1" + + +# ===================================================================== +# API, widget and public-link callers (the wiki's outside-edits setting) +# ===================================================================== + + +def _outside_tool(monkeypatch, allowed): + from docsgpt.agents.tools.wiki import WikiTool + + class _Sources: + def __init__(self, conn): + pass + + def get_by_id(self, sid): + return {"id": sid, "wiki_outside_edits": allowed} + + monkeypatch.setattr("docsgpt.agents.tools.wiki.SourcesRepository", _Sources) + return WikiTool( + { + "source_id": "src-1", + "source_owner_id": "owner-sub", + "decoded_token": {"sub": "owner-sub"}, + "user": "owner-sub", + "outside_caller": True, + } + ) + + +_WRITES = ( + ("create", {"path": "/b.md", "content": "x"}), + ("str_replace", {"path": "/a.md", "old_str": "one", "new_str": "two"}), + ("insert", {"path": "/a.md", "insert_line": 1, "insert_text": "x"}), + ("delete", {"path": "/a.md"}), + ("rename", {"old_path": "/a.md", "new_path": "/c.md"}), +) + + +@pytest.mark.unit +class TestOutsideCallerWrites: + def test_refused_while_the_setting_is_off(self, patched_wiki, monkeypatch, reembed_mock): + _FakeWikiRepo().upsert("src-1", "/a.md", "one") + tool = _outside_tool(monkeypatch, False) + for action, kwargs in _WRITES: + result = tool.execute_action(action, **kwargs) + assert "API or widget" in result, action + reembed_mock.assert_not_called() + assert _FakeWikiRepo().get_by_path("src-1", "/a.md")["content"] == "one" + # Reading stays open to them. + assert "one" in tool.execute_action("view", path="/a.md") + + def test_allowed_once_the_owner_turns_it_on(self, patched_wiki, monkeypatch): + tool = _outside_tool(monkeypatch, True) + assert tool.execute_action("create", path="/b.md", content="x") == "Page created: /b.md" + + def test_a_missing_row_refuses(self, patched_wiki, monkeypatch): + tool = _outside_tool(monkeypatch, True) + + class _Gone: + def __init__(self, conn): + pass + + def get_by_id(self, sid): + return None + + monkeypatch.setattr("docsgpt.agents.tools.wiki.SourcesRepository", _Gone) + assert "API or widget" in tool.execute_action("create", path="/b.md", content="x") + + def test_owner_and_team_runs_skip_the_setting(self, wiki_tool, monkeypatch): + class _Boom: + def __init__(self, conn): + raise AssertionError("an in-app run must not read the setting") + + monkeypatch.setattr("docsgpt.agents.tools.wiki.SourcesRepository", _Boom) + assert wiki_tool.execute_action("create", path="/b.md", content="x") == "Page created: /b.md" + + +@pytest.mark.unit +class TestReadOnlyEntry: + def _entry(self, **extra): + from docsgpt.agents.tools.wiki import WIKI_TOOL_ID, add_wiki_tool + + tools_dict = {} + add_wiki_tool(tools_dict, {"source_id": "s1", "source_owner_id": "owner", "user": "owner", **extra}) + return tools_dict[WIKI_TOOL_ID] + + def test_outside_caller_without_the_setting_is_offered_only_view(self): + entry = self._entry(outside_caller=True, writes_allowed=False) + assert [a["name"] for a in entry["actions"]] == ["wiki_view"] + assert entry["config"]["outside_caller"] is True + + def test_writes_offered_when_allowed(self): + entry = self._entry(outside_caller=True, writes_allowed=True) + names = {a["name"] for a in entry["actions"]} + assert {"wiki_view", "wiki_create", "wiki_str_replace", "wiki_delete"} <= names + assert entry["config"]["outside_caller"] is True + + def test_approval_required_gates_writes_only(self): + entry = self._entry(approval_required=True) + for action in entry["actions"]: + assert bool(action.get("require_approval")) is (action["name"] != "wiki_view"), action["name"] + assert entry["config"]["outside_caller"] is False + + def test_in_app_config_keeps_every_action(self): + entry = self._entry() + assert len(entry["actions"]) == 6 + assert entry["config"]["outside_caller"] is False + + +@pytest.mark.unit +class TestBuildConfigOutsideCallers: + def _cfg(self, monkeypatch, agent_config, allowed=False): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + class _SrcRepo: + def __init__(self, conn): + pass + + def get_any(self, sid, owner): + return {"id": sid, "config": {"kind": "wiki"}, "wiki_outside_edits": allowed} + + monkeypatch.setattr("docsgpt.api.answer.services.stream_processor.SourcesRepository", _SrcRepo) + monkeypatch.setattr("docsgpt.api.answer.services.stream_processor.db_readonly", _noop_conn) + monkeypatch.setattr( + "docsgpt.api.answer.services.stream_processor._wiki_write_owner", lambda conn, sid, uid: "owner-x" + ) + proc = StreamProcessor.__new__(StreamProcessor) + proc.all_sources = [{"id": "wiki-src"}] + proc.decoded_token = {"sub": "owner-x"} + if agent_config is not None: + proc.agent_config = agent_config + return proc._build_wiki_config() + + def test_api_key_caller_gets_read_only_while_off(self, monkeypatch): + cfg = self._cfg(monkeypatch, {"external_api_caller": True}) + assert cfg["outside_caller"] is True + assert cfg["writes_allowed"] is False + assert cfg["approval_required"] is False + + def test_api_key_caller_may_write_when_on(self, monkeypatch): + cfg = self._cfg(monkeypatch, {"external_api_caller": True}, allowed=True) + assert cfg["outside_caller"] is True + assert cfg["writes_allowed"] is True + + @pytest.mark.parametrize("allowed", [False, True]) + def test_public_link_visitor_is_asked_whatever_the_switch(self, monkeypatch, allowed): + # A visitor runs as themselves, so only wikis they can edit get here; + # the switch doesn't apply, but every edit waits for their approval. + cfg = self._cfg(monkeypatch, {"public_link_caller": True}, allowed=allowed) + assert cfg["outside_caller"] is False + assert cfg["writes_allowed"] is True + assert cfg["approval_required"] is True + + def test_v1_key_holder_gets_read_only(self, monkeypatch): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + monkeypatch.setattr(StreamProcessor, "external_caller", True, raising=False) + cfg = self._cfg(monkeypatch, {}) + assert cfg["writes_allowed"] is False + + @pytest.mark.parametrize("agent_config", [None, {}, {"external_api_caller": False}]) + def test_owner_and_team_unaffected(self, monkeypatch, agent_config): + cfg = self._cfg(monkeypatch, agent_config) + assert cfg["outside_caller"] is False + assert cfg["writes_allowed"] is True + assert cfg["approval_required"] is False diff --git a/tests/api/answer/services/test_resume_wiki_callers.py b/tests/api/answer/services/test_resume_wiki_callers.py new file mode 100644 index 00000000..c86a36b5 --- /dev/null +++ b/tests/api/answer/services/test_resume_wiki_callers.py @@ -0,0 +1,201 @@ +"""A resumed turn keeps the wiki and write rules of whoever resumes it. + +Continuation state is looked up by the agent owner, so a request carrying the +agent's key (a widget key is public) could resume the owner's own paused chat. +The resumed run counts as an outside caller when either the saved state or the +resuming request says so, and a request may only resume a turn of the agent it +names. +""" + +from __future__ import annotations + +import copy +import uuid +from contextlib import contextmanager +from unittest.mock import MagicMock + +import pytest + +AGENT = "11111111-1111-1111-1111-111111111111" +OTHER_AGENT = "22222222-2222-2222-2222-222222222222" +OWNER = "owner" + + +def _wiki_entry(**config): + from docsgpt.agents.tools.wiki import WIKI_TOOL_ID, add_wiki_tool + + tools = {} + add_wiki_tool(tools, {"source_id": "wiki-1", "source_owner_id": OWNER, "user": OWNER, **config}) + return tools[WIKI_TOOL_ID] + + +def _state(agent_id=AGENT, **flags): + return { + "messages": [], + "pending_tool_calls": [], + "tools_dict": {"wiki": _wiki_entry()}, + "tool_schemas": [], + "client_tools": None, + "agent_config": { + "model_id": "m1", "llm_name": "openai", "api_key": "k", "user_api_key": "agent-key", + "agent_type": "ClassicAgent", "agent_id": agent_id, **flags, + }, + } + + +@pytest.fixture +def resume(monkeypatch): + """Resume a saved state; returns (tools_dict, executor, continuation service).""" + from docsgpt.agents import agent_creator as ac_mod + from docsgpt.api.answer.services import continuation_service as cont_mod + from docsgpt.api.answer.services import stream_processor as sp_mod + from docsgpt.llm import llm_creator as llm_creator_mod + from docsgpt.llm.handlers import handler_creator as handler_mod + + @contextmanager + def _noop(): + yield None + + agents = {"agent-key": {"id": AGENT, "user_id": OWNER}, "other-key": {"id": OTHER_AGENT, "user_id": OWNER}} + + class _Agents: + def __init__(self, conn): + pass + + def find_by_key(self, key): + return agents.get(key) + + live = {"allowed": False} + + class _Sources: + def __init__(self, conn): + pass + + def get_by_id(self, sid): + return {"id": sid, "wiki_outside_edits": live["allowed"]} + + def _get_agent_key(self, agent_id, user_id): + # Only the owner reaches the agent directly; anyone else by its link. + self.public_link_usage = user_id != OWNER + return "agent-key", user_id != OWNER, None + + monkeypatch.setattr(sp_mod.StreamProcessor, "_get_agent_key", _get_agent_key) + monkeypatch.setattr(sp_mod, "db_readonly", _noop) + monkeypatch.setattr(sp_mod, "AgentsRepository", _Agents) + monkeypatch.setattr("docsgpt.agents.tools.wiki.db_readonly", _noop) + monkeypatch.setattr("docsgpt.agents.tools.wiki.SourcesRepository", _Sources) + monkeypatch.setattr(llm_creator_mod.LLMCreator, "create_llm", lambda *a, **kw: MagicMock()) + monkeypatch.setattr(handler_mod.LLMHandlerCreator, "create_handler", lambda *a, **kw: MagicMock()) + created = {} + monkeypatch.setattr(ac_mod.AgentCreator, "create_agent", lambda *a, **kw: created.update(kw) or MagicMock()) + + def _run(state, data, token, *, external_caller=False, allowed=False): + live["allowed"] = allowed + cont_service = MagicMock() + cont_service.claim_state.return_value = copy.deepcopy(state) + monkeypatch.setattr(cont_mod, "ContinuationService", lambda: cont_service) + processor = sp_mod.StreamProcessor(data, token, external_caller=external_caller) + result = processor.resume_from_tool_actions(tool_actions=[], conversation_id=str(uuid.uuid4())) + return result[2], created["tool_executor"], cont_service + + return _run + + +def _actions(tools_dict): + return [a["name"] for a in tools_dict["wiki"]["actions"]] + + +@pytest.mark.unit +class TestResumeByOutsideCaller: + def test_widget_key_resuming_the_owners_chat_gets_read_only_wiki(self, resume): + tools, executor, _ = resume(_state(), {"api_key": "agent-key"}, None) + assert executor.external_caller is True + assert _actions(tools) == ["wiki_view"] + assert tools["wiki"]["config"]["outside_caller"] is True + + def test_signed_in_stranger_with_the_key_is_outside(self, resume): + tools, executor, _ = resume(_state(), {"api_key": "agent-key"}, {"sub": "stranger"}) + assert executor.external_caller is True + assert _actions(tools) == ["wiki_view"] + + def test_edits_stay_when_the_wiki_allows_them(self, resume): + tools, executor, _ = resume(_state(), {"api_key": "agent-key"}, None, allowed=True) + assert executor.external_caller is True + assert "wiki_create" in _actions(tools) + # The tool still re-checks the live setting on every write. + assert tools["wiki"]["config"]["outside_caller"] is True + + def test_v1_key_holder_is_outside(self, resume): + tools, executor, _ = resume(_state(), {"api_key": "agent-key"}, {"sub": OWNER}, external_caller=True) + assert executor.external_caller is True + assert _actions(tools) == ["wiki_view"] + + def test_saved_outside_state_stays_outside_for_the_owner(self, resume): + tools, executor, _ = resume(_state(external_api_caller=True), {}, {"sub": OWNER}) + assert executor.external_caller is True + assert _actions(tools) == ["wiki_view"] + + def test_owner_resuming_in_app_is_unaffected(self, resume): + tools, executor, _ = resume(_state(), {}, {"sub": OWNER}) + assert executor.external_caller is False + assert executor.public_link_caller is False + assert "wiki_create" in _actions(tools) + assert tools["wiki"]["config"]["outside_caller"] is False + + def test_owner_previewing_with_the_key_is_unaffected(self, resume): + tools, executor, _ = resume(_state(), {"api_key": "agent-key"}, {"sub": OWNER}) + assert executor.external_caller is False + assert "wiki_create" in _actions(tools) + + +@pytest.mark.unit +class TestResumeTargetsTheSameAgent: + def test_a_key_for_another_agent_is_refused(self, resume): + with pytest.raises(ValueError): + resume(_state(), {"api_key": "other-key"}, None) + + def test_release_on_refusal(self, resume, monkeypatch): + from docsgpt.api.answer.services import continuation_service as cont_mod + from docsgpt.api.answer.services import stream_processor as sp_mod + + cont_service = MagicMock() + cont_service.claim_state.return_value = _state() + monkeypatch.setattr(cont_mod, "ContinuationService", lambda: cont_service) + processor = sp_mod.StreamProcessor({"api_key": "other-key"}, None) + conversation_id = str(uuid.uuid4()) + with pytest.raises(ValueError): + processor.resume_from_tool_actions(tool_actions=[], conversation_id=conversation_id) + cont_service.release_claim.assert_called_once_with(conversation_id, OWNER) + + def test_a_key_cannot_resume_an_agentless_chat(self, resume): + with pytest.raises(ValueError): + resume(_state(agent_id=None), {"api_key": "agent-key"}, None) + + def test_a_named_agent_must_match(self, resume): + with pytest.raises(ValueError): + resume(_state(), {"agent_id": OTHER_AGENT}, {"sub": OWNER}) + + def test_same_agent_passes(self, resume): + tools, _executor, _ = resume(_state(), {"agent_id": AGENT.upper()}, {"sub": OWNER}) + assert "wiki_create" in _actions(tools) + + +@pytest.mark.unit +class TestResumePublicLink: + def test_saved_public_link_run_keeps_wiki_writes_behind_approval(self, resume): + state = _state(public_link_caller=True) + tools, executor, _ = resume(state, {}, {"sub": "visitor"}) + assert executor.public_link_caller is True + writes = [a for a in tools["wiki"]["actions"] if a["name"] != "wiki_view"] + assert writes and all(a.get("require_approval") for a in writes) + view = next(a for a in tools["wiki"]["actions"] if a["name"] == "wiki_view") + assert not view.get("require_approval") + # Public-link visitors aren't covered by the switch. + assert tools["wiki"]["config"]["outside_caller"] is False + + def test_a_request_naming_the_agent_by_its_link_is_public(self, resume): + tools, executor, _ = resume(_state(), {"agent_id": AGENT}, {"sub": "visitor"}) + assert executor.public_link_caller is True + assert all( + a.get("require_approval") for a in tools["wiki"]["actions"] if a["name"] != "wiki_view" + ) diff --git a/tests/api/answer/services/test_stream_processor_access.py b/tests/api/answer/services/test_stream_processor_access.py index 1ac58ede..77597db4 100644 --- a/tests/api/answer/services/test_stream_processor_access.py +++ b/tests/api/answer/services/test_stream_processor_access.py @@ -116,3 +116,58 @@ class TestWikiConfigAccess: assert cfg["source_owner_id"] == OWNER and cfg["user"] == "ed" assert self._cfg(use_conn, "vi", sid) is None assert self._cfg(use_conn, "eve", sid) is None + + +class TestWikiOutsideEdits: + """API-key runs edit a wiki only when its owner allows; link visitors are asked.""" + + def _tools(self, conn, caller, sid, agent_config): + from docsgpt.agents.tools.wiki import WIKI_TOOL_ID, add_wiki_tool + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + proc = StreamProcessor.__new__(StreamProcessor) + proc.all_sources = [{"id": sid}] + proc.decoded_token = {"sub": caller} + proc.agent_config = agent_config + cfg = proc._build_wiki_config() + if cfg is None: + return None + tools = {} + add_wiki_tool(tools, cfg) + return {a["name"]: bool(a.get("require_approval")) for a in tools[WIKI_TOOL_ID]["actions"]} + + def _wiki(self, conn, owner=OWNER): + return str(SourcesRepository(conn).create("W", user_id=owner, config={"kind": "wiki"})["id"]) + + def test_api_key_caller_reads_until_the_owner_allows_edits(self, use_conn): + # A widget or API run acts as the agent owner, who owns the wiki. + sid = self._wiki(use_conn) + assert self._tools(use_conn, OWNER, sid, {"external_api_caller": True}) == {"wiki_view": False} + SourcesRepository(use_conn).set_wiki_outside_edits(sid, OWNER, True) + tools = self._tools(use_conn, OWNER, sid, {"external_api_caller": True}) + assert tools["wiki_create"] is False + + def test_owner_and_team_editor_keep_every_action(self, use_conn): + sid = self._wiki(use_conn) + _share(use_conn, "source", sid, "ed", "editor") + assert self._tools(use_conn, OWNER, sid, {})["wiki_create"] is False + assert self._tools(use_conn, "ed", sid, {})["wiki_create"] is False + + def test_link_visitor_edits_their_own_wiki_after_approving(self, use_conn): + # Visitor owns a wiki that the agent (someone else's) also uses. + sid = self._wiki(use_conn, owner="visitor") + tools = self._tools(use_conn, "visitor", sid, {"public_link_caller": True}) + assert tools["wiki_view"] is False + assert tools["wiki_create"] is True and tools["wiki_delete"] is True + + def test_link_visitor_who_edits_the_wiki_through_a_team(self, use_conn): + sid = self._wiki(use_conn) + _share(use_conn, "source", sid, "visitor", "editor") + tools = self._tools(use_conn, "visitor", sid, {"public_link_caller": True}) + assert tools["wiki_create"] is True + + def test_link_visitor_without_edit_rights_gets_no_wiki_tool(self, use_conn): + sid = self._wiki(use_conn) + assert self._tools(use_conn, "visitor", sid, {"public_link_caller": True}) is None + _share(use_conn, "source", sid, "visitor", "viewer") + assert self._tools(use_conn, "visitor", sid, {"public_link_caller": True}) is None diff --git a/tests/api/answer/services/test_stream_processor_utils.py b/tests/api/answer/services/test_stream_processor_utils.py index e6751a20..411ec014 100644 --- a/tests/api/answer/services/test_stream_processor_utils.py +++ b/tests/api/answer/services/test_stream_processor_utils.py @@ -22,7 +22,7 @@ def _patch_db(conn): "docsgpt.api.answer.services.stream_processor.db_readonly", _yield ), patch( "docsgpt.api.answer.services.stream_processor.db_session", _yield - ): + ), patch("docsgpt.agents.tool_executor.db_readonly", _yield): yield diff --git a/tests/api/answer/test_stream_processor_exposure.py b/tests/api/answer/test_stream_processor_exposure.py index 582dac2f..6f2a4896 100644 --- a/tests/api/answer/test_stream_processor_exposure.py +++ b/tests/api/answer/test_stream_processor_exposure.py @@ -22,6 +22,7 @@ def _processor() -> StreamProcessor: sp.agent_config = {} sp.retriever_config = {"retriever_name": "classic", "chunks": 2, "doc_token_limit": 50000} sp.data = {} + sp.decoded_token = {"sub": "u"} return sp diff --git a/tests/api/answer/test_unauthenticated_runs.py b/tests/api/answer/test_unauthenticated_runs.py new file mode 100644 index 00000000..3fd2fe45 --- /dev/null +++ b/tests/api/answer/test_unauthenticated_runs.py @@ -0,0 +1,59 @@ +"""A request with no token and no API key never sets up an agent run. + +The routes answer it with 401, but they used to build the agent first, so a +public agent's prompt tools were pre-fetched (run) for nobody. +""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import pytest + + +@pytest.mark.unit +class TestNoTokenNoRun: + def test_build_agent_sets_nothing_up_without_a_caller(self): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + processor = StreamProcessor({"question": "q", "agent_id": "a-1"}, None) + with patch.object(processor, "initialize") as initialize, \ + patch.object(processor, "pre_fetch_tools") as pre_fetch_tools, \ + patch.object(processor, "create_agent") as create_agent: + assert processor.build_agent("q") is None + initialize.assert_not_called() + pre_fetch_tools.assert_not_called() + create_agent.assert_not_called() + assert processor.decoded_token is None + + def test_api_key_request_still_runs(self): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + processor = StreamProcessor({"question": "q", "api_key": "k"}, None) + agent = MagicMock() + + def _initialize(): + processor.decoded_token = {"sub": "owner"} + + with patch.object(processor, "initialize", side_effect=_initialize), \ + patch.object(processor, "_exposure_partition", return_value=([], [])), \ + patch.object(processor, "pre_fetch_docs", return_value=(None, None)), \ + patch.object(processor, "pre_fetch_tools", return_value=None), \ + patch.object(processor, "create_agent", return_value=agent): + assert processor.build_agent("q") is agent + + def test_stream_route_answers_401_without_pre_fetching(self): + from flask import Flask + + from docsgpt.api.answer.routes.stream import StreamResource + + app = Flask(__name__) + with app.test_request_context("/stream", method="POST", json={"question": "q", "agent_id": "a-1"}), \ + patch("docsgpt.api.answer.services.stream_processor.StreamProcessor.pre_fetch_tools") as pre, \ + patch("docsgpt.api.answer.services.stream_processor.StreamProcessor.initialize"): + from flask import request + + request.decoded_token = None + response = StreamResource().post() + assert response.status_code == 401 + pre.assert_not_called() diff --git a/tests/api/test_agent_team_sharing.py b/tests/api/test_agent_team_sharing.py index 9a264ca4..d843e5a1 100644 --- a/tests/api/test_agent_team_sharing.py +++ b/tests/api/test_agent_team_sharing.py @@ -74,6 +74,8 @@ def _patches(sub, repo, team_access, *, prompt_name="Resolved Prompt", source_de "docsgpt.api.user.agents.routes.resolve_source_details", return_value=source_details, ), + # Run state reads the grants live; covered in test_resource_states. + patch("docsgpt.api.user.agents.routes.resource_states", return_value=[]), ] @@ -188,6 +190,8 @@ class TestGetAgentResolvesNames: def _update_patches(sub, repo, team_access, can_access_mock): + from docsgpt.api.user.resource_access import SponsorPlan + return [ patch("docsgpt.app.handle_auth", return_value={"sub": sub}), patch("docsgpt.app.resolve_roles", return_value=["user"]), @@ -197,7 +201,7 @@ def _update_patches(sub, repo, team_access, can_access_mock): patch("docsgpt.api.user.agents.routes.can_access", can_access_mock), # Sponsor bookkeeping queries the (mocked) connection; covered with a # real database in tests/api/user/test_resource_sponsors.py. - patch("docsgpt.api.user.agents.routes.sponsors_after_save", return_value={}), + patch("docsgpt.api.user.agents.routes.plan_sponsors", return_value=SponsorPlan()), ] diff --git a/tests/api/test_connector_oauth_security.py b/tests/api/test_connector_oauth_security.py index f8b2e521..be44f725 100644 --- a/tests/api/test_connector_oauth_security.py +++ b/tests/api/test_connector_oauth_security.py @@ -9,6 +9,8 @@ from unittest.mock import MagicMock, patch import pytest from flask import Flask +from tests.connectors.conftest import _oauth_connectors_configured # noqa: F401,E402 (autouse) + @pytest.fixture def app(): @@ -21,7 +23,11 @@ def _patch_db(conn, module="docsgpt.api.connector.routes"): def _yield(): yield conn - with patch(f"{module}.db_session", _yield), patch(f"{module}.db_readonly", _yield): + with patch(f"{module}.db_session", _yield), patch(f"{module}.db_readonly", _yield), patch( + "docsgpt.api.connector.routes.db_readonly", _yield + ), patch("docsgpt.connectors.service.db_session", _yield), patch( + "docsgpt.connectors.service.db_readonly", _yield + ): yield diff --git a/tests/api/test_connector_routes_happy.py b/tests/api/test_connector_routes_happy.py index e1302399..a61a4503 100644 --- a/tests/api/test_connector_routes_happy.py +++ b/tests/api/test_connector_routes_happy.py @@ -14,6 +14,8 @@ from unittest.mock import MagicMock, patch import pytest from flask import Flask +from tests.connectors.conftest import _oauth_connectors_configured # noqa: F401,E402 (autouse) + @pytest.fixture def app(): @@ -30,6 +32,10 @@ def _patch_db(conn): "docsgpt.api.connector.routes.db_session", _yield ), patch( "docsgpt.api.connector.routes.db_readonly", _yield + ), patch( + "docsgpt.connectors.service.db_session", _yield + ), patch( + "docsgpt.connectors.service.db_readonly", _yield ): yield @@ -487,6 +493,8 @@ class TestConnectorSync: def test_returns_400_missing_source_id(self, app): from docsgpt.api.connector.routes import ConnectorSync + # The source's own connection is used when none is named, so only + # the source id is required. with app.test_request_context( "/api/connectors/sync", method="POST", json={"session_token": "y"} ): @@ -495,9 +503,9 @@ class TestConnectorSync: r = ConnectorSync().post() assert r.status_code == 400 - def test_owner_without_session_token_returns_400(self, app, pg_conn): - # The owner syncs with their own session token; team editors don't - # send one (their sync uses the owner's session). + def test_owner_without_a_connection_returns_400(self, app, pg_conn): + # The owner names a connection (or the source has its own); team + # editors don't send one (their sync uses the owner's connection). from docsgpt.api.connector.routes import ConnectorSync from docsgpt.storage.db.repositories.sources import SourcesRepository diff --git a/tests/api/test_pat_rules.py b/tests/api/test_pat_rules.py index a1bb0977..88e4205e 100644 --- a/tests/api/test_pat_rules.py +++ b/tests/api/test_pat_rules.py @@ -98,6 +98,8 @@ class TestClassification: ("/api/devices/pairings", "POST"), ("/api/connectors/auth", "GET"), ("/api/mcp_server/callback", "GET"), + ("/api/resource_settings", "PUT"), + ("/api/sources//wiki/settings", "PUT"), ], ) def test_sensitive_routes_are_never_token_reachable(self, rule, method): diff --git a/tests/api/user/agents/test_roles_access.py b/tests/api/user/agents/test_roles_access.py index 022e7130..d6110ce1 100644 --- a/tests/api/user/agents/test_roles_access.py +++ b/tests/api/user/agents/test_roles_access.py @@ -283,7 +283,12 @@ class TestUpdateAgent: denied = self._put(app, pg_conn, agent_id, EDITOR, {"tools": [foreign_tool]}) assert _status(denied) == 403 assert _status(self._put(app, pg_conn, agent_id, OWNER, {"tools": [owner_tool]})) == 200 - ok = self._put(app, pg_conn, agent_id, EDITOR, {"tools": [owner_tool, shared_tool]}) + # The owner can't use the shared tool, so the editor sponsors it, + # which takes their confirmation. + unconfirmed = self._put(app, pg_conn, agent_id, EDITOR, {"tools": [owner_tool, shared_tool]}) + assert _status(unconfirmed) == 409 + ok = self._put(app, pg_conn, agent_id, EDITOR, {"tools": [owner_tool, shared_tool], + "confirm_sponsor": [f"tool:{shared_tool}"]}) assert _status(ok) == 200 set_settings(pg_conn, "tool", shared_tool, {"viewers_can_use_in_agents": False}, STRANGER) diff --git a/tests/api/user/sources/test_routes.py b/tests/api/user/sources/test_routes.py index ecfb386c..70d4e07c 100644 --- a/tests/api/user/sources/test_routes.py +++ b/tests/api/user/sources/test_routes.py @@ -600,6 +600,38 @@ class TestSyncSource: assert response.status_code == 200 assert response.json["task_id"] == "task-123" + def test_syncs_with_the_sources_connection(self, app, pg_conn): + """A source synced from a connection (S3, GitHub) reads with that connection's keys.""" + from sqlalchemy import text + + from docsgpt.api.user.sources.routes import SyncSource + + user = "u-conn-sync" + cid = str(pg_conn.execute(text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status) " + "VALUES (:u, 'github', 'github', 'api_key', 'connected') RETURNING id" + ), {"u": user}).scalar()) + src = _seed_source( + pg_conn, user, name="repo", type="github", + remote_data=json.dumps({"repo_url": "acme/private"}), + ) + pg_conn.execute(text("UPDATE sources SET connection_id = CAST(:c AS uuid) WHERE id = :s"), + {"c": cid, "s": src["id"]}) + + with _patch_db(pg_conn), patch( + "docsgpt.api.user.sources.routes.sync_source.delay", + return_value=MagicMock(id="task-conn"), + ) as mock_delay, app.test_request_context( + "/api/sync_source", method="POST", json={"source_id": str(src["id"])}, + ): + from flask import request + request.decoded_token = {"sub": user} + response = SyncSource().post() + + assert response.status_code == 200 + assert mock_delay.call_args.kwargs["connection_id"] == cid + assert mock_delay.call_args.kwargs["source_data"] == "acme/private" + def test_normalizes_dict_remote_data_before_dispatch(self, app, pg_conn): """The route must hand the sync task the normalized URL string.""" from docsgpt.api.user.sources.routes import SyncSource diff --git a/tests/api/user/sources/test_source_roles.py b/tests/api/user/sources/test_source_roles.py index ed5ec259..b80a2b6a 100644 --- a/tests/api/user/sources/test_source_roles.py +++ b/tests/api/user/sources/test_source_roles.py @@ -52,6 +52,7 @@ ROUTES = "docsgpt.api.user.sources.routes" CHUNKS = "docsgpt.api.user.sources.chunks" UPLOAD = "docsgpt.api.user.sources.upload" CONNECTOR = "docsgpt.api.connector.routes" +CONNECTIONS = "docsgpt.connectors.service" def _shared_source(pg_conn, **kwargs): @@ -456,13 +457,14 @@ def _owner_session(pg_conn, token="st-owner", token_info=None): "token_info": token_info or {"access_token": "a", "refresh_token": "r"}, }, ) + return str(row["id"]) def _connector_sync(app, pg_conn, user, body): from docsgpt.api.connector.routes import ConnectorSync delay = MagicMock(return_value=MagicMock(id="t-sync")) - with _patch_db(pg_conn, CONNECTOR), patch( + with _patch_db(pg_conn, CONNECTOR, CONNECTIONS), patch( f"{CONNECTOR}.ingest_connector_task.delay", delay ): resp = _call(app, user, "/api/connectors/sync", ConnectorSync().post, method="POST", json=body) @@ -472,12 +474,12 @@ def _connector_sync(app, pg_conn, user, body): class TestConnectorSync: def test_editor_syncs_with_owner_session(self, app, pg_conn): sid = _connector_source(pg_conn) - _owner_session(pg_conn) + owner_connection = _owner_session(pg_conn) resp, delay = _connector_sync(app, pg_conn, EDITOR, {"source_id": sid}) assert resp.status_code == 200 kwargs = delay.call_args.kwargs assert kwargs["user"] == OWNER - assert kwargs["session_token"] == "st-owner" + assert kwargs["connection_id"] == owner_connection def test_editor_cannot_substitute_own_session(self, app, pg_conn): from docsgpt.storage.db.repositories.connector_sessions import ( @@ -485,15 +487,16 @@ class TestConnectorSync: ) sid = _connector_source(pg_conn) - _owner_session(pg_conn) + owner_connection = _owner_session(pg_conn) repo = ConnectorSessionsRepository(pg_conn) row = repo.upsert(EDITOR, "google_drive", status="authorized") repo.update(str(row["id"]), {"session_token": "st-editor", "token_info": {"access_token": "x"}}) resp, delay = _connector_sync( - app, pg_conn, EDITOR, {"source_id": sid, "session_token": "st-editor"} + app, pg_conn, EDITOR, + {"source_id": sid, "session_token": "st-editor", "connection_id": str(row["id"])}, ) assert resp.status_code == 200 - assert delay.call_args.kwargs["session_token"] == "st-owner" + assert delay.call_args.kwargs["connection_id"] == owner_connection def test_owner_session_missing_returns_409(self, app, pg_conn): sid = _connector_source(pg_conn) @@ -512,9 +515,50 @@ class TestConnectorSync: def test_owner_still_uses_own_token(self, app, pg_conn): sid = _connector_source(pg_conn) - _owner_session(pg_conn) + owner_connection = _owner_session(pg_conn) resp, delay = _connector_sync( app, pg_conn, OWNER, {"source_id": sid, "session_token": "st-owner"} ) assert resp.status_code == 200 assert delay.call_args.kwargs["user"] == OWNER + assert delay.call_args.kwargs["connection_id"] == owner_connection + + def test_editor_syncs_with_the_sources_own_connection(self, app, pg_conn): + from sqlalchemy import text + + from docsgpt.security.encryption import encrypt_json + from docsgpt.storage.db.repositories.connector_sessions import ( + ConnectorSessionsRepository, + ) + + sid = _connector_source(pg_conn) + _owner_session(pg_conn) + second = str(ConnectorSessionsRepository(pg_conn).create( + OWNER, "google_drive", connector_key="google_drive", auth_kind="oauth", + account_label="second@example.com", + encrypted_credentials=encrypt_json({"token_info": {"access_token": "b", "refresh_token": "r"}}, OWNER), + )["id"]) + pg_conn.execute( + text("UPDATE sources SET connection_id = CAST(:c AS uuid) WHERE id = CAST(:s AS uuid)"), + {"c": second, "s": sid}, + ) + resp, delay = _connector_sync(app, pg_conn, EDITOR, {"source_id": sid}) + assert resp.status_code == 200 + assert delay.call_args.kwargs["connection_id"] == second + + def test_editor_sync_refuses_a_signed_out_source_connection(self, app, pg_conn): + from sqlalchemy import text + + sid = _connector_source(pg_conn) + connection = _owner_session(pg_conn) + pg_conn.execute( + text("UPDATE connector_sessions SET status = 'reconnect_needed' WHERE id = CAST(:c AS uuid)"), + {"c": connection}, + ) + pg_conn.execute( + text("UPDATE sources SET connection_id = CAST(:c AS uuid) WHERE id = CAST(:s AS uuid)"), + {"c": connection, "s": sid}, + ) + resp, delay = _connector_sync(app, pg_conn, EDITOR, {"source_id": sid}) + assert resp.status_code == 409 + delay.assert_not_called() diff --git a/tests/api/user/sources/test_upload_idempotency.py b/tests/api/user/sources/test_upload_idempotency.py index f5b069bc..fe251ff4 100644 --- a/tests/api/user/sources/test_upload_idempotency.py +++ b/tests/api/user/sources/test_upload_idempotency.py @@ -24,6 +24,10 @@ def _patch_db(conn): "docsgpt.api.user.sources.upload.db_session", _yield ), patch( "docsgpt.api.user.sources.upload.db_readonly", _yield + ), patch( + "docsgpt.connectors.service.db_session", _yield + ), patch( + "docsgpt.connectors.service.db_readonly", _yield ): yield diff --git a/tests/api/user/sources/test_wiki_routes.py b/tests/api/user/sources/test_wiki_routes.py index bf4df9aa..3acd8660 100644 --- a/tests/api/user/sources/test_wiki_routes.py +++ b/tests/api/user/sources/test_wiki_routes.py @@ -606,3 +606,114 @@ class TestWikiPageEdit: assert response.status_code == 403 mock_reembed.assert_not_called() + + +def _settings_call(app, pg_conn, sid, user, method="GET", body=None): + from docsgpt.api.user.sources.routes import WikiSettings + + with _patch_db(pg_conn), app.test_request_context( + f"/api/sources/{sid}/wiki/settings", method=method, json=body + ): + from flask import request + request.decoded_token = {"sub": user} if user else None + resource = WikiSettings() + return resource.get(sid) if method == "GET" else resource.put(sid) + + +class TestWikiSettings: + def _wiki(self, pg_conn, owner, kind="wiki"): + from docsgpt.storage.db.repositories.sources import SourcesRepository + + src = SourcesRepository(pg_conn).create( + "wiki", user_id=owner, type=kind, config={"kind": kind} + ) + return str(src["id"]) + + def _stored(self, pg_conn, sid): + from docsgpt.storage.db.repositories.sources import SourcesRepository + + return SourcesRepository(pg_conn).get_by_id(sid)["wiki_outside_edits"] + + def test_returns_401_unauthenticated(self, app, pg_conn): + assert _settings_call(app, pg_conn, str(uuid.uuid4()), None).status_code == 401 + assert _settings_call(app, pg_conn, str(uuid.uuid4()), None, "PUT", {}).status_code == 401 + + def test_defaults_to_off(self, app, pg_conn): + sid = self._wiki(pg_conn, "alice-ws") + response = _settings_call(app, pg_conn, sid, "alice-ws") + assert response.status_code == 200 + assert response.json["allow_outside_edits"] is False + assert "manage_settings" in response.json["allowed_actions"] + + def test_owner_turns_it_on_and_off(self, app, pg_conn): + sid = self._wiki(pg_conn, "alice-ws-on") + response = _settings_call( + app, pg_conn, sid, "alice-ws-on", "PUT", {"allow_outside_edits": True} + ) + assert response.status_code == 200 + assert response.json["allow_outside_edits"] is True + assert self._stored(pg_conn, sid) is True + assert _settings_call(app, pg_conn, sid, "alice-ws-on").json["allow_outside_edits"] is True + _settings_call(app, pg_conn, sid, "alice-ws-on", "PUT", {"allow_outside_edits": False}) + assert self._stored(pg_conn, sid) is False + + def test_change_is_audited(self, app, pg_conn): + from sqlalchemy import text + + sid = self._wiki(pg_conn, "alice-ws-audit") + _settings_call(app, pg_conn, sid, "alice-ws-audit", "PUT", {"allow_outside_edits": True}) + row = pg_conn.execute( + text("SELECT metadata FROM auth_events WHERE event = 'source.wiki_settings_updated' AND actor_id = :a"), + {"a": "alice-ws-audit"}, + ).fetchone() + assert row is not None + assert row[0]["source_id"] == sid and row[0]["allow_outside_edits"] is True + + @pytest.mark.parametrize("level", ["editor", "viewer"]) + def test_team_member_reads_but_cannot_change(self, app, pg_conn, level): + owner, member = f"alice-ws-{level}", f"bob-ws-{level}" + sid = self._wiki(pg_conn, owner) + _grant_team_access(pg_conn, owner, member, sid, level) + read = _settings_call(app, pg_conn, sid, member) + assert read.status_code == 200 + assert read.json["allow_outside_edits"] is False + assert "manage_settings" not in read.json["allowed_actions"] + response = _settings_call( + app, pg_conn, sid, member, "PUT", {"allow_outside_edits": True} + ) + assert response.status_code == 403 + assert self._stored(pg_conn, sid) is False + + def test_stranger_gets_404(self, app, pg_conn): + sid = self._wiki(pg_conn, "alice-ws-404") + assert _settings_call(app, pg_conn, sid, "eve-ws").status_code == 404 + response = _settings_call(app, pg_conn, sid, "eve-ws", "PUT", {"allow_outside_edits": True}) + assert response.status_code == 404 + assert self._stored(pg_conn, sid) is False + + @pytest.mark.parametrize("body", [{}, {"allow_outside_edits": "yes"}, {"allow_outside_edits": 1}]) + def test_bad_body_is_400(self, app, pg_conn, body): + sid = self._wiki(pg_conn, "alice-ws-400") + response = _settings_call(app, pg_conn, sid, "alice-ws-400", "PUT", body) + assert response.status_code == 400 + assert self._stored(pg_conn, sid) is False + + def test_not_a_wiki_is_400(self, app, pg_conn): + sid = self._wiki(pg_conn, "alice-ws-classic", kind="classic") + response = _settings_call( + app, pg_conn, sid, "alice-ws-classic", "PUT", {"allow_outside_edits": True} + ) + assert response.status_code == 400 + assert self._stored(pg_conn, sid) is False + + def test_no_row_updated_is_404(self, app, pg_conn): + sid = self._wiki(pg_conn, "alice-ws-gone") + with patch( + "docsgpt.storage.db.repositories.sources.SourcesRepository.set_wiki_outside_edits", + return_value=False, + ): + response = _settings_call( + app, pg_conn, sid, "alice-ws-gone", "PUT", {"allow_outside_edits": True} + ) + assert response.status_code == 404 + assert response.json["success"] is False diff --git a/tests/api/user/test_resource_sponsors.py b/tests/api/user/test_resource_sponsors.py index 7dff59f6..611a1909 100644 --- a/tests/api/user/test_resource_sponsors.py +++ b/tests/api/user/test_resource_sponsors.py @@ -20,10 +20,10 @@ from flask import Flask from docsgpt.api.user.resource_access import ( active_sponsor, agent_refs, + plan_sponsors, ref_principal, set_settings, sponsor_key, - sponsors_after_save, ) from docsgpt.storage.db.repositories.agents import AgentsRepository from docsgpt.storage.db.repositories.prompts import PromptsRepository @@ -39,12 +39,15 @@ from docsgpt.storage.db.repositories.workflows import WorkflowsRepository OWNER, EDITOR, VIEWER, OTHER = "sp-owner", "sp-editor", "sp-viewer", "sp-other" _DB_MODULES = ( + "docsgpt.agents.headless_runner", "docsgpt.api.user.agents.routes", "docsgpt.api.user.workflows.routes", "docsgpt.api.user.base", "docsgpt.agents.tool_executor", "docsgpt.api.answer.services.stream_processor", "docsgpt.agents.workflows.workflow_engine", + "docsgpt.connectors.resolve", + "docsgpt.connectors.service", "docsgpt.storage.db.session", ) @@ -120,6 +123,20 @@ def _editor_resources(conn): return tool, prompt, source +def _confirm(*refs): + """``confirm_sponsor`` for ``(type, id)`` pairs.""" + return [sponsor_key(t, i) for t, i in refs] + + +def sponsors_after_save(conn, holder_type, holder, owner_id, caller, refs, previous_refs=(), confirm_all=True): + """The stored map after a save that confirms every sponsorship it can.""" + refs = list(refs) + confirmed = [sponsor_key(t, i) for t, i in refs] if confirm_all else [] + plan = plan_sponsors(conn, holder_type, holder, owner_id, caller, refs, + previous_refs=previous_refs, confirmed=confirmed) + return plan.sponsors + + def _put(app, conn, agent_id, user, body): from docsgpt.api.user.agents.routes import UpdateAgent @@ -162,9 +179,17 @@ class TestSponsorsAfterSave: AgentsRepository(pg_conn).update_by_id( agent_id, {"tools": [tool], "resource_sponsors": {sponsor_key("tool", tool): EDITOR}} ) - out = sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, OWNER, [("tool", tool)]) + out = sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, OWNER, [("tool", tool)], + previous_refs=[("tool", tool)], confirm_all=False) assert out == {sponsor_key("tool", tool): EDITOR} + def test_unconfirmed_new_sponsorship_is_not_recorded(self, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + plan = plan_sponsors(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, EDITOR, [("tool", tool)]) + assert plan.sponsors == {} + assert plan.needs_confirmation == [("tool", tool)] + def test_removed_ref_drops_out(self, pg_conn): agent_id, _ = _agent(pg_conn) tool, _, _ = _editor_resources(pg_conn) @@ -173,12 +198,12 @@ class TestSponsorsAfterSave: ) assert sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, EDITOR, []) == {} - def test_another_editor_takes_over_when_sponsor_lost_access(self, pg_conn): + def test_another_editor_takes_over_only_by_confirming(self, pg_conn): agent_id, team_id = _agent(pg_conn) - # A tool shared with the whole team: both editors may use it in their agents. + # A tool the whole team may edit: both editors may sponsor it. tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) TeamMembersRepository(pg_conn).add_member(team_id, OTHER) - TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") TeamResourceGrantsRepository(pg_conn).grant( team_id, "agent", agent_id, OWNER, OWNER, access_level="editor", target_user_id=OTHER ) @@ -187,7 +212,12 @@ class TestSponsorsAfterSave: ) TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) - out = sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, OTHER, [("tool", tool)]) + refs = [("tool", tool)] + agent = _row(pg_conn, agent_id) + kept = sponsors_after_save(pg_conn, "agent", agent, OWNER, OTHER, refs, previous_refs=refs, + confirm_all=False) + assert kept == {sponsor_key("tool", tool): EDITOR} + out = sponsors_after_save(pg_conn, "agent", agent, OWNER, OTHER, refs, previous_refs=refs) assert out == {sponsor_key("tool", tool): OTHER} def test_dead_sponsor_record_kept_when_nobody_qualifies(self, pg_conn): @@ -197,7 +227,8 @@ class TestSponsorsAfterSave: agent_id, {"tools": [tool], "resource_sponsors": {sponsor_key("tool", tool): EDITOR}} ) TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) - out = sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, OWNER, [("tool", tool)]) + out = sponsors_after_save(pg_conn, "agent", _row(pg_conn, agent_id), OWNER, OWNER, [("tool", tool)], + previous_refs=[("tool", tool)]) assert out == {sponsor_key("tool", tool): EDITOR} @@ -245,12 +276,25 @@ class TestActiveSponsor: agent_id, team_id = _agent(pg_conn) tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) TeamResourceGrantsRepository(pg_conn).grant( - team_id, "tool", tool, OTHER, OTHER, target_user_id=EDITOR + team_id, "tool", tool, OTHER, OTHER, access_level="editor", target_user_id=EDITOR ) AgentsRepository(pg_conn).update_by_id( agent_id, {"resource_sponsors": {sponsor_key("tool", tool): EDITOR}} ) assert active_sponsor(pg_conn, "agent", _row(pg_conn, agent_id), "tool", tool) == EDITOR + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "tool", tool, target_user_id=EDITOR) + assert active_sponsor(pg_conn, "agent", _row(pg_conn, agent_id), "tool", tool) is None + + def test_use_only_sponsor_never_runs(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "tool", tool, OTHER, OTHER, target_user_id=EDITOR + ) + AgentsRepository(pg_conn).update_by_id( + agent_id, {"resource_sponsors": {sponsor_key("tool", tool): EDITOR}} + ) + assert active_sponsor(pg_conn, "agent", _row(pg_conn, agent_id), "tool", tool) is None set_settings(pg_conn, "tool", tool, {"viewers_can_use_in_agents": False}, OTHER) assert active_sponsor(pg_conn, "agent", _row(pg_conn, agent_id), "tool", tool) is None @@ -270,7 +314,8 @@ class TestAgentRoutes: agent_id, _ = _agent(pg_conn) tool, prompt, source = _editor_resources(pg_conn) resp = _put(app, pg_conn, agent_id, EDITOR, - {"tools": [tool], "prompt_id": prompt, "source": source}) + {"tools": [tool], "prompt_id": prompt, "source": source, + "confirm_sponsor": _confirm(("tool", tool), ("prompt", prompt), ("source", source))}) assert _status(resp) == 200 row = _row(pg_conn, agent_id) assert row["resource_sponsors"] == { @@ -283,14 +328,16 @@ class TestAgentRoutes: def test_owner_save_does_not_wipe_sponsors(self, app, pg_conn): agent_id, _ = _agent(pg_conn) tool, _, _ = _editor_resources(pg_conn) - assert _status(_put(app, pg_conn, agent_id, EDITOR, {"tools": [tool]})) == 200 + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 assert _status(_put(app, pg_conn, agent_id, OWNER, {"name": "Renamed", "tools": [tool]})) == 200 assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} def test_detaching_clears_sponsor(self, app, pg_conn): agent_id, _ = _agent(pg_conn) tool, _, _ = _editor_resources(pg_conn) - assert _status(_put(app, pg_conn, agent_id, EDITOR, {"tools": [tool]})) == 200 + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 assert _status(_put(app, pg_conn, agent_id, EDITOR, {"tools": []})) == 200 assert _row(pg_conn, agent_id)["resource_sponsors"] == {} @@ -301,12 +348,15 @@ class TestAgentRoutes: agent_id, team_id = _agent(pg_conn) tool, _, _ = _editor_resources(pg_conn) UsersRepository(pg_conn).upsert(EDITOR, email="bob@example.com") - assert _status(_put(app, pg_conn, agent_id, EDITOR, {"tools": [tool]})) == 200 + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 path = f"/api/get_agent?id={agent_id}" owner_view = _call(app, pg_conn, GetAgent, "get", path, OWNER).get_json() assert owner_view["resource_sponsors"] == [ - {"type": "tool", "id": tool, "user_id": EDITOR, "label": "bob@example.com", "active": True} + {"key": f"tool:{tool}", "type": "tool", "id": tool, "name": "api_tool", "user_id": EDITOR, + "label": "bob@example.com", "state": "active", "reason": None, "active": True, + "can_confirm": False} ] viewer_view = _call(app, pg_conn, GetAgent, "get", path, VIEWER).get_json() assert viewer_view["resource_sponsors"] == [] @@ -326,7 +376,8 @@ class TestRunTime: agent_id, team_id = _agent(pg_conn) tool, prompt, source = _editor_resources(pg_conn) resp = _put(app, pg_conn, agent_id, EDITOR, - {"tools": [tool], "prompt_id": prompt, "source": source}) + {"tools": [tool], "prompt_id": prompt, "source": source, + "confirm_sponsor": _confirm(("tool", tool), ("prompt", prompt), ("source", source))}) assert _status(resp) == 200 return _row(pg_conn, agent_id), team_id, tool, prompt, source @@ -374,6 +425,40 @@ class TestRunTime: data = processor._get_data_from_api_key(agent["key"]) assert [s["id"] for s in data["sources"]] == [source, shared] + def test_headless_run_retrieves_from_every_agent_source(self, app, pg_conn, monkeypatch): + """A scheduled or webhook run searches the same sources a chat does.""" + from unittest.mock import MagicMock + + from docsgpt.agents import headless_runner as hr + + agent, team_id, _, _, source = self._sponsored_agent(app, pg_conn) + shared = str(SourcesRepository(pg_conn).create("shared-src", user_id=OTHER)["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OWNER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "source", shared, OTHER, OTHER) + foreign = str(SourcesRepository(pg_conn).create("foreign-src", user_id="sp-stranger")["id"]) + AgentsRepository(pg_conn).update_by_id(str(agent["id"]), {"extra_source_ids": [shared, foreign]}) + + searched = [] + retriever = MagicMock(search=MagicMock(return_value=[])) + + def _create_retriever(cls, *_args, **kwargs): + searched.append(kwargs["source"].get("active_docs")) + return retriever + + run_agent = MagicMock(gen=MagicMock(return_value=iter([{"answer": "ok"}]))) + run_agent.llm.token_usage = {} + monkeypatch.setattr(hr.RetrieverCreator, "create_retriever", classmethod(_create_retriever)) + monkeypatch.setattr(hr, "ToolExecutor", lambda *a, **kw: MagicMock(headless_denials=[])) + monkeypatch.setattr(hr.AgentCreator, "create_agent", classmethod(lambda cls, *a, **kw: run_agent)) + monkeypatch.setattr(hr.QuotaService, "check", lambda *a, **kw: None) + with _patch_db(pg_conn), \ + patch("docsgpt.core.model_utils.validate_model_id", return_value=True), \ + patch("docsgpt.core.model_utils.get_provider_from_model_id", return_value="openai"), \ + patch("docsgpt.core.model_utils.get_api_key_for_provider", return_value="k"), \ + patch("docsgpt.utils.calculate_doc_token_budget", return_value=1000): + hr.run_agent_headless(_row(pg_conn, str(agent["id"])), "q") + assert searched == [[source, shared]] + def test_search_service_authorizes_sponsored_source(self, app, pg_conn): from docsgpt.services.search_service import _authorized_source_ids @@ -384,15 +469,101 @@ class TestRunTime: assert _authorized_source_ids(pg_conn, agent, [source]) == [] +class TestToolPrefetch: + """Prompt tool pre-fetch runs the agent's tools, as the agent run resolves them.""" + + def _prefetched(self, pg_conn, agent_id, caller, required, **flags): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + from docsgpt.core.settings import settings + + agent = _row(pg_conn, agent_id) + processor = StreamProcessor({"agent_id": agent_id}, {"sub": caller}) + processor.agent_id = agent_id + processor.agent_config = {"user_api_key": agent["key"], **flags} + processor._required_tool_actions = required + fetched = {} + + def _fake_fetch(tool_doc, required_actions): + fetched[str(tool_doc["id"])] = required_actions + return {"ok": True} + + with _patch_db(pg_conn), patch.object(settings, "ENABLE_TOOL_PREFETCH", True), \ + patch.object(processor, "_fetch_tool_data", _fake_fetch): + processor.pre_fetch_tools() + return fetched + + def _tool(self, pg_conn, user, name, actions, **extra): + return str(UserToolsRepository(pg_conn).create(user, name, actions=actions, **extra)["id"]) + + def test_prefetch_runs_the_agents_tools_not_the_callers(self, pg_conn): + agent_id, _ = _agent(pg_conn) + owner_tool = self._tool(pg_conn, OWNER, "read_webpage", [{"name": "fetch", "active": True}]) + self._tool(pg_conn, VIEWER, "cryptoprice", [{"name": "price", "active": True}]) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [owner_tool]}) + required = {"read_webpage": {None}, "cryptoprice": {None}} + assert self._prefetched(pg_conn, agent_id, OWNER, required) == {owner_tool: {None}} + assert list(self._prefetched(pg_conn, agent_id, VIEWER, required)) == [owner_tool] + + def test_someone_elses_tool_prefetches_only_what_runs_unasked(self, pg_conn): + from sqlalchemy import text + + agent_id, _ = _agent(pg_conn) + api = self._tool(pg_conn, OWNER, "read_webpage", [ + {"name": "fetch", "active": True}, + {"name": "post", "active": True, "require_approval": True}, + ]) + connection = str(pg_conn.execute(text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status) " + "VALUES (:u, 'telegram', 'telegram', 'api_key', 'connected') RETURNING id" + ), {"u": OWNER}).scalar()) + connected = self._tool(pg_conn, OWNER, "telegram", [{"name": "telegram_get_updates", "active": True}], + connection_id=connection) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [api, connected]}) + required = {"read_webpage": {None}, "telegram": {None}} + + assert self._prefetched(pg_conn, agent_id, OWNER, required) == {api: {None}, connected: {None}} + # Nobody is asked during pre-fetch: a teammate gets no approval-gated + # action and nothing on the owner's connected account. + assert self._prefetched(pg_conn, agent_id, VIEWER, required) == {api: {"fetch"}} + + + def test_tools_that_decide_approval_live_are_not_prefetched_for_others(self, pg_conn): + agent_id, _ = _agent(pg_conn) + device = self._tool(pg_conn, OWNER, "remote_device", [{"name": "run_command", "active": True}]) + code = self._tool(pg_conn, OWNER, "code_executor", [{"name": "execute_code", "active": True}]) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [device, code]}) + required = {"remote_device": {None}, "code_executor": {None}} + assert self._prefetched(pg_conn, agent_id, OWNER, required) == {device: {None}, code: {None}} + assert self._prefetched(pg_conn, agent_id, VIEWER, required) == {} + + def test_api_key_callers_prefetch_like_someone_else(self, pg_conn): + """A widget or API run carries the owner's id, but the caller is not the owner.""" + agent_id, _ = _agent(pg_conn) + key = {"type": "object", "properties": {"X-Key": {"type": "string", "value": "", "has_value": True}}} + api = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool", config={"actions": { + "status": {"url": "https://x.test/s", "method": "GET", "active": True, "headers": key}, + "notify": {"url": "https://x.test/n", "method": "POST", "active": True, "headers": key}, + }})["id"]) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [api]}) + required = {"api_tool": {None}} + assert self._prefetched(pg_conn, agent_id, OWNER, required) == {api: {None}} + # Writes on the owner's credentials never pre-fetch for them. + assert self._prefetched(pg_conn, agent_id, OWNER, required, external_api_caller=True) == {api: {"status"}} + assert self._prefetched(pg_conn, agent_id, VIEWER, required, public_link_caller=True) == {api: {"status"}} + + # --------------------------------------------------------------------------- # workflows # --------------------------------------------------------------------------- -def _wf_body(tool=None, source=None): - agent_cfg = {"agent_type": "classic", "system_prompt": "s", "tools": [tool] if tool else [], +def _wf_body(tool=None, source=None, tools=None, confirm=()): + if tools is None: + tools = [tool] if tool else [] + agent_cfg = {"agent_type": "classic", "system_prompt": "s", "tools": tools, "sources": [source] if source else []} return { + "confirm_sponsor": list(confirm), "name": "WF", "description": "d", "nodes": [ @@ -423,7 +594,8 @@ class TestWorkflows: def test_editor_node_resources_are_sponsored(self, app, pg_conn): wid, _ = self._setup(pg_conn) tool, _, source = _editor_resources(pg_conn) - resp = self._put(app, pg_conn, wid, EDITOR, _wf_body(tool, source)) + resp = self._put(app, pg_conn, wid, EDITOR, + _wf_body(tool, source, confirm=_confirm(("tool", tool), ("source", source)))) assert _status(resp) == 200, resp.get_json() row = WorkflowsRepository(pg_conn).get_by_id(wid) assert row["resource_sponsors"] == { @@ -439,7 +611,8 @@ class TestWorkflows: wid, team_id = self._setup(pg_conn) tool, _, source = _editor_resources(pg_conn) - assert _status(self._put(app, pg_conn, wid, EDITOR, _wf_body(tool, source))) == 200 + confirm = _confirm(("tool", tool), ("source", source)) + assert _status(self._put(app, pg_conn, wid, EDITOR, _wf_body(tool, source, confirm=confirm))) == 200 engine = WorkflowEngine.__new__(WorkflowEngine) engine.agent = SimpleNamespace( @@ -457,3 +630,95 @@ class TestWorkflows: TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) assert engine._node_tool_principals([tool]) == {} assert engine._authorized_node_sources([source]) == [] + + def _node_executor(self, monkeypatch, pg_conn, wid, caller, tools): + """Run one agent node of ``wid`` for ``caller``; return its agent's tool executor.""" + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.agents.workflows.node_agent import WorkflowNodeAgentFactory, _WorkflowNodeMixin + from docsgpt.agents.workflows.schemas import NodeType, Workflow, WorkflowGraph, WorkflowNode + from docsgpt.agents.workflows.workflow_engine import WorkflowEngine + + class _Base: + # Stands in for BaseAgent: the executor is built for the token's user. + def __init__(self, decoded_token=None, **_kwargs): + sub = (decoded_token or {}).get("sub") + self.tool_executor = ToolExecutor(user=sub, decoded_token=decoded_token) + + class _NodeAgent(_WorkflowNodeMixin, _Base): + def gen(self, _prompt): + yield {"answer": "ok"} + + built = [] + + def _create(agent_type, **kwargs): + built.append(_NodeAgent(**kwargs)) + return built[-1] + + monkeypatch.setattr(WorkflowNodeAgentFactory, "create", staticmethod(_create)) + monkeypatch.setattr("docsgpt.core.model_utils.get_api_key_for_provider", lambda _name: None) + agent = SimpleNamespace( + endpoint="stream", llm_name="openai", model_id="gpt-4o-mini", api_key="k", chat_history=[], + decoded_token={"sub": caller}, user=caller, workflow_owner=OWNER, + _resolve_owner_id=lambda: OWNER, workflow_row=WorkflowsRepository(pg_conn).get_by_id(wid), + ) + engine = WorkflowEngine(WorkflowGraph(workflow=Workflow(name="wf"), nodes=[], edges=[]), agent) + engine.state["query"] = "q" + node = WorkflowNode( + id="a1", workflow_id=wid, type=NodeType.AGENT, title="A", position={"x": 0, "y": 0}, + config={"agent_type": "classic", "system_prompt": "s", "tools": tools}, + ) + with _patch_db(pg_conn): + list(engine._execute_agent_node(node)) + return built[0].tool_executor + + @pytest.mark.parametrize("caller", [OWNER, VIEWER, OTHER]) + def test_node_tools_resolve_as_the_owner_whoever_runs_it(self, app, pg_conn, monkeypatch, caller): + """A teammate or public-link user gets the node tools the owner gets.""" + wid, _ = self._setup(pg_conn) + owner_tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + editor_tool, _, _ = _editor_resources(pg_conn) + assert _status(self._put(app, pg_conn, wid, EDITOR, + _wf_body(editor_tool, confirm=_confirm(("tool", editor_tool))))) == 200 + tools = [owner_tool, editor_tool] + assert _status(self._put(app, pg_conn, wid, OWNER, _wf_body(tools=tools))) == 200 + + executor = self._node_executor(monkeypatch, pg_conn, wid, caller, tools) + with _patch_db(pg_conn): + resolved = executor.get_tools() + assert {tid: row["user_id"] for tid, row in resolved.items()} == {owner_tool: OWNER, editor_tool: EDITOR} + # Whoever runs the node stays the invoker: member-mode connections are theirs. + assert executor.user == caller + + def test_caller_only_tool_is_not_in_the_node(self, app, pg_conn, monkeypatch): + """A tool only the caller can use never resolves: nodes are the owner's.""" + wid, _ = self._setup(pg_conn) + viewer_tool = str(UserToolsRepository(pg_conn).create(VIEWER, "api_tool")["id"]) + executor = self._node_executor(monkeypatch, pg_conn, wid, VIEWER, [viewer_tool]) + with _patch_db(pg_conn): + assert executor.get_tools() == {} + + @pytest.mark.parametrize(("mode", "account", "delegated"), [("member", VIEWER, False), ("owner", OWNER, True)]) + def test_node_connection_follows_the_credential_mode(self, app, pg_conn, monkeypatch, mode, account, delegated): + from sqlalchemy import text + + def _connection(user): + return str(pg_conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "account_label) VALUES (:u, 'telegram', 'telegram', 'api_key', 'connected', :u) RETURNING id" + ), + {"u": user}, + ).scalar()) + + wid, _ = self._setup(pg_conn) + _connection(VIEWER) + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(OWNER), credential_mode=mode, + )["id"]) + assert _status(self._put(app, pg_conn, wid, OWNER, _wf_body(tool))) == 200 + + executor = self._node_executor(monkeypatch, pg_conn, wid, VIEWER, [tool]) + with _patch_db(pg_conn): + resolved = executor._resolve_connection(executor.get_tools()[tool]) + assert resolved.row["user_id"] == account + assert resolved.delegated is delegated diff --git a/tests/api/user/test_resource_states.py b/tests/api/user/test_resource_states.py new file mode 100644 index 00000000..80eab955 --- /dev/null +++ b/tests/api/user/test_resource_states.py @@ -0,0 +1,1038 @@ +"""Run state of every resource attached to an agent or a workflow's nodes. + +An agent (or workflow) runs its attached tools, sources and prompt as its +owner, or as the editor who sponsored them. When one stops being usable the +run drops it (a prompt falls back to the default) and the edit page says why: +``resource_states`` on the agent and workflow reads. The state comes from the +same checks the run uses, so a resource marked stopped is never used by a run +and one marked active is. Uses real repositories on ``pg_conn``. +""" + +from __future__ import annotations + +import logging +import uuid + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.api.user.resource_access import ( + REASON_CANNOT_EDIT_HOLDER, + REASON_CANNOT_EDIT_RESOURCE, + REASON_CONNECTION_NEEDS_RECONNECT, + REASON_CONNECTION_REMOVED, + REASON_CONNECTOR_DISABLED, + REASON_DELETED, + REASON_OWNER_LOST_ACCESS, + agent_refs, + ref_access, + resource_states, +) +from docsgpt.storage.db.repositories.agents import AgentsRepository +from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository +from docsgpt.storage.db.repositories.prompts import PromptsRepository +from docsgpt.storage.db.repositories.sources import SourcesRepository +from docsgpt.storage.db.repositories.team_members import TeamMembersRepository +from docsgpt.storage.db.repositories.team_resource_grants import ( + TeamResourceGrantsRepository, +) +from docsgpt.storage.db.repositories.user_tools import UserToolsRepository +from docsgpt.storage.db.repositories.workflows import WorkflowsRepository +from tests.api.user.test_resource_sponsors import ( + EDITOR, + OTHER, + OWNER, + VIEWER, + _agent, + _call, + _confirm, + _editor_resources, + _patch_db, + _put, + _row, + _status, + _wf_body, +) + + +@pytest.fixture +def app(): + return Flask(__name__) + + +def _body(resp) -> dict: + return resp[0] if isinstance(resp, tuple) else resp.get_json() + + +def _by_key(states) -> dict: + return {s["key"]: s for s in states} + + +def _team_source(conn, team_id, level="viewer"): + """OTHER's source, shared with the agent's team (OWNER a member).""" + if not TeamMembersRepository(conn).is_member(OWNER, team_id): + TeamMembersRepository(conn).add_member(team_id, OWNER) + if not TeamMembersRepository(conn).is_member(OTHER, team_id): + TeamMembersRepository(conn).add_member(team_id, OTHER) + source = str(SourcesRepository(conn).create("team-src", user_id=OTHER)["id"]) + TeamResourceGrantsRepository(conn).grant(team_id, "source", source, OTHER, OTHER, access_level=level) + return source + + +def _connection(conn, user=OWNER, status="connected", provider="telegram"): + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "account_label) VALUES (:u, :p, :p, 'api_key', :s, :u) RETURNING id" + ), + {"u": user, "p": provider, "s": status}, + ).scalar()) + + +def _states(conn, agent_id, viewer=OWNER): + agent = _row(conn, agent_id) + return _by_key(resource_states(conn, "agent", agent, agent_refs(agent), viewer)) + + +def _get_agent(app, conn, agent_id, user): + from docsgpt.api.user.agents.routes import GetAgent + + return _call(app, conn, GetAgent, "get", f"/api/get_agent?id={agent_id}", user).get_json() + + +# --------------------------------------------------------------------------- +# Reasons +# --------------------------------------------------------------------------- + + +class TestReasons: + def test_owned_resources_are_active(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + source = str(SourcesRepository(pg_conn).create("mine", user_id=OWNER)["id"]) + prompt = str(PromptsRepository(pg_conn).create(OWNER, "p", "x")["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool], source_id=source, prompt_id=prompt) + states = _states(pg_conn, agent_id) + assert {k: s["state"] for k, s in states.items()} == { + f"tool:{tool}": "active", f"source:{source}": "active", f"prompt:{prompt}": "active", + } + assert all(s["reason"] is None for s in states.values()) + + def test_owner_lost_team_grant(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [source]}) + assert _states(pg_conn, agent_id)[f"source:{source}"]["state"] == "active" + + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + state = _states(pg_conn, agent_id)[f"source:{source}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_OWNER_LOST_ACCESS) + # The owner is told to ask the source's owner, but not who that is: + # they can no longer see the source. + assert state["contact"] is None + assert state["contact_role"] == "resource_owner" + assert state["name"] == "team-src" + + def test_deleted_tool(self, pg_conn): + """A deleted source or prompt leaves the agent by itself (FK, trigger); a tool id stays.""" + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + UserToolsRepository(pg_conn).delete(tool, OWNER) + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_DELETED) + assert state["contact"] is None + assert state["name"] is None + + def test_sponsor_lost_the_agent(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + assert _states(pg_conn, agent_id)[f"tool:{tool}"]["state"] == "active" + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_CANNOT_EDIT_HOLDER) + assert state["sponsor"] == {"user_id": EDITOR, "label": EDITOR} + + def test_sponsor_lost_the_resource(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "tool", tool, OTHER, OTHER, access_level="editor", target_user_id=EDITOR + ) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "tool", tool, target_user_id=EDITOR) + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["reason"] == REASON_CANNOT_EDIT_RESOURCE + + def test_connection_needs_reconnect(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="reconnect_needed"), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_CONNECTION_NEEDS_RECONNECT) + assert state["can_reconnect"] is True + assert state["connection"]["connector_key"] == "telegram" + assert state["connection"]["id"] + assert state["contact"] is None + + def test_disconnected_account_needs_reconnect(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="disconnected"), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["reason"] == REASON_CONNECTION_NEEDS_RECONNECT + + def test_editor_is_told_whom_to_ask_to_reconnect(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="reconnect_needed"), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id, viewer=EDITOR)[f"tool:{tool}"] + assert state["can_reconnect"] is False + # The agent's owner is someone the editor knows. + assert state["contact"] == {"user_id": OWNER, "label": OWNER} + # Only the account's owner gets its connection id and own name. + assert state["connection"] == {"id": None, "connector_key": "telegram", "name": "Telegram"} + + def test_connection_removed_but_tool_kept(self, pg_conn): + from docsgpt.connectors import service + + cid = _connection(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "telegram", connection_id=cid)["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:id AS uuid)"), + {"id": cid}).mappings().one() + service.remove_connection(pg_conn, dict(row), tools="keep") + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_CONNECTION_REMOVED) + assert state["connection"]["name"] == "Telegram" + assert state["can_reconnect"] is False + + def test_connector_disabled_by_admin(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + assert _states(pg_conn, agent_id)[f"tool:{tool}"]["state"] == "active" + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["reason"] == REASON_CONNECTOR_DISABLED + + def test_prompt_the_owner_lost(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + prompt = str(PromptsRepository(pg_conn).create(OTHER, "theirs", "x")["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "prompt", prompt, OTHER, OTHER) + TeamMembersRepository(pg_conn).add_member(team_id, OWNER) + AgentsRepository(pg_conn).update_by_id(agent_id, {"prompt_id": prompt}) + assert _states(pg_conn, agent_id)[f"prompt:{prompt}"]["state"] == "active" + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "prompt", prompt) + assert _states(pg_conn, agent_id)[f"prompt:{prompt}"]["reason"] == REASON_OWNER_LOST_ACCESS + + def test_presets_and_builtin_tools_are_not_listed(self, pg_conn): + from docsgpt.agents.default_tools import loaded_builtin_agent_tools, synthesize_builtin_agent_tool + + builtin = next(iter(loaded_builtin_agent_tools()), None) + tools = [str(synthesize_builtin_agent_tool(builtin)["id"])] if builtin else [] + agent_id, _ = _agent(pg_conn, tools=tools) + assert _states(pg_conn, agent_id) == {} + + +class TestCredentials: + """Whose account or credentials each running tool uses, for the share dialog.""" + + _SEND = {"name": "send_message", "active": True} + _READ = {"name": "get_updates", "active": True} + + def test_owner_mode_connection_names_the_account_holder(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), actions=[self._SEND, self._READ], + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["state"] == "active" + assert state["credential_mode"] == "owner" + assert state["account"] == {"user_id": OWNER, "label": OWNER} + assert state["connection"]["name"] == "Telegram" + # A running tool's connection id is never sent. + assert state["connection"]["id"] is None + assert state["owner_credential_writes"] == ["send_message"] + assert state["writes_allowed"] is True + + def test_member_mode_connection_has_no_account_holder(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), credential_mode="member", + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["credential_mode"] == "member" + assert state["account"] is None + assert state["connection"]["connector_key"] == "telegram" + + def test_admin_forced_mode_wins(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + ConnectorPoliciesRepository(pg_conn).upsert("telegram", credential_mode="member") + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["credential_mode"] == "member" + assert state["account"] is None + + def test_teammates_owner_mode_tool_uses_their_account(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + for member in (OWNER, OTHER): + if not TeamMembersRepository(pg_conn).is_member(member, team_id): + TeamMembersRepository(pg_conn).add_member(team_id, member) + tool = str(UserToolsRepository(pg_conn).create( + OTHER, "telegram", connection_id=_connection(pg_conn, user=OTHER), + )["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [tool]}) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["state"] == "active" + assert state["account"] == {"user_id": OTHER, "label": OTHER} + + def test_tool_without_stored_credentials(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + source = str(SourcesRepository(pg_conn).create("mine", user_id=OWNER)["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool], source_id=source) + states = _states(pg_conn, agent_id) + assert states[f"tool:{tool}"]["credential_mode"] is None + assert states[f"tool:{tool}"]["account"] is None + assert states[f"tool:{tool}"]["connection"] is None + assert states[f"tool:{tool}"]["owner_credential_writes"] == [] + assert states[f"source:{source}"]["owner_credential_writes"] == [] + assert states[f"source:{source}"]["credential_mode"] is None + + def test_api_tool_write_that_sends_a_saved_key(self, pg_conn): + actions = { + "create_ticket": { + "method": "POST", + "headers": {"properties": {"Authorization": {"has_value": True}}}, + }, + "list_tickets": { + "method": "GET", + "headers": {"properties": {"Authorization": {"has_value": True}}}, + }, + } + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "api_tool", config={"actions": actions}, + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["owner_credential_writes"] == ["create_ticket"] + # The saved key is its owner's, whoever runs it. + assert state["account"] == {"user_id": OWNER, "label": OWNER} + assert state["credential_mode"] is None + + def test_mcp_server_signed_in_by_a_teammate(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + for member in (OWNER, OTHER): + if not TeamMembersRepository(pg_conn).is_member(member, team_id): + TeamMembersRepository(pg_conn).add_member(team_id, member) + tool = str(UserToolsRepository(pg_conn).create( + OTHER, "mcp_tool", config={"server_url": "https://mcp.example.com", "auth_type": "bearer"}, + )["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [tool]}) + assert _states(pg_conn, agent_id)[f"tool:{tool}"]["account"] == {"user_id": OTHER, "label": OTHER} + + def test_account_holder_the_reader_shares_no_team_with_is_not_named(self, pg_conn): + """An editor outside the team that shares the tool learns only that it's someone else's.""" + from docsgpt.storage.db.repositories.teams import TeamsRepository + + agent_id, _ = _agent(pg_conn) + private = str(TeamsRepository(pg_conn).create("P", f"p-{uuid.uuid4().hex[:8]}", OTHER)["id"]) + TeamMembersRepository(pg_conn).add_member(private, OWNER) + if not TeamMembersRepository(pg_conn).is_member(OTHER, private): + TeamMembersRepository(pg_conn).add_member(private, OTHER) + tool = str(UserToolsRepository(pg_conn).create( + OTHER, "telegram", connection_id=_connection(pg_conn, user=OTHER), + )["id"]) + TeamResourceGrantsRepository(pg_conn).grant(private, "tool", tool, OTHER, OTHER) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [tool]}) + with _patch_db(pg_conn): + as_owner = _states(pg_conn, agent_id)[f"tool:{tool}"] + as_editor = _states(pg_conn, agent_id, viewer=EDITOR)[f"tool:{tool}"] + assert as_owner["account"] == {"user_id": OTHER, "label": OTHER} + assert as_editor["state"] == "active" + assert as_editor["account"] == {"user_id": None, "label": None} + + def test_admin_turned_writes_off(self, pg_conn): + from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository + from docsgpt.storage.db.repositories.connector_policies import allow_writes_key + + cid = _connection(pg_conn, provider="github") + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "mcp_tool", connection_id=cid, + config={"server_url": "https://api.githubcopilot.com/mcp/", "auth_type": "bearer"}, + actions=[{"name": "create_issue", "active": True}], + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + assert _states(pg_conn, agent_id)[f"tool:{tool}"]["writes_allowed"] is True + AppMetadataRepository(pg_conn).set(allow_writes_key("github"), "false") + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["state"] == "active" + assert state["writes_allowed"] is False + assert state["owner_credential_writes"] == [] + + +class TestRunsAs: + """Whose access a running item runs with: the owner's, or a live sponsor's.""" + + def test_sponsored_item_runs_as_its_sponsor(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["runs_as"] == {"user_id": EDITOR, "label": EDITOR} + + def test_owners_own_item_runs_as_the_owner(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + assert _states(pg_conn, agent_id)[f"tool:{tool}"]["runs_as"] is None + + def test_owner_who_gains_access_runs_it_even_with_a_sponsor_on_record(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + source = str(SourcesRepository(pg_conn).create("editor-src", user_id=EDITOR)["id"]) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"sources": [source], "confirm_sponsor": _confirm(("source", source))})) == 200 + TeamResourceGrantsRepository(pg_conn).grant(team_id, "source", source, EDITOR, EDITOR) + if not TeamMembersRepository(pg_conn).is_member(OWNER, team_id): + TeamMembersRepository(pg_conn).add_member(team_id, OWNER) + state = _states(pg_conn, agent_id)[f"source:{source}"] + assert state["state"] == "active" + assert state["runs_as"] is None + + +class TestTakeOver: + def test_editor_who_can_edit_the_item_may_take_over(self, pg_conn): + """An owner-lost resource the reading editor can edit is theirs to take over.""" + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [source]}) + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "source", source, OTHER, OTHER, access_level="editor", target_user_id=EDITOR + ) + assert _states(pg_conn, agent_id, viewer=EDITOR)[f"source:{source}"]["can_confirm"] is True + assert _states(pg_conn, agent_id, viewer=OWNER)[f"source:{source}"]["can_confirm"] is False + + def test_take_over_of_an_owner_lost_item_is_accepted(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [source]}) + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "source", source, OTHER, OTHER, access_level="editor", target_user_id=EDITOR + ) + resp = _put(app, pg_conn, agent_id, EDITOR, + {"sources": [source], "confirm_sponsor": _confirm(("source", source))}) + assert _status(resp) == 200, _body(resp) + assert _states(pg_conn, agent_id)[f"source:{source}"]["state"] == "active" + + def test_agent_read_carries_the_audience_when_something_can_be_taken_over(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "agent", agent_id, OWNER, OWNER, access_level="editor", target_user_id=OTHER + ) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + data = _get_agent(app, pg_conn, agent_id, OTHER) + assert "sponsor_audience" not in data + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + data = _get_agent(app, pg_conn, agent_id, OTHER) + assert _by_key(data["resource_states"])[f"tool:{tool}"]["can_confirm"] is True + assert data["sponsor_audience"]["teams"] == ["T"] + + +# --------------------------------------------------------------------------- +# Who sees it +# --------------------------------------------------------------------------- + + +class TestVisibility: + def _agent_with_stopped_tool(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + UserToolsRepository(pg_conn).delete(tool, OWNER) + return agent_id, tool + + @pytest.mark.parametrize("user", [OWNER, EDITOR]) + def test_owner_and_editor_get_states(self, app, pg_conn, user): + agent_id, tool = self._agent_with_stopped_tool(pg_conn) + data = _get_agent(app, pg_conn, agent_id, user) + assert _by_key(data["resource_states"])[f"tool:{tool}"]["reason"] == REASON_DELETED + + def test_viewer_gets_none(self, app, pg_conn): + agent_id, _ = self._agent_with_stopped_tool(pg_conn) + data = _get_agent(app, pg_conn, agent_id, VIEWER) + assert data.get("resource_states", []) == [] + assert "sponsor_audience" not in data + + def test_shared_agent_view_carries_no_states(self, app, pg_conn): + """The public-link read never carries run state.""" + from docsgpt.api.user.agents.sharing import SharedAgent + + agent_id, _ = self._agent_with_stopped_tool(pg_conn) + token = uuid.uuid4().hex + AgentsRepository(pg_conn).update_by_id(agent_id, {"shared": True, "shared_token": token}) + resp = _call(app, pg_conn, SharedAgent, "get", f"/api/shared_agent?token={token}", VIEWER) + body = _body(resp) + assert "resource_states" not in body + assert "sponsor_audience" not in body + + +# --------------------------------------------------------------------------- +# Parity with the run +# --------------------------------------------------------------------------- + + +class TestParityWithRun: + def test_stopped_source_is_not_retrieved(self, app, pg_conn): + from docsgpt.api.answer.services.stream_processor import authorized_agent_sources + + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + own = str(SourcesRepository(pg_conn).create("own", user_id=OWNER)["id"]) + AgentsRepository(pg_conn).update_by_id(agent_id, {"source_id": own, "extra_source_ids": [source]}) + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + states = _states(pg_conn, agent_id) + _, rows = authorized_agent_sources(pg_conn, _row(pg_conn, agent_id)) + used = {f"source:{r['id']}" for r in rows} + assert used == {k for k, s in states.items() if s["type"] == "source" and s["state"] == "active"} + assert f"source:{source}" not in used + + def test_stopped_tool_is_not_loaded(self, app, pg_conn): + from docsgpt.agents.tool_executor import ToolExecutor + + own = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, team_id = _agent(pg_conn, tools=[own]) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [own, tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + agent = _row(pg_conn, agent_id) + states = _states(pg_conn, agent_id) + with _patch_db(pg_conn): + loaded = ToolExecutor(user_api_key=agent["key"], user=OWNER)._get_tools_by_api_key(agent["key"]) + assert {f"tool:{t}" for t in loaded} == {k for k, s in states.items() if s["state"] == "active"} + + def test_stopped_prompt_falls_back_to_default(self, app, pg_conn): + from docsgpt.api.answer.services.stream_processor import authorized_prompt_id + + agent_id, team_id = _agent(pg_conn) + _, prompt, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"prompt_id": prompt, "confirm_sponsor": _confirm(("prompt", prompt))})) == 200 + agent = _row(pg_conn, agent_id) + with _patch_db(pg_conn): + assert _states(pg_conn, agent_id)[f"prompt:{prompt}"]["state"] == "active" + assert authorized_prompt_id(prompt, OWNER, agent) == prompt + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + assert _states(pg_conn, agent_id)[f"prompt:{prompt}"]["state"] == "stopped" + assert authorized_prompt_id(prompt, OWNER, _row(pg_conn, agent_id)) == "default" + + def test_ref_access_names_the_principal(self, app, pg_conn): + own = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, _ = _agent(pg_conn, tools=[own]) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [own, tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + agent = _row(pg_conn, agent_id) + assert ref_access(pg_conn, "agent", agent, "tool", own).principal == OWNER + assert ref_access(pg_conn, "agent", agent, "tool", tool).principal == EDITOR + missing = str(uuid.uuid4()) + state = ref_access(pg_conn, "agent", agent, "tool", missing) + assert (state.principal, state.reason) == (None, REASON_DELETED) + + +class TestRunLog: + def test_dropped_source_logs_type_id_and_reason(self, pg_conn, caplog): + from docsgpt.api.answer.services.stream_processor import authorized_agent_sources + + agent_id, _ = _agent(pg_conn) + missing = str(uuid.uuid4()) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [missing]}) + with caplog.at_level(logging.INFO): + authorized_agent_sources(pg_conn, _row(pg_conn, agent_id)) + [record] = [r for r in caplog.records if getattr(r, "event", None) == "resource_stopped"] + assert (record.holder_type, record.holder_id) == ("agent", agent_id) + assert (record.resource_type, record.resource_id, record.reason) == ("source", missing, REASON_DELETED) + assert f"reason={REASON_DELETED}" in record.getMessage() + + def test_dropped_tool_logs_reason(self, pg_conn, caplog): + from docsgpt.agents.tool_executor import ToolExecutor + + missing = str(uuid.uuid4()) + agent_id, _ = _agent(pg_conn, tools=[missing]) + agent = _row(pg_conn, agent_id) + with _patch_db(pg_conn), caplog.at_level(logging.INFO): + assert ToolExecutor(user_api_key=agent["key"], user=OWNER)._get_tools_by_api_key(agent["key"]) == {} + records = [r for r in caplog.records if getattr(r, "event", None) == "resource_stopped"] + assert [(r.resource_type, r.resource_id, r.reason) for r in records] == [("tool", missing, REASON_DELETED)] + + + def test_dropped_node_tool_logs_reason(self, pg_conn, caplog): + from docsgpt.agents.tool_executor import ToolExecutor + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + missing = str(uuid.uuid4()) + executor = ToolExecutor(user=VIEWER) + executor.allowed_tool_ids = [missing] + executor.tool_owner = OWNER + executor.tool_holder = wf + with _patch_db(pg_conn), caplog.at_level(logging.INFO): + assert executor.get_tools() == {} + [record] = [r for r in caplog.records if getattr(r, "event", None) == "resource_stopped"] + assert (record.holder_type, record.holder_id) == ("workflow", str(wf["id"])) + assert (record.resource_type, record.resource_id, record.reason) == ("tool", missing, REASON_DELETED) + + +# --------------------------------------------------------------------------- +# Workflows +# --------------------------------------------------------------------------- + + +class TestWorkflowStates: + def _setup(self, pg_conn): + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + agent_id, team_id = _agent(pg_conn, agent_type="workflow", workflow_id=str(wf["id"])) + return str(wf["id"]), agent_id, team_id + + def _put(self, app, pg_conn, wid, user, body): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + return _call(app, pg_conn, WorkflowDetail, "put", f"/api/workflows/{wid}", user, json=body, args=(wid,)) + + def _get(self, app, pg_conn, wid, user): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + return _call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", user, args=(wid,)) + + def test_node_states_on_the_workflow_read(self, app, pg_conn): + wid, agent_id, team_id = self._setup(pg_conn) + tool, _, source = _editor_resources(pg_conn) + confirm = _confirm(("tool", tool), ("source", source)) + assert _status(self._put(app, pg_conn, wid, EDITOR, _wf_body(tool, source, confirm=confirm))) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + data = _body(self._get(app, pg_conn, wid, OWNER))["data"] + states = _by_key(data["resource_states"]) + assert states[f"tool:{tool}"]["reason"] == REASON_CANNOT_EDIT_HOLDER + assert states[f"source:{source}"]["reason"] == REASON_CANNOT_EDIT_HOLDER + + def test_workflow_engine_drops_what_the_read_marks_stopped(self, app, pg_conn): + from types import SimpleNamespace + + from docsgpt.agents.workflows.workflow_engine import WorkflowEngine + + wid, agent_id, team_id = self._setup(pg_conn) + _, _, source = _editor_resources(pg_conn) + own = str(SourcesRepository(pg_conn).create("own", user_id=OWNER)["id"]) + assert _status(self._put(app, pg_conn, wid, OWNER, _wf_body(source=own))) == 200 + body = _wf_body(source=source, confirm=_confirm(("source", source))) + body["nodes"][1]["data"]["config"]["sources"] = [source, own] + assert _status(self._put(app, pg_conn, wid, EDITOR, body)) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + + engine = WorkflowEngine.__new__(WorkflowEngine) + engine.agent = SimpleNamespace( + workflow_row=WorkflowsRepository(pg_conn).get_by_id(wid), + _resolve_owner_id=lambda: OWNER, user=OWNER, decoded_token={"sub": OWNER}, + ) + with _patch_db(pg_conn): + allowed = engine._authorized_node_sources([source, own]) + states = _by_key(_body(self._get(app, pg_conn, wid, OWNER))["data"]["resource_states"]) + assert {f"source:{s}" for s in allowed} == {k for k, s in states.items() if s["state"] == "active"} + + def test_deleted_node_tool(self, app, pg_conn): + wid, _, _ = self._setup(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + assert _status(self._put(app, pg_conn, wid, OWNER, _wf_body(tool))) == 200 + UserToolsRepository(pg_conn).delete(tool, OWNER) + state = _by_key(_body(self._get(app, pg_conn, wid, OWNER))["data"]["resource_states"])[f"tool:{tool}"] + assert state["reason"] == REASON_DELETED + + def test_sponsor_details_only_for_editors(self, app, pg_conn): + """B: the workflow read gives sponsor details only to people who may edit.""" + from docsgpt.api.user import resource_access + + wid, _, _ = self._setup(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(self._put(app, pg_conn, wid, EDITOR, + _wf_body(tool, confirm=_confirm(("tool", tool))))) == 200 + assert _body(self._get(app, pg_conn, wid, EDITOR))["data"]["resource_sponsors"] + # A future role that may view but not edit reads no details. + original = resource_access.holder_editable_by + try: + resource_access.holder_editable_by = lambda *a, **k: False + data = _body(self._get(app, pg_conn, wid, EDITOR))["data"] + finally: + resource_access.holder_editable_by = original + assert data["resource_sponsors"] == [] + assert data["resource_states"] == [] + assert data["ref_details"] == {"tools": [], "sources": []} + + def test_ref_details_hide_names_nobody_here_can_use(self, app, pg_conn): + """A: a node naming someone else's resource doesn't reveal its name.""" + wid, _, _ = self._setup(pg_conn) + secret = str(UserToolsRepository(pg_conn).create("sp-stranger", "api_tool", custom_name="Secret")["id"]) + own = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool", custom_name="Mine")["id"]) + body = _wf_body(tools=[own]) + assert _status(self._put(app, pg_conn, wid, OWNER, body)) == 200 + # Written straight to the graph, the way an older unchecked save left it. + pg_conn.execute( + text("UPDATE workflow_nodes SET config = jsonb_set(config, '{config,tools}', CAST(:t AS jsonb)) " + "WHERE workflow_id = CAST(:w AS uuid) AND node_type = 'agent'"), + {"t": f'["{own}", "{secret}"]', "w": wid}, + ) + data = _body(self._get(app, pg_conn, wid, OWNER))["data"] + names = {t["id"]: t.get("name") for t in data["ref_details"]["tools"]} + assert own in names + assert secret not in names + state = _by_key(data["resource_states"])[f"tool:{secret}"] + assert state["state"] == "stopped" and state["name"] is None + + def test_owner_save_refuses_a_node_ref_the_owner_cannot_use(self, app, pg_conn): + wid, _, _ = self._setup(pg_conn) + secret = str(UserToolsRepository(pg_conn).create("sp-stranger", "api_tool")["id"]) + resp = self._put(app, pg_conn, wid, OWNER, _wf_body(secret)) + assert _status(resp) == 403 + + def test_owner_create_refuses_a_node_ref_the_owner_cannot_use(self, app, pg_conn): + from docsgpt.api.user.workflows.routes import WorkflowList + + secret = str(SourcesRepository(pg_conn).create("x", user_id="sp-stranger")["id"]) + resp = _call(app, pg_conn, WorkflowList, "post", "/api/workflows", OWNER, json=_wf_body(source=secret)) + assert _status(resp) == 403 + + def test_workflow_audience_when_something_can_be_taken_over(self, app, pg_conn): + wid, agent_id, team_id = self._setup(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "agent", agent_id, OWNER, OWNER, access_level="editor", target_user_id=OTHER + ) + assert _status(self._put(app, pg_conn, wid, EDITOR, + _wf_body(tool, confirm=_confirm(("tool", tool))))) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + data = _body(self._get(app, pg_conn, wid, OTHER))["data"] + assert _by_key(data["resource_states"])[f"tool:{tool}"]["can_confirm"] is True + assert data["sponsor_audience"]["teams"] == ["T"] + + +# --------------------------------------------------------------------------- +# Connections as the run resolves them +# --------------------------------------------------------------------------- + + +class TestConnectionModes: + def test_member_mode_tool_runs_on_each_persons_account(self, pg_conn): + """The owner's own account doesn't decide a member-mode tool; each caller's does.""" + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="reconnect_needed"), + credential_mode="member", + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("active", None) + assert state["note"] == "per_user_account" + + def test_admin_forced_member_mode_runs_on_each_persons_account(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="disconnected"), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + ConnectorPoliciesRepository(pg_conn).upsert("telegram", credential_mode="member") + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["note"]) == ("active", "per_user_account") + + def test_member_mode_tool_still_stops_when_the_service_is_off(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), credential_mode="member", + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["reason"] == REASON_CONNECTOR_DISABLED + + def test_owner_mode_tool_has_no_note(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["note"]) == ("active", None) + + @pytest.mark.parametrize("name", ["ntfy", "brave"]) + def test_service_tool_that_never_had_a_connection_runs(self, pg_conn, name): + """A tokenless ntfy (or a legacy tool) has no connection and needs none.""" + tool = str(UserToolsRepository(pg_conn).create(OWNER, name, config={"server_url": "https://ntfy.sh"})["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("active", None) + + def test_removed_connection_is_remembered_on_the_kept_tool(self, pg_conn): + from docsgpt.connectors import service + + cid = _connection(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "telegram", connection_id=cid)["id"]) + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:id AS uuid)"), + {"id": cid}).mappings().one() + service.remove_connection(pg_conn, dict(row), tools="keep") + kept = UserToolsRepository(pg_conn).get_any(tool, OWNER) + assert kept["connection_id"] is None + assert kept["config"]["removed_connection"] == "telegram" + + +# --------------------------------------------------------------------------- +# Whom to ask, without naming strangers +# --------------------------------------------------------------------------- + + +class TestContact: + def _owners_grant_only(self, conn, team_id, resource_type, resource_id): + """Share OTHER's resource with OWNER alone, so the agent's editors can't see it.""" + for member in (OWNER, OTHER): + if not TeamMembersRepository(conn).is_member(member, team_id): + TeamMembersRepository(conn).add_member(team_id, member) + TeamResourceGrantsRepository(conn).grant( + team_id, resource_type, resource_id, OTHER, OTHER, target_user_id=OWNER + ) + + def test_editor_who_can_see_the_item_is_told_its_owner(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [source]}) + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "source", source, OTHER, OTHER, target_user_id=EDITOR + ) + state = _states(pg_conn, agent_id, viewer=EDITOR)[f"source:{source}"] + assert state["contact"] == {"user_id": OTHER, "label": OTHER} + assert state["contact_role"] == "resource_owner" + + def test_editor_who_cannot_see_the_item_gets_no_identity(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create( + OTHER, "telegram", connection_id=_connection(pg_conn, user=OTHER, status="reconnect_needed"), + )["id"]) + self._owners_grant_only(pg_conn, team_id, "tool", tool) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [tool]}) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id, viewer=EDITOR)[f"tool:{tool}"] + assert state["reason"] == REASON_CONNECTION_NEEDS_RECONNECT + assert state["contact"] is None + assert state["contact_role"] == "resource_owner" + assert state["connection"]["id"] is None + + def test_old_graph_naming_a_strangers_tool_reveals_nobody(self, app, pg_conn): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + wid = str(wf["id"]) + _agent(pg_conn, agent_type="workflow", workflow_id=wid) + stranger_tool = str(UserToolsRepository(pg_conn).create("sp-stranger", "api_tool")["id"]) + own = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + assert _status(_call(app, pg_conn, WorkflowDetail, "put", f"/api/workflows/{wid}", OWNER, + json=_wf_body(own), args=(wid,))) == 200 + pg_conn.execute( + text("UPDATE workflow_nodes SET config = jsonb_set(config, '{config,tools}', CAST(:t AS jsonb)) " + "WHERE workflow_id = CAST(:w AS uuid) AND node_type = 'agent'"), + {"t": f'["{stranger_tool}"]', "w": wid}, + ) + data = _body(_call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", OWNER, args=(wid,))) + state = _by_key(data["data"]["resource_states"])[f"tool:{stranger_tool}"] + assert state["reason"] == REASON_OWNER_LOST_ACCESS + assert state["contact"] is None + assert state["name"] is None + + +class TestNamingPeople: + """One rule names every person on the page: only people the reader knows.""" + + def test_contact_the_reader_shares_no_team_with_is_not_named(self, pg_conn): + """Seeing the item isn't enough: its owner left every team the reader is in.""" + agent_id, team_id = _agent(pg_conn) + source = _team_source(pg_conn, team_id) + AgentsRepository(pg_conn).update_by_id(agent_id, {"extra_source_ids": [source]}) + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "source", source) + TeamResourceGrantsRepository(pg_conn).grant( + team_id, "source", source, OTHER, OTHER, target_user_id=EDITOR + ) + TeamMembersRepository(pg_conn).remove_member(team_id, OTHER) + state = _states(pg_conn, agent_id, viewer=EDITOR)[f"source:{source}"] + assert state["reason"] == REASON_OWNER_LOST_ACCESS + assert state["contact"] is None + assert state["contact_role"] == "resource_owner" + + def test_account_of_a_teammate_whose_tool_the_reader_cannot_see_is_not_named(self, pg_conn): + agent_id, team_id = _agent(pg_conn) + TeamMembersRepository(pg_conn).add_member(team_id, OWNER) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + tool = str(UserToolsRepository(pg_conn).create( + OTHER, "telegram", connection_id=_connection(pg_conn, user=OTHER), + )["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, target_user_id=OWNER) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": [tool]}) + with _patch_db(pg_conn): + as_owner = _states(pg_conn, agent_id)[f"tool:{tool}"] + as_editor = _states(pg_conn, agent_id, viewer=EDITOR)[f"tool:{tool}"] + assert as_owner["account"] == {"user_id": OTHER, "label": OTHER} + assert as_editor["account"] == {"user_id": None, "label": None} + + def test_a_sponsor_from_another_team_is_named_to_the_owner_only(self, app, pg_conn): + from docsgpt.api.user.resource_access import sponsor_details + from docsgpt.storage.db.repositories.teams import TeamsRepository + + second = "sp-editor-2" + agent_id, _ = _agent(pg_conn) + other_team = str(TeamsRepository(pg_conn).create("T2", f"t2-{uuid.uuid4().hex[:8]}", OWNER)["id"]) + TeamMembersRepository(pg_conn).add_member(other_team, second) + TeamResourceGrantsRepository(pg_conn).grant( + other_team, "agent", agent_id, OWNER, OWNER, access_level="editor", target_user_id=second + ) + tool = str(UserToolsRepository(pg_conn).create(second, "api_tool")["id"]) + assert _status(_put(app, pg_conn, agent_id, second, + {"tools": [tool], "confirm_sponsor": _confirm(("tool", tool))})) == 200 + agent = _row(pg_conn, agent_id) + as_owner = _states(pg_conn, agent_id)[f"tool:{tool}"] + as_editor = _states(pg_conn, agent_id, viewer=EDITOR)[f"tool:{tool}"] + assert as_owner["runs_as"] == {"user_id": second, "label": second} + assert as_owner["sponsor"] == {"user_id": second, "label": second} + assert as_editor["runs_as"] == {"user_id": None, "label": None} + assert as_editor["sponsor"] == {"user_id": None, "label": None} + [owner_detail] = sponsor_details(pg_conn, "agent", agent, viewer=OWNER) + [editor_detail] = sponsor_details(pg_conn, "agent", agent, viewer=EDITOR) + assert (owner_detail["user_id"], owner_detail["label"]) == (second, second) + assert (editor_detail["user_id"], editor_detail["label"]) == (None, None) + assert editor_detail["active"] is True + + +class TestRemovedConnectionMarker: + def test_removed_connection_without_a_catalog_key_is_still_marked(self, pg_conn): + from docsgpt.connectors import service + + cid = str(pg_conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, auth_kind, status) " + "VALUES (:u, 'legacy-service', 'api_key', 'connected') RETURNING id" + ), + {"u": OWNER}, + ).scalar()) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "telegram", connection_id=cid)["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:id AS uuid)"), + {"id": cid}).mappings().one() + service.remove_connection(pg_conn, dict(row), tools="keep") + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("stopped", REASON_CONNECTION_REMOVED) + assert state["connection"]["name"] == "Telegram" + + def test_kept_tool_given_its_own_credentials_runs(self, pg_conn): + from docsgpt.security.encryption import encrypt_credentials + + tool = str(UserToolsRepository(pg_conn).create(OWNER, "telegram", config={ + "removed_connection": "telegram", + "encrypted_credentials": encrypt_credentials({"token": "t"}, OWNER), + })["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert (state["state"], state["reason"]) == ("active", None) + + def test_a_stopped_tool_says_nothing_about_how_it_runs(self, pg_conn): + tool = str(UserToolsRepository(pg_conn).create( + OWNER, "telegram", connection_id=_connection(pg_conn, status="reconnect_needed"), + )["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + with _patch_db(pg_conn): + state = _states(pg_conn, agent_id)[f"tool:{tool}"] + assert state["reason"] == REASON_CONNECTION_NEEDS_RECONNECT + assert (state["note"], state["credential_mode"], state["account"]) == (None, None, None) + assert (state["owner_credential_writes"], state["writes_allowed"]) == ([], True) + + +# --------------------------------------------------------------------------- +# Reads never fail on run state +# --------------------------------------------------------------------------- + + +class TestBestEffort: + def test_agent_read_survives_a_state_error(self, app, pg_conn, monkeypatch): + from docsgpt.api.user.agents import routes + + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + agent_id, _ = _agent(pg_conn, tools=[tool]) + + def _boom(*_a, **_k): + raise RuntimeError("state failed") + + monkeypatch.setattr(routes, "resource_states", _boom) + data = _get_agent(app, pg_conn, agent_id, OWNER) + assert data["resource_states"] == [] + assert data["name"] == "Shared" + + def test_workflow_read_survives_a_state_error(self, app, pg_conn, monkeypatch): + from docsgpt.api.user.workflows import routes + from docsgpt.api.user.workflows.routes import WorkflowDetail + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + wid = str(wf["id"]) + _agent(pg_conn, agent_type="workflow", workflow_id=wid) + + def _boom(*_a, **_k): + raise RuntimeError("state failed") + + monkeypatch.setattr(routes, "resource_states", _boom) + resp = _call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", OWNER, args=(wid,)) + assert _status(resp) == 200 + assert _body(resp)["data"]["resource_states"] == [] + + def test_resolves_are_cached_within_a_read(self, pg_conn, monkeypatch): + from docsgpt.api.user import resource_access + + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + calls = [] + real = resource_access._resolve_uncached + + def _spy(*args): + calls.append(args[1:]) + return real(*args) + + monkeypatch.setattr(resource_access, "_resolve_uncached", _spy) + with resource_access.cached_resolves(): + first = resource_access.resolve(pg_conn, "tool", tool, OWNER) + second = resource_access.resolve(pg_conn, "tool", tool.upper(), OWNER) + assert first == second + assert len(calls) == 1 + # Outside a read nothing is cached: a revoked grant denies at once. + resource_access.resolve(pg_conn, "tool", tool, OWNER) + assert len(calls) == 2 diff --git a/tests/api/user/test_scheduler_worker.py b/tests/api/user/test_scheduler_worker.py index 71afc851..1f9f3176 100644 --- a/tests/api/user/test_scheduler_worker.py +++ b/tests/api/user/test_scheduler_worker.py @@ -680,3 +680,63 @@ class TestRunAsOwnerAccessRecheck: assert result["status"] == "success" # Runs with the owner's agent row (owner's context). assert headless.call_args.args[0]["user_id"] == "owner-x" + + +_OK = { + "answer": "ok", "tool_calls": [], "sources": [], "thought": "", + "prompt_tokens": 1, "generated_tokens": 1, "denied": [], + "error_type": None, "model_id": "m", +} + + +class TestCallerRules: + """A scheduled run acts as the owner but keeps the rules of whoever set it.""" + + def _run(self, conn, *, owner="owner-x", user="owner-x", shared=False, created_via="ui", grant=False): + import uuid as _uuid + + from docsgpt.storage.db.repositories.team_members import TeamMembersRepository + from docsgpt.storage.db.repositories.team_resource_grants import ( + TeamResourceGrantsRepository, + ) + from docsgpt.storage.db.repositories.teams import TeamsRepository + + agent_id = _make_agent(conn, owner) + conn.execute(text("UPDATE agents SET shared = :s WHERE id = CAST(:id AS uuid)"), + {"s": shared, "id": agent_id}) + if grant: + team = TeamsRepository(conn).create("T", f"t-{_uuid.uuid4().hex[:8]}", owner) + TeamMembersRepository(conn).add_member(str(team["id"]), user) + TeamResourceGrantsRepository(conn).grant( + str(team["id"]), "agent", agent_id, owner, owner, access_level="viewer", + ) + schedule = SchedulesRepository(conn).create( + user_id=user, agent_id=agent_id, trigger_type="once", instruction="hello", + run_at=_now(), next_run_at=_now(), created_via=created_via, + ) + return ScheduleRunsRepository(conn).record_pending(str(schedule["id"]), user, agent_id, _now()) + + def _caller(self, pg_engine, **kwargs): + with pg_engine.begin() as conn: + run = self._run(conn, **kwargs) + with patch("docsgpt.api.user.scheduler_worker.run_agent_headless", return_value=_OK) as headless: + assert execute_scheduled_run_body(str(run["id"]), "celery-c")["status"] == "success" + return {k: headless.call_args.kwargs.get(k) for k in ("external_caller", "public_link_caller")} + + def test_public_link_users_schedule_runs_as_a_public_link_caller(self, pg_engine, patched_engine, stub_events): + assert self._caller(pg_engine, user="stranger", shared=True) == { + "external_caller": False, "public_link_caller": True, + } + + def test_team_members_schedule_runs_as_a_teammate(self, pg_engine, patched_engine, stub_events): + assert self._caller(pg_engine, user="member-y", shared=True, grant=True) == { + "external_caller": False, "public_link_caller": False, + } + + def test_schedule_set_through_the_api_runs_as_an_external_caller(self, pg_engine, patched_engine, stub_events): + assert self._caller(pg_engine, created_via="api") == { + "external_caller": True, "public_link_caller": False, + } + + def test_owners_own_schedule_runs_as_the_owner(self, pg_engine, patched_engine, stub_events): + assert self._caller(pg_engine) == {"external_caller": False, "public_link_caller": False} diff --git a/tests/api/user/test_sponsor_consent.py b/tests/api/user/test_sponsor_consent.py new file mode 100644 index 00000000..25471cb0 --- /dev/null +++ b/tests/api/user/test_sponsor_consent.py @@ -0,0 +1,613 @@ +"""Who may sponsor a resource, and only with their explicit consent. + +A team editor who attaches a resource the agent's owner can't use makes it +run with the editor's access for everyone who uses the agent. That takes +owning the resource or having edit access to it (plain use is not enough), +an explicit ``confirm_sponsor`` on the save, and never happens silently when +a previous sponsor loses access. Uses real repositories on ``pg_conn``. +""" + +from __future__ import annotations + +import uuid + +import pytest +from flask import Flask + +from docsgpt.api.user.resource_access import ( + CODE_CONFIRMATION_REQUIRED, + CODE_NOT_ALLOWED, + CODE_UNEXPECTED_CONFIRMATION, + REASON_CANNOT_EDIT_HOLDER, + REASON_CANNOT_EDIT_RESOURCE, + active_sponsor, + parse_confirmations, + plan_sponsors, + sponsor_details, + sponsor_key, +) +from docsgpt.storage.db.repositories.agents import AgentsRepository +from docsgpt.storage.db.repositories.sources import SourcesRepository +from docsgpt.storage.db.repositories.team_members import TeamMembersRepository +from docsgpt.storage.db.repositories.team_resource_grants import ( + TeamResourceGrantsRepository, +) +from docsgpt.storage.db.repositories.teams import TeamsRepository +from docsgpt.storage.db.repositories.user_tools import UserToolsRepository +from docsgpt.storage.db.repositories.workflows import WorkflowsRepository +from tests.api.user.test_resource_sponsors import ( + EDITOR, + OTHER, + OWNER, + _agent, + _call, + _editor_resources, + _put, + _row, + _status, + _wf_body, +) + +STRANGER = "sp-stranger" + + +@pytest.fixture +def app(): + return Flask(__name__) + + +def _body(resp) -> dict: + return resp[0] if isinstance(resp, tuple) else resp.get_json() + + +def _other_team(conn, *members): + """A second team, owned by OTHER, that the agent is not shared with.""" + team = TeamsRepository(conn).create("Elsewhere", f"e-{uuid.uuid4().hex[:8]}", OTHER) + tid = str(team["id"]) + for member in (OTHER, *members): + if not TeamMembersRepository(conn).is_member(member, tid): + TeamMembersRepository(conn).add_member(tid, member) + return tid + + +def _others_tool(conn, team_id, user, level): + """OTHER's tool, shared with ``user`` in ``team_id`` at ``level``.""" + tool = str(UserToolsRepository(conn).create(OTHER, "api_tool")["id"]) + TeamResourceGrantsRepository(conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level=level, + target_user_id=user) + return tool + + +def _others_source(conn, team_id, user, level): + source = str(SourcesRepository(conn).create("others-src", user_id=OTHER)["id"]) + TeamResourceGrantsRepository(conn).grant(team_id, "source", source, OTHER, OTHER, access_level=level, + target_user_id=user) + return source + + +def _add_agent_editor(conn, team_id, agent_id, user): + if not TeamMembersRepository(conn).is_member(user, team_id): + TeamMembersRepository(conn).add_member(team_id, user) + TeamResourceGrantsRepository(conn).grant( + team_id, "agent", agent_id, OWNER, OWNER, access_level="editor", target_user_id=user + ) + + +# --------------------------------------------------------------------------- +# A: who may sponsor +# --------------------------------------------------------------------------- + + +class TestWhoMaySponsor: + def test_use_only_share_cannot_sponsor(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + team = _other_team(pg_conn, EDITOR) + source = _others_source(pg_conn, team, EDITOR, "viewer") + tool = _others_tool(pg_conn, team, EDITOR, "viewer") + resp = _put(app, pg_conn, agent_id, EDITOR, {"sources": [source], "tools": [tool], + "confirm_sponsor": [f"source:{source}", f"tool:{tool}"]}) + assert _status(resp) == 403 + body = _body(resp) + assert body["code"] == CODE_NOT_ALLOWED + assert {r["key"] for r in body["resources"]} == {f"source:{source}", f"tool:{tool}"} + row = _row(pg_conn, agent_id) + assert not row["tools"] and not row["extra_source_ids"] and not row["source_id"] + assert row["resource_sponsors"] == {} + + def test_edit_share_from_another_team_can_sponsor(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + team = _other_team(pg_conn, EDITOR) + tool = _others_tool(pg_conn, team, EDITOR, "editor") + resp = _put(app, pg_conn, agent_id, EDITOR, {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 200, _body(resp) + agent = _row(pg_conn, agent_id) + assert agent["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) == EDITOR + + def test_owner_usable_resource_needs_no_sponsor_or_consent(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OWNER, OWNER, target_user_id=EDITOR) + resp = _put(app, pg_conn, agent_id, EDITOR, {"tools": [tool]}) + assert _status(resp) == 200, _body(resp) + assert _row(pg_conn, agent_id)["resource_sponsors"] == {} + + def test_existing_use_only_sponsorship_stops_running(self, pg_conn): + agent_id, _ = _agent(pg_conn) + team = _other_team(pg_conn, EDITOR) + tool = _others_tool(pg_conn, team, EDITOR, "viewer") + AgentsRepository(pg_conn).update_by_id( + agent_id, {"tools": [tool], "resource_sponsors": {sponsor_key("tool", tool): EDITOR}} + ) + agent = _row(pg_conn, agent_id) + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) is None + [detail] = sponsor_details(pg_conn, "agent", agent) + assert detail["state"] == "inactive" + assert detail["reason"] == REASON_CANNOT_EDIT_RESOURCE + + +# --------------------------------------------------------------------------- +# B: explicit consent +# --------------------------------------------------------------------------- + + +class TestConsent: + def test_new_sponsorship_needs_confirmation(self, app, pg_conn): + from docsgpt.storage.db.repositories.users import UsersRepository + + agent_id, _ = _agent(pg_conn) + UsersRepository(pg_conn).upsert(EDITOR) + AgentsRepository(pg_conn).update_by_id(agent_id, {"shared": True, "shared_token": uuid.uuid4().hex}) + tool, prompt, source = _editor_resources(pg_conn) + resp = _put(app, pg_conn, agent_id, EDITOR, {"tools": [tool], "prompt_id": prompt, "source": source}) + assert _status(resp) == 409 + body = _body(resp) + assert body["code"] == CODE_CONFIRMATION_REQUIRED + by_key = {r["key"]: r for r in body["resources"]} + assert set(by_key) == {f"tool:{tool}", f"prompt:{prompt}", f"source:{source}"} + assert by_key[f"source:{source}"] == {"key": f"source:{source}", "type": "source", "id": source, + "name": "editor-src"} + assert by_key[f"prompt:{prompt}"]["name"] == "mine" + assert body["audience"] == {"teams": ["T"], "api_key": True, "public_link": True, "webhook": False} + row = _row(pg_conn, agent_id) + assert not row["tools"] and row["resource_sponsors"] == {} + + def test_exact_confirmation_saves(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, prompt, _ = _editor_resources(pg_conn) + confirm = [f"tool:{tool}", f"prompt:{prompt.upper()}"] + resp = _put(app, pg_conn, agent_id, EDITOR, {"tools": [tool], "prompt_id": prompt, "confirm_sponsor": confirm}) + assert _status(resp) == 200, _body(resp) + assert _row(pg_conn, agent_id)["resource_sponsors"] == { + sponsor_key("tool", tool): EDITOR, + sponsor_key("prompt", prompt): EDITOR, + } + + def test_form_encoded_confirmation(self, app, pg_conn): + from docsgpt.api.user.agents.routes import UpdateAgent + from tests.api.user.test_resource_sponsors import _patch_db + + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + data = {"tools": f'["{tool}"]', "confirm_sponsor": f'["tool:{tool}"]'} + with _patch_db(pg_conn), app.test_request_context(f"/api/update_agent/{agent_id}", method="PUT", + data=data): + from flask import request + + request.decoded_token = {"sub": EDITOR} + resp = UpdateAgent().put(agent_id) + assert _status(resp) == 200, _body(resp) + assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + + def test_partial_confirmation_is_refused(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, prompt, _ = _editor_resources(pg_conn) + resp = _put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "prompt_id": prompt, "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 409 + assert [r["key"] for r in _body(resp)["resources"]] == [f"prompt:{prompt}"] + assert not _row(pg_conn, agent_id)["tools"] + + def test_unrelated_confirmation_is_rejected(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + stray = str(uuid.uuid4()) + resp = _put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}", f"source:{stray}"]}) + assert _status(resp) == 400 + body = _body(resp) + assert body["code"] == CODE_UNEXPECTED_CONFIRMATION + assert body["unexpected"] == [f"source:{stray}"] + assert _row(pg_conn, agent_id)["resource_sponsors"] == {} + + def test_confirming_an_owner_usable_resource_is_rejected(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool")["id"]) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OWNER, OWNER, target_user_id=EDITOR) + resp = _put(app, pg_conn, agent_id, EDITOR, {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 400 + + def test_unchanged_save_needs_no_confirmation(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + assert _status(_put(app, pg_conn, agent_id, EDITOR, {"name": "Again", "tools": [tool]})) == 200 + assert _status(_put(app, pg_conn, agent_id, OWNER, {"name": "Owner", "tools": [tool]})) == 200 + assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + + def test_parse_confirmations(self): + rid = str(uuid.uuid4()) + assert parse_confirmations(None) == set() + assert parse_confirmations([f"tool:{rid.upper()}"]) == {f"tool:{rid}"} + assert parse_confirmations(f'["source:{rid}"]') == {f"source:{rid}"} + assert parse_confirmations(f"prompt:{rid}, agent:{rid}, nonsense") == {f"prompt:{rid}"} + assert parse_confirmations({"tool": rid}) == set() + + +# --------------------------------------------------------------------------- +# C: no silent handover +# --------------------------------------------------------------------------- + + +class TestNoSilentHandover: + def _sponsored_then_lost(self, app, pg_conn): + """EDITOR sponsors OTHER's tool (edit share), then loses edit on the agent. + + OTHER edits the agent too and may sponsor the tool themselves. + """ + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") + _add_agent_editor(pg_conn, team_id, agent_id, OTHER) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + return agent_id, team_id, tool + + def test_lost_sponsor_is_inactive(self, app, pg_conn): + agent_id, _, tool = self._sponsored_then_lost(app, pg_conn) + agent = _row(pg_conn, agent_id) + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) is None + [detail] = sponsor_details(pg_conn, "agent", agent, viewer=OTHER) + assert detail["state"] == "inactive" + assert detail["reason"] == REASON_CANNOT_EDIT_HOLDER + assert detail["user_id"] == EDITOR + assert detail["can_confirm"] is True + assert sponsor_details(pg_conn, "agent", agent, viewer=OWNER)[0]["can_confirm"] is False + + def test_unrelated_save_does_not_transfer(self, app, pg_conn): + agent_id, _, tool = self._sponsored_then_lost(app, pg_conn) + resp = _put(app, pg_conn, agent_id, OTHER, {"name": "Renamed", "tools": [tool]}) + assert _status(resp) == 200, _body(resp) + agent = _row(pg_conn, agent_id) + assert [str(t) for t in agent["tools"]] == [tool] + assert agent["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) is None + + def test_explicit_confirmation_transfers(self, app, pg_conn): + agent_id, _, tool = self._sponsored_then_lost(app, pg_conn) + resp = _put(app, pg_conn, agent_id, OTHER, {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 200, _body(resp) + agent = _row(pg_conn, agent_id) + assert agent["resource_sponsors"] == {sponsor_key("tool", tool): OTHER} + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) == OTHER + + def test_takeover_needs_edit_on_the_resource(self, app, pg_conn): + agent_id, team_id = _agent(pg_conn) + tool = _others_tool(pg_conn, _other_team(pg_conn, EDITOR), EDITOR, "editor") + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + _add_agent_editor(pg_conn, team_id, agent_id, STRANGER) + # STRANGER edits the agent but has no access to the tool at all. + resp = _put(app, pg_conn, agent_id, STRANGER, {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 400 + assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + + def test_sponsor_demoted_on_resource_stops(self, app, pg_conn): + agent_id, _ = _agent(pg_conn) + team = _other_team(pg_conn, EDITOR) + tool = _others_tool(pg_conn, team, EDITOR, "editor") + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + TeamResourceGrantsRepository(pg_conn).grant(team, "tool", tool, OTHER, OTHER, access_level="viewer", + target_user_id=EDITOR) + agent = _row(pg_conn, agent_id) + assert active_sponsor(pg_conn, "agent", agent, "tool", tool) is None + assert sponsor_details(pg_conn, "agent", agent)[0]["reason"] == REASON_CANNOT_EDIT_RESOURCE + # The same editor saving again doesn't quietly keep it running either. + assert _status(_put(app, pg_conn, agent_id, EDITOR, {"name": "x", "tools": [tool]})) == 200 + assert active_sponsor(pg_conn, "agent", _row(pg_conn, agent_id), "tool", tool) is None + + def test_plan_keeps_stopped_record_without_confirmation(self, app, pg_conn): + agent_id, _, tool = self._sponsored_then_lost(app, pg_conn) + agent = _row(pg_conn, agent_id) + refs = [("tool", tool)] + plan = plan_sponsors(pg_conn, "agent", agent, OWNER, OTHER, refs, previous_refs=refs) + assert plan.sponsors == {sponsor_key("tool", tool): EDITOR} + assert not plan.needs_confirmation and not plan.not_allowed and not plan.unexpected + + +# --------------------------------------------------------------------------- +# get_agent +# --------------------------------------------------------------------------- + + +class TestGetAgentDetails: + def test_sponsor_details_carry_state(self, app, pg_conn): + from docsgpt.api.user.agents.routes import GetAgent + + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + data = _call(app, pg_conn, GetAgent, "get", f"/api/get_agent?id={agent_id}", EDITOR).get_json() + assert data["resource_sponsors"] == [{ + "key": f"tool:{tool}", "type": "tool", "id": tool, "name": "api_tool", "user_id": EDITOR, + "label": EDITOR, "state": "active", "reason": None, "active": True, "can_confirm": False, + }] + + +# --------------------------------------------------------------------------- +# workflows +# --------------------------------------------------------------------------- + + +class TestWorkflowConsent: + def _setup(self, pg_conn): + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + agent_id, team_id = _agent(pg_conn, agent_type="workflow", workflow_id=str(wf["id"])) + return str(wf["id"]), agent_id, team_id + + def _put(self, app, pg_conn, wid, user, body): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + return _call(app, pg_conn, WorkflowDetail, "put", f"/api/workflows/{wid}", user, json=body, args=(wid,)) + + def test_new_node_sponsorship_needs_confirmation(self, app, pg_conn): + wid, _, _ = self._setup(pg_conn) + tool, _, source = _editor_resources(pg_conn) + resp = self._put(app, pg_conn, wid, EDITOR, _wf_body(tool, source)) + assert _status(resp) == 409 + body = _body(resp) + assert body["code"] == CODE_CONFIRMATION_REQUIRED + assert {r["key"] for r in body["resources"]} == {f"tool:{tool}", f"source:{source}"} + assert body["audience"]["teams"] == ["T"] + row = WorkflowsRepository(pg_conn).get_by_id(wid) + assert row["resource_sponsors"] == {} + assert row["current_graph_version"] in (None, 0, 1) + + confirmed = {**_wf_body(tool, source), "confirm_sponsor": [f"tool:{tool}", f"source:{source}"]} + resp = self._put(app, pg_conn, wid, EDITOR, confirmed) + assert _status(resp) == 200, _body(resp) + assert WorkflowsRepository(pg_conn).get_by_id(wid)["resource_sponsors"] == { + sponsor_key("tool", tool): EDITOR, + sponsor_key("source", source): EDITOR, + } + + def test_use_only_node_resource_is_refused(self, app, pg_conn): + wid, _, _ = self._setup(pg_conn) + tool = _others_tool(pg_conn, _other_team(pg_conn, EDITOR), EDITOR, "viewer") + resp = self._put(app, pg_conn, wid, EDITOR, {**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 403 + assert _body(resp)["code"] == CODE_NOT_ALLOWED + + def test_unrelated_confirmation_is_rejected(self, app, pg_conn): + wid, _, _ = self._setup(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + body = {**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}", f"tool:{uuid.uuid4()}"]} + assert _status(self._put(app, pg_conn, wid, EDITOR, body)) == 400 + + def test_lost_sponsor_not_transferred_without_confirmation(self, app, pg_conn): + wid, agent_id, team_id = self._setup(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") + _add_agent_editor(pg_conn, team_id, agent_id, OTHER) + assert _status(self._put(app, pg_conn, wid, EDITOR, + {**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}"]})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + + assert _status(self._put(app, pg_conn, wid, OTHER, _wf_body(tool))) == 200 + row = WorkflowsRepository(pg_conn).get_by_id(wid) + assert row["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + assert active_sponsor(pg_conn, "workflow", row, "tool", tool) is None + + assert _status(self._put(app, pg_conn, wid, OTHER, + {**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}"]})) == 200 + row = WorkflowsRepository(pg_conn).get_by_id(wid) + assert row["resource_sponsors"] == {sponsor_key("tool", tool): OTHER} + assert active_sponsor(pg_conn, "workflow", row, "tool", tool) == OTHER + + def test_workflow_details_carry_state(self, app, pg_conn): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + wid, _, _ = self._setup(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(self._put(app, pg_conn, wid, EDITOR, + {**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}"]})) == 200 + resp = _call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", OWNER, args=(wid,)) + [detail] = _body(resp)["data"]["resource_sponsors"] + assert detail["state"] == "active" and detail["reason"] is None and detail["name"] == "api_tool" + + +# --------------------------------------------------------------------------- +# Review follow-ups +# --------------------------------------------------------------------------- + + +def _viewer_share(conn, team_id, resource_type, resource_id, owner, user): + TeamResourceGrantsRepository(conn).grant(team_id, resource_type, resource_id, owner, owner, + access_level="viewer", target_user_id=user) + + +class TestNoResurrectedSponsor: + """A sponsor record left behind never vouches for a newly attached resource.""" + + def _setup(self, app, pg_conn): + """EDITOR sponsors OTHER's tool T (edit share); STRANGER edits the agent, views T.""" + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor", + target_user_id=EDITOR) + _add_agent_editor(pg_conn, team_id, agent_id, STRANGER) + _viewer_share(pg_conn, team_id, "tool", tool, OTHER, STRANGER) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + return agent_id, team_id, tool + + def test_stale_record_does_not_cover_a_new_attachment(self, app, pg_conn): + agent_id, _, tool = self._setup(app, pg_conn) + # Some path drops the tool but leaves EDITOR's record behind. + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": []}) + assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): EDITOR} + resp = _put(app, pg_conn, agent_id, STRANGER, {"tools": [tool]}) + assert _status(resp) == 403 + assert _body(resp)["code"] == CODE_NOT_ALLOWED + assert not _row(pg_conn, agent_id)["tools"] + + def test_new_attachment_by_a_sponsor_capable_editor_asks_them(self, app, pg_conn): + agent_id, team_id, tool = self._setup(app, pg_conn) + AgentsRepository(pg_conn).update_by_id(agent_id, {"tools": []}) + _add_agent_editor(pg_conn, team_id, agent_id, OTHER) + resp = _put(app, pg_conn, agent_id, OTHER, {"tools": [tool]}) + assert _status(resp) == 409 + resp = _put(app, pg_conn, agent_id, OTHER, {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]}) + assert _status(resp) == 200 + assert _row(pg_conn, agent_id)["resource_sponsors"] == {sponsor_key("tool", tool): OTHER} + + def test_reimport_prunes_dropped_sponsors(self, app, pg_conn): + from docsgpt.api.user.agents.portability import API_VERSION, apply_import, ensure_agent_slug + + agent_id, _, tool = self._setup(app, pg_conn) + agent = AgentsRepository(pg_conn).get(agent_id, OWNER) + slug = ensure_agent_slug(pg_conn, agent, OWNER) + doc = {"apiVersion": API_VERSION, "kind": "Agent", "metadata": {"id": agent_id, "slug": slug}, + "spec": {"name": "Shared", "description": "d", "retriever": "classic"}} + assert apply_import(pg_conn, OWNER, doc)["action"] == "updated" + row = _row(pg_conn, agent_id) + assert not row["tools"] + assert row["resource_sponsors"] == {} + # The repro: STRANGER only views T, re-adds it, and must not run it as EDITOR. + resp = _put(app, pg_conn, agent_id, STRANGER, {"tools": [tool]}) + assert _status(resp) == 403 + assert _row(pg_conn, agent_id)["resource_sponsors"] == {} + + def test_workflow_reimport_prunes_dropped_sponsors(self, app, pg_conn): + from docsgpt.api.user.agents.portability import ( + agent_to_yaml, + apply_import, + parse_agent_yaml, + serialize_agent, + ) + from docsgpt.api.user.workflows.routes import WorkflowDetail + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + wid = str(wf["id"]) + agent_id, team_id = _agent(pg_conn, agent_type="workflow", workflow_id=wid) + tool, _, _ = _editor_resources(pg_conn) + resp = _call(app, pg_conn, WorkflowDetail, "put", f"/api/workflows/{wid}", EDITOR, + json={**_wf_body(tool), "confirm_sponsor": [f"tool:{tool}"]}, args=(wid,)) + assert _status(resp) == 200, _body(resp) + assert WorkflowsRepository(pg_conn).get_by_id(wid)["resource_sponsors"] + + agent = AgentsRepository(pg_conn).get(agent_id, OWNER) + doc = parse_agent_yaml(agent_to_yaml(serialize_agent(pg_conn, agent, OWNER))) + for node in doc["spec"]["workflow"]["nodes"]: + if node.get("type") == "agent": + node["config"]["tools"] = [] + apply_import(pg_conn, OWNER, doc) + assert WorkflowsRepository(pg_conn).get_by_id(wid)["resource_sponsors"] == {} + + +class TestTakeOver: + def test_confirming_a_stopped_attached_item_takes_it_over(self, app, pg_conn): + from docsgpt.api.user.agents.routes import GetAgent + + agent_id, team_id = _agent(pg_conn) + tool = str(UserToolsRepository(pg_conn).create(OTHER, "api_tool")["id"]) + TeamMembersRepository(pg_conn).add_member(team_id, OTHER) + TeamResourceGrantsRepository(pg_conn).grant(team_id, "tool", tool, OTHER, OTHER, access_level="editor") + _add_agent_editor(pg_conn, team_id, agent_id, OTHER) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + TeamResourceGrantsRepository(pg_conn).revoke(team_id, "agent", agent_id, target_user_id=EDITOR) + + path = f"/api/get_agent?id={agent_id}" + [stopped] = _call(app, pg_conn, GetAgent, "get", path, OTHER).get_json()["resource_sponsors"] + assert stopped["can_confirm"] is True and stopped["reason"] == REASON_CANNOT_EDIT_HOLDER + # A plain save (no change to the tools) that confirms the stopped key. + resp = _put(app, pg_conn, agent_id, OTHER, {"name": "Kept", "confirm_sponsor": [stopped["key"]]}) + assert _status(resp) == 200, _body(resp) + [running] = _call(app, pg_conn, GetAgent, "get", path, OTHER).get_json()["resource_sponsors"] + assert running["user_id"] == OTHER and running["state"] == "active" + + +class TestDetailsVisibility: + def test_viewers_get_no_sponsor_details(self, app, pg_conn): + from docsgpt.api.user.agents.routes import GetAgent + from tests.api.user.test_resource_sponsors import VIEWER + + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + assert _status(_put(app, pg_conn, agent_id, EDITOR, + {"tools": [tool], "confirm_sponsor": [f"tool:{tool}"]})) == 200 + path = f"/api/get_agent?id={agent_id}" + assert _call(app, pg_conn, GetAgent, "get", path, VIEWER).get_json()["resource_sponsors"] == [] + assert _call(app, pg_conn, GetAgent, "get", path, OWNER).get_json()["resource_sponsors"] + + def test_workflow_viewers_are_refused(self, app, pg_conn): + from docsgpt.api.user.workflows.routes import WorkflowDetail + from tests.api.user.test_resource_sponsors import VIEWER + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + wid = str(wf["id"]) + _agent(pg_conn, agent_type="workflow", workflow_id=wid) + resp = _call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", VIEWER, args=(wid,)) + assert _status(resp) in (403, 404) + + +class TestWorkflowNodeRefDetails: + def test_get_names_every_node_tool_and_source(self, app, pg_conn): + from docsgpt.api.user.workflows.routes import WorkflowDetail + + wf = WorkflowsRepository(pg_conn).create(OWNER, "wf") + wid = str(wf["id"]) + _agent(pg_conn, agent_type="workflow", workflow_id=wid) + tool = str(UserToolsRepository(pg_conn).create(OWNER, "api_tool", display_name="Owner API")["id"]) + source = str(SourcesRepository(pg_conn).create("owner-src", user_id=OWNER)["id"]) + assert _status(_call(app, pg_conn, WorkflowDetail, "put", f"/api/workflows/{wid}", OWNER, + json=_wf_body(tool, source), args=(wid,))) == 200 + data = _body(_call(app, pg_conn, WorkflowDetail, "get", f"/api/workflows/{wid}", EDITOR, + args=(wid,)))["data"] + assert data["ref_details"] == { + "tools": [{"id": tool, "name": "api_tool", "display_name": "Owner API"}], + "sources": [{"id": source, "name": "owner-src"}], + } + + +class TestImageAfterSponsorCheck: + def test_refused_save_stores_no_image(self, app, pg_conn): + import io + from unittest.mock import patch as _patch + + from docsgpt.api.user.agents.routes import UpdateAgent + from tests.api.user.test_resource_sponsors import _patch_db + + agent_id, _ = _agent(pg_conn) + tool, _, _ = _editor_resources(pg_conn) + data = {"tools": f'["{tool}"]', "image": (io.BytesIO(b"png"), "a.png")} + with _patch_db(pg_conn), \ + _patch("docsgpt.api.user.agents.routes.handle_image_upload", + return_value=("stored.png", None)) as upload, \ + app.test_request_context(f"/api/update_agent/{agent_id}", method="PUT", data=data, + content_type="multipart/form-data"): + from flask import request + + request.decoded_token = {"sub": EDITOR} + resp = UpdateAgent().put(agent_id) + assert _status(resp) == 409 + upload.assert_not_called() diff --git a/tests/api/user/test_tasks.py b/tests/api/user/test_tasks.py index a3ff2a19..e2b6a003 100644 --- a/tests/api/user/test_tasks.py +++ b/tests/api/user/test_tasks.py @@ -67,7 +67,8 @@ class TestIngestRemoteTask: mock_worker.assert_called_once_with( ANY, {"url": "http://x"}, "job1", "user1", "web", - config=None, idempotency_key=None, source_id=None, + sync_frequency="never", + config=None, idempotency_key=None, source_id=None, connection_id=None, ) assert result == {"status": "ok"} @@ -164,10 +165,20 @@ class TestSyncSourceTask: ) mock_sync.assert_called_once_with( - ANY, {"data": 1}, "job1", "user1", "web", "daily", "classic", "doc1" + ANY, {"data": 1}, "job1", "user1", "web", "daily", "classic", "doc1", + connection_id=None, ) assert result == {"status": "ok"} + @pytest.mark.unit + @patch("docsgpt.api.user.tasks.sync") + def test_passes_the_sources_connection(self, mock_sync): + from docsgpt.api.user.tasks import sync_source + + sync_source({"data": 1}, "job1", "user1", "s3", "daily", "classic", "doc1", connection_id="c-1") + + assert mock_sync.call_args.kwargs["connection_id"] == "c-1" + class TestStoreAttachmentTask: @pytest.mark.unit @@ -223,6 +234,7 @@ class TestIngestConnectorTask: "user1", "gdrive", session_token=None, + connection_id=None, file_ids=None, folder_ids=None, recursive=True, @@ -248,6 +260,7 @@ class TestIngestConnectorTask: "user1", "sharepoint", session_token="tok", + connection_id=None, file_ids=["f1"], folder_ids=["d1"], recursive=False, @@ -263,6 +276,7 @@ class TestIngestConnectorTask: "user1", "sharepoint", session_token="tok", + connection_id=None, file_ids=["f1"], folder_ids=["d1"], recursive=False, diff --git a/tests/api/user/test_tools_access.py b/tests/api/user/test_tools_access.py index 27ba7b4d..1225924f 100644 --- a/tests/api/user/test_tools_access.py +++ b/tests/api/user/test_tools_access.py @@ -408,6 +408,36 @@ class TestToolWrites: # --------------------------------------------------------------------------- # 5. POST /api/update_tool_status # --------------------------------------------------------------------------- +class TestRemovedConnectionNote: + """Only removing a connection notes it on a kept tool; config saves can't fake or clear it.""" + + @pytest.mark.parametrize("route", ["UpdateToolConfig", "UpdateTool"]) + def test_a_config_save_cannot_set_the_note(self, app, pg_conn, route): + from docsgpt.api.user.tools import routes + + tool = _tool(pg_conn, name="brave", config={}) + body = {"id": str(tool["id"]), "config": {"token": "x", "removed_connection": "brave"}} + assert _call(app, pg_conn, getattr(routes, route), OWNER, json=body).status_code == 200 + assert "removed_connection" not in _row(pg_conn, tool["id"])["config"] + + @pytest.mark.parametrize("route", ["UpdateToolConfig", "UpdateTool"]) + def test_a_config_save_keeps_the_stored_note(self, app, pg_conn, route): + from docsgpt.api.user.tools import routes + + tool = _tool(pg_conn, name="brave", config={"removed_connection": "brave"}) + for config in ({"token": "x"}, {"token": "y", "removed_connection": None}): + body = {"id": str(tool["id"]), "config": config} + assert _call(app, pg_conn, getattr(routes, route), OWNER, json=body).status_code == 200 + assert _row(pg_conn, tool["id"])["config"]["removed_connection"] == "brave" + + def test_an_mcp_save_drops_the_note(self): + from docsgpt.api.user.tools.mcp import _sanitize_mcp_transport + + cfg = {"transport_type": "http", "removed_connection": "github"} + _sanitize_mcp_transport(cfg) + assert "removed_connection" not in cfg + + class TestUpdateToolStatusRoles: def test_owner_writes_status(self, app, pg_conn): from docsgpt.api.user.tools.routes import UpdateToolStatus @@ -452,6 +482,38 @@ def _mcp_tool(conn, url="https://mcp.example.com/mcp", secrets=None, auth_type=" ) +def _stored_mcp_secret(conn, tool_id): + """The secret a saved MCP tool runs with, and whose key it is under. + + A save moves the secret onto a connection of the tool's owner (see + ``docsgpt/connectors``); a tool without one keeps it in its config. + """ + from docsgpt.connectors import service + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + row = UserToolsRepository(conn).get_any(str(tool_id), OWNER) + if not row.get("connection_id"): + return decrypt_credentials(row["config"]["encrypted_credentials"], OWNER), OWNER + assert "encrypted_credentials" not in row["config"] + connection = ConnectorSessionsRepository(conn).get(str(row["connection_id"])) + return service.get_credentials(connection), connection["user_id"] + + +def _mcp_connection_tool(conn, secrets, url="https://mcp.example.com/mcp"): + """An MCP tool whose key lives on an owner's connection, as a save leaves it.""" + from docsgpt.connectors import catalog, service + + connection, _ = service.create_api_key_connection( + conn, OWNER, catalog.get_definition("custom_mcp"), secrets, + server_url=catalog.base_url(url), display_name="M", + ) + return UserToolsRepository(conn).create( + OWNER, "mcp_tool", config={"server_url": url, "auth_type": "bearer", "transport_type": "http"}, + display_name="M", custom_name="M", description="", + actions=[{"name": "old", "active": True}], status=True, connection_id=str(connection["id"]), + ), connection + + class TestMCPSaveAccess: def _save(self, app, conn, user, body, discovered=None): from docsgpt.api.user.tools.mcp import MCPServerSave @@ -482,7 +544,7 @@ class TestMCPSaveAccess: # Stored creds were reused for discovery and kept, encrypted with the owner's key. assert cls.call_args.kwargs.get("user_id") == OWNER assert cls.call_args.kwargs["config"]["auth_credentials"] == {"bearer_token": "tok"} - assert decrypt_credentials(row["config"]["encrypted_credentials"], OWNER) == {"bearer_token": "tok"} + assert _stored_mcp_secret(pg_conn, tool["id"]) == ({"bearer_token": "tok"}, OWNER) def test_viewer_and_stranger_cannot_save_or_duplicate(self, app, pg_conn): tool = _mcp_tool(pg_conn, secrets={"bearer_token": "tok"}) @@ -523,8 +585,45 @@ class TestMCPSaveAccess: "transport_type": "http", "bearer_token": "tok2"}} resp, _ = self._save(app, pg_conn, OWNER, body) assert resp.status_code == 200 - row = _row(pg_conn, tool["id"]) - assert decrypt_credentials(row["config"]["encrypted_credentials"], OWNER) == {"bearer_token": "tok2"} + assert _row(pg_conn, tool["id"])["config"]["server_url"] == "https://elsewhere.example.org/mcp" + assert _stored_mcp_secret(pg_conn, tool["id"]) == ({"bearer_token": "tok2"}, OWNER) + + def test_empty_secret_reuses_the_connections_key(self, app, pg_conn): + tool, connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + _share(pg_conn, tool["id"], "ed", "editor") + body = {"id": str(tool["id"]), "displayName": "Renamed", + "config": {"server_url": "https://mcp.example.com/mcp", "auth_type": "bearer", + "transport_type": "http"}} + resp, cls = self._save(app, pg_conn, "ed", body) + assert resp.status_code == 200, resp.json + assert cls.call_args.kwargs["config"]["auth_credentials"] == {"bearer_token": "tok"} + assert cls.call_args.kwargs["user_id"] == OWNER + assert str(_row(pg_conn, tool["id"])["connection_id"]) == str(connection["id"]) + + def test_editors_new_key_does_not_rotate_the_owners_connection(self, app, pg_conn): + from docsgpt.connectors import service + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + tool, connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + _share(pg_conn, tool["id"], "ed", "editor") + body = {"id": str(tool["id"]), "displayName": "M", + "config": {"server_url": "https://mcp.example.com/mcp", "auth_type": "bearer", + "transport_type": "http", "bearer_token": "tok2"}} + resp, _ = self._save(app, pg_conn, "ed", body) + assert resp.status_code == 200, resp.json + assert _stored_mcp_secret(pg_conn, tool["id"]) == ({"bearer_token": "tok2"}, OWNER) + # The owner's other tools on the old connection keep its key. + old = ConnectorSessionsRepository(pg_conn).get(str(connection["id"])) + assert service.get_credentials(old) == {"bearer_token": "tok"} + + def test_moved_server_does_not_inherit_the_connections_key(self, app, pg_conn): + tool, _connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + body = {"id": str(tool["id"]), "displayName": "M", + "config": {"server_url": "https://elsewhere.example.org/mcp", "auth_type": "bearer", + "transport_type": "http"}} + resp, cls = self._save(app, pg_conn, OWNER, body) + assert resp.status_code == 400 + assert not cls.called def test_editor_blocked_when_credentials_switch_off(self, app, pg_conn): tool = _mcp_tool(pg_conn, secrets={"bearer_token": "tok"}) @@ -709,3 +808,224 @@ class TestSharedOAuthMCPConfig: body = {"id": str(tool["id"]), "config": dict(self.OTHER)} assert _call(app, pg_conn, UpdateToolConfig, OWNER, json=body).status_code == 200 assert _row(pg_conn, tool["id"])["config"]["server_url"] == self.OTHER["server_url"] + + +# --------------------------------------------------------------------------- +# 7. /api/update_tool keeps fixed values and the tool type +# --------------------------------------------------------------------------- +def _pinned_tool(conn, name="ntfy"): + """A tool whose ``server_url`` the owner fixed; ``message`` is the model's.""" + return UserToolsRepository(conn).create( + OWNER, name, config={}, display_name=name, description="", + actions=[{ + "name": "send", "active": True, "require_approval": False, + "parameters": {"type": "object", "properties": { + "server_url": {"type": "string", "filled_by_llm": False, "value": "https://ntfy.example.com"}, + "message": {"type": "string", "filled_by_llm": True, "value": ""}, + }}, + }], + status=True, + ) + + +def _moved_pin(actions): + import copy + + out = copy.deepcopy(actions) + out[0]["parameters"]["properties"]["server_url"]["value"] = "https://evil.example" + return out + + +class TestUpdateToolFixedValues: + def test_editor_cannot_change_a_fixed_value_through_update_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn) + _share(pg_conn, tool["id"], "ed", "editor") + body = {"id": str(tool["id"]), "actions": _moved_pin(tool["actions"])} + resp = _call(app, pg_conn, UpdateTool, "ed", json=body) + assert resp.status_code == 403 + pinned = _row(pg_conn, tool["id"])["actions"][0]["parameters"]["properties"]["server_url"] + assert pinned["value"] == "https://ntfy.example.com" + + def test_editor_cannot_release_a_fixed_value_through_update_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn) + _share(pg_conn, tool["id"], "ed", "editor") + actions = tool["actions"] + actions[0]["parameters"]["properties"]["server_url"]["filled_by_llm"] = True + resp = _call(app, pg_conn, UpdateTool, "ed", json={"id": str(tool["id"]), "actions": actions}) + assert resp.status_code == 403 + assert _row(pg_conn, tool["id"])["actions"][0]["parameters"]["properties"]["server_url"][ + "filled_by_llm"] is False + + def test_editor_round_trip_with_other_edits_still_saves(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn) + _share(pg_conn, tool["id"], "ed", "editor") + actions = tool["actions"] + actions[0]["require_approval"] = True + body = {"id": str(tool["id"]), "name": "ntfy", "customName": "Alerts", "actions": actions} + assert _call(app, pg_conn, UpdateTool, "ed", json=body).status_code == 200 + row = _row(pg_conn, tool["id"]) + assert row["custom_name"] == "Alerts" and row["actions"][0]["require_approval"] is True + + def test_actions_cannot_be_added_through_update_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn) + actions = [*tool["actions"], {"name": "exfiltrate", "active": True}] + resp = _call(app, pg_conn, UpdateTool, OWNER, json={"id": str(tool["id"]), "actions": actions}) + assert resp.status_code == 400 + assert [a["name"] for a in _row(pg_conn, tool["id"])["actions"]] == ["send"] + + def test_owner_changes_a_fixed_value_through_update_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn) + body = {"id": str(tool["id"]), "actions": _moved_pin(tool["actions"])} + assert _call(app, pg_conn, UpdateTool, OWNER, json=body).status_code == 200 + pinned = _row(pg_conn, tool["id"])["actions"][0]["parameters"]["properties"]["server_url"] + assert pinned["value"] == "https://evil.example" + + @pytest.mark.parametrize("user", [OWNER, "ed"]) + def test_tool_type_cannot_change(self, app, pg_conn, user): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = _pinned_tool(pg_conn, name="telegram") + _share(pg_conn, tool["id"], "ed", "editor") + resp = _call(app, pg_conn, UpdateTool, user, json={"id": str(tool["id"]), "name": "ntfy"}) + assert resp.status_code == 400 + assert _row(pg_conn, tool["id"])["name"] == "telegram" + + +# --------------------------------------------------------------------------- +# 8. api_tool: a stored header / query value stays fixed for non-owners +# --------------------------------------------------------------------------- +class TestApiToolFixedValues: + def _tool(self, conn): + from docsgpt.api.user.tools.routes import _seal_api_tool_secrets + + tool = _tool(conn, name="api_tool", config=_seal_api_tool_secrets(_api_config(), {}, OWNER)) + _share(conn, tool["id"], "ed", "editor") + return tool + + def _masked(self): + return TestApiToolSecrets()._masked(_api_config()) + + def _token(self, cfg): + return cfg["actions"]["get_users"]["query_params"]["properties"]["token"] + + @pytest.mark.parametrize("route", ["UpdateTool", "UpdateToolConfig"]) + def test_editor_cannot_hand_a_stored_secret_to_the_model(self, app, pg_conn, route): + from docsgpt.api.user.tools import routes + + tool = self._tool(pg_conn) + cfg = self._masked() + self._token(cfg)["filled_by_llm"] = True + resp = _call(app, pg_conn, getattr(routes, route), "ed", json={"id": str(tool["id"]), "config": cfg}) + assert resp.status_code == 403 + token = _runtime_action(pg_conn, tool["id"])["query_params"]["properties"]["token"] + assert token["filled_by_llm"] is False and token["value"] == "q-secret" + + @pytest.mark.parametrize("change", [ + {"has_value": False}, + {"value": "attacker-token"}, + ]) + def test_editor_cannot_clear_or_replace_a_stored_value(self, app, pg_conn, change): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = self._tool(pg_conn) + cfg = self._masked() + self._token(cfg).update(change) + resp = _call(app, pg_conn, UpdateTool, "ed", json={"id": str(tool["id"]), "config": cfg}) + assert resp.status_code == 403 + assert _runtime_action(pg_conn, tool["id"])["query_params"]["properties"]["token"]["value"] == "q-secret" + + def test_editor_cannot_drop_a_fixed_parameter_then_re_add_it(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = self._tool(pg_conn) + cfg = self._masked() + del cfg["actions"]["get_users"]["query_params"]["properties"]["token"] + resp = _call(app, pg_conn, UpdateTool, "ed", json={"id": str(tool["id"]), "config": cfg}) + assert resp.status_code == 403 + + def test_editor_still_adds_an_empty_parameter_and_edits_descriptions(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = self._tool(pg_conn) + cfg = self._masked() + cfg["actions"]["get_users"]["description"] = "Edited" + cfg["actions"]["get_users"]["query_params"]["properties"]["page"] = { + "type": "string", "description": "", "value": "", "filled_by_llm": False, "required": False, + } + resp = _call(app, pg_conn, UpdateTool, "ed", json={"id": str(tool["id"]), "config": cfg}) + assert resp.status_code == 200, resp.json + action = _runtime_action(pg_conn, tool["id"]) + assert action["description"] == "Edited" + assert action["query_params"]["properties"]["token"]["value"] == "q-secret" + + def test_owner_may_hand_a_value_to_the_model(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool = self._tool(pg_conn) + cfg = self._masked() + self._token(cfg)["filled_by_llm"] = True + assert _call(app, pg_conn, UpdateTool, OWNER, json={"id": str(tool["id"]), "config": cfg}).status_code == 200 + assert _runtime_action(pg_conn, tool["id"])["query_params"]["properties"]["token"]["filled_by_llm"] is True + + +# --------------------------------------------------------------------------- +# 9. A connection-backed MCP tool moves only through /api/mcp_server/save +# --------------------------------------------------------------------------- +class TestConnectionBackedMCPConfig: + SAME = {"server_url": "https://mcp.example.com/mcp", "auth_type": "bearer", "transport_type": "http"} + + @pytest.mark.parametrize("route", ["UpdateTool", "UpdateToolConfig"]) + @pytest.mark.parametrize("change", [ + {"server_url": "https://other.example.org/mcp", "bearer_token": "new"}, + {"server_url": "http://mcp.example.com/mcp", "bearer_token": "new"}, + {"auth_type": "api_key", "api_key": "new"}, + ]) + def test_moving_the_server_or_auth_is_refused(self, app, pg_conn, route, change): + from docsgpt.api.user.tools import routes + + tool, connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + body = {"id": str(tool["id"]), "config": {**self.SAME, **change}} + resp = _call(app, pg_conn, getattr(routes, route), OWNER, json=body) + assert resp.status_code == 400 + assert "mcp_server/save" in resp.json["message"] + row = _row(pg_conn, tool["id"]) + assert row["config"]["server_url"] == self.SAME["server_url"] + assert row["config"]["auth_type"] == "bearer" + assert str(row["connection_id"]) == str(connection["id"]) + + def test_update_tool_config_writes_a_new_key_to_the_connection(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateToolConfig + + tool, _connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + body = {"id": str(tool["id"]), "config": {**self.SAME, "bearer_token": "tok2"}} + assert _call(app, pg_conn, UpdateToolConfig, OWNER, json=body).status_code == 200 + assert _stored_mcp_secret(pg_conn, tool["id"]) == ({"bearer_token": "tok2"}, OWNER) + + def test_update_tool_config_keeps_the_owners_connection_from_editors(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateToolConfig + + tool, _connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + _share(pg_conn, tool["id"], "ed", "editor") + body = {"id": str(tool["id"]), "config": {**self.SAME, "bearer_token": "tok2"}} + assert _call(app, pg_conn, UpdateToolConfig, "ed", json=body).status_code == 403 + assert _stored_mcp_secret(pg_conn, tool["id"]) == ({"bearer_token": "tok"}, OWNER) + + def test_same_server_other_path_still_saves(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + tool, connection = _mcp_connection_tool(pg_conn, {"bearer_token": "tok"}) + body = {"id": str(tool["id"]), "config": {**self.SAME, "server_url": "https://mcp.example.com/v2/mcp"}} + assert _call(app, pg_conn, UpdateTool, OWNER, json=body).status_code == 200 + row = _row(pg_conn, tool["id"]) + assert row["config"]["server_url"] == "https://mcp.example.com/v2/mcp" + assert str(row["connection_id"]) == str(connection["id"]) diff --git a/tests/api/user/test_tools_mcp_pg.py b/tests/api/user/test_tools_mcp_pg.py index 1a3af93b..15008265 100644 --- a/tests/api/user/test_tools_mcp_pg.py +++ b/tests/api/user/test_tools_mcp_pg.py @@ -7,6 +7,16 @@ import pytest from flask import Flask +@pytest.fixture(autouse=True) +def _mcp_servers_allowed(): + """The admin switch fails closed without a database; these tests are not about it. + + The policy itself is covered in ``tests/connectors``. + """ + with patch("docsgpt.api.user.tools.mcp._mcp_policy_error", return_value=None): + yield + + @pytest.fixture def app(): return Flask(__name__) @@ -222,6 +232,8 @@ class TestTestMCPServerConfig: "success": False, "requires_oauth": True, "auth_url": "https://auth/ex", + "task_id": "task-123", + "message": "OAuth required", } with patch( @@ -242,6 +254,8 @@ class TestTestMCPServerConfig: response = TestMCPServerConfig().post() assert response.status_code == 200 assert response.json["requires_oauth"] is True + # The client follows the sign-in by this task's events. + assert response.json["task_id"] == "task-123" def test_unexpected_exception_returns_500(self, app): from docsgpt.api.user.tools.mcp import TestMCPServerConfig @@ -318,7 +332,10 @@ class TestMCPServerSave: def test_oauth_missing_task_id_returns_400(self, app): from docsgpt.api.user.tools.mcp import MCPServerSave - with app.test_request_context( + # No OAuth task and no stored sign-in that works. + unauthorized = MagicMock() + unauthorized.discover_tools.side_effect = RuntimeError("401") + with patch("docsgpt.api.user.tools.mcp.MCPTool", return_value=unauthorized), app.test_request_context( "/api/mcp_server/save", method="POST", json={ "displayName": "Srv", @@ -334,6 +351,39 @@ class TestMCPServerSave: response = MCPServerSave().post() assert response.status_code == 400 + def test_oauth_without_task_uses_the_stored_sign_in(self, app, pg_conn): + """Signed in before: the server answers with the saved tokens, no new handshake.""" + from docsgpt.api.user.tools.mcp import MCPServerSave + + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + connection = ConnectorSessionsRepository(pg_conn).create( + "u-signed-in", "mcp:https://mcp.linear.app", connector_key="mcp:linear", auth_kind="mcp_oauth", + display_name="Linear", account_label="Linear", server_url="https://mcp.linear.app", + ) + signed_in = MagicMock() + signed_in.get_actions_metadata.return_value = [{"name": "search"}] + with _patch_db(pg_conn), patch( + "docsgpt.api.user.tools.mcp.MCPTool", return_value=signed_in, + ), patch( + "docsgpt.api.user.tools.mcp._mcp_connection", return_value=str(connection["id"]), + ), app.test_request_context( + "/api/mcp_server/save", method="POST", + json={ + "displayName": "Linear", + "config": { + "transport_type": "http", + "server_url": "https://mcp.linear.app/mcp", + "auth_type": "oauth", + }, + }, + ): + from flask import request + request.decoded_token = {"sub": "u-signed-in"} + response = MCPServerSave().post() + assert response.status_code == 200 + signed_in.discover_tools.assert_called_once() + def test_creates_mcp_tool_successfully(self, app, pg_conn): from docsgpt.api.user.tools.mcp import MCPServerSave @@ -362,6 +412,74 @@ class TestMCPServerSave: response = MCPServerSave().post() assert response.status_code in (200, 201) + def test_saving_an_existing_server_keeps_fixed_values(self, app, pg_conn): + from docsgpt.api.user.tools.mcp import MCPServerSave + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + user = "u-mcp-resave" + repo = UserToolsRepository(pg_conn) + existing = repo.create( + user, "mcp_tool", + config={"server_url": "https://example.com/mcp", "auth_type": "none"}, + display_name="My MCP", + actions=[{ + "name": "search", + "active": True, + "parameters": {"properties": { + "q": {"type": "string", "filled_by_llm": True, "value": ""}, + "team": {"type": "string", "filled_by_llm": False, "value": "ENG"}, + }}, + }], + ) + fake_tool = MagicMock() + fake_tool.get_actions_metadata.return_value = [{ + "name": "search", + "parameters": {"properties": {"q": {"type": "string"}, "team": {"type": "string"}}}, + }] + with _patch_db(pg_conn), patch( + "docsgpt.api.user.tools.mcp.MCPTool", return_value=fake_tool, + ), app.test_request_context( + "/api/mcp_server/save", method="POST", + json={ + "id": str(existing["id"]), + "displayName": "My MCP", + "config": {"transport_type": "http", "server_url": "https://example.com/mcp", "auth_type": "none"}, + }, + ): + from flask import request + request.decoded_token = {"sub": user} + response = MCPServerSave().post() + assert response.status_code == 200 + team = repo.get_any(str(existing["id"]), user)["actions"][0]["parameters"]["properties"]["team"] + assert team["filled_by_llm"] is False and team["value"] == "ENG" + + +class TestSignInServerWithoutAConnection: + def test_an_oauth_server_is_not_saved_without_its_connection(self, app, pg_conn): + """A client still cached from a removed connection can answer the + discovery; the tool must not then be saved as an unconnected server.""" + from docsgpt.api.user.tools.mcp import MCPServerSave + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + user = "u-mcp-no-connection" + fake_tool = MagicMock() + fake_tool.get_actions_metadata.return_value = [{"name": "list_issues", "parameters": {"properties": {}}}] + with _patch_db(pg_conn), patch( + "docsgpt.api.user.tools.mcp.MCPTool", return_value=fake_tool, + ), app.test_request_context( + "/api/mcp_server/save", method="POST", + json={ + "displayName": "Linear", + "config": {"transport_type": "http", "server_url": "https://mcp.linear.app/mcp", "auth_type": "oauth"}, + "status": True, + }, + ): + from flask import request + request.decoded_token = {"sub": user} + response = MCPServerSave().post() + assert response.status_code == 400 + assert UserToolsRepository(pg_conn).list_for_user(user) == [] + class TestMCPOAuthCallback: def test_error_param_redirects_error(self, app): @@ -460,3 +578,23 @@ class TestMCPAuthStatus: request.decoded_token = None response = MCPAuthStatus().get() assert response.status_code == 401 + + +class TestMCPOAuthCallbackIssuer: + def test_passes_the_issuer_to_the_waiting_sign_in(self, app): + from docsgpt.api.user.tools.mcp import MCPOAuthCallback + + manager = MagicMock() + manager.handle_oauth_callback.return_value = True + with patch( + "docsgpt.api.user.tools.mcp.get_redis_instance", return_value=MagicMock(), + ), patch( + "docsgpt.api.user.tools.mcp.MCPOAuthManager", return_value=manager, + ), app.test_request_context( + "/api/mcp_server/callback?code=c&state=s&iss=https%3A%2F%2Fmcp.linear.app", + ): + response = MCPOAuthCallback().get() + assert response.status_code == 302 + manager.handle_oauth_callback.assert_called_once_with( + "s", "c", None, iss="https://mcp.linear.app", + ) diff --git a/tests/api/user/test_tools_routes.py b/tests/api/user/test_tools_routes.py index 737c0cd6..633ebd08 100644 --- a/tests/api/user/test_tools_routes.py +++ b/tests/api/user/test_tools_routes.py @@ -1306,26 +1306,111 @@ class TestUpdateToolActionsHappy: response = UpdateToolActions().post() assert response.status_code == 404 - def test_updates_actions(self, app, pg_conn): - from docsgpt.api.user.tools.routes import UpdateToolActions + @staticmethod + def _seed_actions_tool(pg_conn, user): + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository - user = "u-actions" - tool = _seed_tool(pg_conn, user=user) + return UserToolsRepository(pg_conn).create( + user, + "telegram", + config={}, + display_name="Telegram", + description="", + actions=[ + { + "name": "telegram_send_message", + "description": "Send a message", + "active": True, + "parameters": { + "type": "object", + "properties": { + "text": {"type": "string", "filled_by_llm": True, "value": ""}, + "chat_id": {"type": "string", "filled_by_llm": True, "value": ""}, + }, + }, + } + ], + status=True, + ) + + def _post(self, app, pg_conn, user, tool_id, actions): + from docsgpt.api.user.tools.routes import UpdateToolActions with _patch_tools_db(pg_conn), app.test_request_context( "/api/update_tool_actions", method="POST", - json={ - "id": str(tool["id"]), - "actions": [ - {"name": "action_1", "active": True, "parameters": {}} - ], - }, + json={"id": tool_id, "actions": actions}, ): from flask import request request.decoded_token = {"sub": user} - response = UpdateToolActions().post() + return UpdateToolActions().post() + + @staticmethod + def _stored(pg_conn, tool): + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + return UserToolsRepository(pg_conn).get_by_id(str(tool["id"]))["actions"][0] + + def test_updates_actions(self, app, pg_conn): + user = "u-actions" + tool = self._seed_actions_tool(pg_conn, user) + response = self._post( + app, pg_conn, user, str(tool["id"]), [{"name": "telegram_send_message", "active": False}], + ) assert response.status_code == 200 + assert self._stored(pg_conn, tool)["active"] is False + + def test_owner_pins_a_parameter(self, app, pg_conn): + user = "u-actions-pin" + tool = self._seed_actions_tool(pg_conn, user) + response = self._post(app, pg_conn, user, str(tool["id"]), [{ + "name": "telegram_send_message", + "parameters": {"properties": {"chat_id": {"filled_by_llm": False, "value": "123"}}}, + }]) + assert response.status_code == 200 + chat_id = self._stored(pg_conn, tool)["parameters"]["properties"]["chat_id"] + assert chat_id["filled_by_llm"] is False and chat_id["value"] == "123" + + @pytest.mark.parametrize( + "actions", + [ + [{"name": "made_up_action", "active": True}], + [{"name": "telegram_send_message", "parameters": {"properties": {"token": {"value": "x"}}}}], + [{"active": True}], + "not a list", + ], + ) + def test_rejects_actions_that_do_not_match_the_schema(self, app, pg_conn, actions): + user = "u-actions-bad" + tool = self._seed_actions_tool(pg_conn, user) + response = self._post(app, pg_conn, user, str(tool["id"]), actions) + assert response.status_code == 400 + assert self._stored(pg_conn, tool)["active"] is True + + def test_team_editor_can_change_actions_but_not_pins(self, app, pg_conn): + from docsgpt.storage.db.repositories.team_members import TeamMembersRepository + from docsgpt.storage.db.repositories.team_resource_grants import TeamResourceGrantsRepository + from docsgpt.storage.db.repositories.teams import TeamsRepository + + owner, editor = "u-owner", "u-editor" + tool = self._seed_actions_tool(pg_conn, owner) + team = TeamsRepository(pg_conn).create("Pins", "t-pins", owner) + TeamMembersRepository(pg_conn).add_member(str(team["id"]), editor) + TeamResourceGrantsRepository(pg_conn).grant( + str(team["id"]), "tool", str(tool["id"]), owner, owner, access_level="editor", + ) + toggled = self._post( + app, pg_conn, editor, str(tool["id"]), [{"name": "telegram_send_message", "active": False}], + ) + pinned = self._post(app, pg_conn, editor, str(tool["id"]), [{ + "name": "telegram_send_message", + "parameters": {"properties": {"chat_id": {"filled_by_llm": False, "value": "666"}}}, + }]) + assert toggled.status_code == 200 + assert pinned.status_code == 403 + stored = self._stored(pg_conn, tool) + assert stored["active"] is False + assert stored["parameters"]["properties"]["chat_id"]["filled_by_llm"] is True class TestUpdateToolStatusHappy: diff --git a/tests/api/v1/test_v1_external_caller.py b/tests/api/v1/test_v1_external_caller.py new file mode 100644 index 00000000..c989c327 --- /dev/null +++ b/tests/api/v1/test_v1_external_caller.py @@ -0,0 +1,151 @@ +"""``/v1`` key holders are external callers, though the run uses the owner's identity. + +The route hands ``StreamProcessor`` the owner's token so state and logs land +under the owner, which made ``is_external_api_caller`` see the owner and let +a key holder run (or approve) writes on the owner's connected accounts. +""" + +from __future__ import annotations + +import uuid +from contextlib import ExitStack, contextmanager +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.security.encryption import encrypt_json + +SP = "docsgpt.api.answer.services.stream_processor" + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with ExitStack() as stack: + for module in (SP, "docsgpt.connectors.service", "docsgpt.connectors.resolve"): + stack.enter_context(patch(f"{module}.db_readonly", _yield)) + stack.enter_context(patch(f"{SP}.db_session", _yield)) + stack.enter_context(patch("docsgpt.connectors.service.db_session", _yield)) + yield + + +def _run_executor(processor): + """The ``ToolExecutor`` the agent run gets from ``create_agent``.""" + processor._get_prompt_content = MagicMock(return_value="p") + processor.prompt_renderer = MagicMock(render_prompt=MagicMock(return_value="p")) + processor._enabled_tool_names = MagicMock(return_value=set()) + processor.model_id = "m1" + with patch(f"{SP}.get_provider_from_model_id", return_value="openai"), \ + patch(f"{SP}.get_api_key_for_provider", return_value="key"), \ + patch("docsgpt.llm.llm_creator.LLMCreator.create_llm", return_value=MagicMock()), \ + patch("docsgpt.llm.handlers.handler_creator.LLMHandlerCreator.create_handler"), \ + patch("docsgpt.agents.agent_creator.AgentCreator.create_agent") as create: + processor.create_agent() + return create.call_args.kwargs["tool_executor"] + + +class TestV1KeyHolderWrites: + def test_connected_write_is_refused_for_a_v1_key_holder(self, pg_conn): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + from docsgpt.storage.db.repositories.agents import AgentsRepository + + key = f"k-{uuid.uuid4().hex}" + AgentsRepository(pg_conn).create("alice", "A", "published", key=key) + connection = str(pg_conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "encrypted_credentials) VALUES ('alice', 'telegram', 'telegram', 'api_key', 'connected', :e) " + "RETURNING id" + ), + {"e": encrypt_json({"credentials": {"token": "t"}}, "alice")}, + ).scalar()) + tool = { + "id": "tool-1", "user_id": "alice", "name": "telegram", "config": {}, + "actions": [{"name": "telegram_send_message", "active": True, "require_approval": False}], + "connection_id": connection, "credential_mode": "owner", + } + + # What /v1 builds: the key in the body and the owner's token. + processor = StreamProcessor({"api_key": key}, {"sub": "alice"}, external_caller=True) + with _db(pg_conn): + processor._configure_agent() + executor = _run_executor(processor) + call = SimpleNamespace(id="c1", name="telegram_send_message", arguments="{}", thought_signature=None) + with patch("docsgpt.agents.tool_executor.ToolActionParser") as parser: + parser.return_value.parse_args.return_value = ("t1", "telegram_send_message", {}) + pause = executor.check_pause({"t1": tool}, call, "OpenAILLM") + assert executor.external_caller is True + assert pause["pause_type"] == "headless_denied" + assert pause["error_type"] == "tool_not_allowed" + + def test_resume_stays_external_even_when_state_says_otherwise(self, monkeypatch): + """State saved before this fix recorded the key holder as not external.""" + from docsgpt.agents import agent_creator as ac_mod + from docsgpt.api.answer.services import continuation_service as cont_mod + from docsgpt.api.answer.services.stream_processor import StreamProcessor + from docsgpt.llm import llm_creator as llm_creator_mod + from docsgpt.llm.handlers import handler_creator as handler_mod + + cont_service = MagicMock() + cont_service.claim_state.return_value = { + "messages": [], "pending_tool_calls": [], "tools_dict": {}, "tool_schemas": [], + "client_tools": None, + "agent_config": {"model_id": "m1", "llm_name": "openai", "api_key": "k", "user_api_key": "uk", + "agent_type": "ClassicAgent", "external_api_caller": False}, + } + monkeypatch.setattr(cont_mod, "ContinuationService", lambda: cont_service) + monkeypatch.setattr(llm_creator_mod.LLMCreator, "create_llm", lambda *a, **kw: MagicMock()) + monkeypatch.setattr(handler_mod.LLMHandlerCreator, "create_handler", lambda *a, **kw: MagicMock()) + created = {} + monkeypatch.setattr( + ac_mod.AgentCreator, "create_agent", lambda *a, **kw: created.update(kw) or MagicMock(), + ) + processor = StreamProcessor({}, {"sub": "alice"}, external_caller=True) + processor.resume_from_tool_actions(tool_actions=[], conversation_id=str(uuid.uuid4())) + assert created["tool_executor"].external_caller is True + + +@pytest.mark.unit +class TestV1Route: + def test_route_marks_the_processor_external(self): + from docsgpt.api.v1.routes import v1_bp + + app = Flask(__name__) + app.register_blueprint(v1_bp) + processor = MagicMock() + processor.decoded_token = {"sub": "owner"} + processor.agent_config = {"user_api_key": "k"} + processor.agent_id = None + processor.build_agent.return_value = MagicMock() + helper = MagicMock() + helper.check_usage.return_value = None + helper.complete_stream.return_value = iter(['data: {"type": "end"}']) + helper.process_response_stream.return_value = { + "error": None, "conversation_id": "c", "answer": "ok", "sources": [], "tool_calls": [], "thought": "", + } + + @contextmanager + def _conn(): + yield MagicMock() + + with patch("docsgpt.api.v1.routes._lookup_agent", + return_value={"id": "agent-1", "name": "Agent", "user_id": "owner"}), \ + patch("docsgpt.api.v1.routes.translate_request", + return_value={"question": "hi", "api_key": "k"}), \ + patch("docsgpt.api.v1.routes.StreamProcessor", return_value=processor) as cls, \ + patch("docsgpt.api.v1.routes._V1AnswerHelper", return_value=helper), \ + patch("docsgpt.api.v1.routes.db_readonly", _conn), \ + patch("docsgpt.api.v1.routes.translate_response", return_value={"id": "x", "choices": []}): + with app.test_client() as client: + resp = client.post( + "/v1/chat/completions", headers={"Authorization": "Bearer k"}, + json={"messages": [{"role": "user", "content": "hi"}]}, + ) + assert resp.status_code == 200 + assert cls.call_args.kwargs.get("external_caller") is True diff --git a/tests/api/v1/test_v1_tool_pause_finalization.py b/tests/api/v1/test_v1_tool_pause_finalization.py index 3c1878dc..c81479e4 100644 --- a/tests/api/v1/test_v1_tool_pause_finalization.py +++ b/tests/api/v1/test_v1_tool_pause_finalization.py @@ -887,10 +887,10 @@ class _PauseThenAnswerAgent: yield {"answer": self.ANSWER_TEXT} -def _seed_agent(conn, user_id: str, key: str) -> None: +def _seed_agent(conn, user_id: str, key: str) -> str: from docsgpt.storage.db.repositories.agents import AgentsRepository - AgentsRepository(conn).create(user_id, "Weather Agent", "published", key=key) + return str(AgentsRepository(conn).create(user_id, "Weather Agent", "published", key=key)["id"]) @contextmanager @@ -898,12 +898,14 @@ def _wire_v1_route_db(engine, monkeypatch): """Full route-level DB wiring for the ``/v1/chat/completions`` blueprint. Extends ``_wire_db`` (conversation/continuation/base services) with the v1 - routes module's own ``db_readonly`` (used by ``_lookup_agent``) and a fake + routes module's own ``db_readonly`` (used by ``_lookup_agent``), the + stream processor's (the resume looks up the key's agent), and a fake title-gen ``LLMCreator`` on the base module, so a real two-POST round-trip runs entirely against the ephemeral Postgres with no live LLM/provider. """ from docsgpt.api.v1 import routes as v1_routes_mod from docsgpt.api.answer.routes import base as base_mod + from docsgpt.api.answer.services import stream_processor as sp_mod @contextmanager def _readonly(): @@ -915,6 +917,7 @@ def _wire_v1_route_db(engine, monkeypatch): with _wire_db(engine, monkeypatch): monkeypatch.setattr(v1_routes_mod, "db_readonly", _readonly) + monkeypatch.setattr(sp_mod, "db_readonly", _readonly) monkeypatch.setattr( base_mod.LLMCreator, "create_llm", @@ -993,14 +996,17 @@ class TestV1ToolRoundTripEndToEnd: api_key = f"key-{uuid.uuid4().hex[:8]}" with pg_engine.begin() as conn: _seed_user(conn, user_id) - _seed_agent(conn, user_id, api_key) + agent_id = _seed_agent(conn, user_id, api_key) app = self._build_app() # ``build_agent`` is the only mock — a fresh pausing/answering agent # per call. The route's ``build_continuation_from_messages`` calls this # internally on POST #2, so the rebuild itself still runs for real. + # It binds the agent as ``_configure_agent`` would, so the paused turn + # is saved under the agent the key resumes. def _fake_build_agent(self, question): # noqa: ARG001 + self.agent_id = agent_id return _PauseThenAnswerAgent( TestV1ToolRoundTripEndToEnd.PENDING ) diff --git a/tests/connectors/__init__.py b/tests/connectors/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/tests/connectors/conftest.py b/tests/connectors/conftest.py new file mode 100644 index 00000000..579b5ff2 --- /dev/null +++ b/tests/connectors/conftest.py @@ -0,0 +1,24 @@ +"""Shared fixtures for connector tests.""" + +from __future__ import annotations + +import pytest + + +@pytest.fixture(autouse=True) +def _oauth_connectors_configured(monkeypatch): + """Give the OAuth connectors their server settings. + + A connector without them is off and hidden from members, so tests about + anything else start from configured connectors. A test about setup sets + a setting back to None itself. + """ + from docsgpt.core.settings import settings + + for name in ( + "GOOGLE_CLIENT_ID", "GOOGLE_CLIENT_SECRET", + "MICROSOFT_CLIENT_ID", "MICROSOFT_CLIENT_SECRET", + "CONFLUENCE_CLIENT_ID", "CONFLUENCE_CLIENT_SECRET", + ): + if not getattr(settings, name, None): + monkeypatch.setattr(settings, name, f"test-{name.lower()}") diff --git a/tests/connectors/test_catalog.py b/tests/connectors/test_catalog.py new file mode 100644 index 00000000..9fa6f2ef --- /dev/null +++ b/tests/connectors/test_catalog.py @@ -0,0 +1,129 @@ +"""Tests for the connector catalog.""" + +from __future__ import annotations + +import pytest + +from docsgpt.connectors import catalog + + +@pytest.fixture(autouse=True) +def _fresh_registry(): + catalog.reset_registry_for_tests() + yield + catalog.reset_registry_for_tests() + + +class TestDefinitions: + def test_built_ins_are_registered(self): + keys = {d.key for d in catalog.all_definitions()} + assert { + "google_drive", "share_point", "confluence", "s3", "reddit", + "brave", "telegram", "ntfy", "postgres", "custom_mcp", "custom_openapi", + } <= keys + + def test_every_definition_uses_known_values(self): + for definition in catalog.all_definitions(): + assert definition.category in catalog.CATEGORIES, definition.key + assert definition.auth_kind in catalog.AUTH_KINDS, definition.key + assert definition.publisher in catalog.PUBLISHERS, definition.key + assert set(definition.capabilities) <= {"sync", "read", "write"}, definition.key + + def test_tool_connectors_match_tool_config_requirements(self): + """Credential fields must be the keys the tool reads its secrets from.""" + import docsgpt.api.user # noqa: F401 (mcp_tool imports it; load it first) + from docsgpt.agents.tools.tool_manager import ToolManager + + tools = ToolManager(config={}).tools + for definition in catalog.all_definitions(): + if definition.publisher != "built_in": + continue + for tool_name in definition.tool_templates: + if tool_name == "mcp_tool": + # GitHub's MCP server gets the connection's token as a bearer token. + continue + requirements = tools[tool_name].get_config_requirements() + secret_keys = {k for k, spec in requirements.items() if spec.get("secret")} + field_keys = {f.key for f in definition.credential_fields} + assert secret_keys <= field_keys, definition.key + + def test_to_dict_has_no_server_secrets(self): + payload = catalog.get_definition("google_drive").to_dict() + assert "required_settings" not in payload + assert payload["capabilities"] == ["sync"] + + +class TestAvailability: + def test_missing_settings_listed(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", None) + monkeypatch.setattr(settings, "GOOGLE_CLIENT_SECRET", "secret") + definition = catalog.get_definition("google_drive") + assert definition.missing_settings == ["GOOGLE_CLIENT_ID"] + assert not definition.configured + + def test_configured_when_all_set(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", "id") + monkeypatch.setattr(settings, "GOOGLE_CLIENT_SECRET", "secret") + assert catalog.get_definition("google_drive").configured + + def test_api_key_connectors_need_no_settings(self): + assert catalog.get_definition("telegram").configured + + +class TestRowMapping: + def test_oauth_provider_maps_to_key(self): + assert catalog.connector_key_for_row({"provider": "share_point"}) == "share_point" + + def test_legacy_mcp_row_is_custom(self): + row = {"provider": "mcp:https://mcp.unknown.dev", "server_url": "https://mcp.unknown.dev"} + assert catalog.connector_key_for_row(row) == "custom_mcp" + + def test_stored_key_wins(self): + assert catalog.connector_key_for_row({"provider": "telegram", "connector_key": "telegram"}) == "telegram" + + def test_unknown_provider(self): + assert catalog.connector_key_for_row({"provider": "nope"}) is None + + def test_definition_for_tool(self): + assert catalog.definition_for_tool("telegram").key == "telegram" + assert catalog.definition_for_tool("memory") is None + assert catalog.definition_for_tool("mcp_tool") is None + + +class TestPresets: + def test_presets_load_from_yaml(self, tmp_path, monkeypatch): + presets = tmp_path / "mcp.yaml" + presets.write_text( + "- key: mcp:example\n" + " name: Example\n" + " description: Example records.\n" + " icon: example\n" + " category: knowledge\n" + " mcp_url: https://mcp.example.com/mcp\n" + " auth_kind: mcp_oauth\n" + ) + monkeypatch.setattr(catalog, "_PRESETS_FILE", presets) + definition = catalog.get_definition("mcp:example") + assert definition.publisher == "preset" + assert definition.mcp_base_url == "https://mcp.example.com" + assert catalog.preset_for_url("https://mcp.example.com/other").key == "mcp:example" + row = {"provider": "mcp:https://mcp.example.com", "server_url": "https://mcp.example.com"} + assert catalog.connector_key_for_row(row) == "mcp:example" + + def test_base_url(self): + assert catalog.base_url("https://a.example.com:8443/x/y") == "https://a.example.com:8443" + assert catalog.base_url("not a url") == "" + + +def test_atlassian_preset_is_part_of_confluence(): + """One Confluence card: syncing pages and the Jira/Confluence agent actions.""" + from docsgpt.connectors import catalog + + atlassian = catalog.get_definition("mcp:atlassian") + assert atlassian.part_of == "confluence" + assert atlassian.to_dict()["part_of"] == "confluence" + assert catalog.get_definition("confluence").to_dict()["part_of"] is None diff --git a/tests/connectors/test_connections_api.py b/tests/connectors/test_connections_api.py new file mode 100644 index 00000000..b81f8c06 --- /dev/null +++ b/tests/connectors/test_connections_api.py @@ -0,0 +1,565 @@ +"""Tests for the connection write endpoints.""" + +from __future__ import annotations + +import json +from contextlib import contextmanager +from unittest.mock import MagicMock, patch + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.connectors import service +from docsgpt.security.encryption import encrypt_json + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.api.connector.connections", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.api.connector.routes", db_session=_yield, db_readonly=_yield): + yield + + +def _call(app, resource, method, path, user="alice", body=None, headers=None, args=()): + with app.test_request_context(path, method=method.upper(), json=body, headers=headers or {}): + from flask import request + + request.decoded_token = {"sub": user} if user else None + return getattr(resource(), method)(*args) + + +def _connection(conn, user="alice", provider="telegram", auth_kind="api_key", secrets=None, status="connected", + **cols) -> str: + values = { + "user_id": user, "provider": provider, "connector_key": provider, "auth_kind": auth_kind, + "status": status, "account_label": cols.pop("account_label", "…abcd"), + "encrypted_credentials": encrypt_json(secrets, user) if secrets is not None else None, **cols, + } + names = ", ".join(values) + params = ", ".join(f":{k}" for k in values) + return str(conn.execute( + text(f"INSERT INTO connector_sessions ({names}) VALUES ({params}) RETURNING id"), values, + ).scalar()) + + +class TestCreate: + def test_telegram_zero_question_setup(self, app, pg_conn): + """Appendix C of the connectors spec: create, then set up, tools appear.""" + from docsgpt.api.connector.connections import ConnectionSetup, ConnectionsList + + with _db(pg_conn): + created = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", "credentials": {"token": "123:abcdefgh"}}) + assert created.status_code == 201 + payload = created.get_json() + assert payload["setup"] == {"tools": "auto", "sync": "off"} + assert "token" not in json.dumps(payload) + cid = payload["connection"]["id"] + setup = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True}, args=[cid]) + assert setup.status_code == 200 + tools = setup.get_json()["tools"] + assert [t["name"] for t in tools] == ["telegram"] + actions = {a["name"]: a for a in tools[0]["actions"]} + assert actions["telegram_send_message"]["access"] == "write" + assert actions["telegram_send_message"]["permission"] == "ask" + + def test_same_bot_with_another_default_chat_is_another_connection(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn): + first = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", "credentials": {"token": "123:abcdefgh"}}) + second = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", + "credentials": {"token": "123:abcdefgh", "chat_id": "-1001"}}) + assert first.status_code == 201 and second.status_code == 201 + assert first.get_json()["connection"]["id"] != second.get_json()["connection"]["id"] + + def test_setup_is_idempotent_for_tools(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + for _ in range(2): + _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True}, args=[cid]) + count = pg_conn.execute( + text("SELECT count(*) FROM user_tools WHERE connection_id = CAST(:c AS uuid)"), {"c": cid}, + ).scalar() + assert count == 1 + + def test_rejects_oauth_connector(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "google_drive", "credentials": {}}) + assert resp.status_code == 400 + + def test_default_key_refused(self, app, pg_conn, monkeypatch): + from docsgpt.api.connector.connections import ConnectionsList + from docsgpt.core.settings import settings + from docsgpt.security.encryption import DEFAULT_ENCRYPTION_KEY + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", DEFAULT_ENCRYPTION_KEY) + monkeypatch.setattr(settings, "AUTH_TYPE", "oidc") + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "brave", "credentials": {"token": "some-long-key"}}) + assert resp.status_code == 400 + assert resp.get_json()["code"] == "encryption_key_default" + + +class TestSetupSync: + def test_oauth_source_is_queued_with_connection(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + task = MagicMock(id="task-1") + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_connector_task.apply_async", + return_value=task) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "create_tools": True, + "sync": {"items": {"folder_ids": ["f1"]}, "frequency": "weekly", "name": "Handbook"}, + }, args=[cid]) + assert resp.status_code == 200 + source = resp.get_json()["sources"][0] + assert source["name"] == "Handbook" and source["sync_frequency"] == "weekly" + kwargs = apply.call_args.kwargs["kwargs"] + assert kwargs["connection_id"] == cid + assert kwargs["folder_ids"] == ["f1"] + assert "session_token" not in kwargs + # Drive has no tools; nothing was created. + assert resp.get_json()["tools"] == [] + + def test_retrieval_settings_apply_to_the_synced_source(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + config = {"chunking": {"strategy": "markdown", "max_tokens": 800}, "retrieval": {"chunks": 4}} + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_connector_task.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"folder_ids": ["f1"]}, "config": config}, + }, args=[cid]) + assert resp.status_code == 200 + sent = apply.call_args.kwargs["kwargs"]["config"] + # Validated and filled in like an upload's config. + assert sent["chunking"]["strategy"] == "markdown" + assert sent["chunking"]["max_tokens"] == 800 + assert sent["retrieval"]["chunks"] == 4 + assert sent["kind"] == "classic" + + def test_remote_sync_carries_retrieval_settings(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="s3", secrets={"credentials": { + "aws_access_key_id": "AKIA", "aws_secret_access_key": "shh-secret"}}) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_remote.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"bucket": "docs"}, "config": {"retrieval": {"chunks": 3}}}, + }, args=[cid]) + assert resp.status_code == 200 + assert apply.call_args.kwargs["kwargs"]["config"]["retrieval"]["chunks"] == 3 + + def test_no_retrieval_settings_keeps_the_defaults(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_connector_task.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"folder_ids": ["f1"]}}, + }, args=[cid]) + assert resp.status_code == 200 + assert apply.call_args.kwargs["kwargs"]["config"] is None + + @pytest.mark.parametrize("config", [ + {"retrieval": {"chunks": 0}}, + {"unknown": True}, + {"retrieval": {"chunks": 6, "prescreen": {"candidate_k": 4, "max_keep": 8}}}, + "not an object", + ]) + def test_invalid_retrieval_settings_are_refused_before_queueing(self, app, pg_conn, config): + from docsgpt.api.connector.connections import ConnectionSetup + + @contextmanager + def _yield(): + yield pg_conn + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + headers = {"Idempotency-Key": f"setup-config-{hash(str(config))}"} + with _db(pg_conn), patch("docsgpt.api.user.sources.upload.db_session", _yield), patch( + "docsgpt.api.user.tasks.ingest_connector_task.apply_async", return_value=MagicMock(id="t"), + ) as apply: + bad = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"folder_ids": ["f1"]}, "config": config}, + }, headers=headers, args=[cid]) + apply.assert_not_called() + # The refusal left the key free for the corrected retry. + fixed = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"folder_ids": ["f1"]}}, + }, headers=headers, args=[cid]) + assert bad.status_code == 400 + assert bad.get_json()["error"] == "Invalid source config" + assert fixed.status_code == 200 + + def test_nothing_picked_is_a_bad_request(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + with _db(pg_conn): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"sync": {"items": {}}}, args=[cid]) + assert resp.status_code == 400 + + def test_rejected_request_does_not_claim_its_idempotency_key(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + @contextmanager + def _yield(): + yield pg_conn + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={"token_info": {}}) + headers = {"Idempotency-Key": "setup-retry-1"} + with _db(pg_conn), patch("docsgpt.api.user.sources.upload.db_session", _yield), patch( + "docsgpt.api.user.tasks.ingest_connector_task.apply_async", return_value=MagicMock(id="t"), + ) as apply: + bad = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"sync": {"items": {}}}, headers=headers, args=[cid]) + fixed = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"sync": {"items": {"folder_ids": ["f1"]}}}, headers=headers, args=[cid]) + assert bad.status_code == 400 + assert fixed.status_code == 200 + apply.assert_called_once() + + def test_s3_keys_stay_on_the_connection(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, provider="s3", secrets={"credentials": { + "aws_access_key_id": "AKIA", "aws_secret_access_key": "shh-secret"}}) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_remote.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "sync": {"items": {"bucket": "docs", "aws_secret_access_key": "smuggled"}}, + }, args=[cid]) + assert resp.status_code == 200 + kwargs = apply.call_args.kwargs["kwargs"] + assert kwargs["source_data"] == {"bucket": "docs"} + assert kwargs["connection_id"] == cid + + def test_flagged_connection_must_reconnect_first(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn, status="reconnect_needed", secrets={}) + with _db(pg_conn): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={}, args=[cid]) + assert resp.status_code == 409 + + +class TestReconnect: + def test_api_key_reconnect_replaces_and_resumes(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionReconnect + + cid = _connection(pg_conn, status="reconnect_needed", secrets={"credentials": {"token": "old"}}) + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, connection_id, metadata) " + "VALUES ('alice', 's', CAST(:c AS uuid), '{\"sync_state\": \"paused_reconnect\"}')" + ), {"c": cid}) + with _db(pg_conn): + resp = _call(app, ConnectionReconnect, "post", f"/api/connections/{cid}/reconnect", + body={"credentials": {"token": "new"}}, args=[cid]) + assert resp.status_code == 200 + assert resp.get_json()["connection"]["status"] == "connected" + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:c AS uuid)"), + {"c": cid}).one()._mapping + assert service.read_secrets(dict(row))["credentials"]["token"] == "new" + meta = pg_conn.execute(text("SELECT metadata FROM sources WHERE user_id = 'alice'")).scalar() + assert "sync_state" not in meta + + def test_reconnect_recovers_undecryptable_credentials_without_a_second_write(self, app, pg_conn): + """After a lost key the reconnect replaces the blob; it must not flag the row it holds locked.""" + from docsgpt.api.connector.connections import ConnectionReconnect + + cid = _connection(pg_conn, status="reconnect_needed") + pg_conn.execute(text( + "UPDATE connector_sessions SET encrypted_credentials = :blob WHERE id = CAST(:c AS uuid)" + ), {"c": cid, "blob": encrypt_json({"credentials": {"token": "old"}}, "someone-else")}) + with _db(pg_conn), patch.object(service, "mark_reconnect_needed") as flag: + resp = _call(app, ConnectionReconnect, "post", f"/api/connections/{cid}/reconnect", + body={"credentials": {"token": "new"}}, args=[cid]) + assert resp.status_code == 200 + flag.assert_not_called() + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:c AS uuid)"), + {"c": cid}).one()._mapping + assert service.read_secrets(dict(row))["credentials"] == {"token": "new"} + + def test_oauth_reconnect_returns_authorization_url(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionReconnect + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={}) + fake_auth = MagicMock() + fake_auth.get_authorization_url.return_value = "https://accounts.example/auth" + with _db(pg_conn), patch("docsgpt.api.connector.routes.ConnectorCreator.create_auth", + return_value=fake_auth): + resp = _call(app, ConnectionReconnect, "post", f"/api/connections/{cid}/reconnect", + body={}, args=[cid]) + assert resp.get_json()["authorization_url"] == "https://accounts.example/auth" + state = fake_auth.get_authorization_url.call_args.kwargs["state"] + import base64 + + assert json.loads(base64.urlsafe_b64decode(state))["object_id"] == cid + + def test_cannot_reconnect_someone_elses(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionReconnect + + cid = _connection(pg_conn, user="bob", secrets={}) + with _db(pg_conn): + resp = _call(app, ConnectionReconnect, "post", f"/api/connections/{cid}/reconnect", + body={"credentials": {"token": "x"}}, args=[cid]) + assert resp.status_code == 404 + + +class TestPickerToken: + def test_returns_access_token_only(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionPickerToken + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", + secrets={"token_info": {"access_token": "at", "refresh_token": "rt"}}) + fake_auth = MagicMock() + fake_auth.is_token_expired.return_value = False + with _db(pg_conn), patch("docsgpt.parser.connectors.connector_creator.ConnectorCreator.create_auth", + return_value=fake_auth): + resp = _call(app, ConnectionPickerToken, "post", f"/api/connections/{cid}/picker-token", + args=[cid]) + payload = resp.get_json() + assert payload["access_token"] == "at" + assert "rt" not in json.dumps(payload) + + def test_owner_only(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionPickerToken + + cid = _connection(pg_conn, user="bob", provider="google_drive", auth_kind="oauth", secrets={}) + with _db(pg_conn): + resp = _call(app, ConnectionPickerToken, "post", f"/api/connections/{cid}/picker-token", + args=[cid]) + assert resp.status_code == 404 + + +class TestClaim: + def test_claims_own_token(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionClaim + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={}, session_token="legacy") + with _db(pg_conn): + resp = _call(app, ConnectionClaim, "post", "/api/connections/claim", + body={"provider": "google_drive", "session_token": "legacy"}) + assert resp.get_json()["connection_id"] == cid + + def test_other_users_token_is_not_found(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionClaim + + _connection(pg_conn, user="bob", provider="google_drive", auth_kind="oauth", secrets={}, session_token="b") + with _db(pg_conn): + resp = _call(app, ConnectionClaim, "post", "/api/connections/claim", + body={"provider": "google_drive", "session_token": "b"}) + assert resp.status_code == 404 + + +class TestPermissions: + def test_set_permissions(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + tool = service.ensure_connection_tools(pg_conn, "alice", service.ConnectorSessionsRepository( + pg_conn).get(cid))[0] + resp = _call(app, ConnectionToolPermissions, "put", + f"/api/connections/{cid}/tools/{tool['id']}/permissions", + body={"permissions": {"telegram_send_image": "off", "telegram_send_message": "always"}}, + args=[cid, str(tool["id"])]) + actions = {a["name"]: a["permission"] for a in resp.get_json()["tool"]["actions"]} + assert actions == {"telegram_send_message": "always", "telegram_send_image": "off"} + + def test_rejects_unknown_permission(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={}) + with _db(pg_conn): + resp = _call(app, ConnectionToolPermissions, "put", "/x", body={"permissions": {"a": "maybe"}}, + args=[cid, "t"]) + assert resp.status_code == 400 + + +class TestParameters: + @staticmethod + def _tool(pg_conn, user="alice"): + cid = _connection(pg_conn, user=user, secrets={"credentials": {"token": "t"}}) + tool = service.ensure_connection_tools(pg_conn, user, service.ConnectorSessionsRepository(pg_conn).get(cid))[0] + return cid, str(tool["id"]) + + @staticmethod + def _put(app, cid, tool_id, body, user="alice"): + from docsgpt.api.connector.connections import ConnectionToolParameters + + return _call(app, ConnectionToolParameters, "put", f"/api/connections/{cid}/tools/{tool_id}/parameters", + user=user, body=body, args=[cid, tool_id]) + + @staticmethod + def _parameters(payload, action="telegram_send_message"): + actions = {a["name"]: a for a in payload["tool"]["actions"]} + return {p["name"]: p for p in actions[action]["parameters"]} + + def test_serialized_actions_list_their_parameters(self, pg_conn): + cid, _ = self._tool(pg_conn) + detail = service.connection_detail(pg_conn, service.ConnectorSessionsRepository(pg_conn).get(cid)) + action = next(a for a in detail["tools"][0]["actions"] if a["name"] == "telegram_send_message") + names = [p["name"] for p in action["parameters"]] + assert names == ["text", "chat_id"] + chat_id = action["parameters"][1] + assert chat_id["fixed"] is False and chat_id["value"] is None + assert chat_id["type"] == "string" + assert chat_id["description"] + + def test_owner_fixes_and_releases_a_parameter(self, app, pg_conn): + cid, tool_id = self._tool(pg_conn) + with _db(pg_conn): + fixed = self._put(app, cid, tool_id, {"action": "telegram_send_message", + "parameters": {"chat_id": "-1001"}}) + assert fixed.status_code == 200 + chat_id = self._parameters(fixed.get_json())["chat_id"] + assert chat_id["fixed"] is True and chat_id["value"] == "-1001" + released = self._put(app, cid, tool_id, {"action": "telegram_send_message", + "parameters": {"chat_id": None}}) + chat_id = self._parameters(released.get_json())["chat_id"] + assert chat_id["fixed"] is False and chat_id["value"] is None + + @pytest.mark.parametrize( + "body", + [ + {"action": "telegram_send_message", "parameters": {"token": "x"}}, + {"action": "not_an_action", "parameters": {"chat_id": "1"}}, + {"action": "telegram_send_message", "parameters": {"chat_id": ""}}, + {"action": "telegram_send_message", "parameters": "chat_id"}, + {"parameters": {"chat_id": "1"}}, + ], + ) + def test_rejects_what_the_action_does_not_have(self, app, pg_conn, body): + cid, tool_id = self._tool(pg_conn) + with _db(pg_conn): + resp = self._put(app, cid, tool_id, body) + assert resp.status_code == 400 + + def test_a_chat_set_on_the_account_shows_as_set_there(self, app, pg_conn): + cid = _connection(pg_conn, secrets={"credentials": {"token": "t", "chat_id": "-1001"}}) + row = service.ConnectorSessionsRepository(pg_conn).get(cid) + tool_id = str(service.ensure_connection_tools(pg_conn, "alice", row)[0]["id"]) + detail = service.connection_detail(pg_conn, row) + action = next(a for a in detail["tools"][0]["actions"] if a["name"] == "telegram_send_message") + chat_id = next(p for p in action["parameters"] if p["name"] == "chat_id") + assert chat_id == {**chat_id, "fixed": True, "value": "-1001", "set_by": "account"} + with _db(pg_conn): + resp = self._put(app, cid, tool_id, {"action": "telegram_send_message", "parameters": {"text": "hi"}}) + chat_id = self._parameters(resp.get_json())["chat_id"] + assert chat_id["set_by"] == "account" + + def test_another_user_cannot_fix_values(self, app, pg_conn): + cid, tool_id = self._tool(pg_conn) + with _db(pg_conn): + resp = self._put(app, cid, tool_id, {"action": "telegram_send_message", + "parameters": {"chat_id": "666"}}, user="mallory") + assert resp.status_code == 404 + stored = pg_conn.execute( + text("SELECT actions FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": tool_id} + ).scalar() + chat_id = stored[0]["parameters"]["properties"]["chat_id"] + assert chat_id["filled_by_llm"] is True + + +class TestRename: + @staticmethod + def _patch(app, cid, body, user="alice"): + from docsgpt.api.connector.connections import ConnectionDetail + + return _call(app, ConnectionDetail, "patch", f"/api/connections/{cid}", user=user, body=body, args=[cid]) + + def test_owner_names_and_unnames_an_account(self, app, pg_conn): + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + named = self._patch(app, cid, {"name": " Alerts bot "}) + assert named.status_code == 200 + connection = named.get_json()["connection"] + assert connection["account_name"] == "Alerts bot" + # The label stays the account's identity. + assert connection["account_label"] == "…abcd" + cleared = self._patch(app, cid, {"name": ""}) + assert cleared.get_json()["connection"]["account_name"] is None + + @pytest.mark.parametrize("body", [{"name": "x" * 81}, {"name": 5}, {}]) + def test_rejects_bad_names(self, app, pg_conn, body): + cid = _connection(pg_conn, secrets={}) + with _db(pg_conn): + assert self._patch(app, cid, body).status_code == 400 + + def test_another_user_cannot_rename(self, app, pg_conn): + cid = _connection(pg_conn, secrets={}) + with _db(pg_conn): + assert self._patch(app, cid, {"name": "Mine now"}, user="mallory").status_code == 404 + name = pg_conn.execute( + text("SELECT account_name FROM connector_sessions WHERE id = CAST(:i AS uuid)"), {"i": cid} + ).scalar() + assert name is None + + +class TestDelete: + def test_delete_with_source_removal(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={}) + pg_conn.execute(text("INSERT INTO sources (user_id, name, connection_id) VALUES ('alice', 's', CAST(:c AS uuid))"), + {"c": cid}) + with _db(pg_conn), patch("docsgpt.connectors.service.revoke_at_provider"), patch( + "docsgpt.api.user.sources.routes.delete_source", return_value=True + ) as delete_source: + resp = _call(app, ConnectionDetail, "delete", f"/api/connections/{cid}", + body={"sources": "delete"}, args=[cid]) + assert resp.status_code == 200 + assert delete_source.call_count == 1 + + def test_bad_mode(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail + + cid = _connection(pg_conn, secrets={}) + with _db(pg_conn): + resp = _call(app, ConnectionDetail, "delete", "/x", body={"sources": "burn"}, args=[cid]) + assert resp.status_code == 400 + + +class TestNoSecretsInResponses: + def test_list_and_detail(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail, ConnectionsList + + cid = _connection(pg_conn, secrets={"credentials": {"token": "super-secret-value"}}, + session_token="sess-tok") + with _db(pg_conn): + listed = _call(app, ConnectionsList, "get", "/api/connections") + detail = _call(app, ConnectionDetail, "get", f"/api/connections/{cid}", args=[cid]) + for resp in (listed, detail): + body = json.dumps(resp.get_json()) + assert "super-secret-value" not in body + assert "sess-tok" not in body + assert "v2:" not in body diff --git a/tests/connectors/test_connections_api_edges.py b/tests/connectors/test_connections_api_edges.py new file mode 100644 index 00000000..a1ec374a --- /dev/null +++ b/tests/connectors/test_connections_api_edges.py @@ -0,0 +1,218 @@ +"""Auth, ownership and failure paths of the connections API.""" + +from __future__ import annotations + +from unittest.mock import patch + +import pytest + +from docsgpt.connectors import service +from tests.connectors.test_connections_api import _call, _connection, _db, app # noqa: F401 (fixture) + +_CID = "00000000-0000-0000-0000-000000000001" + + +def _resources(): + from docsgpt.api.connector import connections as c + + return [ + (c.ConnectorCatalog, "get", ()), + (c.ConnectionsList, "get", ()), + (c.ConnectionsList, "post", ()), + (c.ConnectionDetail, "get", (_CID,)), + (c.ConnectionDetail, "delete", (_CID,)), + (c.ConnectionDisconnect, "post", (_CID,)), + (c.ConnectionSetup, "post", (_CID,)), + (c.ConnectionReconnect, "post", (_CID,)), + (c.ConnectionPickerToken, "post", (_CID,)), + (c.ConnectionClaim, "post", ()), + (c.ConnectionToolPermissions, "put", (_CID, "t1")), + (c.ConnectionRefreshTools, "post", (_CID,)), + (c.ToolCredentialMode, "put", ("t1",)), + ] + + +@pytest.mark.parametrize("index", range(13)) +def test_every_endpoint_needs_a_signed_in_user(app, index): # noqa: F811 + resource, method, args = _resources()[index] + resp = _call(app, resource, method, "/api/connections", user=None, body={}, args=args) + assert resp.status_code == 401 + + +@pytest.mark.parametrize("index", [3, 4, 5, 6, 7, 8, 11]) +def test_someone_elses_connection_is_not_found(app, pg_conn, index): # noqa: F811 + resource, method, _ = _resources()[index] + cid = _connection(pg_conn, user="victim", secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + resp = _call(app, resource, method, f"/api/connections/{cid}", body={"credentials": {"token": "x"}}, + args=(cid,)) + assert resp.status_code == 404 + + +class TestCreateErrors: + def test_oauth_connector_does_not_take_pasted_keys(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "google_drive", "credentials": {"token": "x"}}) + assert resp.status_code == 400 + + def test_credentials_must_be_an_object(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", "credentials": "123:abc"}) + assert resp.status_code == 400 + + def test_disabled_connector_is_forbidden(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionsList + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", "credentials": {"token": "123456:ABCDEFG"}}) + assert resp.status_code == 403 + assert resp.get_json()["code"] == "disabled" + + def test_missing_field_is_a_bad_request(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "telegram", "credentials": {}}) + assert resp.status_code == 400 + + +class TestToolPermissions: + def _tool(self, pg_conn, cid): + from sqlalchemy import text + + return str(pg_conn.execute(text( + "INSERT INTO user_tools (user_id, name, config, connection_id, actions) VALUES ('alice', 'telegram', " + "'{}'::jsonb, CAST(:c AS uuid), '[{\"name\": \"telegram_send_message\", \"active\": true}]'::jsonb) " + "RETURNING id" + ), {"c": cid}).scalar()) + + def test_sets_permissions_on_the_connection_tool(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + tool_id = self._tool(pg_conn, cid) + with _db(pg_conn): + resp = _call(app, ConnectionToolPermissions, "put", "/p", + body={"permissions": {"telegram_send_message": "off"}}, args=(cid, tool_id)) + assert resp.status_code == 200 + action = resp.get_json()["tool"]["actions"][0] + assert action["permission"] == "off" + + def test_unknown_permission_is_rejected(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + resp = _call(app, ConnectionToolPermissions, "put", "/p", + body={"permissions": {"telegram_send_message": "sometimes"}}, args=(cid, "t1")) + assert resp.status_code == 400 + + def test_tool_of_another_connection_is_not_found(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + other = _connection(pg_conn, account_label="…zzzz", secrets={"credentials": {"token": "u"}}) + tool_id = self._tool(pg_conn, other) + with _db(pg_conn): + resp = _call(app, ConnectionToolPermissions, "put", "/p", + body={"permissions": {"telegram_send_message": "off"}}, args=(cid, tool_id)) + assert resp.status_code == 404 + + def test_tool_of_another_connection_is_left_unchanged(self, app, pg_conn): # noqa: F811 + from sqlalchemy import text + + from docsgpt.api.connector.connections import ConnectionToolPermissions + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + other = _connection(pg_conn, account_label="…zzzz", secrets={"credentials": {"token": "u"}}) + tool_id = self._tool(pg_conn, other) + with _db(pg_conn): + _call(app, ConnectionToolPermissions, "put", "/p", + body={"permissions": {"telegram_send_message": "off"}}, args=(cid, tool_id)) + actions = pg_conn.execute(text("SELECT actions FROM user_tools WHERE id = CAST(:t AS uuid)"), + {"t": tool_id}).scalar() + assert actions[0]["active"] is True + + +class TestRefreshTools: + def test_returns_the_diff(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionRefreshTools + + cid = _connection(pg_conn, provider="mcp:https://m.example.com", auth_kind="mcp_oauth", secrets={}) + with _db(pg_conn), patch("docsgpt.connectors.mcp.refresh_mcp_tools", + return_value={"added": ["a"], "removed": [], "tools": []}): + resp = _call(app, ConnectionRefreshTools, "post", "/r", args=(cid,)) + assert resp.status_code == 200 + assert resp.get_json()["added"] == ["a"] + + def test_signed_out_server_asks_to_reconnect(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionRefreshTools + + cid = _connection(pg_conn, provider="mcp:https://m.example.com", auth_kind="mcp_oauth", secrets={}) + with _db(pg_conn), patch("docsgpt.connectors.mcp.refresh_mcp_tools", + side_effect=service.ConnectionUnavailable("gone")): + resp = _call(app, ConnectionRefreshTools, "post", "/r", args=(cid,)) + assert resp.status_code == 409 + assert resp.get_json()["code"] == "reconnect" + + def test_server_failure_is_a_bad_gateway(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionRefreshTools + + cid = _connection(pg_conn, provider="mcp:https://m.example.com", auth_kind="mcp_oauth", secrets={}) + with _db(pg_conn), patch("docsgpt.connectors.mcp.refresh_mcp_tools", side_effect=RuntimeError("boom")): + resp = _call(app, ConnectionRefreshTools, "post", "/r", args=(cid,)) + assert resp.status_code == 502 + + +class TestPickerToken: + def test_signed_out_connection_asks_to_reconnect(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionPickerToken + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={}) + with _db(pg_conn), patch.object(service, "picker_token", side_effect=service.ConnectionUnavailable("x")): + resp = _call(app, ConnectionPickerToken, "post", "/p", args=(cid,)) + assert resp.status_code == 409 + + def test_provider_outage_is_retryable(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionPickerToken + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", secrets={}) + with _db(pg_conn), patch.object(service, "picker_token", + side_effect=service.TransientConnectionError("x")): + resp = _call(app, ConnectionPickerToken, "post", "/p", args=(cid,)) + assert resp.status_code == 503 + + def test_api_key_connection_has_no_picker(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionPickerToken + + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _db(pg_conn): + resp = _call(app, ConnectionPickerToken, "post", "/p", args=(cid,)) + assert resp.status_code == 404 + + +class TestClaim: + def test_needs_provider_and_token(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionClaim + + with _db(pg_conn): + resp = _call(app, ConnectionClaim, "post", "/c", body={"provider": "google_drive"}) + assert resp.status_code == 400 + + def test_unknown_token_is_not_found(self, app, pg_conn): # noqa: F811 + from docsgpt.api.connector.connections import ConnectionClaim + + with _db(pg_conn): + resp = _call(app, ConnectionClaim, "post", "/c", + body={"provider": "google_drive", "session_token": "nope"}) + assert resp.status_code == 404 diff --git a/tests/connectors/test_github.py b/tests/connectors/test_github.py new file mode 100644 index 00000000..f8f4ad80 --- /dev/null +++ b/tests/connectors/test_github.py @@ -0,0 +1,343 @@ +"""The built-in GitHub connector: two sign-ins, repository sync and read-only MCP tools.""" + +from __future__ import annotations + +from contextlib import contextmanager +from unittest.mock import MagicMock, patch + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.connectors import catalog, service +from docsgpt.security.encryption import encrypt_json + +READONLY_MCP = "https://api.githubcopilot.com/mcp/readonly" + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@pytest.fixture +def app_settings(monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_CLIENT_ID", "Iv1.client") + monkeypatch.setattr(settings, "GITHUB_CLIENT_SECRET", "app-secret") + monkeypatch.setattr(settings, "GITHUB_APP_SLUG", "docsgpt-acme") + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.api.connector.connections", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.mcp", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.resolve", db_readonly=_yield), \ + patch.multiple("docsgpt.api.connector.routes", db_session=_yield, db_readonly=_yield): + yield + + +def _call(app, resource, method, path, user="alice", body=None, args=(), query=None): + with app.test_request_context(path, method=method.upper(), json=body, query_string=query): + from flask import request + + request.decoded_token = {"sub": user} if user else None + return getattr(resource(), method)(*args) + + +def _connection(conn, user="alice", auth_kind="api_key", secrets=None, status="connected", label="octocat") -> str: + secrets = secrets if secrets is not None else {"credentials": {"access_token": "github_pat_alice"}} + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "encrypted_credentials) VALUES (:u, 'github', 'github', :a, :s, :l, :e) RETURNING id" + ), + {"u": user, "a": auth_kind, "s": status, "l": label, "e": encrypt_json(secrets, user)}, + ).scalar()) + + +def _response(payload, status=200, headers=None): + response = MagicMock(status_code=status, headers=headers or {}) + response.json.return_value = payload + response.ok = status < 400 + return response + + +class TestCatalog: + def test_github_syncs_and_reads_with_a_token_and_no_admin_setup(self, monkeypatch): + from docsgpt.core.settings import settings + + for name in ("GITHUB_CLIENT_ID", "GITHUB_CLIENT_SECRET", "GITHUB_APP_SLUG"): + monkeypatch.setattr(settings, name, None) + definition = catalog.get_definition("github") + assert definition.configured + assert definition.capabilities == ("sync", "read") + assert definition.sync_ingestor == "github" + assert definition.mcp_url == READONLY_MCP + assert definition.setup == {"tools": "ask", "sync": "ask"} + payload = definition.to_dict() + assert payload["sign_in_methods"] == ["api_key"] + assert [f["key"] for f in payload["credential_fields"]] == ["access_token"] + + def test_github_app_sign_in_is_offered_once_configured(self, app_settings): + assert catalog.get_definition("github").to_dict()["sign_in_methods"] == ["oauth", "api_key"] + + def test_other_connectors_have_one_sign_in(self): + assert catalog.get_definition("google_drive").to_dict()["sign_in_methods"] == ["oauth"] + assert catalog.get_definition("s3").to_dict()["sign_in_methods"] == ["api_key"] + + def test_generic_mcp_tools_do_not_belong_to_github(self): + """Every MCP tool would otherwise be listed as GitHub's.""" + assert catalog.definition_for_tool("mcp_tool") is None + assert "mcp_tool" not in catalog.tool_connector_keys() + + +class TestTokenSignIn: + def test_token_is_checked_and_named_after_the_account(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn), patch("docsgpt.connectors.github.requests.get", + return_value=_response({"login": "octocat"})) as get: + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "github", "credentials": {"access_token": "github_pat_abc"}}) + assert resp.status_code == 201 + assert resp.get_json()["connection"]["account_label"] == "octocat" + assert get.call_args.kwargs["headers"]["Authorization"] == "Bearer github_pat_abc" + assert resp.get_json()["setup"] == {"tools": "ask", "sync": "ask"} + + def test_rejected_token_is_not_stored(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionsList + + with _db(pg_conn), patch("docsgpt.connectors.github.requests.get", + return_value=_response({"message": "Bad credentials"}, 401)): + resp = _call(app, ConnectionsList, "post", "/api/connections", + body={"connector_key": "github", "credentials": {"access_token": "nope"}}) + assert resp.status_code == 400 + assert resp.get_json()["code"] == "invalid_credentials" + assert pg_conn.execute(text("SELECT count(*) FROM connector_sessions")).scalar() == 0 + + +class TestAppSignIn: + def test_callback_stores_the_app_token_under_the_login(self, app, pg_conn, app_settings): + import base64 + import json + + from docsgpt.api.connector.routes import ConnectorsCallback, build_authorization + + with _db(pg_conn), patch("docsgpt.api.connector.routes.service.ensure_can_store_credentials"): + started = build_authorization("github", "alice") + assert started["authorization_url"].startswith("https://github.com/login/oauth/authorize?") + state = started["state"] + token_info = { + "access_token": "ghu_a", "refresh_token": "ghr_r", "expiry": "2099-01-01T00:00:00+00:00", + "user_info": {"login": "octocat", "name": "Octo"}, + } + with _db(pg_conn), patch("docsgpt.parser.connectors.github.auth.GitHubAuth.exchange_code_for_tokens", + return_value=token_info): + with app.test_request_context(f"/api/connectors/callback?code=c&state={state}"): + page = ConnectorsCallback().get() + assert page.status_code == 200 + assert b"github_auth_success" in page.data + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE provider = 'github'")).one()._mapping + assert row["account_label"] == "octocat" and row["auth_kind"] == "oauth" + assert service.read_secrets(dict(row))["token_info"]["refresh_token"] == "ghr_r" + assert json.loads(base64.urlsafe_b64decode(state))["provider"] == "github" + + def test_installation_link_carries_the_same_state(self, pg_conn, app_settings): + from docsgpt.api.connector.routes import build_authorization + + cid = _connection(pg_conn, auth_kind="oauth", secrets={"token_info": {"access_token": "ghu"}}) + with _db(pg_conn), patch("docsgpt.api.connector.routes.service.ensure_can_store_credentials"): + started = build_authorization("github", "alice", cid, install=True) + assert started["authorization_url"].startswith( + "https://github.com/apps/docsgpt-acme/installations/new?state=" + ) + + def test_installation_redirect_without_state_is_a_friendly_page(self, app, app_settings): + """Installing the app from GitHub itself lands on the callback with no state.""" + from docsgpt.api.connector.routes import ConnectorsCallback + + with app.test_request_context("/api/connectors/callback?code=c&installation_id=9&setup_action=install"): + page = ConnectorsCallback().get() + assert page.status_code == 200 + assert b"installed" in page.data.lower() + + def test_expired_app_token_is_refreshed_before_use(self, pg_conn, app_settings): + cid = _connection(pg_conn, auth_kind="oauth", secrets={"token_info": { + "access_token": "ghu_old", "refresh_token": "ghr_r", "expiry": "2000-01-01T00:00:00+00:00", + }}) + row = pg_conn.execute(text("SELECT * FROM connector_sessions WHERE id = CAST(:c AS uuid)"), + {"c": cid}).one()._mapping + with _db(pg_conn), patch( + "docsgpt.parser.connectors.github.auth.GitHubAuth.refresh_access_token", + return_value={"access_token": "ghu_new", "refresh_token": "ghr_next", "expiry": "2099-01-01T00:00:00+00:00"}, + ): + assert service.access_credentials(dict(row)) == {"access_token": "ghu_new"} + + +class TestRepositories: + def test_token_lists_the_repositories_it_can_read(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionRepositories + + cid = _connection(pg_conn) + page = [ + {"full_name": "octocat/private", "name": "private", "owner": {"login": "octocat"}, "private": True, + "description": "Secret", "default_branch": "main", "pushed_at": "2026-09-01T00:00:00Z", + "html_url": "https://github.com/octocat/private"}, + ] + with _db(pg_conn), patch("docsgpt.connectors.github.requests.get", return_value=_response(page)) as get: + resp = _call(app, ConnectionRepositories, "get", f"/api/connections/{cid}/repositories", args=[cid]) + assert resp.status_code == 200 + data = resp.get_json() + assert data["repositories"] == [{ + "full_name": "octocat/private", "private": True, "description": "Secret", + "default_branch": "main", "updated_at": "2026-09-01T00:00:00Z", + "html_url": "https://github.com/octocat/private", + }] + assert data["install_url"] is None + assert get.call_args.args[0] == "https://api.github.com/user/repos" + assert "github_pat" not in resp.get_data(as_text=True) + + def test_app_sign_in_lists_the_installations_repositories(self, app, pg_conn, app_settings): + from docsgpt.api.connector.connections import ConnectionRepositories + + cid = _connection(pg_conn, auth_kind="oauth", secrets={"token_info": { + "access_token": "ghu_a", "expiry": "2099-01-01T00:00:00+00:00"}}) + + def fake_get(url, **kwargs): + if url.endswith("/user/installations"): + return _response({"installations": [{"id": 7}]}) + assert url.endswith("/user/installations/7/repositories") + return _response({"repositories": [{"full_name": "acme/api", "private": True}]}) + + with _db(pg_conn), patch("docsgpt.connectors.github.requests.get", side_effect=fake_get): + resp = _call(app, ConnectionRepositories, "get", f"/api/connections/{cid}/repositories", args=[cid]) + data = resp.get_json() + assert [r["full_name"] for r in data["repositories"]] == ["acme/api"] + assert data["install_url"] == "https://github.com/apps/docsgpt-acme/installations/new" + + def test_rejected_token_asks_to_reconnect(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionRepositories + + cid = _connection(pg_conn) + with _db(pg_conn), patch("docsgpt.connectors.github.requests.get", + return_value=_response({"message": "Bad credentials"}, 401)), \ + patch.object(service, "mark_reconnect_needed") as flag: + resp = _call(app, ConnectionRepositories, "get", f"/api/connections/{cid}/repositories", args=[cid]) + assert resp.status_code == 409 + flag.assert_called_once() + + def test_only_the_owner_and_only_github(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionRepositories + + cid = _connection(pg_conn, user="bob") + with _db(pg_conn): + resp = _call(app, ConnectionRepositories, "get", f"/api/connections/{cid}/repositories", args=[cid]) + assert resp.status_code == 404 + + +class TestSetup: + def test_sync_queues_the_repository_with_the_connection(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_remote.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "create_tools": False, + "sync": {"items": {"repo_url": "https://github.com/octocat/private.git"}, "frequency": "daily"}, + }, args=[cid]) + assert resp.status_code == 200 + kwargs = apply.call_args.kwargs["kwargs"] + assert kwargs["loader"] == "github" + assert kwargs["source_data"] == {"repo_url": "octocat/private"} + assert kwargs["connection_id"] == cid + assert kwargs["sync_frequency"] == "daily" + # Named after the repository, not the connector. + assert resp.get_json()["sources"][0]["name"] == "octocat/private" + + def test_not_a_repository_is_a_bad_request(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn) + with _db(pg_conn): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "create_tools": False, "sync": {"items": {"repo_url": "https://example.com/x"}}, + }, args=[cid]) + assert resp.status_code == 400 + + def test_tools_are_the_read_only_mcp_server_with_discovered_actions(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn) + actions = [{"name": "get_file_contents", "description": "Read a file", "annotations": {"readOnlyHint": True}, + "parameters": {"type": "object", "properties": {"path": {"type": "string"}}}}] + with _db(pg_conn), patch("docsgpt.agents.tools.mcp_tool.MCPTool.discover_tools"), \ + patch("docsgpt.agents.tools.mcp_tool.MCPTool.get_actions_metadata", return_value=actions), \ + patch("docsgpt.agents.tools.mcp_tool.MCPTool.__init__", return_value=None) as init: + for _ in range(2): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True}, args=[cid]) + assert resp.status_code == 200 + # The first MCPTool is the discovery one (the tool registry builds more). + config = init.call_args_list[0].args[0] + assert config["server_url"] == READONLY_MCP + assert config["auth_credentials"] == {"access_token": "github_pat_alice"} + rows = pg_conn.execute(text( + "SELECT name, config, actions FROM user_tools WHERE connection_id = CAST(:c AS uuid)" + ), {"c": cid}).all() + assert len(rows) == 1 + name, stored, stored_actions = rows[0] + assert name == "mcp_tool" + assert stored == {"server_url": READONLY_MCP, "auth_type": "bearer", "transport_type": "http", "timeout": 30} + assert "github_pat" not in str(stored) + assert stored_actions[0]["name"] == "get_file_contents" + assert stored_actions[0]["access"] == "read" and not stored_actions[0].get("require_approval") + + def test_unreachable_mcp_server_creates_nothing(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn) + with _db(pg_conn), patch("docsgpt.connectors.mcp._discover", side_effect=RuntimeError("down")): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True}, args=[cid]) + assert resp.status_code == 502 + assert resp.get_json()["code"] == "tools_unavailable" + assert pg_conn.execute(text("SELECT count(*) FROM user_tools")).scalar() == 0 + + +class TestToolRuntime: + def _tool(self, cid, server_url=READONLY_MCP): + return { + "id": "tool-gh", "user_id": "alice", "name": "mcp_tool", "connection_id": cid, + "config": {"server_url": server_url, "auth_type": "bearer"}, + "actions": [{"name": "get_me", "active": True}], "credential_mode": "owner", + } + + def test_app_token_reaches_the_tool_as_a_bearer_token(self, pg_conn, app_settings): + from docsgpt.agents.tool_executor import ToolExecutor + + cid = _connection(pg_conn, auth_kind="oauth", secrets={"token_info": { + "access_token": "ghu_live", "expiry": "2099-01-01T00:00:00+00:00"}}) + with _db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + ToolExecutor(user="alice")._get_or_load_tool(self._tool(cid), "t1", "get_me") + config = manager.return_value.load_tool.call_args.kwargs["tool_config"] + assert config["auth_credentials"] == {"access_token": "ghu_live"} + assert config["server_url"] == READONLY_MCP + + def test_token_never_goes_to_another_server(self, pg_conn): + from docsgpt.agents.tool_executor import ToolExecutor + + cid = _connection(pg_conn) + with _db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + with pytest.raises(service.ConnectionUnavailable): + ToolExecutor(user="alice")._get_or_load_tool( + self._tool(cid, "https://evil.example.com/mcp"), "t1", "get_me", + ) + manager.return_value.load_tool.assert_not_called() diff --git a/tests/connectors/test_github_writes.py b/tests/connectors/test_github_writes.py new file mode 100644 index 00000000..f7edee52 --- /dev/null +++ b/tests/connectors/test_github_writes.py @@ -0,0 +1,363 @@ +"""GitHub write access: opted into per connection, forbidden by an admin, enforced at run time.""" + +from __future__ import annotations + +import json +from contextlib import contextmanager +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +import pytest +from flask import Flask +from sqlalchemy import text + +import docsgpt.api.user # noqa: F401 (loads mcp_tool without the circular import) +import docsgpt.agents.tools.mcp_tool # noqa: F401,E402 (patched below) +from docsgpt.connectors import catalog, mcp, service +from docsgpt.security.encryption import encrypt_json + +READONLY_MCP = "https://api.githubcopilot.com/mcp/readonly" +WRITE_MCP = "https://api.githubcopilot.com/mcp/" + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.api.connector.connections", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.api.admin.connectors", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.mcp", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.resolve", db_readonly=_yield): + yield + + +def _call(app, resource, method, path, user="alice", body=None, args=(), roles=None): + with app.test_request_context(path, method=method.upper(), json=body): + from flask import request + + request.decoded_token = {"sub": user, "roles": roles or ["user"]} if user else None + return getattr(resource(), method)(*args) + + +def _connection(conn, user="alice") -> str: + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "encrypted_credentials) VALUES (:u, 'github', 'github', 'api_key', 'connected', 'octocat', :e) " + "RETURNING id" + ), + {"u": user, "e": encrypt_json({"credentials": {"access_token": "github_pat_alice"}}, user)}, + ).scalar()) + + +def _tool(conn, cid, server_url, actions) -> str: + return str(conn.execute( + text( + "INSERT INTO user_tools (user_id, name, config, actions, connection_id) VALUES ('alice', 'mcp_tool', " + "CAST(:c AS jsonb), CAST(:a AS jsonb), CAST(:cid AS uuid)) RETURNING id" + ), + {"c": json.dumps({"server_url": server_url, "auth_type": "bearer", "transport_type": "http", "timeout": 30}), + "a": json.dumps(actions), "cid": cid}, + ).scalar()) + + +def _row(conn, cid) -> dict: + return dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = CAST(:c AS uuid)"), {"c": cid}, + ).one()._mapping) + + +def _stored(conn, cid) -> tuple[dict, dict]: + config, actions = conn.execute( + text("SELECT config, actions FROM user_tools WHERE connection_id = CAST(:c AS uuid)"), {"c": cid}, + ).one() + return config, {a["name"]: a for a in actions} + + +def _forbid(conn): + from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository + from docsgpt.storage.db.repositories.connector_policies import allow_writes_key + + AppMetadataRepository(conn).set(allow_writes_key("github"), "false") + + +def _action(name, read_only=None): + action = {"name": name, "description": name, + "parameters": {"type": "object", "properties": {"owner": {"type": "string"}}}} + if read_only is not None: + action["annotations"] = {"readOnlyHint": read_only} + return action + + +# What GitHub's servers list: the read-only endpoint only reads; the full one +# adds writes, annotated ``readOnlyHint: false`` (or not annotated at all). +READ_ACTIONS = [_action("get_issue", True), _action("search_code", True)] +WRITE_ACTIONS = READ_ACTIONS + [ + _action("create_issue", False), + _action("add_issue_comment", False), + _action("mark_all_notifications_read"), +] + + +def _fake_discovery(calls): + """Stands in for GitHub's MCP server: lists the actions of the endpoint asked for.""" + + def discover(user_id, connection, tool): + url = tool["config"]["server_url"] + calls.append(url) + return [dict(a) for a in (WRITE_ACTIONS if url == WRITE_MCP else READ_ACTIONS)] + + return discover + + +class TestCatalog: + def test_github_offers_writes_on_its_full_endpoint(self): + definition = catalog.get_definition("github") + assert definition.mcp_url == READONLY_MCP + assert definition.mcp_write_url == WRITE_MCP + assert definition.to_dict()["writes_opt_in"] is True + assert catalog.get_definition("telegram").to_dict()["writes_opt_in"] is False + + def test_members_see_whether_writes_are_allowed(self, pg_conn): + entries = {e["key"]: e for e in service.catalog_for_user(pg_conn, "alice", is_admin=False)} + assert entries["github"]["writes_allowed"] is True + assert entries["telegram"]["writes_allowed"] is False + _forbid(pg_conn) + entries = {e["key"]: e for e in service.catalog_for_user(pg_conn, "alice", is_admin=False)} + assert entries["github"]["writes_allowed"] is False + + def test_builtin_config_picks_the_endpoint(self): + definition = catalog.get_definition("github") + assert service.builtin_mcp_config(definition)["server_url"] == READONLY_MCP + assert service.builtin_mcp_config(definition, writes=True)["server_url"] == WRITE_MCP + + +class TestClassification: + def test_full_endpoint_actions_are_writes_unless_marked_read_only(self): + """A GitHub action is a read only when GitHub says so: a name like + ``mark_all_notifications_read`` must not pass for one.""" + connection = {"id": "c1", "auth_kind": "api_key", "status": "connected", "connector_key": "github"} + fake = MagicMock() + fake.get_actions_metadata.return_value = [dict(a) for a in WRITE_ACTIONS] + with patch.object(service, "access_credentials", return_value={"access_token": "t"}), \ + patch("docsgpt.agents.tools.mcp_tool.MCPTool", return_value=fake): + actions = mcp.discover_builtin_actions("alice", connection, writes=True) + access = {a["name"]: a["access"] for a in actions} + assert access == { + "get_issue": "read", "search_code": "read", + "create_issue": "write", "add_issue_comment": "write", "mark_all_notifications_read": "write", + } + + +class TestSetup: + def test_opting_in_uses_the_full_endpoint_and_writes_ask_first(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail, ConnectionSetup + + cid = _connection(pg_conn) + calls = [] + with _db(pg_conn), patch.object(mcp, "_discover", side_effect=_fake_discovery(calls)): + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True, "allow_writes": True}, args=[cid]) + detail = _call(app, ConnectionDetail, "get", f"/api/connections/{cid}", args=[cid]).get_json() + assert resp.status_code == 200 + assert calls == [WRITE_MCP] + config, actions = _stored(pg_conn, cid) + assert config["server_url"] == WRITE_MCP + assert actions["create_issue"]["access"] == "write" and actions["create_issue"]["require_approval"] is True + assert actions["mark_all_notifications_read"]["access"] == "write" + assert actions["get_issue"]["access"] == "read" and not actions["get_issue"].get("require_approval") + assert detail["connection"]["writes"] is True + permissions = {a["name"]: a["permission"] for a in detail["connection"]["tools"][0]["actions"]} + assert permissions["create_issue"] == "ask" and permissions["get_issue"] == "always" + + def test_read_only_is_the_default(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail, ConnectionSetup + + cid = _connection(pg_conn) + calls = [] + with _db(pg_conn), patch.object(mcp, "_discover", side_effect=_fake_discovery(calls)): + _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True}, args=[cid]) + detail = _call(app, ConnectionDetail, "get", f"/api/connections/{cid}", args=[cid]).get_json() + assert calls == [READONLY_MCP] + assert _stored(pg_conn, cid)[0]["server_url"] == READONLY_MCP + assert detail["connection"]["writes"] is False + + def test_forbidden_writes_are_refused(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _connection(pg_conn) + _forbid(pg_conn) + with _db(pg_conn), patch.object(mcp, "_discover") as discover: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", + body={"create_tools": True, "allow_writes": True}, args=[cid]) + assert resp.status_code == 403 + assert resp.get_json()["code"] == "writes_forbidden" + discover.assert_not_called() + assert pg_conn.execute(text("SELECT count(*) FROM user_tools")).scalar() == 0 + + +class TestSwitchWrites: + def _read_only_tool(self, conn, cid): + actions = mcp.apply_default_permissions("mcp_tool", service._transform_actions( + [dict(a) for a in READ_ACTIONS])) + # The user turned one read off; that choice survives switching. + actions = [{**a, "active": a["name"] != "search_code"} for a in actions] + return _tool(conn, cid, READONLY_MCP, actions) + + def test_turning_writes_on_and_off_keeps_choices(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionWrites + + cid = _connection(pg_conn) + self._read_only_tool(pg_conn, cid) + calls = [] + with _db(pg_conn), patch.object(mcp, "_discover", side_effect=_fake_discovery(calls)): + on = _call(app, ConnectionWrites, "put", f"/api/connections/{cid}/writes", + body={"allow": True}, args=[cid]) + config, actions = _stored(pg_conn, cid) + assert on.status_code == 200 + assert on.get_json()["writes"] is True + assert sorted(on.get_json()["added"]) == ["add_issue_comment", "create_issue", + "mark_all_notifications_read"] + assert config["server_url"] == WRITE_MCP + assert actions["search_code"]["active"] is False + assert actions["create_issue"]["require_approval"] is True + off = _call(app, ConnectionWrites, "put", f"/api/connections/{cid}/writes", + body={"allow": False}, args=[cid]) + assert calls == [WRITE_MCP, READONLY_MCP] + config, actions = _stored(pg_conn, cid) + assert off.get_json()["writes"] is False + assert config["server_url"] == READONLY_MCP + assert set(actions) == {"get_issue", "search_code"} + assert actions["search_code"]["active"] is False + + def test_forbidden_writes_cannot_be_turned_on_but_can_be_turned_off(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionWrites + + cid = _connection(pg_conn) + _tool(pg_conn, cid, WRITE_MCP, [_action("create_issue", False)]) + _forbid(pg_conn) + calls = [] + with _db(pg_conn), patch.object(mcp, "_discover", side_effect=_fake_discovery(calls)): + on = _call(app, ConnectionWrites, "put", "/x", body={"allow": True}, args=[cid]) + off = _call(app, ConnectionWrites, "put", "/x", body={"allow": False}, args=[cid]) + assert on.status_code == 403 and on.get_json()["code"] == "writes_forbidden" + assert off.status_code == 200 + assert _stored(pg_conn, cid)[0]["server_url"] == READONLY_MCP + + def test_owner_only_github_only_and_needs_the_tool(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionWrites + + cid = _connection(pg_conn) + with _db(pg_conn): + bob = _call(app, ConnectionWrites, "put", "/x", user="bob", body={"allow": True}, args=[cid]) + no_tool = _call(app, ConnectionWrites, "put", "/x", body={"allow": True}, args=[cid]) + bad = _call(app, ConnectionWrites, "put", "/x", body={"allow": "yes"}, args=[cid]) + assert bob.status_code == 404 + assert no_tool.status_code == 409 and no_tool.get_json()["code"] == "no_tools" + assert bad.status_code == 400 + telegram = str(pg_conn.execute(text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label) " + "VALUES ('alice', 'telegram', 'telegram', 'api_key', 'connected', 'bot') RETURNING id" + )).scalar()) + with _db(pg_conn): + other = _call(app, ConnectionWrites, "put", "/x", body={"allow": True}, args=[telegram]) + assert other.status_code == 400 + + def test_refresh_after_an_admin_forbids_writes_goes_read_only(self, pg_conn): + cid = _connection(pg_conn) + _tool(pg_conn, cid, WRITE_MCP, [_action("create_issue", False)]) + _forbid(pg_conn) + calls = [] + with _db(pg_conn), patch.object(mcp, "_discover", side_effect=_fake_discovery(calls)): + result = mcp.refresh_mcp_tools("alice", _row(pg_conn, cid)) + assert calls == [READONLY_MCP] + assert result["removed"] == ["create_issue"] + assert _stored(pg_conn, cid)[0]["server_url"] == READONLY_MCP + + +class TestAdmin: + def test_admin_can_forbid_github_writes(self, app, pg_conn): + from docsgpt.api.admin.connectors import AdminConnectorsResource + + with _db(pg_conn): + before = _call(app, AdminConnectorsResource, "get", "/x", roles=["admin"]).get_json() + resp = _call(app, AdminConnectorsResource, "put", "/x", roles=["admin"], + body={"policies": {"github": {"allow_writes": False}}}) + rows = {c["key"]: c for c in before["connectors"]} + assert rows["github"]["allow_writes"] is True + assert rows["telegram"]["allow_writes"] is None + assert resp.status_code == 200 + assert {c["key"]: c for c in resp.get_json()["connectors"]}["github"]["allow_writes"] is False + assert service.writes_allowed(service.load_policies(pg_conn), "github") is False + + def test_rejects_writes_policy_where_it_means_nothing(self, app, pg_conn): + from docsgpt.api.admin.connectors import AdminConnectorsResource + + with _db(pg_conn): + other = _call(app, AdminConnectorsResource, "put", "/x", roles=["admin"], + body={"policies": {"telegram": {"allow_writes": False}}}) + not_bool = _call(app, AdminConnectorsResource, "put", "/x", roles=["admin"], + body={"policies": {"github": {"allow_writes": "no"}}}) + assert other.status_code == 400 + assert not_bool.status_code == 400 + + +def _tool_data(cid, server_url=WRITE_MCP): + return { + "id": "tool-gh", "user_id": "alice", "name": "mcp_tool", "connection_id": cid, + "config": {"server_url": server_url, "auth_type": "bearer"}, "credential_mode": "owner", + "actions": [ + {"name": "create_issue", "access": "write", "active": True, "require_approval": False}, + {"name": "get_issue", "access": "read", "active": True}, + ], + } + + +class TestRuntime: + def _loaded_url(self, pg_conn, tool): + from docsgpt.agents.tool_executor import ToolExecutor + + with _db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + ToolExecutor(user="alice")._get_or_load_tool(tool, "t1", "create_issue") + return manager.return_value.load_tool.call_args.kwargs["tool_config"]["server_url"] + + def test_write_endpoint_is_used_while_allowed(self, pg_conn): + assert self._loaded_url(pg_conn, _tool_data(_connection(pg_conn))) == WRITE_MCP + + def test_forbidden_writes_fall_back_to_the_read_only_endpoint(self, pg_conn): + cid = _connection(pg_conn) + _forbid(pg_conn) + assert self._loaded_url(pg_conn, _tool_data(cid)) == READONLY_MCP + + def test_any_other_path_on_githubs_host_is_read_only(self, pg_conn): + cid = _connection(pg_conn) + assert self._loaded_url(pg_conn, _tool_data(cid, "https://api.githubcopilot.com/mcp/x/all")) == READONLY_MCP + + def _pause(self, pg_conn, action_name): + from docsgpt.agents.tool_executor import ToolExecutor + + cid = _connection(pg_conn) + _forbid(pg_conn) + with _db(pg_conn), patch("docsgpt.agents.tool_executor.ToolActionParser") as parser: + parser.return_value.parse_args.return_value = ("t1", action_name, {}) + return ToolExecutor(user="alice").check_pause( + {"t1": _tool_data(cid)}, SimpleNamespace(id="c1", name=action_name, thought_signature=None), + "OpenAILLM", + ) + + def test_forbidden_write_is_denied_with_a_reason(self, pg_conn): + pause = self._pause(pg_conn, "create_issue") + assert pause["pause_type"] == "headless_denied" + assert "admin" in pause["deny_reason"] + + def test_reads_still_run_when_writes_are_forbidden(self, pg_conn): + assert self._pause(pg_conn, "get_issue") is None diff --git a/tests/connectors/test_linear.py b/tests/connectors/test_linear.py new file mode 100644 index 00000000..7e94ed10 --- /dev/null +++ b/tests/connectors/test_linear.py @@ -0,0 +1,279 @@ +"""Linear as Knowledge: the same MCP sign-in that gives agents Linear's tools syncs its issues.""" + +from __future__ import annotations + +import asyncio +from contextlib import contextmanager +from unittest.mock import MagicMock, patch + +import pytest +from flask import Flask +from sqlalchemy import text + +import docsgpt.api.user # noqa: F401 (loads mcp_tool without the circular import) +from docsgpt.connectors import catalog, linear, service +from docsgpt.security.encryption import encrypt_json + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.api.connector.connections", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield): + yield + + +def _call(app, resource, method, path, user="alice", body=None, args=()): + with app.test_request_context(path, method=method.upper(), json=body): + from flask import request + + request.decoded_token = {"sub": user} if user else None + return getattr(resource(), method)(*args) + + +def _linear(conn, user="alice", status="connected") -> str: + tokens = {"tokens": {"access_token": "lin-mcp", "token_type": "Bearer", "refresh_token": "r"}} + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "server_url, encrypted_credentials) VALUES (:u, 'mcp:https://mcp.linear.app', 'mcp:linear', " + "'mcp_oauth', :s, 'alice@acme.com', 'https://mcp.linear.app', :e) RETURNING id" + ), + {"u": user, "s": status, "e": encrypt_json(tokens, user)}, + ).scalar()) + + +class FakeLinear: + def __init__(self, answers): + self.answers = answers + + async def input_schema(self, name): + return {"properties": {"limit": {}, "cursor": {}}} if name in self.answers else None + + async def call(self, name, arguments): + return self.answers[name] + + +def _session(answers): + """``run_connection_session`` answering from ``answers``, recording the connection it was given.""" + seen = {} + + def run(connection, server_url, work, **kwargs): + seen.update(connection=connection, server_url=server_url) + return asyncio.run(work(FakeLinear(answers))) + + return run, seen + + +class TestCatalog: + @pytest.fixture(autouse=True) + def _fresh_registry(self): + catalog.reset_registry_for_tests() + yield + catalog.reset_registry_for_tests() + + def test_one_linear_sign_in_syncs_and_gives_tools(self): + definition = catalog.get_definition("mcp:linear") + assert definition.capabilities == ("sync", "read", "write") + assert definition.sync_ingestor == "linear" + assert definition.setup == {"tools": "auto", "sync": "ask"} + assert definition.tool_templates == ("mcp_tool",) + assert definition.auth_kind == "mcp_oauth" + + def test_other_presets_still_only_give_tools(self): + notion = catalog.get_definition("mcp:notion") + assert notion.sync_ingestor is None + assert notion.setup == {"tools": "auto", "sync": "off"} + assert "sync" not in notion.capabilities + + def test_linear_sources_have_a_loader(self): + from docsgpt.parser.remote.linear_loader import LinearLoader + from docsgpt.parser.remote.remote_creator import RemoteCreator, normalize_remote_data + + assert isinstance(RemoteCreator.create_loader("linear"), LinearLoader) + stored = {"teams": [{"id": "t1"}], "include_comments": True} + assert normalize_remote_data("linear", stored) == stored + + +class TestWorkspace: + def test_lists_the_teams_and_projects_to_pick(self, app, pg_conn): + from docsgpt.api.connector.connections import LinearWorkspace + + cid = _linear(pg_conn) + run, seen = _session({ + "list_teams": {"teams": [{"id": "t2", "key": "OPS", "name": "Operations"}, + {"id": "t1", "key": "ENG", "name": "Engineering"}]}, + "list_projects": [{"id": "p1", "name": "Acme", "state": {"name": "Started"}, "teams": ["Engineering"]}], + }) + with _db(pg_conn), patch("docsgpt.connectors.mcp.run_connection_session", side_effect=run): + resp = _call(app, LinearWorkspace, "get", f"/api/connections/{cid}/linear", args=[cid]) + assert resp.status_code == 200 + data = resp.get_json() + assert data["teams"] == [ + {"id": "t1", "key": "ENG", "name": "Engineering"}, {"id": "t2", "key": "OPS", "name": "Operations"}, + ] + assert data["projects"] == [{"id": "p1", "name": "Acme", "state": "Started", "teams": ["Engineering"]}] + assert str(seen["connection"]["id"]) == cid + assert seen["server_url"] == "https://mcp.linear.app/mcp" + assert "lin-mcp" not in resp.get_data(as_text=True) + + def test_a_lost_sign_in_asks_to_reconnect(self, app, pg_conn): + from docsgpt.api.connector.connections import LinearWorkspace + + cid = _linear(pg_conn) + with _db(pg_conn), patch("docsgpt.connectors.mcp.run_connection_session", + side_effect=service.ConnectionUnavailable("expired", connection_id=cid)): + resp = _call(app, LinearWorkspace, "get", f"/api/connections/{cid}/linear", args=[cid]) + assert resp.status_code == 409 + assert resp.get_json()["code"] == "reconnect" + + def test_linear_not_answering_is_worth_a_retry(self, app, pg_conn): + from docsgpt.api.connector.connections import LinearWorkspace + + cid = _linear(pg_conn) + with _db(pg_conn), patch("docsgpt.connectors.mcp.run_connection_session", + side_effect=service.TransientConnectionError("rate limit")): + resp = _call(app, LinearWorkspace, "get", f"/api/connections/{cid}/linear", args=[cid]) + assert resp.status_code == 503 + + def test_only_the_owner_and_only_linear(self, app, pg_conn): + from docsgpt.api.connector.connections import LinearWorkspace + + theirs = _linear(pg_conn, user="bob") + github = str(pg_conn.execute(text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status) " + "VALUES ('alice', 'github', 'github', 'api_key', 'connected') RETURNING id" + )).scalar()) + with _db(pg_conn): + for cid in (theirs, github): + resp = _call(app, LinearWorkspace, "get", f"/api/connections/{cid}/linear", args=[cid]) + assert resp.status_code == 404 + + +class TestSetup: + def test_sync_queues_the_picked_teams_with_the_connection(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _linear(pg_conn) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_remote.apply_async", + return_value=MagicMock(id="t")) as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "create_tools": False, + "sync": {"items": {"teams": [{"id": "t1", "key": "ENG", "name": "Engineering"}], + "include_comments": False}, "frequency": "daily"}, + }, args=[cid]) + assert resp.status_code == 200 + kwargs = apply.call_args.kwargs["kwargs"] + assert kwargs["loader"] == "linear" + assert kwargs["connection_id"] == cid + assert kwargs["sync_frequency"] == "daily" + assert kwargs["source_data"] == { + "teams": [{"id": "t1", "key": "ENG", "name": "Engineering"}], "projects": [], + "include_comments": False, "include_documents": False, + } + assert resp.get_json()["sources"][0]["name"] == "Linear · Engineering" + + def test_nothing_picked_is_a_bad_request(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionSetup + + cid = _linear(pg_conn) + with _db(pg_conn), patch("docsgpt.api.user.tasks.ingest_remote.apply_async") as apply: + resp = _call(app, ConnectionSetup, "post", f"/api/connections/{cid}/setup", body={ + "create_tools": False, "sync": {"items": {"teams": []}}, + }, args=[cid]) + assert resp.status_code == 400 + apply.assert_not_called() + + +class TestWorker: + @staticmethod + @contextmanager + def _readonly(conn): + from docsgpt import worker + + @contextmanager + def _yield(): + yield conn + + with patch.object(worker, "db_readonly", _yield): + yield worker + + def test_the_loader_gets_the_connection_not_its_tokens(self, pg_conn): + cid = _linear(pg_conn) + with self._readonly(pg_conn) as worker: + data = worker._with_connection_credentials({"teams": [{"id": "t1"}]}, cid) + assert data == {"teams": [{"id": "t1"}], "connection_id": cid} + + def test_a_connection_that_needs_reconnecting_gives_nothing(self, pg_conn): + cid = _linear(pg_conn, status="reconnect_needed") + with self._readonly(pg_conn) as worker: + assert worker._with_connection_credentials({"teams": [{"id": "t1"}]}, cid) is None + + def test_scheduled_sync_reads_with_the_connection(self, pg_conn): + cid = _linear(pg_conn) + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data) " + "VALUES ('alice', 'Linear · Engineering', 'linear', 'daily', CAST(:c AS uuid), " + "'{\"teams\": [{\"id\": \"t1\"}]}')" + ), {"c": cid}) + with self._readonly(pg_conn) as worker, patch.object(worker, "sync", return_value={"status": "success"}) as sync: + counts = worker.sync_worker(MagicMock(), "daily") + assert counts["sync_success"] == 1 + args, kwargs = sync.call_args + assert args[1] == {"teams": [{"id": "t1"}]} + assert args[4] == "linear" + assert kwargs["connection_id"] == cid + + def test_scheduled_sync_skips_a_source_paused_for_reconnect(self, pg_conn): + cid = _linear(pg_conn, status="reconnect_needed") + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data, metadata) " + "VALUES ('alice', 'Linear', 'linear', 'daily', CAST(:c AS uuid), '{\"teams\": [{\"id\": \"t1\"}]}', " + "'{\"sync_state\": \"paused_reconnect\"}')" + ), {"c": cid}) + with self._readonly(pg_conn) as worker, patch.object(worker, "sync") as sync: + counts = worker.sync_worker(MagicMock(), "daily") + sync.assert_not_called() + assert counts["sync_skipped"] == 1 + + def test_scheduled_sync_skips_a_source_whose_connection_was_removed(self, pg_conn): + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, remote_data) " + "VALUES ('alice', 'Linear', 'linear', 'daily', '{\"teams\": [{\"id\": \"t1\"}]}')" + )) + with self._readonly(pg_conn) as worker, patch.object(worker, "sync") as sync: + counts = worker.sync_worker(MagicMock(), "daily") + sync.assert_not_called() + assert counts["sync_skipped"] == 1 + + def test_a_lost_sign_in_fails_the_ingest_without_retrying_forever(self): + from docsgpt import worker + + loader = MagicMock() + loader.load_data.side_effect = service.ConnectionUnavailable("expired", connection_id="c-1") + task = MagicMock() + task.request.retries = 1 + with patch.object(worker.RemoteCreator, "create_loader", return_value=loader), patch.object( + worker, "_with_connection_credentials", return_value={"teams": [{"id": "t1"}], "connection_id": "c-1"}, + ), patch.object(worker, "publish_user_event") as publish: + with pytest.raises(service.ConnectionUnavailable): + worker.remote_worker(task, {"teams": [{"id": "t1"}]}, "Linear", "alice", "linear", + connection_id="c-1") + assert publish.call_args.args[1] == "source.ingest.failed" + + +def test_the_picker_lists_are_capped(monkeypatch): + monkeypatch.setattr(linear, "MAX_PICKER_ITEMS", 2) + session = FakeLinear({ + "list_teams": {"teams": [{"id": f"t{n}", "name": f"Team {n}"} for n in range(5)]}, + "list_projects": {"projects": []}, + }) + assert len(asyncio.run(linear.list_workspace(session))["teams"]) == 2 diff --git a/tests/connectors/test_mcp_refresh.py b/tests/connectors/test_mcp_refresh.py new file mode 100644 index 00000000..957df212 --- /dev/null +++ b/tests/connectors/test_mcp_refresh.py @@ -0,0 +1,229 @@ +"""Tests for re-reading an MCP connection's actions.""" + +from __future__ import annotations + +import json +from contextlib import contextmanager +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy import text + +import docsgpt.api.user # noqa: F401 (loads mcp_tool without the circular import) +import docsgpt.agents.tools.mcp_tool # noqa: F401,E402 (patched below) +from docsgpt.connectors import mcp, service +from docsgpt.security.encryption import encrypt_json + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.connectors.mcp", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield): + yield + + +def _connection(conn, *, auth_kind="mcp_oauth", status="connected", secrets=None) -> dict: + row = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "server_url, encrypted_credentials) VALUES ('alice', 'mcp:https://mcp.linear.app', 'mcp:linear', :a, " + ":s, 'alice@example.com', 'https://mcp.linear.app', :e) RETURNING *" + ), + {"a": auth_kind, "s": status, "e": encrypt_json(secrets or {}, "alice")}, + ).one() + return dict(row._mapping) + + +def _tool(conn, connection_id, actions) -> str: + return str(conn.execute( + text( + "INSERT INTO user_tools (user_id, name, config, actions, connection_id) VALUES ('alice', 'mcp_tool', " + "CAST(:c AS jsonb), CAST(:a AS jsonb), CAST(:cid AS uuid)) RETURNING id" + ), + {"c": json.dumps({"server_url": "https://mcp.linear.app/mcp", "auth_type": "oauth"}), + "a": json.dumps(actions), "cid": str(connection_id)}, + ).scalar()) + + +def _action(name, **extra): + return {"name": name, "description": name, "parameters": {"properties": {"q": {"type": "string"}}}, **extra} + + +class TestRefresh: + def test_keeps_choices_adds_new_actions_and_drops_removed_ones(self, pg_conn): + connection = _connection(pg_conn) + tool_id = _tool(pg_conn, connection["id"], [ + _action("list_issues", active=False, require_approval=True), + _action("archived_action"), + ]) + discovered = [ + _action("list_issues", annotations={"readOnlyHint": True}), + _action("create_issue", annotations={"readOnlyHint": False}), + ] + with _db(pg_conn), patch.object(mcp, "_discover", return_value=discovered): + result = mcp.refresh_mcp_tools("alice", connection) + assert result["added"] == ["create_issue"] + assert result["removed"] == ["archived_action"] + actions = {a["name"]: a for a in pg_conn.execute( + text("SELECT actions FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": tool_id} + ).scalar()} + assert set(actions) == {"list_issues", "create_issue"} + # The user's choices survive a refresh. + assert actions["list_issues"]["active"] is False + assert actions["list_issues"]["require_approval"] is True + # New writes default to needing approval. + assert actions["create_issue"]["access"] == "write" + assert actions["create_issue"]["require_approval"] is True + assert actions["create_issue"]["parameters"]["properties"]["q"]["filled_by_llm"] is True + assert [t["id"] for t in result["tools"]] == [tool_id] + + def test_fixed_values_survive_for_parameters_that_still_exist(self, pg_conn): + connection = _connection(pg_conn) + pinned = _action("search_issues") + pinned["parameters"]["properties"] = { + "q": {"type": "string", "filled_by_llm": True, "value": ""}, + "team": {"type": "string", "filled_by_llm": False, "value": "ENG"}, + "gone": {"type": "string", "filled_by_llm": False, "value": "x"}, + } + tool_id = _tool(pg_conn, connection["id"], [pinned]) + fresh = _action("search_issues") + fresh["parameters"]["properties"] = {"q": {"type": "string"}, "team": {"type": "string"}} + with _db(pg_conn), patch.object(mcp, "_discover", return_value=[fresh]): + mcp.refresh_mcp_tools("alice", connection) + action = pg_conn.execute( + text("SELECT actions FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": tool_id} + ).scalar()[0] + properties = action["parameters"]["properties"] + assert properties["team"] == {"type": "string", "filled_by_llm": False, "value": "ENG"} + assert properties["q"]["filled_by_llm"] is True + assert "gone" not in properties + + def test_other_tools_on_the_connection_are_left_alone(self, pg_conn): + connection = _connection(pg_conn) + pg_conn.execute( + text("INSERT INTO user_tools (user_id, name, config, connection_id) VALUES ('alice', 'telegram', " + "'{}'::jsonb, CAST(:c AS uuid))"), + {"c": str(connection["id"])}, + ) + with _db(pg_conn), patch.object(mcp, "_discover") as discover: + result = mcp.refresh_mcp_tools("alice", connection) + discover.assert_not_called() + assert result == {"added": [], "removed": [], "tools": []} + + +class TestDiscover: + def test_oauth_connection_passes_its_id_not_tokens(self): + connection = {"id": "c1", "auth_kind": "mcp_oauth", "status": "connected"} + tool = {"config": {"server_url": "https://mcp.linear.app/mcp", "encrypted_credentials": "v1-blob"}} + fake = MagicMock() + fake.get_actions_metadata.return_value = [_action("search")] + with patch("docsgpt.agents.tools.mcp_tool.MCPTool", return_value=fake) as tool_cls: + assert mcp._discover("alice", connection, tool) == [_action("search")] + config = tool_cls.call_args.args[0] + assert config["connection_id"] == "c1" + assert config["query_mode"] is True + assert "encrypted_credentials" not in config + fake.discover_tools.assert_called_once() + + def test_api_key_connection_supplies_its_credentials(self): + connection = {"id": "c1", "auth_kind": "api_key", "status": "connected"} + with patch.object(service, "get_credentials", return_value={"bearer_token": "tok"}), \ + patch("docsgpt.agents.tools.mcp_tool.MCPTool") as tool_cls: + mcp._discover("alice", connection, {"config": {"server_url": "https://m.example.com/mcp"}}) + assert tool_cls.call_args.args[0]["auth_credentials"] == {"bearer_token": "tok"} + + def test_signed_out_oauth_connection_must_reconnect(self): + connection = {"id": "c1", "auth_kind": "mcp_oauth", "status": "reconnect_needed"} + with pytest.raises(service.ConnectionUnavailable), patch("docsgpt.agents.tools.mcp_tool.MCPTool") as tool_cls: + mcp._discover("alice", connection, {"config": {}}) + tool_cls.assert_not_called() + + +def test_oauth_resource_is_the_full_mcp_endpoint(): + """Linear and Sentry publish ``https://host/mcp`` as their protected + resource; the SDK must be given that endpoint, not only the origin.""" + from mcp.shared.auth_utils import check_resource_allowed, resource_url_from_server_url + + from docsgpt.agents.tools.mcp_tool import DocsGPTOAuth + + oauth = DocsGPTOAuth( + mcp_url="https://mcp.linear.app/mcp", + redis_client=MagicMock(), + redirect_uri="https://example.com/callback", + user_id="user1", + ) + assert oauth.context.server_url == "https://mcp.linear.app/mcp" + requested = resource_url_from_server_url(oauth.context.server_url) + # Servers that publish the endpoint, and those that publish the origin. + assert check_resource_allowed(requested_resource=requested, configured_resource="https://mcp.linear.app/mcp") + assert check_resource_allowed(requested_resource=requested, configured_resource="https://mcp.linear.app") + # Stored tokens stay keyed by the server's origin. + assert oauth.context.storage.server_url == "https://mcp.linear.app" + + +def test_callback_hands_the_sdk_an_authorization_result(): + """The MCP SDK reads ``.code``, ``.state`` and the RFC 9207 ``.iss`` off the + callback's result; Linear advertises ``iss`` and refuses a sign-in without it.""" + import asyncio + + from mcp.shared.auth import AuthorizationCodeResult + + from docsgpt.agents.tools.mcp_tool import DocsGPTOAuth + + stored = { + "mcp_oauth:code:st": b"the-code", + "mcp_oauth:iss:st": b"https://mcp.linear.app", + } + redis_client = MagicMock() + redis_client.get.side_effect = stored.get + oauth = DocsGPTOAuth( + mcp_url="https://mcp.linear.app/mcp", + redis_client=redis_client, + redirect_uri="https://example.com/callback", + user_id="user1", + ) + oauth.extracted_state = "st" + + result = asyncio.run(oauth.callback_handler()) + + assert isinstance(result, AuthorizationCodeResult) + assert (result.code, result.state, result.iss) == ("the-code", "st", "https://mcp.linear.app") + redis_client.delete.assert_any_call("mcp_oauth:iss:st") + + +@pytest.mark.parametrize( + "method, basic, body_client_id", + [("client_secret_basic", True, False), ("client_secret_post", False, True), ("none", False, True)], +) +def test_token_request_uses_one_client_authentication(method, basic, body_client_id): + """Linear registers clients for ``client_secret_basic`` and refuses a token + request that also names the client in the body; RFC 6749 wants one method.""" + from mcp.shared.auth import OAuthClientInformationFull + + from docsgpt.agents.tools.mcp_tool import DocsGPTOAuth + + oauth = DocsGPTOAuth( + mcp_url="https://mcp.linear.app/mcp", + redis_client=MagicMock(), + redirect_uri="https://example.com/callback", + user_id="user1", + ) + oauth.context.client_info = OAuthClientInformationFull( + client_id="cid", + client_secret=None if method == "none" else "secret", + token_endpoint_auth_method=method, + redirect_uris=["https://example.com/callback"], + ) + + data, headers = oauth.context.prepare_token_auth( + {"grant_type": "authorization_code", "code": "c", "client_id": "cid"}, + {"Content-Type": "application/x-www-form-urlencoded"}, + ) + + assert headers.get("Authorization", "").startswith("Basic ") is basic + assert ("client_id" in data) is body_client_id + assert data["code"] == "c" diff --git a/tests/connectors/test_mcp_session.py b/tests/connectors/test_mcp_session.py new file mode 100644 index 00000000..25387876 --- /dev/null +++ b/tests/connectors/test_mcp_session.py @@ -0,0 +1,211 @@ +"""Calling an MCP server's tools from the server side, signed in with a connection.""" + +from __future__ import annotations + +import json +from types import SimpleNamespace +from unittest.mock import patch + +import pytest + +import docsgpt.api.user # noqa: F401 (loads mcp_tool without the circular import) +from docsgpt.agents.tools.mcp_tool import MCPReauthorizationRequired +from docsgpt.connectors import mcp, service + +LINEAR = "https://mcp.linear.app/mcp" + + +def _connection(status="connected", server_url="https://mcp.linear.app") -> dict: + return { + "id": "c1", "user_id": "alice", "provider": "mcp:https://mcp.linear.app", "connector_key": "mcp:linear", + "auth_kind": "mcp_oauth", "status": status, "server_url": server_url, "encrypted_credentials": "v2:x", + } + + +def _text(payload) -> SimpleNamespace: + return SimpleNamespace(type="text", text=payload if isinstance(payload, str) else json.dumps(payload)) + + +def _result(*content, structured=None, error=False) -> SimpleNamespace: + return SimpleNamespace(content=list(content), structured_content=structured, is_error=error) + + +class FakeClient: + """An MCP client that answers from a table of tool results.""" + + def __init__(self, results=None, tools=(), enter_error=None): + self.results = results or {} + self.tools = [SimpleNamespace(name=name, inputSchema=schema) for name, schema in tools] + self.enter_error = enter_error + self.calls: list[tuple[str, dict]] = [] + self.listed = 0 + self.open = False + + async def __aenter__(self): + if self.enter_error: + raise self.enter_error + self.open = True + return self + + async def __aexit__(self, *exc): + self.open = False + + async def list_tools(self): + self.listed += 1 + return self.tools + + async def call_tool(self, name, arguments=None, raise_on_error=True): + assert self.open + self.calls.append((name, dict(arguments or {}))) + result = self.results[name] + if isinstance(result, BaseException): + raise result + return result + + +def _run(client, work, connection=None): + with patch.object(mcp, "_client_for", return_value=client) as make: + value = mcp.run_connection_session(connection or _connection(), LINEAR, work) + return value, make + + +class TestRunConnectionSession: + def test_calls_share_one_session_signed_in_with_the_connection(self): + client = FakeClient({"list_teams": _result(_text({"teams": [{"id": "t1"}]}))}) + + async def work(session): + first = await session.call("list_teams", {"limit": 5}) + second = await session.call("list_teams", {}) + return first, second + + (first, second), make = _run(client, work) + assert first == {"teams": [{"id": "t1"}]} == second + assert client.calls == [("list_teams", {"limit": 5}), ("list_teams", {})] + assert make.call_args.args[0]["id"] == "c1" + assert make.call_args.args[1] == LINEAR + + def test_structured_content_is_preferred_to_text(self): + client = FakeClient({"get_issue": _result(_text("prose"), structured={"id": "ENG-1"})}) + + async def work(session): + return await session.call("get_issue", {"id": "ENG-1"}) + + assert _run(client, work)[0] == {"id": "ENG-1"} + + def test_text_that_is_not_json_comes_back_as_text(self): + client = FakeClient({"search": _result(_text("No results"))}) + + async def work(session): + return await session.call("search", {}) + + assert _run(client, work)[0] == "No results" + + def test_a_tool_error_names_the_tool(self): + client = FakeClient({"get_issue": _result(_text("Entity not found"), error=True)}) + + async def work(session): + return await session.call("get_issue", {"id": "X"}) + + with pytest.raises(mcp.MCPToolError, match="get_issue: Entity not found"): + _run(client, work) + + def test_a_rate_limit_is_worth_retrying(self): + client = FakeClient({"list_issues": _result(_text("Rate limit exceeded, retry later"), error=True)}) + + async def work(session): + return await session.call("list_issues", {}) + + with pytest.raises(service.TransientConnectionError): + _run(client, work) + + def test_input_schemas_are_listed_once(self): + client = FakeClient(tools=[("list_issues", {"properties": {"team": {}}}), ("get_issue", None)]) + + async def work(session): + return ( + await session.input_schema("list_issues"), + await session.input_schema("get_issue"), + await session.input_schema("missing"), + ) + + assert _run(client, work)[0] == ({"properties": {"team": {}}}, {}, None) + assert client.listed == 1 + + @pytest.mark.parametrize("error", [ + MCPReauthorizationRequired("OAuth session expired"), + RuntimeError("Client failed to connect: OAuth session expired — please re-authorize"), + ExceptionGroup("task group", [MCPReauthorizationRequired("OAuth session expired")]), + ]) + def test_a_lost_sign_in_flags_the_connection(self, error): + client = FakeClient(enter_error=error) + + async def work(session): # pragma: no cover - never reached + return None + + with patch.object(service, "mark_reconnect_needed") as flag, pytest.raises(service.ConnectionUnavailable): + _run(client, work) + flag.assert_called_once() + assert flag.call_args.args[0] == "c1" + + def test_a_lost_sign_in_during_a_call_flags_the_connection(self): + client = FakeClient({"list_issues": MCPReauthorizationRequired("OAuth session expired")}) + + async def work(session): + return await session.call("list_issues", {}) + + with patch.object(service, "mark_reconnect_needed") as flag, pytest.raises(service.ConnectionUnavailable): + _run(client, work) + flag.assert_called_once() + + def test_a_connection_that_needs_reconnecting_is_not_used(self): + client = FakeClient() + + async def work(session): # pragma: no cover - never reached + return None + + with pytest.raises(service.ConnectionUnavailable): + _run(client, work, _connection(status="reconnect_needed")) + assert client.calls == [] + + def test_tokens_never_go_to_another_server(self): + async def work(session): # pragma: no cover - never reached + return None + + with pytest.raises(ValueError): + _run(FakeClient(), work, _connection(server_url="https://mcp.notion.com")) + + def test_network_trouble_is_worth_retrying(self): + client = FakeClient({"list_issues": ConnectionError("reset")}) + + async def work(session): + return await session.call("list_issues", {}) + + with patch.object(service, "mark_reconnect_needed") as flag, pytest.raises(service.TransientConnectionError): + _run(client, work) + flag.assert_not_called() + + def test_the_client_signs_in_with_the_connections_stored_tokens(self, monkeypatch): + from fastmcp import Client + + import docsgpt.agents.tools.mcp_tool as mcp_tool + + monkeypatch.setattr(mcp_tool, "validate_url", lambda url, **kw: url) + client = mcp._client_for(_connection(), LINEAR, 30) + assert isinstance(client, Client) + auth = client.transport.auth + assert isinstance(auth, mcp_tool.NonInteractiveOAuth) + assert auth.context.storage.connection_id == "c1" + assert auth.context.storage.user_id == "alice" + + def test_runs_from_inside_a_running_event_loop(self): + import asyncio + + client = FakeClient({"list_teams": _result(_text([]))}) + + async def work(session): + return await session.call("list_teams", {}) + + async def caller(): + return _run(client, work)[0] + + assert asyncio.run(caller()) == [] diff --git a/tests/connectors/test_permissions.py b/tests/connectors/test_permissions.py new file mode 100644 index 00000000..84c37595 --- /dev/null +++ b/tests/connectors/test_permissions.py @@ -0,0 +1,64 @@ +"""Tests for read / write classification and permissions.""" + +from __future__ import annotations + +import pytest + +from docsgpt.connectors import permissions as p + + +class TestActionAccess: + def test_explicit_access_wins(self): + assert p.action_access("telegram", {"name": "search", "access": "write"}) == "write" + + def test_mcp_annotations(self): + assert p.action_access("mcp_tool", {"name": "create_page", "annotations": {"readOnlyHint": True}}) == "read" + assert p.action_access("mcp_tool", {"name": "search", "annotations": {"destructiveHint": True}}) == "write" + + def test_api_tool_method(self): + assert p.action_access("api_tool", {"name": "x", "method": "get"}) == "read" + assert p.action_access("api_tool", {"name": "x", "method": "POST"}) == "write" + + @pytest.mark.parametrize( + "name,expected", + [("search_pages", "read"), ("list_issues", "read"), ("create_issue", "write"), ("send_message", "write")], + ) + def test_name_heuristic(self, name, expected): + assert p.action_access("mcp_tool", {"name": name}) == expected + + @pytest.mark.parametrize( + "name", + ["update_spreadsheet", "create_thread", "set_budget", "update_target", "enlist_member", "threadReply"], + ) + def test_read_word_inside_another_word_is_not_a_read(self, name): + assert p.action_access("mcp_tool", {"name": name}) == "write" + + @pytest.mark.parametrize( + "name", ["get_or_create_page", "search_and_replace", "list-and-delete", "findAndUpdateRecord"] + ) + def test_a_write_verb_wins_over_a_read_verb(self, name): + assert p.action_access("mcp_tool", {"name": name}) == "write" + + @pytest.mark.parametrize("name", ["getSpreadsheet", "list-issues", "Search Pages", "fetchURLContent", "get"]) + def test_read_verbs_match_as_whole_words(self, name): + assert p.action_access("mcp_tool", {"name": name}) == "read" + + +class TestPermissions: + def test_permission_from_flags(self): + assert p.action_permission({"active": False, "require_approval": True}) == "off" + assert p.action_permission({"active": True, "require_approval": True}) == "ask" + assert p.action_permission({"active": True}) == "always" + + def test_apply_permission(self): + assert p.apply_permission({"name": "a"}, "ask") == {"name": "a", "active": True, "require_approval": True} + assert p.apply_permission({"name": "a"}, "off")["active"] is False + with pytest.raises(ValueError): + p.apply_permission({}, "sometimes") + + def test_defaults_writes_to_approval(self): + stamped = p.apply_default_permissions( + "mcp_tool", [{"name": "search"}, {"name": "delete_page"}] + ) + assert stamped[0] == {"name": "search", "access": "read"} + assert stamped[1]["access"] == "write" and stamped[1]["require_approval"] is True diff --git a/tests/connectors/test_policies_and_attribution.py b/tests/connectors/test_policies_and_attribution.py new file mode 100644 index 00000000..73e3dbdc --- /dev/null +++ b/tests/connectors/test_policies_and_attribution.py @@ -0,0 +1,392 @@ +"""Admin connector policies, sharing modes and connector attribution.""" + +from __future__ import annotations + +import json +from contextlib import contextmanager +from types import SimpleNamespace +from unittest.mock import patch + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.connectors import catalog, service +from docsgpt.security.encryption import encrypt_json + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + modules = ( + "docsgpt.connectors.service", + "docsgpt.connectors.resolve", + "docsgpt.connectors.attribution", + "docsgpt.api.connector.connections", + "docsgpt.api.admin.connectors", + "docsgpt.api.user.tools.mcp", + ) + patches = [] + for module in modules: + for name in ("db_session", "db_readonly"): + target = f"{module}.{name}" + try: + patches.append(patch(target, _yield)) + patches[-1].start() + except AttributeError: + patches.pop() + try: + yield + finally: + for p in patches: + p.stop() + + +def _connection(conn, user="alice", provider="telegram", status="connected", auth_kind="api_key", + server_url=None, secrets=None) -> str: + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "server_url, encrypted_credentials) VALUES (:u, :p, :p, :a, :s, :l, :url, :e) RETURNING id" + ), + {"u": user, "p": provider, "a": auth_kind, "s": status, "l": f"{user}@example.com", "url": server_url, + "e": encrypt_json(secrets or {"credentials": {"token": "tok"}}, user)}, + ).scalar()) + + +def _call(app, resource, method, path, *, user="alice", body=None, roles=None, args=()): + with app.test_request_context(path, method=method.upper(), json=body): + from flask import request + + request.decoded_token = {"sub": user, "roles": roles or ["user"]} if user else None + return getattr(resource(), method)(*args) + + +class TestPresets: + def test_presets_are_in_the_catalog(self): + presets = [d for d in catalog.all_definitions() if d.publisher == "preset"] + assert {p.key for p in presets} >= {"mcp:notion", "mcp:linear"} + for preset in presets: + assert preset.mcp_url.startswith("https://") + assert preset.auth_kind in ("mcp_oauth", "none") + assert preset.tool_templates == ("mcp_tool",) + + def test_existing_mcp_connection_maps_to_its_preset(self): + row = {"provider": "mcp:https://mcp.linear.app", "server_url": "https://mcp.linear.app", + "connector_key": "custom_mcp"} + assert catalog.connector_key_for_row(row) == "mcp:linear" + + +class TestAdminPolicies: + def test_requires_admin(self, app, pg_conn): + from docsgpt.api.admin.connectors import AdminConnectorsResource + + with _db(pg_conn): + resp = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors") + assert resp.status_code == 403 + + def test_lists_setup_state_without_values(self, app, pg_conn, monkeypatch): + from docsgpt.api.admin.connectors import AdminConnectorsResource + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", "secret-client-id") + monkeypatch.setattr(settings, "GOOGLE_CLIENT_SECRET", None) + _connection(pg_conn, provider="google_drive", auth_kind="oauth") + with _db(pg_conn): + resp = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors", roles=["admin"]) + payload = resp.get_json() + drive = next(c for c in payload["connectors"] if c["key"] == "google_drive") + assert drive["required_settings"] == [ + {"name": "GOOGLE_CLIENT_ID", "set": True}, {"name": "GOOGLE_CLIENT_SECRET", "set": False}, + ] + assert drive["connection_count"] == 1 + # Sync-only: the page shows no sharing policy for it. + assert drive["capabilities"] == ["sync"] + assert "secret-client-id" not in json.dumps(payload) + assert payload["allow_custom_mcp"] is True + + def test_github_is_ready_and_lists_its_optional_app_settings(self, app, pg_conn, monkeypatch): + """Tokens need no setup; the GitHub App settings only add Sign in with GitHub.""" + from docsgpt.api.admin.connectors import AdminConnectorsResource + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_CLIENT_ID", "Iv1.secret-client") + monkeypatch.setattr(settings, "GITHUB_CLIENT_SECRET", None) + monkeypatch.setattr(settings, "GITHUB_APP_SLUG", None) + with _db(pg_conn): + resp = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors", roles=["admin"]) + payload = resp.get_json() + github = next(c for c in payload["connectors"] if c["key"] == "github") + assert github["configured"] is True and github["enabled"] is True + assert github["required_settings"] == [] + assert github["oauth_settings"] == [ + {"name": "GITHUB_CLIENT_ID", "set": True}, + {"name": "GITHUB_CLIENT_SECRET", "set": False}, + {"name": "GITHUB_APP_SLUG", "set": False}, + ] + assert github["oauth_configured"] is False + assert "Iv1.secret-client" not in json.dumps(payload) + drive = next(c for c in payload["connectors"] if c["key"] == "google_drive") + assert drive["oauth_settings"] == [] and drive["oauth_configured"] is False + + def test_disable_connector_and_custom_mcp(self, app, pg_conn): + from docsgpt.api.admin.connectors import AdminConnectorsResource + + with _db(pg_conn): + resp = _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], body={ + "policies": {"telegram": {"enabled": False, "credential_mode": "member"}}, + "allow_custom_mcp": False, + }) + assert resp.status_code == 200 + entries = {e["key"]: e for e in service.catalog_for_user(pg_conn, "bob", is_admin=False)} + assert "telegram" not in entries + assert "custom_mcp" not in entries + with pytest.raises(service.ConnectorDisabled): + service.create_api_key_connection( + pg_conn, "bob", catalog.get_definition("telegram"), {"token": "long-enough-token"}, + ) + + def test_a_preset_server_on_a_key_follows_the_preset_switch(self, app, pg_conn): + """A key-based connection to a preset's server is that preset, not a + custom server: turning custom servers off leaves it alone, and turning + the preset off stops it.""" + from docsgpt.api.admin.connectors import AdminConnectorsResource + + linear = catalog.get_definition("mcp:linear") + custom = catalog.get_definition("custom_mcp") + with _db(pg_conn): + _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], body={ + "policies": {}, "allow_custom_mcp": False, + }) + row, _ = service.create_api_key_connection( + pg_conn, "bob", custom, {"api_key": "lin-key-123456"}, server_url=linear.mcp_base_url, + ) + assert catalog.connector_key_for_row(row) == "mcp:linear" + with pytest.raises(service.ConnectorDisabled): + service.create_api_key_connection( + pg_conn, "bob", custom, {"api_key": "other-key-123456"}, server_url="https://mcp.example.com", + ) + _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], body={ + "policies": {"mcp:linear": {"enabled": False}}, "allow_custom_mcp": True, + }) + with pytest.raises(service.ConnectorDisabled): + service.create_api_key_connection( + pg_conn, "bob", custom, {"api_key": "lin-key-654321"}, server_url=linear.mcp_base_url, + ) + + def test_unconfigured_connectors_are_off_until_an_admin_turns_them_on(self, app, pg_conn, monkeypatch): + from docsgpt.api.admin.connectors import AdminConnectorsResource + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "MICROSOFT_CLIENT_ID", None) + with _db(pg_conn): + before = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors", roles=["admin"]).get_json() + # Changing only the sharing mode must not switch it on. + _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], + body={"policies": {"share_point": {"credential_mode": "member"}}}) + middle = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors", roles=["admin"]).get_json() + _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], + body={"policies": {"share_point": {"enabled": True}}}) + after = _call(app, AdminConnectorsResource, "get", "/api/admin/connectors", roles=["admin"]).get_json() + + def enabled(payload, key): + return next(c for c in payload["connectors"] if c["key"] == key)["enabled"] + + assert enabled(before, "share_point") is False + assert enabled(before, "telegram") is True + assert enabled(middle, "share_point") is False + assert enabled(after, "share_point") is True + + def test_rejects_unknown_connector(self, app, pg_conn): + from docsgpt.api.admin.connectors import AdminConnectorsResource + + with _db(pg_conn): + resp = _call(app, AdminConnectorsResource, "put", "/api/admin/connectors", roles=["admin"], + body={"policies": {"nope": {"enabled": False}}}) + assert resp.status_code == 400 + + def test_custom_mcp_switch_is_enforced_server_side(self, app, pg_conn): + from docsgpt.api.user.tools.mcp import TestMCPServerConfig + from docsgpt.storage.db.repositories.app_metadata import AppMetadataRepository + + AppMetadataRepository(pg_conn).set("connectors.allow_custom_mcp", "false") + with _db(pg_conn), patch("docsgpt.api.user.tools.mcp._validate_mcp_server_url"): + custom = _call(app, TestMCPServerConfig, "post", "/api/mcp_server/test", + body={"config": {"server_url": "https://tools.example.com/mcp"}}) + assert custom.status_code == 403 + + def test_disabled_connector_stops_existing_tools(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + cid = _connection(pg_conn) + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _db(pg_conn): + resolved = resolve_connection({"connection_id": cid, "user_id": "alice"}, "alice") + assert resolved.available is False + + def test_forced_member_mode_overrides_the_share(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + owner = _connection(pg_conn) + bobs = _connection(pg_conn, user="bob") + ConnectorPoliciesRepository(pg_conn).upsert("telegram", credential_mode="member") + with _db(pg_conn): + resolved = resolve_connection( + {"connection_id": owner, "user_id": "alice", "credential_mode": "owner"}, "bob", + ) + assert resolved.connection_id == bobs + + +class TestCredentialMode: + def _tool(self, conn, cid, user="alice"): + return str(conn.execute( + text("INSERT INTO user_tools (user_id, name, connection_id) VALUES (:u, 'telegram', CAST(:c AS uuid)) " + "RETURNING id"), + {"u": user, "c": cid}, + ).scalar()) + + def test_owner_sets_member_mode(self, app, pg_conn): + from docsgpt.api.connector.connections import ToolCredentialMode + + tool = self._tool(pg_conn, _connection(pg_conn)) + with _db(pg_conn): + resp = _call(app, ToolCredentialMode, "put", "/x", body={"mode": "member"}, args=[tool]) + assert resp.status_code == 200 + mode = pg_conn.execute(text("SELECT credential_mode FROM user_tools WHERE id = CAST(:i AS uuid)"), + {"i": tool}).scalar() + assert mode == "member" + + def test_other_users_cannot(self, app, pg_conn): + from docsgpt.api.connector.connections import ToolCredentialMode + + tool = self._tool(pg_conn, _connection(pg_conn)) + with _db(pg_conn): + resp = _call(app, ToolCredentialMode, "put", "/x", user="bob", body={"mode": "member"}, args=[tool]) + assert resp.status_code == 404 + + def test_forced_policy_wins(self, app, pg_conn): + from docsgpt.api.connector.connections import ToolCredentialMode + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + tool = self._tool(pg_conn, _connection(pg_conn)) + ConnectorPoliciesRepository(pg_conn).upsert("telegram", credential_mode="owner") + with _db(pg_conn): + resp = _call(app, ToolCredentialMode, "put", "/x", body={"mode": "member"}, args=[tool]) + assert resp.status_code == 409 + assert resp.get_json()["mode"] == "owner" + + +def _executor(user): + from docsgpt.agents.tool_executor import ToolExecutor + + return ToolExecutor(user=user) + + +def _pause(executor, tool, action_name): + with patch("docsgpt.agents.tool_executor.ToolActionParser") as parser: + parser.return_value.parse_args.return_value = ("t1", action_name, {}) + return executor.check_pause( + {"t1": tool}, SimpleNamespace(id="c1", name=action_name, thought_signature=None), "OpenAILLM", + ) + + +class TestSharedWrites: + def _tool(self, cid): + return { + "id": "tool-1", + "user_id": "alice", + "name": "postgres", + "config": {}, + "connection_id": cid, + "credential_mode": "owner", + "actions": [ + {"name": "postgres_execute_sql", "access": "write", "require_approval": False, "active": True}, + {"name": "postgres_get_schema", "access": "read", "require_approval": False, "active": True}, + ], + } + + def test_member_on_owners_account_must_approve_writes(self, pg_conn): + cid = _connection(pg_conn, provider="postgres") + with _db(pg_conn): + pause = _pause(_executor("bob"), self._tool(cid), "postgres_execute_sql") + assert pause["pause_type"] == "awaiting_approval" + + def test_member_reads_are_not_gated(self, pg_conn): + cid = _connection(pg_conn, provider="postgres") + with _db(pg_conn): + assert _pause(_executor("bob"), self._tool(cid), "postgres_get_schema") is None + + def test_owner_keeps_their_own_choice(self, pg_conn): + cid = _connection(pg_conn, provider="postgres") + with _db(pg_conn): + assert _pause(_executor("alice"), self._tool(cid), "postgres_execute_sql") is None + + +class TestAttribution: + def test_tool_call_events_name_the_connector_not_the_account(self, pg_conn): + cid = _connection(pg_conn, provider="mcp:https://mcp.notion.com", auth_kind="mcp_oauth", + server_url="https://mcp.notion.com", secrets={"tokens": {"access_token": "t"}}) + # MCP rows are stored as custom_mcp and named after their preset. + pg_conn.execute(text("UPDATE connector_sessions SET connector_key = 'custom_mcp' WHERE id = CAST(:i AS uuid)"), + {"i": cid}) + executor = _executor("alice") + tool = { + "id": "tool-1", "user_id": "alice", "name": "mcp_tool", "connection_id": cid, + "config": {"server_url": "https://mcp.notion.com/mcp", "auth_type": "oauth"}, + "actions": [{"name": "search", "active": True, "parameters": {"properties": {}}, + "annotations": {"readOnlyHint": True}}], + } + fake_tool = SimpleNamespace(execute_action=lambda *a, **k: "ok", config={}) + call = SimpleNamespace(id="c1", name="search", arguments="{}", thought_signature=None) + with _db(pg_conn), patch("docsgpt.agents.tool_executor.ToolActionParser") as parser, patch.object( + type(executor), "_get_or_load_tool", return_value=fake_tool + ), patch("docsgpt.agents.tool_executor._record_proposed", return_value=True), patch( + "docsgpt.agents.tool_executor._mark_executed" + ): + parser.return_value.parse_args.return_value = ("t1", "search", {}) + events = list(executor._execute({"t1": tool}, call, "OpenAILLM")) + final = events[-1]["data"] + assert final["connector_key"] == "mcp:notion" + assert final["connector_name"] == "Notion" + assert final["access"] == "read" + assert "alice@example.com" not in json.dumps(events) + + def test_built_in_tools_carry_no_connector(self): + executor = _executor("alice") + assert executor._resolve_connection({"id": "x", "name": "memory"}) is None + + def test_citations_name_the_connector(self, pg_conn): + from docsgpt.connectors.attribution import connector_labels + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth") + source = str(pg_conn.execute(text( + "INSERT INTO sources (user_id, name, connection_id) VALUES ('alice', 'Handbook', CAST(:c AS uuid)) " + "RETURNING id" + ), {"c": cid}).scalar()) + upload = str(pg_conn.execute(text( + "INSERT INTO sources (user_id, name) VALUES ('alice', 'Upload') RETURNING id" + )).scalar()) + with _db(pg_conn): + labels = connector_labels([source, upload, "not-a-uuid"]) + assert labels == {source: {"connector_key": "google_drive", "connector_name": "Google Drive"}} + + def test_classic_retriever_stamps_connector(self, pg_conn): + from docsgpt.retriever.classic_rag import ClassicRAG + + retriever = ClassicRAG.__new__(ClassicRAG) + with patch("docsgpt.connectors.attribution.connector_labels", + return_value={"s1": {"connector_key": "google_drive", "connector_name": "Google Drive"}}) as lookup: + first = retriever._connector_labels.for_source("s1") + again = retriever._connector_labels.for_source("s1") + assert first == again == {"connector_key": "google_drive", "connector_name": "Google Drive"} + assert lookup.call_count == 1 diff --git a/tests/connectors/test_public_link_callers.py b/tests/connectors/test_public_link_callers.py new file mode 100644 index 00000000..25f93b66 --- /dev/null +++ b/tests/connectors/test_public_link_callers.py @@ -0,0 +1,362 @@ +"""Someone reaching an agent only through its public link can't approve for the owner. + +A signed-in stranger with the link runs the agent like a teammate, but a +write on the owner's connected account (a tool in owner mode) is refused +unless the owner allowlisted it, exactly as for an API-key caller. Their own +account (member mode) stays theirs to approve, and teammates keep the card. +""" + +from __future__ import annotations + +import json +import uuid +from contextlib import ExitStack, contextmanager +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy import text + +from docsgpt.security.encryption import encrypt_json + +ACTION = "telegram_send_message" + + +@contextmanager +def _db(conn, *modules): + @contextmanager + def _yield(): + yield conn + + with ExitStack() as stack: + stack.enter_context(patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield)) + stack.enter_context(patch.multiple("docsgpt.connectors.resolve", db_readonly=_yield)) + for module in modules: + stack.enter_context(patch.multiple(module, db_session=_yield, db_readonly=_yield)) + yield + + +def _connection(conn, user: str) -> str: + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "account_label, encrypted_credentials) VALUES (:u, 'telegram', 'telegram', 'api_key', " + "'connected', :u, :e) RETURNING id" + ), + {"u": user, "e": encrypt_json({"credentials": {"token": f"{user}-token"}}, user)}, + ).scalar()) + + +def _tool(connection_id: str, mode: str = "owner") -> dict: + return { + "id": "tool-1", "user_id": "alice", "name": "telegram", "config": {}, + "actions": [{"name": ACTION, "active": True, "require_approval": False}], + "connection_id": connection_id, "credential_mode": mode, + } + + +def _pause(executor, tool, action=ACTION): + call = SimpleNamespace(id="call-1", name=action, arguments="{}", thought_signature=None) + with patch("docsgpt.agents.tool_executor.ToolActionParser") as parser: + parser.return_value.parse_args.return_value = ("t1", action, {}) + return executor.check_pause({"t1": tool}, call, "OpenAILLM") + + +def _executor(public: bool, allowlist=None): + from docsgpt.agents.tool_executor import ToolExecutor + + return ToolExecutor(user="bob", public_link_caller=public, api_write_allowlist=allowlist) + + +class TestWritesOnTheOwnersAccount: + def test_public_link_write_is_refused_without_a_card(self, pg_conn): + cid = _connection(pg_conn, "alice") + with _db(pg_conn): + pause = _pause(_executor(public=True), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + assert pause["error_type"] == "tool_not_allowed" + assert "public link" in pause["deny_reason"] + assert "Access details" in pause["deny_reason"] + + def test_allowlisted_write_runs(self, pg_conn): + cid = _connection(pg_conn, "alice") + with _db(pg_conn): + assert _pause(_executor(public=True, allowlist=[f"tool-1:{ACTION}"]), _tool(cid)) is None + + def test_allowlist_covers_only_its_action(self, pg_conn): + cid = _connection(pg_conn, "alice") + with _db(pg_conn): + pause = _pause(_executor(public=True, allowlist=["tool-1:telegram_send_image"]), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + + def test_teammate_keeps_the_approval_card(self, pg_conn): + cid = _connection(pg_conn, "alice") + with _db(pg_conn): + pause = _pause(_executor(public=False), _tool(cid)) + assert pause["pause_type"] == "awaiting_approval" + + def test_own_account_in_member_mode_is_not_refused(self, pg_conn): + cid = _connection(pg_conn, "alice") + _connection(pg_conn, "bob") + with _db(pg_conn): + assert _pause(_executor(public=True), _tool(cid, mode="member")) is None + + +def _team_grant(conn, agent_id: str, owner: str, member: str) -> None: + from docsgpt.storage.db.repositories.team_members import TeamMembersRepository + from docsgpt.storage.db.repositories.team_resource_grants import TeamResourceGrantsRepository + from docsgpt.storage.db.repositories.teams import TeamsRepository + + team = TeamsRepository(conn).create("T", f"t-{uuid.uuid4().hex[:8]}", owner) + TeamMembersRepository(conn).add_member(str(team["id"]), member) + TeamResourceGrantsRepository(conn).grant( + str(team["id"]), "agent", agent_id, owner, owner, access_level="viewer", target_user_id=member, + ) + + +class TestPublicLinkDetection: + SP = "docsgpt.api.answer.services.stream_processor" + + def _agent(self, conn, shared=True) -> str: + from docsgpt.storage.db.repositories.agents import AgentsRepository + + return str(AgentsRepository(conn).create( + "alice", "A", "published", key=f"k-{uuid.uuid4().hex}", shared=shared, + )["id"]) + + def _run(self, conn, agent_id: str, caller: str): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + processor = StreamProcessor({"agent_id": agent_id}, {"sub": caller}) + with _db(conn, self.SP): + processor._configure_agent() + return processor + + @pytest.mark.parametrize(("caller", "public"), [("alice", False), ("stranger", True)]) + def test_only_a_link_user_is_a_public_caller(self, pg_conn, caller, public): + processor = self._run(pg_conn, self._agent(pg_conn), caller) + assert processor.agent_config["public_link_caller"] is public + + def test_teammate_of_a_public_agent_is_not_a_link_user(self, pg_conn): + agent_id = self._agent(pg_conn) + _team_grant(pg_conn, agent_id, "alice", "bob") + processor = self._run(pg_conn, agent_id, "bob") + assert processor.is_shared_usage is True + assert processor.agent_config["public_link_caller"] is False + + def test_run_executor_carries_the_flag(self): + from docsgpt.api.answer.services.stream_processor import StreamProcessor + + processor = StreamProcessor({}, {"sub": "stranger"}) + processor.agent_config = { + "agent_type": "classic", "prompt_id": "default", "user_api_key": "k", + "public_link_caller": True, "api_write_allowlist": [f"tool-1:{ACTION}"], + } + processor._get_prompt_content = MagicMock(return_value="p") + processor.prompt_renderer = MagicMock(render_prompt=MagicMock(return_value="p")) + processor._enabled_tool_names = MagicMock(return_value=set()) + processor.model_id = "m1" + with patch(f"{self.SP}.get_provider_from_model_id", return_value="openai"), \ + patch(f"{self.SP}.get_api_key_for_provider", return_value="key"), \ + patch("docsgpt.llm.llm_creator.LLMCreator.create_llm", return_value=MagicMock()), \ + patch("docsgpt.llm.handlers.handler_creator.LLMHandlerCreator.create_handler"), \ + patch("docsgpt.agents.agent_creator.AgentCreator.create_agent") as create: + processor.create_agent() + executor = create.call_args.kwargs["tool_executor"] + assert executor.public_link_caller is True + assert executor.api_write_allowlist == {f"tool-1:{ACTION}"} + + def test_resumed_run_stays_a_public_caller(self, monkeypatch): + from docsgpt.agents import agent_creator as ac_mod + from docsgpt.api.answer.services import continuation_service as cont_mod + from docsgpt.api.answer.services.stream_processor import StreamProcessor + from docsgpt.llm import llm_creator as llm_creator_mod + from docsgpt.llm.handlers import handler_creator as handler_mod + + cont_service = MagicMock() + cont_service.claim_state.return_value = { + "messages": [], "pending_tool_calls": [], "tools_dict": {}, "tool_schemas": [], + "client_tools": None, + "agent_config": {"model_id": "m1", "llm_name": "openai", "api_key": "k", "user_api_key": "uk", + "agent_type": "ClassicAgent", "public_link_caller": True}, + } + monkeypatch.setattr(cont_mod, "ContinuationService", lambda: cont_service) + monkeypatch.setattr(llm_creator_mod.LLMCreator, "create_llm", lambda *a, **kw: MagicMock()) + monkeypatch.setattr(handler_mod.LLMHandlerCreator, "create_handler", lambda *a, **kw: MagicMock()) + created = {} + monkeypatch.setattr( + ac_mod.AgentCreator, "create_agent", lambda *a, **kw: created.update(kw) or MagicMock(), + ) + processor = StreamProcessor({}, {"sub": "stranger"}) + processor.resume_from_tool_actions(tool_actions=[], conversation_id=str(uuid.uuid4())) + assert created["tool_executor"].public_link_caller is True + + +class TestWorkflowNodes: + """A node's tools follow the same caller rules as the run that started it.""" + + def test_node_executor_inherits_the_run_policy(self, monkeypatch): + from docsgpt.agents.tool_executor import ToolExecutor + from docsgpt.agents.workflows.node_agent import WorkflowNodeAgentFactory, _WorkflowNodeMixin + from docsgpt.agents.workflows.schemas import NodeType, Workflow, WorkflowGraph, WorkflowNode + from docsgpt.agents.workflows.workflow_engine import WorkflowEngine + + class _Base: + def __init__(self, decoded_token=None, **_kwargs): + self.tool_executor = ToolExecutor(user=(decoded_token or {}).get("sub")) + + class _NodeAgent(_WorkflowNodeMixin, _Base): + def gen(self, _prompt): + yield {"answer": "ok"} + + built = [] + monkeypatch.setattr( + WorkflowNodeAgentFactory, "create", + staticmethod(lambda agent_type, **kw: built.append(_NodeAgent(**kw)) or built[-1]), + ) + monkeypatch.setattr("docsgpt.core.model_utils.get_api_key_for_provider", lambda _name: None) + run_executor = ToolExecutor( + user="stranger", headless=True, tool_allowlist=["t9"], external_caller=True, + public_link_caller=True, api_write_allowlist=[f"tool-1:{ACTION}"], + ) + agent = SimpleNamespace( + endpoint="stream", llm_name="openai", model_id="gpt-4o-mini", api_key="k", chat_history=[], + decoded_token={"sub": "stranger"}, user="stranger", _resolve_owner_id=lambda: "alice", + tool_executor=run_executor, + ) + engine = WorkflowEngine(WorkflowGraph(workflow=Workflow(name="wf"), nodes=[], edges=[]), agent) + engine.state["query"] = "q" + node = WorkflowNode( + id="a1", workflow_id="wf", type=NodeType.AGENT, title="A", position={"x": 0, "y": 0}, + config={"agent_type": "classic", "system_prompt": "s", "tools": []}, + ) + list(engine._execute_agent_node(node)) + + node_executor = built[0].tool_executor + assert node_executor.headless is True + assert node_executor.tool_allowlist == {"t9"} + assert node_executor.external_caller is True + assert node_executor.public_link_caller is True + assert node_executor.api_write_allowlist == {f"tool-1:{ACTION}"} + + +def _stored_tool(name: str, action: dict, config: dict) -> dict: + return {"id": "tool-9", "user_id": "alice", "name": name, "config": config, "actions": [action]} + + +def _api_tool(method: str, header: dict | None = None, **config) -> dict: + """An API tool with one ``call`` action; ``header`` is its Authorization header spec. + + By default the header holds a sealed secret (``has_value``), the way a + saved key is stored. + """ + action = {"url": "https://x.test", "method": method, "active": True, "require_approval": False} + if header is None: + header = {"type": "string", "value": "", "has_value": True} + if header: + action["headers"] = {"type": "object", "properties": {"Authorization": header}} + tool = _stored_tool("api_tool", {}, {"actions": {"call": action}, **config}) + tool["actions"] = [] + return tool + + +class TestOwnerHeldCredentialsWithoutAConnection: + """Writes with the owner's stored credentials are gated like connected ones.""" + + def _caller(self, **flags): + from docsgpt.agents.tool_executor import ToolExecutor + + flags.setdefault("user", "bob") + return ToolExecutor(**flags) + + @pytest.mark.parametrize("flags", [{"public_link_caller": True}, {"external_caller": True, "user": "alice"}]) + def test_api_tool_write_is_refused(self, flags): + pause = _pause(self._caller(**flags), _api_tool("POST"), action="call") + assert pause["pause_type"] == "headless_denied" + assert "Access details" in pause["deny_reason"] + + @pytest.mark.parametrize("header", [ + {"type": "string", "value": "", "has_value": True}, + {"type": "string", "value": "Bearer legacy-plaintext"}, + ]) + def test_api_tool_with_a_stored_header_value_is_gated(self, header): + pause = _pause(self._caller(public_link_caller=True), _api_tool("POST", header), action="call") + assert pause["pause_type"] == "headless_denied" + + def test_api_tool_with_stored_credentials_in_config_is_gated(self): + tool = _api_tool("POST", {}, encrypted_credentials="blob") + assert _pause(self._caller(public_link_caller=True), tool, action="call")["pause_type"] == "headless_denied" + + @pytest.mark.parametrize("header", [{}, {"type": "string", "value": "", "has_value": False}, + {"type": "string", "filled_by_llm": True}]) + def test_api_tool_without_credentials_is_not_gated(self, header): + assert _pause(self._caller(public_link_caller=True), _api_tool("POST", header), action="call") is None + + def test_api_tool_read_runs(self): + assert _pause(self._caller(public_link_caller=True), _api_tool("GET"), action="call") is None + + def test_allowlisted_api_tool_write_runs(self): + caller = self._caller(public_link_caller=True, api_write_allowlist=["tool-9:call"]) + assert _pause(caller, _api_tool("POST"), action="call") is None + + def test_mcp_tool_with_stored_sign_in_is_gated(self): + tool = _stored_tool("mcp_tool", {"name": "create_issue", "active": True}, + {"server_url": "https://m.test/mcp", "auth_type": "bearer"}) + pause = _pause(self._caller(public_link_caller=True), tool, action="create_issue") + assert pause["pause_type"] == "headless_denied" + + def test_mcp_tool_without_credentials_is_not_gated(self): + tool = _stored_tool("mcp_tool", {"name": "create_issue", "active": True}, + {"server_url": "https://m.test/mcp", "auth_type": "none"}) + assert _pause(self._caller(public_link_caller=True), tool, action="create_issue") is None + + def test_teammate_is_not_gated(self): + assert _pause(self._caller(), _api_tool("POST"), action="call") is None + + +class TestScheduledRunsForOthers: + """A scheduled run acts as the owner, for someone who can't approve for them.""" + + def test_public_link_schedule_cannot_write_on_the_owners_account(self, pg_conn): + from docsgpt.agents.tool_executor import ToolExecutor + + cid = _connection(pg_conn, "alice") + executor = ToolExecutor(user="alice", headless=True, public_link_caller=True) + with _db(pg_conn): + pause = _pause(executor, _tool(cid)) + assert pause["pause_type"] == "headless_denied" + assert "public link" in pause["deny_reason"] + + def test_owners_own_schedule_is_not_gated(self, pg_conn): + from docsgpt.agents.tool_executor import ToolExecutor + + cid = _connection(pg_conn, "alice") + with _db(pg_conn): + assert _pause(ToolExecutor(user="alice", headless=True), _tool(cid)) is None + + +class TestSchedulesSetThroughTheApi: + def test_scheduler_loaded_for_an_api_caller_records_it(self): + from docsgpt.agents.tool_executor import ToolExecutor + + executor = ToolExecutor(user="alice", agent_id="11111111-1111-1111-1111-111111111111", external_caller=True) + row = {"id": "sched", "name": "scheduler", "config": {}, "actions": []} + with patch("docsgpt.agents.tool_executor.ToolManager") as manager: + executor._get_or_load_tool(row, "sched", "schedule_task") + assert manager.return_value.load_tool.call_args.kwargs["tool_config"]["created_via"] == "api" + + def test_api_schedule_is_stored_as_such(self, pg_conn): + from docsgpt.agents.tools.scheduler import SchedulerTool + from docsgpt.storage.db.repositories.agents import AgentsRepository + from docsgpt.storage.db.repositories.schedules import SchedulesRepository + + agent_id = str(AgentsRepository(pg_conn).create("alice", "A", "published", key=f"k-{uuid.uuid4().hex}")["id"]) + tool = SchedulerTool({"agent_id": agent_id, "created_via": "api"}, user_id="alice") + + @contextmanager + def _yield(): + yield pg_conn + + with patch.multiple("docsgpt.agents.tools.scheduler", db_session=_yield, db_readonly=_yield): + created = json.loads(tool.execute_action("schedule_task", instruction="ping", delay="1h")) + assert SchedulesRepository(pg_conn).get_internal(created["task_id"])["created_via"] == "api" diff --git a/tests/connectors/test_runtime.py b/tests/connectors/test_runtime.py new file mode 100644 index 00000000..c2a2201f --- /dev/null +++ b/tests/connectors/test_runtime.py @@ -0,0 +1,696 @@ +"""Runtime use of connections: tool execution, sharing modes and scheduled sync.""" + +from __future__ import annotations + +import json +import logging +from contextlib import contextmanager +from types import SimpleNamespace +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy import text + +from docsgpt.security.encryption import encrypt_json + + +@contextmanager +def _service_db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.resolve", db_readonly=_yield): + yield + + +def _connection(conn, user="alice", provider="telegram", status="connected", secrets=None, + auth_kind="api_key", server_url=None) -> str: + return str(conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "account_label, server_url, encrypted_credentials) VALUES (:u, :p, :p, :a, :s, :l, :url, :e) RETURNING id" + ), + {"u": user, "p": provider, "a": auth_kind, "s": status, "l": f"{user}-label", "url": server_url, + "e": encrypt_json(secrets or {"credentials": {"token": f"{user}-token"}}, user)}, + ).scalar()) + + +def _tool(connection_id, *, user="alice", name="telegram", mode="owner", tool_id="tool-1"): + return { + "id": tool_id, + "user_id": user, + "name": name, + "config": {}, + "actions": [{"name": "telegram_send_message", "active": True, "require_approval": False}], + "connection_id": connection_id, + "credential_mode": mode, + } + + +def _call(action="telegram_send_message"): + return SimpleNamespace(id="call-1", name=action, arguments="{}", thought_signature=None) + + +def _executor(user="alice", headless=False): + from docsgpt.agents.tool_executor import ToolExecutor + + executor = ToolExecutor(user=user, headless=headless) + return executor + + +def _pause(executor, tool): + with patch("docsgpt.agents.tool_executor.ToolActionParser") as parser: + parser.return_value.parse_args.return_value = ("t1", "telegram_send_message", {}) + return executor.check_pause({"t1": tool}, _call(), "OpenAILLM") + + +class TestResolution: + def test_owner_mode_uses_owner_connection(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + cid = _connection(pg_conn) + with _service_db(pg_conn): + resolved = resolve_connection(_tool(cid), "bob") + assert resolved.available and resolved.connection_id == cid + assert resolved.delegated is True + assert resolved.connector_name == "Telegram" + + def test_member_mode_uses_invokers_connection(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + owner = _connection(pg_conn) + bobs = _connection(pg_conn, user="bob") + with _service_db(pg_conn): + resolved = resolve_connection(_tool(owner, mode="member"), "bob") + assert resolved.connection_id == bobs + assert resolved.delegated is False + + def test_member_with_several_accounts_uses_the_one_last_used(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + owner = _connection(pg_conn) + older = _connection(pg_conn, user="bob") + pg_conn.execute(text( + "UPDATE connector_sessions SET account_label = 'bob-home', last_used_at = now() - interval '1 day' " + "WHERE id = CAST(:i AS uuid)" + ), {"i": older}) + newer = _connection(pg_conn, user="bob") + pg_conn.execute(text( + "UPDATE connector_sessions SET last_used_at = now() WHERE id = CAST(:i AS uuid)" + ), {"i": newer}) + with _service_db(pg_conn): + resolved = resolve_connection(_tool(owner, mode="member"), "bob") + assert resolved.available and resolved.connection_id == newer + + def test_member_gets_the_account_they_just_connected(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + owner = _connection(pg_conn) + used = _connection(pg_conn, user="bob") + pg_conn.execute(text( + "UPDATE connector_sessions SET account_label = 'bob-work', created_at = now() - interval '30 days', " + "updated_at = now() - interval '30 days', last_used_at = now() - interval '1 hour' " + "WHERE id = CAST(:i AS uuid)" + ), {"i": used}) + # Added by "Connect to continue": never used yet. + added = _connection(pg_conn, user="bob") + with _service_db(pg_conn): + resolved = resolve_connection(_tool(owner, mode="member"), "bob") + assert resolved.available and resolved.connection_id == added + + def test_member_mode_without_own_connection_is_unavailable(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + owner = _connection(pg_conn) + with _service_db(pg_conn): + resolved = resolve_connection(_tool(owner, mode="member"), "bob") + assert resolved.available is False + assert resolved.connector_name == "Telegram" + + def test_member_mode_matches_mcp_server(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + owner = _connection(pg_conn, provider="mcp:https://a.example.com", auth_kind="mcp_oauth", + server_url="https://a.example.com", secrets={"tokens": {"access_token": "x"}}) + _connection(pg_conn, user="bob", provider="mcp:https://b.example.com", auth_kind="mcp_oauth", + server_url="https://b.example.com", secrets={"tokens": {"access_token": "y"}}) + with _service_db(pg_conn): + resolved = resolve_connection(_tool(owner, name="mcp_tool", mode="member"), "bob") + assert resolved.available is False + + def test_resource_cannot_borrow_another_users_connection(self, pg_conn): + from docsgpt.connectors.resolve import resolve_connection + + mallorys_target = _connection(pg_conn, user="victim") + with _service_db(pg_conn): + resolved = resolve_connection(_tool(mallorys_target, user="mallory"), "mallory") + assert resolved.available is False and resolved.row is None + + +class TestExecutor: + def test_credentials_come_from_the_connection(self, pg_conn): + cid = _connection(pg_conn) + executor = _executor() + tool = _tool(cid) + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + executor._get_or_load_tool(tool, "t1", "telegram_send_message") + config = manager.return_value.load_tool.call_args.kwargs["tool_config"] + assert config["token"] == "alice-token" + assert "encrypted_credentials" not in config + + def test_owner_mode_delegation_is_audited(self, pg_conn, caplog): + cid = _connection(pg_conn) + executor = _executor(user="bob") + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager"), \ + caplog.at_level(logging.INFO, logger="docsgpt.connectors.resolve"): + executor._get_or_load_tool(_tool(cid), "t1", "telegram_send_message") + record = next(r for r in caplog.records if r.message == "tool_credential_delegation") + assert record.connection_id == cid and record.invoker == "bob" + + def test_needs_reconnect_pauses_on_connect_card(self, pg_conn): + cid = _connection(pg_conn, status="reconnect_needed") + with _service_db(pg_conn): + pause = _pause(_executor(), _tool(cid)) + assert pause["pause_type"] == "awaiting_approval" + # The caller's own connection: the card can reconnect it in place. + assert pause["connection_required"] == { + "connector_key": "telegram", "connector_name": "Telegram", "status": "reconnect_needed", + "connection_id": cid, "owner_account": False, + } + + def test_owners_broken_account_is_not_handed_to_the_member(self, pg_conn): + cid = _connection(pg_conn, status="reconnect_needed") + with _service_db(pg_conn): + pause = _pause(_executor(user="bob"), _tool(cid)) + required = pause["connection_required"] + assert required["owner_account"] is True + assert "connection_id" not in required + + def test_member_without_connection_pauses(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + pause = _pause(_executor(user="bob"), _tool(cid, mode="member")) + assert pause["connection_required"]["status"] == "missing" + + def test_headless_run_is_denied_not_paused(self, pg_conn): + cid = _connection(pg_conn, status="disconnected") + with _service_db(pg_conn): + pause = _pause(_executor(headless=True), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + assert pause["error_type"] == "connection_required" + + def test_connected_tool_does_not_pause(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + assert _pause(_executor(), _tool(cid)) is None + + def test_mcp_tool_gets_connection_id_not_tokens(self, pg_conn): + cid = _connection(pg_conn, provider="mcp:https://m.example.com", auth_kind="mcp_oauth", + server_url="https://m.example.com", secrets={"tokens": {"access_token": "secret"}}) + tool = {**_tool(cid, name="mcp_tool"), "config": {"server_url": "https://m.example.com/mcp", + "auth_type": "oauth"}} + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + _executor()._get_or_load_tool(tool, "t1", "search") + config = manager.return_value.load_tool.call_args.kwargs["tool_config"] + assert config["connection_id"] == cid + assert "secret" not in str(config) + + def test_stored_connection_id_in_config_is_ignored(self, pg_conn): + """Only a resolved connection reaches the tool; a config value never does.""" + victim = _connection(pg_conn, user="victim", provider="mcp:https://m.example.com", auth_kind="mcp_oauth", + server_url="https://m.example.com", secrets={"tokens": {"access_token": "v"}}) + tool = {**_tool(None, name="mcp_tool"), "config": {"server_url": "https://m.example.com/mcp", + "auth_type": "oauth", "connection_id": victim}} + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + _executor()._get_or_load_tool(tool, "t1", "search") + config = manager.return_value.load_tool.call_args.kwargs["tool_config"] + assert "connection_id" not in config + + +def _telegram_tool(connection_id, *, user="alice", mode="owner"): + from docsgpt.agents.tools.telegram import TelegramTool + from docsgpt.connectors.service import _transform_actions + + return { + **_tool(connection_id, user=user, mode=mode), + "actions": _transform_actions(TelegramTool({}).get_actions_metadata()), + } + + +def _run_send(executor, tool, arguments): + with patch("docsgpt.agents.tool_executor.ToolActionParser") as parser, \ + patch("docsgpt.agents.tool_executor.ToolManager") as manager: + parser.return_value.parse_args.return_value = ("t1", "telegram_send_message", arguments) + gen = executor.execute({"t1": tool}, _call(), "OpenAILLM") + while True: + try: + next(gen) + except StopIteration: + break + return manager.return_value.load_tool.return_value.execute_action.call_args + + +class TestTelegramDefaultChat: + def test_the_connection_offers_a_default_chat_field(self): + from docsgpt.connectors import catalog + + fields = {f.key: f for f in catalog.get_definition("telegram").credential_fields} + chat = fields["chat_id"] + assert chat.secret is False and chat.required is False + assert chat.parameter == "chat_id" + assert chat.hint + assert chat.to_dict()["hint"] == chat.hint + + def test_the_model_is_not_asked_for_a_chat_the_connection_sets(self, pg_conn): + cid = _connection(pg_conn, secrets={"credentials": {"token": "t", "chat_id": "-1001"}}) + with _service_db(pg_conn): + functions = _executor().prepare_tools_for_llm({"t1": _telegram_tool(cid)}) + by_name = {f["function"]["name"]: f["function"]["parameters"] for f in functions} + assert "chat_id" not in by_name["telegram_send_message"]["properties"] + assert "chat_id" not in by_name["telegram_send_image"]["properties"] + + def test_without_a_default_chat_the_model_still_names_one(self, pg_conn): + cid = _connection(pg_conn, secrets={"credentials": {"token": "t"}}) + with _service_db(pg_conn): + functions = _executor().prepare_tools_for_llm({"t1": _telegram_tool(cid)}) + params = {f["function"]["name"]: f["function"]["parameters"] for f in functions} + assert "chat_id" in params["telegram_send_message"]["properties"] + + def test_the_default_chat_wins_over_what_the_model_sends(self, pg_conn): + cid = _connection(pg_conn, secrets={"credentials": {"token": "t", "chat_id": "-1001"}}) + with _service_db(pg_conn): + call = _run_send(_executor(), _telegram_tool(cid), {"text": "hi", "chat_id": "666"}) + assert call.kwargs == {"text": "hi", "chat_id": "-1001"} + + def test_each_member_uses_their_own_chat(self, pg_conn): + owner = _connection(pg_conn, secrets={"credentials": {"token": "t", "chat_id": "-1001"}}) + _connection(pg_conn, user="bob", secrets={"credentials": {"token": "b", "chat_id": "-2002"}}) + with _service_db(pg_conn): + call = _run_send(_executor(user="bob"), _telegram_tool(owner, mode="member"), {"text": "hi"}) + assert call.kwargs == {"text": "hi", "chat_id": "-2002"} + + +class TestAccountsTellApartForTheModel: + @staticmethod + def _two_bots(pg_conn, names): + tools = {} + for index, name in enumerate(names): + cid = _connection(pg_conn) + pg_conn.execute(text( + "UPDATE connector_sessions SET account_label = :l, account_name = :n WHERE id = CAST(:i AS uuid)" + ), {"l": f"…{index}abc", "n": name, "i": cid}) + tools[f"t{index}"] = {**_telegram_tool(cid), "id": f"tool-{index}"} + return tools + + def test_named_accounts_name_the_functions(self, pg_conn): + tools = self._two_bots(pg_conn, ["Alerts bot", "Ops: on-call!"]) + with _service_db(pg_conn): + executor = _executor() + functions = {f["function"]["name"]: f["function"] for f in executor.prepare_tools_for_llm(tools)} + assert {"telegram_send_message_alerts_bot", "telegram_send_message_ops_on_call"} <= set(functions) + assert "Alerts bot" in functions["telegram_send_message_alerts_bot"]["description"] + assert executor._name_to_tool["telegram_send_message_ops_on_call"] == ("t1", "telegram_send_message") + + def test_unnamed_accounts_use_their_labels(self, pg_conn): + tools = self._two_bots(pg_conn, [None, None]) + with _service_db(pg_conn): + names = {f["function"]["name"] for f in _executor().prepare_tools_for_llm(tools)} + assert {"telegram_send_message_0abc", "telegram_send_message_1abc"} <= names + + def test_different_services_are_named_after_the_service(self, pg_conn): + tools = {} + for index, (host, name) in enumerate((("a.example.com", "Wiki"), ("b.example.com", "Tracker"))): + cid = _connection(pg_conn, provider=f"mcp:https://{host}", auth_kind="mcp_oauth", + server_url=f"https://{host}", secrets={"tokens": {"access_token": "x"}}) + pg_conn.execute(text("UPDATE connector_sessions SET connector_key = 'custom_mcp', display_name = :n " + "WHERE id = CAST(:i AS uuid)"), {"n": name, "i": cid}) + tools[f"t{index}"] = {**_tool(cid, name="mcp_tool", tool_id=f"tool-{index}"), + "actions": [{"name": "search", "description": "Search", "active": True}]} + with _service_db(pg_conn): + functions = {f["function"]["name"]: f["function"] for f in _executor().prepare_tools_for_llm(tools)} + assert set(functions) == {"search_wiki", "search_tracker"} + assert functions["search_wiki"]["description"] == "Search (Wiki)" + + def test_names_stay_within_provider_limits(self, pg_conn): + tools = self._two_bots(pg_conn, ["x" * 80, "x" * 80]) + with _service_db(pg_conn): + names = [f["function"]["name"] for f in _executor().prepare_tools_for_llm(tools)] + assert len(names) == len(set(names)) + assert all(len(n) <= 64 and n.replace("_", "").replace("-", "").isalnum() for n in names) + + +class TestScheduledSync: + def test_connector_sources_with_a_connection_are_dispatched(self, pg_conn): + from docsgpt import worker + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", + secrets={"token_info": {"access_token": "a"}}) + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data) " + "VALUES ('alice', 'Drive', 'connector:file', 'weekly', CAST(:c AS uuid), '{\"provider\": \"google_drive\"}')" + ), {"c": cid}) + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency) VALUES ('alice', 'Old', 'connector:file', 'weekly')" + )) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), patch( + "docsgpt.api.user.tasks.sync_connector_source.delay" + ) as delay: + counts = worker.sync_worker(MagicMock(), "weekly") + assert delay.call_count == 1 + assert counts["sync_dispatched"] == 1 + assert counts["sync_skipped"] == 1 + + def test_paused_repository_is_skipped_until_reconnected(self, pg_conn): + """A GitHub or S3 source paused for reconnect is not retried (and failed) on every schedule.""" + from docsgpt import worker + + cid = _connection(pg_conn, provider="github", status="reconnect_needed", + secrets={"credentials": {"access_token": "revoked"}}) + pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data, metadata) " + "VALUES ('alice', 'acme/api', 'github', 'daily', CAST(:c AS uuid), '{\"repo_url\": \"acme/api\"}', " + "'{\"sync_state\": \"paused_reconnect\"}')" + ), {"c": cid}) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), patch.object(worker, "sync") as sync: + counts = worker.sync_worker(MagicMock(), "daily") + sync.assert_not_called() + assert counts["sync_skipped"] == 1 + + def test_paused_connection_is_not_synced(self, pg_conn): + from docsgpt import worker + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", status="reconnect_needed", + secrets={"token_info": {}}) + source = pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id) " + "VALUES ('alice', 'Drive', 'connector:file', 'weekly', CAST(:c AS uuid)) RETURNING id" + ), {"c": cid}).scalar() + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), patch.object(worker, "ingest_connector") as ingest: + result = worker.sync_connector_source(MagicMock(), str(source)) + assert result == {"status": "paused"} + ingest.assert_not_called() + + def test_disabled_connector_is_not_synced(self, pg_conn): + from docsgpt import worker + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", + secrets={"token_info": {"access_token": "a"}}) + source = pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data) " + "VALUES ('alice', 'Drive', 'connector:file', 'weekly', CAST(:c AS uuid), " + "'{\"provider\": \"google_drive\"}') RETURNING id" + ), {"c": cid}).scalar() + ConnectorPoliciesRepository(pg_conn).upsert("google_drive", enabled=False) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), patch.object(worker, "ingest_connector") as ingest: + result = worker.sync_connector_source(MagicMock(), str(source)) + assert result == {"status": "disabled"} + ingest.assert_not_called() + + def test_disabled_connector_gives_remote_sync_no_credentials(self, pg_conn): + from docsgpt import worker + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + cid = _connection(pg_conn, provider="s3", auth_kind="api_key", + secrets={"credentials": {"aws_access_key_id": "AKIA", "aws_secret_access_key": "s"}}) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield): + assert worker._with_connection_credentials({"bucket": "b"}, cid)["aws_access_key_id"] == "AKIA" + ConnectorPoliciesRepository(pg_conn).upsert("s3", enabled=False) + assert worker._with_connection_credentials({"bucket": "b"}, cid) is None + + def test_repository_url_gets_the_connections_token(self, pg_conn): + """A GitHub source's loader input is a plain URL, not JSON.""" + from docsgpt import worker + + cid = _connection(pg_conn, provider="github", auth_kind="api_key", + secrets={"credentials": {"access_token": "github_pat_x"}}) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield): + data = worker._with_connection_credentials("https://github.com/acme/private", cid) + as_json = worker._with_connection_credentials('{"search_queries": ["x"]}', cid) + assert data == {"url": "https://github.com/acme/private", "access_token": "github_pat_x"} + assert json.loads(as_json)["access_token"] == "github_pat_x" + + def test_oauth_connection_gives_its_current_access_token(self, pg_conn): + """A GitHub App sign-in keeps an OAuth token, refreshed before use.""" + from docsgpt import worker + + cid = _connection(pg_conn, provider="github", auth_kind="oauth", + secrets={"token_info": {"access_token": "ghu_fresh", "refresh_token": "ghr_x"}}) + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), _service_db(pg_conn), patch( + "docsgpt.connectors.service.get_valid_token_info", return_value={"access_token": "ghu_fresh"}, + ) as valid: + data = worker._with_connection_credentials({"repo_url": "acme/private"}, cid) + valid.assert_called_once_with(cid) + assert data == {"repo_url": "acme/private", "access_token": "ghu_fresh"} + + def test_rejected_token_flags_the_connection(self, pg_conn): + """A revoked token pauses the source for reconnect instead of failing every sync.""" + from docsgpt import worker + from docsgpt.connectors.service import ConnectionUnavailable + from docsgpt.parser.remote.github_loader import GitHubTokenRejected + + loader = MagicMock() + loader.load_data.side_effect = GitHubTokenRejected("revoked") + task = MagicMock() + task.request.retries = 1 + with patch.object(worker.RemoteCreator, "create_loader", return_value=loader), patch.object( + worker, "_with_connection_credentials", return_value={"url": "acme/r", "access_token": "t"}, + ), patch.object(worker, "publish_user_event"), patch( + "docsgpt.connectors.service.mark_reconnect_needed", + ) as flag: + with pytest.raises(ConnectionUnavailable): + worker.remote_worker(task, "acme/r", "repo", "alice", "github", connection_id="c-1") + flag.assert_called_once() + assert flag.call_args.args[0] == "c-1" + + def test_sync_runs_as_the_connection_without_a_browser(self, pg_conn): + from docsgpt import worker + + cid = _connection(pg_conn, provider="google_drive", auth_kind="oauth", + secrets={"token_info": {"access_token": "a"}}) + source = pg_conn.execute(text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id, remote_data) " + "VALUES ('alice', 'Drive', 'connector:file', 'daily', CAST(:c AS uuid), " + "'{\"provider\": \"google_drive\", \"folder_ids\": [\"f\"], \"recursive\": false}') RETURNING id" + ), {"c": cid}).scalar() + + @contextmanager + def _yield(): + yield pg_conn + + with patch.object(worker, "db_readonly", _yield), patch.object( + worker, "ingest_connector", return_value={"tokens": 1} + ) as ingest: + result = worker.sync_connector_source(MagicMock(), str(source)) + assert result["status"] == "success" + kwargs = ingest.call_args.kwargs + assert kwargs["connection_id"] == cid + assert kwargs["operation_mode"] == "sync" + assert kwargs["folder_ids"] == ["f"] and kwargs["recursive"] is False + assert "session_token" not in kwargs + + +@pytest.mark.parametrize("secret_key", ["token_info", "tokens", "client_info", "encrypted_credentials", + "client_secret", "refresh_token", "access_token"]) +def test_redaction_covers_connection_secrets(secret_key): + from docsgpt.storage.db.redaction import REDACTED, redact_secrets + + assert redact_secrets({secret_key: {"x": "y"}})[secret_key] == REDACTED + + +class TestMcpServerMismatch: + def test_connection_for_another_server_is_not_applied(self, pg_conn): + """A key stored for one MCP server is never sent to a tool now pointing at another.""" + from docsgpt.connectors import service + + cid = _connection(pg_conn, provider="custom_mcp", server_url="https://old.example.com", + secrets={"credentials": {"bearer_token": "old-server-secret"}}) + tool = {**_tool(cid, name="mcp_tool"), "config": {"server_url": "https://new.example.com/mcp", + "auth_type": "bearer"}} + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + with pytest.raises(service.ConnectionUnavailable): + _executor()._get_or_load_tool(tool, "t1", "search") + manager.return_value.load_tool.assert_not_called() + + @pytest.mark.parametrize("tool_name, provider, server_url", [ + # A service's key without a server of its own is not an MCP server's. + ("mcp_tool", "telegram", None), + ("mcp_tool", "ntfy", None), + # A custom server connection that names no server has nowhere to go. + ("mcp_tool", "custom_mcp", None), + # A tool only runs on a connection of the connector that provides it. + ("ntfy", "telegram", None), + ("telegram", "ntfy", None), + ("telegram", "custom_mcp", "https://new.example.com"), + ]) + def test_connection_of_another_connector_is_not_applied(self, pg_conn, tool_name, provider, server_url): + from docsgpt.connectors import service + + cid = _connection(pg_conn, provider=provider, server_url=server_url, + secrets={"credentials": {"token": "bot-token", "bearer_token": "bot-token"}}) + tool = {**_tool(cid, name=tool_name), "config": {"server_url": "https://new.example.com/mcp", + "auth_type": "bearer"}} + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + with pytest.raises(service.ConnectionUnavailable): + _executor()._get_or_load_tool(tool, "t1", "search") + manager.return_value.load_tool.assert_not_called() + + def test_legacy_mcp_connection_is_still_applied_to_its_server(self, pg_conn): + """Rows from before connector keys are named from their ``mcp:`` provider.""" + cid = _connection(pg_conn, provider="mcp:https://m.example.com", auth_kind="mcp_oauth", + server_url=None, secrets={"tokens": {"access_token": "x"}}) + pg_conn.execute(text("UPDATE connector_sessions SET connector_key = NULL WHERE id = CAST(:i AS uuid)"), + {"i": cid}) + tool = {**_tool(cid, name="mcp_tool"), "config": {"server_url": "https://m.example.com/mcp", + "auth_type": "oauth"}} + with _service_db(pg_conn), patch("docsgpt.agents.tool_executor.ToolManager") as manager: + _executor()._get_or_load_tool(tool, "t1", "search") + assert manager.return_value.load_tool.call_args.kwargs["tool_config"]["connection_id"] == cid + + def test_save_keeps_previous_connection_only_for_the_same_server(self, pg_conn): + from docsgpt.api.user.tools.mcp import _previous_connection + + @contextmanager + def _yield(): + yield pg_conn + + cid = _connection(pg_conn, provider="custom_mcp", server_url="https://old.example.com") + existing = {"connection_id": cid} + same = {"server_url": "https://old.example.com/mcp", "auth_type": "bearer"} + with patch("docsgpt.api.user.tools.mcp.db_readonly", _yield): + assert _previous_connection(existing, same, "alice") == cid + assert _previous_connection(existing, {**same, "server_url": "https://new.example.com/mcp"}, "alice") is None + assert _previous_connection(None, same, "alice") is None + # Someone else's connection, or one that signs in another way, is not kept. + assert _previous_connection(existing, same, "bob") is None + assert _previous_connection(existing, {**same, "auth_type": "oauth"}, "alice") is None + + +class TestExternalApiCallers: + """An agent called with its API key runs as the owner, and nobody can approve there.""" + + def _external(self, allowlist=None): + from docsgpt.agents.tool_executor import ToolExecutor + + return ToolExecutor(user="alice", external_caller=True, api_write_allowlist=allowlist) + + def test_owner_account_write_is_denied(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + pause = _pause(self._external(), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + assert pause["error_type"] == "tool_not_allowed" + assert "Access details" in pause["deny_reason"] + + def test_even_always_allow_writes_are_denied(self, pg_conn): + cid = _connection(pg_conn) + tool = _tool(cid) + tool["actions"][0]["require_approval"] = False + with _service_db(pg_conn): + assert _pause(self._external(), tool)["pause_type"] == "headless_denied" + + def test_allowlisted_write_runs(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + pause = _pause(self._external(["tool-1:telegram_send_message"]), _tool(cid)) + assert pause is None + + def test_allowlist_does_not_cover_other_actions(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + pause = _pause(self._external(["tool-1:telegram_send_image"]), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + + def test_missing_connection_is_denied_not_paused(self, pg_conn): + """The widget cannot show a Connect card.""" + cid = _connection(pg_conn, status="reconnect_needed") + with _service_db(pg_conn): + pause = _pause(self._external(), _tool(cid)) + assert pause["pause_type"] == "headless_denied" + assert pause["error_type"] == "connection_required" + + def test_the_owner_in_the_app_is_not_external(self, pg_conn): + cid = _connection(pg_conn) + with _service_db(pg_conn): + assert _pause(_executor(), _tool(cid)) is None + + +class TestExternalCallerDetection: + def test_api_key_request_from_someone_else_is_external(self): + from docsgpt.api.answer.services.stream_processor import is_external_api_caller + + assert is_external_api_caller({"api_key": "k"}, {"sub": "visitor"}, "alice") is True + assert is_external_api_caller({"api_key": "k"}, None, "alice") is True + + def test_owner_previewing_their_agent_is_not_external(self): + from docsgpt.api.answer.services.stream_processor import is_external_api_caller + + assert is_external_api_caller({"api_key": "k"}, {"sub": "alice"}, "alice") is False + assert is_external_api_caller({}, {"sub": "visitor"}, "alice") is False + + +class TestApiWriteAllowlistConfig: + def test_accepts_tool_action_pairs(self): + from docsgpt.guardrails.config import AgentConfig + + config = AgentConfig.model_validate({"api_write_allowlist": ["tool-1:telegram_send_message"]}) + assert config.api_write_allowlist == ["tool-1:telegram_send_message"] + + def test_rejects_malformed_entries(self): + from docsgpt.guardrails.config import AgentConfig + + with pytest.raises(Exception): + AgentConfig.model_validate({"api_write_allowlist": ["no-action-part"]}) + + def test_old_configs_still_parse(self): + from docsgpt.guardrails.config import AgentConfig + + assert AgentConfig.parse({"guardrails": {}}).api_write_allowlist == [] + + +class TestAllowlistOwnership: + def test_team_editor_cannot_change_the_allowlist(self): + from docsgpt.api.user.agents.routes import keep_owner_only_config + + existing = {"config": {"api_write_allowlist": ["t:a"]}} + sent = {"guardrails": {"controls": []}, "api_write_allowlist": ["t:a", "t:b"]} + assert keep_owner_only_config(sent, existing, True)["api_write_allowlist"] == ["t:a"] + assert keep_owner_only_config(sent, existing, False)["api_write_allowlist"] == ["t:a", "t:b"] + assert keep_owner_only_config({}, {"config": None}, True) == {"api_write_allowlist": []} diff --git a/tests/connectors/test_service_credentials.py b/tests/connectors/test_service_credentials.py new file mode 100644 index 00000000..721a3d41 --- /dev/null +++ b/tests/connectors/test_service_credentials.py @@ -0,0 +1,604 @@ +"""Tests for connection credentials: storage, refresh, failure handling.""" + +from __future__ import annotations + +import json +import threading +import time +from contextlib import contextmanager +from unittest.mock import MagicMock, patch + +import pytest +from sqlalchemy import text + +from docsgpt.connectors import service +from docsgpt.security.encryption import encrypt_json + + +def _patch_service_db(conn): + @contextmanager + def _yield(): + yield conn + + return patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield) + + +def _connection(conn, *, user="alice", provider="google_drive", secrets=None, status="connected", **extra) -> str: + cols = { + "user_id": user, + "provider": provider, + "connector_key": provider, + "auth_kind": extra.pop("auth_kind", "oauth"), + "status": status, + "account_label": extra.pop("account_label", f"{user}@example.com"), + "encrypted_credentials": encrypt_json(secrets, user) if secrets is not None else None, + **extra, + } + names = ", ".join(cols) + values = ", ".join(f":{k}" for k in cols) + return str( + conn.execute( + text(f"INSERT INTO connector_sessions ({names}) VALUES ({values}) RETURNING id"), cols, + ).scalar() + ) + + +def _source(conn, connection_id, user="alice") -> str: + return str( + conn.execute( + text( + "INSERT INTO sources (user_id, name, type, connection_id) " + "VALUES (:u, 'Docs', 'connector:file', CAST(:c AS uuid)) RETURNING id" + ), + {"u": user, "c": connection_id}, + ).scalar() + ) + + +def _row(conn, connection_id): + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + return ConnectorSessionsRepository(conn).get(connection_id) + + +class _FakeAuth: + """A provider whose refresh rotates the refresh token on every call.""" + + def __init__(self, *, expired=True, fail=None, delay=0.0): + self.expired = expired + self.fail = fail + self.delay = delay + self.calls = [] + self._lock = threading.Lock() + + def is_token_expired(self, token_info): + return self.expired and token_info.get("access_token") == "old-at" + + def refresh_access_token(self, refresh_token): + with self._lock: + self.calls.append(refresh_token) + n = len(self.calls) + if self.delay: + time.sleep(self.delay) + if self.fail: + raise self.fail + if refresh_token != "rt-0": + # A reused rotating refresh token is rejected, like Microsoft's. + raise ValueError("invalid_grant: refresh token already used") + return {"access_token": f"new-at-{n}", "refresh_token": f"rt-{n}", "expiry": None} + + def sanitize_token_info(self, token_info, **extra): + return {k: token_info.get(k) for k in ("access_token", "refresh_token", "expiry", "cloud_id")} + + +def _auth(fake): + return patch("docsgpt.parser.connectors.connector_creator.ConnectorCreator.create_auth", return_value=fake) + + +class TestSecrets: + def test_write_and_read(self, pg_conn): + cid = _connection(pg_conn, secrets=None, status="pending") + service.write_secrets(pg_conn, _row(pg_conn, cid), {"token_info": {"access_token": "a", "refresh_token": "r"}}) + row = _row(pg_conn, cid) + assert row["token_info"] is None + assert row["has_refresh_token"] is True + assert "access_token" not in json.dumps(row["session_data"]) + assert service.read_secrets(row) == {"token_info": {"access_token": "a", "refresh_token": "r"}} + + def test_legacy_plaintext_is_read(self, pg_conn): + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + row = ConnectorSessionsRepository(pg_conn).upsert( + "alice", "share_point", token_info={"access_token": "p"}, status="authorized", + ) + assert service.read_secrets(row)["token_info"]["access_token"] == "p" + + def test_decrypt_failure_flags_reconnect(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "a"}}) + pg_conn.execute( + text("UPDATE connector_sessions SET encrypted_credentials = 'v2:ffffffff:AAAA' WHERE id = CAST(:i AS uuid)"), + {"i": cid}, + ) + with _patch_service_db(pg_conn), patch("docsgpt.events.publisher.publish_user_event"): + assert service.load_secrets(_row(pg_conn, cid)) is None + row = _row(pg_conn, cid) + assert row["status"] == "reconnect_needed" + assert row["last_error"] == service.DECRYPT_ERROR + + +class TestGetValidTokenInfo: + def test_returns_unexpired_token(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "fresh", "refresh_token": "rt-0"}}) + fake = _FakeAuth(expired=True) + with _patch_service_db(pg_conn), _auth(fake): + info = service.get_valid_token_info(cid) + assert info["access_token"] == "fresh" + assert fake.calls == [] + + def test_refreshes_and_persists_rotated_token(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "old-at", "refresh_token": "rt-0"}}) + fake = _FakeAuth() + with _patch_service_db(pg_conn), _auth(fake): + info = service.get_valid_token_info(cid) + assert info["access_token"] == "new-at-1" + stored = service.read_secrets(_row(pg_conn, cid))["token_info"] + assert stored["refresh_token"] == "rt-1" + + def test_rejected_access_token_forces_refresh(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "fresh", "refresh_token": "rt-0"}}) + fake = _FakeAuth(expired=False) + with _patch_service_db(pg_conn), _auth(fake): + info = service.get_valid_token_info(cid, rejected_access_token="fresh") + assert info["access_token"] == "new-at-1" + + def test_rejected_token_already_replaced_is_not_refreshed_again(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "newer", "refresh_token": "rt-0"}}) + fake = _FakeAuth(expired=False) + with _patch_service_db(pg_conn), _auth(fake): + info = service.get_valid_token_info(cid, rejected_access_token="stale") + assert info["access_token"] == "newer" + assert fake.calls == [] + + def test_revoked_grant_pauses_sources_and_notifies(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "old-at", "refresh_token": "rt-0"}}) + source = _source(pg_conn, cid) + fake = _FakeAuth(fail=ValueError("Error refreshing token: invalid_grant")) + with _patch_service_db(pg_conn), _auth(fake), patch( + "docsgpt.events.publisher.publish_user_event" + ) as publish: + with pytest.raises(service.ConnectionUnavailable): + service.get_valid_token_info(cid) + assert _row(pg_conn, cid)["status"] == "reconnect_needed" + meta = pg_conn.execute( + text("SELECT metadata FROM sources WHERE id = CAST(:i AS uuid)"), {"i": source} + ).scalar() + assert meta["sync_state"] == "paused_reconnect" + publish.assert_called_once() + assert publish.call_args.args[1] == "connection.reconnect_needed" + # The toast says what stopped working. + assert publish.call_args.args[2]["source_count"] == 1 + assert publish.call_args.args[2]["tool_count"] == 0 + # No account data or secrets in the event. + assert "alice@example.com" not in json.dumps(publish.call_args.args[2]) + + def test_second_failure_does_not_notify_again(self, pg_conn): + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "old-at", "refresh_token": "rt-0"}}) + fake = _FakeAuth(fail=ValueError("invalid_grant")) + with _patch_service_db(pg_conn), _auth(fake), patch( + "docsgpt.events.publisher.publish_user_event" + ) as publish: + for _ in range(2): + with pytest.raises(service.ConnectionUnavailable): + service.get_valid_token_info(cid) + assert publish.call_count == 1 + + def test_transient_failure_keeps_status(self, pg_conn): + import requests + + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "old-at", "refresh_token": "rt-0"}}) + fake = _FakeAuth(fail=requests.exceptions.ConnectionError("network down")) + with _patch_service_db(pg_conn), _auth(fake): + with pytest.raises(service.TransientConnectionError): + service.get_valid_token_info(cid) + assert _row(pg_conn, cid)["status"] == "connected" + + def test_disconnected_connection_is_unavailable(self, pg_conn): + cid = _connection(pg_conn, secrets={}, status="disconnected") + with _patch_service_db(pg_conn): + with pytest.raises(service.ConnectionUnavailable) as exc: + service.get_valid_token_info(cid) + assert exc.value.status == "disconnected" + + def test_concurrent_refresh_spends_the_refresh_token_once(self, pg_engine): + """Two workers with an expiring rotating token: one refresh, both succeed.""" + with pg_engine.begin() as conn: + cid = _connection(conn, secrets={"token_info": {"access_token": "old-at", "refresh_token": "rt-0"}}) + + @contextmanager + def _session(): + with pg_engine.begin() as conn: + yield conn + + fake = _FakeAuth(delay=0.3) + results, errors = [], [] + + def worker(): + try: + results.append(service.get_valid_token_info(cid)["access_token"]) + except Exception as exc: # pragma: no cover - reported below + errors.append(exc) + + with patch.multiple("docsgpt.connectors.service", db_session=_session, db_readonly=_session), _auth(fake): + threads = [threading.Thread(target=worker) for _ in range(2)] + for thread in threads: + thread.start() + for thread in threads: + thread.join() + assert errors == [] + assert fake.calls == ["rt-0"] + assert results == ["new-at-1", "new-at-1"] + + +class TestApiKeyConnections: + def test_create_and_reuse(self, pg_conn): + from docsgpt.connectors import catalog + + telegram = catalog.get_definition("telegram") + row, created = service.create_api_key_connection(pg_conn, "alice", telegram, {"token": "123456:ABCDEFG"}) + assert created is True + assert row["account_label"] == "…DEFG" + assert row["status"] == "connected" + again, created = service.create_api_key_connection(pg_conn, "alice", telegram, {"token": "123456:ABCDEFG"}) + assert created is False and again["id"] == row["id"] + assert service.get_credentials(again) == {"token": "123456:ABCDEFG"} + + def test_different_keys_with_the_same_hint_stay_apart(self, pg_conn): + """Two keys ending in the same four characters are two accounts, not one.""" + from docsgpt.connectors import catalog + + postgres = catalog.get_definition("postgres") + first, _ = service.create_api_key_connection( + pg_conn, "alice", postgres, {"token": "postgresql://ro@db-a/app"}, + ) + second, created = service.create_api_key_connection( + pg_conn, "alice", postgres, {"token": "postgresql://rw@db-b/app"}, + ) + assert created is True and second["id"] != first["id"] + assert second["account_label"] != first["account_label"] + assert service.get_credentials(_row(pg_conn, str(first["id"]))) == {"token": "postgresql://ro@db-a/app"} + again, created = service.create_api_key_connection( + pg_conn, "alice", postgres, {"token": "postgresql://rw@db-b/app"}, + ) + assert created is False and again["id"] == second["id"] + + def test_same_label_different_key_does_not_overwrite(self, pg_conn): + from docsgpt.connectors import catalog + + brave = catalog.get_definition("brave") + first, _ = service.create_api_key_connection(pg_conn, "alice", brave, {"token": "key-one-111111"}, label="Team") + second, created = service.create_api_key_connection( + pg_conn, "alice", brave, {"token": "key-two-222222"}, label="Team", + ) + assert created is True and second["id"] != first["id"] + assert service.get_credentials(_row(pg_conn, str(first["id"]))) == {"token": "key-one-111111"} + + def test_missing_field(self, pg_conn): + from docsgpt.connectors import catalog + + with pytest.raises(ValueError): + service.create_api_key_connection(pg_conn, "alice", catalog.get_definition("telegram"), {}) + + def test_default_key_refused_with_auth(self, pg_conn, monkeypatch): + from docsgpt.connectors import catalog + from docsgpt.core.settings import settings + from docsgpt.security.encryption import DEFAULT_ENCRYPTION_KEY + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", DEFAULT_ENCRYPTION_KEY) + monkeypatch.setattr(settings, "AUTH_TYPE", "session_jwt") + with pytest.raises(service.EncryptionKeyNotConfigured): + service.create_api_key_connection( + pg_conn, "alice", catalog.get_definition("brave"), {"token": "secret-key-1234"}, + ) + + def test_disabled_connector_refused_before_other_errors(self, pg_conn, monkeypatch): + from docsgpt.connectors import catalog + from docsgpt.core.settings import settings + from docsgpt.security.encryption import DEFAULT_ENCRYPTION_KEY + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", DEFAULT_ENCRYPTION_KEY) + monkeypatch.setattr(settings, "AUTH_TYPE", "session_jwt") + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + telegram = catalog.get_definition("telegram") + # Callers fall back to a legacy per-tool secret on these other errors. + for credentials in ({"token": "123456:ABCDEFG"}, {}): + with pytest.raises(service.ConnectorDisabled): + service.create_api_key_connection(pg_conn, "alice", telegram, credentials) + + def test_default_key_allowed_single_user(self, pg_conn, monkeypatch): + from docsgpt.connectors import catalog + from docsgpt.core.settings import settings + from docsgpt.security.encryption import DEFAULT_ENCRYPTION_KEY + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", DEFAULT_ENCRYPTION_KEY) + monkeypatch.setattr(settings, "AUTH_TYPE", None) + row, _ = service.create_api_key_connection( + pg_conn, "local", catalog.get_definition("brave"), {"token": "secret-key-1234"}, + ) + assert row["encrypted_credentials"].startswith("v2:") + + def test_flagged_connection_credentials_unavailable(self, pg_conn): + cid = _connection( + pg_conn, provider="telegram", auth_kind="api_key", status="reconnect_needed", + secrets={"credentials": {"token": "t"}}, + ) + with pytest.raises(service.ConnectionUnavailable): + service.get_credentials(_row(pg_conn, cid)) + + +class TestOAuthCompletion: + def test_new_account(self, pg_conn): + state_row = service.begin_oauth(pg_conn, "alice", "google_drive") + assert state_row["status"] == "pending" + row = service.complete_oauth(pg_conn, state_row, "google_drive", {"access_token": "a"}, "a@example.com") + assert row["id"] == state_row["id"] + assert row["account_label"] == "a@example.com" + assert row["status"] == "connected" + assert row["session_token"] + + def test_same_account_updates_existing_connection(self, pg_conn): + first = service.complete_oauth( + pg_conn, service.begin_oauth(pg_conn, "alice", "google_drive"), "google_drive", + {"access_token": "a"}, "a@example.com", + ) + source = _source(pg_conn, str(first["id"])) + pg_conn.execute( + text("UPDATE connector_sessions SET status = 'reconnect_needed' WHERE id = CAST(:i AS uuid)"), + {"i": str(first["id"])}, + ) + pg_conn.execute( + text("UPDATE sources SET metadata = '{\"sync_state\": \"paused_reconnect\"}' WHERE id = CAST(:i AS uuid)"), + {"i": source}, + ) + pending = service.begin_oauth(pg_conn, "alice", "google_drive") + again = service.complete_oauth(pg_conn, pending, "google_drive", {"access_token": "b"}, "a@example.com") + assert again["id"] == first["id"] + assert again["status"] == "connected" + assert _row(pg_conn, str(pending["id"])) is None + meta = pg_conn.execute( + text("SELECT metadata FROM sources WHERE id = CAST(:i AS uuid)"), {"i": source} + ).scalar() + assert "sync_state" not in meta + + def test_second_account_is_a_second_connection(self, pg_conn): + first = service.complete_oauth( + pg_conn, service.begin_oauth(pg_conn, "alice", "google_drive"), "google_drive", + {"access_token": "a"}, "a@example.com", + ) + second = service.complete_oauth( + pg_conn, service.begin_oauth(pg_conn, "alice", "google_drive"), "google_drive", + {"access_token": "b"}, "b@example.com", + ) + assert first["id"] != second["id"] + assert len(service.list_connections(pg_conn, "alice")) == 2 + + def test_reconnect_as_the_same_account_heals_the_connection(self, pg_conn): + cid = _connection(pg_conn, status="reconnect_needed", secrets={"token_info": {"access_token": "old"}}, + account_label="a@example.com") + state_row = service.begin_oauth(pg_conn, "alice", "google_drive", cid) + row = service.complete_oauth(pg_conn, state_row, "google_drive", {"access_token": "new"}, "a@example.com") + assert str(row["id"]) == cid + assert row["status"] == "connected" + + def test_reconnect_as_another_account_leaves_the_connection_alone(self, pg_conn): + cid = _connection(pg_conn, status="reconnect_needed", secrets={"token_info": {"access_token": "old"}}, + account_label="a@example.com") + source = _source(pg_conn, cid) + pg_conn.execute( + text("UPDATE sources SET metadata = '{\"sync_state\": \"paused_reconnect\"}' WHERE id = CAST(:i AS uuid)"), + {"i": source}, + ) + state_row = service.begin_oauth(pg_conn, "alice", "google_drive", cid) + row = service.complete_oauth(pg_conn, state_row, "google_drive", {"access_token": "b"}, "b@example.com") + assert str(row["id"]) != cid + assert row["account_label"] == "b@example.com" + assert row["status"] == "connected" + original = _row(pg_conn, cid) + assert original["account_label"] == "a@example.com" + assert original["user_email"] != "b@example.com" + assert original["status"] == "reconnect_needed" + assert service.read_secrets(original)["token_info"]["access_token"] == "old" + meta = pg_conn.execute( + text("SELECT metadata FROM sources WHERE id = CAST(:i AS uuid)"), {"i": source} + ).scalar() + assert meta["sync_state"] == "paused_reconnect" + + def test_reconnect_must_name_own_connection(self, pg_conn): + cid = _connection(pg_conn, user="bob", secrets={}) + with pytest.raises(service.ConnectionUnavailable): + service.begin_oauth(pg_conn, "alice", "google_drive", cid) + + +class TestReencrypt: + def test_rewrites_rows_on_previous_key(self, pg_conn, monkeypatch): + from docsgpt.core.settings import settings + from docsgpt.security.encryption import current_key_id, envelope_key_id + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "old-key") + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "a"}}) + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "new-key") + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY_PREVIOUS", "old-key") + with _patch_service_db(pg_conn): + counts = service.reencrypt_all() + assert counts == {"rewritten": 1, "current": 0, "failed": 0} + row = _row(pg_conn, cid) + assert envelope_key_id(row["encrypted_credentials"]) == current_key_id() + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY_PREVIOUS", None) + assert service.read_secrets(row)["token_info"]["access_token"] == "a" + + def test_unreadable_rows_flagged(self, pg_conn, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "lost-key") + cid = _connection(pg_conn, secrets={"token_info": {"access_token": "a"}}) + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "new-key") + with _patch_service_db(pg_conn), patch("docsgpt.events.publisher.publish_user_event"): + counts = service.reencrypt_all() + assert counts["failed"] == 1 + assert _row(pg_conn, cid)["status"] == "reconnect_needed" + + +class TestRemove: + def test_keep_sources_delete_tools(self, pg_conn): + cid = _connection(pg_conn, provider="telegram", auth_kind="api_key", secrets={"credentials": {"token": "t"}}) + source = _source(pg_conn, cid) + pg_conn.execute( + text( + "INSERT INTO user_tools (user_id, name, connection_id) VALUES ('alice', 'telegram', CAST(:c AS uuid))" + ), + {"c": cid}, + ) + with patch("docsgpt.connectors.service.revoke_at_provider") as revoke: + to_delete = service.remove_connection(pg_conn, _row(pg_conn, cid)) + assert to_delete == [] + revoke.assert_called_once() + assert _row(pg_conn, cid) is None + assert pg_conn.execute(text("SELECT count(*) FROM user_tools WHERE user_id = 'alice'")).scalar() == 0 + kept = pg_conn.execute( + text("SELECT connection_id, sync_frequency FROM sources WHERE id = CAST(:i AS uuid)"), {"i": source} + ).one() + assert kept.connection_id is None and kept.sync_frequency == "never" + + def test_deleted_tools_leave_no_sharing_rows(self, pg_conn): + from docsgpt.api.user.resource_access import set_settings + from docsgpt.storage.db.repositories.user_tool_preferences import UserToolPreferencesRepository + + cid = _connection(pg_conn, provider="telegram", auth_kind="api_key", secrets={"credentials": {"token": "t"}}) + tool_id = str(pg_conn.execute( + text( + "INSERT INTO user_tools (user_id, name, connection_id) " + "VALUES ('alice', 'telegram', CAST(:c AS uuid)) RETURNING id" + ), + {"c": cid}, + ).scalar()) + set_settings(pg_conn, "tool", tool_id, {"editors_can_share": True}, "alice") + UserToolPreferencesRepository(pg_conn).set_in_chat("bob", tool_id, True) + with patch("docsgpt.connectors.service.revoke_at_provider"): + service.remove_connection(pg_conn, _row(pg_conn, cid)) + for table, column in (("resource_share_settings", "resource_id"), ("user_tool_preferences", "tool_id")): + count = pg_conn.execute( + text(f"SELECT count(*) FROM {table} WHERE {column} = CAST(:i AS uuid)"), {"i": tool_id} + ).scalar() + assert count == 0, table + + def test_delete_sources_returns_them(self, pg_conn): + cid = _connection(pg_conn, secrets={}) + source = _source(pg_conn, cid) + with patch("docsgpt.connectors.service.revoke_at_provider"): + to_delete = service.remove_connection(pg_conn, _row(pg_conn, cid), sources="delete") + assert [str(s["id"]) for s in to_delete] == [source] + + +class TestRevoke: + def test_google_revocation_posts_token(self): + response = MagicMock(status_code=200) + with patch("requests.post", return_value=response) as post: + ok = service.revoke_at_provider( + {"provider": "google_drive"}, {"token_info": {"refresh_token": "rt", "access_token": "at"}}, + ) + assert ok + assert post.call_args.kwargs["data"] == {"token": "rt"} + + def test_other_providers_are_local_only(self): + with patch("requests.post") as post: + assert not service.revoke_at_provider({"provider": "share_point"}, {"token_info": {"access_token": "a"}}) + post.assert_not_called() + + def test_failure_never_raises(self): + import requests + + with patch("requests.post", side_effect=requests.exceptions.Timeout()): + assert not service.revoke_at_provider({"provider": "google_drive"}, {"token_info": {"access_token": "a"}}) + + +class TestMcpConnectionScope: + def _mcp(self, conn, user="alice", base="https://mcp.notion.com"): + return _connection( + conn, user=user, provider=f"mcp:{base}", auth_kind="mcp_oauth", server_url=base, + secrets={"tokens": {"access_token": f"{user}-mcp-token"}}, + ) + + def test_reads_tokens_of_the_named_connection(self, pg_conn): + cid = self._mcp(pg_conn) + with _patch_service_db(pg_conn): + data = service.read_mcp_secrets("alice", "https://mcp.notion.com", cid) + assert data["tokens"]["access_token"] == "alice-mcp-token" + + def test_connection_for_another_server_yields_nothing(self, pg_conn): + """A connection id never sends its tokens to a different server.""" + cid = self._mcp(pg_conn) + with _patch_service_db(pg_conn): + assert service.read_mcp_secrets("alice", "https://attacker.example", cid) == {} + + def test_writes_never_land_on_another_servers_connection(self, pg_conn): + cid = self._mcp(pg_conn) + with _patch_service_db(pg_conn): + with pytest.raises(service.ConnectionUnavailable): + service.update_mcp_secrets( + "alice", "https://attacker.example", {"tokens": {"access_token": "planted"}}, connection_id=cid, + ) + assert service.read_mcp_secrets("alice", "https://mcp.notion.com", cid)["tokens"]["access_token"] == ( + "alice-mcp-token" + ) + assert service.read_mcp_secrets("alice", "https://attacker.example") == {} + + def test_a_removed_connection_is_not_brought_back_by_a_late_token_write(self, pg_conn): + """A client or sync still running when its connection is removed may + renew the tokens afterwards; that must not re-create the connection.""" + cid = self._mcp(pg_conn) + with _patch_service_db(pg_conn): + service.remove_connection(pg_conn, service.ConnectorSessionsRepository(pg_conn).get(cid)) + with pytest.raises(service.ConnectionUnavailable): + service.update_mcp_secrets( + "alice", "https://mcp.notion.com", {"tokens": {"access_token": "renewed"}}, connection_id=cid, + ) + assert service.read_mcp_secrets("alice", "https://mcp.notion.com") == {} + + def test_removing_a_connection_forgets_its_cached_mcp_clients(self, pg_conn): + import docsgpt.api.user # noqa: F401 (loads mcp_tool without the circular import) + from docsgpt.agents.tools import mcp_tool + + cid = self._mcp(pg_conn) + mcp_tool._mcp_clients_cache.update({ + f"https://mcp.notion.com/mcp#http#oauth:{cid}:DocsGPT:none:cb": {"client": object(), "created_at": 0}, + "https://mcp.notion.com/mcp#http#oauth:alice:DocsGPT:none:cb": {"client": object(), "created_at": 0}, + "https://mcp.notion.com/mcp#http#oauth:bob:DocsGPT:none:cb": {"client": object(), "created_at": 0}, + }) + try: + with _patch_service_db(pg_conn): + service.remove_connection(pg_conn, service.ConnectorSessionsRepository(pg_conn).get(cid)) + keys = [k for k in mcp_tool._mcp_clients_cache if "mcp.notion.com" in k] + assert keys == ["https://mcp.notion.com/mcp#http#oauth:bob:DocsGPT:none:cb"] + finally: + mcp_tool._mcp_clients_cache.clear() + + def test_mcp_routes_drop_client_supplied_connection_id(self): + """Only the tool executor may pick the connection whose tokens a tool uses.""" + from docsgpt.api.user.tools.mcp import _sanitize_mcp_transport + + config = {"transport_type": "http", "connection_id": "someone-elses-connection"} + _sanitize_mcp_transport(config) + assert "connection_id" not in config + + def test_mcp_policy_check_fails_closed_when_unreadable(self): + """A server whose admin switch cannot be read is not contacted.""" + from flask import Flask + + from docsgpt.api.user.tools import mcp as mcp_routes + + def _broken(): + raise RuntimeError("database is down") + + with Flask(__name__).app_context(), patch.object(mcp_routes, "db_readonly", _broken): + resp = mcp_routes._mcp_policy_error({"server_url": "https://mcp.linear.app/mcp"}) + assert resp is not None and resp.status_code == 503 diff --git a/tests/connectors/test_service_read.py b/tests/connectors/test_service_read.py new file mode 100644 index 00000000..9e3af7bc --- /dev/null +++ b/tests/connectors/test_service_read.py @@ -0,0 +1,306 @@ +"""Tests for the read side of the connection service and its API.""" + +from __future__ import annotations + +import json +from contextlib import contextmanager +from unittest.mock import patch + +import pytest +from flask import Flask +from sqlalchemy import text + +from docsgpt.connectors import service + + +def _session(conn, user="alice", provider="google_drive", **cols) -> str: + cols = {"status": "authorized", "user_email": f"{user}@example.com", **cols} + casts = {"token_info": "jsonb", "session_data": "jsonb"} + names = ", ".join(["user_id", "provider", *cols]) + values = ", ".join( + [":user_id", ":provider", *[f"CAST(:{k} AS {casts[k]})" if k in casts else f":{k}" for k in cols]] + ) + return str( + conn.execute( + text(f"INSERT INTO connector_sessions ({names}) VALUES ({values}) RETURNING id"), + {"user_id": user, "provider": provider, **cols}, + ).scalar() + ) + + +def _source(conn, connection_id, user="alice", name="Handbook") -> str: + return str( + conn.execute( + text( + "INSERT INTO sources (user_id, name, type, sync_frequency, connection_id) " + "VALUES (:u, :n, 'connector:file', 'weekly', CAST(:c AS uuid)) RETURNING id" + ), + {"u": user, "n": name, "c": connection_id}, + ).scalar() + ) + + +def _tool(conn, connection_id, user="alice") -> str: + actions = [ + {"name": "search_pages", "description": "Search", "active": True}, + {"name": "create_page", "description": "Create", "active": True, "require_approval": True}, + ] + return str( + conn.execute( + text( + "INSERT INTO user_tools (user_id, name, display_name, actions, connection_id) " + "VALUES (:u, 'mcp_tool', 'Notion', CAST(:a AS jsonb), CAST(:c AS uuid)) RETURNING id" + ), + {"u": user, "a": json.dumps(actions), "c": connection_id}, + ).scalar() + ) + + +class TestNormalizeStatus: + @pytest.mark.parametrize( + "row,expected", + [ + ({"status": "authorized"}, "connected"), + ({"status": "reconnect_needed"}, "reconnect_needed"), + ({"status": "pending"}, "pending"), + ({"status": "pending", "token_info": {"access_token": "x"}}, "connected"), + ({"status": None, "session_data": {"tokens": {"access_token": "x"}}}, "connected"), + ({"status": None, "session_data": {"client_info": {}}}, "pending"), + ({"status": None, "encrypted_credentials": "v2:..."}, "connected"), + ], + ) + def test_statuses(self, row, expected): + assert service.normalize_status(row) == expected + + def test_worst_status(self): + assert service.worst_status(["connected", "reconnect_needed"]) == "reconnect_needed" + assert service.worst_status([]) is None + + +class TestListing: + def test_lists_finished_connections_with_counts(self, pg_conn): + drive = _session(pg_conn) + _session(pg_conn, provider="confluence", status="pending", user_email=None) + _source(pg_conn, drive) + _source(pg_conn, drive, name="Wiki") + connections = service.list_connections(pg_conn, "alice") + assert [c["connector_key"] for c in connections] == ["google_drive"] + only = connections[0] + assert only["name"] == "Google Drive" + assert only["account_label"] == "alice@example.com" + assert only["status"] == "connected" + assert only["source_count"] == 2 + assert "token_info" not in only and "session_token" not in only + + def test_other_users_rows_hidden(self, pg_conn): + _session(pg_conn, user="bob") + assert service.list_connections(pg_conn, "alice") == [] + + def test_detail_lists_sources_and_tools(self, pg_conn): + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + drive = _session(pg_conn, status="reconnect_needed") + _source(pg_conn, drive) + _tool(pg_conn, drive) + row = ConnectorSessionsRepository(pg_conn).get(drive) + detail = service.connection_detail(pg_conn, row) + assert detail["sources"][0]["sync_state"] == "paused_reconnect" + actions = {a["name"]: a for a in detail["tools"][0]["actions"]} + assert actions["search_pages"] == { + "name": "search_pages", "description": "Search", "access": "read", "permission": "always", + "parameters": [], + } + assert actions["create_page"]["access"] == "write" + assert actions["create_page"]["permission"] == "ask" + + +class TestCatalogForUser: + def test_states(self, pg_conn, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", "id") + monkeypatch.setattr(settings, "GOOGLE_CLIENT_SECRET", "secret") + monkeypatch.setattr(settings, "MICROSOFT_CLIENT_ID", None) + _session(pg_conn) + _session(pg_conn, provider="confluence", status="reconnect_needed") + entries = {e["key"]: e for e in service.catalog_for_user(pg_conn, "alice", is_admin=False)} + assert entries["google_drive"]["state"] == "connected" + assert entries["google_drive"]["connected_count"] == 1 + assert entries["confluence"]["state"] == "reconnect" + # Needs server settings and nobody connected it: members never see it. + assert "share_point" not in entries + assert entries["telegram"]["state"] == "available" + assert entries["custom_mcp"]["state"] == "custom" + + def test_unconfigured_connector_stays_hidden_even_when_switched_on(self, pg_conn, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "MICROSOFT_CLIENT_ID", None) + entries = { + e["key"]: e + for e in service.catalog_for_user( + pg_conn, "alice", is_admin=True, policies={"share_point": {"enabled": True}} + ) + } + assert "share_point" not in entries + + def test_policy_hides_a_disabled_connector(self, pg_conn): + entries = { + e["key"]: e + for e in service.catalog_for_user( + pg_conn, "alice", is_admin=False, policies={"telegram": {"enabled": False}} + ) + } + assert "telegram" not in entries + + def test_disabled_connector_with_a_connection_shows_as_turned_off(self, pg_conn): + """Members can still see and remove what they connected before it was turned off.""" + _session(pg_conn, provider="telegram") + entries = { + e["key"]: e + for e in service.catalog_for_user( + pg_conn, "alice", is_admin=False, policies={"telegram": {"enabled": False}} + ) + } + assert entries["telegram"]["state"] == "disabled" + assert entries["telegram"]["available"] is False + + +class TestConnectorIsEnabled: + def test_unconfigured_connector_is_off_by_default(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", None) + assert service.connector_is_enabled({}, "google_drive") is False + assert service.connector_is_enabled({}, "telegram") is True + + def test_configured_connector_is_on_by_default(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", "id") + monkeypatch.setattr(settings, "GOOGLE_CLIENT_SECRET", "secret") + assert service.connector_is_enabled({}, "google_drive") is True + + def test_an_explicit_switch_wins(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GOOGLE_CLIENT_ID", None) + assert service.connector_is_enabled({"google_drive": {"enabled": True}}, "google_drive") is True + assert service.connector_is_enabled({"telegram": {"enabled": False}}, "telegram") is False + assert service.connector_is_enabled({"telegram": {"enabled": None}}, "telegram") is True + + +@contextmanager +def _patched_db(conn): + @contextmanager + def _yield(): + yield conn + + with patch("docsgpt.api.connector.connections.db_readonly", _yield): + yield + + +@pytest.fixture +def app(): + return Flask(__name__) + + +def _call(app, resource, path, token, *args): + with app.test_request_context(path): + from flask import request + + request.decoded_token = token + return resource().get(*args) + + +class TestRoutes: + def test_catalog_requires_auth(self, app): + from docsgpt.api.connector.connections import ConnectorCatalog + + assert _call(app, ConnectorCatalog, "/api/connectors/catalog", None).status_code == 401 + + def test_catalog(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectorCatalog + + with _patched_db(pg_conn): + resp = _call(app, ConnectorCatalog, "/api/connectors/catalog", {"sub": "alice"}) + assert resp.status_code == 200 + keys = {c["key"] for c in resp.get_json()["connectors"]} + assert "telegram" in keys + + def test_list_and_detail(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail, ConnectionsList + + drive = _session(pg_conn) + with _patched_db(pg_conn): + listed = _call(app, ConnectionsList, "/api/connections", {"sub": "alice"}) + detail = _call(app, ConnectionDetail, f"/api/connections/{drive}", {"sub": "alice"}, drive) + assert listed.get_json()["connections"][0]["id"] == drive + assert detail.get_json()["connection"]["id"] == drive + + def test_detail_of_another_users_connection_is_404(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail + + drive = _session(pg_conn, user="bob") + with _patched_db(pg_conn): + resp = _call(app, ConnectionDetail, f"/api/connections/{drive}", {"sub": "alice"}, drive) + assert resp.status_code == 404 + + def test_detail_of_bad_id_is_404(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDetail + + with _patched_db(pg_conn): + resp = _call(app, ConnectionDetail, "/api/connections/nope", {"sub": "alice"}, "nope") + assert resp.status_code == 404 + + +class TestDisconnect: + def test_clears_credentials_keeps_resources(self, pg_conn): + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + drive = _session( + pg_conn, session_token="tok", token_info=json.dumps({"access_token": "at", "refresh_token": "rt"}), + ) + source = _source(pg_conn, drive) + repo = ConnectorSessionsRepository(pg_conn) + result = service.disconnect(pg_conn, repo.get(drive)) + assert result["status"] == "disconnected" + row = repo.get(drive) + assert row["token_info"] is None and row["session_token"] is None + linked = pg_conn.execute( + text("SELECT connection_id FROM sources WHERE id = CAST(:id AS uuid)"), {"id": source} + ).scalar() + assert str(linked) == drive + + def test_mcp_keeps_client_registration(self, pg_conn): + from docsgpt.storage.db.repositories.connector_sessions import ConnectorSessionsRepository + + mcp = _session( + pg_conn, + provider="mcp:https://mcp.example.com", + status=None, + session_data=json.dumps({"tokens": {"access_token": "x"}, "client_info": {"client_id": "c"}}), + ) + repo = ConnectorSessionsRepository(pg_conn) + service.disconnect(pg_conn, repo.get(mcp)) + row = repo.get(mcp) + assert row["session_data"] == {} + assert service.read_secrets(row) == {"client_info": {"client_id": "c"}} + + def test_route_rejects_other_users(self, app, pg_conn): + from docsgpt.api.connector.connections import ConnectionDisconnect + + drive = _session(pg_conn, user="bob") + + @contextmanager + def _yield(): + yield pg_conn + + with patch("docsgpt.api.connector.connections.db_session", _yield), app.test_request_context( + f"/api/connections/{drive}/disconnect", method="POST" + ): + from flask import request + + request.decoded_token = {"sub": "alice"} + resp = ConnectionDisconnect().post(drive) + assert resp.status_code == 404 diff --git a/tests/connectors/test_tool_routes.py b/tests/connectors/test_tool_routes.py new file mode 100644 index 00000000..7535f079 --- /dev/null +++ b/tests/connectors/test_tool_routes.py @@ -0,0 +1,330 @@ +"""Tests for creating and editing tools whose secret lives on a connection.""" + +from __future__ import annotations + +from contextlib import contextmanager +from unittest.mock import patch + +import pytest +from flask import Flask +from sqlalchemy import text + +import docsgpt.api.user # noqa: F401 (import order: avoids the tools/tasks cycle) +from docsgpt.connectors import service +from docsgpt.security.encryption import encrypt_json + + +@pytest.fixture +def app(): + return Flask(__name__) + + +@contextmanager +def _db(conn): + @contextmanager + def _yield(): + yield conn + + with patch.multiple("docsgpt.api.user.tools.routes", db_session=_yield, db_readonly=_yield), \ + patch.multiple("docsgpt.connectors.service", db_session=_yield, db_readonly=_yield): + yield + + +def _call(app, resource, body, user="alice"): + with app.test_request_context("/api/tools", method="POST", json=body): + from flask import request + + request.decoded_token = {"sub": user} + return resource().post() + + +def _telegram(token="123456:SECRETTOKEN", **extra): + return { + "name": "telegram", + "displayName": "Telegram", + "description": "Send messages", + "config": {"token": token}, + "status": True, + **extra, + } + + +def _connection_row(conn, connection_id): + return dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = CAST(:i AS uuid)"), {"i": connection_id} + ).one()._mapping) + + +class TestCreateConnectedTool: + def test_pasted_key_goes_on_a_connection_not_the_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + resp = _call(app, CreateTool, _telegram()) + assert resp.status_code == 200 + body = resp.get_json() + tool = dict(pg_conn.execute( + text("SELECT * FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": body["id"]} + ).one()._mapping) + assert str(tool["connection_id"]) == body["connection_id"] + assert "SECRETTOKEN" not in str(tool["config"]) + assert "encrypted_credentials" not in (tool["config"] or {}) + row = _connection_row(pg_conn, body["connection_id"]) + assert service.read_secrets(row)["credentials"] == {"token": "123456:SECRETTOKEN"} + + def test_same_key_twice_shares_one_connection(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + first = _call(app, CreateTool, _telegram()).get_json() + second = _call(app, CreateTool, _telegram()).get_json() + assert first["id"] != second["id"] + assert first["connection_id"] == second["connection_id"] + + def test_existing_connection_is_used_by_id(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + first = _call(app, CreateTool, _telegram()).get_json() + resp = _call(app, CreateTool, _telegram(token="", connection_id=first["connection_id"])) + assert resp.status_code == 200 + assert resp.get_json()["connection_id"] == first["connection_id"] + + def test_existing_connection_path_still_validates_the_config(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + first = _call(app, CreateTool, _telegram()).get_json() + with patch("docsgpt.api.user.tools.routes._validate_config", + return_value={"timeout": "Timeout must be between 1 and 300"}) as validate: + resp = _call(app, CreateTool, _telegram(token="", connection_id=first["connection_id"])) + assert resp.status_code == 400 + # The connection supplies the secret, so a missing key is not an error. + assert validate.call_args.kwargs["has_existing_secrets"] is True + + def test_someone_elses_connection_is_not_found(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + victims = _call(app, CreateTool, _telegram(), user="victim").get_json() + resp = _call(app, CreateTool, _telegram(token="", connection_id=victims["connection_id"])) + assert resp.status_code == 404 + + def test_connection_of_another_service_is_not_found(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + telegram = _call(app, CreateTool, _telegram()).get_json() + resp = _call(app, CreateTool, { + **_telegram(token=""), "name": "ntfy", "config": {}, "connection_id": telegram["connection_id"], + }) + assert resp.status_code == 404 + + def test_missing_key_is_a_validation_error(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + resp = _call(app, CreateTool, _telegram(token="")) + assert resp.status_code == 400 + + def test_disabled_connector_is_refused(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _db(pg_conn): + resp = _call(app, CreateTool, _telegram()) + assert resp.status_code == 403 + + def test_disabled_connector_is_refused_on_the_default_key(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _db(pg_conn), patch.object(service, "ensure_can_store_credentials", + side_effect=service.EncryptionKeyNotConfigured("set a key")): + resp = _call(app, CreateTool, _telegram()) + assert resp.status_code == 403 + assert pg_conn.execute(text("SELECT count(*) FROM user_tools")).scalar() == 0 + + def test_default_key_on_multi_user_falls_back_to_the_tool(self, app, pg_conn): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn), patch.object(service, "ensure_can_store_credentials", + side_effect=service.EncryptionKeyNotConfigured("set a key")): + resp = _call(app, CreateTool, _telegram()) + assert resp.status_code == 200 + tool = dict(pg_conn.execute( + text("SELECT * FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": resp.get_json()["id"]} + ).one()._mapping) + assert tool["connection_id"] is None + assert "encrypted_credentials" in tool["config"] + + +class TestUpdateConnectedTool: + def _create(self, app, pg_conn, user="alice"): + from docsgpt.api.user.tools.routes import CreateTool + + with _db(pg_conn): + return _call(app, CreateTool, _telegram(), user=user).get_json() + + def test_new_key_is_written_to_the_connection(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + created = self._create(app, pg_conn) + with _db(pg_conn): + resp = _call(app, UpdateTool, {"id": created["id"], "config": {"token": "999999:ROTATED"}}) + assert resp.status_code == 200 + row = _connection_row(pg_conn, created["connection_id"]) + assert service.read_secrets(row)["credentials"]["token"] == "999999:ROTATED" + config = pg_conn.execute( + text("SELECT config FROM user_tools WHERE id = CAST(:i AS uuid)"), {"i": created["id"]} + ).scalar() + assert "ROTATED" not in str(config) + + def test_new_key_replaces_unreadable_credentials(self, app, pg_conn): + """After a lost encryption key, editing the tool is a way back in.""" + from docsgpt.api.user.tools.routes import UpdateTool + + created = self._create(app, pg_conn) + pg_conn.execute( + text("UPDATE connector_sessions SET encrypted_credentials = :b WHERE id = CAST(:i AS uuid)"), + {"b": encrypt_json({"credentials": {"token": "x"}}, "someone-else"), "i": created["connection_id"]}, + ) + with _db(pg_conn): + resp = _call(app, UpdateTool, {"id": created["id"], "config": {"token": "999999:ROTATED"}}) + assert resp.status_code == 200 + row = _connection_row(pg_conn, created["connection_id"]) + assert service.read_secrets(row)["credentials"] == {"token": "999999:ROTATED"} + + def test_editing_without_a_new_key_keeps_the_connection(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + created = self._create(app, pg_conn) + with _db(pg_conn): + resp = _call(app, UpdateTool, {"id": created["id"], "config": {}}) + assert resp.status_code == 200 + row = _connection_row(pg_conn, created["connection_id"]) + assert service.read_secrets(row)["credentials"]["token"] == "123456:SECRETTOKEN" + + def test_rejected_edit_leaves_the_connection_untouched(self, app, pg_conn): + from docsgpt.api.user.tools.routes import UpdateTool + + created = self._create(app, pg_conn) + with _db(pg_conn), patch("docsgpt.api.user.tools.routes._validate_config", + return_value={"timeout": "Timeout must be between 1 and 300"}): + resp = _call(app, UpdateTool, {"id": created["id"], "config": {"token": "999999:ROTATED"}}) + assert resp.status_code == 400 + row = _connection_row(pg_conn, created["connection_id"]) + assert service.read_secrets(row)["credentials"]["token"] == "123456:SECRETTOKEN" + + +class TestAvailableTools: + def test_tools_of_a_turned_off_connector_are_not_offered(self, app, pg_conn): + from docsgpt.api.user.tools.routes import AvailableTools + from docsgpt.storage.db.repositories.connector_policies import ConnectorPoliciesRepository + + ConnectorPoliciesRepository(pg_conn).upsert("telegram", enabled=False) + with _db(pg_conn), app.test_request_context("/api/available_tools"): + from flask import request + + request.decoded_token = {"sub": "alice"} + names = {t["name"] for t in AvailableTools().get().get_json()["data"]} + assert "telegram" not in names + assert "brave" in names + + def test_service_tools_use_the_connector_name(self, app, pg_conn): + """One name everywhere: "Telegram", not the tool's "Telegram Bot".""" + from docsgpt.api.user.tools.routes import AvailableTools + + with _db(pg_conn), app.test_request_context("/api/available_tools"): + from flask import request + + request.decoded_token = {"sub": "alice"} + tools = {t["name"]: t for t in AvailableTools().get().get_json()["data"]} + assert tools["telegram"]["displayName"] == "Telegram" + assert tools["ntfy"]["displayName"] == "ntfy" + assert tools["postgres"]["displayName"] == "PostgreSQL" + + +def _telegram_connection(conn, user, label, name=None): + row = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, account_label, " + "account_name, encrypted_credentials) VALUES (:u, 'telegram', 'telegram', 'api_key', 'connected', " + ":l, :n, :e) RETURNING *" + ), + {"u": user, "l": label, "n": name, "e": encrypt_json({"credentials": {"token": label}}, user)}, + ).one() + return dict(row._mapping) + + +class TestAccountNamesInToolNames: + def _listed(self, app, pg_conn, user="alice"): + from docsgpt.api.user.tools.routes import GetTools + + with _db(pg_conn), app.test_request_context("/api/get_tools"): + from flask import request + + request.decoded_token = {"sub": user} + tools = GetTools().get().get_json()["tools"] + return sorted(t["customName"] for t in tools if t.get("name") == "telegram") + + def test_two_accounts_are_named_after_their_accounts(self, app, pg_conn): + for label, name in (("…aaaa", "Alerts bot"), ("…bbbb", None)): + service.ensure_connection_tools(pg_conn, "alice", _telegram_connection(pg_conn, "alice", label, name)) + assert self._listed(app, pg_conn) == ["Telegram · Alerts bot", "Telegram · …bbbb"] + + def test_one_account_keeps_the_plain_name(self, app, pg_conn): + connection = _telegram_connection(pg_conn, "alice", "…aaaa", "Alerts bot") + service.ensure_connection_tools(pg_conn, "alice", connection) + assert self._listed(app, pg_conn) == ["Telegram"] + + def test_a_name_the_user_chose_is_kept(self, app, pg_conn): + for label in ("…aaaa", "…bbbb"): + service.ensure_connection_tools(pg_conn, "alice", _telegram_connection(pg_conn, "alice", label)) + pg_conn.execute(text("UPDATE user_tools SET custom_name = 'Ops' WHERE name = 'telegram' " + "AND connection_id = (SELECT id FROM connector_sessions WHERE account_label = '…aaaa')")) + assert self._listed(app, pg_conn) == ["Ops", "Telegram · …bbbb"] + + def test_the_drawer_names_the_tool_after_its_account_too(self, pg_conn): + named = _telegram_connection(pg_conn, "alice", "…aaaa", "Alerts bot") + for connection in (named, _telegram_connection(pg_conn, "alice", "…bbbb")): + service.ensure_connection_tools(pg_conn, "alice", connection) + detail = service.connection_detail(pg_conn, named) + assert detail["account_name"] == "Alerts bot" + assert detail["tools"][0]["display_name"] == "Telegram · Alerts bot" + + +class TestOwnerCredentialWrites: + """The tool list names the writes an agent's API allowlist can cover.""" + + def _listed(self, app, pg_conn): + from docsgpt.api.user.tools.routes import GetTools + + with _db(pg_conn), app.test_request_context("/api/get_tools"): + from flask import request + + request.decoded_token = {"sub": "alice"} + tools = GetTools().get().get_json()["tools"] + return {t["name"]: t.get("owner_credential_writes") for t in tools if t.get("ownership") == "user"} + + def test_writes_on_stored_credentials_are_listed(self, app, pg_conn): + from docsgpt.storage.db.repositories.user_tools import UserToolsRepository + + repo = UserToolsRepository(pg_conn) + key = {"type": "object", "properties": {"X-Key": {"type": "string", "value": "", "has_value": True}}} + repo.create("alice", "api_tool", config={"actions": { + "status": {"url": "https://x.test/s", "method": "GET", "active": True, "headers": key}, + "notify": {"url": "https://x.test/n", "method": "POST", "active": True, "headers": key}, + # A write that sends nothing of the owner's is not listed. + "ping": {"url": "https://x.test/p", "method": "POST", "active": True}, + }}) + repo.create("alice", "mcp_tool", config={"server_url": "https://m.test/mcp", "auth_type": "bearer"}, + actions=[{"name": "create_issue", "active": True}, {"name": "list_issues", "active": True}]) + repo.create("alice", "read_webpage", actions=[{"name": "post_page", "active": True}]) + listed = self._listed(app, pg_conn) + assert listed["api_tool"] == ["notify"] + assert listed["mcp_tool"] == ["create_issue"] + # No credentials: nothing of the owner's to write with. + assert listed["read_webpage"] == [] diff --git a/tests/parser/connectors/confluence/test_auth_session.py b/tests/parser/connectors/confluence/test_auth_session.py index c4d11797..49ef5f43 100644 --- a/tests/parser/connectors/confluence/test_auth_session.py +++ b/tests/parser/connectors/confluence/test_auth_session.py @@ -3,7 +3,6 @@ from contextlib import contextmanager from unittest.mock import patch -import pytest @contextmanager @@ -18,82 +17,3 @@ def _patch_db(conn): yield -class TestGetTokenInfoFromSession: - def test_invalid_session_token_raises(self, pg_conn): - from docsgpt.parser.connectors.confluence.auth import ( - ConfluenceAuth, - ) - - auth = ConfluenceAuth.__new__(ConfluenceAuth) - with _patch_db(pg_conn), pytest.raises(ValueError): - auth.get_token_info_from_session("no-such-token") - - def test_missing_token_info_raises(self, pg_conn): - from docsgpt.parser.connectors.confluence.auth import ( - ConfluenceAuth, - ) - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - - repo = ConnectorSessionsRepository(pg_conn) - repo.upsert("u", "confluence", status="authorized") - # Set session_token but no token_info - session = repo.get_by_user_provider("u", "confluence") - repo.update(str(session["id"]), {"session_token": "tok-no-info"}) - - auth = ConfluenceAuth.__new__(ConfluenceAuth) - with _patch_db(pg_conn), pytest.raises(ValueError): - auth.get_token_info_from_session("tok-no-info") - - def test_missing_required_fields_raises(self, pg_conn): - from docsgpt.parser.connectors.confluence.auth import ( - ConfluenceAuth, - ) - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - - repo = ConnectorSessionsRepository(pg_conn) - repo.upsert("u", "confluence", status="authorized") - session = repo.get_by_user_provider("u", "confluence") - repo.update( - str(session["id"]), - { - "session_token": "tok-partial", - "token_info": {"access_token": "at"}, # missing refresh + cloud_id - }, - ) - - auth = ConfluenceAuth.__new__(ConfluenceAuth) - with _patch_db(pg_conn), pytest.raises(ValueError): - auth.get_token_info_from_session("tok-partial") - - def test_complete_token_info_returned(self, pg_conn): - from docsgpt.parser.connectors.confluence.auth import ( - ConfluenceAuth, - ) - from docsgpt.storage.db.repositories.connector_sessions import ( - ConnectorSessionsRepository, - ) - - repo = ConnectorSessionsRepository(pg_conn) - repo.upsert("u", "confluence", status="authorized") - session = repo.get_by_user_provider("u", "confluence") - repo.update( - str(session["id"]), - { - "session_token": "tok-good", - "token_info": { - "access_token": "at", - "refresh_token": "rt", - "cloud_id": "cid-1", - }, - }, - ) - - auth = ConfluenceAuth.__new__(ConfluenceAuth) - with _patch_db(pg_conn): - got = auth.get_token_info_from_session("tok-good") - assert got["access_token"] == "at" - assert got["cloud_id"] == "cid-1" diff --git a/tests/parser/connectors/confluence/test_loader.py b/tests/parser/connectors/confluence/test_loader.py index ba891d66..61cc3433 100644 --- a/tests/parser/connectors/confluence/test_loader.py +++ b/tests/parser/connectors/confluence/test_loader.py @@ -2,6 +2,7 @@ import os import tempfile +from tests.parser.connectors.token_patch import patch_tokens from unittest.mock import MagicMock, patch import pytest @@ -25,11 +26,12 @@ def _make_loader(token_info=None): with patch("docsgpt.parser.connectors.confluence.loader.ConfluenceAuth") as MockAuth: mock_auth = MagicMock() - mock_auth.get_token_info_from_session.return_value = token_info + token_info = token_info MockAuth.return_value = mock_auth from docsgpt.parser.connectors.confluence.loader import ConfluenceLoader - loader = ConfluenceLoader("session_tok") + with patch_tokens(token_info): + loader = ConfluenceLoader("session_tok") loader.auth = mock_auth return loader @@ -69,7 +71,7 @@ class TestConfluenceLoaderInit: def test_init_sets_attributes(self, loader): assert loader.session_token == "session_tok" assert loader.access_token == "test_at" - assert loader.refresh_token == "test_rt" + assert loader.connection_id == "conn-1" assert loader.cloud_id == "test_cloud" assert loader.next_page_token is None assert "test_cloud" in loader.base_url @@ -834,18 +836,16 @@ class TestRetryOnAuthFailure: # Second call succeeds return _mock_response({"results": [], "_links": {}}) - loader.auth.refresh_access_token = MagicMock(return_value={ - "access_token": "new_at", - "refresh_token": "new_rt", - }) - loader._persist_refreshed_tokens = MagicMock() - - with patch("requests.get", side_effect=flaky_request): + with patch( + "docsgpt.connectors.service.get_valid_token_info", + return_value={"access_token": "new_at", "cloud_id": "test_cloud"}, + ) as tokens, patch("requests.get", side_effect=flaky_request): loader.load_data({}) - assert loader.auth.refresh_access_token.called + # The rejected token goes to the service, which refreshes under the + # connection's row lock and stores the rotated refresh token. + tokens.assert_called_once_with("conn-1", rejected_access_token="test_at") assert loader.access_token == "new_at" - assert loader._persist_refreshed_tokens.called @pytest.mark.unit def test_raises_non_auth_http_error(self, loader): @@ -867,11 +867,9 @@ class TestRetryOnAuthFailure: err = requests.exceptions.HTTPError(response=resp) err.response = resp - loader.auth.refresh_access_token = MagicMock( - side_effect=Exception("refresh failed") - ) - - with patch("requests.get", side_effect=err): + with patch( + "docsgpt.connectors.service.get_valid_token_info", side_effect=Exception("refresh failed"), + ), patch("requests.get", side_effect=err): with pytest.raises(ValueError, match="Authentication failed"): loader.load_data({}) @@ -890,29 +888,12 @@ class TestRetryOnAuthFailure: raise err return _mock_response({"results": [], "_links": {}}) - loader.auth.refresh_access_token = MagicMock(return_value={ - "access_token": "new_at", - "refresh_token": "new_rt", - }) - loader._persist_refreshed_tokens = MagicMock() - - with patch("requests.get", side_effect=flaky): + with patch( + "docsgpt.connectors.service.get_valid_token_info", + return_value={"access_token": "new_at", "cloud_id": "test_cloud"}, + ) as tokens, patch("requests.get", side_effect=flaky): loader.load_data({}) - assert loader.auth.refresh_access_token.called + assert tokens.called -# --------------------------------------------------------------------------- -# _persist_refreshed_tokens -# --------------------------------------------------------------------------- - - -class TestPersistRefreshedTokens: - pass - - @pytest.mark.unit - def test_logs_warning_on_failure(self, loader): - loader.auth.sanitize_token_info = MagicMock(side_effect=Exception("db error")) - - # Should not raise, just log a warning - loader._persist_refreshed_tokens({"access_token": "at"}) diff --git a/tests/parser/connectors/test_auth_token_leakage.py b/tests/parser/connectors/test_auth_token_leakage.py index 98d63aa4..217b437e 100644 --- a/tests/parser/connectors/test_auth_token_leakage.py +++ b/tests/parser/connectors/test_auth_token_leakage.py @@ -51,12 +51,11 @@ def _patches(session_return: Optional[Dict[str, Any]]): ) return ( patch( - "docsgpt.storage.db.repositories.connector_sessions." - "ConnectorSessionsRepository", + "docsgpt.connectors.service.ConnectorSessionsRepository", fake_repo_cls, ), patch( - "docsgpt.storage.db.session.db_readonly", + "docsgpt.connectors.service.db_readonly", lambda: _FakeReadonlyCtx(), ), ) @@ -88,57 +87,18 @@ class TestSessionTokenFingerprint: assert session_token_fingerprint("a") != session_token_fingerprint("b") -class TestConfluenceAuthDoesNotLeakToken: +class TestConnectionLookupDoesNotLeakToken: + """A session token that names no connection is reported by fingerprint only.""" @pytest.mark.unit def test_invalid_session_does_not_interpolate_token(self) -> None: - from docsgpt.parser.connectors.confluence.auth import ConfluenceAuth + from docsgpt.connectors import service - auth = ConfluenceAuth.__new__(ConfluenceAuth) repo_patch, ctx_patch = _patches(None) with repo_patch, ctx_patch: with pytest.raises(ValueError) as excinfo: - auth.get_token_info_from_session(SECRET_TOKEN) + service.connection_id_for_session_token(SECRET_TOKEN) message = str(excinfo.value) assert SECRET_TOKEN not in message assert session_token_fingerprint(SECRET_TOKEN) in message - - -class TestGoogleDriveAuthDoesNotLeakToken: - - @pytest.mark.unit - def test_invalid_session_does_not_interpolate_token(self) -> None: - from docsgpt.parser.connectors.google_drive.auth import GoogleDriveAuth - - auth = GoogleDriveAuth.__new__(GoogleDriveAuth) - repo_patch, ctx_patch = _patches(None) - with repo_patch, ctx_patch: - with pytest.raises(ValueError) as excinfo: - auth.get_token_info_from_session(SECRET_TOKEN) - - # The Google Drive module wraps the inner ValueError in a broad - # ``except Exception as e: raise ValueError(... {str(e)})`` block, - # so the outer message still carries the fingerprint from the - # inner raise but must never carry the raw token. - message = str(excinfo.value) - assert SECRET_TOKEN not in message - assert session_token_fingerprint(SECRET_TOKEN) in message - - -class TestSharePointAuthDoesNotLeakToken: - - @pytest.mark.unit - def test_invalid_session_does_not_interpolate_token(self) -> None: - from docsgpt.parser.connectors.share_point.auth import SharePointAuth - - auth = SharePointAuth.__new__(SharePointAuth) - repo_patch, ctx_patch = _patches(None) - with repo_patch, ctx_patch: - with pytest.raises(ValueError) as excinfo: - auth.get_token_info_from_session(SECRET_TOKEN) - - # SharePoint also wraps the inner ValueError. Same invariants. - message = str(excinfo.value) - assert SECRET_TOKEN not in message - assert session_token_fingerprint(SECRET_TOKEN) in message diff --git a/tests/parser/connectors/test_connector_creator.py b/tests/parser/connectors/test_connector_creator.py index 535441db..321aa016 100644 --- a/tests/parser/connectors/test_connector_creator.py +++ b/tests/parser/connectors/test_connector_creator.py @@ -1,5 +1,6 @@ """Tests for ConnectorCreator factory class.""" +from tests.parser.connectors.token_patch import patch_tokens from unittest.mock import patch, MagicMock import pytest @@ -74,7 +75,7 @@ class TestConnectorCreator: def test_create_connector_google_drive(self): with patch("docsgpt.parser.connectors.google_drive.loader.GoogleDriveAuth") as MockAuth: mock_auth_instance = MagicMock() - mock_auth_instance.get_token_info_from_session.return_value = { + token_info = { "access_token": "at", "refresh_token": "rt" } mock_creds = MagicMock() @@ -84,7 +85,8 @@ class TestConnectorCreator: mock_auth_instance.build_drive_service.return_value = MagicMock() MockAuth.return_value = mock_auth_instance - loader = self.ConnectorCreator.create_connector("google_drive", "session_tok") + with patch_tokens(token_info): + loader = self.ConnectorCreator.create_connector("google_drive", "session_tok") from docsgpt.parser.connectors.google_drive.loader import GoogleDriveLoader assert isinstance(loader, GoogleDriveLoader) @@ -92,11 +94,12 @@ class TestConnectorCreator: def test_create_connector_share_point(self): with patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth") as MockAuth: mock_auth_instance = MagicMock() - mock_auth_instance.get_token_info_from_session.return_value = { + token_info = { "access_token": "at", "refresh_token": "rt" } MockAuth.return_value = mock_auth_instance - loader = self.ConnectorCreator.create_connector("share_point", "session_tok") + with patch_tokens(token_info): + loader = self.ConnectorCreator.create_connector("share_point", "session_tok") from docsgpt.parser.connectors.share_point.loader import SharePointLoader assert isinstance(loader, SharePointLoader) diff --git a/tests/parser/connectors/test_github_auth.py b/tests/parser/connectors/test_github_auth.py new file mode 100644 index 00000000..16c8013f --- /dev/null +++ b/tests/parser/connectors/test_github_auth.py @@ -0,0 +1,128 @@ +"""Tests for the GitHub App user sign-in.""" + +from __future__ import annotations + +import datetime +from unittest.mock import MagicMock, patch +from urllib.parse import parse_qs, urlparse + +import pytest + + +@pytest.fixture +def configured(monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_CLIENT_ID", "Iv1.client") + monkeypatch.setattr(settings, "GITHUB_CLIENT_SECRET", "app-secret") + monkeypatch.setattr(settings, "GITHUB_APP_SLUG", "docsgpt-acme") + monkeypatch.setattr(settings, "CONNECTOR_REDIRECT_BASE_URI", "https://docs.example/api/connectors/callback") + + +def _response(payload, status=200): + response = MagicMock(status_code=status) + response.json.return_value = payload + response.raise_for_status.return_value = None + return response + + +def test_needs_the_app_settings(monkeypatch): + from docsgpt.core.settings import settings + from docsgpt.parser.connectors.github.auth import GitHubAuth + + monkeypatch.setattr(settings, "GITHUB_CLIENT_ID", None) + with pytest.raises(ValueError, match="GITHUB_CLIENT_ID"): + GitHubAuth() + + +def test_authorization_url_carries_state_and_callback(configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + url = urlparse(GitHubAuth().get_authorization_url(state="st")) + query = parse_qs(url.query) + assert f"{url.scheme}://{url.netloc}{url.path}" == "https://github.com/login/oauth/authorize" + assert query["client_id"] == ["Iv1.client"] + assert query["state"] == ["st"] + assert query["redirect_uri"] == ["https://docs.example/api/connectors/callback"] + + +def test_installation_url_carries_state(configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + assert GitHubAuth().get_installation_url(state="st") == ( + "https://github.com/apps/docsgpt-acme/installations/new?state=st" + ) + + +def test_exchange_returns_expiring_tokens_and_the_login(configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + token = _response({ + "access_token": "ghu_a", "refresh_token": "ghr_r", "expires_in": 28800, "token_type": "bearer", + }) + user = _response({"login": "octocat", "name": "The Octocat"}) + with patch("docsgpt.parser.connectors.github.auth.requests.post", return_value=token) as post, \ + patch("docsgpt.parser.connectors.github.auth.requests.get", return_value=user): + info = GitHubAuth().exchange_code_for_tokens("code-1") + assert post.call_args.kwargs["data"]["code"] == "code-1" + assert post.call_args.kwargs["headers"]["Accept"] == "application/json" + assert info["access_token"] == "ghu_a" and info["refresh_token"] == "ghr_r" + assert info["user_info"]["login"] == "octocat" + expiry = datetime.datetime.fromisoformat(info["expiry"]) + assert expiry > datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(hours=7) + + +def test_exchange_error_in_a_200_body_raises(configured): + """GitHub reports a bad code with 200 and an ``error`` field.""" + from docsgpt.parser.connectors.github.auth import GitHubAuth + + with patch("docsgpt.parser.connectors.github.auth.requests.post", + return_value=_response({"error": "bad_verification_code"})): + with pytest.raises(ValueError, match="bad_verification_code"): + GitHubAuth().exchange_code_for_tokens("stale") + + +def test_refused_refresh_raises(configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + with patch("docsgpt.parser.connectors.github.auth.requests.post", + return_value=_response({"error": "bad_refresh_token"})): + with pytest.raises(ValueError, match="bad_refresh_token"): + GitHubAuth().refresh_access_token("ghr_old") + + +def test_refresh_rotates_the_refresh_token(configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + with patch("docsgpt.parser.connectors.github.auth.requests.post", return_value=_response({ + "access_token": "ghu_b", "refresh_token": "ghr_new", "expires_in": 28800, + })) as post: + info = GitHubAuth().refresh_access_token("ghr_old") + assert post.call_args.kwargs["data"]["grant_type"] == "refresh_token" + assert info["access_token"] == "ghu_b" and info["refresh_token"] == "ghr_new" + + +class TestExpiry: + def test_token_without_expiry_never_expires(self, configured): + """An app with token expiry turned off issues tokens with no expiry: they stay valid.""" + from docsgpt.parser.connectors.github.auth import GitHubAuth + + assert GitHubAuth().is_token_expired({"access_token": "ghu_a"}) is False + + def test_expired_and_fresh(self, configured): + from docsgpt.parser.connectors.github.auth import GitHubAuth + + now = datetime.datetime.now(datetime.timezone.utc) + auth = GitHubAuth() + assert auth.is_token_expired({"access_token": "a", "expiry": (now - datetime.timedelta(minutes=1)).isoformat()}) + assert not auth.is_token_expired({"access_token": "a", "expiry": (now + datetime.timedelta(hours=1)).isoformat()}) + assert auth.is_token_expired({}) + + +def test_registered_as_an_auth_provider_but_not_a_file_connector(): + """GitHub signs in like the OAuth connectors but ingests through the remote loader.""" + from docsgpt.parser.connectors.connector_creator import ConnectorCreator + + assert ConnectorCreator.has_auth("github") + assert not ConnectorCreator.is_supported("github") + assert "github" not in ConnectorCreator.get_supported_connectors() diff --git a/tests/parser/connectors/test_google_drive_auth.py b/tests/parser/connectors/test_google_drive_auth.py index 610e6d11..63c50a4a 100644 --- a/tests/parser/connectors/test_google_drive_auth.py +++ b/tests/parser/connectors/test_google_drive_auth.py @@ -364,71 +364,6 @@ class _FakeReadonlyCtx: return False -class TestGetTokenInfoFromSession: - - def _patches(self, session_return): - fake_repo_cls = type( - "FakeRepo", - (_FakeRepo,), - {"_session": session_return}, - ) - return ( - patch( - "docsgpt.storage.db.repositories.connector_sessions.ConnectorSessionsRepository", - fake_repo_cls, - ), - patch( - "docsgpt.storage.db.session.db_readonly", - lambda: _FakeReadonlyCtx(), - ), - ) - - @pytest.mark.unit - def test_valid_session(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({ - "session_token": "st", - "token_info": {"access_token": "at", "refresh_token": "rt"}, - }) - with repo_patch, ctx_patch: - result = auth.get_token_info_from_session("st") - assert result["access_token"] == "at" - assert result["token_uri"] == "https://oauth2.googleapis.com/token" - - @pytest.mark.unit - def test_session_not_found_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches(None) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve Google Drive token"): - auth.get_token_info_from_session("bad_token") - - @pytest.mark.unit - def test_session_missing_token_info_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({"session_token": "st"}) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve Google Drive token"): - auth.get_token_info_from_session("st") - - @pytest.mark.unit - def test_missing_required_fields_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({ - "session_token": "st", - "token_info": {"access_token": "at"}, - }) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve Google Drive token"): - auth.get_token_info_from_session("st") - - @pytest.mark.unit - def test_empty_token_info_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({ - "session_token": "st", - "token_info": None, - }) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve Google Drive token"): - auth.get_token_info_from_session("st") - - class TestValidateCredentials: @pytest.mark.unit diff --git a/tests/parser/connectors/test_google_drive_loader.py b/tests/parser/connectors/test_google_drive_loader.py index ed765e60..3a6dcb67 100644 --- a/tests/parser/connectors/test_google_drive_loader.py +++ b/tests/parser/connectors/test_google_drive_loader.py @@ -1,5 +1,6 @@ """Tests for GoogleDriveLoader.""" +from tests.parser.connectors.token_patch import patch_tokens from unittest.mock import MagicMock, patch import pytest @@ -11,7 +12,7 @@ def _make_loader(service=None): """Create a GoogleDriveLoader with mocked dependencies.""" with patch("docsgpt.parser.connectors.google_drive.loader.GoogleDriveAuth") as MockAuth: mock_auth = MagicMock() - mock_auth.get_token_info_from_session.return_value = { + token_info = { "access_token": "at", "refresh_token": "rt", } @@ -24,7 +25,8 @@ def _make_loader(service=None): MockAuth.return_value = mock_auth from docsgpt.parser.connectors.google_drive.loader import GoogleDriveLoader - loader = GoogleDriveLoader("session_tok") + with patch_tokens(token_info): + loader = GoogleDriveLoader("session_tok") return loader @@ -51,7 +53,7 @@ class TestGoogleDriveLoaderInit: def test_init_service_failure_sets_none(self): with patch("docsgpt.parser.connectors.google_drive.loader.GoogleDriveAuth") as MockAuth: mock_auth = MagicMock() - mock_auth.get_token_info_from_session.return_value = { + token_info = { "access_token": "at", "refresh_token": "rt" } mock_creds = MagicMock() @@ -61,7 +63,8 @@ class TestGoogleDriveLoaderInit: MockAuth.return_value = mock_auth from docsgpt.parser.connectors.google_drive.loader import GoogleDriveLoader - loader = GoogleDriveLoader("st") + with patch_tokens(token_info): + loader = GoogleDriveLoader("st") assert loader.service is None diff --git a/tests/parser/connectors/test_share_point_auth.py b/tests/parser/connectors/test_share_point_auth.py index ecf25a7d..23529ddd 100644 --- a/tests/parser/connectors/test_share_point_auth.py +++ b/tests/parser/connectors/test_share_point_auth.py @@ -285,63 +285,3 @@ class _FakeReadonlyCtx: return False -class TestGetTokenInfoFromSession: - - def _patches(self, session_return): - fake_repo_cls = type( - "FakeRepo", - (_FakeRepo,), - {"_session": session_return}, - ) - return ( - patch( - "docsgpt.storage.db.repositories.connector_sessions.ConnectorSessionsRepository", - fake_repo_cls, - ), - patch( - "docsgpt.storage.db.session.db_readonly", - lambda: _FakeReadonlyCtx(), - ), - ) - - @pytest.mark.unit - def test_valid_session(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({ - "session_token": "st", - "token_info": {"access_token": "at", "refresh_token": "rt"}, - }) - with repo_patch, ctx_patch: - result = auth.get_token_info_from_session("st") - assert result["access_token"] == "at" - assert "token_uri" in result - - @pytest.mark.unit - def test_session_not_found_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches(None) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve SharePoint token"): - auth.get_token_info_from_session("bad") - - @pytest.mark.unit - def test_missing_token_info_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({"session_token": "st"}) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve SharePoint token"): - auth.get_token_info_from_session("st") - - @pytest.mark.unit - def test_empty_token_info_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({"session_token": "st", "token_info": None}) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve SharePoint token"): - auth.get_token_info_from_session("st") - - @pytest.mark.unit - def test_missing_required_fields_raises(self, auth, mock_settings): - repo_patch, ctx_patch = self._patches({ - "session_token": "st", - "token_info": {"access_token": "at"}, - }) - with repo_patch, ctx_patch: - with pytest.raises(ValueError, match="Failed to retrieve SharePoint token"): - auth.get_token_info_from_session("st") diff --git a/tests/parser/connectors/test_share_point_loader.py b/tests/parser/connectors/test_share_point_loader.py index b54eb02b..4390bc2d 100644 --- a/tests/parser/connectors/test_share_point_loader.py +++ b/tests/parser/connectors/test_share_point_loader.py @@ -1,6 +1,7 @@ """Tests for SharePointLoader.""" import os +from tests.parser.connectors.token_patch import patch_tokens from unittest.mock import MagicMock, patch import pytest @@ -13,7 +14,7 @@ def _make_loader(access_token="at", refresh_token="rt", allows_shared=False): """Create a SharePointLoader with mocked dependencies.""" with patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth") as MockAuth: mock_auth = MagicMock() - mock_auth.get_token_info_from_session.return_value = { + token_info = { "access_token": access_token, "refresh_token": refresh_token, "allows_shared_content": allows_shared, @@ -22,7 +23,8 @@ def _make_loader(access_token="at", refresh_token="rt", allows_shared=False): MockAuth.return_value = mock_auth from docsgpt.parser.connectors.share_point.loader import SharePointLoader - loader = SharePointLoader("session_tok") + with patch_tokens(token_info): + loader = SharePointLoader("session_tok") return loader @@ -42,7 +44,7 @@ class TestSharePointLoaderInit: def test_init_sets_attributes(self, loader): assert loader.session_token == "session_tok" assert loader.access_token == "at" - assert loader.refresh_token == "rt" + assert loader.connection_id == "conn-1" assert loader.allows_shared_content is False assert loader.next_page_token is None @@ -50,7 +52,7 @@ class TestSharePointLoaderInit: def test_no_access_token_raises(self): with patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth") as MockAuth: mock_auth = MagicMock() - mock_auth.get_token_info_from_session.return_value = { + token_info = { "access_token": None, "refresh_token": "rt", } @@ -58,7 +60,8 @@ class TestSharePointLoaderInit: from docsgpt.parser.connectors.share_point.loader import SharePointLoader with pytest.raises(ValueError, match="No access token"): - SharePointLoader("st") + with patch_tokens(token_info): + SharePointLoader("st") class TestGetHeaders: @@ -86,16 +89,19 @@ class TestEnsureValidToken: @pytest.mark.unit def test_expired_token_refreshes(self, loader): loader.auth.is_token_expired.return_value = True - loader.auth.refresh_access_token.return_value = {"access_token": "new_at"} - loader._ensure_valid_token() + with patch( + "docsgpt.connectors.service.get_valid_token_info", return_value={"access_token": "new_at"}, + ) as tokens: + loader._ensure_valid_token() assert loader.access_token == "new_at" + tokens.assert_called_once_with("conn-1") @pytest.mark.unit def test_refresh_failure_raises(self, loader): loader.auth.is_token_expired.return_value = True - loader.auth.refresh_access_token.side_effect = Exception("fail") - with pytest.raises(ValueError, match="Failed to refresh"): - loader._ensure_valid_token() + with patch("docsgpt.connectors.service.get_valid_token_info", side_effect=Exception("fail")): + with pytest.raises(ValueError, match="Failed to refresh"): + loader._ensure_valid_token() class TestGetItemUrl: @@ -305,7 +311,7 @@ class TestLoadFileById: return mock_resp_ok loader._process_file = MagicMock(return_value=Document(text="", doc_id="f1", extra_info={})) - loader.auth.refresh_access_token.return_value = {"access_token": "new_at"} + loader._refresh_rejected_token = MagicMock(return_value={"access_token": "new_at"}) with patch("docsgpt.parser.connectors.share_point.loader.requests.get", side_effect=get_side_effect): doc = loader._load_file_by_id("f1") @@ -481,7 +487,7 @@ class TestDownloadFileContent: raise http_error return mock_resp_ok - loader.auth.refresh_access_token.return_value = {"access_token": "new_at"} + loader._refresh_rejected_token = MagicMock(return_value={"access_token": "new_at"}) with patch("docsgpt.parser.connectors.share_point.loader.requests.get", side_effect=get_side_effect): content = loader._download_file_content("f1") @@ -1004,7 +1010,7 @@ class TestRetryOnAuthFailureDecorator: raise http_error return mock_resp_ok - loader.auth.refresh_access_token.return_value = {"access_token": "new_at"} + loader._refresh_rejected_token = MagicMock(return_value={"access_token": "new_at"}) with patch("docsgpt.parser.connectors.share_point.loader.requests.get", side_effect=get_side_effect): content = loader._download_file_content("f1") @@ -1018,7 +1024,7 @@ class TestRetryOnAuthFailureDecorator: mock_resp_403.status_code = 403 http_error = real_requests.exceptions.HTTPError(response=mock_resp_403) - loader.auth.refresh_access_token.side_effect = Exception("refresh fail") + loader._refresh_rejected_token = MagicMock(side_effect=Exception("refresh fail")) with patch("docsgpt.parser.connectors.share_point.loader.requests.get", side_effect=http_error): with pytest.raises(ValueError, match="could not be refreshed"): diff --git a/tests/parser/connectors/token_patch.py b/tests/parser/connectors/token_patch.py new file mode 100644 index 00000000..dafcd115 --- /dev/null +++ b/tests/parser/connectors/token_patch.py @@ -0,0 +1,23 @@ +"""Stand-in for the connection service in connector loader tests.""" + +from __future__ import annotations + +from contextlib import ExitStack, contextmanager +from unittest.mock import patch + + +@contextmanager +def patch_tokens(token_info: dict, connection_id: str = "conn-1"): + """Make every loader resolve ``connection_id`` and read ``token_info``. + + Yields the ``get_valid_token_info`` mock so a test can assert on or + change what a refresh returns. + """ + with ExitStack() as stack: + stack.enter_context( + patch("docsgpt.connectors.service.connection_id_for_session_token", return_value=connection_id) + ) + tokens = stack.enter_context( + patch("docsgpt.connectors.service.get_valid_token_info", return_value=token_info) + ) + yield tokens diff --git a/tests/parser/remote/test_github_loader.py b/tests/parser/remote/test_github_loader.py index 61684af1..7dcb7756 100644 --- a/tests/parser/remote/test_github_loader.py +++ b/tests/parser/remote/test_github_loader.py @@ -6,6 +6,16 @@ import requests from docsgpt.parser.remote.github_loader import GitHubLoader +@pytest.fixture(autouse=True) +def _no_instance_token(monkeypatch): + """No ``GITHUB_ACCESS_TOKEN`` from the environment: with one, ``load_data`` + checks the repository's visibility against the real GitHub API. Tests that + need the instance token set it themselves.""" + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_ACCESS_TOKEN", None) + + def make_response(json_data=None, status_code=200, raise_error=None): resp = MagicMock() resp.status_code = status_code @@ -512,3 +522,107 @@ class TestGitHubLoaderStaleTokenFallback: assert mock_get.call_count == 2 # Second attempt carried no Authorization header. assert "Authorization" not in mock_get.call_args.kwargs["headers"] + + +def _stub_repo(loader, monkeypatch): + monkeypatch.setattr(loader, "get_default_branch", lambda repo: "main") + monkeypatch.setattr(loader, "fetch_repo_tree", lambda repo, branch: ([("README.md", 5)], False)) + monkeypatch.setattr(loader, "fetch_file_content", lambda repo, path: "readme") + + +class TestGitHubLoaderConnectionToken: + def test_uses_the_connections_token(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_ACCESS_TOKEN", "instance-token") + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + with patch("docsgpt.parser.remote.github_loader.requests.get") as get: + docs = loader.load_data({"repo_url": "owner/private-repo", "access_token": "user-token"}) + # A user's own token reads their private repos: no public check. + get.assert_not_called() + assert loader.headers["Authorization"] == "Bearer user-token" + assert [d.doc_id for d in docs] == ["README.md"] + + def test_accepts_a_url_key_and_a_plain_string(self, monkeypatch): + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + assert loader.load_data({"url": "https://github.com/owner/repo"})[0].extra_info["source"].startswith( + "https://github.com/owner/repo/" + ) + assert loader.load_data("owner/repo") + + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_rejected_connection_token_is_not_retried_anonymously(self, mock_get): + from docsgpt.parser.remote.github_loader import GitHubTokenRejected + + loader = GitHubLoader(access_token="revoked") + mock_get.return_value = MagicMock(status_code=401) + with pytest.raises(GitHubTokenRejected): + loader._make_request("https://api.github.com/repos/o/r") + assert mock_get.call_count == 1 + + +class TestGitHubLoaderInstanceTokenIsPublicOnly: + """GITHUB_ACCESS_TOKEN is shared by every user: it must never read a private repo.""" + + @pytest.fixture(autouse=True) + def _instance_token(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "GITHUB_ACCESS_TOKEN", "instance-token") + + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_private_repo_is_refused(self, mock_get, monkeypatch): + from docsgpt.parser.remote.github_loader import PrivateRepositoryError + + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + fetched = MagicMock() + monkeypatch.setattr(loader, "fetch_file_content", fetched) + mock_get.return_value = make_response({"private": True, "visibility": "private"}) + + with pytest.raises(PrivateRepositoryError, match="Connect"): + loader.load_data("https://github.com/acme/secret") + fetched.assert_not_called() + + @pytest.mark.parametrize("status", [404, 403]) + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_unreadable_repo_is_refused(self, mock_get, status, monkeypatch): + from docsgpt.parser.remote.github_loader import PrivateRepositoryError + + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + mock_get.return_value = make_response({"message": "Not Found"}, status_code=status) + with pytest.raises(PrivateRepositoryError): + loader.load_data("acme/secret") + + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_internal_repo_is_refused(self, mock_get, monkeypatch): + from docsgpt.parser.remote.github_loader import PrivateRepositoryError + + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + mock_get.return_value = make_response({"private": False, "visibility": "internal"}) + with pytest.raises(PrivateRepositoryError): + loader.load_data("acme/internal") + + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_public_repo_uses_the_instance_token(self, mock_get, monkeypatch): + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + mock_get.return_value = make_response({"private": False, "visibility": "public"}) + + docs = loader.load_data("owner/repo") + + assert [d.doc_id for d in docs] == ["README.md"] + assert mock_get.call_args.kwargs["headers"]["Authorization"] == "Bearer instance-token" + + @patch("docsgpt.parser.remote.github_loader.requests.get") + def test_stale_instance_token_reads_public_repos_anonymously(self, mock_get, monkeypatch): + loader = GitHubLoader() + _stub_repo(loader, monkeypatch) + mock_get.side_effect = [MagicMock(status_code=401), make_response({"private": False})] + + assert loader.load_data("owner/repo") + assert "Authorization" not in loader.headers diff --git a/tests/parser/remote/test_linear_loader.py b/tests/parser/remote/test_linear_loader.py new file mode 100644 index 00000000..a8b6e750 --- /dev/null +++ b/tests/parser/remote/test_linear_loader.py @@ -0,0 +1,327 @@ +"""Linear issues and documents as Knowledge, read through Linear's MCP server (mocked).""" + +from __future__ import annotations + +import asyncio +from unittest.mock import patch + +import pytest + +from docsgpt.connectors import linear +from docsgpt.parser.remote.linear_loader import LinearLoader + +# Input schemas shaped like the ones Linear's MCP server publishes. +SCHEMAS = { + "list_issues": {"properties": { + "team": {}, "project": {}, "limit": {"maximum": 250}, "cursor": {}, "orderBy": {}, "includeArchived": {}, + }}, + "get_issue": {"properties": {"id": {}}}, + "list_comments": {"properties": {"issueId": {}}}, + "list_documents": {"properties": {"projectId": {}, "limit": {}, "cursor": {}}}, + "get_document": {"properties": {"id": {}}}, + "list_teams": {"properties": {"limit": {}, "cursor": {}}}, + "list_projects": {"properties": {"limit": {}, "cursor": {}, "includeArchived": {}}}, +} + +ISSUE = { + "id": "ENG-1310", + "title": "Fix the loader", + "description": "The spinner stays for 10 s.", + "url": "https://linear.app/acme/issue/ENG-1310/fix-the-loader", + "status": "In Progress", + "priority": {"value": 2, "name": "High"}, + "assignee": "Sam", + "labels": ["bug", {"id": "l2", "name": "ui"}], + "project": "Acme", + "team": "Engineering", + "createdAt": "2026-09-20T10:00:00.000Z", + "updatedAt": "2026-09-25T09:30:00.000Z", +} + + +class FakeLinear: + """Linear's MCP server, answering tool calls from handlers.""" + + def __init__(self, handlers, schemas=None): + self.handlers = handlers + self.schemas = SCHEMAS if schemas is None else schemas + self.calls: list[tuple[str, dict]] = [] + + async def input_schema(self, name): + return self.schemas.get(name) + + async def call(self, name, arguments): + self.calls.append((name, dict(arguments))) + return self.handlers[name](arguments) + + def called(self, name): + return [args for tool, args in self.calls if tool == name] + + +def _collect(session, **selection): + selection = linear.normalize_selection({"teams": [], "projects": [], **selection}) + return asyncio.run(LinearLoader().collect(session, selection)) + + +class TestIssues: + def test_one_document_per_issue_with_what_it_says(self): + session = FakeLinear({ + "list_issues": lambda args: {"issues": [ISSUE], "hasNextPage": False}, + "list_comments": lambda args: {"comments": [ + {"id": "c1", "body": "Reproduced on the current release.", "user": {"name": "Alex"}, + "createdAt": "2026-09-21T11:00:00.000Z"}, + {"id": "c2", "body": "Fixed.", "author": "Sam", "createdAt": "2026-09-22T11:00:00.000Z"}, + ]}, + }) + [doc] = _collect(session, teams=[{"id": "team-1", "key": "ENG", "name": "Engineering"}]) + text = doc.text + assert text.startswith("# ENG-1310: Fix the loader") + for fragment in ("In Progress", "Sam", "High", "bug, ui", "Acme", "The spinner stays for 10 s.", + "Reproduced on the current release.", "Alex", "Fixed."): + assert fragment in text + assert doc.extra_info["title"] == "ENG-1310: Fix the loader" + assert doc.extra_info["source"] == ISSUE["url"] + assert doc.extra_info["file_path"] == "ENG/ENG-1310.md" + assert session.called("list_comments") == [{"issueId": "ENG-1310"}] + + def test_pages_through_a_teams_issues(self): + pages = { + None: {"issues": [{**ISSUE, "id": "ENG-2"}], "hasNextPage": True, "cursor": "p2"}, + "p2": {"issues": [{**ISSUE, "id": "ENG-1"}], "hasNextPage": False}, + } + session = FakeLinear({"list_issues": lambda args: pages[args.get("cursor")]}) + docs = _collect(session, teams=["team-1"], include_comments=False) + assert [d.extra_info["title"].split(":")[0] for d in docs] == ["ENG-2", "ENG-1"] + first, second = session.called("list_issues") + assert first == {"team": "team-1", "limit": 100, "orderBy": "updatedAt", "includeArchived": False} + assert second == {**first, "cursor": "p2"} + + def test_a_clipped_description_is_read_in_full(self): + clipped = {**ISSUE, "description": "The spinner (truncated, use get_issue to read the full description)"} + session = FakeLinear({ + "list_issues": lambda args: {"issues": [clipped]}, + "get_issue": lambda args: {"issue": {**ISSUE, "description": "The whole story."}}, + }) + [doc] = _collect(session, teams=["team-1"], include_comments=False) + assert "The whole story." in doc.text + assert "truncated" not in doc.text + assert session.called("get_issue") == [{"id": "ENG-1310"}] + + def test_comments_are_paged_through(self): + schemas = {**SCHEMAS, "list_comments": {"properties": {"issueId": {}, "cursor": {}}}} + pages = { + None: {"comments": [{"body": "First."}], "hasNextPage": True, "cursor": "c2"}, + "c2": {"comments": [{"body": "Second."}], "hasNextPage": False}, + } + session = FakeLinear({ + "list_issues": lambda args: [ISSUE], + "list_comments": lambda args: pages[args.get("cursor")], + }, schemas) + [doc] = _collect(session, teams=["team-1"]) + assert "First." in doc.text and "Second." in doc.text + + def test_comments_linear_cannot_list_are_skipped(self): + schemas = {k: v for k, v in SCHEMAS.items() if k != "list_comments"} + session = FakeLinear({"list_issues": lambda args: [ISSUE]}, schemas) + [doc] = _collect(session, teams=["team-1"]) + assert "## Comments" not in doc.text + + def test_comments_are_left_out_when_not_wanted(self): + session = FakeLinear({"list_issues": lambda args: [ISSUE]}) + [doc] = _collect(session, teams=["team-1"], include_comments=False) + assert session.called("list_comments") == [] + assert "## Comments" not in doc.text + + def test_an_issue_in_a_picked_team_and_project_is_synced_once(self): + session = FakeLinear({"list_issues": lambda args: {"issues": [ISSUE]}}) + docs = _collect(session, teams=["team-1"], projects=["project-1"], include_comments=False) + assert len(docs) == 1 + assert [args.get("team") or args.get("project") for args in session.called("list_issues")] == [ + "team-1", "project-1", + ] + + def test_stops_at_the_cap(self, monkeypatch): + monkeypatch.setattr(linear, "MAX_ISSUES", 3) + pages = iter(range(100)) + + def endless(args): + page = next(pages) + issues = [{**ISSUE, "id": f"ENG-{page}{n}"} for n in range(2)] + return {"issues": issues, "hasNextPage": True, "cursor": f"p{page + 1}"} + + session = FakeLinear({"list_issues": endless}) + docs = _collect(session, teams=["team-1"], projects=["project-1"], include_comments=False) + assert len(docs) == 3 + assert len(session.called("list_issues")) == 2 + + def test_a_repeated_cursor_ends_the_listing(self): + session = FakeLinear({"list_issues": lambda args: {"issues": [ISSUE], "hasNextPage": True, "cursor": "same"}}) + _collect(session, teams=["team-1"], include_comments=False) + assert len(session.called("list_issues")) == 2 + + def test_nested_records_and_graphql_page_info_are_read(self): + record = { + "id": "5c1e3d2a-uuid", "identifier": "DES-7", "title": "Logo", + "state": {"id": "s", "name": "Done"}, "assignee": {"id": "u", "displayName": "kim"}, + "team": {"id": "t", "key": "DES", "name": "Design"}, "labels": {"nodes": [{"name": "brand"}]}, + } + session = FakeLinear({"list_issues": lambda args: { + "nodes": [record], "pageInfo": {"hasNextPage": False, "endCursor": "x"}, + }}) + [doc] = _collect(session, teams=["t"], include_comments=False) + assert doc.text.startswith("# DES-7: Logo") + assert "Done" in doc.text and "kim" in doc.text and "brand" in doc.text + assert doc.extra_info["file_path"] == "DES/DES-7.md" + + def test_argument_names_follow_the_tools_schema(self): + schemas = {**SCHEMAS, "list_issues": {"properties": {"teamId": {}, "first": {"maximum": 50}, "after": {}}}} + pages = {None: {"issues": [ISSUE], "hasNextPage": True, "cursor": "p2"}, "p2": {"issues": []}} + session = FakeLinear({"list_issues": lambda args: pages[args.get("after")]}, schemas) + _collect(session, teams=["team-1"], include_comments=False) + assert session.called("list_issues") == [ + {"teamId": "team-1", "first": 50}, {"teamId": "team-1", "first": 50, "after": "p2"}, + ] + + def test_a_tool_that_cannot_filter_by_team_syncs_nothing(self): + schemas = {**SCHEMAS, "list_issues": {"properties": {"query": {}, "limit": {}}}} + session = FakeLinear({"list_issues": lambda args: pytest.fail("must not list the whole workspace")}, schemas) + with pytest.raises(linear.LinearSyncError, match="team"): + _collect(session, teams=["team-1"]) + + def test_a_missing_tool_is_an_error(self): + session = FakeLinear({}, schemas={}) + with pytest.raises(linear.LinearSyncError, match="list_issues"): + _collect(session, teams=["team-1"]) + + +class TestDocuments: + def test_documents_of_the_picked_projects(self): + session = FakeLinear({ + "list_issues": lambda args: {"issues": []}, + "list_documents": lambda args: {"documents": [ + {"id": "d1", "title": "Launch plan", "url": "https://linear.app/acme/document/launch-plan-d1"}, + ]}, + "get_document": lambda args: {"id": "d1", "title": "Launch plan", "content": "We ship on Monday.", + "url": "https://linear.app/acme/document/launch-plan-d1"}, + }) + [doc] = _collect(session, projects=[{"id": "project-1", "name": "Acme"}], include_documents=True) + assert doc.text.startswith("# Launch plan") + assert "We ship on Monday." in doc.text + assert doc.extra_info["source"] == "https://linear.app/acme/document/launch-plan-d1" + assert doc.extra_info["file_path"] == "Acme/Documents/Launch plan.md" + assert session.called("list_documents") == [{"projectId": "project-1", "limit": 100}] + + def test_a_listed_document_with_its_content_is_not_read_again(self): + session = FakeLinear({ + "list_issues": lambda args: [], + "list_documents": lambda args: [{"id": "d1", "title": "Notes", "content": "All here."}], + }) + [doc] = _collect(session, projects=["project-1"], include_documents=True) + assert "All here." in doc.text + assert session.called("get_document") == [] + + def test_documents_with_one_title_stay_two_files(self): + session = FakeLinear({ + "list_issues": lambda args: [], + "list_documents": lambda args: [ + {"id": "d1aaaaaaaa", "title": "Notes", "content": "One."}, + {"id": "d2bbbbbbbb", "title": "Notes", "content": "Two."}, + ], + }) + docs = _collect(session, projects=[{"id": "p1", "name": "Acme"}], include_documents=True) + assert [d.extra_info["file_path"] for d in docs] == [ + "Acme/Documents/Notes.md", "Acme/Documents/Notes (d2bbbbbb).md", + ] + + def test_documents_are_left_out_when_not_wanted(self): + session = FakeLinear({"list_issues": lambda args: []}) + assert _collect(session, projects=["project-1"]) == [] + assert session.called("list_documents") == [] + + +class TestSelection: + def test_ids_or_records_are_kept_once(self): + selection = linear.normalize_selection({ + "teams": ["t1", {"id": "t1", "key": "ENG", "name": "Engineering"}, {"id": "t2"}, "", None], + "projects": [{"id": "p1", "name": "Acme"}], + "include_comments": "false", + "include_documents": True, + "unrelated": "dropped", + }) + assert selection == { + "teams": [{"id": "t1", "key": "ENG", "name": "Engineering"}, {"id": "t2", "key": "", "name": ""}], + "projects": [{"id": "p1", "name": "Acme"}], + "include_comments": False, + "include_documents": True, + } + + def test_comments_are_on_by_default(self): + assert linear.normalize_selection({"teams": ["t1"]})["include_comments"] is True + + def test_something_must_be_picked(self): + with pytest.raises(ValueError): + linear.normalize_selection({"teams": [], "projects": []}) + + def test_a_json_string_is_read(self): + assert linear.normalize_selection('{"teams": ["t1"]}')["teams"] == [{"id": "t1", "key": "", "name": ""}] + + def test_default_name_lists_what_was_picked(self): + selection = linear.normalize_selection({ + "teams": [{"id": "t1", "name": "Engineering"}], "projects": [{"id": "p1", "name": "Acme"}], + }) + assert linear.selection_name(selection) == "Linear · Engineering, Acme" + assert linear.selection_name(linear.normalize_selection({"teams": ["t1"]})) == "Linear" + + +class TestLoadData: + def test_reads_with_the_sources_connection(self): + connection = {"id": "c1", "user_id": "alice", "server_url": "https://mcp.linear.app"} + loader = LinearLoader() + with patch("docsgpt.parser.remote.linear_loader._connection", return_value=connection) as load, \ + patch("docsgpt.parser.remote.linear_loader.run_connection_session", + return_value=["doc"]) as run: + result = loader.load_data({"teams": ["t1"], "connection_id": "c1"}) + assert result == ["doc"] + load.assert_called_once_with("c1") + assert run.call_args.args[:2] == (connection, "https://mcp.linear.app/mcp") + + def test_needs_a_connection(self): + with pytest.raises(ValueError): + LinearLoader().load_data({"teams": ["t1"]}) + + +class TestOneBadItemDoesNotStopTheSync: + """An issue deleted since it was listed, or comments the token cannot + read, lose that detail; the rest of the sync still lands.""" + + @staticmethod + def _refuse(args): + from docsgpt.connectors.mcp import MCPToolError + + raise MCPToolError("Entity not found") + + def test_an_issue_that_cannot_be_read_in_full_keeps_what_was_listed(self): + clipped = {**ISSUE, "description": "The spinner (truncated, use get_issue to read the full description)"} + session = FakeLinear({ + "list_issues": lambda args: {"issues": [clipped, {**ISSUE, "id": "ENG-2", "title": "Other"}]}, + "get_issue": self._refuse, + }) + docs = _collect(session, teams=["team-1"], include_comments=False) + assert [d.text.splitlines()[0] for d in docs] == ["# ENG-1310: Fix the loader", "# ENG-2: Other"] + + def test_comments_that_cannot_be_read_are_left_out(self): + session = FakeLinear({"list_issues": lambda args: [ISSUE], "list_comments": self._refuse}) + [doc] = _collect(session, teams=["team-1"]) + assert "## Comments" not in doc.text + + def test_a_document_that_cannot_be_read_is_skipped(self): + session = FakeLinear({ + "list_issues": lambda args: {"issues": []}, + "list_documents": lambda args: {"documents": [ + {"id": "d1", "title": "Gone"}, + {"id": "d2", "title": "Plan", "content": "We ship on Monday."}, + ]}, + "get_document": self._refuse, + }) + docs = _collect(session, projects=[{"id": "project-1", "name": "Acme"}], include_documents=True) + assert [d.text.splitlines()[0] for d in docs] == ["# Plan"] diff --git a/tests/parser/remote/test_share_point_loader.py b/tests/parser/remote/test_share_point_loader.py index 07944ee6..5ee13392 100644 --- a/tests/parser/remote/test_share_point_loader.py +++ b/tests/parser/remote/test_share_point_loader.py @@ -66,7 +66,8 @@ class TestSharePointLoaderLoadFileById: """Test _load_file_by_id method.""" @patch("docsgpt.parser.connectors.share_point.loader.requests.get") - @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.get_token_info_from_session") + @patch("docsgpt.connectors.service.connection_id_for_session_token", new=lambda token: "conn-1") + @patch("docsgpt.connectors.service.get_valid_token_info") @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.__init__", return_value=None) @patch("docsgpt.parser.connectors.share_point.loader.SharePointLoader._ensure_valid_token") def test_load_file_by_id_includes_size_in_select(self, mock_ensure_token, mock_auth_init, mock_get_token, mock_get): @@ -97,7 +98,8 @@ class TestSharePointLoaderLoadFileById: assert "size" in params["$select"] @patch("docsgpt.parser.connectors.share_point.loader.requests.get") - @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.get_token_info_from_session") + @patch("docsgpt.connectors.service.connection_id_for_session_token", new=lambda token: "conn-1") + @patch("docsgpt.connectors.service.get_valid_token_info") @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.__init__", return_value=None) @patch("docsgpt.parser.connectors.share_point.loader.SharePointLoader._ensure_valid_token") def test_load_file_by_id_returns_document_with_size(self, mock_ensure_token, mock_auth_init, mock_get_token, mock_get): @@ -134,7 +136,8 @@ class TestSharePointLoaderListItems: """Test _list_items_in_parent method.""" @patch("docsgpt.parser.connectors.share_point.loader.requests.get") - @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.get_token_info_from_session") + @patch("docsgpt.connectors.service.connection_id_for_session_token", new=lambda token: "conn-1") + @patch("docsgpt.connectors.service.get_valid_token_info") @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.__init__", return_value=None) @patch("docsgpt.parser.connectors.share_point.loader.SharePointLoader._ensure_valid_token") def test_list_items_includes_size_in_select(self, mock_ensure_token, mock_auth_init, mock_get_token, mock_get): @@ -169,7 +172,8 @@ class TestSharePointLoaderListItems: assert "size" in params["$select"] @patch("docsgpt.parser.connectors.share_point.loader.requests.get") - @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.get_token_info_from_session") + @patch("docsgpt.connectors.service.connection_id_for_session_token", new=lambda token: "conn-1") + @patch("docsgpt.connectors.service.get_valid_token_info") @patch("docsgpt.parser.connectors.share_point.loader.SharePointAuth.__init__", return_value=None) @patch("docsgpt.parser.connectors.share_point.loader.SharePointLoader._ensure_valid_token") def test_list_items_folders_include_size(self, mock_ensure_token, mock_auth_init, mock_get_token, mock_get): diff --git a/tests/security/test_encryption_v2.py b/tests/security/test_encryption_v2.py new file mode 100644 index 00000000..1551c45b --- /dev/null +++ b/tests/security/test_encryption_v2.py @@ -0,0 +1,94 @@ +"""Tests for the v2 credential envelope.""" + +from __future__ import annotations + +import base64 + +import pytest + +from docsgpt.security import encryption as enc + + +@pytest.fixture(autouse=True) +def _keys(monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "current-key-for-tests") + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY_PREVIOUS", None) + yield + + +class TestEnvelope: + def test_round_trip(self): + blob = enc.encrypt_json({"access_token": "at", "n": 1}, "alice") + assert blob.startswith("v2:") + assert enc.decrypt_json(blob, "alice") == {"access_token": "at", "n": 1} + + def test_plaintext_never_in_blob(self): + blob = enc.encrypt_json({"refresh_token": "very-secret-refresh"}, "alice") + assert "very-secret-refresh" not in blob + assert "very-secret-refresh".encode() not in base64.b64decode(blob.split(":", 2)[2]) + + def test_each_blob_is_unique(self): + assert enc.encrypt_json({"a": 1}, "alice") != enc.encrypt_json({"a": 1}, "alice") + + def test_bound_to_owner(self): + blob = enc.encrypt_json({"a": 1}, "alice") + with pytest.raises(enc.CredentialDecryptionError): + enc.decrypt_json(blob, "bob") + + def test_tampering_is_detected(self): + blob = enc.encrypt_json({"a": 1}, "alice") + prefix, key_id, payload = blob.split(":", 2) + raw = bytearray(base64.b64decode(payload)) + raw[-1] ^= 0x01 + tampered = f"{prefix}:{key_id}:{base64.b64encode(bytes(raw)).decode()}" + with pytest.raises(enc.CredentialDecryptionError): + enc.decrypt_json(tampered, "alice") + + @pytest.mark.parametrize("blob", ["", "v1:abc", "v2:only-two", "v2:deadbeef:%%%not-base64"]) + def test_malformed(self, blob): + with pytest.raises(enc.CredentialDecryptionError): + enc.decrypt_json(blob, "alice") + + def test_key_id_names_the_key(self): + blob = enc.encrypt_json({"a": 1}, "alice") + assert enc.envelope_key_id(blob) == enc.current_key_id() + assert enc.envelope_key_id("not an envelope") is None + + +class TestRotation: + def test_unknown_key_fails(self, monkeypatch): + from docsgpt.core.settings import settings + + blob = enc.encrypt_json({"a": 1}, "alice") + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "a-new-key") + with pytest.raises(enc.CredentialDecryptionError): + enc.decrypt_json(blob, "alice") + + def test_previous_key_still_decrypts(self, monkeypatch): + from docsgpt.core.settings import settings + + blob = enc.encrypt_json({"a": 1}, "alice") + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", "a-new-key") + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY_PREVIOUS", "current-key-for-tests") + assert enc.decrypt_json(blob, "alice") == {"a": 1} + assert enc.envelope_key_id(enc.encrypt_json({"a": 1}, "alice")) == enc.current_key_id() + assert enc.envelope_key_id(blob) != enc.current_key_id() + + +class TestDefaultKey: + def test_detects_default(self, monkeypatch): + from docsgpt.core.settings import settings + + monkeypatch.setattr(settings, "ENCRYPTION_SECRET_KEY", enc.DEFAULT_ENCRYPTION_KEY) + assert enc.is_default_encryption_key() + + def test_custom_key(self): + assert not enc.is_default_encryption_key() + + +class TestLegacyV1Unchanged: + def test_v1_round_trip_still_works(self): + blob = enc.encrypt_credentials({"token": "t"}, "alice") + assert enc.decrypt_credentials(blob, "alice") == {"token": "t"} diff --git a/tests/storage/db/test_migration_0040.py b/tests/storage/db/test_migration_0040.py new file mode 100644 index 00000000..8cf924c3 --- /dev/null +++ b/tests/storage/db/test_migration_0040.py @@ -0,0 +1,451 @@ +"""Migration tests for 0040_connections: columns, backfill and round trip.""" + +from __future__ import annotations + +import json +import os +import subprocess +import sys +from pathlib import Path + +import pytest +from sqlalchemy import text + + +pytestmark = pytest.mark.integration + +_0037 = "0039_resource_sponsors" # the revision before 0040_connections + + +def _alembic_ini() -> Path: + return Path(__file__).resolve().parents[3] / "docsgpt" / "alembic.ini" + + +def _run_alembic(url: str, *args: str) -> None: + subprocess.check_call( + [sys.executable, "-m", "alembic", "-c", str(_alembic_ini()), *args], + timeout=120, + env={**os.environ, "POSTGRES_URI": url}, + ) + + +def _columns(conn, table: str) -> set[str]: + rows = conn.execute( + text("SELECT column_name FROM information_schema.columns WHERE table_name = :t"), + {"t": table}, + ).fetchall() + return {r[0] for r in rows} + + +def _seed_legacy(conn) -> dict: + """Rows as a pre-0040 install left them.""" + ids = {} + ids["drive"] = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, session_token, user_email, status, token_info) " + "VALUES ('alice', 'google_drive', 'tok-a', 'alice@example.com', 'authorized', " + "CAST(:ti AS jsonb)) RETURNING id" + ), + {"ti": json.dumps({"access_token": "at", "refresh_token": "rt"})}, + ).scalar() + ids["mcp"] = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, server_url, session_data) " + "VALUES ('alice', 'mcp:https://mcp.example.com', 'https://mcp.example.com', CAST(:sd AS jsonb)) " + "RETURNING id" + ), + {"sd": json.dumps({"tokens": {"access_token": "m"}})}, + ).scalar() + ids["source"] = conn.execute( + text( + "INSERT INTO sources (user_id, name, type, remote_data) " + "VALUES ('alice', 'Handbook', 'connector:file', CAST(:rd AS jsonb)) RETURNING id" + ), + {"rd": json.dumps({"provider": "google_drive", "file_ids": ["f1"]})}, + ).scalar() + ids["other_source"] = conn.execute( + text( + "INSERT INTO sources (user_id, name, type, remote_data) " + "VALUES ('bob', 'Bob files', 'connector:file', CAST(:rd AS jsonb)) RETURNING id" + ), + {"rd": json.dumps({"provider": "google_drive"})}, + ).scalar() + ids["mcp_tool"] = conn.execute( + text( + "INSERT INTO user_tools (user_id, name, config) " + "VALUES ('alice', 'mcp_tool', CAST(:c AS jsonb)) RETURNING id" + ), + {"c": json.dumps({"server_url": "https://mcp.example.com/mcp", "auth_type": "oauth"})}, + ).scalar() + ids["bearer_tool"] = conn.execute( + text( + "INSERT INTO user_tools (user_id, name, config) " + "VALUES ('alice', 'mcp_tool', CAST(:c AS jsonb)) RETURNING id" + ), + {"c": json.dumps({"server_url": "https://mcp.example.com/mcp", "auth_type": "bearer"})}, + ).scalar() + return ids + + +class TestMigration0040: + def test_head_has_connection_columns(self, pg_engine): + with pg_engine.connect() as conn: + assert {"connector_key", "display_name", "account_label", "auth_kind", "updated_at"} <= _columns( + conn, "connector_sessions" + ) + assert "connection_id" in _columns(conn, "sources") + assert "connection_id" in _columns(conn, "user_tools") + assert {"connector_key", "enabled", "credential_mode"} <= _columns(conn, "connector_policies") + + def test_backfill_links_legacy_rows(self, pg_engine): + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0037) + with pg_engine.begin() as conn: + ids = _seed_legacy(conn) + _run_alembic(url, "upgrade", "head") + + with pg_engine.connect() as conn: + drive = conn.execute( + text( + "SELECT connector_key, auth_kind, display_name, account_label " + "FROM connector_sessions WHERE id = :id" + ), + {"id": ids["drive"]}, + ).one() + assert tuple(drive) == ("google_drive", "oauth", "Google Drive", "alice@example.com") + + mcp = conn.execute( + text("SELECT connector_key, auth_kind, display_name FROM connector_sessions WHERE id = :id"), + {"id": ids["mcp"]}, + ).one() + assert tuple(mcp) == ("custom_mcp", "mcp_oauth", "mcp.example.com") + + linked = conn.execute( + text("SELECT connection_id FROM sources WHERE id = :id"), {"id": ids["source"]} + ).scalar() + assert linked == ids["drive"] + # Bob has no session for the provider: nothing to link. + assert ( + conn.execute( + text("SELECT connection_id FROM sources WHERE id = :id"), {"id": ids["other_source"]} + ).scalar() + is None + ) + tool = conn.execute( + text("SELECT connection_id FROM user_tools WHERE id = :id"), {"id": ids["mcp_tool"]} + ).scalar() + assert tool == ids["mcp"] + # A bearer MCP tool has no OAuth session to point at. + bearer = conn.execute( + text("SELECT connection_id FROM user_tools WHERE id = :id"), {"id": ids["bearer_tool"]} + ).scalar() + assert bearer is None + + def test_downgrade_then_upgrade_is_idempotent(self, pg_engine): + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0037) + with pg_engine.connect() as conn: + assert "connector_key" not in _columns(conn, "connector_sessions") + assert "connection_id" not in _columns(conn, "sources") + _run_alembic(url, "upgrade", "head") + _run_alembic(url, "downgrade", _0037) + _run_alembic(url, "upgrade", "head") + with pg_engine.connect() as conn: + assert "connector_key" in _columns(conn, "connector_sessions") + assert "connection_id" in _columns(conn, "user_tools") + + +def _seed_secrets(conn) -> dict: + """Plaintext tokens and v1 tool secrets, as a pre-0040 install stored them.""" + from docsgpt.security.encryption import encrypt_credentials + + ids = {} + ids["drive"] = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, session_token, user_email, status, token_info) " + "VALUES ('carol', 'google_drive', 'tok-c', 'carol@example.com', 'authorized', CAST(:ti AS jsonb)) " + "RETURNING id" + ), + {"ti": json.dumps({"access_token": "plain-at", "refresh_token": "plain-rt", "scopes": ["drive"]})}, + ).scalar() + ids["mcp"] = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, server_url, session_data) " + "VALUES ('carol', 'mcp:https://mcp.notion.com', 'https://mcp.notion.com', CAST(:sd AS jsonb)) " + "RETURNING id" + ), + {"sd": json.dumps({ + "tokens": {"access_token": "mcp-at", "refresh_token": "mcp-rt"}, + "client_info": {"client_id": "cid", "client_secret": "dcr-secret"}, + "other": 1, + })}, + ).scalar() + ids["mcp_pending"] = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, server_url, session_data) " + "VALUES ('carol', 'mcp:https://half.example.com', 'https://half.example.com', CAST(:sd AS jsonb)) " + "RETURNING id" + ), + {"sd": json.dumps({"client_info": {"client_id": "x"}})}, + ).scalar() + for name, token in (("telegram_a", "111111:AAAAAAAA"), ("telegram_b", "111111:AAAAAAAA")): + ids[name] = conn.execute( + text("INSERT INTO user_tools (user_id, name, config) VALUES ('carol', 'telegram', CAST(:c AS jsonb)) " + "RETURNING id"), + {"c": json.dumps({"encrypted_credentials": encrypt_credentials({"token": token}, "carol")})}, + ).scalar() + ids["mcp_key_tool"] = conn.execute( + text("INSERT INTO user_tools (user_id, name, config) VALUES ('carol', 'mcp_tool', CAST(:c AS jsonb)) " + "RETURNING id"), + {"c": json.dumps({ + "server_url": "https://tools.example.com/mcp", "auth_type": "bearer", + "encrypted_credentials": encrypt_credentials({"bearer_token": "bearer-secret-9999"}, "carol"), + })}, + ).scalar() + ids["mcp_oauth_tool"] = conn.execute( + text("INSERT INTO user_tools (user_id, name, config) VALUES ('carol', 'mcp_tool', CAST(:c AS jsonb)) " + "RETURNING id"), + {"c": json.dumps({"server_url": "https://mcp.notion.com/mcp", "auth_type": "oauth"})}, + ).scalar() + return ids + + +class TestMigration0040Credentials: + def _upgrade_with(self, pg_engine, seed): + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0037) + with pg_engine.begin() as conn: + ids = seed(conn) + _run_alembic(url, "upgrade", "head") + return url, ids + + def test_tokens_are_encrypted_and_plaintext_removed(self, pg_engine): + from docsgpt.connectors.service import read_secrets + + _, ids = self._upgrade_with(pg_engine, _seed_secrets) + with pg_engine.connect() as conn: + rows = { + r.id: dict(r._mapping) + for r in conn.execute(text("SELECT * FROM connector_sessions WHERE user_id = 'carol'")) + } + dump = json.dumps([ + {k: v for k, v in r.items() if k not in ("encrypted_credentials",)} for r in rows.values() + ], default=str) + # Nothing readable is left outside the envelope. + for secret in ("plain-at", "plain-rt", "mcp-at", "mcp-rt", "dcr-secret", "AAAAAAAA", "bearer-secret"): + assert secret not in dump + drive = rows[ids["drive"]] + assert drive["token_info"] is None + assert drive["has_refresh_token"] is True + assert drive["scopes"] == ["drive"] + assert drive["status"] == "connected" + assert read_secrets(drive)["token_info"]["refresh_token"] == "plain-rt" + mcp = rows[ids["mcp"]] + assert mcp["session_data"] == {"other": 1} + assert mcp["status"] == "connected" + assert read_secrets(mcp)["client_info"]["client_secret"] == "dcr-secret" + assert rows[ids["mcp_pending"]]["status"] == "pending" + + def test_api_key_tools_share_one_connection(self, pg_engine): + from docsgpt.connectors.service import read_secrets + + _, ids = self._upgrade_with(pg_engine, _seed_secrets) + with pg_engine.connect() as conn: + links = dict(conn.execute( + text("SELECT id, connection_id FROM user_tools WHERE user_id = 'carol'") + ).fetchall()) + assert links[ids["telegram_a"]] is not None + assert links[ids["telegram_a"]] == links[ids["telegram_b"]] + telegram = dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = :i"), {"i": links[ids["telegram_a"]]} + ).one()._mapping) + assert telegram["auth_kind"] == "api_key" + assert telegram["account_label"] == "…AAAA" + assert read_secrets(telegram) == {"credentials": {"token": "111111:AAAAAAAA"}} + bearer = dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = :i"), {"i": links[ids["mcp_key_tool"]]} + ).one()._mapping) + assert (bearer["connector_key"], bearer["server_url"]) == ("custom_mcp", "https://tools.example.com") + # Rollback safety: the tool keeps its v1 copy for a release. + config = conn.execute( + text("SELECT config FROM user_tools WHERE id = :i"), {"i": ids["telegram_a"]} + ).scalar() + assert "encrypted_credentials" in config + + def test_different_keys_with_the_same_hint_get_their_own_connections(self, pg_engine): + from docsgpt.connectors.service import read_secrets + from docsgpt.security.encryption import encrypt_credentials + + def seed(conn): + ids = {} + for name, token in (("a", "111111:SAMEEND1"), ("b", "222222:SAMEEND1")): + ids[name] = conn.execute( + text("INSERT INTO user_tools (user_id, name, config) VALUES ('frank', 'telegram', " + "CAST(:c AS jsonb)) RETURNING id"), + {"c": json.dumps({"encrypted_credentials": encrypt_credentials({"token": token}, "frank")})}, + ).scalar() + return ids + + _, ids = self._upgrade_with(pg_engine, seed) + with pg_engine.connect() as conn: + links = dict(conn.execute( + text("SELECT id, connection_id FROM user_tools WHERE user_id = 'frank'") + ).fetchall()) + assert links[ids["a"]] != links[ids["b"]] + for name, token in (("a", "111111:SAMEEND1"), ("b", "222222:SAMEEND1")): + row = dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = :i"), {"i": links[ids[name]]} + ).one()._mapping) + assert read_secrets(row) == {"credentials": {"token": token}} + + def test_non_object_token_info_is_kept_in_the_envelope(self, pg_engine): + from docsgpt.connectors.service import read_secrets + from docsgpt.security.encryption import decrypt_json + + values = { + # An object stored as a JSON string by an older writer. + "gina": json.dumps(json.dumps({"access_token": "dbl-at", "refresh_token": "dbl-rt", "scopes": "a b"})), + "hank": json.dumps("opaque-token"), + "ivan": "null", + } + + def seed(conn): + return { + user: conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, session_token, status, token_info) " + "VALUES (:u, 'google_drive', :t, 'authorized', CAST(:ti AS jsonb)) RETURNING id" + ), + {"u": user, "t": f"tok-{user}", "ti": raw}, + ).scalar() + for user, raw in values.items() + } + + url, ids = self._upgrade_with(pg_engine, seed) + with pg_engine.connect() as conn: + rows = { + user: dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = :i"), {"i": row_id} + ).one()._mapping) + for user, row_id in ids.items() + } + plaintext = conn.execute( + text("SELECT count(*) FROM connector_sessions WHERE id = ANY(:ids) AND token_info IS NOT NULL"), + {"ids": list(ids.values())}, + ).scalar() + assert plaintext == 0 + gina = rows["gina"] + assert read_secrets(gina)["token_info"]["refresh_token"] == "dbl-rt" + assert gina["has_refresh_token"] is True + assert gina["scopes"] == ["a", "b"] + assert decrypt_json(rows["hank"]["encrypted_credentials"], "hank")["legacy_token_info"] == "opaque-token" + assert "token_info" not in read_secrets(rows["hank"]) + + _run_alembic(url, "downgrade", _0037) + with pg_engine.connect() as conn: + restored = dict(conn.execute( + text("SELECT id, token_info::text FROM connector_sessions WHERE id = ANY(:ids)"), + {"ids": list(ids.values())}, + ).fetchall()) + assert json.loads(restored[ids["gina"]])["access_token"] == "dbl-at" + assert restored[ids["hank"]] == '"opaque-token"' + assert restored[ids["ivan"]] == "null" + _run_alembic(url, "upgrade", "head") + + def test_oauth_mcp_tools_keep_member_credentials(self, pg_engine): + _, ids = self._upgrade_with(pg_engine, _seed_secrets) + with pg_engine.connect() as conn: + modes = dict(conn.execute( + text("SELECT id, credential_mode FROM user_tools WHERE user_id = 'carol'") + ).fetchall()) + assert modes[ids["mcp_oauth_tool"]] == "member" + assert modes[ids["telegram_a"]] == "owner" + + def test_multiple_accounts_per_provider_allowed(self, pg_engine): + with pg_engine.begin() as conn: + for label in ("a@example.com", "b@example.com"): + conn.execute( + text("INSERT INTO connector_sessions (user_id, provider, account_label, status) " + "VALUES ('dan', 'google_drive', :l, 'connected')"), + {"l": label}, + ) + with pg_engine.connect() as conn: + assert conn.execute( + text("SELECT count(*) FROM connector_sessions WHERE user_id = 'dan'") + ).scalar() == 2 + + def test_downgrade_restores_plaintext(self, pg_engine): + url, ids = self._upgrade_with(pg_engine, _seed_secrets) + _run_alembic(url, "downgrade", _0037) + with pg_engine.connect() as conn: + token_info = conn.execute( + text("SELECT token_info FROM connector_sessions WHERE id = :i"), {"i": ids["drive"]} + ).scalar() + session_data = conn.execute( + text("SELECT session_data FROM connector_sessions WHERE id = :i"), {"i": ids["mcp"]} + ).scalar() + api_rows = conn.execute( + text("SELECT count(*) FROM connector_sessions WHERE provider = 'telegram'") + ).scalar() + assert token_info["refresh_token"] == "plain-rt" + assert session_data["tokens"]["access_token"] == "mcp-at" + assert api_rows == 0 + _run_alembic(url, "upgrade", "head") + + def test_downgrade_keeps_secrets_of_tools_made_after_upgrade(self, pg_engine): + from docsgpt.security.encryption import decrypt_credentials, encrypt_json + + url, _ = self._upgrade_with(pg_engine, _seed_secrets) + # A tool added through the wizard keeps its key only on the connection. + with pg_engine.begin() as conn: + connection_id = conn.execute( + text( + "INSERT INTO connector_sessions (user_id, provider, connector_key, auth_kind, status, " + "account_label, encrypted_credentials, session_data) VALUES ('erin', 'ntfy', 'ntfy', " + "'api_key', 'connected', '…9999', :blob, '{}'::jsonb) RETURNING id" + ), + {"blob": encrypt_json({"credentials": {"token": "ntfy-secret-9999"}}, "erin")}, + ).scalar() + tool_id = conn.execute( + text("INSERT INTO user_tools (user_id, name, config, connection_id) " + "VALUES ('erin', 'ntfy', CAST(:c AS jsonb), :cid) RETURNING id"), + {"c": json.dumps({"server_url": "https://ntfy.sh"}), "cid": connection_id}, + ).scalar() + _run_alembic(url, "downgrade", _0037) + with pg_engine.connect() as conn: + config = conn.execute(text("SELECT config FROM user_tools WHERE id = :i"), {"i": tool_id}).scalar() + assert config["server_url"] == "https://ntfy.sh" + assert decrypt_credentials(config["encrypted_credentials"], "erin") == {"token": "ntfy-secret-9999"} + _run_alembic(url, "upgrade", "head") + with pg_engine.connect() as conn: + relinked = conn.execute( + text("SELECT connection_id FROM user_tools WHERE id = :i"), {"i": tool_id} + ).scalar() + assert relinked is not None + + def test_second_upgrade_is_a_no_op(self, pg_engine): + from docsgpt.connectors.service import read_secrets + + url, ids = self._upgrade_with(pg_engine, _seed_secrets) + _run_alembic(url, "downgrade", _0037) + _run_alembic(url, "upgrade", "head") + with pg_engine.connect() as conn: + drive = dict(conn.execute( + text("SELECT * FROM connector_sessions WHERE id = :i"), {"i": ids["drive"]} + ).one()._mapping) + count = conn.execute( + text("SELECT count(*) FROM connector_sessions WHERE provider = 'telegram'") + ).scalar() + assert read_secrets(drive)["token_info"]["access_token"] == "plain-at" + assert count == 1 + + +def test_model_metadata_matches_the_policy_and_custom_model_columns(): + """The SQLAlchemy tables mirror the migrations for both ``enabled`` columns.""" + from docsgpt.storage.db.models import connector_policies_table, user_custom_models_table + + policy_enabled = connector_policies_table.c.enabled + assert policy_enabled.nullable is True and policy_enabled.server_default is None + model_enabled = user_custom_models_table.c.enabled + assert model_enabled.nullable is False + assert str(model_enabled.server_default.arg) == "true" diff --git a/tests/storage/db/test_migration_0041.py b/tests/storage/db/test_migration_0041.py new file mode 100644 index 00000000..cfef75f7 --- /dev/null +++ b/tests/storage/db/test_migration_0041.py @@ -0,0 +1,50 @@ +"""Migration round-trip test for 0041_connection_account_name.""" + +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + +import pytest +from sqlalchemy import text + + +pytestmark = pytest.mark.integration + +_0039 = "0041_connection_account_name" +_0038 = "0040_connections" + + +def _run_alembic(url: str, *args: str) -> None: + ini = Path(__file__).resolve().parents[3] / "docsgpt" / "alembic.ini" + subprocess.check_call( + [sys.executable, "-m", "alembic", "-c", str(ini), *args], + timeout=120, + env={**os.environ, "POSTGRES_URI": url}, + ) + + +def _has_column(conn) -> bool: + return conn.execute( + text( + "SELECT 1 FROM information_schema.columns WHERE table_schema = 'public' " + "AND table_name = 'connector_sessions' AND column_name = 'account_name'" + ) + ).fetchone() is not None + + +class TestMigration0041RoundTrip: + def test_head_has_account_name(self, pg_engine): + with pg_engine.connect() as conn: + assert _has_column(conn) + + def test_downgrade_drops_then_upgrade_restores(self, pg_engine): + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0038) + with pg_engine.connect() as conn: + assert not _has_column(conn) + _run_alembic(url, "upgrade", "head") + with pg_engine.connect() as conn: + assert _has_column(conn) diff --git a/tests/storage/db/test_migration_0042.py b/tests/storage/db/test_migration_0042.py new file mode 100644 index 00000000..95f297c8 --- /dev/null +++ b/tests/storage/db/test_migration_0042.py @@ -0,0 +1,55 @@ +"""Migration round-trip test for 0042_schedule_created_via_api.""" + +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + +import pytest +from sqlalchemy import text +from sqlalchemy.exc import IntegrityError + + +pytestmark = pytest.mark.integration + +_0041 = "0041_connection_account_name" + + +def _run_alembic(url: str, *args: str) -> None: + ini = Path(__file__).resolve().parents[3] / "docsgpt" / "alembic.ini" + subprocess.check_call( + [sys.executable, "-m", "alembic", "-c", str(ini), *args], + timeout=120, + env={**os.environ, "POSTGRES_URI": url}, + ) + + +def _insert(conn, created_via: str) -> None: + conn.execute( + text( + "INSERT INTO schedules (user_id, trigger_type, instruction, run_at, next_run_at, created_via) " + "VALUES ('u', 'once', 'x', now(), now(), :v)" + ), + {"v": created_via}, + ) + + +class TestMigration0042RoundTrip: + def test_head_accepts_api(self, pg_engine): + with pg_engine.begin() as conn: + _insert(conn, "api") + + def test_downgrade_folds_api_into_chat_then_upgrade_restores(self, pg_engine): + with pg_engine.begin() as conn: + _insert(conn, "api") + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0041) + with pg_engine.connect() as conn: + assert conn.execute(text("SELECT count(*) FROM schedules WHERE created_via = 'api'")).scalar() == 0 + with pytest.raises(IntegrityError), pg_engine.begin() as conn: + _insert(conn, "api") + _run_alembic(url, "upgrade", "head") + with pg_engine.begin() as conn: + _insert(conn, "api") diff --git a/tests/storage/db/test_migration_0043.py b/tests/storage/db/test_migration_0043.py new file mode 100644 index 00000000..942e47fe --- /dev/null +++ b/tests/storage/db/test_migration_0043.py @@ -0,0 +1,52 @@ +"""Migration round-trip test for 0043_wiki_outside_edits.""" + +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + +import pytest +from sqlalchemy import text + + +pytestmark = pytest.mark.integration + +_0042 = "0042_schedule_created_via_api" + + +def _run_alembic(url: str, *args: str) -> None: + ini = Path(__file__).resolve().parents[3] / "docsgpt" / "alembic.ini" + subprocess.check_call( + [sys.executable, "-m", "alembic", "-c", str(ini), *args], + timeout=120, + env={**os.environ, "POSTGRES_URI": url}, + ) + + +def _column_exists(conn) -> bool: + return conn.execute( + text( + "SELECT 1 FROM information_schema.columns WHERE table_schema = 'public' " + "AND table_name = 'sources' AND column_name = 'wiki_outside_edits'" + ) + ).fetchone() is not None + + +class TestMigration0043RoundTrip: + def test_head_defaults_to_off(self, pg_engine): + with pg_engine.begin() as conn: + value = conn.execute( + text("INSERT INTO sources (user_id, name) VALUES ('u', 'w') RETURNING wiki_outside_edits") + ).scalar() + assert value is False + + def test_downgrade_drops_then_upgrade_restores(self, pg_engine): + url = pg_engine.url.render_as_string(hide_password=False) + _run_alembic(url, "downgrade", _0042) + with pg_engine.connect() as conn: + assert not _column_exists(conn) + _run_alembic(url, "upgrade", "head") + with pg_engine.connect() as conn: + assert _column_exists(conn) diff --git a/tests/test_continuation.py b/tests/test_continuation.py index 56c06b34..5b4b63d0 100644 --- a/tests/test_continuation.py +++ b/tests/test_continuation.py @@ -1010,7 +1010,7 @@ class TestResumeMarkResuming: "llm_name": "openai", "api_key": "k", "user_api_key": None, - "agent_id": None, + "agent_id": "agent-1", "agent_type": "ClassicAgent", "prompt": "", "json_schema": None, @@ -1038,7 +1038,7 @@ class TestResumeMarkResuming: # The body api_key resolves to its owning user. fake_repo = MagicMock() - fake_repo.find_by_key.return_value = {"user_id": "owner-1"} + fake_repo.find_by_key.return_value = {"id": "agent-1", "user_id": "owner-1"} @contextmanager def _fake_db_readonly(): diff --git a/tests/test_extra_coverage.py b/tests/test_extra_coverage.py index fcb3333f..4731df1c 100644 --- a/tests/test_extra_coverage.py +++ b/tests/test_extra_coverage.py @@ -44,6 +44,10 @@ def _patch_upload_db(conn): "docsgpt.api.user.sources.upload.db_session", _yield ), patch( "docsgpt.api.user.sources.upload.db_readonly", _yield + ), patch( + "docsgpt.connectors.service.db_session", _yield + ), patch( + "docsgpt.connectors.service.db_readonly", _yield ): yield @@ -58,6 +62,10 @@ def _patch_conn_db(conn): "docsgpt.api.connector.routes.db_session", _yield ), patch( "docsgpt.api.connector.routes.db_readonly", _yield + ), patch( + "docsgpt.connectors.service.db_session", _yield + ), patch( + "docsgpt.connectors.service.db_readonly", _yield ): yield @@ -210,7 +218,7 @@ class TestConnectorExtra: yield with patch( - "docsgpt.api.connector.routes.db_session", _broken + "docsgpt.connectors.service.db_readonly", _broken ), app.test_request_context( "/api/connectors/disconnect", method="POST", json={"provider": "x", "session_token": "y"}, diff --git a/tests/test_tool_approval.py b/tests/test_tool_approval.py index ee1f99b0..de9e2d91 100644 --- a/tests/test_tool_approval.py +++ b/tests/test_tool_approval.py @@ -215,6 +215,30 @@ class TestHandlerApprovalPause: ] assert len(approval_events) == 1 + def test_approval_event_carries_what_will_be_sent(self): + handler = ConcreteHandler() + agent = self._make_agent({ + "call_id": "c1", + "name": "send_msg_0", + "tool_name": "telegram", + "tool_id": "0", + "action_name": "send_msg", + "arguments": {"text": "hello", "chat_id": "666"}, + "sent_arguments": {"text": "hello", "chat_id": "111"}, + "pause_type": "awaiting_approval", + "thought_signature": None, + }) + call = ToolCall(id="c1", name="send_msg_0", arguments='{"text": "hello"}') + gen = handler.handle_tool_calls(agent, [call], {"0": {"name": "telegram"}}, []) + events = [] + try: + while True: + events.append(next(gen)) + except StopIteration: + pass + (event,) = [e for e in events if e.get("data", {}).get("status") == "awaiting_approval"] + assert event["data"]["sent_arguments"] == {"text": "hello", "chat_id": "111"} + def test_mixed_normal_and_approval(self): """First tool runs normally, second needs approval.""" handler = ConcreteHandler() diff --git a/tests/tracing/test_entry_points.py b/tests/tracing/test_entry_points.py index 218432d2..ffeb8570 100644 --- a/tests/tracing/test_entry_points.py +++ b/tests/tracing/test_entry_points.py @@ -82,7 +82,8 @@ class TestHeadless: assert trace.request_id == "run-1" assert trace.user_id == "owner-1" assert trace.status == "ok" - assert [s.kind for s in trace.spans] == ["retrieval", "agent"] + # The per-source dispatcher's span wraps its retriever's own. + assert [s.kind for s in trace.spans] == ["retrieval", "retrieval", "agent"] def test_trace_can_belong_to_the_scheduling_user(self, monkeypatch, flushed): """A schedule on a shared agent is the scheduler's run, not the owner's.""" diff --git a/tests/vectorstore/test_embeddings_local.py b/tests/vectorstore/test_embeddings_local.py index 5d61a72b..f226881b 100644 --- a/tests/vectorstore/test_embeddings_local.py +++ b/tests/vectorstore/test_embeddings_local.py @@ -413,3 +413,74 @@ class TestPoolingOverrides: with patch.object(embeddings_local.settings, "EMBEDDINGS_POOLING", "banana"): spec = embeddings_local._spec_for("some-org/plain-onnx-export") assert spec.pooling == embeddings_local._FALLBACK_POOLING + + +class TestIncompleteModelCache: + """The chunker caches only a model's ``tokenizer.json`` in the same + directory. FastEmbed treats any cached snapshot as the model and then + fails to open its ONNX graph, so the loader completes the snapshot first.""" + + @staticmethod + def _description(): + from types import SimpleNamespace + + return SimpleNamespace( + model="sentence-transformers/all-mpnet-base-v2", + model_file="onnx/model.onnx", + additional_files=[], + sources=SimpleNamespace(hf="sentence-transformers/all-mpnet-base-v2"), + ) + + def _complete(self, monkeypatch, tmp_path, cached: set, offline: str = ""): + from docsgpt.vectorstore import embeddings_local + + downloads = [] + + def hf_hub_download(repo_id, filename, cache_dir=None, local_files_only=False): + if filename not in cached: + raise FileNotFoundError(filename) + return f"{cache_dir}/{filename}" + + def snapshot_download(**kwargs): + downloads.append(kwargs) + return "/snapshot" + + monkeypatch.setenv("HF_HUB_OFFLINE", offline) + with patch("fastembed.TextEmbedding._list_supported_models", return_value=[self._description()]), \ + patch("huggingface_hub.hf_hub_download", side_effect=hf_hub_download), \ + patch("huggingface_hub.snapshot_download", side_effect=snapshot_download): + embeddings_local._complete_model_cache("sentence-transformers/all-mpnet-base-v2", str(tmp_path)) + return downloads + + def test_a_snapshot_with_only_the_tokenizer_gets_its_model(self, monkeypatch, tmp_path): + downloads = self._complete(monkeypatch, tmp_path, cached={"tokenizer.json"}) + assert len(downloads) == 1 + assert downloads[0]["repo_id"] == "sentence-transformers/all-mpnet-base-v2" + assert downloads[0]["cache_dir"] == str(tmp_path) + assert "onnx/model.onnx" in downloads[0]["allow_patterns"] + assert "tokenizer_config.json" in downloads[0]["allow_patterns"] + + def test_a_failed_repair_leaves_loading_to_fastembed(self, monkeypatch, tmp_path): + """The repair is best effort: a network error or rate limit here must + not stop FastEmbed from trying its own download.""" + from docsgpt.vectorstore import embeddings_local + + monkeypatch.delenv("HF_HUB_OFFLINE", raising=False) + with patch("fastembed.TextEmbedding._list_supported_models", return_value=[self._description()]), \ + patch("huggingface_hub.hf_hub_download", side_effect=FileNotFoundError("missing")), \ + patch("huggingface_hub.snapshot_download", side_effect=OSError("rate limited")): + embeddings_local._complete_model_cache("sentence-transformers/all-mpnet-base-v2", str(tmp_path)) + + def test_a_complete_snapshot_downloads_nothing(self, monkeypatch, tmp_path): + assert self._complete(monkeypatch, tmp_path, cached={"tokenizer.json", "onnx/model.onnx"}) == [] + + def test_offline_never_downloads(self, monkeypatch, tmp_path): + assert self._complete(monkeypatch, tmp_path, cached={"tokenizer.json"}, offline="1") == [] + + def test_loading_completes_the_cache_first(self, fake_fastembed, monkeypatch): + from docsgpt.vectorstore import embeddings_local + + calls = [] + monkeypatch.setattr(embeddings_local, "_complete_model_cache", lambda repo, cache: calls.append(repo)) + embeddings_local.EmbeddingsWrapper("huggingface_sentence-transformers/all-mpnet-base-v2") + assert calls == ["sentence-transformers/all-mpnet-base-v2"] diff --git a/tests/worker/test_agent_workers.py b/tests/worker/test_agent_workers.py index 033f97d0..59381f9a 100644 --- a/tests/worker/test_agent_workers.py +++ b/tests/worker/test_agent_workers.py @@ -75,6 +75,9 @@ class TestAgentWebhookWorker: assert captured["input"] == '{"event": "ping"}' # Webhook caller should pass endpoint='webhook'. assert captured["kwargs"].get("endpoint") == "webhook" + # The owner set the webhook up (its URL is a secret): the run keeps the + # owner's rules, and headless runs still refuse approval-gated tools. + assert not captured["kwargs"].get("external_caller") def test_missing_agent_raises( self, pg_conn, patch_worker_db, task_self, monkeypatch @@ -375,4 +378,4 @@ class TestRunAgentHeadlessFromWebhook: outcome = headless_runner.run_agent_headless(agent_config, "hello") assert outcome["answer"] == "done" - assert captured_source.get("active_docs") == source_id + assert captured_source.get("active_docs") == [source_id] diff --git a/tests/worker/test_ingest_connector.py b/tests/worker/test_ingest_connector.py index 998b6d38..2e566bc6 100644 --- a/tests/worker/test_ingest_connector.py +++ b/tests/worker/test_ingest_connector.py @@ -24,6 +24,7 @@ def _mock_connector_pipeline(monkeypatch): from docsgpt import worker fake_connector = MagicMock(name="connector") + fake_connector.connection_id = None fake_connector.download_to_directory.return_value = { "files_downloaded": 1, "empty_result": False, @@ -34,7 +35,7 @@ def _mock_connector_pipeline(monkeypatch): monkeypatch.setattr( worker.ConnectorCreator, "create_connector", - staticmethod(lambda source_type, session_token: fake_connector), + staticmethod(lambda source_type, session_token=None, connection_id=None: fake_connector), ) fake_reader = MagicMock(name="reader") diff --git a/tests/worker/test_remote_worker.py b/tests/worker/test_remote_worker.py index 4f4e0bb6..2eab9b54 100644 --- a/tests/worker/test_remote_worker.py +++ b/tests/worker/test_remote_worker.py @@ -125,7 +125,8 @@ class TestSyncWorker: captured: list[dict] = [] def _fake_sync(self, source_data, name_job, user, loader, - sync_frequency, retriever, doc_id=None, directory="temp"): + sync_frequency, retriever, doc_id=None, directory="temp", + connection_id=None): captured.append({ "name_job": name_job, "user": user, diff --git a/tests/worker/test_worker_sse_publishes.py b/tests/worker/test_worker_sse_publishes.py index f50869b5..e8a1ba1e 100644 --- a/tests/worker/test_worker_sse_publishes.py +++ b/tests/worker/test_worker_sse_publishes.py @@ -470,6 +470,7 @@ def _stub_connector_pipeline( from docsgpt import worker fake_connector = MagicMock(name="connector") + fake_connector.connection_id = None fake_connector.download_to_directory.return_value = { "files_downloaded": files_downloaded, "empty_result": empty_result, @@ -482,7 +483,7 @@ def _stub_connector_pipeline( monkeypatch.setattr( worker.ConnectorCreator, "create_connector", - staticmethod(lambda source_type, session_token: fake_connector), + staticmethod(lambda source_type, session_token=None, connection_id=None: fake_connector), ) fake_reader = MagicMock(name="reader")