A run from an agent's API key or widget acts as the agent's owner, so it
could rewrite any wiki the owner can edit. A new per-wiki setting,
wiki_outside_edits (off by default), decides whether such runs, and runs
from the agent's public link, get the wiki's edit actions. While it is off
they are offered only wiki_view, and the tool refuses writes itself after
reading the live setting. The owner changes it through the owner-only
/api/sources/<id>/wiki/settings route; tokens can read it but not change it.
A scheduled or webhook run acts as the agent's owner with no one to
approve, so a public-link user or API-key caller could have the agent
schedule a write and have it run on the owner's accounts. Runs now keep
the caller's rules: a schedule set by someone who reaches the agent only
by its public link runs as a public-link caller, one set through the API
(recorded as created_via 'api', migration 0042) and every webhook run as
an external caller, each with the agent's API write allowlist.
Migration 0040 dropped token_info values that were not objects before
clearing the plaintext column. Objects stored as JSON strings are now
decoded, and other values are kept in the envelope under
legacy_token_info so downgrade restores them.
Main's access model (team editors and viewers, edit_credentials, owner-only
OAuth servers, per-user tool preferences, resource sponsors) now applies to
connection-backed tools and sources. Our migrations are renumbered to
0040_connections and 0041_connection_account_name, after main's
0038_resource_access_settings and 0039_resource_sponsors.
Where the two sides met: MCP tools save and load their connections as the
tool owner, editors may add a key (a new connection on the owner's account)
but never rewrite an existing connection's secret, fixed values stay
owner-only, and a member's own connection is used in member mode.
Connections get an account_name (migration 0039) that the owner sets with
PATCH /api/connections/<id>; the account label stays the account's
identity, so signing in again still finds it. When someone has more than
one account of a service, its tools are listed as "Telegram · <account>"
and the model sees each account's actions under the account's name
(telegram_send_message_alerts_bot) with the account in the description,
instead of _1 and _2. Actions shared by different services are named
after the service. Tools a user renamed keep their name.
- models.py: connector_policies.enabled is the nullable column; the
earlier edit had loosened user_custom_models.enabled instead. A test
pins both against their migrations.
- Admin > Connectors saves one change at a time, so an older snapshot
can no longer overwrite a newer one.
- In member mode, a member with several connected accounts of the same
service uses the one they used (or connected) most recently, not the
oldest.
- An MCP connection's tokens only go to its own server, and a
connection_id can no longer come from a client: the MCP test/save
routes drop it and the tool executor uses only the resolved one.
- API-key connections are reused only when they hold the same
credentials. Two keys that share a hint or a label get separate
connections ("…abcd (2)"), at runtime and in the 0038 backfill.
- Reconnecting a connection whose secrets cannot be decrypted no longer
flags it from a second transaction while holding its row lock.
- Connection setup validates the sync request before claiming its
Idempotency-Key, so a corrected retry is queued.
- A request that names no connection resolves to none without opening
a database connection; fixes two tests that reached a real database.
Downgrade removes API-key connections and assumed every tool still had
its v1 copy. Tools added through the wizard after the upgrade never had
one, and a reconnect can change the key after the backfill, so write the
connection's current credentials back to each linked tool first.
Six vendor-run MCP servers (Notion, Linear, Atlassian, Sentry, Asana,
Stripe) join the catalog as presets from docsgpt/connectors/presets/
mcp.yaml; existing connections to those servers show under them.
connector_policies lets an admin turn a connector off or force whose
account every share uses, and app_metadata's
connectors.allow_custom_mcp turns custom MCP servers off. Both are
enforced on the server: new connections, OAuth sign-ins, MCP test and
save, and the tools of a disabled connector stop resolving.
GET/PUT /api/admin/connectors reads and changes them.
A shared tool's owner picks owner or member credentials with
PUT /api/connections/tools/<id>/credential-mode, and a member running
the owner's account always confirms write actions.
Tool-call events and retrieved chunks name the connector they came
from (key and display name, never the account), and tool calls keep
those fields when the conversation is reloaded.
Migration 0038 moves every stored secret (OAuth tokens, MCP OAuth
tokens and client registrations, API keys) into the connection's
encrypted envelope, links API-key tools to one connection per distinct
credential, allows several accounts per provider, and adds
credential_mode to sources and tools. OAuth MCP tools keep resolving
each member's own token, as they did before.
docsgpt.connectors.service is now the only reader of OAuth tokens:
get_valid_token_info refreshes under a row lock and persists rotated
refresh tokens, and a revoked grant flags the connection, pauses its
sources and notifies the owner. Loaders build from a connection
(BaseConnectorLoader.from_connection), so scheduled sync covers Drive,
SharePoint and Confluence sources with no browser. S3 and Reddit keys
stay on the connection instead of in remote_data.
New endpoints: POST /api/connections, /setup, /reconnect,
/picker-token, /claim, DELETE /api/connections/<id>, per-action
permissions and MCP refresh-tools. Upload, file listing, sync and
validate-session take a connection_id; session tokens keep working for
this release. The tool executor reads credentials from the resolved
connection (owner or member mode) and pauses on a Connect card when a
connection needs signing in. docsgpt connectors reencrypt rewrites
stored credentials after a key rotation.
connector_sessions already holds one row per signed-in account or MCP
server. Migration 0038 names each row (connector_key, auth_kind,
display_name, account_label) and lets sources and tools point at the
row they use through connection_id, backfilling the links it can prove.
docsgpt/connectors/catalog.py declares every service DocsGPT connects
to, with the server settings it needs and what a connection provides.
New endpoints: GET /api/connectors/catalog, GET /api/connections,
GET /api/connections/<id> and POST /api/connections/<id>/disconnect.
/api/available_tools now groups tools into built_in, service and
custom, and source listings carry connectionId.
chunks is a total per request, split across the attached sources, so an
agent with two sources and the default of 2 got a single chunk from each,
and its answers changed with whichever chunk won. 6 gives three per
source for about 4-5k more input tokens per retrieval turn.
Every literal default moves from 2 to 6: the request default, the
retrievers, the internal search tool, workflow agent nodes, scheduled and
headless runs, agent create/update/import, the source retrieval config and
the frontend forms. Existing agents and sources keep what they store; a
source saved with chunks=2 now counts as configured at 2, which is pinned
by a test.
Headless runs also read chunks=0 as unset (`or 2`), so an agent with
retrieval switched off retrieved anyway on scheduled runs; 0 now stays 0.
Only message-linked traces cascaded, so scheduled runs, stateless /v1
rounds and unreserved turns kept their previews after the conversation was
deleted; they are now deleted by conversation_id. The Logs search and graph
branches filter by source, so the listing indexes lead with it.
One row per execution holding its span tree as JSONB, linked to Logs rows
by request, message, activity and workflow-run ids. message_id cascades so
deleting or truncating a conversation drops its traces; a daily beat task
enforces TRACES_RETENTION_DAYS.
The DEFAULT 'unknown' added in the last pass stopped a previous-release
process from raising NotNullViolation mid-rollout, but it threw the
attribution away to do it. The highest-volume auth_events writers are OIDC
login and silent renewal, where the actor is simply the user, so a rollout
window would have flattened exactly the rows that were trivially recoverable.
Lifts both derivation rules into SQL functions -- auth_events_derive_actor
and auth_events_derive_target -- so the backfill and a BEFORE INSERT trigger
share one definition rather than two copies of the same CASE drifting apart.
The trigger guards on NEW.actor_id IS NULL, which identifies a legacy insert
exactly because the repository always supplies one. That distinction matters
for target_id: a current writer sets it NULL deliberately for events with no
user target, and deriving there would name the actor as their own target.
Kept rather than scheduled for removal: it is a no-op on the path the
repository takes, and it keeps the column's contract true for any writer that
bypasses it.
Correctness
- /api/remote never recorded source.created, so URL, GitHub and connector
sources had a source.deleted with no matching creation. All three creation
paths now go through one _audit_source_created helper.
- The prompt-cache rate divided cached tokens by a whole bucket's prompt
tokens. A bucket is a day and mixes calls whose provider reports a cache
breakdown with calls whose provider does not, so filtering buckets in the
client could not separate them and the rate was understated by however much
traffic ran on a non-reporting provider. The denominator is now computed in
SQL over the reporting rows.
- The outcome pill matched values nothing writes. Guardrails emit triggered /
not_evaluated and the device feed emits dispatched; the map had blocked /
denied / allowed, so a guardrail that fired rendered neutral grey -- the one
signal the merged feed exists to surface. Fixtures were seeding the
fictional values, so the tests passed on it too.
- Stream duration_ms timed the consumer. stream_token_usage is a generator,
so start-to-exhaustion includes the agent loop's tool handling and the SSE
client's pace; a slow browser recorded ~30s for a sub-second call. It now
accumulates only the time spent inside next().
Safety
- Activity filters failed open: an unknown facet or unparseable timestamp was
dropped, and no filter means every row, so a typo widened an audit view and
on the export streamed the full history. Both are now a 400.
- The search term was interpolated into an ILIKE pattern, so "100%" matched
everything and "q1_report" matched more than it should. Escaped.
- 0034 set actor_id NOT NULL with no default. A previous-release process
inserting mid-rollout would raise, and in admin/routes.py that insert shares
the request transaction, so a role grant beside it would roll back too.
Noise and dead code
- The per-user panel is a security panel: data-plane events file under the
actor, so an active account's routine deletes pushed a denied login out of
the 20-row window. It now excludes them; the Activity tab shows everything.
- device_audit_log had no created_at-leading index, so the merged feed
sequentially scanned that branch every page (migration 0036).
- conversation.deleted_all no longer records when nothing was deleted, and
agent.updated no longer records an empty field list.
- Dropped by_model from /admin/usage (no consumer; an extra aggregate per page
load), the duplicate filter surface on AuthEventsRepository that nothing
called, and the unreachable FLOW_LABELS.schedule entry.
- Type hints on record_event's conn and the remaining unannotated helpers.
- 0034's backfill set target_id to the acting admin for instance- and
team-scoped quota policy changes, which are filed under the actor and have
no user target. It now returns NULL for those, matching what quotas.py
records going forward, with a test covering all three scopes.
- The CSV export wrote every cell verbatim. user_agent is an attacker's raw
header, recorded without authenticating on a denied login, and the export
is opened in a spreadsheet by an admin -- a cell starting "=" would be
evaluated there. Every cell now has a leading formula trigger neutralized.
- The activity feed applied every response it received, so a slow reply for
an old filter could overwrite the current one. Guarded by a request id,
the same way settings/Analytics already does.
- The activity detail expander and the top-user drill-down were row onClick
handlers, unreachable without a mouse. Both are real buttons now, the
expander carrying aria-expanded and a label naming its event.
- FLOW_LABELS was applied to both breakdowns in the spend modal, so a model
named "fallback" or "workflow" rendered as a flow description. Only the
flow table maps keys now.
- Changing the range in that modal left the previous range's totals and
chart on screen while the new request was in flight.
bucketed_totals now returns cost and cached_tokens, so the exact-equality
assertion in test_day_bucket_sums_per_day had to grow the two fields; it
pins that an unreported cache bin reads NULL rather than 0.
The per-user breakdown test was seeded with a source='schedule' row, which
is a run-level rollup and no longer counted. Reseeded on a real flow, and
both spend queries gained a test that a rollup is excluded.
Three gaps in one view.
Cost: token_usage.cost has been written on every call since quotas landed, and
tokens_by_model already returned it, but bucketed_totals and top_token_users
selected tokens only. An admin could set a USD quota in /admin/quotas and had
no way to see the spend it was capping. Buckets and top users now carry cost,
the endpoint reports a window total and a per-model split, and the chart takes
a Tokens/Cost toggle with a currency axis.
Group-by: the endpoint has supported group_by=model|agent|source from the
start and the UI only ever sent bucket=day. The selector is now wired, and a
grouped series pads missing buckets so a model that was idle on Tuesday plots
a zero instead of shifting its whole row one bar left.
Latency: surfaced from the columns the previous commit added, as p50/p95 with
a median time-to-first-token underneath. Unmeasured rows are excluded rather
than counted as zero, and the card says so when nothing was measured.
Also surfaces the prompt-cache hit rate, computed only over rows whose
provider reported a cache breakdown -- NULL means "not reported", and folding
those in as 0% would understate it.
Per-user drill-down: GET /api/admin/users/<id>/usage returns a daily series
plus splits by model and by flow, reachable from the top-users table and from
the user detail dialog. The detail dialog previously showed one tokens_30d
number, which answers neither "what is this person costing" nor "what is
driving it".
The wrappers in docsgpt/usage.py already measured how long each call took --
for the llm_gen_finished / llm_stream_finished log lines -- and then threw the
number away. Nothing in the schema recorded it, so an operator could see what
an instance spent but never how slow it was.
Adds token_usage.duration_ms and token_usage.ttft_ms, and threads the
measurements the wrappers already take into the insert. The stream wrapper now
also stamps the moment of the first yielded chunk.
Both columns are nullable, and deliberately so: ttft_ms is NULL for a
non-streaming call and for a stream that failed before yielding anything, and
every row written before this migration is NULL too. A 0 there would drag a
p50 toward an instant first token that never happened.
DocsGPT keeps three append-only journals -- auth_events, device_audit_log and
guardrail_events -- each readable only on its own terms, and the second of
those had an admin endpoint that no UI ever called. An operator reviewing an
incident wants one timeline, not three.
Adds GET /api/admin/activity: all three journals projected onto a common row
shape and merged, ordered newest first, with facets for journal, category,
event name, actor, affected user, a time window and free-text search over the
detail payload. A category filter that only one journal can satisfy drops the
others from the union rather than scanning and discarding them.
GET /api/admin/activity/events returns the distinct (event, category) pairs
the instance has actually recorded, so the filter UI can offer real choices
instead of asking an operator to type "oidc_login_denied" from memory.
GET /api/admin/activity/export streams the same filtered feed as CSV or
NDJSON. Streamed rather than buffered, and capped, so a compliance export of a
busy instance is bounded work.
guardrail_events.api_key (a raw agent key) and matched_value (unredacted
source text) are excluded from the projection, mirroring the exclusion the
per-agent guardrail view already makes. Admin-gating is not a reason to widen
what a list response carries.
auth_events.user_id was overloaded and meant something different per call
site: admin mutations filed the row under the target user and hid the acting
admin in metadata->>'by', team events filed it under the actor, and quota
events switched between the two depending on scope. The practical consequence
was that "show me everything admin X did" had no answer.
Adds actor_id (never NULL) and target_id (NULL when the event is not about a
user) alongside the existing column, which keeps its meaning as the per-user
feed key so nothing historical is rewritten. The backfill recovers the actor
from metadata for rows that predate the columns.
Also adds the indexes the cross-user admin feed needs. The only index was
(user_id, created_at DESC), so the global feed -- which orders by created_at
with no user predicate -- fell back to a sequential scan plus a sort on every
page.
The feed gains actor, multi-event, until and free-text search filters, plus a
distinct-event catalogue so the UI can offer what the instance recorded
instead of asking operators to type a name from memory.
- A tool continuation refused for usage now releases the resume claim it
took; before, retries got a 409 until the stale claim was reverted.
- Agent traffic is any row with an agent key or an agent id, so keyless
agents and workflow nodes count toward the agent bucket, not direct.
- The user quota modal discards responses for a previously opened user.
- The usage meter shows every limited bucket, not only 'all'.
- Restore the class separator on the analytics stat card that a formatter
run removed, and align the OpenRouter DeepSeek description with its rates.
Validation problems are returned as values rather than raised and echoed
with str(exc), and a huge integer limit is rejected as out of range instead
of overflowing. Tests no longer call mutating endpoints inside asserts.
docsgpt/quotas resolves a user's effective token and cost limits from the
policy rows that apply to them: their own override, then the most generous
allowance among their teams, then the instance default, then any defaults a
registered provider supplies. Each budget resolves on its own, and a team
counts once however many memberships the user holds in it.
QuotaService compares those limits with the user's token_usage totals over
the current QUOTA_PERIOD window (calendar-aligned, UTC, computed at read
time). It fails open, and skips the usage query for unlimited users.
sum_tokens_in_range now skips the scheduler's per-run rollup rows, whose
tokens are already on the run's per-call rows, so the per-agent 24h limit
and the admin total no longer count scheduled spend twice.
Workflow node LLMs carry the workflow agent's id, so their usage rows are
attributed to the agent instead of landing with a user id only.
usage_totals returns a user's tokens and cost since a window start, split
by interactive and agent-key traffic.
Migration 0033 adds quota_policies (instance default, team per-member
allowance, user override; a token budget and a USD budget per row) and
token_usage.cost. The column is added IF NOT EXISTS so a database that
already carries it upgrades cleanly.
Every usage row now records the call's USD cost from the model catalog;
bring-your-own models are recorded at $0.
POST /api/user/tokens/<id>/regenerate swaps the secret of an existing token
in place: name, scopes and restrictions stay, the old secret stops matching
at once, and the expiry is reset. The lifetime defaults to the one the token
was last issued with (clamped to today's policy) or to expires_in_days when
given. An expired token can be renewed this way; a revoked one cannot. It is
session only like the rest of token management, and writes a pat_regenerated
audit event. regenerated_at records the rotation.
A resource restriction was checked on ids in the request, not on what the
addressed row pulls in or belongs to. Closed:
- Workflow writes for tokens restricted on sources, tools or prompts (a graph
names those inside its nodes), and attaching a workflow to an agent unless
the token is restricted on workflows too.
- Chat for tools-restricted tokens (chat executes tools; rejected at token
creation as well), and agent-less chat for tokens restricted on prompts or
workflows.
- conversation_id on chat: it must belong to the agent being run, or to no
agent for agent-less chat. Otherwise the server continued, appended to, or
resumed pending tool calls of another agent's conversation.
- Schedules for tokens restricted on anything but agents; schedule-id routes
for every restricted token.
- Conversations and analytics for every restricted token, not only
agent-restricted ones.
Also: create, first publish and adopt return the agent API key masked to a
token without agents:keys; token ids must be canonical UUIDs (urn:uuid: gave
a 500); an expired token is reported as expired; token creation takes a
per-user advisory lock so the cap cannot be raced; admin revoke-sessions
writes a pat_revoked event per token. The UI drops a row whose revoke returns
404 and does not offer a tools restriction next to chat:run.
The ASGI message events route now accepts the same scopes as its Flask
sibling (conversations:read or chat:run) through a shared constant. A token
request that fails routing gets Flask's 404/405 instead of a 403. Creating a
token retires an expired token that still held the name. allowed_ids uses
is_pat instead of a bare literal. PAT_ENABLED is documented as the master
switch it is: turning it off stops every existing token from authenticating.
The docs explain that sources are matched by name (oldest wins) and point CI
flows at sources upload --replace.
A personal_access_tokens table (migration 0032) holds scoped user-level API
credentials. Only the SHA-256 of the secret is stored, like device session
tokens. Lookups exclude revoked and expired tokens and the tokens of
deactivated users. PAT_* settings cover the feature switch, default and
maximum lifetime, the operator opt-in for non-expiring tokens and the
per-user cap.
The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.
Kept for one release:
- A top-level application package whose meta-path finder resolves
application.x.y to the already-imported docsgpt.x.y object, so old imports
and entry points (celery -A application.app.celery,
uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
docsgpt.* task on start-up, so messages queued by the previous release still
run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
the previous release wrote are left unread instead of firing twice.
The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
The duplicate check for append_compression_point ran as a SELECT before
the UPDATE, so two sessions persisting the same point could both pass it.
The predicate now lives inside the UPDATE's CASE expression and is
evaluated under the row lock; a point without query_index or
compressed_summary never matches and is always appended. Adds a
two-session regression test that races the same point and expects one.
- append_compression_point only skips a point when both query_index and
compressed_summary are present and match the last one; points without
those fields (as in the repository tests) were all being treated as
duplicates.
- The incremental compression tail and the orchestrator's "anything new
since the last point" check exclude the visible summary row, which the
prompt already receives through existing_compressions.
- Summary rows carry a persisted metadata marker; replay filters on the
marker, and falls back to the label only for rows written before it that
have no tool calls and no per-turn metadata, so a user who types the
label text keeps their turn.
- The prompt_cache_key is a hash of the user id, never the id itself.
- Describe truncation="auto" as dropping the oldest items.
Five defects from a review of the embeddings work, four of them silent.
- Write local files atomically. `LocalStorage.save_file` streamed straight onto
the destination, so an interrupted write left a truncated file. `reembed`
rewrites every index it touches, and a half-written `index.faiss` loads at
neither the old width nor the new one -- the source was unrecoverable, with
no backup and no temp file left behind. Bytes now land beside the destination
and move into place with `os.replace`. S3 was already safe (single PUT).
- Read pgvector chunks a page at a time. `reembed_pgvector` materialised every
`(id, text)` row for a source before embedding -- ~1.6 GB at 200k chunks and
several times that for non-Latin scripts, with the `PGresult` held alongside
until the cursor closed. Inside the shipped 4Gi limit, while also holding the
model, that is an OOMKill -- which is exactly the SIGKILL the point above
turned into a destroyed index. It now walks the source by keyset.
- Bound the first wave of delegated embeds. The failure cooldown is only latched
once the first `get()` returns, so every request already in flight paid the
full EMBEDDINGS_DELEGATE_TIMEOUT: measured 64 threads all timing out together,
and at the shipped 60s across a 96-thread WSGI pool that is an API serving
nothing at all, health checks included. One caller now probes while the rest
fail fast; after a single success the gate leaves the path entirely.
- Ship EMBEDDINGS_NAME commented in .env-template. The comment directly above it
says to leave it commented when upgrading, and the line shipped set. Any value
reaching `.env` lands in `model_fields_set`, which makes `resolve_embeddings_pin`
bail -- so a template-derived `.env` disabled the legacy pin outright and
repointed a populated index at a different 768-dim model, where no width check
fires. The pin already picks granite for a fresh install and mpnet for an
existing one, so nothing needs to be set by hand.
- Stamp `sources.model` on wiki sources. They were created with the column NULL
and then embedded like any other source, and the boot check reads NULL as
"pre-dates the column, therefore the legacy model" -- reporting a correctly
embedded source as stale on every startup of every process. Stamped at
creation, and again on each page re-embed so existing rows heal.
The two docs that promised the FAISS index survives a failed run said so of the
embed only; both now describe the write, and upgrading.mdx says to stop ingest
for the duration.
Query embedding moved to the Celery worker, but nothing that ships was
updated to consume the queue it dispatches to.
- Add `embeddings` to every worker `-Q` list (compose x3, k8s, devcontainer,
sandbox README). Without it a search blocked for EMBEDDINGS_DELEGATE_TIMEOUT
and then answered with no retrieved context, because classic_rag swallows the
dispatch error and skips the source -- bad answers, not an error.
- Skip the task_postrun heap reclaim for the embed task. The full gc.collect()
was written for docling/torch parses; on a worker holding the ONNX model it
measured ~86ms against ~8ms for the embed itself, a 9x slowdown of the round
trip for a task that allocates a few kilobytes.
- Resolve the installation pin in the re-embed script. It never imports
application.app, so an install pinned in app_metadata with no EMBEDDINGS_NAME
set -- every stock k8s deployment, whose manifests carry no embedding config
-- would rewrite its whole index with the legacy default and stamp
sources.model to match, then be told by the boot warning to run it again.
- Fail fast for 30s after a failed dispatch. fanout.embed_questions falls back
to letting each store embed its own query, so one dead-worker retrieval paid
the timeout once in the fan-out and again per source.
- Forget the task result. Nothing reads it back: the key is per-dispatch UUID,
not content-addressed, so a repeated query mints another. Left alone every
search leaked ~17KB for result_expires (7 days) into the Redis the broker
shares -- on the bundled k8s manifest (1Gi, no maxmemory policy) that is an
OOMKill that takes the broker with it.
- Release the model ensure_vector_schema loads to read the width of an
unregistered model, in a process that delegates and would never call it.
The width still comes from the model, not the table, so the mismatch check
the hook exists for keeps working.
- Correct the docs that said otherwise: embeddings.md claimed the standard
deployment worked unchanged, upgrading.mdx said no action was needed, and
the settings table listed none of the three delegation settings.
EMBEDDINGS_NAME has a code-level default, and moving that default re-points an
existing index at a different vector space without anything noticing: mpnet and
granite are both 768-dimensional, so no width check fires and retrieval simply
gets worse. Which model an index was built with is a property of the
installation, not of the release it happens to be running.
So it is resolved once at boot and stored in app_metadata, which already exists
for exactly this kind of one-off state and needs no migration. An installation
that already has sources is pinned to the legacy model it has been using all
along and told, once, how to move; an empty one is pinned to the current
recommendation. An explicit EMBEDDINGS_NAME in the environment still wins over
both, and an unreachable database falls back to the code default.
This also closes a gap in what "new installs get granite" meant: it held for
setup.sh and for anyone copying .env-template, but a hand-written .env plus
docker compose fell through to the settings default and quietly got mpnet --
English-only, and a 384-token window against a 1250-token chunk default.
Emptiness is counted in sources rather than vector rows so the answer is the
same for every vector store, including the FAISS ones whose vectors are not in
this database at all. Concurrent workers converge through the same
INSERT ... ON CONFLICT DO NOTHING the instance id already uses.
The API embeds every query it serves, so it held its own copy of the model:
~890 MB it never needed. EMBEDDINGS_DELEGATE_TO_WORKER (on by default) sends
the text to the Celery worker instead and gets the vector back, taking an API
process from 1176 MB to 285 MB with no ONNX Runtime imported at all. The client
embeds locally when it finds itself inside a worker task, so the worker never
dispatches to itself -- the same self-deadlock DOCUMENT_PARSE_QUEUE avoids on
the parsing side. EMBEDDINGS_BASE_URL still wins over it, and remains the right
answer for production.
ensure_vector_schema was constructing the embeddings instance purely to read
.dimension off it, loading several hundred MB of ONNX into every API and worker
process at import. For a model the registry describes that is a lookup; only an
unregistered name now falls back to loading.
EMBEDDINGS_BATCH_SIZE was sizing two unrelated things: chunks per store
transaction (and per remote embed request) and documents per ONNX forward pass.
Each pass pads every input up to its longest, and that waste grows with the
square of chunk length, so at the 1250-token default a batch of 32 peaked at
6.6 GB and took 326s where a batch of 1 peaked at 2.9 GB and took 90s. The
forward pass is now sized by EMBEDDINGS_MODEL_BATCH_SIZE, defaulting to 1;
storage and remote batching are unchanged at 32.
reembed embeds in-process: a batch job that walks the whole index should not
round-trip every chunk through a broker, and loading the model there reports a
real failure instead of timing out against an empty queue.
Also drops the mpnet zip download from the docs and the devcontainer, which
pointed at a SentenceTransformers export with no ONNX graph and had been inert
since the FastEmbed swap; corrects the claim that any sentence-transformers
model works; and settles the Configuring/Settings pages on what the registry
and the repository metadata actually decide.
arc53-machine