The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.
Kept for one release:
- A top-level application package whose meta-path finder resolves
application.x.y to the already-imported docsgpt.x.y object, so old imports
and entry points (celery -A application.app.celery,
uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
docsgpt.* task on start-up, so messages queued by the previous release still
run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
the previous release wrote are left unread instead of firing twice.
The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
Bumps the Python backend dependency set, holding back the ones that are
resolver-blocked or that regress behaviour this repo depends on.
Notable upgrades:
cryptography 46.0.7 -> 50.0.0 (requires msal 1.37.0, which relaxes its cap)
protobuf 6.33.6 -> 7.35.1 (floats opentelemetry-* to 1.44.0)
openai 2.32.0 -> 2.53.0, anthropic 0.88.0 -> 0.121.0
google-genai 1.73.1 -> 2.17.0 (the 2.0 break is scoped to the Interactions
API, which this repo does not use)
fastmcp 3.2.4 -> 3.4.6, gunicorn 25.3.0 -> 26.0.0
starlette 1.0.0 -> 1.6.0, uvicorn 0.42.0 -> 0.52.1
sentence-transformers 5.3.0 -> 5.7.0, numpy 2.4.4 -> 2.5.1
faiss-cpu 1.13.2 -> 1.15.0, pillow -> 12.3.0 (pinned; security release)
pypdf 6.9.2 -> 6.15.0, lxml 6.0.2 -> 6.1.1 (CVE-2026-41066)
Code changes needed by the bumps:
- openai >= 2.53 rejects a falsy api_key at construction. Keyless
OpenAI-compatible backends (Ollama, llama.cpp, vLLM) legitimately have
none, and pydantic-settings yields "" for a bare `API_KEY=` in .env, so
both call sites now fall back to a placeholder.
- Flask >= 3.1.2 tears a stream_with_context request down twice, and a
ContextVar token may only be reset once. The log-context teardown hook
is now idempotent.
- sentence-transformers renamed get_sentence_embedding_dimension to
get_embedding_dimension in 5.4; use the new name with a fallback.
Held back deliberately:
torch 2.11.0 - 2.13.0 drags torchvision 0.26 -> 0.28 and the whole
docling stack; torch is only probed for cuda.empty_cache()
tokenizers 0.22.2 - transformers pins <=0.23.0 and no stable 0.23.0 exists
transformers - capped <5.9.0: 5.9+ breaks docling's PDF layout model on
Apple Silicon (MPS float64), matching docling-core's own
darwin pin
docling 2.84.0 - 2.118.1 needs rapidocr >=3.9.1 and conflicts with
transformers on macOS; wants its own PR with a
golden-corpus diff
redis 7.4.0 - 8.x defaults socket_timeout to 5s, silently capping the
blocking reads in the device broker and SSE tail
websockets 16.0 - google-genai caps <17.0
marshmallow - dataclasses-json hard-caps <4; spec tightened to match
langchain block - langchain-community 0.4.2 deletes the Qdrant vectorstore
this repo imports; langchain 1.3.x needs websockets <16
_upload_file_to_openai short-circuited on `"openai_file_id" in attachment`.
Attachment dicts are built by row_to_dict from `SELECT *`, so that key is
always present and is NULL until an upload caches one — every fresh
attachment read as a cache hit. The method returned None without
uploading and, critically, without raising, so the except branch that
inlines the extracted text never ran. The part went out as
`{"file_id": None}`, which _resolve_file_part degraded to
`[File 'upload.pdf' could not be processed]`.
The model then relayed that to the user while thousands of tokens of
extracted text sat unused on the attachment row. Reproduced live against
the Foundry deployment: the model answers "I couldn't access the attached
PDF because it could not be processed; please re-upload it or paste the
text". The same bug was fixed in google_ai.py by 8e9f661e ("Truthy check,
not membership") which never touched this file; d29a7aaf later turned the
resulting hard 400 into this silent wrong answer.
Truthy check restored, a falsy file_id now raises so the fallback runs,
and the fallback tests `content` truthily — it too was a membership test,
so an empty extraction sent a bare "File content:" header that reads as a
successfully-read document. With no content at all we emit a note naming
the user's own file instead of the `upload.pdf` placeholder.
The file part deliberately carries `file_id` only: sending `filename`
alongside it is rejected with
`400 Unknown parameter: 'messages[0].content[1].file.filename'`.
Two consequences of making this path live for the first time are handled
here. `attachments.openai_file_id` is a single global column, so ids are
now stamped with an endpoint fingerprint — an id minted against one
deployment replayed against another answers "No such File object" on
every retry, permanently, since the row keeps the bad id. A foreign or
legacy unscoped value is treated as a miss and re-uploaded, so it
self-heals. And inlined text is bounded against the model's context
window, because _enforce_context_window runs before attachments are
merged into the messages and nothing downstream would trim a 200-page
extraction.
Route opted-in models to /v1/responses while keeping Chat Completions as
the default. The flavor is selected per-model in the catalog YAML and
threaded through ModelCapabilities (with a new reasoning_effort field).
- Translate chat-shaped messages/tools/structured-output into the
Responses input / text.format / flattened-tool shapes at the provider
boundary; stored history and intra-turn tool round-trips need no
migration.
- Normalize the typed SSE event stream back into the existing handler
contract (content / thought / tool-call chunks) so the streaming
accumulator and OpenAI handler stay unchanged.
- Carry encrypted reasoning items across the in-turn tool loop
(provider-local, keyed by call_id) for stateless reasoning continuity.
- OPENAI_RESPONSES_STORE opt-in enables server-side state and
previous_response_id chaining across turns, persisted in message
metadata.
- Set gpt-5.5 to api_flavor: responses, reasoning_effort: medium.
- Remove AzureOpenAILLM and the azure_openai provider/catalog.
Alex