Merge branch 'arc53:main' into main

This commit is contained in:
Manish Madan authored and GitHub committed 2026-09-15 04:18:40 +05:30
commit 2afc711b57
60 files changed
+4696 -1690

No files matched your search

+2 -1
View File
@@ -70,7 +70,7 @@ GET /api/messages/<message_id>/events
This replays the message's events past your last-seen sequence number and tails the rest live. It is backed by the Postgres `message_events` journal (retained for `MESSAGE_EVENTS_RETENTION_DAYS`, default 14).
<Callout type="warning" emoji="⚠️">
The chat reconnect endpoint is a native-async route served by the ASGI entrypoint. Under a plain `flask run` dev server it returns `404`; run the backend via the ASGI app (`uvicorn docsgpt.asgi:asgi_app`) or the production gunicorn uvicorn worker to use it. See the [Development Environment](/Deploying/Development-Environment) guide.
Both channels are native-async routes served by the ASGI entrypoint. Under a plain `flask run` dev server they return `404`; run the backend via the ASGI app (`uvicorn docsgpt.asgi:asgi_app`) or the production gunicorn uvicorn worker to use them. See the [Development Environment](/Deploying/Development-Environment) guide.
</Callout>
## Settings
@@ -81,6 +81,7 @@ The chat reconnect endpoint is a native-async route served by the ASGI entrypoin
| `EVENTS_STREAM_MAXLEN` | `1000` | Per-user backlog cap (approximate). |
| `SSE_KEEPALIVE_SECONDS` | `15` | Keepalive comment-frame cadence (keep below your proxy's idle timeout). |
| `SSE_MAX_CONCURRENT_PER_USER` | `8` | Max simultaneous SSE connections per user (`0` disables the cap). |
| `ASYNC_REDIS_MAX_CONNECTIONS` | `2000` | Redis connection pool per API worker. Each open SSE connection holds one, so this caps concurrent streams per worker. |
| `EVENTS_REPLAY_MAX_PER_REQUEST` | `200` | Max backlog entries per replay request. |
| `EVENTS_REPLAY_BUDGET_REQUESTS_PER_WINDOW` | `30` | Per-user replay requests per window (`0` disables). |
| `EVENTS_REPLAY_BUDGET_WINDOW_SECONDS` | `60` | Replay budget window length. |
@@ -133,7 +133,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
flask --app docsgpt/app.py run --host=0.0.0.0 --port=7091
```
But it serves **only** the WSGI Flask app and omits the routes mounted on the ASGI shell in `docsgpt/asgi.py`: the `/mcp` FastMCP endpoint and the native-async SSE reconnect reader `GET /api/messages/<id>/events`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but a stream interrupted by a disconnect won't auto-resume on reconnect. Use `flask run` only when you don't need those routes.
But it serves **only** the WSGI Flask app and omits the native-async routes mounted on the ASGI shell in `docsgpt/asgi.py`: the `/mcp` FastMCP endpoint, the chat reconnect reader `GET /api/messages/<id>/events`, the notification stream `GET /api/events`, the remote-device command stream `GET /api/devices/sessions/<id>/events`, and artifact downloads `GET /api/artifacts/<id>/download`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but live notifications, stream auto-resume, paired devices and artifact downloads don't. Use `flask run` only when you don't need them.
6. **Start the Celery Worker:**
+3 -3
View File
@@ -22,7 +22,7 @@ flowchart TB
Clients["Web UI, widgets and API clients"] --> API
subgraph Runtime["DocsGPT processes"]
API["API process<br/>Flask routes and agent execution<br/>ASGI shell: MCP and chat reconnect"]
API["API process<br/>Flask routes and agent execution<br/>ASGI shell: MCP and streaming routes"]
Redis["Redis<br/>Task broker, results and realtime delivery"]
Worker["Celery workers<br/>Ingestion, parsing, embeddings<br/>and background agent runs"]
API -->|"Enqueue work"| Redis
@@ -63,7 +63,7 @@ The storage boxes describe responsibilities, not a required number of servers. F
### API entry points
[The ASGI application](https://github.com/arc53/DocsGPT/blob/main/docsgpt/asgi.py) mounts Flask through a WSGI adapter, the FastMCP server, and native asynchronous chat-reconnect readers in one process.
[The ASGI application](https://github.com/arc53/DocsGPT/blob/main/docsgpt/asgi.py) mounts Flask through a WSGI adapter, the FastMCP server, and native asynchronous routes in one process. The asynchronous routes are the ones that hold a response open: the notification stream, chat reconnect, the remote-device command stream, and artifact downloads. They run on the event loop, so an idle connection doesn't occupy a Flask worker thread.
| Interface | Purpose |
| --- | --- |
@@ -76,7 +76,7 @@ The storage boxes describe responsibilities, not a required number of servers. F
DocsGPT also acts as an **MCP client**: agents can call external MCP servers through the [MCP tool integration](/Guides/Integrations/mcp-tool-integration). This is a separate direction from exposing DocsGPT search at `/mcp`.
<Callout type="info">
Use the ASGI entry point for the complete application. A plain `flask run` server omits `/mcp` and the native async chat-reconnect routes. See [Development Environment](/Deploying/Development-Environment).
Use the ASGI entry point for the complete application. A plain `flask run` server omits `/mcp` and the native asynchronous routes: notifications, chat reconnect, the remote-device command stream, and artifact downloads. See [Development Environment](/Deploying/Development-Environment).
</Callout>
### Agents and retrieval
+9
View File
@@ -150,6 +150,15 @@ takes over as the fallback parser. If that fallback also extracts almost
nothing — OCR off, or no engine available — the upload fails with a clear
message instead of silently indexing an empty document.
Chat attachments are the exception: a scanned PDF attached from the message
box is kept with no text (`extraction.status: no_text`), because the file
itself is what a model reads. Models that take PDFs get the document; models
that take only images get its pages as images; a text-only model is told it
cannot read the file, and the message box warns before sending. Images work
the same way with OCR off, and TIFF and BMP attachments are stored as PNG,
since model providers do not accept those formats. A TIFF or BMP larger than
40 million pixels is refused instead of converted.
Mixed documents — text pages with scanned pages among them — convert
through anydoc, which reads the text pages and skips the scanned ones. With
OCR on, DocsGPT probes every page's text layer, OCRs the pages that have