12 Commits
Author SHA1 Message Date
Alex e889392eba fix(api): address review on the event-loop streams
- Per-user SSE cap uses per-connection leases in a sorted set instead of
  a shared INCR/DECR counter with a TTL. A stream that outlived the TTL
  could let the counter expire, then decrement another stream's slot or
  drive the count negative. Leases refresh while a stream sends frames
  and age out when a stream dies without cleanup.
- Shielded cleanup is bounded per step: stream close and on_close in
  ClosingStreamingResponse, unsubscribe and close in AsyncTopic, lease
  release, and the replay-budget check, so a dead Redis connection can't
  hold a request or a graceful shutdown.
- Artifact downloads send an ASCII filename plus an RFC 5987 filename*
  for non-ASCII names, build headers before opening the file, and close
  disk handles off the event loop.
- Type hints and docstrings on the new helpers.
2026-09-14 10:19:59 +01:00
Alex 54b540ea42 feat(api): serve long-lived streams on the event loop
Move GET /api/events, the remote-device command stream and artifact
downloads from Flask to Starlette routes mounted ahead of the Flask
catch-all. On Flask each held an a2wsgi threadpool slot for as long as
its response stayed open, and because uvicorn drops writes after a
client disconnects, a closed tab never released it.

- asgi_auth: one JWT/OIDC gate for Starlette routes; the chat reconnect
  reader uses it too
- ClosingStreamingResponse closes the body iterator and releases the
  SSE slot or file handle even when the client leaves before the first
  frame
- AsyncTopic liveness probe replaces the sync client's socket_timeout
  guard against half-open pub/sub sockets
- ASYNC_REDIS_MAX_CONNECTIONS sizes the async Redis pool; every open
  stream holds a connection and redis-py defaults to 100
2026-09-14 08:33:46 +01:00
Alex 574f96341e refactor: rename the application package to docsgpt
The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.

Kept for one release:
- A top-level application package whose meta-path finder resolves
  application.x.y to the already-imported docsgpt.x.y object, so old imports
  and entry points (celery -A application.app.celery,
  uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
  docsgpt.* task on start-up, so messages queued by the previous release still
  run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
  the previous release wrote are left unread instead of firing twice.

The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
2026-09-07 10:20:43 +01:00
Alex 66786c2760 fix: remove code exec as default and sec improvements 2026-07-08 18:50:10 +01:00
Alex 94a845aa82 fix: more artefact hardening 2026-07-04 11:42:27 +02:00
Alex 47cc0314af feat: more executor guards 2026-07-03 23:53:51 +02:00
Alex a0f10925af fix: artefact fixes 2026-07-03 20:16:31 +02:00
Alex bd88dd6807 feat: scope artefacts on api 2026-07-01 20:34:27 +02:00
Alex 75e07af78f fix: minor issue fixes 2026-06-30 18:36:06 +02:00
Alex b144265094 fix: workflow security pass 2026-06-29 19:21:35 +02:00
Alex 00896eb828 Clear bot-flagged test nits and annotate the sandbox workspace path
Wrap a storage read in a context manager (close the file handle), add a comment
on the expected SandboxCapacityError in the churn test, and unify the
artifacts-routes test on a single import style. Also annotate the intentional
per-session sandbox workspace path with # nosec B108 (controlled dir inside the
runner container, not an insecure shared temp file).
2026-06-25 18:09:48 +01:00
Alex 922ed53a70 Add artifact REST endpoints (list, get, download, restore)
Serve artifact metadata and bytes over HTTP with parent-derived
authorization: conversation-parented artifacts inherit conversation
access (owner, shared_with, or a public share token whose conversation
matches the parent), workflow-run artifacts check run ownership, and
access fails closed when the parent is missing or deleted.

Adds list/get/versions/download/restore routes, an authenticated
storage-agnostic download (sanitized Content-Disposition, 302 to a
short-lived private S3 presigned URL when that strategy is configured),
a generate_presigned_url primitive on the storage base and S3 backend,
and generalizes the tools artifact endpoint for documents/files. Shared
authorization helpers live in a dedicated module used by both surfaces.
2026-06-24 10:53:41 +01:00