The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.
Kept for one release:
- A top-level application package whose meta-path finder resolves
application.x.y to the already-imported docsgpt.x.y object, so old imports
and entry points (celery -A application.app.celery,
uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
docsgpt.* task on start-up, so messages queued by the previous release still
run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
the previous release wrote are left unread instead of firing twice.
The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
Wrap a storage read in a context manager (close the file handle), add a comment
on the expected SandboxCapacityError in the churn test, and unify the
artifacts-routes test on a single import style. Also annotate the intentional
per-session sandbox workspace path with # nosec B108 (controlled dir inside the
runner container, not an insecure shared temp file).
Remove the integration tests that launch a real Jupyter Kernel Gateway or hit
Daytona (artifact_generator / code_executor / workflow-code-node e2e, the gateway
integration test, and the Daytona live test) — these belong in a separate e2e PR
with proper harnessing. The ephemeral-Postgres integration tests (artifacts
repository, run-scoped + input-document authz) stay: they run in CI without any
external service and hold the cross-tenant coverage. Trim the now-unused
test-only deps (jupyter-kernel-gateway, ipykernel, websocket-client); keep
jupyter-client for the retained kernelspec-resolution test.
Run each kernel under a scrubbed environment so untrusted code can never read
the host's secrets. A custom 'docsgpt-python' kernelspec launches ipykernel
through a wrapper that keeps only what the kernel needs (PATH, HOME, LANG, and
the Jupyter runtime/data dirs), dropping API keys, tokens, the database URL, and
the gateway token. The app selects this kernel by name via SANDBOX_KERNEL_NAME,
so the distinct name is never shadowed by the stock python3 spec. Per-session
workspaces are created mode 0700 (defense in depth under the shared uid). The
README documents the runner as a single trust domain and points to the Daytona
backend for per-tenant isolation.
The output-cap test flooded ~100MB through a 20s wall-clock timeout, so under
heavy parallel load the timeout could fire before enough output accumulated to
trip the cap, intermittently failing. Emit a bounded ~200 KiB (4x the cap)
under a generous timeout so the truncation path triggers deterministically.
Add runtime governance for the sandbox and artifact store: a per-process
concurrent-session cap with least-recently-used eviction of idle sessions and a
periodic idle reaper (Celery beat), so sandbox kernels do not accumulate. The
session manager performs all backend start/stop outside its lock and tears down
the captured handle, so eviction never closes a concurrently re-opened session.
Add per-user artifact quotas (count, total bytes, and per-file size) enforced at
persistence time as a soft cap, and best-effort cleanup of per-render scratch
directories in the sandbox workspace.
Add a pluggable Daytona backend (SANDBOX_BACKEND=daytona) that runs code on
Daytona Cloud via the Apache-2.0 SDK and DAYTONA_API_KEY, conforming to the
CodeSandbox interface. Sessions reattach to a labelled cloud sandbox by id
(waking a stopped one) and are created crash-safe so a failed setup never
orphans a paid sandbox; a configurable ceiling caps concurrent sandboxes and
auto-delete is clamped on so orphans always self-reap. File transfer is
workspace-contained with a size guard. Includes mocked unit tests and an
opt-in live smoke test.
Introduce a pluggable CodeSandbox abstraction with a SandboxManager and a
Jupyter Kernel Gateway backend: the app is a client of a single always-on
runner that executes code in stateful in-process kernels (no child-container
spawning, no docker socket). Sessions bind to a conversation or workflow run
with an agent-selectable TTL clamped by a global cap; execution enforces a
wall-clock deadline with interrupt-on-timeout and capped output, and file
transfer is workspace-contained with size and integrity checks. Adds a
docsgpt-sandbox docker-compose service (resource-capped, read-only, internal
network) plus settings, with a real local-gateway integration test.
Alex