Add semi-persistent code-execution sandbox (Jupyter Kernel Gateway runner)

Introduce a pluggable CodeSandbox abstraction with a SandboxManager and a
Jupyter Kernel Gateway backend: the app is a client of a single always-on
runner that executes code in stateful in-process kernels (no child-container
spawning, no docker socket). Sessions bind to a conversation or workflow run
with an agent-selectable TTL clamped by a global cap; execution enforces a
wall-clock deadline with interrupt-on-timeout and capped output, and file
transfer is workspace-contained with size and integrity checks. Adds a
docsgpt-sandbox docker-compose service (resource-capped, read-only, internal
network) plus settings, with a real local-gateway integration test.

deployment/docker-compose.yaml

@@ -24,6 +24,8 @@ services:
       - CELERY_RESULT_BACKEND=redis://redis:6379/1
       - CACHE_REDIS_URL=redis://redis:6379/2
       - POSTGRES_URI=postgresql://docsgpt:docsgpt@postgres:5432/docsgpt
+      # Code-execution runner reached over HTTP + WebSocket (no docker socket).
+      - SANDBOX_GATEWAY_URL=http://docsgpt-sandbox:8888
     ports:
       - "7091:7091"
     volumes:
@@ -49,6 +51,7 @@ services:
       - API_URL=http://backend:7091
       - CACHE_REDIS_URL=redis://redis:6379/2
       - POSTGRES_URI=postgresql://docsgpt:docsgpt@postgres:5432/docsgpt
+      - SANDBOX_GATEWAY_URL=http://docsgpt-sandbox:8888
     volumes:
       - ../application/indexes:/app/indexes
       - ../application/inputs:/app/inputs
@@ -59,6 +62,30 @@ services:
       postgres:
         condition: service_healthy
 
+  # Always-on code-execution runner (Jupyter Kernel Gateway). Sessions are
+  # in-process kernels, never child containers; the Docker socket is NOT
+  # mounted. On an internal-only network — no host port is published, so the
+  # runner is reachable only from backend/worker, not from the host/internet.
+  # Egress/SSRF blocks, the gVisor `runsc` runtime, and seccomp profile come in
+  # the hardening slice.
+  docsgpt-sandbox:
+    build: ./sandbox
+    mem_limit: ${SANDBOX_MEMORY:-1g}
+    cpus: ${SANDBOX_CPUS:-1.0}
+    pids_limit: 256
+    read_only: true
+    environment:
+      # Keep Jupyter's runtime/connection files on the writable tmpfs.
+      - JUPYTER_RUNTIME_DIR=/tmp/jupyter-runtime
+      - JUPYTER_DATA_DIR=/tmp/jupyter-data
+    tmpfs:
+      # Per-session workspaces (/tmp/docsgpt-sandbox/<session_id>) and Jupyter
+      # runtime files live on tmpfs; the root FS is read-only everywhere else.
+      - /tmp
+    networks:
+      - sandbox-net
+      - default
+
   redis:
     image: redis:6-alpine
     ports:
@@ -80,6 +107,12 @@ services:
       timeout: 5s
       retries: 10
 
+networks:
+  # Internal-only network for the sandbox runner (no external egress route via
+  # this network; the runner still reaches the internet via the default bridge).
+  sandbox-net:
+    internal: true
+
 volumes:
   postgres_data:
 

deployment/sandbox/Dockerfile

@@ -0,0 +1,48 @@
+# docsgpt-sandbox: the always-on code-execution runner.
+#
+# One container runs a Jupyter Kernel Gateway; each sandbox session is an
+# in-process kernel (a child process), NOT a child container. This image does
+# NOT mount the Docker socket and never spawns containers — that avoids the
+# host-root risk of Docker-in-Docker and is the core security win.
+#
+# The app (backend/worker) is the CLIENT and reaches this service over
+# HTTP + WebSocket via SANDBOX_GATEWAY_URL.
+#
+# Pre-baked doc libs are all permissive (MIT/BSD/Apache) for speed/reliability;
+# runtime `pip install` still works because egress is open. PyMuPDF and any
+# other AGPL lib are intentionally excluded; Docling is deferred to its own slice.
+#
+# HARDENING (separate slice — NOT done here): run under the gVisor `runsc`
+# runtime, add network-layer SSRF blocks (drop RFC1918 / link-local /
+# 169.254.169.254), seccomp profile, read-only root FS + quota'd scratch dir,
+# and cgroup CPU/mem/PID caps wired from SANDBOX_MEMORY / SANDBOX_CPUS.
+FROM python:3.12-slim
+
+# Non-root user for the runner (untrusted code runs as this UID).
+RUN useradd --create-home --uid 10001 sandbox
+
+# Exact pins for reproducible builds and no license drift (all MIT/BSD/Apache).
+RUN pip install --no-cache-dir \
+    jupyter-kernel-gateway==3.0.1 \
+    ipykernel==6.29.5 \
+    python-pptx==1.0.2 \
+    python-docx==1.1.2 \
+    openpyxl==3.1.5 \
+    reportlab==4.2.5 \
+    pandas==2.2.3 \
+    matplotlib==3.9.2
+
+USER sandbox
+WORKDIR /home/sandbox
+
+EXPOSE 8888
+
+# ip=0.0.0.0 so the backend can reach it over the internal compose network.
+# allow_origin is intentionally NOT set to "*" — publishing port 8888 requires
+# setting SANDBOX_GATEWAY_AUTH_TOKEN (see README). limit_rate=False raises the
+# iopub data-rate cap so large get_file base64 payloads are not truncated; the
+# get_file integrity check still guards against truncation if this is ever off.
+CMD ["jupyter", "kernelgateway", \
+     "--KernelGatewayApp.ip=0.0.0.0", \
+     "--KernelGatewayApp.port=8888", \
+     "--ZMQChannelsWebsocketConnection.limit_rate=False"]

deployment/sandbox/README.md

@@ -0,0 +1,50 @@
+# docsgpt-sandbox runner
+
+Always-on Jupyter Kernel Gateway that executes sandboxed LLM code. The DocsGPT
+backend/worker is the **client** and connects over HTTP + WebSocket via
+`SANDBOX_GATEWAY_URL`. Each session is an **in-process kernel** (child process),
+never a child container; the Docker socket is **not** mounted.
+
+## Run standalone for dev
+
+Build and run the runner on its own, then point the app at it:
+
+```bash
+docker build -t docsgpt-sandbox deployment/sandbox
+docker run --rm -p 8888:8888 docsgpt-sandbox
+# in the app's .env:  SANDBOX_GATEWAY_URL=http://localhost:8888
+```
+
+Without Docker (matches the test harness) you can run the gateway directly from
+a venv that has `jupyter-kernel-gateway` installed:
+
+```bash
+jupyter kernelgateway --KernelGatewayApp.ip=0.0.0.0 --KernelGatewayApp.port=8888 \
+  --ZMQChannelsWebsocketConnection.limit_rate=False
+```
+
+`--ZMQChannelsWebsocketConnection.limit_rate=False` raises the iopub data-rate
+limit so large `get_file` base64 payloads aren't silently truncated. (On older
+gateways the trait may live elsewhere; the client's `get_file` integrity check
+catches any truncation regardless.)
+
+## Exposing the port requires auth
+
+The image does **not** set `--KernelGatewayApp.allow_origin=*`. If you publish
+port 8888 (e.g. `docker run -p 8888:8888`), set `SANDBOX_GATEWAY_AUTH_TOKEN`
+and launch the gateway with a matching `--KernelGatewayApp.auth_token` so the
+runner is not an open arbitrary-code-execution endpoint. In compose the runner
+stays on the internal-only network with no published port, so no token is
+required there.
+
+## In docker-compose
+
+The `docsgpt-sandbox` service is defined in `deployment/docker-compose.yaml` on
+an internal-only network. The backend and worker reach it at
+`http://docsgpt-sandbox:8888`.
+
+## Hardening (separate slice — not in this image)
+
+Egress/SSRF blocks (drop RFC1918, link-local, `169.254.169.254`), the gVisor
+`runsc` runtime, seccomp, read-only root FS, and cgroup CPU/mem/PID caps wired
+from `SANDBOX_MEMORY` / `SANDBOX_CPUS` land in the hardening slice.