build(deps): declare dependencies in pyproject.toml with docling and milvus extras

requirements.txt pinned torch and transformers in core although only docling
needs them, and on Linux torch pulls the CUDA 13 stack: 2.7 GB of the 3.0 GB
wheel download. Direct dependencies now live in pyproject.toml, uv.lock pins
everything, and application/requirements*.txt are exported from the lock by
scripts/export_requirements.sh (each file is the core set plus one extra).

The docling extra pins torch/torchvision/transformers itself and, on Linux,
resolves torch from the CPU-only PyTorch index (no nvidia packages). milvus
(pymilvus + milvus-lite, which pulls pyarrow) is the second extra.

application/core/optional_deps.py is the one place install hints come from;
the milvus store and the docling call sites use it so a missing extra fails
with the exact command to run.
This commit is contained in:
Alex committed 2026-09-05 15:50:20 +01:00
1 parent 55a38b3132
commit 196865846c
14 files changed
+10545 -123

No files matched your search

@@ -94,13 +94,25 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
pip install -r application/requirements.txt
```
Optionally add the docling parser engine (OCR, `structured` output for
`read_document`; the default anydoc engine does not need it):
Dependencies are declared in `pyproject.toml` and locked in `uv.lock`; the
`requirements*.txt` files are exported from that lock, so with
[uv](https://docs.astral.sh/uv/) installed `uv sync` sets up the same
environment (plus the test tools) in one step.
Optional extras are not installed by default. Add them when you need the
feature (each file is the core set plus the extra):
```bash
pip install -r application/requirements-docling.txt
pip install -r application/requirements-docling.txt # docling parser engine: OCR backend, read_document structured output
pip install -r application/requirements-milvus.txt # VECTOR_STORE=milvus
# or with uv: uv sync --extra docling --extra milvus
```
A feature whose extra is missing fails with the exact command to run.
After changing `pyproject.toml`, run `uv lock` and
`bash scripts/export_requirements.sh` so the exported files stay in sync
(CI checks this).
5. **Run the Backend:**
For local development, run the ASGI composition under uvicorn. It serves the **whole** application, hot-reloads on source changes, and matches the production runtime: