refactor: rename the application package to docsgpt

The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.

Kept for one release:
- A top-level application package whose meta-path finder resolves
  application.x.y to the already-imported docsgpt.x.y object, so old imports
  and entry points (celery -A application.app.celery,
  uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
  docsgpt.* task on start-up, so messages queued by the previous release still
  run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
  the previous release wrote are left unread instead of firing twice.

The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
This commit is contained in:
Alex committed 2026-09-07 10:20:43 +01:00
1 parent 3e7f1f91b4
commit 574f96341e
984 files changed
+9847 -9682

No files matched your search

@@ -53,7 +53,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
* **Option 1: Using a `.env` file (Recommended):**
* If you haven't already, create a file named `.env` in the **root directory** of your DocsGPT project.
* Modify the `.env` file to adjust settings as needed. You can find a comprehensive list of configurable options in [`application/core/settings.py`](https://github.com/arc53/DocsGPT/blob/main/application/core/settings.py).
* Modify the `.env` file to adjust settings as needed. You can find a comprehensive list of configurable options in [`docsgpt/core/settings.py`](https://github.com/arc53/DocsGPT/blob/main/docsgpt/core/settings.py).
* **Option 2: Exporting Environment Variables:**
* Alternatively, you can export environment variables directly in your terminal. However, using a `.env` file is generally more organized for development.
@@ -83,7 +83,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
For an offline or air-gapped machine, fetch it ahead of time instead:
```bash
python -m application.scripts.prefetch_models
python -m docsgpt.scripts.prefetch_models
```
4. **Install Backend Dependencies:**
@@ -91,7 +91,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
Navigate to the root of your DocsGPT repository and install the required Python packages:
```bash
pip install -r application/requirements.txt
pip install -r docsgpt/requirements.txt
```
Dependencies are declared in `pyproject.toml` and locked in `uv.lock`; the
@@ -103,8 +103,8 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
feature (each file is the core set plus the extra):
```bash
pip install -r application/requirements-docling.txt # docling parser engine: OCR backend, read_document structured output
pip install -r application/requirements-milvus.txt # VECTOR_STORE=milvus
pip install -r docsgpt/requirements-docling.txt # docling parser engine: OCR backend, read_document structured output
pip install -r docsgpt/requirements-milvus.txt # VECTOR_STORE=milvus
# or with uv: uv sync --extra docling --extra milvus
```
@@ -122,25 +122,25 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
For local development, run the ASGI composition under uvicorn. It serves the **whole** application, hot-reloads on source changes, and matches the production runtime:
```bash
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091 --reload
uvicorn docsgpt.asgi:asgi_app --host 0.0.0.0 --port 7091 --reload
```
This makes the backend accessible on `http://localhost:7091`. Production uses `gunicorn -k uvicorn_worker.UvicornWorker` against the same `application.asgi:asgi_app` target.
This makes the backend accessible on `http://localhost:7091`. Production uses `gunicorn -k uvicorn_worker.UvicornWorker` against the same `docsgpt.asgi:asgi_app` target.
A plain Flask run is a faster inner loop (quick startup, the Werkzeug interactive debugger):
```bash
flask --app application/app.py run --host=0.0.0.0 --port=7091
flask --app docsgpt/app.py run --host=0.0.0.0 --port=7091
```
But it serves **only** the WSGI Flask app and omits the routes mounted on the ASGI shell in `application/asgi.py`: the `/mcp` FastMCP endpoint and the native-async SSE reconnect reader `GET /api/messages/<id>/events`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but a stream interrupted by a disconnect won't auto-resume on reconnect. Use `flask run` only when you don't need those routes.
But it serves **only** the WSGI Flask app and omits the routes mounted on the ASGI shell in `docsgpt/asgi.py`: the `/mcp` FastMCP endpoint and the native-async SSE reconnect reader `GET /api/messages/<id>/events`. Under `flask run` those paths return 404 — chat still works (`POST /stream` is a Flask route), but a stream interrupted by a disconnect won't auto-resume on reconnect. Use `flask run` only when you don't need those routes.
6. **Start the Celery Worker:**
Open a new terminal window (and activate your virtual environment if you used one). Start the Celery worker to handle background tasks:
```bash
celery -A application.app.celery worker -l INFO
celery -A docsgpt.app.celery worker -l INFO
```
This command will start the Celery worker, which processes tasks such as document parsing and vector embedding.
@@ -148,7 +148,7 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
**macOS note:** Due to a threading issue, start Celery with the solo pool:
```bash
python -m celery -A application.app.celery worker -l INFO --pool=solo
python -m celery -A docsgpt.app.celery worker -l INFO --pool=solo
```
**Running in Debugger (VSCode):**
+2 -2
View File
@@ -69,8 +69,8 @@ checkout.
## Using the Source Checkout
With a clone of the repository, `deployment/docker-compose-hub.yaml` runs the
same pre-built images while keeping your data in `application/indexes`,
`application/inputs` and `application/vectors`, and `deployment/docker-compose.yaml`
same pre-built images while keeping your data in `docsgpt/indexes`,
`docsgpt/inputs` and `docsgpt/vectors`, and `deployment/docker-compose.yaml`
builds the images from your working tree (for local changes, or a build with
extra packages: `EXTRAS=docling` in `.env`).
+9 -9
View File
@@ -29,11 +29,11 @@ LLM_NAME=gpt-4o
### 2. Configuration via `settings.py` file (Advanced)
For more advanced configurations or if you prefer to manage settings directly in code, you can modify the `settings.py` file. This file is located in the `application/core` directory of your DocsGPT project.
For more advanced configurations or if you prefer to manage settings directly in code, you can modify the `settings.py` file. This file is located in the `docsgpt/core` directory of your DocsGPT project.
While modifying `settings.py` offers more flexibility, it's generally recommended to use the `.env` file for basic settings and reserve `settings.py` for more complex adjustments or when you need to configure settings programmatically.
**Location of `settings.py`:** `application/core/settings.py`
**Location of `settings.py`:** `docsgpt/core/settings.py`
## Basic Settings Explained
@@ -61,7 +61,7 @@ Here are some of the most fundamental settings you'll likely want to configure:
- **Default value:** leave it unset and DocsGPT picks for you at first boot, recording the choice so it never changes underneath you: a fresh install is pinned to `ibm-granite/granite-embedding-311m-multilingual-r2` (multilingual, 32k context, same 768 dimensions), and an install that already has sources is pinned to `huggingface_sentence-transformers/all-mpnet-base-v2` so its index stays readable. Setting it here overrides that pin.
- **Other options:** Any FastEmbed built-in model, or any Hugging Face repository shipping an ONNX export. See [Embeddings](/Models/embeddings).
- **Changing it on an existing index requires re-embedding** — same-width models swap without any error and silently degrade retrieval. Run `python -m application.scripts.reembed`.
- **Changing it on an existing index requires re-embedding** — same-width models swap without any error and silently degrade retrieval. Run `python -m docsgpt.scripts.reembed`.
- **`API_KEY`**: Required for most cloud-based LLM providers. This is your authentication key to access the LLM provider's API. You'll need to obtain this key from your chosen provider's platform.
@@ -133,7 +133,7 @@ models:
After restart, those models appear in `/api/models` and are selectable
in the UI. A working template lives at
`application/core/models/examples/mistral.yaml.example`.
`docsgpt/core/models/examples/mistral.yaml.example`.
**What you can do:**
@@ -148,8 +148,8 @@ in the UI. A working template lives at
**What you cannot do via `MODELS_CONFIG_DIR`:** add a brand-new
non-OpenAI provider. That requires a Python plugin under
`application/llm/providers/`. See
`application/core/models/README.md` for the full schema reference.
`docsgpt/llm/providers/`. See
`docsgpt/core/models/README.md` for the full schema reference.
### Docker
@@ -207,7 +207,7 @@ for the engines and flows.
| Setting | Default | Description |
| --- | --- | --- |
| `DOC_PARSER_ENGINE` | `anydoc` | Parser engine: `anydoc` (fast Rust converter, no ML models) or `docling` (layout/table models, structured output). Files anydoc cannot read fall back to Docling when installed. Docling is an optional extra: `pip install -r application/requirements-docling.txt`, or Docker builds with `--build-arg INSTALL_DOCLING=true`. |
| `DOC_PARSER_ENGINE` | `anydoc` | Parser engine: `anydoc` (fast Rust converter, no ML models) or `docling` (layout/table models, structured output). Files anydoc cannot read fall back to Docling when installed. Docling is an optional extra: `pip install -r docsgpt/requirements-docling.txt`, or Docker builds with `--build-arg INSTALL_DOCLING=true`. |
| `OCR_ENABLED` | `false` | OCR for source ingestion. Alias: `DOCLING_OCR_ENABLED`. |
| `OCR_ATTACHMENTS_ENABLED` | `false` | OCR for chat attachments. Alias: `DOCLING_OCR_ATTACHMENTS_ENABLED`. |
| `OCR_BACKEND` | `auto` | Who performs OCR: `auto` (Docling when installed, else native), `native` (pypdfium2 + Pillow rendering into tesseract or DeepSeek-OCR; no docling needed), or `docling` (layout-model hybrid OCR). See the [OCR guide](/Guides/ocr#ocr-backends). |
@@ -469,7 +469,7 @@ See [Embeddings](/Models/embeddings) for full guidance.
| Setting | Default | Description |
| --- | --- | --- |
| `EMBEDDINGS_NAME` | `huggingface_sentence-transformers/all-mpnet-base-v2` | The embedding model. New installs use `ibm-granite/granite-embedding-311m-multilingual-r2`. Changing it on a populated index requires `application.scripts.reembed`. |
| `EMBEDDINGS_NAME` | `huggingface_sentence-transformers/all-mpnet-base-v2` | The embedding model. New installs use `ibm-granite/granite-embedding-311m-multilingual-r2`. Changing it on a populated index requires `docsgpt.scripts.reembed`. |
| `EMBEDDINGS_BASE_URL` | unset | Base URL of a remote OpenAI-compatible embeddings server. Setting it routes all embedding calls there. |
| `EMBEDDINGS_THREADS` | unset (Docker image: `4`) | Threads one local FastEmbed/onnxruntime session may use. onnxruntime otherwise sizes its pool to the host's core count, which a CPU-limited container still reports, so the image pins it like `OMP_NUM_THREADS`. Raise it on a large dedicated worker. |
| `EMBEDDINGS_KEY` | unset | Optional bearer token for the remote embeddings server. |
@@ -550,4 +550,4 @@ These are just the basic settings to get you started. The `settings.py` file con
- Cache settings (`CACHE_REDIS_URL`)
- And many more!
For a complete list of available settings and their descriptions, refer to the `settings.py` file in `application/core`. Remember to restart your Docker containers after making changes to your `.env` file or `settings.py` for the changes to take effect.
For a complete list of available settings and their descriptions, refer to the `settings.py` file in `docsgpt/core`. Remember to restart your Docker containers after making changes to your `.env` file or `settings.py` for the changes to take effect.
+7 -7
View File
@@ -8,7 +8,7 @@ import { Callout } from 'nextra/components'
# Observability
DocsGPT bundles the OpenTelemetry SDK and auto-instrumentation packages
in `application/requirements.txt` — they install with the rest of the
in `docsgpt/requirements.txt` — they install with the rest of the
backend deps. Telemetry is **off by default**; opt in by prefixing the
launch command with `opentelemetry-instrument` and setting OTLP env
vars.
@@ -42,12 +42,12 @@ services:
backend:
command: >
opentelemetry-instrument gunicorn -w 1 -k uvicorn_worker.UvicornWorker
--bind 0.0.0.0:7091 --config application/gunicorn_conf.py
application.asgi:asgi_app
--bind 0.0.0.0:7091 --config docsgpt/gunicorn_conf.py
docsgpt.asgi:asgi_app
environment:
- OTEL_SERVICE_NAME=docsgpt-backend
worker:
command: opentelemetry-instrument celery -A application.app.celery worker -l INFO -B
command: opentelemetry-instrument celery -A docsgpt.app.celery worker -l INFO -B
environment:
- OTEL_SERVICE_NAME=docsgpt-celery-worker
```
@@ -56,14 +56,14 @@ For local dev, prepend `dotenv run --` so the `OTEL_*` vars from `.env`
reach `opentelemetry-instrument` before it boots the SDK:
```bash
dotenv run -- opentelemetry-instrument flask --app application/app.py run --port=7091
dotenv run -- opentelemetry-instrument celery -A application.app.celery worker -l INFO --pool=solo
dotenv run -- opentelemetry-instrument flask --app docsgpt/app.py run --port=7091
dotenv run -- opentelemetry-instrument celery -A docsgpt.app.celery worker -l INFO --pool=solo
```
<Callout type="info" emoji="ℹ️">
Logs are exported in-process when `OTEL_LOGS_EXPORTER=otlp` is set —
`application/core/logging_config.py` detects the flag and preserves
`docsgpt/core/logging_config.py` detects the flag and preserves
the OTEL log handler. Without it, `logging` writes only to stdout.
</Callout>
@@ -37,7 +37,7 @@ schema on first boot.
```bash
export POSTGRES_URI="postgresql://user:pass@host/docsgpt?sslmode=require"
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091
uvicorn docsgpt.asgi:asgi_app --host 0.0.0.0 --port 7091
```
### Bare-metal Postgres
@@ -47,7 +47,7 @@ First boot creates both the database and the schema.
```bash
export POSTGRES_URI="postgresql://postgres@localhost/docsgpt"
uvicorn application.asgi:asgi_app --host 0.0.0.0 --port 7091
uvicorn docsgpt.asgi:asgi_app --host 0.0.0.0 --port 7091
```
Prefer a dedicated non-superuser role? Create it once as superuser — the
@@ -93,7 +93,7 @@ init-container ahead of the app rollout:
```bash
python scripts/db/init_postgres.py
# equivalently:
alembic -c application/alembic.ini upgrade head
alembic -c docsgpt/alembic.ini upgrade head
```
The reasoning: the app's runtime role shouldn't carry DDL privileges,
@@ -112,7 +112,7 @@ One-shot, offline, app stopped. The app itself will create the
Postgres schema when it boots — you only need to run the data copy.
```bash
pip install -r application/requirements.txt
pip install -r docsgpt/requirements.txt
pip install 'pymongo>=4.6'
export POSTGRES_URI="postgresql://docsgpt:docsgpt@localhost:5432/docsgpt"