mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-13 14:15:52 +00:00
Alex
12dd7c7eab
Five defects from a review of the embeddings work, four of them silent. - Write local files atomically. `LocalStorage.save_file` streamed straight onto the destination, so an interrupted write left a truncated file. `reembed` rewrites every index it touches, and a half-written `index.faiss` loads at neither the old width nor the new one -- the source was unrecoverable, with no backup and no temp file left behind. Bytes now land beside the destination and move into place with `os.replace`. S3 was already safe (single PUT). - Read pgvector chunks a page at a time. `reembed_pgvector` materialised every `(id, text)` row for a source before embedding -- ~1.6 GB at 200k chunks and several times that for non-Latin scripts, with the `PGresult` held alongside until the cursor closed. Inside the shipped 4Gi limit, while also holding the model, that is an OOMKill -- which is exactly the SIGKILL the point above turned into a destroyed index. It now walks the source by keyset. - Bound the first wave of delegated embeds. The failure cooldown is only latched once the first `get()` returns, so every request already in flight paid the full EMBEDDINGS_DELEGATE_TIMEOUT: measured 64 threads all timing out together, and at the shipped 60s across a 96-thread WSGI pool that is an API serving nothing at all, health checks included. One caller now probes while the rest fail fast; after a single success the gate leaves the path entirely. - Ship EMBEDDINGS_NAME commented in .env-template. The comment directly above it says to leave it commented when upgrading, and the line shipped set. Any value reaching `.env` lands in `model_fields_set`, which makes `resolve_embeddings_pin` bail -- so a template-derived `.env` disabled the legacy pin outright and repointed a populated index at a different 768-dim model, where no width check fires. The pin already picks granite for a fresh install and mpnet for an existing one, so nothing needs to be set by hand. - Stamp `sources.model` on wiki sources. They were created with the column NULL and then embedded like any other source, and the boot check reads NULL as "pre-dates the column, therefore the legacy model" -- reporting a correctly embedded source as stale on every startup of every process. Stamped at creation, and again on each page re-embed so existing rows heal. The two docs that promised the FAISS index survives a failed run said so of the embed only; both now describe the write, and upgrading.mdx says to stop ingest for the duration.
146 lines
7.1 KiB
Plaintext
146 lines
7.1 KiB
Plaintext
---
|
|
title: Upgrading DocsGPT
|
|
description: Upgrade your DocsGPT deployment across Docker Compose, source builds, and Kubernetes.
|
|
---
|
|
|
|
import { Callout } from 'nextra/components'
|
|
|
|
# Upgrading DocsGPT
|
|
|
|
<Callout type="warning">
|
|
**Upgrading from 0.16.x?** User data moved from MongoDB to Postgres in 0.17.0. Follow the [Postgres Migration guide](/Deploying/Postgres-Migration) before running `docker compose pull` or `git pull` — existing deployments will not start cleanly without it.
|
|
</Callout>
|
|
|
|
## Embedding models
|
|
|
|
DocsGPT now runs embeddings through [FastEmbed](https://github.com/qdrant/fastembed) (ONNX Runtime) instead of SentenceTransformer. The models are the same and the vectors are identical, so **your existing index needs no action** — `all-mpnet-base-v2` keeps working exactly as before.
|
|
|
|
<Callout type="warning">
|
|
**Your worker command does need one change.** Query embedding now runs on the Celery worker (`EMBEDDINGS_DELEGATE_TO_WORKER`, on by default), which keeps the API from loading a model of its own. If you start your worker with an explicit `-Q`, add the `embeddings` queue:
|
|
|
|
```diff
|
|
- celery -A application.app.celery worker -l INFO -Q docsgpt,parsing
|
|
+ celery -A application.app.celery worker -l INFO -Q docsgpt,parsing,embeddings
|
|
```
|
|
|
|
The bundled Compose and Kubernetes manifests already do this — pull them along with the code. Without it, every search blocks for `EMBEDDINGS_DELEGATE_TIMEOUT` (60s) and then answers with no retrieved context rather than raising, so the symptom is bad answers, not an error. To keep the model out of the worker too, set `EMBEDDINGS_BASE_URL`; to run the API on its own, set `EMBEDDINGS_DELEGATE_TO_WORKER=false`.
|
|
</Callout>
|
|
|
|
New installs default to `ibm-granite/granite-embedding-311m-multilingual-r2`: multilingual, a 32k-token context, and the same 768 dimensions.
|
|
|
|
### Switching an existing deployment to granite
|
|
|
|
<Callout type="error">
|
|
Changing `EMBEDDINGS_NAME` on an index that already has vectors **breaks retrieval silently**. Both models are 768-dimensional, so nothing raises an error — queries are simply compared against vectors that mean something else, and answers quietly get worse. Always re-embed.
|
|
</Callout>
|
|
|
|
Set the model, then rebuild the vectors:
|
|
|
|
```bash
|
|
# 1. In your .env
|
|
EMBEDDINGS_NAME=ibm-granite/granite-embedding-311m-multilingual-r2
|
|
|
|
# 2. Rebuild the vectors from the chunk text already in your index
|
|
docker compose exec backend python -m application.scripts.reembed --dry-run
|
|
docker compose exec backend python -m application.scripts.reembed
|
|
```
|
|
|
|
Re-embedding reads the chunk text already stored in your index. It does not re-download, re-parse or re-chunk your documents, so no source files are needed and the run is proportional to index size, not corpus size. Both `pgvector` and `faiss` are supported.
|
|
|
|
Useful flags:
|
|
|
|
| Flag | Effect |
|
|
| --- | --- |
|
|
| `--dry-run` | Report how many chunks would change, write nothing |
|
|
| `--sources a,b` | Only these source ids — also how you retry a failed source |
|
|
| `--batch-size N` | Chunks per embed call (default 64) |
|
|
|
|
The script processes sources independently: one failing source is reported and skipped rather than aborting the run, and the exit code is non-zero if any failed. For `pgvector` it reads a page of chunks at a time and updates rows in place, so memory stays flat on a large index and an interrupted run simply re-does its last batch. For `faiss` it builds the replacement index in memory, writes each file to a temporary path, and moves it into place — so an interrupt during either the rebuild or the write leaves the existing index intact rather than truncated.
|
|
|
|
<Callout type="warning">
|
|
Stop ingest before you run this. It reads each source's chunks and writes the vectors back; anything ingested while it runs can be overwritten by the rebuild (`faiss`) or missed by it (`pgvector`).
|
|
</Callout>
|
|
|
|
<Callout type="info">
|
|
Running [GraphRAG](/Sources/GraphRAG)? The script also rewrites `graph_nodes.name_embedding`, which seeds every graph traversal. Those vectors are written once at extraction time and share the chunk vectors' width, so leaving them in the old model's space degrades graph retrieval just as silently as the chunk vectors would — and needs no LLM re-extraction to fix.
|
|
</Callout>
|
|
|
|
### Custom local models
|
|
|
|
A local model now runs through ONNX Runtime, so its repository must ship an ONNX
|
|
export (`onnx/model.onnx`) or be one of FastEmbed's built-in models. Repositories
|
|
with PyTorch weights only no longer load; `hkunlp/instructor-large`, previously
|
|
supported by name, is one of them. Serve such a model over `EMBEDDINGS_BASE_URL`
|
|
instead, or switch to a model with an export.
|
|
|
|
How to run the model — pooling, and whether outputs are L2-normalised — is read
|
|
from the repository's own `1_Pooling/config.json` and `modules.json`. Two cases
|
|
need attention:
|
|
|
|
- **Models with a Dense projection layer** (`sentence-transformers/LaBSE`,
|
|
`distiluse-base-multilingual-cased-v1`) are now **refused at startup**.
|
|
FastEmbed cannot apply the projection, so it would have produced vectors of the
|
|
wrong width in a different space. If you were running one, its stored vectors
|
|
were already wrong; move it to `EMBEDDINGS_BASE_URL` or pick another model.
|
|
- **Repositories that declare nothing** fall back to mean pooling with
|
|
normalisation and log a warning. Pin the real values with `EMBEDDINGS_POOLING`
|
|
(`cls` or `mean`) and `EMBEDDINGS_NORMALIZE`.
|
|
|
|
<Callout type="info">
|
|
Staying on `all-mpnet-base-v2` is a supported choice — it remains in the model registry and in `setup.sh`. You only need this section if you want to move to granite.
|
|
</Callout>
|
|
|
|
## Check your version
|
|
|
|
```bash
|
|
docker compose exec backend python -c "from application.version import get_version; print(get_version())"
|
|
```
|
|
|
|
Release notes: [changelog](/changelog). Tags: [GitHub releases](https://github.com/arc53/DocsGPT/releases).
|
|
|
|
## Docker Compose — hub images
|
|
|
|
```bash
|
|
cd DocsGPT/deployment
|
|
docker compose -f docker-compose-hub.yaml pull
|
|
docker compose -f docker-compose-hub.yaml up -d
|
|
```
|
|
|
|
`pull` fetches the latest image for whichever tag your compose file references. To move to a specific release, edit `image: arc53/docsgpt:<tag>` first.
|
|
|
|
## Docker Compose — from source
|
|
|
|
```bash
|
|
cd DocsGPT
|
|
git pull
|
|
docker compose -f deployment/docker-compose.yaml build
|
|
docker compose -f deployment/docker-compose.yaml up -d
|
|
```
|
|
|
|
Swap `git pull` for `git checkout <tag>` if you want to pin a specific release.
|
|
|
|
## Kubernetes
|
|
|
|
```bash
|
|
kubectl set image deployment/docsgpt-backend backend=arc53/docsgpt:<tag>
|
|
kubectl set image deployment/docsgpt-worker worker=arc53/docsgpt:<tag>
|
|
kubectl rollout status deployment/docsgpt-backend
|
|
kubectl rollout status deployment/docsgpt-worker
|
|
```
|
|
|
|
Full manifests: [Kubernetes deployment guide](/Deploying/Kubernetes-Deploying).
|
|
|
|
## Migrations
|
|
|
|
Alembic migrations run on worker startup. To apply manually:
|
|
|
|
```bash
|
|
docker compose exec backend alembic -c application/alembic.ini upgrade head
|
|
```
|
|
|
|
`upgrade head` is idempotent.
|
|
|
|
## Rollback
|
|
|
|
Set the image tag to the previous release and `up -d` again. Schema changes are not reversible without a backup — take one before upgrading any release that mentions migrations in the changelog.
|