mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-14 02:15:06 +00:00
Merge branch 'main' into anydoc-support
Conflicts, and how each was taken: - application/core/settings.py — ours. The renamed OCR_ENABLED / OCR_ATTACHMENTS_ENABLED / OCR_MIN_CHARS_PER_PAGE accept main's DOCLING_OCR_* spellings as AliasChoices, so nothing is dropped. - application/Dockerfile — both. Main's install layers plus the INSTALL_DOCLING build arg. - application/parser/file/constants.py — both imports. - deployment/docker-compose.yaml — both. The INSTALL_DOCLING / INSTALL_TESSERACT build args on backend and worker, and main's -Q docsgpt,parsing,embeddings, which query embedding needs. - tests/conftest.py — theirs. Both sides fixed the same pytest-postgresql 9.0.0 autocommit= breakage; main's spelling is the one already on main. - application/requirements.txt — the comments claimed different reasons torch is in core. Main's is the true one now: it removed sentence-transformers, so docling is torch's only remaining consumer. Two things the merge broke without conflicting: - onnxruntime. This branch moved it out of core into the docling extra; main meanwhile made it the runtime local embeddings execute on (fastembed). Git took the deletion, leaving fastembed with no pinned runtime in a repo that pins everything. Restored to core, and no longer pinned twice from the extra. - The frontend copy of ATTACHMENT_PARSER_EXTENSIONS. The backend list is derived and picked up the anydoc suffixes; the hand-kept frontend mirror did not, so the composer would refuse files the API accepts. tests/parser/file/test_constants.py is what caught it. ruff, pytest (9897 passed), frontend build and docs build all pass. The image build is unverified: no Docker daemon on this machine.
This commit is contained in:
commit
3947c66cda
97 files changed
+8661
-1049
No files matched your search
@@ -76,18 +76,16 @@ To run the DocsGPT backend locally, you'll need to set up a Python environment a
|
||||
venv/Scripts/activate
|
||||
```
|
||||
|
||||
3. **Download Embedding Model:**
|
||||
3. **Embedding Model (no action needed):**
|
||||
|
||||
The backend requires an embedding model. Download the `mpnet-base-v2` model and place it in the `models/` directory within the project root. You can use the following script:
|
||||
The embedding model is downloaded automatically the first time you ingest a document, and cached for subsequent runs. Set `EMBEDDINGS_CACHE_DIR` to control where.
|
||||
|
||||
For an offline or air-gapped machine, fetch it ahead of time instead:
|
||||
|
||||
```bash
|
||||
wget https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip
|
||||
unzip mpnet-base-v2.zip -d model
|
||||
rm mpnet-base-v2.zip
|
||||
python -m application.scripts.prefetch_models
|
||||
```
|
||||
|
||||
Alternatively, you can manually download the zip file from [here](https://d3dg1063dc54p9.cloudfront.net/models/embeddings/mpnet-base-v2.zip), unzip it, and place the extracted folder in `models/`.
|
||||
|
||||
4. **Install Backend Dependencies:**
|
||||
|
||||
Navigate to the root of your DocsGPT repository and install the required Python packages:
|
||||
|
||||
Reference in new issue
Block a user