tiennm99/DocsGPT
1
0 Fork 0
mirror of https://github.com/tiennm99/DocsGPT.git synced 2026-10-13 20:14:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

standalone OCR

Browse Source
This commit is contained in:
Pavel committed 2026-09-02 23:12:36 +04:00
1 parent d4e92be0ab
commit 96b878217d
27 files changed
+1995 -152
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/content/Deploying/DocsGPT-Settings.mdx
+8 -4
View File
@@ -206,19 +206,23 @@ for the engines and flows.
| Setting | Default | Description |
| --- | --- | --- |
| `DOC_PARSER_ENGINE` | `anydoc` | Parser engine: `anydoc` (fast Rust converter, no ML models) or `docling` (layout/table models, OCR, structured output). Files anydoc cannot read fall back to Docling when installed. Docling is an optional extra: `pip install -r application/requirements-docling.txt`, or Docker builds with `--build-arg INSTALL_DOCLING=true`. |
| `DOCLING_OCR_ENABLED` | `false` | OCR for source ingestion (Docling only). |
| `DOCLING_OCR_ATTACHMENTS_ENABLED` | `false` | OCR for chat attachments (Docling only). |
| `DOC_PARSER_ENGINE` | `anydoc` | Parser engine: `anydoc` (fast Rust converter, no ML models) or `docling` (layout/table models, structured output). Files anydoc cannot read fall back to Docling when installed. Docling is an optional extra: `pip install -r application/requirements-docling.txt`, or Docker builds with `--build-arg INSTALL_DOCLING=true`. |
| `OCR_ENABLED` | `false` | OCR for source ingestion. Alias: `DOCLING_OCR_ENABLED`. |
| `OCR_ATTACHMENTS_ENABLED` | `false` | OCR for chat attachments. Alias: `DOCLING_OCR_ATTACHMENTS_ENABLED`. |
| `OCR_BACKEND` | `auto` | Who performs OCR: `auto` (Docling when installed, else native), `native` (pypdfium2 + Pillow rendering into tesseract or DeepSeek-OCR; no docling needed), or `docling` (layout-model hybrid OCR). See the [OCR guide](/Guides/ocr#ocr-backends). |
| `ATTACHMENT_PDF_TEXT_FAST_PATH` | `true` | Under the `docling` engine, read PDF attachments through their text layer (pypdfium2) and only hand scans to Docling. Not needed under `anydoc`. |
| `DOCLING_TABULAR_MAX_BYTES` | `2000000` | Docling engine: CSV/XLSX larger than this (by content) use the lightweight tabular parsers. |
| `DOCLING_MARKUP_MAX_BYTES` | `8000000` | Docling engine: HTML/VTT larger than this are head-truncated before parsing. |
| `MARKUP_MAX_BYTES` | `8000000` | anydoc engine: HTML/XHTML larger than this are head-truncated before the `markdownify` parser runs (its tree costs ~50x the input). `0` disables the gate. |
| `PDF_TRUST_CHECK` | `true` | Trust-check anydoc's PDF output for the two silent-loss classes (Type0 fonts without ToUnicode, CJK-declaring PDFs with CJK-less text). Flagged files re-parse on Docling when installed, else carry `parse_warnings` metadata. |
| `ANYDOC_TABLEIZE` | `false` | Rewrite dot-leader / whitespace-aligned table runs in anydoc's PDF markdown into GFM tables. |
| `OCR_ENGINE` | `tesseract` | OCR engine when OCR is on: `tesseract` (recommended), `auto`, `ocrmac`, `rapidocr`, or `deepseek` (VLM endpoint). Unavailable engines degrade to `auto` with a warning. See the [OCR guide](/Guides/ocr). |
| `OCR_ENGINE` | `tesseract` | OCR engine when OCR is on: `tesseract` (recommended, both backends), `deepseek` (VLM endpoint, both backends), or the Docling-only `auto`, `ocrmac`, `rapidocr`. Unavailable engines degrade with a warning. See the [OCR guide](/Guides/ocr). |
| `OCR_LANGS` | `eng` | Tesseract language packs, `+`-separated (e.g. `eng+chi_sim+deu`). |
| `OCR_DEEPSEEK_URL` | `http://localhost:11434/v1/chat/completions` | OpenAI-compatible endpoint for `OCR_ENGINE=deepseek` (Ollama or vLLM). |
| `OCR_DEEPSEEK_MODEL` | `deepseek-ocr:3b` | Model name at that endpoint. |
| `OCR_DEEPSEEK_TIMEOUT` | `300` | Native backend: seconds allowed per page request to the DeepSeek endpoint (pages are sent one at a time). |
| `OCR_RENDER_DPI` | `200` | Native backend: resolution at which pages without a text layer are rendered before OCR (clamped to 72-600). |
| `OCR_MIN_CHARS_PER_PAGE` | `20` | Chars-per-page floor below which an OCR'd parse is treated as an OCR dropout and fails loudly instead of indexing an empty document; `0` disables. Alias: `DOCLING_OCR_MIN_CHARS_PER_PAGE`. |
## Speech-to-Text Settings
docs/content/Guides/ocr.mdx
+126 -40
View File
@@ -26,9 +26,10 @@ DOC_PARSER_ENGINE=anydoc
anydoc's other formats (DOC, PPT/PPS/POT, XLS, ODT/ODS/ODP, RTF and the
macro-enabled Office variants). It never performs OCR:
a scanned or image-only PDF is *detected* and handed to the fallback
parser — Docling when it is installed, the legacy text parsers otherwise —
and if nothing can read the file the upload fails with a clear error
instead of storing an empty document.
parser — the active OCR backend when OCR is on (see below), Docling when
it is installed, the legacy text parsers otherwise — and if nothing can
read the file the upload fails with a clear error instead of storing an
empty document.
- `docling`: the previous default. Docling's layout and table models produce
structured Markdown, support OCR, and back the `read_document` tool's
`structured` output, at the cost of a large dependency tree (torch,
@@ -36,14 +37,16 @@ DOC_PARSER_ENGINE=anydoc
this one variable; nothing else changes.
With `DOC_PARSER_ENGINE=anydoc`, Docling still handles what anydoc cannot when
it is installed: image OCR (`DOCLING_OCR_ENABLED`), `.adoc`, `.vtt` and `.xml`
sources, and the fallback for files anydoc rejects. Without Docling those
formats use the standard parsers, and images are only read when
it is installed: `.adoc`, `.vtt` and `.xml` sources, the fallback for files
anydoc rejects, and — when it is the OCR backend — scanned PDFs and images.
Without Docling those formats use the standard parsers; OCR still works
through the native backend, and with OCR off images are only read when
`PARSE_IMAGE_REMOTE=true`.
## Installing the docling engine
docling is not part of the base install. Add it when you need OCR,
docling is not part of the base install, and OCR does not need it (see
[OCR backends](#ocr-backends)). Add it when you want its layout-model OCR,
`.adoc`/`.vtt`/`.xml` source parsing, or `read_document`'s `structured`
output:
@@ -59,44 +62,79 @@ docker build --build-arg INSTALL_DOCLING=true ./application
`deployment/docker-compose.yaml` forwards the same switch, so setting
`INSTALL_DOCLING=true` in `.env` (or the shell) bakes docling into locally
built backend and worker images. `setup.sh` writes it automatically when you
answer yes to the OCR question. Either way no code changes are needed —
docling is picked up as the fallback engine as soon as it is importable, and
`DOC_PARSER_ENGINE=docling` makes it the primary one.
built backend and worker images; `setup.sh` offers it as a follow-up to the
OCR question. Either way no code changes are needed — docling is picked up as
the fallback engine (and, under `OCR_BACKEND=auto`, as the OCR backend) as
soon as it is importable, and `DOC_PARSER_ENGINE=docling` makes it the
primary parser.
## Docling OCR
## OCR
OCR is optional, Docling-only, and controlled by two on/off settings plus an
OCR is optional and controlled by two on/off settings, a backend and an
engine choice:
```env
DOCLING_OCR_ENABLED=false
DOCLING_OCR_ATTACHMENTS_ENABLED=false
OCR_ENABLED=false
OCR_ATTACHMENTS_ENABLED=false
OCR_BACKEND=auto
OCR_ENGINE=tesseract
```
- `DOCLING_OCR_ENABLED`: OCR behavior for Source Docs ingestion.
- `DOCLING_OCR_ATTACHMENTS_ENABLED`: OCR behavior for chat attachments uploaded from the message box.
- `OCR_ENGINE`: which engine performs the OCR when it is on (next section).
- `OCR_ENABLED`: OCR behavior for Source Docs ingestion.
- `OCR_ATTACHMENTS_ENABLED`: OCR behavior for chat attachments uploaded from the message box.
- `OCR_BACKEND`: which stack performs the OCR (next section).
- `OCR_ENGINE`: which recognition engine it uses ([Choosing the OCR engine](#choosing-the-ocr-engine)).
The older names `DOCLING_OCR_ENABLED` and `DOCLING_OCR_ATTACHMENTS_ENABLED`
are still accepted as aliases.
Under the default anydoc engine, a scanned PDF reaches OCR through anydoc's
own detection: anydoc refuses it ("OCR is required") and the docling fallback
takes over. If that fallback also extracts almost nothing — OCR off, or no
docling installed — the upload now fails with a clear message instead of
silently indexing an empty document.
own detection: anydoc refuses it ("OCR is required") and the OCR backend
takes over as the fallback parser. If that fallback also extracts almost
nothing — OCR off, or no engine available — the upload fails with a clear
message instead of silently indexing an empty document.
Mixed documents — text pages with scanned pages among them — convert
through anydoc, which reads the text pages and skips the scanned ones. With
OCR on, DocsGPT probes every page's text layer, OCRs the pages that have
none through the active backend, and appends their text; the document's
metadata records the count as `ocr_pages`. With OCR off, only the text pages
are indexed.
## OCR backends
Docling is not the only way to OCR. The `native` backend renders the pages
that need it with pypdfium2 and Pillow — both already core dependencies —
and feeds them straight into tesseract or a DeepSeek-OCR endpoint. No ML
models load in the worker, and nothing beyond the ~35 MB tesseract binary
(shipped in the Docker image) is installed.
| `OCR_BACKEND` | What runs | When to pick it |
|---|---|---|
| `auto` (default) | Docling when the docling extra is installed, `native` otherwise. | Leave it: a plain install gets working OCR from tesseract alone, and installing docling upgrades OCR without touching config. |
| `native` | pypdfium2 + Pillow page rendering into `tesseract` or `deepseek`. Pages that carry a text layer are read directly and never OCR'd; pages without one are rendered at `OCR_RENDER_DPI` (200) and OCR'd. Multi-frame TIFFs are read frame by frame. | You do not want docling's dependency tree or memory footprint, or you have docling installed for `structured` output but want lightweight OCR. |
| `docling` | Docling's layout-model pipeline: hybrid OCR (only the bitmap regions of a page), reading-order recovery, TableFormer table structure, and the `auto` / `ocrmac` / `rapidocr` engines. | Multi-column scans, scanned tables you need as Markdown tables under tesseract, or macOS `ocrmac`. Needs the docling extra. |
The trade-off is the layout model. Under `native`, multi-column scans rely
on tesseract's own page segmentation and tesseract yields tables as plain
lines; DeepSeek-OCR emits Markdown tables itself, so it remains the quality
path on either backend. Under `DOC_PARSER_ENGINE=docling` with the `native`
backend, PDFs whose every page has a text layer still go through Docling
(OCR off) for its structured Markdown; only documents with scanned pages
take the native path.
## Choosing the OCR engine
Benchmarked 2026-08 on English, bilingual EN/ZH, table-heavy and degraded
scans (all engines driven through docling so layout handling is identical):
| `OCR_ENGINE` | Role | Notes |
|---|---|---|
| `tesseract` | **recommended default** | Best classic-engine accuracy in the bench: perfect EN word recall on all docs, 0.000 CER on the bilingual page, 100% table cells, robust to mild degradation. ~35 MB of *system* packages, CPU-only. Needs the `tesseract` binary + language packs in the image — installed automatically when the Docker image is built with `INSTALL_DOCLING=true`; set languages via `OCR_LANGS` (e.g. `eng+chi_sim`). |
| `auto` | convenience | docling picks: `ocrmac` on macOS (excellent, ~1 s/page), `rapidocr` on Linux — see below before relying on it server-side. Also the automatic fallback whenever the selected engine is not installed. |
| `ocrmac` | macOS only | Best raw accuracy and fastest of all classic engines; irrelevant for Linux deploys. |
| `rapidocr` | pip-only fallback | No system packages needed, perfect on tables/CJK — but it silently shreds some long text lines into garbage at every setting tried, which is content loss for RAG ingestion. Avoid as a server default until fixed upstream. |
| `deepseek` | best quality, heavy on the *server* | DeepSeek-OCR through docling's VLM pipeline. Only engine that reconstructs totals rows as table rows; near-perfect CJK; barely affected by degradation. The ingestion worker gets *lighter* (~370 MB RSS — no layout models); the model runs in Ollama or vLLM. Costs: a GPU/Apple-Silicon endpoint, ~seconds per page, and occasional silent drops of page-level elements (titles). |
| `OCR_ENGINE` | Backends | Role | Notes |
|---|---|---|---|
| `tesseract` | native, docling | **recommended default** | Best classic-engine accuracy in the bench: perfect EN word recall on all docs, 0.000 CER on the bilingual page, 100% table cells, robust to mild degradation. ~35 MB of *system* packages, CPU-only. Needs the `tesseract` binary + language packs — shipped in the Docker image (opt out with `--build-arg INSTALL_TESSERACT=false`); set languages via `OCR_LANGS` (e.g. `eng+chi_sim`). |
| `deepseek` | native, docling | best quality, heavy on the *server* | DeepSeek-OCR against an OpenAI-compatible endpoint. Only engine that reconstructs totals rows as table rows; near-perfect CJK; barely affected by degradation. The ingestion worker stays light (no layout models); the model runs in Ollama or vLLM. Costs: a GPU/Apple-Silicon endpoint, ~seconds per page, and occasional silent drops of page-level elements (titles). |
| `auto` | docling | convenience | docling picks: `ocrmac` on macOS (excellent, ~1 s/page), `rapidocr` on Linux — see below before relying on it server-side. Also docling's automatic fallback whenever the selected engine is not installed. The native backend runs tesseract for it. |
| `ocrmac` | docling | macOS only | Best raw accuracy and fastest of all classic engines; irrelevant for Linux deploys. |
| `rapidocr` | docling | pip-only fallback | No system packages needed, perfect on tables/CJK — but it silently shreds some long text lines into garbage at every setting tried, which is content loss for RAG ingestion. Avoid as a server default until fixed upstream. |
For `deepseek`, point the worker at an OpenAI-compatible endpoint:
@@ -104,14 +142,61 @@ For `deepseek`, point the worker at an OpenAI-compatible endpoint:
OCR_ENGINE=deepseek
OCR_DEEPSEEK_URL=http://localhost:11434/v1/chat/completions # Ollama default
OCR_DEEPSEEK_MODEL=deepseek-ocr:3b
OCR_DEEPSEEK_TIMEOUT=300 # native backend: seconds per page
```
Ollama works out of the box (`ollama pull deepseek-ocr:3b`); for real
throughput serve `deepseek-ai/DeepSeek-OCR` with vLLM on a GPU and set the
URL accordingly.
URL accordingly. The native backend sends pages one at a time, so a slow
laptop-hosted model only needs a generous `OCR_DEEPSEEK_TIMEOUT`; docling's
VLM pipeline uses its own 90 s timeout and four concurrent requests.
A selected engine that is not available (no tesseract binary, no macOS for
ocrmac) degrades to `auto` with a warning rather than failing parses.
A selected engine that is not available degrades rather than failing parses:
under docling, a missing tesseract binary or non-macOS `ocrmac` falls back to
`auto` with a warning; under `native`, a docling-only engine becomes
tesseract, and a missing tesseract binary fails the scanned file with an
install hint (text-layer documents are unaffected).
<Callout type="warning" emoji="⚠️">
A language listed in `OCR_LANGS` whose tesseract pack is not installed fails
every scanned page loudly (tesseract exits with "Error opening data file
... chi_sim.traineddata"), on both backends. Install the pack before
listing it: `apt-get install tesseract-ocr-chi-sim` in the image, or on
macOS download `chi_sim.traineddata` into `$(brew --prefix)/share/tessdata`.
</Callout>
### DeepSeek through docling versus native
`OCR_ENGINE=deepseek` behaves differently on the two backends, and the
difference matters more than for tesseract:
- **native** sends one request per page that lacks a text layer and reads
every other page directly. Pages go one at a time with
`OCR_DEEPSEEK_TIMEOUT` per request, so a slow model server just takes
longer.
- **docling** uses its VLM pipeline, which *replaces* the whole converter:
every page of every PDF that reaches docling goes to the model, text layer
or not (a 39-page text PDF measured 721 s against 20 s on the other paths),
with four concurrent requests and a fixed 90 s timeout. In testing it also
dropped page-level elements — a title, an intro paragraph — that the same
model kept through the native path, and it logs a burst of pydantic
serialization warnings per conversion. Its one advantage was table
structure: it was the only configuration that returned an invoice's
subtotal/VAT/total rows as table rows.
Measured on the same scans (2026-09, MacBook Air, Ollama `deepseek-ocr:3b`):
| Configuration | Scanned page | Image | Accuracy | Tables | Chinese |
|---|---|---|---|---|---|
| tesseract, native | 1-2 s | 1-2 s | exact | flat lines | clean |
| deepseek, native | 15-30 s | 15-25 s | exact | Markdown table, totals as bold lines | clean |
| tesseract, docling | 3-4 s | ~8 s | exact | Markdown body, totals shredded | clean |
| deepseek, docling | ~20 s | ~20 s | drops titles/paragraphs | exact | clean |
Recommendation: `native` with `tesseract` for throughput, `native` with
`deepseek` for quality, `docling` with `tesseract` when you want its layout
model, and `docling` with `deepseek` only for table-heavy scans where the
costs above are acceptable.
## Processing Flow
@@ -120,7 +205,7 @@ ocrmac) degrades to `auto` with a warning rather than failing parses.
1. Files are uploaded through `/api/upload`.
2. Ingestion runs asynchronously in Celery (`ingest_worker`).
3. `SimpleDirectoryReader` parses files with `get_default_file_extractor`.
4. Documents are parsed by the `DOC_PARSER_ENGINE` engine; images (and, under `anydoc`, scanned PDFs) reach Docling when it is installed. OCR in this path is controlled by `DOCLING_OCR_ENABLED`.
4. Documents are parsed by the `DOC_PARSER_ENGINE` engine; images (and, under `anydoc`, scanned PDFs) reach the OCR backend. OCR in this path is controlled by `OCR_ENABLED`.
5. Parsed text is chunked, embedded, and stored in the vector store.
6. Retrieval during chat uses this indexed text and returns source citations.
@@ -128,13 +213,13 @@ ocrmac) degrades to `auto` with a warning rather than failing parses.
1. Files are uploaded through `/api/store_attachment`.
2. Celery task `attachment_worker` parses and stores the attachment in Postgres (`attachments` table).
3. OCR in this path is controlled by `DOCLING_OCR_ATTACHMENTS_ENABLED`.
3. OCR in this path is controlled by `OCR_ATTACHMENTS_ENABLED`.
4. Attachments are not vectorized and are not added to the source index.
5. During answer generation, selected attachment IDs are loaded and passed directly to the LLM pipeline.
## How Docling OCR Works
Docling OCR behavior is different for PDFs vs images:
With `OCR_BACKEND=docling`, OCR behavior is different for PDFs vs images:
- PDF parser defaults to hybrid OCR:
- text regions: extracted directly
@@ -182,18 +267,19 @@ This means OCR quality is especially important for text fallback paths and for m
## Recommended Configuration
For most OCR-enabled use cases, enable both flags:
For most OCR-enabled use cases, enable both flags and leave the backend on
`auto`:
```env
DOCLING_OCR_ENABLED=true
DOCLING_OCR_ATTACHMENTS_ENABLED=true
OCR_ENABLED=true
OCR_ATTACHMENTS_ENABLED=true
```
After changing these settings, restart the API and Celery worker.
## Legacy Fallback Notes
- If Docling is unavailable, DocsGPT falls back to legacy parsers for the formats anydoc does not cover, and anydoc's own refusals become upload errors.
- If Docling is unavailable, DocsGPT falls back to the native OCR parsers (OCR on) or the legacy parsers (OCR off) for the formats anydoc does not cover, and anydoc's refusals of scanned PDFs become upload errors only when no OCR is available.
- With OCR disabled, text-based PDFs can still parse, but scanned/image-heavy content may produce little text.
- For image parsing without Docling OCR, the legacy image parser only extracts text when `PARSE_IMAGE_REMOTE=true`.
- For image parsing without OCR, the legacy image parser only extracts text when `PARSE_IMAGE_REMOTE=true`.