Implement anydoc with docling

docs/content/Deploying/DocsGPT-Settings.mdx

@@ -198,6 +198,22 @@ These byte/count limits and the non-negative nesting depth can be overridden in
 The request limit intentionally applies only to public upload routes; internal
 worker index transfers use their own trusted service boundary.
 
+## Document Parsing
+
+Uploaded sources and chat attachments are converted to Markdown before
+chunking or prompting. See the [Document Parsing and OCR](/Guides/ocr) guide
+for the engines and flows.
+
+| Setting | Default | Description |
+| --- | --- | --- |
+| `DOC_PARSER_ENGINE` | `anydoc` | Parser engine: `anydoc` (fast Rust converter, no ML models) or `docling` (layout/table models, OCR, structured output; heavy optional install). Files anydoc cannot read fall back to Docling when installed. |
+| `DOCLING_OCR_ENABLED` | `false` | OCR for source ingestion (Docling only). |
+| `DOCLING_OCR_ATTACHMENTS_ENABLED` | `false` | OCR for chat attachments (Docling only). |
+| `ATTACHMENT_PDF_TEXT_FAST_PATH` | `true` | Under the `docling` engine, read PDF attachments through their text layer (pypdfium2) and only hand scans to Docling. Not needed under `anydoc`. |
+| `DOCLING_TABULAR_MAX_BYTES` | `2000000` | Docling engine: CSV/XLSX larger than this (by content) use the lightweight tabular parsers. |
+| `DOCLING_MARKUP_MAX_BYTES` | `8000000` | Docling engine: HTML/VTT larger than this are head-truncated before parsing. |
+| `MARKUP_MAX_BYTES` | `8000000` | anydoc engine: HTML/XHTML larger than this are head-truncated before the `markdownify` parser runs (its tree costs ~50x the input). `0` disables the gate. |
+
 ## Speech-to-Text Settings
 
 DocsGPT can transcribe audio in two places:

docs/content/Guides/ocr.mdx

@@ -1,13 +1,49 @@
 ---
-title: OCR for Sources and Attachments
-description: How OCR works in DocsGPT, how to configure it, and what changes for source ingestion vs chat attachments.
+title: Document Parsing and OCR
+description: Which engine parses uploaded documents, how OCR works in DocsGPT, how to configure it, and what changes for source ingestion vs chat attachments.
 ---
 
 import { Callout } from 'nextra/components'
 
-# Docling OCR for Sources and Attachments
+# Document Parsing and OCR for Sources and Attachments
 
-DocsGPT uses Docling as the default parser layer for many document formats. OCR is optional and controlled by two settings:
+## Parser engine
+
+DocsGPT converts uploaded documents to Markdown before chunking, embedding or
+handing them to a model. The engine is selected with one setting:
+
+```env
+DOC_PARSER_ENGINE=anydoc
+```
+
+- `anydoc` (default): [firecrawl-anydoc](https://github.com/firecrawl/anydoc),
+  a Rust converter with no ML models. It reads PDF, DOCX, PPTX, XLSX and CSV
+  in milliseconds with ~100 MB peak memory; HTML/XHTML is converted with
+  `markdownify`, head-truncated at `MARKUP_MAX_BYTES`. Source uploads and the
+  `read_document` tool accept the same file types as before (the
+  `SUPPORTED_SOURCE_EXTENSIONS` whitelist is unchanged); chat attachments,
+  which are not whitelisted by extension, can additionally be parsed from
+  anydoc's other formats (DOC, PPT/PPS/POT, XLS, ODT/ODS/ODP, RTF and the
+  macro-enabled Office variants). It never performs OCR:
+  a scanned or image-only PDF is *detected* and handed to the fallback
+  parser — Docling when it is installed, the legacy text parsers otherwise —
+  and if nothing can read the file the upload fails with a clear error
+  instead of storing an empty document.
+- `docling`: the previous default. Docling's layout and table models produce
+  structured Markdown, support OCR, and back the `read_document` tool's
+  `structured` output, at the cost of a large dependency tree (torch,
+  transformers, ONNX models) and seconds to minutes per PDF. Switching back is
+  this one variable; nothing else changes.
+
+With `DOC_PARSER_ENGINE=anydoc`, Docling still handles what anydoc cannot when
+it is installed: image OCR (`DOCLING_OCR_ENABLED`), `.adoc`, `.vtt` and `.xml`
+sources, and the fallback for files anydoc rejects. Without Docling those
+formats use the standard parsers, and images are only read when
+`PARSE_IMAGE_REMOTE=true`.
+
+## Docling OCR
+
+OCR is optional, Docling-only, and controlled by two settings:
 
 ```env
 DOCLING_OCR_ENABLED=false
@@ -24,7 +60,7 @@ DOCLING_OCR_ATTACHMENTS_ENABLED=false
 1. Files are uploaded through `/api/upload`.
 2. Ingestion runs asynchronously in Celery (`ingest_worker`).
 3. `SimpleDirectoryReader` parses files with `get_default_file_extractor`.
-4. For PDFs and image formats, Docling parsers are used. OCR in this path is controlled by `DOCLING_OCR_ENABLED`.
+4. Documents are parsed by the `DOC_PARSER_ENGINE` engine; images (and, under `anydoc`, scanned PDFs) reach Docling when it is installed. OCR in this path is controlled by `DOCLING_OCR_ENABLED`.
 5. Parsed text is chunked, embedded, and stored in the vector store.
 6. Retrieval during chat uses this indexed text and returns source citations.
 
@@ -90,7 +126,7 @@ After changing these settings, restart the API and Celery worker.
 
 ## Legacy Fallback Notes
 
-- If Docling is unavailable, DocsGPT falls back to legacy parsers.
+- If Docling is unavailable, DocsGPT falls back to legacy parsers for the formats anydoc does not cover, and anydoc's own refusals become upload errors.
 - With OCR disabled, text-based PDFs can still parse, but scanned/image-heavy content may produce little text.
 - For image parsing without Docling OCR, the legacy image parser only extracts text when `PARSE_IMAGE_REMOTE=true`.