Commit Graph
15 Commits
Author SHA1 Message Date
Alex 574f96341e refactor: rename the application package to docsgpt
The backend import package is now docsgpt, the name it will carry on PyPI;
application was far too generic to install into anyone's site-packages.
git mv plus a mechanical rewrite of every import, dotted string and path
reference: 734 Python files, the compose files, Dockerfile, workflows, docs,
setup scripts, devcontainer, k8s manifests, vscode config, pytest and coverage
config, .gitignore. Behaviour is unchanged.

Kept for one release:
- A top-level application package whose meta-path finder resolves
  application.x.y to the already-imported docsgpt.x.y object, so old imports
  and entry points (celery -A application.app.celery,
  uvicorn application.asgi:asgi_app) keep working with a FutureWarning.
- Celery registers every application.* task name as an alias of its
  docsgpt.* task on start-up, so messages queued by the previous release still
  run. The redbeat key prefix moves to redbeat:docsgpt:v2: so schedule entries
  the previous release wrote are left unread instead of firing twice.

The backend image builds from the repository root (docker build -f
docsgpt/Dockerfile .) so it can ship the alias package; a root .dockerignore
allow-lists docsgpt/ and application/ and keeps caches, local data, .env
files, the sample index files and the Dockerfile out. Compose and the image
workflows point at the new context.
2026-09-07 10:20:43 +01:00
Alex 6860a21541 fix: address review on the slim-image branch
- The frontend image ran the Vite dev server in development mode, so
  .env.development supplied its defaults (notification banner, Google client
  id, local API host). The static build only loads .env.production, so the
  build stage now copies .env.development in as the baseline and the compose
  files pass every VITE_* the app reads through from .env; the runtime script
  skips empty values so a blank passthrough keeps the build-time default.
  .dockerignore kept only the .local variants out.
- VITE_DISABLE_SOURCE_FE disables sources only when it is the string true.
- DoclingParser: find_spec raises when docling itself is absent; the install
  hint now covers that path, with a regression test.
- verify_offline: direct tests for verify(); the PR image check builds and
  verifies the -docling variant as well as slim.
- Workflows this branch adds or rewrites pin actions by commit, pass the
  release tag through env instead of template expansion, and do not persist
  checkout credentials.
- OCR guide no longer claims pre-built images never include docling.
2026-09-06 21:44:34 +01:00
Alex aecb596e99 build(docker): slim backend image, static frontend image, -docling variant
Backend (arc53/docsgpt): 4.5 GB compressed -> 0.9 GB with both embedding
models and tiktoken baked in.
- torch/transformers gone from the default install (docling extra only).
- Ubuntu 24.04 ships python3.12: no deadsnakes PPA, no software-properties-
  common; every pin is a wheel, so no gcc/g++/rust in the builder.
- COPY --chown and a prefetch that runs as the process user replace the
  trailing chown -R, which duplicated the 600 MB model layer.
- .dockerignore keeps __pycache__, .coverage, local indexes and .env out.
- EXTRAS build arg (INSTALL_DOCLING kept as an alias); the docling variant
  also bakes docling's layout/table/RapidOCR models (DOCLING_ARTIFACTS_PATH)
  and tesseract, and drops only the discovery documents of Google APIs the
  app never builds.
- FLASK_DEBUG env removed (unused); OCI labels added.

Frontend (arc53/docsgpt-fe): 302 MB Vite dev server -> 25 MB static build
behind nginx. VITE_* variables are injected at container start into
/config.js and read through src/env.ts, so the image no longer needs a
rebuild per deployment; docker-compose.yaml keeps hot reload via the dev
target.

Publishing: every release and develop build now pushes a slim tag and a
-docling tag (docling engine + models + tesseract). docker-compose-hub.yaml
takes DOCSGPT_IMAGE_TAG / DOCSGPT_IMAGE_VARIANT; docker-compose-standalone.yaml
runs the stack from pre-built images without a checkout and is attached to
each release. setup.sh selects the -docling variant for OCR instead of
requiring a local build. A new workflow builds the image on PRs that touch
it and runs verify_offline under --network none; lint checks the exported
requirements match uv.lock.
2026-09-05 15:50:21 +01:00
Alex 2a9a02427f fix(setup,docs): write INSTALL_TESSERACT from setup.ps1, correct the OCR upgrade note
setup.ps1 wrote OCR_ENABLED=true but never INSTALL_TESSERACT=true, so a
Windows user answering yes to the OCR question ended up with OCR on and no
engine in the image; it also claimed tesseract was "shipped in the Docker
image", which this change makes false, and offered OCR for the pre-built
Docker Hub images that cannot include it. Mirror setup.sh: skip the question
for hub images (naming all three settings the DeepSeek path needs), and bake
tesseract in for locally built ones.

The upgrade callout said earlier images always included tesseract. They never
did -- they included docling, and OCR ran on the RapidOCR engine bundled with
it, needing no system package. It also covered only OCR_ENABLED, missing
OCR_ATTACHMENTS_ENABLED, which is a separate switch onto the same native OCR
path.
2026-09-04 15:13:23 +01:00
Pavel 6276158d0d Small fixes 3 2026-09-04 17:45:38 +04:00
Pavel 4e14a79923 Fixes batch 2 2026-09-04 13:38:29 +04:00
Pavel ed0892b39b Batch fixes 2 2026-09-03 00:30:59 +04:00
Pavel 47eb92fc42 Fixes batch 1 2026-09-02 23:42:35 +04:00
Pavel 96b878217d standalone OCR 2026-09-02 23:12:36 +04:00
Pavel d4e92be0ab ocr update 2026-08-27 23:46:12 +04:00
Pavel d7a7d4d084 docling separation 2026-08-27 17:36:38 +04:00
Pavel e101c0a7f1 Implement anydoc with docling 2026-08-27 13:51:09 +04:00
Alex ca53a3ac4b chore: bump docling 2026-08-12 09:05:00 +01:00
81b6ee5daa Pg 4 (#2390)
* feat: postgres tests

* feat: mongo cutoff

* feat: mongo cutoff

* feat: adjust docs and compose files

* fix: mini code mongo removals

* fix: tests and k8s mongo stuff

* feat: test fixes

* fix: ruff

* fix: vale

* Potential fix for pull request finding 'CodeQL / Clear-text logging of sensitive information'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix: mini suggestions

* vale lint fix 2

* fix: codeql columns thing

* fix: test mongo

* fix: tests coverage

* feat: better tests 4

* feat: more tests

* feat: decent coverage

* fix: ruff fixes

* fix: remove mongo mock

* feat: enhance workflow engine and API routes; add document retrieval and source handling

* feat: e2e tests

* fix: mcp, mongo and more

* fix: mini codeql warning

* fix: agent chunk view

* fix: mini issues

* fix: more pg fixes

* feat: postgres prep on start

* feat: qa tests

* fix: mini improvements

* fix: tests

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Rai <siddhant.rai.5686@gmail.com>
2026-04-18 13:13:57 +01:00
Alex 17a736a927 docs: migrate to Nextra 4 and Next.js App Router 2026-02-18 17:13:24 +00:00