docs(settings): generate the settings reference from the definitions

The hand-maintained settings page documented 95 of 258 settings and
.env-template 42, and both drifted as fields were added. The field
descriptions now live on the model, so the reference is rendered from it:

  python -m docsgpt.core.settings.reference --write

writes docs/content/Deploying/Settings-Reference.mdx, one section per
settings group with each field's type, default, constraints, aliases and
description. --check reports a stale page, and tests/core/test_settings.py
fails when the checked-in page no longer matches the definitions, so a
new setting cannot land undocumented.

test_settings.py also pins the composition contract: every group field is
a flat Settings attribute, no field is defined twice, every field has a
description, and the secret-normalising validator of every group is
applied (the case that a shared method name would silently drop).

The App Configuration page points at the reference instead of at
settings.py, and the reference is listed in the Deploying navigation.

docs/content/Deploying/DocsGPT-Settings.mdx

@@ -27,13 +27,13 @@ API_KEY=YOUR_OPENAI_API_KEY
 LLM_NAME=gpt-4o
 ```
 
-### 2. Configuration via `settings.py` file (Advanced)
+### 2. Configuration in code (Advanced)
 
-For more advanced configurations or if you prefer to manage settings directly in code, you can modify the `settings.py` file. This file is located in the `docsgpt/core` directory of your DocsGPT project.
+Every setting is defined in the `docsgpt/core/settings/` package, one module per domain (`auth.py`, `llm.py`, `embeddings.py`, ...). If you prefer to manage defaults directly in code, change them there; the `.env` file and the process environment still override whatever the code says.
 
-While modifying `settings.py` offers more flexibility, it's generally recommended to use the `.env` file for basic settings and reserve `settings.py` for more complex adjustments or when you need to configure settings programmatically.
+Using the `.env` file is recommended for day-to-day configuration. Reserve code changes for new settings or for defaults you want every deployment of your fork to share.
 
-**Location of `settings.py`:** `docsgpt/core/settings.py`
+The [Settings Reference](/Deploying/Settings-Reference) lists every setting with its type, default and description, generated from those definitions.
 
 ## Basic Settings Explained
 
@@ -289,7 +289,7 @@ DocsGPT includes a JWT (JSON Web Token) based authentication feature for managin
 
 ### `AUTH_TYPE` Overview
 
-The `AUTH_TYPE` setting in your `.env` file or `settings.py` determines the authentication method used by DocsGPT. This allows you to control how users authenticate with your DocsGPT instance.
+The `AUTH_TYPE` setting in your `.env` file determines the authentication method used by DocsGPT. This allows you to control how users authenticate with your DocsGPT instance.
 
 | Value         | Description                                                                                 |
 | ------------- | ------------------------------------------------------------------------------------------- |
@@ -300,7 +300,7 @@ The `AUTH_TYPE` setting in your `.env` file or `settings.py` determines the auth
 
 #### How to Configure
 
-Add the following to your `.env` file (or set in `settings.py`):
+Add the following to your `.env` file:
 
 ```env
 # Shared signing key (required in production for every authentication mode)
@@ -547,11 +547,10 @@ recovers.
 
 ## Exploring More Settings
 
-These are just the basic settings to get you started. The `settings.py` file contains many more advanced options that you can explore to further customize DocsGPT, such as:
+These are just the basic settings to get you started. DocsGPT has many more advanced options, such as:
 
 - Vector store configuration (`VECTOR_STORE`, Qdrant, Milvus, LanceDB settings) If you're looking for an easy way to set up a vector store with pgvector, try [Neon](https://get.neon.com/docsgpt).
-- Retriever settings (`RETRIEVERS_ENABLED`)
 - Cache settings (`CACHE_REDIS_URL`)
-- And many more!
+- Sandbox, scheduler, guardrails and event-stream tuning
 
-For a complete list of available settings and their descriptions, refer to the `settings.py` file in `docsgpt/core`. Remember to restart your Docker containers after making changes to your `.env` file or `settings.py` for the changes to take effect.
+The [Settings Reference](/Deploying/Settings-Reference) lists every setting with its type, default and description. Remember to restart your Docker containers after making changes to your `.env` file for the changes to take effect.

docs/content/Deploying/_meta.js

@@ -3,6 +3,10 @@ export default {
     "title": "⚙️ App Configuration",
     "href": "/Deploying/DocsGPT-Settings"
   },
+  "Settings-Reference": {
+    "title": "📖 Settings Reference",
+    "href": "/Deploying/Settings-Reference"
+  },
   "OIDC-SSO": {
     "title": "🔐 SSO with OIDC",
     "href": "/Deploying/OIDC-SSO"

docsgpt/core/settings/reference.py

@@ -0,0 +1,169 @@
+"""Render the settings reference page from the ``Settings`` definitions.
+
+The page under ``docs/content/Deploying/Settings-Reference.mdx`` is generated
+from the field types, defaults and descriptions in this package, so the model
+is the single source of truth. Regenerate it after changing a setting::
+
+    python -m docsgpt.core.settings.reference --write
+
+``--check`` exits non-zero when the checked-in page is stale; the test suite
+runs the same comparison.
+"""
+
+from __future__ import annotations
+
+import argparse
+import inspect
+import json
+import sys
+import types
+import typing
+from pathlib import Path
+from typing import Any, Literal, Optional, Union
+
+from pydantic import AliasChoices
+from pydantic.fields import FieldInfo
+
+from docsgpt.core.paths import home_dir
+from docsgpt.core.settings import SETTINGS_GROUPS, Settings
+
+REFERENCE_PATH = Path("docs") / "content" / "Deploying" / "Settings-Reference.mdx"
+
+HOME_PLACEHOLDER = "<DOCSGPT_HOME>"
+
+_HEADER = """\
+---
+title: Settings Reference
+description: Every DocsGPT setting, grouped by domain, with its type, default and purpose.
+---
+
+{/* GENERATED FILE. Do not edit by hand: run `python -m docsgpt.core.settings.reference --write`. */}
+
+# Settings Reference
+
+Every setting DocsGPT reads, generated from `docsgpt/core/settings/`. Each one is
+an environment variable of the same name, set in `.env` or the process
+environment; see [App Configuration](/Deploying/DocsGPT-Settings) for how the
+file is found and for worked examples. `<DOCSGPT_HOME>` below is the data home
+described there.
+"""
+
+
+def _mdx(text: str) -> str:
+    """Escape prose for MDX, where braces open expressions and ``<`` opens JSX."""
+    return text.replace("{", "\\{").replace("}", "\\}").replace("<", "&lt;")
+
+
+def _type_name(annotation: Any) -> str:
+    origin = typing.get_origin(annotation)
+    if origin in (Union, types.UnionType):
+        args = [a for a in typing.get_args(annotation) if a is not type(None)]
+        return " | ".join(_type_name(a) for a in args)
+    if origin is Literal:
+        return " | ".join(json.dumps(v) for v in typing.get_args(annotation))
+    if origin is not None:
+        return getattr(origin, "__name__", str(origin))
+    return getattr(annotation, "__name__", str(annotation))
+
+
+def _default_text(field: FieldInfo) -> str:
+    value = field.default_factory() if field.default_factory is not None else field.default
+    if value is None:
+        return "unset"
+    if isinstance(value, bool):
+        return "`true`" if value else "`false`"
+    if isinstance(value, str):
+        value = value.replace(str(home_dir()), HOME_PLACEHOLDER)
+        return "`\"\"`" if value == "" else f"`{value}`"
+    if isinstance(value, (list, dict)):
+        return f"`{json.dumps(value)}`"
+    return f"`{value}`"
+
+
+def _constraints(field: FieldInfo) -> list[str]:
+    out = []
+    for item in field.metadata:
+        for attr, symbol in (("gt", ">"), ("ge", ">="), ("lt", "<"), ("le", "<=")):
+            if hasattr(item, attr):
+                out.append(f"{symbol} {getattr(item, attr)}")
+    return out
+
+
+def _aliases(name: str, field: FieldInfo) -> list[str]:
+    alias = field.validation_alias
+    if isinstance(alias, AliasChoices):
+        return [str(c) for c in alias.choices if str(c) != name]
+    if isinstance(alias, str) and alias != name:
+        return [alias]
+    return []
+
+
+def _render_field(name: str, field: FieldInfo) -> str:
+    facts = [f"Type `{_type_name(field.annotation)}`", f"default {_default_text(field)}"]
+    constraints = _constraints(field)
+    if constraints:
+        facts.append("must be " + " and ".join(constraints))
+    aliases = _aliases(name, field)
+    if aliases:
+        facts.append("also read from " + ", ".join(f"`{a}`" for a in aliases))
+    # The facts are code spans, which MDX leaves alone; only prose needs escaping.
+    lines = [f"### `{name}`", "", ", ".join(facts) + "."]
+    if field.deprecated:
+        note = field.deprecated if isinstance(field.deprecated, str) else "This setting is deprecated."
+        lines += ["", f"**Deprecated.** {_mdx(str(note))}"]
+    if field.description:
+        lines += ["", _mdx(field.description)]
+    return "\n".join(lines)
+
+
+def _group_intro(group: type) -> str:
+    doc = inspect.getdoc(group) or ""
+    return doc.split("\n\n", 1)[0].replace("\n", " ").strip()
+
+
+def render_reference() -> str:
+    """The full reference page as MDX text."""
+    parts = [_HEADER]
+    for title, group in SETTINGS_GROUPS:
+        parts.append(f"\n## {title}\n")
+        intro = _group_intro(group)
+        if intro:
+            parts.append(_mdx(intro) + "\n")
+        for name in group.model_fields:
+            parts.append(_render_field(name, Settings.model_fields[name]) + "\n")
+    return "\n".join(parts).rstrip("\n") + "\n"
+
+
+def reference_path(root: Optional[Path] = None) -> Path:
+    """Where the generated page lives in a checkout; ``root`` defaults to the repository root."""
+    if root is None:
+        root = Path(__file__).resolve().parents[3]
+    return root / REFERENCE_PATH
+
+
+def main(argv: Optional[list[str]] = None) -> int:
+    parser = argparse.ArgumentParser(description=__doc__.split("\n\n", 1)[0])
+    action = parser.add_mutually_exclusive_group()
+    action.add_argument("--write", action="store_true", help="write the page into the docs tree")
+    action.add_argument("--check", action="store_true", help="exit 1 if the checked-in page is stale")
+    args = parser.parse_args(argv)
+
+    rendered = render_reference()
+    path = reference_path()
+    if args.write:
+        path.write_text(rendered, encoding="utf-8")
+        print(f"wrote {path}")
+        return 0
+    if args.check:
+        current = path.read_text(encoding="utf-8") if path.exists() else ""
+        if current != rendered:
+            print(f"{path} is stale; run: python -m docsgpt.core.settings.reference --write", file=sys.stderr)
+            return 1
+        print(f"{path} is up to date")
+        return 0
+    sys.stdout.write(rendered)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

tests/core/test_settings.py

@@ -0,0 +1,109 @@
+"""Contract tests for ``docsgpt.core.settings``.
+
+``Settings`` is composed from one ``SettingsGroup`` per domain; these tests pin
+the properties that composition must keep (flat names, no duplicate fields,
+validators from every group applied) and that the generated reference page
+tracks the definitions.
+"""
+
+from pathlib import Path
+
+import pytest
+
+from docsgpt.core.settings import SETTINGS_GROUPS, Settings, settings
+from docsgpt.core.settings.reference import reference_path, render_reference
+
+SECRET_FIELDS = (
+    "API_KEY",
+    "OPENAI_API_KEY",
+    "ANTHROPIC_API_KEY",
+    "GOOGLE_API_KEY",
+    "GROQ_API_KEY",
+    "HUGGINGFACE_API_KEY",
+    "NOVITA_API_KEY",
+    "EMBEDDINGS_KEY",
+    "FALLBACK_LLM_API_KEY",
+    "QDRANT_API_KEY",
+    "ELEVENLABS_API_KEY",
+    "INTERNAL_KEY",
+)
+
+
+@pytest.mark.unit
+class TestComposition:
+    def test_every_group_field_is_a_flat_settings_attribute(self):
+        for _, group in SETTINGS_GROUPS:
+            for name in group.model_fields:
+                assert name in Settings.model_fields, name
+                assert hasattr(settings, name), name
+
+    def test_no_field_is_defined_in_two_groups(self):
+        owners: dict[str, str] = {}
+        for title, group in SETTINGS_GROUPS:
+            for name in group.model_fields:
+                assert name not in owners, f"{name} is defined in both {owners[name]} and {title}"
+                owners[name] = title
+        assert len(owners) == len(Settings.model_fields)
+
+    def test_every_field_has_a_description(self):
+        missing = [name for name, field in Settings.model_fields.items() if not field.description]
+        assert not missing, f"settings without a description: {missing}"
+
+    def test_defaults_load_without_an_env_file(self, monkeypatch):
+        for name in Settings.model_fields:
+            monkeypatch.delenv(name, raising=False)
+        fresh = Settings(_env_file=None)
+        assert fresh.LLM_PROVIDER == "docsgpt"
+        assert fresh.VECTOR_STORE == "faiss"
+
+
+@pytest.mark.unit
+class TestValidators:
+    """Validators live on the group that owns the field; composition must keep all of them.
+
+    Two groups defining a validator under the same method name would silently
+    keep only one, so this checks every secret field, across every group.
+    """
+
+    @pytest.mark.parametrize("name", SECRET_FIELDS)
+    @pytest.mark.parametrize("raw", ["None", "none", "", "   "])
+    def test_unset_secret_spellings_become_none(self, name, raw):
+        assert getattr(Settings.model_validate({name: raw}), name) is None
+
+    @pytest.mark.parametrize("name", SECRET_FIELDS)
+    def test_secret_is_stripped(self, name):
+        assert getattr(Settings.model_validate({name: "  k3y  "}), name) == "k3y"
+
+    def test_normalize_api_key_classmethod_is_kept(self):
+        assert Settings.normalize_api_key("None") is None
+        assert Settings.normalize_api_key("  x ") == "x"
+        assert Settings.normalize_api_key(42) == 42
+
+    def test_postgres_uris_are_normalized(self):
+        loaded = Settings.model_validate(
+            {"POSTGRES_URI": "postgres://u:p@h/db", "PGVECTOR_CONNECTION_STRING": "postgresql+psycopg://u:p@h/v"}
+        )
+        assert loaded.POSTGRES_URI.startswith("postgresql+psycopg://")
+        assert loaded.PGVECTOR_CONNECTION_STRING.startswith("postgresql://")
+
+    def test_legacy_docling_ocr_aliases_are_read(self):
+        loaded = Settings.model_validate({"DOCLING_OCR_ENABLED": "true", "DOCLING_OCR_MIN_CHARS_PER_PAGE": "7"})
+        assert loaded.OCR_ENABLED is True
+        assert loaded.OCR_MIN_CHARS_PER_PAGE == 7
+
+
+@pytest.mark.unit
+class TestReference:
+    def test_reference_lists_every_setting_once(self):
+        page = render_reference()
+        for name in Settings.model_fields:
+            assert page.count(f"### `{name}`") == 1, name
+
+    def test_checked_in_reference_is_current(self):
+        path: Path = reference_path()
+        if not path.exists():
+            pytest.skip("docs tree not present (installed package, not a checkout)")
+        assert path.read_text(encoding="utf-8") == render_reference(), (
+            "docs/content/Deploying/Settings-Reference.mdx is stale; "
+            "run: python -m docsgpt.core.settings.reference --write"
+        )