Let a fork's own catalogs live in docsgpt/core/models without failing CI

test_each_provider_has_expected_ids asserts the exact set of model ids per
provider against a snapshot. For the first-party catalogs that is what we
want: the snapshot exists so an upstream id typo fails before it silently
breaks every agent that references the old id.

openai_compatible is different. It is the zero-Python extension point —
the README's "Add an OpenAI-compatible provider" section and
examples/mistral.yaml.example both tell you to drop a YAML next to the
built-ins, and the loader globs the whole directory. So anyone who follows
those instructions gets a permanently red test on their fork, for having
done the documented thing. The _internal suffix carve-out only covers the
one filename we happen to gitignore.

Treat the snapshot as a floor for that provider: the built-in ids must all
still be present, extra ones are somebody else's provider. The rename this
guards against still drops an id, so it still fails, and the assertion
names the missing ids instead of printing two large sets to diff by eye.

Verified by copying examples/mistral.yaml.example into the catalog
directory: red before, green after, and still red when a built-in id is
renamed.
This commit is contained in:
Alex committed 2026-09-22 10:03:12 +01:00
1 parent c2e04fa7bf
commit 2f1b728e23
1 file changed
+19 -1
+19 -1
View File
@@ -67,6 +67,16 @@ EXPECTED_IDS = {
"huggingface": {"huggingface-local"},
}
# Providers whose catalog is an open extension point. ``openai_compatible``
# is the zero-Python way to add an OpenAI-shaped endpoint — copy
# ``examples/mistral.yaml.example`` next to the built-ins and it loads — so
# a fork or a deployment can legitimately have catalogs here that upstream
# does not. For these the snapshot is a floor, not an exact set: the
# built-in ids must all still be present, but extra ones are somebody's
# own provider rather than a regression. A rename, which is what this
# snapshot exists to catch, still drops an id and still fails.
EXTENSIBLE_PROVIDERS = {"openai_compatible"}
def _make_settings(**overrides):
s = MagicMock()
@@ -135,7 +145,15 @@ class TestYAMLLoader:
)
]
actual = {m.id for c in canonical for m in c.models}
assert actual == expected, f"{provider}: expected {expected}, got {actual}"
if provider in EXTENSIBLE_PROVIDERS:
missing = expected - actual
assert not missing, (
f"{provider}: built-in ids missing from the catalog: {missing}"
)
else:
assert actual == expected, (
f"{provider}: expected {expected}, got {actual}"
)
def test_attachment_alias_image_expands_to_five_mime_types(self):
grouped = _by_provider(load_model_yamls([BUILTIN_MODELS_DIR]))