Merge pull request #1781 from oraios/test-uv-python-compat

Test Python version compatibility in CI

.github/workflows/python-versions.yml

@@ -0,0 +1,94 @@
+# =============================================================================
+# Python version compatibility -- strategy & rationale
+# =============================================================================
+# WHY THIS EXISTS
+#   `pyproject.toml` declares support for the whole range `>=3.11, <3.15`, but the
+#   test suite (see pytest.yml) runs on a single interpreter. This workflow is the
+#   cheap backstop for the rest of the range: it does not run the test suite, it only
+#   checks that the locked dependency set can be *installed* on each supported
+#   interpreter and that Serena is then functional there (via `project health-check`).
+#   That catches the failure mode a single-version test matrix structurally cannot
+#   see -- a dependency with no wheel (or no support) for a given Python version,
+#   e.g. right after a new CPython release.
+#
+# WHY A SEPARATE WORKFLOW
+#   The check depends only on `pyproject.toml` + `uv.lock` + the import graph, not on
+#   any language toolchain, so it shares nothing with the test matrix's per-batch
+#   setup. Keeping it separate also gives it its own concurrency group and keeps the
+#   job count out of pytest.yml's (os x batch) cross-product.
+#
+# CONCURRENCY LIMITS (GitHub free tier)
+#   Hard cap of 20 concurrent jobs across the account. pytest.yml already uses 13,
+#   so this workflow is deliberately kept at 4 (Linux only) to stay inside the cap
+#   when both run on the same push; adding OSes here must respect that ceiling.
+#
+# DELIBERATELY NOT CACHED
+#   The `.venv` is intentionally NOT cached (unlike in pytest.yml): a restored venv
+#   would make `uv sync` a no-op and thereby skip the very thing under test. Only
+#   uv's own wheel cache (managed by setup-uv) is reused, which speeds up downloads
+#   without weakening the check.
+# =============================================================================
+name: Python Versions
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  uv-sync:
+    name: uv sync (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    # Backstop against hung jobs; a healthy run is ~2 minutes.
+    timeout-minutes: 20
+    strategy:
+      # Report every unsupported version in one run instead of stopping at the first.
+      fail-fast: false
+      matrix:
+        # The full range declared by `requires-python` in pyproject.toml. Keep in sync
+        # with it (and with the `Programming Language :: Python :: 3.x` classifiers).
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Node.js
+        # Required by the smoke test below, not by the sync: the default Python language
+        # server (Pyright) is auto-installed by Serena and runs on Node.
+        uses: actions/setup-node@v4
+        with:
+          node-version: '20.x'
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+      - name: Create virtual environment with Python ${{ matrix.python-version }}
+        # No actions/setup-python: uv fetches the interpreter itself, so the version under
+        # test is stated exactly once here and cannot drift from what uv would pick up
+        # from PATH. `-p` is what makes this job meaningful -- a bare `uv venv` would test
+        # whatever interpreter happens to be on the runner.
+        shell: bash
+        run: uv venv -p ${{ matrix.python-version }}
+      - name: Install Python environment
+        # --locked additionally asserts that uv.lock is up to date with pyproject.toml,
+        # i.e. that the resolution being installed is the one contributors and CI use.
+        shell: bash
+        run: uv sync --extra dev --locked
+      - name: List Python dependencies
+        shell: bash
+        run: uv pip list
+      - name: Smoke-test against the Python test repo
+        # Installability alone does not prove usability. The health check starts a real
+        # SerenaAgent on the Python test repo and exercises the symbolic tools end to end
+        # (symbols overview -> find symbol -> find references -> pattern search), so it
+        # covers the import graph, the agent's startup path and the language server
+        # integration -- the parts most likely to break on a new interpreter. Note that
+        # Pyright itself is launched via `uvx -p 3.13`, i.e. on its own interpreter: what
+        # varies across this matrix is Serena, not the language server. Exits non-zero on
+        # failure.
+        shell: bash
+        run: uv run serena project health-check test/resources/repos/python/test_repo

CHANGELOG.md

@@ -17,6 +17,9 @@ Status of the `main` branch. Changes prior to the next official version change w
 * CLI:
   - Fix: `start-mcp-server` help text for `--project-from-cwd` falsely promised a fallback to the CWD, which was 
     removed in v1.0.0 #1773
+  - Fix: `project health-check` always exited with code 0, even when the check failed, so callers 
+    (CI, scripts) could not act on its verdict; it now exits with code 1 on failure. A `find_symbol` 
+    result without any matches is now reported as a failure rather than as a warning.
 
 * Tools:
   - `find_symbol`, `jet_brains_find_symbol`: Change tool description to improve tool search results in clients that load tools dynamically

src/serena/cli.py

@@ -909,6 +909,12 @@ class ProjectCommands(AutoRegisteringGroup):
         finally:
             ls_mgr.stop_all()
 
+    class _HealthCheckFailure(Exception):
+        """
+        A condition under which the health check is to be considered failed.
+        The exception message is the explanation presented to the user.
+        """
+
     @staticmethod
     @click.command(
         "health-check",
@@ -939,6 +945,9 @@ class ProjectCommands(AutoRegisteringGroup):
         os.makedirs(log_dir, exist_ok=True)
         log_file = os.path.join(log_dir, f"health_check_{timestamp}.log")
 
+        # the check's verdict: the explanation of the failure, or None if the check passed (see below)
+        failure_reason: str | None = None
+
         with FileLoggerContext(log_file, append=False, enabled=True):
             log.info("Starting health check for project: %s", project_path)
 
@@ -957,7 +966,7 @@ class ProjectCommands(AutoRegisteringGroup):
                 for file_path in files:
                     try:
                         full_path = os.path.join(project_path, file_path)
-                        if os.path.getsize(full_path) > 0:
+                        if os.path.getsize(full_path) > 1000:
                             target_file = file_path
                             log.info("Found analyzable file: %s", target_file)
                             break
@@ -965,10 +974,7 @@ class ProjectCommands(AutoRegisteringGroup):
                         continue
 
                 if not target_file:
-                    log.error("No analyzable files found in project")
-                    click.echo("❌ Health check failed: No analyzable files found")
-                    click.echo(f"Log saved to: {log_file}")
-                    return
+                    raise ProjectCommands._HealthCheckFailure("No analyzable files found")
 
                 # Get tools from agent
                 overview_tool = agent.get_tool(GetSymbolsOverviewTool)
@@ -982,10 +988,7 @@ class ProjectCommands(AutoRegisteringGroup):
                 log.info(f"GetSymbolsOverviewTool returned: {overview_data}")
 
                 if not overview_data:
-                    log.error("No symbols found in file %s", target_file)
-                    click.echo("❌ Health check failed: No symbols found in target file")
-                    click.echo(f"Log saved to: {log_file}")
-                    return
+                    raise ProjectCommands._HealthCheckFailure(f"No symbols found in target file {target_file}")
 
                 # Extract suitable symbol (prefer class or function over variables)
                 preferred_kinds = {SymbolKind.Class.name, SymbolKind.Function.name, SymbolKind.Method.name, SymbolKind.Constructor.name}
@@ -1036,28 +1039,32 @@ class ProjectCommands(AutoRegisteringGroup):
                     pattern_matches = 0
 
                 # Verify tools worked as expected
-                tools_working = True
                 if not find_symbol_data:
-                    log.error("FindSymbolTool returned no results")
-                    tools_working = False
+                    raise ProjectCommands._HealthCheckFailure("FindSymbolTool returned no results")
 
                 if len(find_refs_data) == 0 and pattern_matches == 0:
                     log.warning("Both FindReferencingSymbolsTool and SearchForPatternTool found no matches - this might indicate an issue")
 
                 log.info("Health check completed successfully")
 
-                if tools_working:
-                    click.echo("✅ Health check passed - All tools working correctly")
-                else:
-                    click.echo("⚠️  Health check completed with warnings - Check log for details")
-
+            # expected failures and unexpected exceptions are reported alike: both mean the project's
+            # tooling is not functional, which is the single thing this command is asked to determine
             except Exception as e:
                 log.exception("Health check failed with exception: %s", str(e))
-                click.echo(f"❌ Health check failed: {e!s}")
+                failure_reason = str(e)
 
             finally:
                 click.echo(f"Log saved to: {log_file}")
 
+        # the verdict is reported outside the checked region, so that a failure to write the report
+        # cannot be mistaken for a failure of the check itself; the exit code lets callers
+        # (CI, scripts) act on the verdict
+        if failure_reason is None:
+            click.echo("✅ Health check passed - All tools working correctly")
+        else:
+            click.echo(f"❌ Health check failed: {failure_reason}")
+            raise SystemExit(1)
+
 
 class ToolCommands(AutoRegisteringGroup):
     """Group for 'tool' subcommands."""