Unreal Engine: reorder the setup guide, fail on an empty compile_commands.json (#1610)

CHANGELOG.md

@@ -29,14 +29,7 @@ Status of the `main` branch. Changes prior to the next official version change w
     (which does not correctly unpack structured output) #1042
 
 * Language Servers:
-  - C/C++ (clangd): add Unreal Engine 5 fixture and tests verifying that reflection-macro
-    code (`UCLASS`, `UFUNCTION`, `UPROPERTY`, `GENERATED_BODY`) yields correct symbols,
-    references, definitions, and rename edits in hand-written sources (never in
-    UHT-generated files); add an Unreal Engine setup guide.
-  - C/C++ (clangd, ccls): detect Unreal Engine projects by a `.uproject` file at the project
-    root and skip the engine build/cache directories (`Binaries`, `DerivedDataCache`,
-    `Intermediate`, `Saved`) during indexing. clangd fails with a pointer to the Unreal Engine
-    setup guide when started in such a project without a `compile_commands.json`. #1566
+  - C/C++ (clangd): improve support and documentation for Unreal Engine 5 projects.
   - `typescript_vts`: Add `initialization_options` setting in `ls_specific_settings.typescript_vts`.
     The dict is forwarded to vtsls via `initializationOptions`, `workspace/didChangeConfiguration`,
     and `workspace/configuration` pulls. Enables Yarn PnP setups with `typescript.tsdk` pointing

docs/03-special-guides/unreal_engine_setup_guide_for_serena.md

@@ -21,30 +21,80 @@ file by default; this guide shows how to obtain it.
 ---
 ## Getting a compilation database
 
-If clangd starts in a project that has a `.uproject` file but no `compile_commands.json`,
-Serena fails the language server startup with an error that points back to this guide,
-because clangd cannot resolve engine headers or reflection macros without the database.
-Pick one of the following routes to create it.
+If clangd starts in a project that has a `.uproject` file but no usable
+`compile_commands.json`, Serena fails the language server startup with an error that
+inlines the command below, because clangd cannot resolve engine headers or reflection
+macros without the database. Pick one of the following routes to create it.
 
-### Route 1: VSCode project files (no extra installs)
+None of these routes change how you build. You still compile with MSVC. A clang toolchain,
+where a route uses one, only generates the database that clangd reads.
 
-UnrealBuildTool's VSCode project generator emits per-project compile commands.
-Run it from your engine installation (VSCode itself is not required):
-
-    <Engine>\Build\BatchFiles\Build.bat -projectfiles -project="<YourProject>.uproject" -game -VSCode
-
-This produces `.vscode/compileCommands_<YourProject>.json` inside your project.
-Copy or symlink it to the project root as `compile_commands.json`.
-
-If you already use VSCode with UE, the file likely exists; the editor's
-"Tools > Refresh Visual Studio Code Project" action maintains it.
-
-### Route 2: UnrealBuildTool's clang database mode (requires LLVM installed)
+### Route 1 (recommended): UnrealBuildTool's clang database
 
     <Engine>\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.exe -mode=GenerateClangDatabase -project="<YourProject>.uproject" <YourProject>Editor Win64 Development -OutputDir="<YourProject's directory>"
 
-This emits clang-native commands (cleanest flags for clangd) but requires a Clang
-toolchain installed on Windows.
+This emits clang-native commands. Each entry carries the system-include paths, so clangd
+resolves system and engine headers without guessing.
+
+It needs a clang toolchain. Install one from the Visual Studio Installer:
+**Modify > Individual Components > "C++ Clang tools for Windows"**. This is not a separate
+LLVM download. UnrealBuildTool auto-detects it at `VC\Tools\Llvm\x64`.
+
+- UnrealBuildTool searches for clang in this order: `C:\Program Files\LLVM`, then the
+  `LLVM_PATH` environment variable, then the VS-bundled `VC\Tools\Llvm\x64`, then AutoSDK.
+  A standalone LLVM install wins if present. Remove it to fall back to the version-matched VS one.
+- UE 5.7 expects clang in roughly the 18.1.8 to 20.1.8 range, and the VS component is
+  matched. A newer standalone (e.g. 22.x) still generates the database but prints
+  `Clang compiler version ... is not a preferred version`.
+- `-OutputDir` is required. Without it the file lands in the engine root.
+- Build the editor target once before generating, so the `*.generated.h` headers exist.
+  After a build you can add `-NoExecCodeGenActions` to skip redundant codegen.
+
+### Route 2 (no clang toolchain): MSVC database
+
+If you cannot add the clang component, append `-Compiler=VisualStudio2022`:
+
+    <Engine>\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.exe -mode=GenerateClangDatabase -project="<YourProject>.uproject" <YourProject>Editor Win64 Development -Compiler=VisualStudio2022 -OutputDir="<YourProject's directory>"
+
+This produces an MSVC `cl.exe` database and needs no extra toolchain. The entries omit
+system-include paths, which MSVC reads from the `INCLUDE` environment variable rather than
+the database, so clangd may not resolve standard headers like `<vector>` and logs
+`Failed to compile ... index may be incomplete` per file. Symbol tools still work, since
+clangd indexes through those errors. To stop them from truncating symbol trees, add a
+`.clangd` at the project root (see Troubleshooting):
+
+    CompileFlags:
+      Add: [-D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH, -ferror-limit=200]
+
+### Route 3 (situational): VSCode project files
+
+UnrealBuildTool's VSCode generator emits per-project compile commands:
+
+    <Engine>\Build\BatchFiles\Build.bat -projectfiles -project="<YourProject>.uproject" -game -VSCode
+
+On an installed engine (Launcher/Fab) with a project outside the engine tree, this writes
+an empty array. UnrealBuildTool treats the project as foreign and emits no commands. Verify
+the output is a non-empty JSON array before relying on it:
+
+    <YourProject>\.vscode\compileCommands_Default.json
+
+If it contains entries, copy or symlink it to the project root as `compile_commands.json`.
+If it is `[]`, use Route 1 or Route 2. The generator works reliably only for source-built
+or in-tree engines, or when refreshed from inside the editor via
+**Tools > Refresh Visual Studio Code Project**.
+
+### Notes for Rider users
+
+Building the project in Rider never produces `compile_commands.json`. `-VSCode` is a
+project-file generator, not an editor mode, and the `.vscode/` directory it creates is
+safe to delete afterwards. Use Route 1.
+
+### When to regenerate
+
+Regenerate only when you add a module or change a `*.Build.cs` (new files or compiler
+flags). clangd watches `compile_commands.json` and reloads automatically, so routine edits
+need no regeneration. If Serena failed to start because the database was missing or empty,
+reconnect or restart the MCP after creating it. Serena reads the database only at startup.
 
 ---
 ## Build artifact directories
@@ -73,6 +123,10 @@ To exclude further paths, add them to `ignored_paths` in your project's
 - **Symbol searches on large projects:** prefer passing `relative_path` to
   `find_symbol`. An unscoped search visits every translation unit, and UE
   files are expensive to parse because each pulls in large engine headers.
+- **clangd index logs (MSVC database):** an `Indexed ... (N symbols)` line is a success.
+  A following `Failed to compile ... index may be incomplete` means clangd hit errors
+  while parsing that file but indexed it anyway. Raise `-ferror-limit` via `.clangd` if
+  symbol trees look truncated (see Troubleshooting).
 
 ---
 ## Troubleshooting

src/solidlsp/language_servers/clangd_language_server.py

@@ -111,6 +111,27 @@ class ClangdLanguageServer(SolidLanguageServer):
             or (is_unreal_engine_project(self.repository_root_path) and dirname in UE_IGNORED_DIRNAMES)
         )
 
+    def _raise_if_unreal_without_compile_db(self) -> None:
+        """Raise if this is an Unreal Engine project without a usable compilation database.
+
+        clangd cannot resolve engine headers or macros without one. Non-UE projects return
+        without raising, since clangd works without a database there.
+        """
+        if not is_unreal_engine_project(self.repository_root_path):
+            return
+        raise SolidLSPException(
+            f"No usable compile_commands.json at {self.repository_root_path}, but this looks like an "
+            "Unreal Engine project (.uproject present). clangd needs a non-empty compilation database "
+            "to resolve engine headers and macros.\n\n"
+            "Generate one from the engine's <Engine>\\Binaries\\DotNET\\UnrealBuildTool\\ directory:\n"
+            '  UnrealBuildTool.exe -mode=GenerateClangDatabase -project="<Proj>.uproject" '
+            '<Proj>Editor Win64 Development -OutputDir="<projectDir>"\n'
+            "Without a clang toolchain, append -Compiler=VisualStudio2022 to build an MSVC database instead.\n\n"
+            "Build the editor target once first so the generated headers exist. After creating the "
+            "database, reconnect or restart Serena's MCP so the language server re-checks for it.\n"
+            "See docs/03-special-guides/unreal_engine_setup_guide_for_serena.md for details."
+        )
+
     def _prepare_compile_commands(self) -> str | None:
         """
         Prepare clangd compilation database with absolute directory paths.
@@ -132,13 +153,7 @@ class ClangdLanguageServer(SolidLanguageServer):
 
         if not os.path.exists(compile_db_path):
             # clangd can't resolve UE engine headers without the database; fail with guidance.
-            if is_unreal_engine_project(self.repository_root_path):
-                raise SolidLSPException(
-                    f"No compile_commands.json found at {self.repository_root_path}, but this looks like an "
-                    "Unreal Engine project (.uproject present). clangd needs a compilation database to resolve "
-                    "engine headers and macros. See docs/03-special-guides/"
-                    "unreal_engine_setup_guide_for_serena.md to generate one."
-                )
+            self._raise_if_unreal_without_compile_db()
             return None
 
         try:
@@ -146,6 +161,8 @@ class ClangdLanguageServer(SolidLanguageServer):
                 compile_commands = json.load(f)
 
             if not compile_commands:
+                # An empty [] database is treated like a missing one.
+                self._raise_if_unreal_without_compile_db()
                 return None
 
             # Check if any entries have relative directory paths

test/solidlsp/cpp/test_cpp_unreal_unit.py

@@ -43,6 +43,21 @@ def test_missing_compile_commands_returns_none_for_non_unreal_project(tmp_path):
     assert server._prepare_compile_commands() is None
 
 
+def test_empty_compile_commands_raises_for_unreal_project(tmp_path):
+    (tmp_path / "MyGame.uproject").write_text("{}")
+    (tmp_path / "compile_commands.json").write_text("[]")
+    server = _make_clangd(tmp_path)
+
+    with pytest.raises(SolidLSPException, match="Unreal Engine"):
+        server._prepare_compile_commands()
+
+
+def test_empty_compile_commands_returns_none_for_non_unreal_project(tmp_path):
+    (tmp_path / "compile_commands.json").write_text("[]")
+    server = _make_clangd(tmp_path)
+    assert server._prepare_compile_commands() is None
+
+
 @pytest.mark.parametrize("make_server", [_make_clangd, _make_ccls])
 def test_unreal_dirs_ignored_only_for_unreal_project(make_server, tmp_path):
     non_ue = make_server(tmp_path)