tiennm99/serena
1
0 Fork 0
mirror of https://github.com/tiennm99/serena.git synced 2026-10-14 16:18:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into hover_budget

Browse Source 
# Conflicts:
#	src/serena/config/serena_config.py
#	src/serena/symbol.py
#	src/serena/tools/symbol_tools.py
#	src/solidlsp/ls.py
#	test/serena/test_symbol.py
This commit is contained in:
Michael Panchenko committed 2026-02-19 17:44:15 +01:00
2 parents bb41cef097 3791c1b050
commit 4086ce5e6e
110 files changed
+4445 -1188
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.dockerignore
+6
View File
@@ -7,3 +7,9 @@ docs/.jupyter_cache
docs/_build
coverage.xml
docker_build_and_run.sh
# Python artifacts (prevents Docker build conflicts when .venv exists locally)
.venv
__pycache__
*.pyc
.pytest_cache
.github/workflows/pytest.yml
+25 -4
View File
@@ -1,4 +1,4 @@
name: Tests on CI
name: Tests
on:
pull_request:
@@ -58,16 +58,19 @@ jobs:
uses: actions/cache@v3
with:
path: .venv
key: uv-venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('uv.lock') }}
key: uv-venv-${{ runner.os }}-${{ matrix.python-version }}-lock-${{ hashFiles('uv.lock') }}
- name: Create virtual environment
shell: bash
run: |
if [ ! -d ".venv" ]; then
uv venv
fi
- name: Install dependencies
- name: Install Python environment
shell: bash
run: uv pip install -e ".[dev]"
run: uv sync --extra dev --locked
- name: List Python dependencies
shell: bash
run: uv pip list
- name: Check formatting
shell: bash
run: uv run poe lint
@@ -110,6 +113,24 @@ jobs:
uses: DeLaGuardo/setup-clojure@13.4
with:
cli: latest
- name: Install ccls (C/C++ Language Server)
shell: bash
run: |
if [[ "${{ runner.os }}" == "Linux" ]]; then
sudo apt-get update
sudo apt-get install -y ccls
elif [[ "${{ runner.os }}" == "macOS" ]]; then
brew install ccls
elif [[ "${{ runner.os }}" == "Windows" ]]; then
choco install ccls -y
fi
# Verify installation
if command -v ccls &> /dev/null; then
echo "ccls installed: $(ccls --version 2>&1 | head -1)"
else
echo "ERROR: ccls installation failed"
exit 1
fi
- name: Setup Java (for JVM based languages)
uses: actions/setup-java@v4
with:
CHANGELOG.md
+2
View File
@@ -38,6 +38,7 @@ Status of the `main` branch. Changes prior to the next official version change w
* Language support:
* **Add Phpactor as alternative PHP language server** (specify `php_phpactor` as language; requires PHP 8.1+)
* **Add support for Fortran** via fortls language server (requires `pip install fortls`)
* **Add partial support for Groovy** requires user-provided Groovy language server JAR (see [setup guide](docs/03-special-guides/groovy_setup_guide_for_serena.md))
* **Add support for Julia** via LanguageServer.jl
@@ -61,6 +62,7 @@ Status of the `main` branch. Changes prior to the next official version change w
* **Add support for PowerShell** via PowerShell Editor Services (PSES). Requires `pwsh` (PowerShell Core) to be installed and available in PATH. Supports symbol navigation, go-to-definition, and within-file references for .ps1 files.
* **Add support for MATLAB** via the official MathWorks MATLAB Language Server. Requires MATLAB R2021b or later and Node.js. Set `MATLAB_PATH` environment variable or configure `matlab_path` in `ls_specific_settings`. Supports .m, .mlx, and .mlapp files with code completion, diagnostics, go-to-definition, find references, document symbols, formatting, and rename.
* **Add support for Pascal** via the official Pascal Language Server.
* **C/C++ alternate LS (ccls)**: Add experimental, opt-in support for ccls as an alternative backend to clangd. Enable via `cpp_ccls` in project configuration. Requires `ccls` installed and ideally a `compile_commands.json` at repo root.
# 0.1.4
CONTRIBUTING.md
+13 -17
View File
@@ -11,18 +11,11 @@ The following types of contributions can be submitted directly via pull requests
For other changes, please open an issue first to discuss your ideas with the maintainers.
## Developer Environment Setup
### Adding Support for a New Language Server
You can have a local setup via `uv` or a docker interpreter-based setup.
The repository is also configured to seamlessly work within a GitHub Codespace. See the instructions
for the various setup scenarios below.
See the corresponding [memory](.serena/memories/adding_new_language_support_guide.md).
Independently of how the setup was done, the virtual environment can be
created and activated via `uv` (see below), and the various tasks like formatting, testing, and documentation building
can be executed using `poe`. For example, `poe format` will format the code, including the
notebooks. Just run `poe` to see the available commands.
### Python (uv) setup
## Python Environment Setup
You can install a virtual environment with the required as follows
@@ -30,15 +23,18 @@ You can install a virtual environment with the required as follows
2. Activate the environment:
* On Linux/Unix/macOS or Windows with Git Bash: `source .venv/bin/activate`
* On Windows outside of Git Bash: `.venv\Scripts\activate.bat` (in cmd/ps) or `source .venv/Scripts/activate` (in git-bash)
3. Install the required packages with all extras: `uv pip install --all-extras -r pyproject.toml -e .`
3. Install the required packages with all extras: `uv sync --extra dev`
## Running Tools Locally
## Poe Tasks
We use poe to execute development tasks:
- `poe format` - run code auto-formatters
- `poe type-check` - run type checkers
## Testing Tool Executions
The Serena tools (and in fact all Serena code) can be executed without an LLM, and also without
any MCP specifics (though you can use the mcp inspector, if you want).
An example script for running tools is provided in [scripts/demo_run_tools.py](scripts/demo_run_tools.py).
## Adding a New Supported Language
See the corresponding [memory](.serena/memories/adding_new_language_support_guide.md).
An example script for running tools is provided in [scripts/demo_run_tools.py](scripts/demo_run_tools.py).
docs/01-about/020_programming-languages.md
+14 -9
View File
@@ -29,9 +29,13 @@ Some languages require additional installations or setup steps, as noted.
* **AL**
* **Bash**
* **C#**
* **C#**
(by default, uses the Roslyn language server (language `csharp`), requires `pwsh` ([PowerShell 7+](https://learn.microsoft.com/en-us/powershell/scripting/install/install-powershell-on-windows?view=powershell-7.5)) on Windows;
set language to `csharp_omnisharp` to use OmiSharp instead)
* **C/C++**
(you may experience issues with finding references, we are working on it)
(by default, uses the clangd language server (language `cpp`) but we also support ccls (language `cpp_ccls`);
for best results, provide a `compile_commands.json` at the repository root;
see the [C/C++ Setup Guide](../03-special-guides/cpp_setup) for details.)
* **Clojure**
* **Dart**
* **Elixir**
@@ -47,7 +51,7 @@ Some languages require additional installations or setup steps, as noted.
* **Go**
(requires installation of `gopls`)
* **Groovy**
(requires local groovy-language-server.jar setup via GROOVY_LS_JAR_PATH or configuration)
(requires local groovy-language-server.jar setup via `GROOVY_LS_JAR_PATH` or configuration)
* **Haskell**
(automatically locates HLS via ghcup, stack, or system PATH; supports Stack and Cabal projects)
* **Java**
@@ -57,20 +61,21 @@ Some languages require additional installations or setup steps, as noted.
(uses the pre-alpha [official kotlin LS](https://github.com/Kotlin/kotlin-lsp), some issues may appear)
* **Lua**
* **Markdown**
(must be explicitly specified via `--language markdown` when generating project config, primarily useful for documentation-heavy projects)
(must explicitly enable language `markdown`, primarily useful for documentation-heavy projects)
* **Nix**
(requires nixd installation)
* **Pascal**
(Free Pascal/Lazarus; automatically downloads pasls binary; set PP and FPCDIR environment variables for source navigation)
* **Perl**
* **Pascal**
(uses Pascal/Lazarus, which is automatically downloaded; set `PP` and `FPCDIR` environment variables for source navigation)
* **Perl**
(requires installation of Perl::LanguageServer)
* **PHP**
(uses Intelephense LSP; set `INTELEPHENSE_LICENSE_KEY` environment variable for premium features)
(by default, uses the Intelephense language server (language `php`), set `INTELEPHENSE_LICENSE_KEY` environment variable for premium features;
we also support [Phpactor](https://github.com/phpactor/phpactor) (language `php_phpactor`), which requires PHP 8.1+)
* **Python**
* **R**
(requires installation of the `languageserver` R package)
* **Ruby**
(by default, uses [ruby-lsp](https://github.com/Shopify/ruby-lsp), specify ruby_solargraph as your language to use the previous solargraph based implementation)
(by default, uses [ruby-lsp](https://github.com/Shopify/ruby-lsp) (language `ruby`); use language `ruby_solargraph` to use Solargraph instead.)
* **Rust**
(requires [rustup](https://rustup.rs/) - uses rust-analyzer from your toolchain)
* **Scala**
docs/02-usage/020_running.md
+2 -2
View File
@@ -162,8 +162,8 @@ to get a list of all available options.
Some useful options include:
* `--project <path|name>`: specify the project to work on by name or path.
* `--project-from-cwd`: auto-detect the project from current working directory
(looking for a directory containing `.serena/project.yml` or `.git` in parent directories, activating the current directory if none is found);
* `--project-from-cwd`: auto-detect the project from current working directory
(looking for a directory containing `.serena/project.yml` or `.git` in parent directories and activating the containing directory as the project root, if any).
This option is intended for CLI-based agents like Claude Code, Gemini and Codex, which are typically started from within the project directory
and which do not change directories during their operation.
* `--language-backend JetBrains`: use the Serena JetBrains Plugin as the language backend (overriding the default backend configured in the central configuration)
docs/02-usage/025_jetbrains_plugin.md
+60
View File
@@ -73,6 +73,66 @@ If you need to work on multiple projects in the same agent session, create a mon
containing all the projects and open that folder in both Serena and your IDE.
:::
## Advanced Usage and Configuration
### Using Serena with Multi-Module Projects
JetBrains IDEs support *multi-module projects*, where a project can reference other projects as modules.
Serena, however, requires that a project is self-contained within a single root folder.
There has to be a one-to-one relationship between the project root folder and the folder that is open in the IDE.
Therefore, to get a multi-module setup working with Serena, the recommended approach is to create a **monorepo folder**,
i.e. a folder that contains all the projects as sub-folders, and open that monorepo folder in both Serena and your IDE.
You do not necessarily need to physically move your projects into a common parent folder;
you can also use symbolic links to achieve the same effect
(i.e. use `mklink` on Windows or `ln` on Linux/macOS to link the project folders into a common parent folder).
### Using Serena with Windows Subsystem for Linux (WSL)
JetBrains IDEs have built-in support for WSL, allowing you to run the IDE on Windows while working with code in the WSL environment.
The Serena JetBrains plugin works seamlessly in this setup as well. You have two options:
* **Windows-centric approach** (keeping things mainly in Windows)
* run both Serena and the IDE on Windows
* keep your project in the Windows file system, access it in WSL via `/mnt/`
(optionally linking the folder where you need it to be in the WSL file system)
* **WSL-centric approach** (keeping things mainly in WSL)
* run Serena in WSL (while the IDE runs in Windows)
* make sure Serena in WSL can communicate with services running on Windows.
This will normally be the case when using mirrored networking;
otherwise, you can configure `jetbrains_plugin_server_address` in your [serena_config.yml](050_configuration) and,
if necessary, configure the listen address of the Serena plugin in the IDE (Settings / Tools / Serena).
The WSL-centric approach has the downside that the WSL file system cannot be efficiently monitored by the IDE,
which can result in slower performance, especially for larger projects. If this is a problem, consider
* using the Windows-centric approach instead, or
* disabling the automatic synchronisation in the plugin and manually triggering synchronisation when needed (see below).
## Serena Plugin Configuration Options
You can configure plugin options in the IDE under Settings / Tools / Serena.
* **Listen address** (default: `127.0.0.1`)
the address the plugin's server listens on.
The default will work as long as Serena is running on the same machine (or on a virtual machine using mirrored networking).
But if the Serena MCP server is running on a different machine, configure the listen address to ensure that connections are possible.
You can use `0.0.0.0` to listen on all interfaces (but be aware of the security implications of doing so).
* **Sync file system before every operation** (default: enabled)
whether to synchronise the file system state before processing requests from Serena.
This is important to ensure that the plugin does not read stale data, but it can have a performance impact,
especially when using slow file systems (e.g. WSL file system while the IDE is running on Windows).
Note, however, that without synchronisation being forced by the Serena plugin, you will have to ensure synchronisation yourself.
Operations that apply changes to files in your project that are *not* made either in the IDE itself or by Serena may not be seen by the IDE.
Normally, the IDE synchronises automatically when it has the focus, using file watchers to achieve this (though this may or may not work reliably for the WSL file system).
Also, if you are working primarily in another application (e.g. AI chat), the IDE may not have the focus frequently.
So when external changes are made to your project, you will have to either give the IDE the focus (if that works) or trigger a sync manually (right-click root folder / Reload from Disk).
Further, note that even an edit made using, for example, Claude Code's internal editing tools would count as an external modification.
Only Serena's editing tools are "JetBrains-aware" and will tell the IDE to update the state of the edited file.
So if you are making AI-based edits using tools other than Serena's tools, do make sure that the lack of synchronisation is not a problem if you decide to disable this option.
## Usage with Other Editors
We realize that not everyone uses a JetBrains IDE as their main code editor.
docs/02-usage/030_clients.md
+1 -1
View File
@@ -96,7 +96,7 @@ claude mcp add --scope user serena -- uvx --from git+https://github.com/oraios/s
```
Whenever you start Claude Code, Serena will search up from the current directory for `.serena/project.yml` or `.git` markers,
activating the current directory as the project if neither is found.
activating the containing directory as the project (if any).
This mechanism makes it suitable for a single global MCP configuration.
**Maximum Token Efficiency.** To maximize token efficiency, you may want to use Claude Code's
docs/02-usage/050_configuration.md
+48 -3
View File
@@ -14,7 +14,8 @@ Serena is configured in using a multi-layered approach:
* **contexts and modes** for composable configuration, which can be enabled on a case-by-case basis (see below)
* **command-line parameters** passed to the `start-mcp-server` server command (overriding/extending configured settings)
See [MCP Server Command-Line Arguments](mcp-args) for further information.
(global-config)=
## Global Configuration
The global configuration file allows you to change general settings and defaults that will apply to all projects unless overridden.
@@ -173,7 +174,7 @@ You can change this location by setting the `SERENA_HOME` environment variable t
(ls-specific-settings)=
### Language Server-Specific Settings
:::note
:::{note}
**Advanced Users Only**: The settings described in this section are intended for advanced users who need to fine-tune language server behavior.
Most users will not need to adjust these settings.
:::
@@ -209,9 +210,53 @@ ls_specific_settings:
```
This is supported by all language servers deriving their dependency provider from `LanguageServerDependencyProviderSinglePath`.
Currently, this includes the following languages: `clojure`, `cpp`, `php`, `python`, `typescript`.
Currently, this includes the following languages: `bash`, `clojure`, `cpp`, `markdown`, `php`, `php_phpactor`, `python`, `rust`, `toml`, `typescript`, `yaml`.
We will add support for more languages over time.
#### C# (Roslyn Language Server)
Serena uses [Microsoft's Roslyn Language Server](https://github.com/dotnet/roslyn) for C# support.
**Runtime Requirements:**
- .NET 10 or higher is required. If not found in PATH, Serena automatically installs it using Microsoft's official install scripts.
- The Roslyn Language Server is automatically downloaded from NuGet.org.
**Supported Platforms:**
Automatic download is supported for: Windows (x64, ARM64), macOS (x64, ARM64), Linux (x64, ARM64).
**Configuration:**
The `runtime_dependencies` setting allows you to override the download URLs for the Roslyn Language Server. This is useful if you need to use a private package mirror or a specific version.
Example configuration to override the language server download URL:
```yaml
ls_specific_settings:
csharp:
runtime_dependencies:
- id: "CSharpLanguageServer"
platform_id: "linux-x64" # or win-x64, win-arm64, osx-x64, osx-arm64, linux-arm64
url: "https://your-mirror.example.com/roslyn-language-server.linux-x64.5.5.0-2.26078.4.nupkg"
package_version: "5.5.0-2.26078.4"
```
Available fields for `runtime_dependencies` entries:
| Field | Description |
| ----------------- | --------------------------------------------------------------------------- |
| `id` | Dependency identifier (use `CSharpLanguageServer`) |
| `platform_id` | Target platform: `win-x64`, `win-arm64`, `osx-x64`, `osx-arm64`, `linux-x64`, `linux-arm64` |
| `url` | Download URL for the NuGet package |
| `package_version` | Package version string |
| `extract_path` | Path within the package to extract (default: `tools/net10.0/<platform>`) |
Notes:
- Only specify the platforms you want to override; others will use the defaults.
- The language server package is a `.nupkg` file (ZIP format) downloaded from NuGet.org by default.
- If you have .NET 10+ already installed, Serena will use your system installation.
#### Go (`gopls`)
Serena forwards `ls_specific_settings.go.gopls_settings` to `gopls` as LSP `initializationOptions` when the Go language server is started.
docs/02-usage/065_logs.md
+14
View File
@@ -0,0 +1,14 @@
# Logs
It can be vital to understand what is happening in Serena, especially when something goes wrong.
You can access Serena's live logs via
* the [Serena dashboard](060_dashboard) (tab "Logs")
* the [GUI tool](060_dashboard).
Additionally, logs are persisted in the Serena home directory, which, by default, is located at
* `%USERPROFILE%\.serena\logs` on Windows
* `~/.serena/logs` on Linux and macOS.
You can adjust the log level via the [global configuration](global-config).
You additionally have the option of enabling full tracing of language server communication (mostly for development purposes).
docs/03-special-guides/cpp_setup.md
+100
View File
@@ -0,0 +1,100 @@
# C/C++ Setup Guide
This guide explains how to prepare a C/C++ project so that Serena can provide reliable code intelligence via clangd or ccls language servers.
This is only necessary if you use the language server variant of Serena, for users of the Serena JetBrains plugin no setup is required
and the limitations described below do not apply.
---
## General
Serena supports two C/C++ language servers, clangd (default) and ccls.
Both have their pros and cons and require a properly configured `compile_commands.json`
for cross-file reference finding, see below for details.
Your project must have a `compile_commands.json` file at the repository root.
This file is essential for correct parsing and cross-file reference finding.
## compile_commands.json Requirements
For reliable cross-file reference finding with clangd, your `compile_commands.json` must:
1. **Include proper C++ standard flags** (e.g., `-std=c++17`)
2. **Include all necessary include paths** (`-I` flags)
---
### With clangd
Serena automatically downloads and manages clangd. Since clangd does not properly work with relative paths in `compile_commands.json`,
Serena will detect them and transform them into absolute paths automatically (writing a new `compile_commands.json` file), if needed.
#### Customizing the Compilation Database Location
By default, Serena creates the transformed compilation database at `.serena/compile_commands.json`.
You can customize this location via project settings:
```yaml
# .serena/project.yml
language_servers:
cpp:
compile_commands_dir: custom/rel/path (defaults to .serena)
```
### With ccls
ccls requires manual installation and configuration. It may perform better in some situations.
#### Installation
**Linux:**
```bash
# Ubuntu/Debian (22.04+)
sudo apt-get install ccls
# Fedora/RHEL
sudo dnf install ccls
# Arch Linux
sudo pacman -S ccls
```
**macOS:**
```bash
brew install ccls
```
**Windows:**
```bash
choco install ccls
```
#### Configuration
After installing ccls, configure Serena to use it via project settings (in `.serena/project.yml`)
by adding `cpp_ccls` to the `languages` list. Replace `cpp` with `cpp_ccls` if you already have the `cpp` entry.
ccls can handle relative paths in `compile_commands.json`, so no transformation is necessary
and no transformed `compile_commands.json` file will be created.
---
## Known Limitations
### Files Created After Server Initialization
Both clangd and ccls have a fundamental limitation:
**files created by external mechanisms after the language server starts are not automatically indexed**.
Cross-file references to newly created files will not work unless the new file is at some point opened by the language server (for example, by a symbol lookup in it), or until `compile_commands.json` is updated and
the language server is restarted.
---
## Reference
- Clangd official documentation: https://clangd.llvm.org/
- Clangd project setup: https://clangd.llvm.org/installation#project-setup
- CCLS repository: https://github.com/MaskRay/ccls
docs/03-special-guides/scala_setup_guide_for_serena.md
+37 -1
View File
@@ -63,5 +63,41 @@ These instructions cover the setup for projects that use sbt as the build tool,
Notes:
- Ensure you completed the manual or auto‑import steps so that the build is compiled and indexed; otherwise, code navigation and references may be incomplete until the first successful compile.
---
## Running Multiple Metals Instances
Serena can run alongside other Metals instances (e.g., VS Code with Metals extension) on the same project. This is **fully supported** by Metals via H2 AUTO_SERVER mode.
### How It Works
Metals uses an H2 database (`.metals/metals.mv.db`) to cache semantic information. When multiple Metals instances run on the same project:
- **H2 AUTO_SERVER**: The first instance becomes the TCP server; subsequent instances connect as clients
- **Bloop Build Server**: All instances share a single Bloop process (port 8212)
- **Compilation Results**: Shared via Bloop — no duplicate compilation
### Stale Lock Detection
If a Metals process crashes without proper cleanup, it may leave a stale lock file (`.metals/metals.mv.db.lock.db`). This can prevent proper AUTO_SERVER coordination, causing new instances to fall back to in-memory database mode (degraded experience).
Serena automatically detects and handles stale locks based on your configuration:
```yaml
# ~/.serena/serena_config.yml or .serena/project.yml
ls_specific_settings:
scala:
on_stale_lock: "auto-clean" # auto-clean | warn | fail
log_multi_instance_notice: true # Log info when another Metals detected
```
#### Stale Lock Modes
| Mode | Behavior |
|------|----------|
| `auto-clean` | **(Default, Recommended)** Automatically removes stale lock files and proceeds normally. |
| `warn` | Logs a warning but proceeds. Metals may use in-memory database (slower). |
| `fail` | Raises an error and refuses to start. Useful for debugging lock issues. |
---
## Reference
- Metals + sbt: [https://scalameta.org/metals/docs/build-tools/sbt](https://scalameta.org/metals/docs/build-tools/sbt)
- Metals + sbt: [https://scalameta.org/metals/docs/build-tools/sbt](https://scalameta.org/metals/docs/build-tools/sbt)
pyproject.toml
+9 -3
View File
@@ -114,7 +114,11 @@ warn_redundant_casts = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = false
exclude = "^build/|^docs/"
exclude = "^build/|^docs/|^test/resources/"
[[tool.mypy.overrides]]
module = "test.*"
disallow_untyped_defs = false # untyped defs are OK in tests
[tool.poe.env]
PYDEVD_DISABLE_FILE_VALIDATION = "1"
@@ -137,8 +141,9 @@ _black_format = "black src scripts test"
_ruff_format = "ruff check --fix src scripts test"
lint = ["_black_check", "_ruff_check"]
format = ["_ruff_format", "_black_format"]
_mypy = "mypy src/serena src/solidlsp"
type-check = ["_mypy"]
_mypy_core = "mypy src/serena src/solidlsp"
_mypy_test = "mypy --disable-error-code no-untyped-def test"
type-check = ["_mypy_core", "_mypy_test"]
# docs
_autogen_rst = "python docs/autogen_rst.py"
_sphinx_build = "sphinx-build -b html docs docs/_build -W --keep-going"
@@ -313,6 +318,7 @@ markers = [
"yaml: language server running for YAML",
"powershell: language server running for PowerShell",
"pascal: language server running for Pascal (Free Pascal/Lazarus)",
"cpp: language server running for C/C++",
"slow: tests that require additional Expert instances and have long startup times (~60-90s each)",
"toml: language server running for TOML",
"matlab: language server running for MATLAB (requires MATLAB R2021b+)",
src/serena/agent.py
+1 -1
View File
@@ -522,7 +522,7 @@ class SerenaAgent:
:param mode_names: List of mode names or paths to use
"""
self._active_modes.apply(ModeSelectionDefinition(default_modes=mode_names))
self._mode_overrides = ModeSelectionDefinition(default_modes=mode_names)
self._update_active_modes_and_tools()
log.info(f"Set modes to {[mode.name for mode in self.get_active_modes()]}")
src/serena/cli.py
+20 -15
View File
@@ -39,6 +39,7 @@ from serena.tools import FindReferencingSymbolsTool, FindSymbolTool, GetSymbolsO
from serena.util.dataclass import get_dataclass_default
from serena.util.logging import MemoryLogHandler
from solidlsp.ls_config import Language
from solidlsp.ls_types import SymbolKind
from solidlsp.util.subprocess_util import subprocess_kwargs
log = logging.getLogger(__name__)
@@ -55,15 +56,14 @@ Overriding them means that they no longer apply, so you will need to
re-specify them in addition to further modes if you want to keep them."""
def find_project_root(root: str | Path | None = None) -> str:
def find_project_root(root: str | Path | None = None) -> str | None:
"""Find project root by walking up from CWD.
Checks for .serena/project.yml first (explicit Serena project), then .git (git root).
Falls back to CWD if no marker is found.
:param root: If provided, constrains the search to this directory and below
(acts as a virtual filesystem root). Search stops at this boundary.
:return: absolute path to project root (falls back to CWD if no marker found)
:return: absolute path to project root or None if not suitable root is found
"""
current = Path.cwd().resolve()
boundary = Path(root).resolve() if root is not None else None
@@ -86,8 +86,7 @@ def find_project_root(root: str | Path | None = None) -> str:
if (directory / ".git").exists(): # .git can be file (worktree) or dir
return str(directory)
# Fall back to CWD
return str(current)
return None
# --------------------- Utilities -------------------------------------
@@ -274,7 +273,10 @@ class TopLevelCommands(AutoRegisteringGroup):
if project is not None or project_file_arg is not None:
raise click.UsageError("--project-from-cwd cannot be used with --project or positional project argument")
project = find_project_root()
log.info("Auto-detected project root: %s", project)
if project is not None:
log.info("Auto-detected project root: %s", project)
else:
log.warning("No project root found from %s; not activating any project", os.getcwd())
project_file = project_file_arg or project
factory = SerenaMCPFactory(context=context, project=project_file, memory_log_handler=memory_log_handler)
@@ -636,7 +638,7 @@ class ProjectCommands(AutoRegisteringGroup):
lvl = logging.getLevelNamesMapping()[log_level.upper()]
logging.configure(level=lvl)
serena_config = SerenaConfig.from_config_file()
proj = registered_project.get_project_instance()
proj = registered_project.get_project_instance(serena_config=serena_config)
click.echo(f"Indexing symbols in {proj} …")
ls_mgr = proj.create_language_server_manager(
log_level=lvl, ls_timeout=timeout, ls_specific_settings=serena_config.ls_specific_settings
@@ -689,7 +691,8 @@ class ProjectCommands(AutoRegisteringGroup):
:param path: The path to check.
:param project: The path to the project directory, defaults to the current working directory.
"""
proj = Project.load(os.path.abspath(project))
serena_config = SerenaConfig.from_config_file()
proj = Project.load(os.path.abspath(project), serena_config=serena_config)
if os.path.isabs(path):
path = os.path.relpath(path, start=proj.project_root)
is_ignored = proj.is_ignored_path(path)
@@ -711,7 +714,8 @@ class ProjectCommands(AutoRegisteringGroup):
:param project: path to the project directory, defaults to the current working directory.
:param verbose: if set, prints detailed information about the indexed symbols.
"""
proj = Project.load(os.path.abspath(project))
serena_config = SerenaConfig.from_config_file()
proj = Project.load(os.path.abspath(project), serena_config=serena_config)
if os.path.isabs(file):
file = os.path.relpath(file, start=proj.project_root)
if proj.is_ignored_path(file, ignore_non_source_files=True):
@@ -748,7 +752,8 @@ class ProjectCommands(AutoRegisteringGroup):
# NOTE: completely written by Claude Code, only functionality was reviewed, not implementation
logging.configure(level=logging.INFO)
project_path = os.path.abspath(project)
proj = Project.load(project_path)
serena_config = SerenaConfig.from_config_file()
proj = Project.load(project_path, serena_config=serena_config)
# Create log file with timestamp
timestamp = datetime_tag()
@@ -762,8 +767,10 @@ class ProjectCommands(AutoRegisteringGroup):
try:
# Create SerenaAgent with dashboard disabled
log.info("Creating SerenaAgent with disabled dashboard...")
config = SerenaConfig(gui_log_window=False, web_dashboard=False)
agent = SerenaAgent(project=project_path, serena_config=config)
serena_config = SerenaConfig.from_config_file()
serena_config.gui_log_window = False
serena_config.web_dashboard = False
agent = SerenaAgent(project=project_path, serena_config=serena_config)
log.info("SerenaAgent created successfully")
# Find first non-empty file that can be analyzed
@@ -805,9 +812,7 @@ class ProjectCommands(AutoRegisteringGroup):
return
# Extract suitable symbol (prefer class or function over variables)
# LSP symbol kinds: 5=class, 12=function, 6=method, 9=constructor
preferred_kinds = [5, 12, 6, 9] # class, function, method, constructor
preferred_kinds = {SymbolKind.Class.name, SymbolKind.Function.name, SymbolKind.Method.name, SymbolKind.Constructor.name}
selected_symbol = None
for symbol in overview_data:
if symbol.get("kind") in preferred_kinds:
src/serena/code_editor.py
+3 -2
View File
@@ -6,6 +6,7 @@ from collections.abc import Iterable, Iterator, Reversible
from contextlib import contextmanager
from typing import TYPE_CHECKING, Generic, Optional, TypeVar, cast
from serena.jetbrains.jetbrains_plugin_client import JetBrainsPluginClient
from serena.symbol import JetBrainsSymbol, LanguageServerSymbol, LanguageServerSymbolRetriever, PositionInFile, Symbol
from solidlsp import SolidLanguageServer, ls_types
from solidlsp.ls import LSPFileBuffer
@@ -13,7 +14,6 @@ from solidlsp.ls_utils import PathUtils, TextUtils
from .constants import DEFAULT_SOURCE_FILE_ENCODING
from .project import Project
from .tools.jetbrains_plugin_client import JetBrainsPluginClient
if TYPE_CHECKING:
from .agent import SerenaAgent
@@ -81,8 +81,9 @@ class CodeEditor(Generic[TSymbol], ABC):
def _save_edited_file(self, edited_file: "CodeEditor.EditedFile") -> None:
abs_path = os.path.join(self.project_root, edited_file.relative_path)
new_contents = edited_file.get_contents()
with open(abs_path, "w", encoding=self.encoding) as f:
f.write(edited_file.get_contents())
f.write(new_contents)
@abstractmethod
def _find_unique_symbol(self, name_path: str, relative_file_path: str) -> TSymbol:
src/serena/config/serena_config.py
+39 -7
View File
@@ -8,7 +8,7 @@ import shutil
from collections.abc import Iterator, Sequence
from copy import deepcopy
from dataclasses import dataclass, field
from datetime import datetime
from datetime import UTC, datetime
from enum import Enum
from functools import cached_property
from pathlib import Path
@@ -415,12 +415,18 @@ class ProjectConfig(ToolInclusionDefinition, ModeSelectionDefinition, ToStringMi
class RegisteredProject(ToStringMixin):
def __init__(self, project_root: str, project_config: "ProjectConfig", project_instance: Optional["Project"] = None) -> None:
def __init__(
self,
project_root: str,
project_config: "ProjectConfig",
project_instance: Optional["Project"] = None,
) -> None:
"""
Represents a registered project in the Serena configuration.
:param project_root: the root directory of the project
:param project_config: the configuration of the project
:param project_instance: an existing project instance (if already loaded)
"""
self.project_root = Path(project_root).resolve()
self.project_config = project_config
@@ -456,9 +462,9 @@ class RegisteredProject(ToStringMixin):
:param path: the path to check
:return: True if the path matches the project root, False otherwise
"""
return self.project_root == Path(path).resolve()
return self.project_root.samefile(Path(path).resolve())
def get_project_instance(self) -> "Project":
def get_project_instance(self, serena_config: "SerenaConfig | None") -> "Project":
"""
Returns the project instance for this registered project, loading it if necessary.
"""
@@ -466,7 +472,11 @@ class RegisteredProject(ToStringMixin):
from ..project import Project
with LogTime(f"Loading project instance for {self}", logger=log):
self._project_instance = Project(project_root=str(self.project_root), project_config=self.project_config)
self._project_instance = Project(
project_root=str(self.project_root),
project_config=self.project_config,
serena_config=serena_config,
)
return self._project_instance
@@ -511,6 +521,10 @@ class SerenaConfig(ToolInclusionDefinition, ModeSelectionDefinition, ToStringMix
ls_specific_settings: dict = field(default_factory=dict)
"""Advanced configuration option allowing to configure language server implementation specific options, see SolidLSPSettings for more info."""
ignored_paths: list[str] = field(default_factory=list)
"""List of paths to ignore across all projects. Same syntax as gitignore, so you can use * and **.
These patterns are merged additively with each project's own ignored_paths."""
hover_budget: float = 10.0
"""
Time budget (seconds) for LSP hover requests when tools request include_info.
@@ -537,6 +551,19 @@ class SerenaConfig(ToolInclusionDefinition, ModeSelectionDefinition, ToStringMix
CONFIG_FIELDS_WITH_TYPE_CONVERSION = {"projects", "language_backend"}
# *** methods ***
@classmethod
def get_config_file_creation_date(cls) -> datetime | None:
"""
:return: the creation date of the configuration file, or None if the configuration file does not exist
"""
config_file_path = cls._determine_config_file_path()
if not os.path.exists(config_file_path):
return None
# for unix systems st_ctime is the inode change time (change of metadata),
# which is good enough for our purposes
creation_timestamp = os.stat(config_file_path).st_ctime
return datetime.fromtimestamp(creation_timestamp, UTC)
@property
def config_file_path(self) -> str | None:
@@ -737,7 +764,7 @@ class SerenaConfig(ToolInclusionDefinition, ModeSelectionDefinition, ToStringMix
if registered_project is None:
return None
else:
return registered_project.get_project_instance()
return registered_project.get_project_instance(serena_config=self)
def add_registered_project(self, registered_project: RegisteredProject) -> None:
"""
@@ -771,7 +798,12 @@ class SerenaConfig(ToolInclusionDefinition, ModeSelectionDefinition, ToStringMix
project_config = ProjectConfig.load(project_root, autogenerate=True)
new_project = Project(project_root=str(project_root), project_config=project_config, is_newly_created=True)
new_project = Project(
project_root=str(project_root),
project_config=project_config,
is_newly_created=True,
serena_config=self,
)
self.add_registered_project(RegisteredProject.from_project_instance(new_project))
return new_project
src/serena/constants.py
+3
View File
@@ -28,3 +28,6 @@ PROJECT_TEMPLATE_FILE = str(_serena_pkg_path / "resources" / "project.template.y
SERENA_CONFIG_TEMPLATE_FILE = str(_serena_pkg_path / "resources" / "serena_config.template.yml")
SERENA_LOG_FORMAT = "%(levelname)-5s %(asctime)-15s [%(threadName)s] %(name)s:%(funcName)s:%(lineno)d - %(message)s"
LOG_MESSAGES_BUFFER_SIZE = 2500
"""The maximum number of log messages to keep in the buffer (for the dashboard)."""
src/serena/dashboard.py
+18 -6
View File
@@ -10,7 +10,7 @@ from pydantic import BaseModel
from sensai.util import logging
from serena.analytics import ToolUsageStats
from serena.config.serena_config import LanguageBackend, SerenaPaths
from serena.config.serena_config import LanguageBackend, SerenaConfig, SerenaPaths
from serena.constants import SERENA_DASHBOARD_DIR
from serena.task_executor import TaskExecutor
from serena.util.logging import MemoryLogHandler
@@ -324,12 +324,25 @@ class SerenaDashboardAPI:
def _get_unread_news_ids() -> list[int]:
all_news_files = (Path(SERENA_DASHBOARD_DIR) / "news").glob("*.html")
all_news_ids = [int(f.stem) for f in all_news_files]
"""News ids are ints of format YYYYMMDD (publication dates)"""
# Filter news items by installation date
serena_config_creation_date = SerenaConfig.get_config_file_creation_date()
if serena_config_creation_date is None:
# should not normally happen, since config file should exist when the dashboard is started
# We assume a fresh installation in this case
log.error("Serena config file not found when starting the dashboard")
return []
serena_config_creation_date_int = int(serena_config_creation_date.strftime("%Y%m%d"))
# Only include news items published on or after the installation date
post_installation_news_ids = [news_id for news_id in all_news_ids if news_id >= serena_config_creation_date_int]
news_snippet_id_file = SerenaPaths().news_snippet_id_file
if not os.path.exists(news_snippet_id_file):
return all_news_ids
return post_installation_news_ids
with open(news_snippet_id_file, encoding="utf-8") as f:
last_read_news_id = int(f.read().strip())
return [news_id for news_id in all_news_ids if news_id > last_read_news_id]
return [news_id for news_id in post_installation_news_ids if news_id > last_read_news_id]
try:
unread_news_ids = _get_unread_news_ids()
@@ -350,11 +363,10 @@ class SerenaDashboardAPI:
return {"status": "error", "message": str(e)}
def _get_log_messages(self, request_log: RequestLog) -> ResponseLog:
all_messages = self._memory_log_handler.get_log_messages()
requested_messages = all_messages[request_log.start_idx :] if request_log.start_idx <= len(all_messages) else []
messages = self._memory_log_handler.get_log_messages(from_idx=request_log.start_idx)
project = self._agent.get_active_project()
project_name = project.project_name if project else None
return ResponseLog(messages=requested_messages, max_idx=len(all_messages) - 1, active_project=project_name)
return ResponseLog(messages=messages.messages, max_idx=messages.max_idx, active_project=project_name)
def _get_tool_names(self) -> ResponseToolNames:
return ResponseToolNames(tool_names=self._tool_names)
src/serena/gui_log_viewer.py
+1 -1
View File
@@ -68,7 +68,7 @@ class GuiLogViewer:
}
if memory_log_handler is not None:
for msg in memory_log_handler.get_log_messages():
for msg in memory_log_handler.get_log_messages().messages:
self.message_queue.put(msg)
memory_log_handler.add_emit_callback(lambda msg: self.message_queue.put(msg))
src/serena/tools/jetbrains_plugin_client.py → src/serena/jetbrains/jetbrains_plugin_client.py
+59 -28
View File
@@ -12,10 +12,10 @@ import requests
from requests import Response
from sensai.util.string import ToStringMixin
import serena.tools.jetbrains_types as jb
import serena.jetbrains.jetbrains_types as jb
from serena.jetbrains.jetbrains_types import PluginStatusDTO
from serena.project import Project
from serena.text_utils import render_html
from serena.tools.jetbrains_types import PluginStatusDTO
from serena.util.version import Version
T = TypeVar("T")
@@ -49,7 +49,7 @@ class PluginServerError(SerenaClientError):
unrecoverable_error_prefix = (
"The request failed due to an error on the Serena JetBrains plugin side. This may be caused "
"by a bug or an incompatibility in the plugin. In that case, please report the error below together with "
"your setup (IDE + version, Serena plugin version) at https://github.com/oraios/serena/issues "
"your setup (IDE + version, Serena plugin version, programming language) at https://github.com/oraios/serena/issues "
"(if you are an LLM, tell the user about this). The Oraios team will then provide support on this.\n"
)
super().__init__(unrecoverable_error_prefix + message)
@@ -96,7 +96,7 @@ class JetBrainsPluginClient(ToStringMixin):
except ConnectionError: # expected if no server is running at the port
pass
except Exception as e:
log.warning("Failed to obtain status from JetBrains plugin service at port %d: %s", e, port, exc_info=True)
log.warning("Failed to obtain status from JetBrains plugin service at port %d: %s", port, e, exc_info=e)
@property
def _base_url(self) -> str:
@@ -118,43 +118,71 @@ class JetBrainsPluginClient(ToStringMixin):
if client.matches(resolved_path):
return client
plugin_paths_found = []
for port in range(cls.BASE_PORT, cls.BASE_PORT + 20):
client = JetBrainsPluginClient(port)
if client.matches(resolved_path):
log.info("Found matching %s", client)
cls._last_port = port
return client
elif client._project_root is not None:
plugin_paths_found.append(client._project_root)
log.warning(
"Searched for Serena JetBrains plugin service for project at %s but found no matching service. "
"Found plugin instances for the following project paths: %s",
resolved_path,
plugin_paths_found,
)
raise ServerNotFoundError("Found no Serena service in a JetBrains IDE instance for the project at " + str(resolved_path))
@staticmethod
def _normalize_wsl_path(path_str: str) -> Path:
def _paths_match(resolved_serena_path: str, plugin_path: str) -> bool:
"""
Normalize WSL UNC paths to Linux paths for comparison.
Checks whether the resolved Serena path matches the plugin path, accounting for possible prefixes
in the plugin path, different file system perspectives, and case sensitivity.
When JetBrains IDE runs on Windows with a project opened from WSL,
it returns paths like `//wsl.localhost/Ubuntu-24.04/home/user/project`
or `//wsl$/Ubuntu/home/user/project`. This method converts such paths
to standard Linux format `/home/user/project` for proper matching.
Concrete aspects considered:
- The plugin path may contain prefixes:
- The plugin path may be a WSL UNC path, e.g. `//wsl.localhost/Ubuntu-24.04/home/user/project`
or `//wsl$/Ubuntu/home/user/project` while Serena will just have `/home/user/project`
- Other prefixes like `/workspaces/serena/C:/Users/user/projects/my-app`
- One path may use a different file system perspective (particularly WSL vs Windows-native) but still
point to the same location, e.g. `/mnt/c/` vs `C:/`
- Case sensitivity
:param path_str: Path string that may be a WSL UNC path
:return: Normalized Path object
:param resolved_serena_path: The resolved project root path from Serena's perspective
:param plugin_path: The project root path reported by the plugin (which may be a WSL UNC path)
:return: True if the paths match, False otherwise
"""
path_str = str(path_str)
# Match //wsl.localhost/<distro>/... or //wsl$/<distro>/...
match = re.match(r"^//wsl(?:\.localhost|\$)/[^/]+(.*)$", path_str, re.IGNORECASE)
if match:
return Path(match.group(1))
return Path(path_str)
# try to resolve the plugin path, checking for a direct match
# (this is robust against symlinks as long as there are no prefixes)
try:
resolved_plugin_path = str(Path(plugin_path).resolve())
if resolved_plugin_path == resolved_serena_path:
return True
except:
pass
def normalise_wsl_mnt(path_str: str) -> str:
# normalise WSL /mnt/c/ to c:/ for comparison
return re.sub(r"/mnt/([a-z])/", r"\1:/", path_str, flags=re.IGNORECASE)
# standardise paths for comparison: normalise WSL /mnt/ to Windows paths and ignore case
std_serena_path = normalise_wsl_mnt(str(resolved_serena_path)).lower()
std_plugin_path = normalise_wsl_mnt(str(plugin_path)).lower()
# At this point, the plugin path may still contain prefixes, so we check if the Serena path is a suffix of the plugin path
return std_plugin_path.endswith(std_serena_path)
def matches(self, resolved_path: Path) -> bool:
"""
:param resolved_path: the resolved project root path from Serena's perspective
:return: whether this client instance matches the given project path
"""
if self._project_root is None:
return False
try:
plugin_root = self._normalize_wsl_path(self._project_root)
return plugin_root.resolve() == resolved_path
except ConnectionError:
return False
return self._paths_match(str(resolved_path), self._project_root)
def is_version_at_least(self, *version_parts: int) -> bool:
if self._plugin_version is None:
@@ -252,11 +280,14 @@ class JetBrainsPluginClient(ToStringMixin):
else:
del symbol[key]
# convert documentation and quick info from HTML to plain text (if present)
symbols = response_dict["symbols"]
for s in symbols:
convert_html("documentation", s)
convert_html("quick_info", s)
def convert_symbol_list(l: list) -> None:
for s in l:
convert_html("documentation", s)
convert_html("quick_info", s)
if "children" in s:
convert_symbol_list(s["children"])
convert_symbol_list(response_dict["symbols"])
def find_symbol(
self,
src/serena/tools/jetbrains_types.py → src/serena/jetbrains/jetbrains_types.py
+4 -1
View File
@@ -1,4 +1,4 @@
from typing import NotRequired, TypedDict
from typing import Literal, NotRequired, TypedDict
class PluginStatusDTO(TypedDict):
@@ -30,6 +30,9 @@ class SymbolDTO(TypedDict):
num_usages: NotRequired[int]
SymbolDTOKey = Literal["name_path", "relative_path", "type", "body", "quick_info", "documentation", "text_range", "children", "num_usages"]
class SymbolCollectionResponse(TypedDict):
symbols: list[SymbolDTO]
src/serena/mcp.py
+1
View File
@@ -342,6 +342,7 @@ class SerenaMCPFactory:
self._set_mcp_tools(mcp_server, openai_tool_compatible=openai_tool_compatible)
log.info("MCP server lifetime setup complete")
yield
log.info("MCP server shutting down")
def _get_initial_instructions(self) -> str:
assert self.agent is not None
src/serena/project.py
+76 -27
View File
@@ -1,10 +1,12 @@
import json
import logging
import os
import threading
from pathlib import Path
from typing import Any
from typing import TYPE_CHECKING, Any
import pathspec
from sensai.util.logging import LogTime
from sensai.util.string import ToStringMixin
from serena.config.serena_config import DEFAULT_TOOL_TIMEOUT, ProjectConfig, get_serena_managed_in_project_dir
@@ -16,6 +18,9 @@ from solidlsp import SolidLanguageServer
from solidlsp.ls_config import Language
from solidlsp.ls_utils import FileUtils
if TYPE_CHECKING:
from serena.config.serena_config import SerenaConfig
log = logging.getLogger(__name__)
@@ -54,7 +59,13 @@ class MemoriesManager:
class Project(ToStringMixin):
def __init__(self, project_root: str, project_config: ProjectConfig, is_newly_created: bool = False):
def __init__(
self,
project_root: str,
project_config: ProjectConfig,
is_newly_created: bool = False,
serena_config: "SerenaConfig | None" = None,
):
self.project_root = project_root
self.project_config = project_config
self.memories_manager = MemoriesManager(project_root)
@@ -69,27 +80,44 @@ class Project(ToStringMixin):
with open(serena_data_gitignore_path, "w", encoding="utf-8") as f:
f.write(f"/{SolidLanguageServer.CACHE_FOLDER_NAME}\n")
# gather ignored paths from the project configuration and gitignore files
ignored_patterns = list(project_config.ignored_paths)
if len(ignored_patterns) > 0:
log.info(f"Using {len(ignored_patterns)} ignored paths from the explicit project configuration.")
log.debug(f"Ignored paths: {ignored_patterns}")
if project_config.ignore_all_files_in_gitignore:
gitignore_parser = GitignoreParser(self.project_root)
for spec in gitignore_parser.get_ignore_specs():
log.debug(f"Adding {len(spec.patterns)} patterns from {spec.file_path} to the ignored paths.")
ignored_patterns.extend(spec.patterns)
self._ignored_patterns = ignored_patterns
# prepare ignore spec asynchronously, ensuring immediate project activation.
self._serena_config = serena_config
self.__ignored_patterns: list[str]
self.__ignore_spec: pathspec.PathSpec
self._ignore_spec_available = threading.Event()
threading.Thread(name=f"gather-ignorespec[{self.project_config.project_name}]", target=self._gather_ignorespec, daemon=True).start()
# Set up the pathspec matcher for the ignored paths
# for all absolute paths in ignored_paths, convert them to relative paths
processed_patterns = []
for pattern in set(ignored_patterns):
# Normalize separators (pathspec expects forward slashes)
pattern = pattern.replace(os.path.sep, "/")
processed_patterns.append(pattern)
log.debug(f"Processing {len(processed_patterns)} ignored paths")
self._ignore_spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, processed_patterns)
def _gather_ignorespec(self) -> None:
with LogTime(f"Gathering ignore spec for project {self.project_config.project_name}", logger=log):
# gather ignored paths from the global configuration, project configuration, and gitignore files
global_ignored_paths = self._serena_config.ignored_paths if self._serena_config else []
ignored_patterns = list(global_ignored_paths) + list(self.project_config.ignored_paths)
if len(global_ignored_paths) > 0:
log.info(f"Using {len(global_ignored_paths)} ignored paths from the global configuration.")
log.debug(f"Global ignored paths: {list(global_ignored_paths)}")
if len(self.project_config.ignored_paths) > 0:
log.info(f"Using {len(self.project_config.ignored_paths)} ignored paths from the project configuration.")
log.debug(f"Project ignored paths: {self.project_config.ignored_paths}")
log.debug(f"Combined ignored patterns: {ignored_patterns}")
if self.project_config.ignore_all_files_in_gitignore:
gitignore_parser = GitignoreParser(self.project_root)
for spec in gitignore_parser.get_ignore_specs():
log.debug(f"Adding {len(spec.patterns)} patterns from {spec.file_path} to the ignored paths.")
ignored_patterns.extend(spec.patterns)
self.__ignored_patterns = ignored_patterns
# Set up the pathspec matcher for the ignored paths
# for all absolute paths in ignored_paths, convert them to relative paths
processed_patterns = []
for pattern in set(ignored_patterns):
# Normalize separators (pathspec expects forward slashes)
pattern = pattern.replace(os.path.sep, "/")
processed_patterns.append(pattern)
log.debug(f"Processing {len(processed_patterns)} ignored paths")
self.__ignore_spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, processed_patterns)
self._ignore_spec_available.set()
def _tostring_includes(self) -> list[str]:
return []
@@ -102,12 +130,17 @@ class Project(ToStringMixin):
return self.project_config.project_name
@classmethod
def load(cls, project_root: str | Path, autogenerate: bool = True) -> "Project":
def load(
cls,
project_root: str | Path,
serena_config: "SerenaConfig | None",
autogenerate: bool = True,
) -> "Project":
project_root = Path(project_root).resolve()
if not project_root.exists():
raise FileNotFoundError(f"Project root not found: {project_root}")
project_config = ProjectConfig.load(project_root, autogenerate=autogenerate)
return Project(project_root=str(project_root), project_config=project_config)
return Project(project_root=str(project_root), project_config=project_config, serena_config=serena_config)
def save_config(self) -> None:
"""
@@ -151,12 +184,28 @@ class Project(ToStringMixin):
abs_path = Path(self.project_root) / relative_path
return FileUtils.read_file(str(abs_path), self.project_config.encoding)
def get_ignore_spec(self) -> pathspec.PathSpec:
@property
def _ignore_spec(self) -> pathspec.PathSpec:
"""
:return: the pathspec matcher for the paths that were configured to be ignored,
either explicitly or implicitly through .gitignore files.
"""
return self._ignore_spec
if not self._ignore_spec_available.is_set():
log.info("Waiting for ignore spec to become available ...")
self._ignore_spec_available.wait()
log.info("Ignore spec is now available for project; proceeding")
return self.__ignore_spec
@property
def _ignored_patterns(self) -> list[str]:
"""
:return: the list of ignored path patterns
"""
if not self._ignore_spec_available.is_set():
log.info("Waiting for ignored patterns to become available ...")
self._ignore_spec_available.wait()
log.info("Ignore patterns are now available for project; proceeding")
return self.__ignored_patterns
def _is_ignored_relative_path(self, relative_path: str | Path, ignore_non_source_files: bool = True) -> bool:
"""
@@ -198,7 +247,7 @@ class Project(ToStringMixin):
if len(rel_path.parts) > 0 and rel_path.parts[0] == ".git":
return True
return match_path(str(relative_path), self.get_ignore_spec(), root_path=self.project_root)
return match_path(str(relative_path), self._ignore_spec, root_path=self.project_root)
def is_ignored_path(self, path: str | Path, ignore_non_source_files: bool = False) -> bool:
"""
src/serena/resources/dashboard/dashboard.css
+47 -3
View File
@@ -213,13 +213,48 @@ body {
.header-nav {
position: relative;
display: flex;
flex-direction: column;
align-items: flex-end;
width: 200px;
justify-content: space-between;
height: 100%;
gap: 10px;
flex-shrink: 0;
}
.header-actions {
position: relative;
display: flex;
align-items: center;
gap: 10px;
}
.header-tabs {
display: flex;
align-items: flex-end;
gap: 0;
}
.header-tab {
display: flex;
align-items: center;
padding: 8px 20px;
color: var(--text-muted);
text-decoration: none;
font-size: 15px;
font-weight: 500;
border-bottom: 3px solid transparent;
transition: color 0.2s ease, border-color 0.2s ease;
white-space: nowrap;
}
.header-tab:hover {
color: var(--text-primary);
}
.header-tab.active {
color: var(--btn-primary);
border-bottom-color: var(--btn-primary);
}
.menu-button {
background-color: var(--bg-secondary);
color: var(--text-primary);
@@ -240,7 +275,8 @@ body {
.menu-dropdown {
position: absolute;
top: calc(var(--header-height) + 10px);
top: 100%;
margin-top: 6px;
right: 0;
background-color: var(--bg-secondary);
border: 1px solid var(--border-color);
@@ -966,6 +1002,14 @@ body {
.header-nav {
width: 100%;
align-items: center;
}
.header-tabs {
justify-content: center;
}
.header-actions {
justify-content: center;
}
src/serena/resources/dashboard/index.html
+18 -14
View File
@@ -30,20 +30,24 @@
</div>
<nav class="header-nav">
<button id="theme-toggle" class="theme-toggle">
<span id="theme-icon" style="height: 21px">🌙</span>
<span id="theme-text">Dark</span>
</button>
<button id="menu-toggle" class="menu-button">
<span>☰</span>
<span>Menu</span>
</button>
<div id="menu-dropdown" class="menu-dropdown" style="display:none">
<a href="#" data-page="overview" class="active">Overview</a>
<a href="#" data-page="logs">Logs</a>
<a href="#" data-page="stats">Advanced Stats</a>
<hr>
<a href="#" id="menu-shutdown">Shutdown Server</a>
<div class="header-actions">
<button id="theme-toggle" class="theme-toggle">
<span id="theme-icon" style="height: 21px">🌙</span>
<span id="theme-text">Dark</span>
</button>
<button id="menu-toggle" class="menu-button">
<span>☰</span>
<span>Menu</span>
</button>
<div id="menu-dropdown" class="menu-dropdown" style="display:none">
<a href="#" data-page="stats">Advanced Stats</a>
<hr>
<a href="#" id="menu-shutdown">Shutdown Server</a>
</div>
</div>
<div class="header-tabs">
<a href="#" data-page="overview" class="header-tab active">Overview</a>
<a href="#" data-page="logs" class="header-tab">Logs</a>
</div>
</nav>
</header>
src/serena/resources/project.template.yml
+7 -6
View File
@@ -8,10 +8,10 @@ project_name: "project_name"
# fortran fsharp go groovy haskell
# java julia kotlin lua markdown
# matlab nix pascal perl php
# powershell python python_jedi r rego
# ruby ruby_solargraph rust scala swift
# terraform toml typescript typescript_vts vue
# yaml zig
# php_phpactor powershell python python_jedi r
# rego ruby ruby_solargraph rust scala
# swift terraform toml typescript typescript_vts
# vue yaml zig
# (This list may be outdated. For the current list, see values of Language enum here:
# https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py
# For some languages, there are alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
@@ -34,8 +34,9 @@ encoding: "utf-8"
# whether to use project's .gitignore files to ignore files
ignore_all_files_in_gitignore: true
# list of additional paths to ignore in all projects
# same syntax as gitignore, so you can use * and **
# list of additional paths to ignore in this project.
# Same syntax as gitignore, so you can use * and **.
# Note: global ignored_paths from serena_config.yml are also applied additively.
ignored_paths: []
# whether the project is in read-only mode
src/serena/resources/serena_config.template.yml
+5
View File
@@ -63,6 +63,11 @@ trace_lsp_communication: False
# No documentation on options means no options are available.
ls_specific_settings: {}
# list of paths to ignore across all projects.
# Same syntax as gitignore, so you can use * and **.
# These patterns are merged additively with each project's own ignored_paths.
ignored_paths: []
# timeout, in seconds, after which tool executions are terminated
tool_timeout: 240
src/serena/symbol.py
+190 -51
View File
@@ -5,11 +5,11 @@ from abc import ABC, abstractmethod
from collections.abc import Callable, Iterator, Sequence
from dataclasses import asdict, dataclass
from time import perf_counter
from typing import TYPE_CHECKING, Any, Self, Union
from typing import TYPE_CHECKING, Any, Generic, Literal, NotRequired, Self, TypedDict, TypeVar, Union
from sensai.util.string import ToStringMixin
import serena.tools.jetbrains_types as jb
import serena.jetbrains.jetbrains_types as jb
from solidlsp import SolidLanguageServer
from solidlsp.ls import ReferenceInSymbol as LSPReferenceInSymbol
from solidlsp.ls_types import Position, SymbolKind, UnifiedSymbolInformation
@@ -213,14 +213,17 @@ class LanguageServerSymbol(Symbol, ToStringMixin):
return []
def _tostring_additional_entries(self) -> dict[str, Any]:
return dict(name=self.name, kind=self.kind, num_children=len(self.symbol_root["children"]))
return dict(name=self.name, kind=self.symbol_kind_name, num_children=len(self.symbol_root["children"]))
@property
def name(self) -> str:
return self.symbol_root["name"]
@property
def kind(self) -> str:
def symbol_kind_name(self) -> str:
"""
:return: string representation of the symbol kind (name attribute of the `SymbolKind` enum item)
"""
return SymbolKind(self.symbol_kind).name
@property
@@ -396,67 +399,99 @@ class LanguageServerSymbol(Symbol, ToStringMixin):
traverse(self)
return result
class OutputDict(TypedDict):
name_path: NotRequired[str]
name: NotRequired[str]
location: NotRequired[dict[str, Any]]
relative_path: NotRequired[str | None]
body_location: NotRequired[dict[str, Any]]
body: NotRequired[str | None]
kind: NotRequired[str]
"""
string representation of the symbol kind (name attribute of the `SymbolKind` enum item)
"""
children: NotRequired[list["LanguageServerSymbol.OutputDict"]]
OutputDictKey = Literal["name", "name_path", "relative_path", "location", "body_location", "body", "kind", "children"]
def to_dict(
self,
*,
name_path: bool = True,
name: bool = False,
kind: bool = False,
location: bool = False,
depth: int = 0,
include_body: bool = False,
include_children_body: bool = False,
include_relative_path: bool = True,
body: bool = False,
body_location: bool = False,
children_body: bool = False,
relative_path: bool = False,
child_inclusion_predicate: Callable[[Self], bool] | None = None,
) -> dict[str, Any]:
) -> OutputDict:
"""
Converts the symbol to a dictionary.
:param name_path: whether to include the name path of the symbol
:param name: whether to include the name of the symbol
:param kind: whether to include the kind of the symbol
:param location: whether to include the location of the symbol
:param depth: the depth up to which to include child symbols (0 = do not include children)
:param include_body: whether to include the body of the top-level symbol.
:param include_children_body: whether to also include the body of the children.
:param body: whether to include the body of the top-level symbol.
:param children_body: whether to also include the body of the children.
Note that the body of the children is part of the body of the parent symbol,
so there is usually no need to set this to True unless you want process the output
and pass the children without passing the parent body to the LM.
:param include_relative_path: whether to include the relative path of the symbol in the location
entry. Relative paths of the symbol's children are always excluded.
:param relative_path: whether to include the relative path of the symbol.
If `location` is True, this defines whether to include the path in the location entry.
If `location` is False, this defines whether to include the relative path as a top-level entry.
Relative paths of the symbol's children are always excluded.
:param child_inclusion_predicate: an optional predicate that decides whether a child symbol
should be included.
:return: a dictionary representation of the symbol
"""
result: dict[str, Any] = {"name": self.name, "name_path": self.get_name_path()}
result: LanguageServerSymbol.OutputDict = {}
if name_path:
result["name_path"] = self.get_name_path()
if name:
result["name"] = self.name
if kind:
result["kind"] = self.kind
result["kind"] = self.symbol_kind_name
if location:
result["location"] = self.location.to_dict(include_relative_path=include_relative_path)
result["location"] = self.location.to_dict(include_relative_path=relative_path)
elif relative_path:
result["relative_path"] = self.relative_path
if body_location:
body_start_line, body_end_line = self.get_body_line_numbers()
result["body_location"] = {"start_line": body_start_line, "end_line": body_end_line}
if include_body:
body = self.body
if body is None:
log.warning("Requested body for symbol, but it is not present. The symbol might have been loaded with include_body=False.")
result["body"] = body
if body:
result["body"] = self.body
if child_inclusion_predicate is None:
child_inclusion_predicate = lambda s: True
def included_children(s: Self) -> list[dict[str, Any]]:
def included_children(s: Self) -> list[LanguageServerSymbol.OutputDict]:
children = []
for c in s.iter_children():
if not child_inclusion_predicate(c):
continue
children.append(
c.to_dict(
name_path=name_path,
name=name,
kind=kind,
location=location,
body_location=body_location,
depth=depth - 1,
child_inclusion_predicate=child_inclusion_predicate,
include_body=include_children_body,
include_children_body=include_children_body,
body=children_body,
children_body=children_body,
# all children have the same relative path as the parent
include_relative_path=False,
relative_path=False,
)
)
return children
@@ -464,7 +499,7 @@ class LanguageServerSymbol(Symbol, ToStringMixin):
if depth > 0:
children = included_children(self)
if len(children) > 0:
result["children"] = included_children(self)
result["children"] = children
return result
@@ -716,8 +751,7 @@ class LanguageServerSymbolRetriever:
include_rel_path = within_relative_path is not None
raise ValueError(
f"Found multiple {len(symbol_candidates)} symbols matching '{name_path_pattern}'. "
"They are: \n"
+ json.dumps([s.to_dict(kind=True, include_relative_path=include_rel_path) for s in symbol_candidates], indent=2)
"They are: \n" + json.dumps([s.to_dict(kind=True, relative_path=include_rel_path) for s in symbol_candidates], indent=2)
)
def find_by_location(self, location: LanguageServerSymbolLocation) -> LanguageServerSymbol | None:
@@ -801,36 +835,15 @@ class LanguageServerSymbolRetriever:
return [ReferenceInLanguageServerSymbol.from_lsp_reference(r) for r in references]
def get_symbol_overview(self, relative_path: str, depth: int = 0) -> dict[str, list[dict]]:
def get_symbol_overview(self, relative_path: str) -> dict[str, list[LanguageServerSymbol]]:
"""
:param relative_path: the path of the file or directory for which to get the symbol overview
:param depth: the depth up to which to include child symbols (0 = only top-level symbols)
:return: a mapping from file paths to lists of symbol dictionaries.
:return: a mapping from file paths to lists of symbols.
For the case where a file is passed, the mapping will contain a single entry.
"""
lang_server = self.get_language_server(relative_path)
path_to_unified_symbols = lang_server.request_overview(relative_path)
def child_inclusion_predicate(s: LanguageServerSymbol) -> bool:
return not s.is_low_level()
result = {}
for file_path, unified_symbols in path_to_unified_symbols.items():
symbols_in_file = []
for us in unified_symbols:
symbol = LanguageServerSymbol(us)
symbols_in_file.append(
symbol.to_dict(
depth=depth,
kind=True,
include_relative_path=False,
location=False,
child_inclusion_predicate=child_inclusion_predicate,
)
)
result[file_path] = symbols_in_file
return result
return {k: [LanguageServerSymbol(us) for us in v] for k, v in path_to_unified_symbols.items()}
class JetBrainsSymbol(Symbol):
@@ -887,3 +900,129 @@ class JetBrainsSymbol(Symbol):
def is_neighbouring_definition_separated_by_empty_line(self) -> bool:
# NOTE: Symbol types cannot really be differentiated, because types are not handled in a language-agnostic way.
return False
TSymbolDict = TypeVar("TSymbolDict")
GroupedSymbolDict = dict[str, list[dict] | dict[str, dict]]
class SymbolDictGrouper(Generic[TSymbolDict], ABC):
"""
A utility class for grouping a list of symbol dictionaries by one or more specified keys.
If an instance is statically initialised (upon module import), then this establishes a guarantee
that the specified keys are defined in the symbol dictionary type, ensuring at least basic type safety.
The respective ValueError will immediately be apparent.
"""
def __init__(
self,
symbol_dict_type: type[TSymbolDict],
children_key: Any,
group_keys: list[Any],
group_children_keys: list[Any],
collapse_singleton: bool,
) -> None:
"""
:param symbol_dict_type: the TypedDict type that represents the type of the symbol dictionaries to be grouped
:param children_key: the key in the symbol dictionaries that contains the list of child symbols (for recursive grouping).
:param group_keys: keys by which to group the symbol dictionaries. Must be a subset of the keys of `symbol_dict_type`.
:param group_children_keys: keys by which to group the child symbol dictionaries. Must be a subset of the keys of `symbol_dict_type`.
:param collapse_singleton: whether to collapse dictionaries containing a single entry after regrouping to just the entry's value
"""
# check whether the type contains all the keys specified in `keys` and raise an error if not.
if not hasattr(symbol_dict_type, "__annotations__"):
raise ValueError(f"symbol_dict_type must be a TypedDict type, got {symbol_dict_type}")
symbol_dict_keys = set(symbol_dict_type.__annotations__.keys())
for key in group_keys + [children_key] + group_children_keys:
if key not in symbol_dict_keys:
raise ValueError(f"symbol_dict_type {symbol_dict_type} does not contain key '{key}'")
self._children_key = children_key
self._group_keys = group_keys
self._group_children_keys = group_children_keys
self._collapse_singleton = collapse_singleton
def _group_by(self, l: list[dict], keys: list[str], children_keys: list[str]) -> dict[str, Any]:
assert len(keys) > 0, "keys must not be empty"
# group by the first key
grouped: dict[str, Any] = {}
for item in l:
key_value = item.pop(keys[0], "unknown")
if key_value not in grouped:
grouped[key_value] = []
grouped[key_value].append(item)
if len(keys) > 1:
# continue grouping by the remaining keys
for k, group in grouped.items():
grouped[k] = self._group_by(group, keys[1:], children_keys)
else:
# grouping is complete; now group the children if necessary
if children_keys:
for k, group in grouped.items():
for item in group:
if self._children_key in item:
children = item[self._children_key]
item[self._children_key] = self._group_by(children, children_keys, children_keys)
# post-process final group items
grouped = {k: [self._transform_item(i) for i in v] for k, v in grouped.items()}
return grouped
def _transform_item(self, item: dict) -> dict:
"""
Post-processes a final group item (which has been regrouped, i.e. some keys may have been removed),
collapsing singleton items (and items containing only a single non-children key)
"""
if self._collapse_singleton:
if len(item) == 1:
# {"name": "foo"} -> "foo"
# if there is only a single entry, collapse the dictionary to just the value of that entry
return next(iter(item.values()))
elif len(item) == 2 and self._children_key in item:
# {"name": "foo", "children": {...}} -> {"foo": {...}}
# if there are exactly two entries and one of them is the children key,
# convert to {other_value: children}
other_key = next(k for k in item.keys() if k != self._children_key)
new_item = {item[other_key]: item[self._children_key]}
return new_item
return item
def group(self, symbols: list[TSymbolDict]) -> GroupedSymbolDict:
"""
:param symbols: the symbols to group
:return: dictionary with the symbols grouped as defined at construction
"""
return self._group_by(symbols, self._group_keys, self._group_children_keys) # type: ignore
class LanguageServerSymbolDictGrouper(SymbolDictGrouper[LanguageServerSymbol.OutputDict]):
def __init__(
self,
group_keys: list[LanguageServerSymbol.OutputDictKey],
group_children_keys: list[LanguageServerSymbol.OutputDictKey],
collapse_singleton: bool = False,
) -> None:
super().__init__(LanguageServerSymbol.OutputDict, "children", group_keys, group_children_keys, collapse_singleton)
class JetBrainsSymbolDictGrouper(SymbolDictGrouper[jb.SymbolDTO]):
def __init__(
self,
group_keys: list[jb.SymbolDTOKey],
group_children_keys: list[jb.SymbolDTOKey],
collapse_singleton: bool = False,
map_name_path_to_name: bool = False,
) -> None:
super().__init__(jb.SymbolDTO, "children", group_keys, group_children_keys, collapse_singleton)
self._map_name_path_to_name = map_name_path_to_name
def _transform_item(self, item: dict) -> dict:
if self._map_name_path_to_name:
# {"name_path: "Class/myMethod"} -> {"name: "myMethod"}
new_item = dict(item)
if "name_path" in item:
name_path = new_item.pop("name_path")
new_item["name"] = name_path.split("/")[-1]
return super()._transform_item(new_item)
else:
return super()._transform_item(item)
src/serena/tools/jetbrains_tools.py
+12 -38
View File
@@ -1,10 +1,10 @@
import logging
from collections import defaultdict
from typing import Any, Literal
import serena.tools.jetbrains_types as jb
import serena.jetbrains.jetbrains_types as jb
from serena.jetbrains.jetbrains_plugin_client import JetBrainsPluginClient
from serena.symbol import JetBrainsSymbolDictGrouper
from serena.tools import Tool, ToolMarkerOptional, ToolMarkerSymbolicRead
from serena.tools.jetbrains_plugin_client import JetBrainsPluginClient
log = logging.getLogger(__name__)
@@ -91,6 +91,8 @@ class JetBrainsFindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead, ToolMark
Finds symbols that reference the given symbol using the JetBrains backend
"""
symbol_dict_grouper = JetBrainsSymbolDictGrouper(["relative_path", "type"], ["type"], collapse_singleton=True)
# TODO: (maybe) - add content snippets showing the references like in LS based version?
def apply(
self,
@@ -116,10 +118,12 @@ class JetBrainsFindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead, ToolMark
response_dict = client.find_references(
name_path=name_path,
relative_path=relative_path,
include_quick_info=include_info,
include_quick_info=False, # TODO: Hotfix for serena-jetbrains-plugin/issues/13; revert once fixed
)
result = self._to_json(response_dict)
return self._limit_length(result, max_answer_chars)
symbol_dicts = response_dict["symbols"]
result = self.symbol_dict_grouper.group(symbol_dicts)
result_json = self._to_json(result)
return self._limit_length(result_json, max_answer_chars)
class JetBrainsGetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead, ToolMarkerOptional):
@@ -128,37 +132,7 @@ class JetBrainsGetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead, ToolMarkerOp
"""
USE_COMPACT_FORMAT = True
@staticmethod
def _transform_symbols_to_compact_format(symbols: list[jb.SymbolDTO]) -> dict[str, list]:
"""
Transform symbol overview from verbose format to compact grouped format.
Groups symbols by kind and uses names instead of full symbol objects.
For symbols with children, creates nested dictionaries.
The name_path can be inferred from the hierarchical structure:
- Top-level symbols: name_path = name
- Nested symbols: name_path = parent_name + "/" + name
For example, "convert" under class "ProjectType" has name_path "ProjectType/convert".
"""
result = defaultdict(list)
for symbol in symbols:
kind = symbol.get("type", "Unknown")
name_path = symbol["name_path"]
name = name_path.split("/")[-1]
children = symbol.get("children", [])
if children:
# Symbol has children: create nested dict {name: children_dict}
children_dict = JetBrainsGetSymbolsOverviewTool._transform_symbols_to_compact_format(children)
result[kind].append({name: children_dict})
else:
# Symbol has no children: just add the name
result[kind].append(name) # type: ignore
return result
symbol_dict_grouper = JetBrainsSymbolDictGrouper(["type"], ["type"], collapse_singleton=True, map_name_path_to_name=True)
def apply(
self,
@@ -187,7 +161,7 @@ class JetBrainsGetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead, ToolMarkerOp
)
if self.USE_COMPACT_FORMAT:
symbols = symbol_overview["symbols"]
result: dict[str, Any] = {"symbols": self._transform_symbols_to_compact_format(symbols)}
result: dict[str, Any] = {"symbols": self.symbol_dict_grouper.group(symbols)}
documentation = symbol_overview.pop("documentation", None)
if documentation:
result["docstring"] = documentation
src/serena/tools/symbol_tools.py
+32 -54
View File
@@ -3,11 +3,9 @@ Language server-related tools
"""
import os
from collections import defaultdict
from collections.abc import Sequence
from copy import copy
from typing import Any
from serena.symbol import LanguageServerSymbol, LanguageServerSymbolDictGrouper
from serena.tools import (
SUCCESS_RESULT,
Tool,
@@ -18,22 +16,6 @@ from serena.tools.tools_base import ToolMarkerOptional
from solidlsp.ls_types import SymbolKind
def _sanitize_symbol_dict(symbol_dict: dict[str, Any]) -> dict[str, Any]:
"""
Sanitize a symbol dictionary inplace by removing unnecessary information.
"""
# We replace the location entry, which repeats line information already included in body_location
# and has unnecessary information on column, by just the relative path.
symbol_dict = copy(symbol_dict)
s_relative_path = symbol_dict.get("location", {}).get("relative_path")
if s_relative_path is not None:
symbol_dict["relative_path"] = s_relative_path
symbol_dict.pop("location", None)
# also remove name, name_path should be enough
symbol_dict.pop("name")
return symbol_dict
class RestartLanguageServerTool(Tool, ToolMarkerOptional):
"""Restarts the language server, may be necessary when edits not through Serena happen."""
@@ -50,6 +32,8 @@ class GetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead):
Gets an overview of the top-level symbols defined in a given file.
"""
symbol_dict_grouper = LanguageServerSymbolDictGrouper(["kind"], ["kind"], collapse_singleton=True)
def apply(self, relative_path: str, depth: int = 0, max_answer_chars: int = -1) -> str:
"""
Use this tool to get a high-level understanding of the code symbols in a file.
@@ -65,11 +49,11 @@ class GetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead):
:return: a JSON object containing symbols grouped by kind in a compact format.
"""
result = self.get_symbol_overview(relative_path, depth=depth)
compact_result = self._transform_symbols_to_compact_format(result)
compact_result = self.symbol_dict_grouper.group(result)
result_json_str = self._to_json(compact_result)
return self._limit_length(result_json_str, max_answer_chars)
def get_symbol_overview(self, relative_path: str, depth: int = 0) -> list[dict]:
def get_symbol_overview(self, relative_path: str, depth: int = 0) -> list[LanguageServerSymbol.OutputDict]:
"""
:param relative_path: relative path to a source file
:param depth: the depth up to which descendants shall be retrieved
@@ -85,37 +69,25 @@ class GetSymbolsOverviewTool(Tool, ToolMarkerSymbolicRead):
if os.path.isdir(file_path):
raise ValueError(f"Expected a file path, but got a directory path: {relative_path}. ")
return symbol_retriever.get_symbol_overview(relative_path, depth=depth)[relative_path]
symbols = symbol_retriever.get_symbol_overview(relative_path)[relative_path]
@staticmethod
def _transform_symbols_to_compact_format(symbols: list[dict[str, Any]]) -> dict[str, list]:
"""
Transform symbol overview from verbose format to compact grouped format.
Groups symbols by kind and uses names instead of full symbol objects.
For symbols with children, creates nested dictionaries.
The name_path can be inferred from the hierarchical structure:
- Top-level symbols: name_path = name
- Nested symbols: name_path = parent_name + "/" + name
For example, "convert" under class "ProjectType" has name_path "ProjectType/convert".
"""
result = defaultdict(list)
def child_inclusion_predicate(s: LanguageServerSymbol) -> bool:
return not s.is_low_level()
symbol_dicts = []
for symbol in symbols:
kind = symbol.get("kind", "Unknown")
name = symbol.get("name", "unknown")
children = symbol.get("children", [])
if children:
# Symbol has children: create nested dict {name: children_dict}
children_dict = GetSymbolsOverviewTool._transform_symbols_to_compact_format(children)
result[kind].append({name: children_dict})
else:
# Symbol has no children: just add the name
result[kind].append(name)
return result
symbol_dicts.append(
symbol.to_dict(
name_path=False,
name=True,
depth=depth,
kind=True,
relative_path=False,
location=False,
child_inclusion_predicate=child_inclusion_predicate,
)
)
return symbol_dicts
class FindSymbolTool(Tool, ToolMarkerSymbolicRead):
@@ -185,7 +157,7 @@ class FindSymbolTool(Tool, ToolMarkerSymbolicRead):
substring_matching=substring_matching,
within_relative_path=relative_path,
)
symbol_dicts = [_sanitize_symbol_dict(s.to_dict(kind=True, location=True, depth=depth, include_body=include_body)) for s in symbols]
symbol_dicts = [dict(s.to_dict(kind=True, relative_path=True, body_location=True, depth=depth, body=include_body)) for s in symbols]
if not include_body and include_info:
info_by_symbol = symbol_retriever.request_info_for_symbol_batch(symbols)
for s, s_dict in zip(symbols, symbol_dicts, strict=True):
@@ -201,6 +173,8 @@ class FindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead):
Finds symbols that reference the given symbol using the language server backend
"""
symbol_dict_grouper = LanguageServerSymbolDictGrouper(["relative_path", "kind"], ["kind"], collapse_singleton=True)
# noinspection PyDefaultArgument
def apply(
self,
@@ -229,6 +203,7 @@ class FindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead):
parsed_include_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in include_kinds] if include_kinds else None
parsed_exclude_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in exclude_kinds] if exclude_kinds else None
symbol_retriever = self.create_language_server_symbol_retriever()
references_in_symbols = symbol_retriever.find_referencing_symbols(
name_path,
relative_file_path=relative_path,
@@ -245,8 +220,8 @@ class FindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead):
reference_dicts = []
for ref in references_in_symbols:
ref_dict = ref.symbol.to_dict(kind=True, location=True, depth=0, include_body=include_body)
ref_dict = _sanitize_symbol_dict(ref_dict)
ref_dict_orig = ref.symbol.to_dict(kind=True, relative_path=True, depth=0, body=include_body, body_location=True)
ref_dict = dict(ref_dict_orig)
if not include_body:
ref_relative_path = ref.symbol.location.relative_path
assert ref_relative_path is not None, f"Referencing symbol {ref.symbol.name} has no relative path, this is likely a bug."
@@ -257,8 +232,11 @@ class FindReferencingSymbolsTool(Tool, ToolMarkerSymbolicRead):
)
ref_dict["content_around_reference"] = content_around_ref.to_display_string()
reference_dicts.append(ref_dict)
result = self._to_json(reference_dicts)
return self._limit_length(result, max_answer_chars)
result = self.symbol_dict_grouper.group(reference_dicts) # type: ignore
result_json = self._to_json(result)
return self._limit_length(result_json, max_answer_chars)
class ReplaceSymbolBodyTool(Tool, ToolMarkerSymbolicEdit):
src/serena/tools/workflow_tools.py
+3 -3
View File
@@ -53,7 +53,7 @@ class OnboardingTool(Tool):
return self.prompt_factory.create_onboarding_prompt(system=system)
class ThinkAboutCollectedInformationTool(Tool):
class ThinkAboutCollectedInformationTool(Tool, ToolMarkerOptional):
"""
Thinking tool for pondering the completeness of collected information.
"""
@@ -67,7 +67,7 @@ class ThinkAboutCollectedInformationTool(Tool):
return self.prompt_factory.create_think_about_collected_information()
class ThinkAboutTaskAdherenceTool(Tool):
class ThinkAboutTaskAdherenceTool(Tool, ToolMarkerOptional):
"""
Thinking tool for determining whether the agent is still on track with the current task.
"""
@@ -83,7 +83,7 @@ class ThinkAboutTaskAdherenceTool(Tool):
return self.prompt_factory.create_think_about_task_adherence()
class ThinkAboutWhetherYouAreDoneTool(Tool):
class ThinkAboutWhetherYouAreDoneTool(Tool, ToolMarkerOptional):
"""
Thinking tool for determining whether the task is truly completed.
"""
src/serena/util/logging.py
+44 -9
View File
@@ -1,20 +1,33 @@
import queue
import threading
from collections.abc import Callable
from dataclasses import dataclass
from typing import Optional
from sensai.util import logging
from serena.constants import SERENA_LOG_FORMAT
from serena.constants import LOG_MESSAGES_BUFFER_SIZE, SERENA_LOG_FORMAT
lg = logging
@dataclass
class LogMessages:
messages: list[str]
"""
the list of log messages, ordered from oldest to newest
"""
max_idx: int
"""
the 0-based index of the last message in `messages` (in the full log history)
"""
class MemoryLogHandler(logging.Handler):
def __init__(self, level: int = logging.NOTSET) -> None:
def __init__(self, level: int = logging.NOTSET, max_messages: int | None = LOG_MESSAGES_BUFFER_SIZE) -> None:
super().__init__(level=level)
self.setFormatter(logging.Formatter(SERENA_LOG_FORMAT))
self._log_buffer = LogBuffer()
self._log_buffer = LogBuffer(max_messages=max_messages)
self._log_queue: queue.Queue[str] = queue.Queue()
self._stop_event = threading.Event()
self._emit_callbacks: list[Callable[[str], None]] = []
@@ -48,26 +61,48 @@ class MemoryLogHandler(logging.Handler):
except queue.Empty:
continue
def get_log_messages(self) -> list[str]:
return self._log_buffer.get_log_messages()
def get_log_messages(self, from_idx: int = 0) -> LogMessages:
return self._log_buffer.get_log_messages(from_idx=from_idx)
class LogBuffer:
"""
A thread-safe buffer for storing log messages.
A thread-safe buffer for storing (an optionally limited number of) log messages.
"""
def __init__(self) -> None:
def __init__(self, max_messages: int | None = None) -> None:
self._max_messages = max_messages
self._log_messages: list[str] = []
self._lock = threading.Lock()
self._max_idx = -1
"""
the 0-based index of the most recently added log message
"""
def append(self, msg: str) -> None:
with self._lock:
self._log_messages.append(msg)
self._max_idx += 1
if self._max_messages is not None and len(self._log_messages) > self._max_messages:
excess = len(self._log_messages) - self._max_messages
self._log_messages = self._log_messages[excess:]
def get_log_messages(self) -> list[str]:
def get_log_messages(self, from_idx: int = 0) -> LogMessages:
"""
:param from_idx: the 0-based index of the first log message to return.
If from_idx is less than or equal to the index of the oldest message in the buffer,
then all messages in the buffer will be returned.
:return: the list of messages
"""
from_idx = max(from_idx, 0)
with self._lock:
return self._log_messages.copy()
first_stored_idx = self._max_idx - len(self._log_messages) + 1
if from_idx <= first_stored_idx:
messages = self._log_messages.copy()
else:
start_idx = from_idx - first_stored_idx
messages = self._log_messages[start_idx:].copy()
return LogMessages(messages=messages, max_idx=self._max_idx)
class SuspendedLoggersContext:
src/solidlsp/language_servers/bash_language_server.py
+52 -43
View File
@@ -10,10 +10,15 @@ import shutil
import threading
from solidlsp.language_servers.common import RuntimeDependency, RuntimeDependencyCollection
from solidlsp.ls import DocumentSymbols, LSPFileBuffer, SolidLanguageServer
from solidlsp.ls import (
DocumentSymbols,
LanguageServerDependencyProvider,
LanguageServerDependencyProviderSinglePath,
LSPFileBuffer,
SolidLanguageServer,
)
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
@@ -30,57 +35,63 @@ class BashLanguageServer(SolidLanguageServer):
Creates a BashLanguageServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
bash_lsp_executable_path = self._setup_runtime_dependencies(config, solidlsp_settings)
super().__init__(
config,
repository_root_path,
ProcessLaunchInfo(cmd=bash_lsp_executable_path, cwd=repository_root_path),
None,
"bash",
solidlsp_settings,
)
self.server_ready = threading.Event()
self.initialize_searcher_command_available = threading.Event()
@classmethod
def _setup_runtime_dependencies(cls, config: LanguageServerConfig, solidlsp_settings: SolidLSPSettings) -> str:
"""
Setup runtime dependencies for Bash Language Server and return the command to start the server.
"""
# Verify both node and npm are installed
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
deps = RuntimeDependencyCollection(
[
RuntimeDependency(
id="bash-language-server",
description="bash-language-server package",
command="npm install --prefix ./ bash-language-server@5.6.0",
platform_id="any",
),
]
)
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
def _get_or_install_core_dependency(self) -> str:
"""
Setup runtime dependencies for Bash Language Server and return the command to start the server.
"""
# Verify both node and npm are installed
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
# Install bash-language-server if not already installed
bash_ls_dir = os.path.join(cls.ls_resources_dir(solidlsp_settings), "bash-lsp")
bash_executable_path = os.path.join(bash_ls_dir, "node_modules", ".bin", "bash-language-server")
# Handle Windows executable extension
if os.name == "nt":
bash_executable_path += ".cmd"
if not os.path.exists(bash_executable_path):
log.info(f"Bash Language Server executable not found at {bash_executable_path}. Installing...")
deps.install(bash_ls_dir)
log.info("Bash language server dependencies installed successfully")
if not os.path.exists(bash_executable_path):
raise FileNotFoundError(
f"bash-language-server executable not found at {bash_executable_path}, something went wrong with the installation."
deps = RuntimeDependencyCollection(
[
RuntimeDependency(
id="bash-language-server",
description="bash-language-server package",
command="npm install --prefix ./ bash-language-server@5.6.0",
platform_id="any",
),
]
)
return f"{bash_executable_path} start"
# Install bash-language-server if not already installed
bash_ls_dir = os.path.join(self._ls_resources_dir, "bash-lsp")
bash_executable_path = os.path.join(bash_ls_dir, "node_modules", ".bin", "bash-language-server")
# Handle Windows executable extension
if os.name == "nt":
bash_executable_path += ".cmd"
if not os.path.exists(bash_executable_path):
log.info(f"Bash Language Server executable not found at {bash_executable_path}. Installing...")
deps.install(bash_ls_dir)
log.info("Bash language server dependencies installed successfully")
if not os.path.exists(bash_executable_path):
raise FileNotFoundError(
f"bash-language-server executable not found at {bash_executable_path}, something went wrong with the installation."
)
return bash_executable_path
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path, "start"]
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -148,7 +159,6 @@ class BashLanguageServer(SolidLanguageServer):
if "Analyzing" in message_text or "analysis complete" in message_text.lower():
log.info("Bash language server analysis signals detected")
self.server_ready.set()
self.completions_available.set()
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
@@ -183,7 +193,6 @@ class BashLanguageServer(SolidLanguageServer):
# This is common. bash-language-server doesn't always send explicit ready signals. Log as info
log.info("Timeout waiting for bash server ready signal, proceeding anyway")
self.server_ready.set()
self.completions_available.set()
else:
log.info("Bash server initialization complete")
src/solidlsp/language_servers/ccls_language_server.py
+154
View File
@@ -0,0 +1,154 @@
"""
This is an alternative to clangd for large C++ codebases where ccls may perform
better for indexing and navigation. Requires ccls to be installed and available
on PATH, or configured via ls_specific_settings with key "ls_path".
Installation
------------
ccls must be installed manually as there are no prebuilt binaries available for
direct download. Install using your system package manager:
**Linux:**
- Ubuntu/Debian (22.04+): ``sudo apt-get install ccls``
- Fedora/RHEL: ``sudo dnf install ccls``
- Arch Linux: ``sudo pacman -S ccls``
- openSUSE Tumbleweed: ``sudo zypper install ccls``
- Gentoo: ``sudo emerge dev-util/ccls``
**macOS:**
- Homebrew: ``brew install ccls``
**Windows:**
- Chocolatey: ``choco install ccls``
For alternative installation methods and build-from-source instructions, see:
https://github.com/MaskRay/ccls/wiki/Build
Official documentation:
https://github.com/MaskRay/ccls
"""
import logging
import os
import pathlib
import threading
from typing import Any, cast
from solidlsp.ls import (
LanguageServerDependencyProvider,
LanguageServerDependencyProviderSinglePath,
SolidLanguageServer,
)
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
class CCLS(SolidLanguageServer):
"""
C/C++ language server implementation using ccls.
Notes:
- ccls should be installed and on PATH (or specify ls_path in settings)
- compile_commands.json at repo root is recommended for accurate indexing
"""
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
"""
Creates a CclsLanguageServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
super().__init__(config, repository_root_path, None, "cpp", solidlsp_settings)
self.server_ready = threading.Event()
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
def _get_or_install_core_dependency(self) -> str:
"""
Resolve ccls path from system or raise helpful error if missing.
Allows override via ls_specific_settings[language].ls_path.
"""
import shutil
ccls_path = shutil.which("ccls")
if not ccls_path:
raise FileNotFoundError(
"ccls is not installed on your system.\n"
"Please install ccls using your system package manager:\n"
" Linux (Ubuntu/Debian): sudo apt-get install ccls\n"
" Linux (Fedora/RHEL): sudo dnf install ccls\n"
" Linux (Arch): sudo pacman -S ccls\n"
" macOS (Homebrew): brew install ccls\n"
" Windows: choco install ccls\n\n"
"For build instructions and more details, see:\n"
" https://github.com/MaskRay/ccls/wiki/Build"
)
log.info(f"Using system-installed ccls at {ccls_path}")
return ccls_path
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path]
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the ccls Language Server.
"""
root_uri = pathlib.Path(repository_absolute_path).as_uri()
initialize_params = {
"locale": "en",
"capabilities": {
"textDocument": {
"synchronization": {"didSave": True, "dynamicRegistration": True},
"completion": {"dynamicRegistration": True, "completionItem": {"snippetSupport": True}},
"definition": {"dynamicRegistration": True},
"references": {"dynamicRegistration": True},
"documentSymbol": {"dynamicRegistration": True},
},
"workspace": {"workspaceFolders": True, "didChangeConfiguration": {"dynamicRegistration": True}},
},
"processId": os.getpid(),
"rootPath": repository_absolute_path,
"rootUri": root_uri,
"workspaceFolders": [
{
"uri": root_uri,
"name": "$name",
}
],
# ccls supports initializationOptions but none are required for basic functionality
}
return cast(InitializeParams, initialize_params)
def _start_server(self) -> None:
"""
Starts the ccls language server and initializes the LSP connection.
"""
def do_nothing(params: Any) -> None:
pass
def window_log_message(msg: dict) -> None:
log.info(f"LSP: window/logMessage: {msg}")
# Register minimal handlers
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
log.info("Starting ccls server process")
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
log.info("Sending initialize request from LSP client to ccls and awaiting response")
self.server.send.initialize(initialize_params)
# Do not assert clangd-specific capability shapes; ccls differs
self.server.notify.initialized({})
# Basic readiness
self.server_ready.set()
src/solidlsp/language_servers/clangd_language_server.py
+99 -10
View File
@@ -1,14 +1,11 @@
"""
Provides C/C++ specific instantiation of the LanguageServer class. Contains various configurations and settings specific to C/C++.
"""
import json
import logging
import os
import pathlib
import threading
from typing import Any, cast
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, ProcessLaunchInfo, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.settings import SolidLSPSettings
@@ -35,6 +32,89 @@ class ClangdLanguageServer(SolidLanguageServer):
self.initialize_searcher_command_available = threading.Event()
self.resolve_main_method_available = threading.Event()
def _prepare_compile_commands(self) -> str | None:
"""
Prepare clangd compilation database with absolute directory paths.
Clangd requires absolute directory paths in compile_commands.json for correct
cross-file reference finding. This method reads the compile_commands.json,
converts relative directory paths to absolute paths, and writes a transformed
compilation database to the serena managed directory.
The transformed file is persisted in .serena/serena_compile_commands.json
(or a configurable directory via ls_specific_settings) and is not deleted
on cleanup. This allows clangd to use the absolute-path version without
modifying the user's original compile_commands.json.
Returns the path to the serena directory containing the transformed database,
or None if no transformation was needed.
"""
compile_db_path = os.path.join(self.repository_root_path, "compile_commands.json")
if not os.path.exists(compile_db_path):
# No compile_commands.json, nothing to do
return None
try:
with open(compile_db_path, encoding="utf-8") as f:
compile_commands = json.load(f)
if not compile_commands:
return None
# Check if any entries have relative directory paths
has_relative = False
for entry in compile_commands:
directory = entry.get("directory", "")
if directory and not os.path.isabs(directory):
has_relative = True
# Convert to absolute path
entry["directory"] = os.path.abspath(os.path.join(self.repository_root_path, directory))
if not has_relative:
# No relative paths found, no need to create transformed database
return None
# Get the target directory from ls_specific_settings, default to .serena
cpp_settings: dict[str, Any] = self._custom_settings or {}
compile_commands_rel_dir = cpp_settings.get("compile_commands_dir", ".serena")
compile_commands_dir = os.path.join(self.repository_root_path, compile_commands_rel_dir)
os.makedirs(compile_commands_dir, exist_ok=True)
# Write the transformed compile_commands.json
# clangd looks for compile_commands.json in the --compile-commands-dir
compile_commands_path = os.path.join(compile_commands_dir, "compile_commands.json")
with open(compile_commands_path, "w", encoding="utf-8") as f:
json.dump(compile_commands, f, indent=2)
# Track the directory for --compile-commands-dir
log.info(f"Created serena compilation database with absolute paths at {compile_commands_path}")
return compile_commands_dir
except (OSError, json.JSONDecodeError) as e:
log.warning(f"Failed to prepare compile_commands.json: {e}")
return None
def _create_process_launch_info(self) -> ProcessLaunchInfo:
"""
Override to add --compile-commands-dir argument if we created a serena compilation database.
"""
# First, ensure the serena compile commands database is prepared
compile_commands_dir = self._prepare_compile_commands()
# Get the default launch info from parent
launch_info = super()._create_process_launch_info()
# If we created a serena compilation database, add --compile-commands-dir to the command
if compile_commands_dir:
# Insert --compile-commands-dir after the executable path
cmd = launch_info.cmd
assert isinstance(cmd, list)
launch_info.cmd = [cmd[0], f"--compile-commands-dir={compile_commands_dir}"] + cmd[1:]
return launch_info
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
@@ -116,8 +196,10 @@ class ClangdLanguageServer(SolidLanguageServer):
os.chmod(clangd_executable_path, 0o755)
return clangd_executable_path
def _create_launch_command(self, core_path: str) -> list[str] | str:
return [core_path]
def _create_launch_command(self, core_path: str) -> list[str]:
# --background-index enables clangd to index all files in the project,
# which is required for finding cross-file references
return [core_path, "--background-index"]
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -132,6 +214,11 @@ class ClangdLanguageServer(SolidLanguageServer):
"synchronization": {"didSave": True, "dynamicRegistration": True},
"completion": {"dynamicRegistration": True, "completionItem": {"snippetSupport": True}},
"definition": {"dynamicRegistration": True},
"references": {"dynamicRegistration": True},
"documentSymbol": {
"dynamicRegistration": True,
"hierarchicalDocumentSymbolSupport": True,
},
},
"workspace": {"workspaceFolders": True, "didChangeConfiguration": {"dynamicRegistration": True}},
},
@@ -160,6 +247,7 @@ class ClangdLanguageServer(SolidLanguageServer):
await lsp.request_references(...)
# Shutdown the LanguageServer on exit from scope
# LanguageServer has been shutdown
```
"""
def register_capability_handler(params: Any) -> None:
@@ -213,8 +301,9 @@ class ClangdLanguageServer(SolidLanguageServer):
}
self.server.notify.initialized({})
self.completions_available.set()
# set ready flag
# set ready flag, clangd sends no meaningful notification when ready
# TODO This defeats the purpose of the event; we should wait for the server to actually be ready
self.server_ready.set()
# wait for server to be ready
self.server_ready.wait()
src/solidlsp/language_servers/clojure_lsp.py
+1 -2
View File
@@ -123,7 +123,7 @@ class ClojureLSP(SolidLanguageServer):
os.chmod(clojurelsp_executable_path, 0o755)
return clojurelsp_executable_path
def _create_launch_command(self, core_path: str) -> list[str] | str:
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path]
@staticmethod
@@ -214,4 +214,3 @@ class ClojureLSP(SolidLanguageServer):
self.server.notify.initialized({})
# after initialize, Clojure-lsp is ready to serve
self.server_ready.set()
self.completions_available.set()
src/solidlsp/language_servers/csharp_language_server.py
+263 -188
View File
@@ -1,26 +1,24 @@
"""
CSharp Language Server using Microsoft.CodeAnalysis.LanguageServer (Official Roslyn-based LSP server)
CSharp Language Server using Roslyn Language Server (Official Roslyn-based LSP server from NuGet.org)
"""
import json
import logging
import os
import platform
import shutil
import subprocess
import tarfile
import threading
import urllib.request
import zipfile
from collections.abc import Iterable
from pathlib import Path
from typing import Any, cast
from overrides import override
from solidlsp.ls import LanguageServerDependencyProvider, SolidLanguageServer
from solidlsp.ls import DocumentSymbols, LanguageServerDependencyProvider, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_exceptions import SolidLSPException
from solidlsp.ls_types import Hover, UnifiedSymbolInformation
from solidlsp.ls_utils import PathUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams, InitializeResult
from solidlsp.settings import SolidLSPSettings
@@ -33,111 +31,69 @@ log = logging.getLogger(__name__)
_RUNTIME_DEPENDENCIES = [
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for Windows (x64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.win-x64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for Windows (x64)",
package_name="roslyn-language-server.win-x64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.win-x64/5.5.0-2.26078.4",
platform_id="win-x64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/win-x64",
extract_path="tools/net10.0/win-x64",
),
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for Windows (ARM64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.win-arm64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for Windows (ARM64)",
package_name="roslyn-language-server.win-arm64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.win-arm64/5.5.0-2.26078.4",
platform_id="win-arm64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/win-arm64",
extract_path="tools/net10.0/win-arm64",
),
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for macOS (x64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.osx-x64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for macOS (x64)",
package_name="roslyn-language-server.osx-x64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.osx-x64/5.5.0-2.26078.4",
platform_id="osx-x64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/osx-x64",
extract_path="tools/net10.0/osx-x64",
),
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for macOS (ARM64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.osx-arm64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for macOS (ARM64)",
package_name="roslyn-language-server.osx-arm64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.osx-arm64/5.5.0-2.26078.4",
platform_id="osx-arm64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/osx-arm64",
extract_path="tools/net10.0/osx-arm64",
),
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for Linux (x64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.linux-x64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for Linux (x64)",
package_name="roslyn-language-server.linux-x64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.linux-x64/5.5.0-2.26078.4",
platform_id="linux-x64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/linux-x64",
extract_path="tools/net10.0/linux-x64",
),
RuntimeDependency(
id="CSharpLanguageServer",
description="Microsoft.CodeAnalysis.LanguageServer for Linux (ARM64)",
package_name="Microsoft.CodeAnalysis.LanguageServer.linux-arm64",
package_version="5.0.0-1.25329.6",
description="Roslyn Language Server for Linux (ARM64)",
package_name="roslyn-language-server.linux-arm64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.linux-arm64/5.5.0-2.26078.4",
platform_id="linux-arm64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/linux-arm64",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for Windows (x64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-win-x64.zip",
platform_id="win-x64",
archive_type="zip",
binary_name="dotnet.exe",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for Linux (x64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-linux-x64.tar.gz",
platform_id="linux-x64",
archive_type="tar.gz",
binary_name="dotnet",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for Linux (ARM64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-linux-arm64.tar.gz",
platform_id="linux-arm64",
archive_type="tar.gz",
binary_name="dotnet",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for macOS (x64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-osx-x64.tar.gz",
platform_id="osx-x64",
archive_type="tar.gz",
binary_name="dotnet",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for macOS (ARM64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-osx-arm64.tar.gz",
platform_id="osx-arm64",
archive_type="tar.gz",
binary_name="dotnet",
),
RuntimeDependency(
id="DotNetRuntime",
description=".NET 9 Runtime for Windows (ARM64)",
url="https://builds.dotnet.microsoft.com/dotnet/Runtime/9.0.6/dotnet-runtime-9.0.6-win-arm64.zip",
platform_id="win-arm64",
archive_type="zip",
binary_name="dotnet.exe",
extract_path="tools/net10.0/linux-arm64",
),
]
@@ -188,22 +144,26 @@ def find_solution_or_project_file(root_dir: str) -> str | None:
class CSharpLanguageServer(SolidLanguageServer):
"""
Provides C# specific instantiation of the LanguageServer class using `Microsoft.CodeAnalysis.LanguageServer`,
the official Roslyn-based language server from Microsoft.
Provides C# specific instantiation of the LanguageServer class using the official Roslyn-based
language server from NuGet.org.
You can pass a list of runtime dependency overrides in ls_specific_settings["csharp"]["runtime_dependencies"]. This is a list of
dicts, each containing at least the "id" key, and optionally "platform_id" to uniquely identify the dependency to override.
For example, to override the URL of the .NET runtime on windows-x64, add the entry:
You can pass a list of runtime dependency overrides in ls_specific_settings["csharp"]["runtime_dependencies"].
This is a list of dicts, each containing at least the "id" key, and optionally "platform_id" to uniquely
identify the dependency to override.
Example - Override Roslyn Language Server URL:
```
{
"id": "DotNetRuntime",
"id": "CSharpLanguageServer",
"platform_id": "win-x64",
"url": "https://example.com/custom-dotnet-runtime.zip"
"url": "https://example.com/custom-roslyn-server.nupkg"
}
```
See the `_RUNTIME_DEPENDENCIES` variable above for the available dependency ids and platform_ids.
Note: .NET runtime (version 10+) is required and installed automatically via Microsoft's official install
scripts. If you have a custom .NET installation, ensure 'dotnet' is available in PATH with version 10 or higher.
"""
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
@@ -212,8 +172,9 @@ class CSharpLanguageServer(SolidLanguageServer):
Use LanguageServer.create() instead.
"""
super().__init__(config, repository_root_path, None, "csharp", solidlsp_settings)
self.initialization_complete = threading.Event()
# Cache for original Roslyn symbol names with type annotations
# Key: (relative_file_path, line, character) -> Value: original name
self._original_symbol_names: dict[tuple[str, int, int], str] = {}
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir, self._solidlsp_settings, self.repository_root_path)
@@ -222,6 +183,116 @@ class CSharpLanguageServer(SolidLanguageServer):
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["bin", "obj", "packages", ".vs"]
@override
def request_document_symbols(self, relative_file_path: str, file_buffer: Any = None) -> DocumentSymbols:
"""
Override to normalize Roslyn symbol names and cache originals.
Roslyn 5.5.0+ returns symbol names with type annotations:
- Properties: "Name : string"
- Methods: "Add(int, int) : int"
This method:
1. Normalizes names to base form ("Name", "Add")
2. Caches original names for rich information display
3. Populates LSP spec's 'detail' field with type/signature info
"""
symbols = super().request_document_symbols(relative_file_path, file_buffer)
# Normalize all symbols recursively
for symbol in symbols.iter_symbols():
self._normalize_symbol_name(symbol, relative_file_path)
return symbols
@override
def request_hover(self, relative_file_path: str, line: int, column: int) -> Hover | None:
"""
Override to inject original Roslyn symbol names (with type annotations) into hover responses.
When hovering over a symbol whose name was normalized, we prepend the original
full name (e.g., 'Add(int, int) : int') to the hover content.
"""
hover = super().request_hover(relative_file_path, line, column)
if hover is None:
return None
# Check if we have an original name for this position
original_name = self._original_symbol_names.get((relative_file_path, line, column))
if original_name and "contents" in hover:
contents = hover["contents"]
if isinstance(contents, dict) and "value" in contents:
# Prepend the original full name with type information to the hover content
prefix = f"**{original_name}**\n\n---\n\n"
contents["value"] = prefix + contents["value"]
return hover
def _normalize_symbol_name(self, symbol: UnifiedSymbolInformation, relative_file_path: str) -> None:
"""
Normalize a single symbol's name and cache the original.
Processes children recursively.
"""
original_name = symbol.get("name", "")
# Extract base name and type/signature info
normalized_name, type_info = self._extract_base_name_and_type(original_name)
# Store original name if it was normalized
if original_name != normalized_name:
sel_range = symbol.get("selectionRange")
if sel_range:
start = sel_range.get("start")
if start and "line" in start and "character" in start:
line = start["line"]
char = start["character"]
cache_key = (relative_file_path, line, char)
self._original_symbol_names[cache_key] = original_name
# Populate LSP spec's 'detail' field with type/signature information
if type_info and "detail" not in symbol:
symbol["detail"] = type_info
# Update the symbol name
symbol["name"] = normalized_name
# Process children recursively
children = symbol.get("children", [])
for child in children:
self._normalize_symbol_name(child, relative_file_path)
@staticmethod
def _extract_base_name_and_type(roslyn_name: str) -> tuple[str, str]:
"""
Extract base name and type/signature information from Roslyn symbol names.
Examples:
"Name : string" -> ("Name", ": string")
"Add(int, int) : int" -> ("Add", "(int, int) : int")
"ToString()" -> ("ToString", "()")
"SimpleMethod" -> ("SimpleMethod", "")
Returns:
Tuple of (base_name, type_info)
"""
# Check for property pattern: "Name : Type"
if " : " in roslyn_name and "(" not in roslyn_name:
base_name, type_part = roslyn_name.split(" : ", 1)
return base_name.strip(), f": {type_part.strip()}"
# Check for method pattern: "MethodName(params) : ReturnType"
if "(" in roslyn_name:
paren_idx = roslyn_name.index("(")
base_name = roslyn_name[:paren_idx].strip()
signature = roslyn_name[paren_idx:].strip()
return base_name, signature
# No type annotation
return roslyn_name, ""
class DependencyProvider(LanguageServerDependencyProvider):
def __init__(
self,
@@ -235,7 +306,7 @@ class CSharpLanguageServer(SolidLanguageServer):
self._repository_root_path = repository_root_path
self._dotnet_path, self._language_server_path = self._ensure_server_installed()
def create_launch_command(self) -> list[str] | str:
def create_launch_command(self) -> list[str]:
# Find solution or project file
solution_or_project = find_solution_or_project_file(self._repository_root_path)
@@ -263,11 +334,23 @@ class CSharpLanguageServer(SolidLanguageServer):
"""
runtime_dependency_overrides = cast(list[dict[str, Any]], self._custom_settings.get("runtime_dependencies", []))
# Filter out deprecated DotNetRuntime overrides and warn users
filtered_overrides = []
for dep_override in runtime_dependency_overrides:
if dep_override.get("id") == "DotNetRuntime":
log.warning(
"The 'DotNetRuntime' runtime_dependencies override is no longer supported. "
".NET is now installed automatically via Microsoft's official install scripts. "
"Please remove this override from your configuration."
)
else:
filtered_overrides.append(dep_override)
log.debug("Resolving runtime dependencies")
runtime_dependencies = RuntimeDependencyCollection(
_RUNTIME_DEPENDENCIES,
overrides=runtime_dependency_overrides,
overrides=filtered_overrides,
)
log.debug(
@@ -276,30 +359,28 @@ class CSharpLanguageServer(SolidLanguageServer):
# Find the dependencies for our platform
lang_server_dep = runtime_dependencies.get_single_dep_for_current_platform("CSharpLanguageServer")
dotnet_runtime_dep = runtime_dependencies.get_single_dep_for_current_platform("DotNetRuntime")
dotnet_path = self._ensure_dotnet_runtime(dotnet_runtime_dep)
dotnet_path = self._ensure_dotnet_runtime()
server_dll_path = self._ensure_language_server(lang_server_dep)
return dotnet_path, server_dll_path
def _ensure_dotnet_runtime(self, dotnet_runtime_dep: RuntimeDependency) -> str:
def _ensure_dotnet_runtime(self) -> str:
"""Ensure .NET runtime is available and return the dotnet executable path."""
# TODO: use RuntimeDependency util methods instead of custom validation/download logic
# Check if dotnet is already available on the system
system_dotnet = shutil.which("dotnet")
if system_dotnet:
# Check if it's .NET 9
# Check if it's .NET 10 or compatible
try:
result = subprocess.run([system_dotnet, "--list-runtimes"], capture_output=True, text=True, check=True)
if "Microsoft.NETCore.App 9." in result.stdout:
log.info("Found system .NET 9 runtime")
# Accept .NET 10 or higher (10.x, 11.x, etc.)
if any(f"Microsoft.NETCore.App {v}." in result.stdout for v in range(10, 20)):
log.info("Found system .NET 10+ runtime")
return system_dotnet
except subprocess.CalledProcessError:
pass
# Download .NET 9 runtime using config
return self._ensure_dotnet_runtime_from_config(dotnet_runtime_dep)
# Install .NET 10 runtime using Microsoft's install script
return self._install_dotnet_with_script()
def _ensure_language_server(self, lang_server_dep: RuntimeDependency) -> str:
"""Ensure language server is available and return the DLL path."""
@@ -311,26 +392,24 @@ class CSharpLanguageServer(SolidLanguageServer):
server_dll = server_dir / lang_server_dep.binary_name
if server_dll.exists():
log.info(f"Using cached Microsoft.CodeAnalysis.LanguageServer from {server_dll}")
log.info(f"Using cached Roslyn Language Server from {server_dll}")
return str(server_dll)
# Download and install the language server
log.info(f"Downloading {package_name} version {package_version}...")
assert package_version is not None
assert package_name is not None
package_path = self._download_nuget_package_direct(package_name, package_version)
log.info(f"Downloading {package_name} version {package_version} from NuGet.org...")
package_path = self._download_nuget_package(lang_server_dep)
# Extract and install
self._extract_language_server(lang_server_dep, package_path, server_dir)
if not server_dll.exists():
raise SolidLSPException("Microsoft.CodeAnalysis.LanguageServer.dll not found after extraction")
raise SolidLSPException("Roslyn Language Server DLL not found after extraction")
# Make executable on Unix systems
if platform.system().lower() != "windows":
server_dll.chmod(0o755)
log.info(f"Successfully installed Microsoft.CodeAnalysis.LanguageServer to {server_dll}")
log.info(f"Successfully installed Roslyn Language Server to {server_dll}")
return str(server_dll)
@staticmethod
@@ -356,43 +435,28 @@ class CSharpLanguageServer(SolidLanguageServer):
server_dir.mkdir(parents=True, exist_ok=True)
shutil.copytree(source_dir, server_dir, dirs_exist_ok=True)
def _download_nuget_package_direct(self, package_name: str, package_version: str) -> Path:
def _download_nuget_package(self, dependency: RuntimeDependency) -> Path:
"""
Download a NuGet package directly from the Azure NuGet feed.
Download a NuGet package from NuGet.org and extract it.
Returns the path to the extracted package directory.
"""
azure_feed_url = "https://pkgs.dev.azure.com/azure-public/vside/_packaging/vs-impl/nuget/v3/index.json"
package_name = dependency.package_name
package_version = dependency.package_version
url = dependency.url
if url is None:
raise SolidLSPException(f"No URL specified for package {package_name} version {package_version}")
# Create temporary directory for package download
temp_dir = Path(self._ls_resources_dir) / "temp_downloads"
temp_dir.mkdir(parents=True, exist_ok=True)
try:
# First, get the service index from the Azure feed
log.debug("Fetching NuGet service index from Azure feed...")
with urllib.request.urlopen(azure_feed_url) as response:
service_index = json.loads(response.read().decode())
# Find the package base address (for downloading packages)
package_base_address = None
for resource in service_index.get("resources", []):
if resource.get("@type") == "PackageBaseAddress/3.0.0":
package_base_address = resource.get("@id")
break
if not package_base_address:
raise SolidLSPException("Could not find package base address in Azure NuGet feed")
# Construct the download URL for the specific package
package_id_lower = package_name.lower()
package_version_lower = package_version.lower()
package_url = f"{package_base_address.rstrip('/')}/{package_id_lower}/{package_version_lower}/{package_id_lower}.{package_version_lower}.nupkg"
log.debug(f"Downloading package from: {package_url}")
log.debug(f"Downloading package from: {url}")
# Download the .nupkg file
nupkg_file = temp_dir / f"{package_name}.{package_version}.nupkg"
urllib.request.urlretrieve(package_url, nupkg_file)
urllib.request.urlretrieve(url, nupkg_file)
# Extract the .nupkg file (it's just a zip file)
package_extract_dir = temp_dir / f"{package_name}.{package_version}"
@@ -405,82 +469,94 @@ class CSharpLanguageServer(SolidLanguageServer):
# Clean up the nupkg file
nupkg_file.unlink()
log.info(f"Successfully downloaded and extracted {package_name} version {package_version}")
log.info(f"Successfully downloaded and extracted {package_name} version {package_version} from NuGet.org")
return package_extract_dir
except Exception as e:
raise SolidLSPException(
f"Failed to download package {package_name} version {package_version} from Azure NuGet feed: {e}"
) from e
raise SolidLSPException(f"Failed to download package {package_name} version {package_version} from NuGet.org: {e}") from e
def _ensure_dotnet_runtime_from_config(self, dotnet_runtime_dep: RuntimeDependency) -> str:
def _install_dotnet_with_script(self, version: str = "10.0") -> str:
"""
Ensure .NET 9 runtime is available using runtime dependency configuration.
Install .NET runtime using Microsoft's official install script.
Returns the path to the dotnet executable.
"""
# TODO: use RuntimeDependency util methods instead of custom download logic
dotnet_dir = Path(self._ls_resources_dir) / f"dotnet-runtime-{version}"
# Check if dotnet is already available on the system
system_dotnet = shutil.which("dotnet")
if system_dotnet:
# Check if it's .NET 9
try:
result = subprocess.run([system_dotnet, "--list-runtimes"], capture_output=True, text=True, check=True)
if "Microsoft.NETCore.App 9." in result.stdout:
log.info("Found system .NET 9 runtime")
return system_dotnet
except subprocess.CalledProcessError:
pass
# Download .NET 9 runtime using config
dotnet_dir = Path(self._ls_resources_dir) / "dotnet-runtime-9.0"
assert dotnet_runtime_dep.binary_name is not None, "Runtime dependency must have a binary_name"
dotnet_exe = dotnet_dir / dotnet_runtime_dep.binary_name
# Determine binary name based on platform
is_windows = platform.system().lower() == "windows"
dotnet_exe = dotnet_dir / ("dotnet.exe" if is_windows else "dotnet")
if dotnet_exe.exists():
log.info(f"Using cached .NET runtime from {dotnet_exe}")
log.info(f"Using cached .NET {version} runtime from {dotnet_exe}")
return str(dotnet_exe)
# Download .NET runtime
log.info("Downloading .NET 9 runtime...")
# Download and run install script
log.info(f"Installing .NET {version} runtime using official Microsoft install script...")
dotnet_dir.mkdir(parents=True, exist_ok=True)
custom_dotnet_runtime_url = self._custom_settings.get("dotnet_runtime_url")
if custom_dotnet_runtime_url is not None:
log.info(f"Using custom .NET runtime url: {custom_dotnet_runtime_url}")
url = custom_dotnet_runtime_url
else:
url = dotnet_runtime_dep.url
archive_type = dotnet_runtime_dep.archive_type
# Download the runtime
download_path = dotnet_dir / f"dotnet-runtime.{archive_type}"
try:
log.debug(f"Downloading from {url}")
urllib.request.urlretrieve(url, download_path)
if is_windows:
# PowerShell script for Windows
script_url = "https://dot.net/v1/dotnet-install.ps1"
script_path = dotnet_dir / "dotnet-install.ps1"
urllib.request.urlretrieve(script_url, script_path)
# Extract the archive
if archive_type == "zip":
with zipfile.ZipFile(download_path, "r") as zip_ref:
zip_ref.extractall(dotnet_dir)
cmd = [
"pwsh",
"-NoProfile",
"-ExecutionPolicy",
"Bypass",
"-File",
str(script_path),
"-Version",
version,
"-InstallDir",
str(dotnet_dir),
"-Runtime",
"dotnet",
"-NoPath",
]
else:
# tar.gz
with tarfile.open(download_path, "r:gz") as tar_ref:
tar_ref.extractall(dotnet_dir)
# Bash script for Linux/macOS
script_url = "https://dot.net/v1/dotnet-install.sh"
script_path = dotnet_dir / "dotnet-install.sh"
urllib.request.urlretrieve(script_url, script_path)
script_path.chmod(0o755)
# Remove the archive
download_path.unlink()
cmd = [
"bash",
str(script_path),
"--version",
version,
"--install-dir",
str(dotnet_dir),
"--runtime",
"dotnet",
"--no-path",
]
# Make dotnet executable on Unix
if platform.system().lower() != "windows":
dotnet_exe.chmod(0o755)
# Run the install script
log.info("Running .NET install script: %s", cmd)
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
log.debug(f"Install script output: {result.stdout}")
log.info(f"Successfully installed .NET 9 runtime to {dotnet_exe}")
if not dotnet_exe.exists():
raise SolidLSPException(f"dotnet executable not found at {dotnet_exe} after installation")
log.info(f"Successfully installed .NET {version} runtime to {dotnet_exe}")
return str(dotnet_exe)
except subprocess.CalledProcessError as e:
raise SolidLSPException(
f"Failed to install .NET {version} runtime using install script: {e.stderr if e.stderr else e}"
) from e
except Exception as e:
raise SolidLSPException(f"Failed to download .NET 9 runtime from {url}: {e}") from e
message = f"Failed to install .NET {version} runtime: {e}"
if is_windows and isinstance(e, FileNotFoundError):
message += (
"; pwsh, i.e. PowerShell 7+, is required to install .NET runtime. Make sure pwsh is available on your system."
)
raise SolidLSPException(message) from e
def _get_initialize_params(self) -> InitializeParams:
"""
@@ -538,6 +614,8 @@ class CSharpLanguageServer(SolidLanguageServer):
)
def _start_server(self) -> None:
indexing_complete = threading.Event()
def do_nothing(params: dict) -> None:
return
@@ -645,7 +723,7 @@ class CSharpLanguageServer(SolidLanguageServer):
return
def handle_workspace_indexing_complete(params: dict) -> None:
self.completions_available.set()
indexing_complete.set()
# Set up notification handlers
self.server.on_notification("window/logMessage", window_log_message)
@@ -699,19 +777,16 @@ class CSharpLanguageServer(SolidLanguageServer):
# Open solution and project files
self._open_solution_and_projects()
self.initialization_complete.set()
log.info(
"Microsoft.CodeAnalysis.LanguageServer initialized and ready\n"
"Waiting for language server to index project files...\n"
"This may take a while for large projects"
)
if self.completions_available.wait(30): # Wait up to 30 seconds for indexing
if indexing_complete.wait(30): # Wait up to 30 seconds for indexing
log.info("Indexing complete")
else:
log.warning("Timeout waiting for indexing to complete, proceeding anyway")
self.completions_available.set()
def _force_pull_diagnostics(self, init_response: dict | InitializeResult) -> None:
"""
src/solidlsp/language_servers/eclipse_jdtls.py
+1 -2
View File
@@ -271,7 +271,7 @@ class EclipseJDTLS(SolidLanguageServer):
intellisense_members_path=intellisense_members_path,
)
def create_launch_command(self) -> list[str] | str:
def create_launch_command(self) -> list[str]:
# ws_dir is the workspace directory for the EclipseJDTLS server
ws_dir = str(
PurePath(
@@ -739,7 +739,6 @@ class EclipseJDTLS(SolidLanguageServer):
"*",
" ",
]
self.completions_available.set()
if registration["method"] == "workspace/executeCommand":
if "java.intellicode.enable" in registration["registerOptions"]["commands"]:
self._intellicode_enable_command_available.set()
src/solidlsp/language_servers/elixir_tools/elixir_tools.py
-1
View File
@@ -328,7 +328,6 @@ class ElixirTools(SolidLanguageServer):
assert "textDocumentSync" in init_response["capabilities"], f"Missing textDocumentSync in {init_response['capabilities']}"
self.server.notify.initialized({})
self.completions_available.set()
# Expert needs time to compile the project and build indexes on first run.
# This can take 2-3+ minutes for mid-sized codebases.
src/solidlsp/language_servers/elm_language_server.py
-3
View File
@@ -50,7 +50,6 @@ class ElmLanguageServer(SolidLanguageServer):
"elm",
solidlsp_settings,
)
self.server_ready = threading.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
@@ -196,8 +195,6 @@ class ElmLanguageServer(SolidLanguageServer):
else:
log.warning("Timeout waiting for Elm workspace scan, proceeding anyway")
self.server_ready.set()
self.completions_available.set()
log.info("Elm server ready")
@override
src/solidlsp/language_servers/erlang_language_server.py
-1
View File
@@ -155,7 +155,6 @@ class ErlangLanguageServer(SolidLanguageServer):
log.info(f"Erlang LS capabilities: {list(init_response['capabilities'].keys())}")
self.server.notify.initialized({})
self.completions_available.set()
# Wait for Erlang LS to be ready - adjust timeout based on environment
is_ci = os.getenv("CI") == "true" or os.getenv("GITHUB_ACTIONS") == "true"
src/solidlsp/language_servers/fortran_language_server.py
-4
View File
@@ -7,7 +7,6 @@ import os
import pathlib
import re
import shutil
import threading
from overrides import override
@@ -190,7 +189,6 @@ class FortranLanguageServer(SolidLanguageServer):
super().__init__(
config, repository_root_path, ProcessLaunchInfo(cmd=fortls_cmd, cwd=repository_root_path), "fortran", solidlsp_settings
)
self.server_ready = threading.Event()
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -283,8 +281,6 @@ class FortranLanguageServer(SolidLanguageServer):
log.info("Fortran LSP document symbol provider available")
self.server.notify.initialized({})
self.completions_available.set()
# Fortran Language Server is ready after initialization
self.server_ready.set()
log.info("Fortran Language Server initialization complete")
src/solidlsp/language_servers/gopls.py
+1 -5
View File
@@ -2,7 +2,6 @@ import logging
import os
import pathlib
import subprocess
import threading
from typing import Any, cast
from overrides import override
@@ -95,7 +94,6 @@ class Gopls(SolidLanguageServer):
self._setup_runtime_dependency()
super().__init__(config, repository_root_path, ProcessLaunchInfo(cmd="gopls", cwd=repository_root_path), "go", solidlsp_settings)
self.server_ready = threading.Event()
self.request_id = 0
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
@@ -234,8 +232,6 @@ class Gopls(SolidLanguageServer):
assert "definitionProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# gopls server is typically ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
# (no need to wait for events)
src/solidlsp/language_servers/groovy_language_server.py
-1
View File
@@ -271,4 +271,3 @@ class GroovyLanguageServer(SolidLanguageServer):
assert "workspaceSymbolProvider" in capabilities, "Server must support workspace symbols"
self.server.notify.initialized({})
self.completions_available.set()
src/solidlsp/language_servers/haskell_language_server.py
-1
View File
@@ -374,7 +374,6 @@ class HaskellLanguageServer(SolidLanguageServer):
log.info(f"HLS capabilities: {list(capabilities.keys())}")
self.server.notify.initialized({})
self.completions_available.set()
# Give HLS time to index the project
# HLS can be slow to index, especially on first run
src/solidlsp/language_servers/intelephense.py
-1
View File
@@ -172,7 +172,6 @@ class Intelephense(SolidLanguageServer):
assert "definitionProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# Intelephense server is typically ready immediately after initialization
# TODO: This is probably incorrect; the server does send an initialized notification, which we could wait for!
src/solidlsp/language_servers/jedi_server.py
+3 -1
View File
@@ -5,6 +5,7 @@ Provides Python specific instantiation of the LanguageServer class. Contains var
import logging
import os
import pathlib
import threading
from typing import cast
from overrides import override
@@ -155,6 +156,7 @@ class JediServer(SolidLanguageServer):
"""
Starts the JEDI Language Server
"""
completions_available = threading.Event()
def execute_client_command_handler(params: dict) -> list:
return []
@@ -164,7 +166,7 @@ class JediServer(SolidLanguageServer):
def check_experimental_status(params: dict) -> None:
if params["quiescent"] == True:
self.completions_available.set()
completions_available.set()
def window_log_message(msg: dict) -> None:
log.info(f"LSP: window/logMessage: {msg}")
src/solidlsp/language_servers/julia_server.py
-1
View File
@@ -166,5 +166,4 @@ class JuliaLanguageServer(SolidLanguageServer):
assert "documentSymbolProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
log.info("Julia Language Server is initialized and ready.")
src/solidlsp/language_servers/kotlin_language_server.py
-1
View File
@@ -485,4 +485,3 @@ class KotlinLanguageServer(SolidLanguageServer):
assert "semanticTokensProvider" in capabilities, "Server must support semantic tokens"
self.server.notify.initialized({})
self.completions_available.set()
src/solidlsp/language_servers/lua_ls.py
+1 -5
View File
@@ -8,7 +8,6 @@ import pathlib
import platform
import shutil
import tarfile
import threading
import zipfile
from pathlib import Path
@@ -161,7 +160,6 @@ class LuaLanguageServer(SolidLanguageServer):
super().__init__(
config, repository_root_path, ProcessLaunchInfo(cmd=lua_ls_path, cwd=repository_root_path), "lua", solidlsp_settings
)
self.server_ready = threading.Event()
self.request_id = 0
@staticmethod
@@ -281,8 +279,6 @@ class LuaLanguageServer(SolidLanguageServer):
assert "referencesProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# Lua Language Server is typically ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
# (no need to wait for events)
src/solidlsp/language_servers/marksman.py
+64 -65
View File
@@ -6,14 +6,12 @@ Contains various configurations and settings specific to Markdown.
import logging
import os
import pathlib
import threading
from overrides import override
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
from .common import RuntimeDependency, RuntimeDependencyCollection
@@ -26,80 +24,83 @@ class Marksman(SolidLanguageServer):
Provides Markdown specific instantiation of the LanguageServer class using marksman.
"""
marksman_releases = "https://github.com/artempyanykh/marksman/releases/download/2024-12-18"
runtime_dependencies = RuntimeDependencyCollection(
[
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-linux-x64",
platform_id="linux-x64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-linux-arm64",
platform_id="linux-arm64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-macos",
platform_id="osx-x64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-macos",
platform_id="osx-arm64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman.exe",
platform_id="win-x64",
archive_type="binary",
binary_name="marksman.exe",
),
]
)
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
marksman_releases = "https://github.com/artempyanykh/marksman/releases/download/2024-12-18"
runtime_dependencies = RuntimeDependencyCollection(
[
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-linux-x64",
platform_id="linux-x64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-linux-arm64",
platform_id="linux-arm64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-macos",
platform_id="osx-x64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman-macos",
platform_id="osx-arm64",
archive_type="binary",
binary_name="marksman",
),
RuntimeDependency(
id="marksman",
url=f"{marksman_releases}/marksman.exe",
platform_id="win-x64",
archive_type="binary",
binary_name="marksman.exe",
),
]
)
@classmethod
def _setup_runtime_dependencies(cls, config: LanguageServerConfig, solidlsp_settings: SolidLSPSettings) -> str:
"""Setup runtime dependencies for marksman and return the command to start the server."""
deps = cls.runtime_dependencies
dependency = deps.get_single_dep_for_current_platform()
def _get_or_install_core_dependency(self) -> str:
"""Setup runtime dependencies for marksman and return the command to start the server."""
deps = self.runtime_dependencies
dependency = deps.get_single_dep_for_current_platform()
marksman_ls_dir = cls.ls_resources_dir(solidlsp_settings)
marksman_executable_path = deps.binary_path(marksman_ls_dir)
if not os.path.exists(marksman_executable_path):
log.info(
f"Downloading marksman from {dependency.url} to {marksman_ls_dir}",
)
deps.install(marksman_ls_dir)
if not os.path.exists(marksman_executable_path):
raise FileNotFoundError(f"Download failed? Could not find marksman executable at {marksman_executable_path}")
os.chmod(marksman_executable_path, 0o755)
return marksman_executable_path
marksman_ls_dir = self._ls_resources_dir
marksman_executable_path = deps.binary_path(marksman_ls_dir)
if not os.path.exists(marksman_executable_path):
log.info(
f"Downloading marksman from {dependency.url} to {marksman_ls_dir}",
)
deps.install(marksman_ls_dir)
if not os.path.exists(marksman_executable_path):
raise FileNotFoundError(f"Download failed? Could not find marksman executable at {marksman_executable_path}")
os.chmod(marksman_executable_path, 0o755)
return marksman_executable_path
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path, "server"]
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
"""
Creates a Marksman instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
marksman_executable_path = self._setup_runtime_dependencies(config, solidlsp_settings)
super().__init__(
config,
repository_root_path,
ProcessLaunchInfo(cmd=f"{marksman_executable_path} server", cwd=repository_root_path),
None,
"markdown",
solidlsp_settings,
)
self.server_ready = threading.Event()
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
@override
def is_ignored_dirname(self, dirname: str) -> bool:
@@ -181,5 +182,3 @@ class Marksman(SolidLanguageServer):
# marksman is typically ready immediately after initialization
log.info("Marksman server initialization complete")
self.server_ready.set()
self.completions_available.set()
src/solidlsp/language_servers/matlab_language_server.py
+2 -4
View File
@@ -38,7 +38,7 @@ import platform
import shutil
import threading
import zipfile
from typing import Any, Union, cast
from typing import Any, cast
import requests
@@ -339,7 +339,7 @@ class MatlabLanguageServer(SolidLanguageServer):
self._matlab_path = matlab_path
return matlab_path
def create_launch_command(self) -> Union[str, list[str]]:
def create_launch_command(self) -> list[str]:
# Verify node is installed
node_path = shutil.which("node")
if node_path is None:
@@ -463,7 +463,6 @@ class MatlabLanguageServer(SolidLanguageServer):
if "mvm attach success" in message_text.lower() or "adding workspace folder" in message_text.lower():
log.info("MATLAB language server ready signal detected (MVM attached)")
self.server_ready.set()
self.completions_available.set()
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
@@ -508,7 +507,6 @@ class MatlabLanguageServer(SolidLanguageServer):
# Fallback: assume server is ready after timeout
log.info("Timeout waiting for MATLAB server ready signal, proceeding anyway")
self.server_ready.set()
self.completions_available.set()
else:
log.info("MATLAB server initialization complete")
src/solidlsp/language_servers/nixd_ls.py
-5
View File
@@ -11,7 +11,6 @@ import pathlib
import platform
import shutil
import subprocess
import threading
from pathlib import Path
from overrides import override
@@ -249,7 +248,6 @@ class NixLanguageServer(SolidLanguageServer):
nixd_path = self._setup_runtime_dependency()
super().__init__(config, repository_root_path, ProcessLaunchInfo(cmd=nixd_path, cwd=repository_root_path), "nix", solidlsp_settings)
self.server_ready = threading.Event()
self.request_id = 0
@staticmethod
@@ -374,8 +372,5 @@ class NixLanguageServer(SolidLanguageServer):
assert "referencesProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# nixd server is typically ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
src/solidlsp/language_servers/omnisharp.py
+1
View File
@@ -108,6 +108,7 @@ class OmniSharp(SolidLanguageServer):
self.server_ready = threading.Event()
self.definition_available = threading.Event()
self.references_available = threading.Event()
self.completions_available = threading.Event()
@override
def is_ignored_dirname(self, dirname: str) -> bool:
src/solidlsp/language_servers/pascal_server.py
-3
View File
@@ -141,7 +141,6 @@ class PascalLanguageServer(SolidLanguageServer):
solidlsp_settings,
)
self.server_ready = threading.Event()
self.completions_available_event = threading.Event()
# ============== Metadata Directory Management ==============
@@ -859,7 +858,6 @@ class PascalLanguageServer(SolidLanguageServer):
if "initialized" in message_text.lower() or "ready" in message_text.lower():
log.info("Pascal language server ready signal detected")
self.server_ready.set()
self.completions_available.set()
def publish_diagnostics(params: dict) -> None:
log.debug(f"Diagnostics: {params}")
@@ -904,7 +902,6 @@ class PascalLanguageServer(SolidLanguageServer):
# pasls may not send explicit ready signals, so we proceed after timeout
log.info("Timeout waiting for Pascal server ready signal, assuming server is ready")
self.server_ready.set()
self.completions_available.set()
else:
log.info("Pascal server initialization complete")
src/solidlsp/language_servers/perl_language_server.py
-2
View File
@@ -206,8 +206,6 @@ class PerlLanguageServer(SolidLanguageServer):
log.info(f"Sending workspace/didChangeConfiguration notification with config: {perl_config}")
self.server.notify.workspace_did_change_configuration(perl_config)
self.completions_available.set()
# Perl::LanguageServer needs time to index files and resolve cross-file references
# Without this delay, requests for definitions/references may return empty results
settling_time = 0.5
src/solidlsp/language_servers/phpactor.py
+161
View File
@@ -0,0 +1,161 @@
"""
Provides PHP specific instantiation of the LanguageServer class using Phpactor.
"""
import logging
import os
import pathlib
import re
import shutil
import stat
import subprocess
from overrides import override
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls_config import Language, LanguageServerConfig
from solidlsp.ls_utils import FileUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
PHPACTOR_VERSION = "2025.12.21.1"
PHPACTOR_PHAR_URL = f"https://github.com/phpactor/phpactor/releases/download/{PHPACTOR_VERSION}/phpactor.phar"
class PhpactorServer(SolidLanguageServer):
"""
Provides PHP specific instantiation of the LanguageServer class using Phpactor.
Phpactor is an open-source (MIT) PHP language server that requires PHP 8.1+ on the system.
It is an alternative to Intelephense, which is the default PHP language server.
You can pass the following entries in ls_specific_settings["php_phpactor"]:
- ignore_vendor: whether to ignore directories named "vendor" (default: true)
"""
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in self._ignored_dirnames
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
def _get_or_install_core_dependency(self) -> str:
"""
Setup runtime dependencies for Phpactor and return the path to the PHAR file.
"""
# Verify PHP is installed
php_path = shutil.which("php")
assert (
php_path is not None
), "PHP is not installed or not found in PATH. Phpactor requires PHP 8.1+. Please install PHP and try again."
# Check PHP version (Phpactor requires PHP 8.1+)
result = subprocess.run(["php", "--version"], capture_output=True, text=True, check=False)
php_version_output = result.stdout.strip()
log.info(f"PHP version: {php_version_output}")
version_match = re.search(r"PHP (\d+)\.(\d+)", php_version_output)
if version_match:
major, minor = int(version_match.group(1)), int(version_match.group(2))
if major < 8 or (major == 8 and minor < 1):
raise RuntimeError(f"PHP {major}.{minor} detected, but Phpactor requires PHP 8.1+. Please upgrade PHP.")
else:
log.warning("Could not parse PHP version from output. Continuing anyway.")
phpactor_phar_path = os.path.join(self._ls_resources_dir, "phpactor.phar")
if not os.path.exists(phpactor_phar_path):
os.makedirs(self._ls_resources_dir, exist_ok=True)
log.info(f"Downloading phpactor PHAR from {PHPACTOR_PHAR_URL}")
FileUtils.download_and_extract_archive(PHPACTOR_PHAR_URL, phpactor_phar_path, "binary")
assert os.path.exists(phpactor_phar_path), f"phpactor PHAR not found at {phpactor_phar_path}, download may have failed."
# Ensure the PHAR is executable
current_mode = os.stat(phpactor_phar_path).st_mode
os.chmod(phpactor_phar_path, current_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)
return phpactor_phar_path
def _create_launch_command(self, core_path: str) -> list[str]:
return ["php", core_path, "language-server"]
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
super().__init__(config, repository_root_path, None, "php", solidlsp_settings)
# Override internal language enum for correct file matching
self.language = Language.PHP_PHPACTOR
self._ignored_dirnames = {"node_modules", "cache"}
if self._custom_settings.get("ignore_vendor", True):
self._ignored_dirnames.add("vendor")
log.info(f"Ignoring the following directories for PHP (Phpactor): {', '.join(sorted(self._ignored_dirnames))}")
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialization params for the Phpactor Language Server.
"""
root_uri = pathlib.Path(repository_absolute_path).as_uri()
initialize_params = {
"processId": os.getpid(),
"rootPath": repository_absolute_path,
"rootUri": root_uri,
"capabilities": {
"textDocument": {
"synchronization": {"didSave": True, "dynamicRegistration": True},
"definition": {"dynamicRegistration": True},
"documentSymbol": {
"hierarchicalDocumentSymbolSupport": True,
"symbolKind": {"valueSet": list(range(1, 27))},
},
},
"workspace": {
"workspaceFolders": True,
"didChangeConfiguration": {"dynamicRegistration": True},
},
},
"workspaceFolders": [
{
"uri": root_uri,
"name": os.path.basename(repository_absolute_path),
}
],
"initializationOptions": {
"language_server_phpstan.enabled": False,
"language_server_psalm.enabled": False,
"language_server_php_cs_fixer.enabled": False,
},
}
return initialize_params # type: ignore
def _start_server(self) -> None:
"""Start Phpactor server process."""
def register_capability_handler(params: dict) -> None:
return
def window_log_message(msg: dict) -> None:
log.info(f"LSP: window/logMessage: {msg}")
def do_nothing(params: dict) -> None:
return
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
log.info("Starting Phpactor server process")
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
log.info("Sending initialize request from LSP client to LSP server and awaiting response")
init_response = self.server.send.initialize(initialize_params)
log.info("After sent initialize params")
# Verify server capabilities
assert "capabilities" in init_response
assert init_response["capabilities"].get("definitionProvider"), "Phpactor did not advertise definition support"
self.server.notify.initialized({})
src/solidlsp/language_servers/powershell_language_server.py
-2
View File
@@ -287,7 +287,6 @@ class PowerShellLanguageServer(SolidLanguageServer):
if "started" in message_text.lower() or "ready" in message_text.lower():
log.info("PowerShell Editor Services ready signal detected")
self.server_ready.set()
self.completions_available.set()
def do_nothing(params: dict) -> None:
return
@@ -320,6 +319,5 @@ class PowerShellLanguageServer(SolidLanguageServer):
# Fallback: assume server is ready after timeout
log.info("Timeout waiting for PSES ready signal, proceeding anyway")
self.server_ready.set()
self.completions_available.set()
else:
log.info("PowerShell Editor Services initialization complete")
src/solidlsp/language_servers/pyright_server.py
-3
View File
@@ -154,7 +154,6 @@ class PyrightServer(SolidLanguageServer):
log.info("Pyright workspace scanning complete")
self.found_source_files = True
self.analysis_complete.set()
self.completions_available.set()
def check_experimental_status(params: dict) -> None:
"""
@@ -164,7 +163,6 @@ class PyrightServer(SolidLanguageServer):
log.info("Received experimental/serverStatus with quiescent=true")
if not self.found_source_files:
self.analysis_complete.set()
self.completions_available.set()
# Set up notification handlers
self.server.on_request("client/registerCapability", do_nothing)
@@ -203,4 +201,3 @@ class PyrightServer(SolidLanguageServer):
log.warning("Timeout waiting for Pyright analysis completion, proceeding anyway")
# Fallback: assume analysis is complete after timeout
self.analysis_complete.set()
self.completions_available.set()
src/solidlsp/language_servers/r_language_server.py
-4
View File
@@ -2,7 +2,6 @@ import logging
import os
import pathlib
import subprocess
import threading
from typing import Any
from overrides import override
@@ -68,7 +67,6 @@ class RLanguageServer(SolidLanguageServer):
r_cmd = 'R --vanilla --quiet --slave -e "options(languageserver.debug_mode = FALSE); languageserver::run()"'
super().__init__(config, repository_root_path, ProcessLaunchInfo(cmd=r_cmd, cwd=repository_root_path), "r", solidlsp_settings)
self.server_ready = threading.Event()
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -158,7 +156,5 @@ class RLanguageServer(SolidLanguageServer):
log.info("R LSP definition provider available")
self.server.notify.initialized({})
self.completions_available.set()
# R Language Server is ready after initialization
self.server_ready.set()
src/solidlsp/language_servers/regal_server.py
-5
View File
@@ -3,7 +3,6 @@
import logging
import os
import shutil
import threading
from overrides import override
@@ -53,7 +52,6 @@ class RegalLanguageServer(SolidLanguageServer):
"rego",
solidlsp_settings,
)
self.server_ready = threading.Event()
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -129,8 +127,5 @@ class RegalLanguageServer(SolidLanguageServer):
assert "textDocumentSync" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# Regal server is ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
src/solidlsp/language_servers/ruby_lsp.py
-4
View File
@@ -350,7 +350,6 @@ class RubyLsp(SolidLanguageServer):
if params.get("type") == "ready":
log.info("ruby-lsp service is ready.")
self.analysis_complete.set()
self.completions_available.set()
def execute_client_command_handler(params: dict) -> list:
return []
@@ -370,7 +369,6 @@ class RubyLsp(SolidLanguageServer):
if value.get("kind") == "end":
log.info("ruby-lsp indexing complete ($/progress end)")
self.analysis_complete.set()
self.completions_available.set()
elif value.get("kind") == "begin":
log.info("ruby-lsp indexing started ($/progress begin)")
elif "percentage" in value:
@@ -384,7 +382,6 @@ class RubyLsp(SolidLanguageServer):
if value.get("kind") == "end" or value.get("percentage") == 100:
log.info("ruby-lsp indexing complete (token progress)")
self.analysis_complete.set()
self.completions_available.set()
def window_work_done_progress_create(params: dict) -> dict:
"""Handle workDoneProgress/create requests from ruby-lsp"""
@@ -429,7 +426,6 @@ class RubyLsp(SolidLanguageServer):
log.warning("Timeout waiting for ruby-lsp indexing completion, proceeding anyway")
# Fallback: assume indexing is complete after timeout
self.analysis_complete.set()
self.completions_available.set()
def _handle_initialization_response(self, init_response: InitializeResult) -> None:
"""
src/solidlsp/language_servers/rust_analyzer.py
+118 -113
View File
@@ -13,10 +13,9 @@ from typing import cast
from overrides import override
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
@@ -43,136 +42,140 @@ class RustAnalyzer(SolidLanguageServer):
return SolidLanguageServer._determine_log_level(line)
@staticmethod
def _get_rustup_version() -> str | None:
"""Get installed rustup version or None if not found."""
try:
result = subprocess.run(["rustup", "--version"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
@staticmethod
def _get_rustup_version() -> str | None:
"""Get installed rustup version or None if not found."""
try:
result = subprocess.run(["rustup", "--version"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
return None
return None
return None
@staticmethod
def _get_rust_analyzer_via_rustup() -> str | None:
"""Get rust-analyzer path via rustup. Returns None if not found."""
try:
result = subprocess.run(["rustup", "which", "rust-analyzer"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
pass
return None
@staticmethod
def _get_rust_analyzer_via_rustup() -> str | None:
"""Get rust-analyzer path via rustup. Returns None if not found."""
try:
result = subprocess.run(["rustup", "which", "rust-analyzer"], capture_output=True, text=True, check=False)
if result.returncode == 0:
return result.stdout.strip()
except FileNotFoundError:
pass
return None
@staticmethod
def _ensure_rust_analyzer_installed() -> str:
"""
Ensure rust-analyzer is available.
@staticmethod
def _ensure_rust_analyzer_installed() -> str:
"""
Ensure rust-analyzer is available.
Priority order:
1. Rustup existing installation (preferred - matches toolchain version)
2. Rustup auto-install if rustup is available (ensures correct version)
3. Common installation locations as fallback (only if rustup not available)
4. System PATH last (can pick up incompatible versions)
Priority order:
1. Rustup existing installation (preferred - matches toolchain version)
2. Rustup auto-install if rustup is available (ensures correct version)
3. Common installation locations as fallback (only if rustup not available)
4. System PATH last (can pick up incompatible versions)
:return: path to rust-analyzer executable
"""
# Try rustup FIRST (preferred - avoids picking up incompatible versions from PATH)
rustup_path = RustAnalyzer._get_rust_analyzer_via_rustup()
if rustup_path:
return rustup_path
:return: path to rust-analyzer executable
"""
# Try rustup FIRST (preferred - avoids picking up incompatible versions from PATH)
rustup_path = RustAnalyzer.DependencyProvider._get_rust_analyzer_via_rustup()
if rustup_path:
return rustup_path
# If rustup is available but rust-analyzer not installed, auto-install it BEFORE
# checking common paths. This ensures we get the correct version matching the toolchain.
if RustAnalyzer._get_rustup_version():
result = subprocess.run(["rustup", "component", "add", "rust-analyzer"], check=False, capture_output=True, text=True)
if result.returncode == 0:
# Verify installation worked
rustup_path = RustAnalyzer._get_rust_analyzer_via_rustup()
if rustup_path:
return rustup_path
# If auto-install failed, fall through to common paths as last resort
# If rustup is available but rust-analyzer not installed, auto-install it BEFORE
# checking common paths. This ensures we get the correct version matching the toolchain.
if RustAnalyzer.DependencyProvider._get_rustup_version():
result = subprocess.run(["rustup", "component", "add", "rust-analyzer"], check=False, capture_output=True, text=True)
if result.returncode == 0:
# Verify installation worked
rustup_path = RustAnalyzer.DependencyProvider._get_rust_analyzer_via_rustup()
if rustup_path:
return rustup_path
# If auto-install failed, fall through to common paths as last resort
# Determine platform-specific binary name and paths
is_windows = platform.system() == "Windows"
binary_name = "rust-analyzer.exe" if is_windows else "rust-analyzer"
# Determine platform-specific binary name and paths
is_windows = platform.system() == "Windows"
binary_name = "rust-analyzer.exe" if is_windows else "rust-analyzer"
# Fallback to common installation locations (only used if rustup not available)
common_paths: list[str | None] = []
# Fallback to common installation locations (only used if rustup not available)
common_paths: list[str | None] = []
if is_windows:
# Windows-specific paths
home = pathlib.Path.home()
common_paths.extend(
[
str(home / ".cargo" / "bin" / binary_name), # cargo install / rustup
str(home / "scoop" / "shims" / binary_name), # Scoop package manager
str(home / "scoop" / "apps" / "rust-analyzer" / "current" / binary_name), # Scoop direct
str(
pathlib.Path(os.environ.get("LOCALAPPDATA", "")) / "Programs" / "rust-analyzer" / binary_name
), # Standalone install
]
)
else:
# Unix-like paths (macOS, Linux)
common_paths.extend(
[
"/opt/homebrew/bin/rust-analyzer", # macOS Homebrew (Apple Silicon)
"/usr/local/bin/rust-analyzer", # macOS Homebrew (Intel) / Linux system
os.path.expanduser("~/.cargo/bin/rust-analyzer"), # cargo install
os.path.expanduser("~/.local/bin/rust-analyzer"), # User local bin
]
if is_windows:
# Windows-specific paths
home = pathlib.Path.home()
common_paths.extend(
[
str(home / ".cargo" / "bin" / binary_name), # cargo install / rustup
str(home / "scoop" / "shims" / binary_name), # Scoop package manager
str(home / "scoop" / "apps" / "rust-analyzer" / "current" / binary_name), # Scoop direct
str(
pathlib.Path(os.environ.get("LOCALAPPDATA", "")) / "Programs" / "rust-analyzer" / binary_name
), # Standalone install
]
)
else:
# Unix-like paths (macOS, Linux)
common_paths.extend(
[
"/opt/homebrew/bin/rust-analyzer", # macOS Homebrew (Apple Silicon)
"/usr/local/bin/rust-analyzer", # macOS Homebrew (Intel) / Linux system
os.path.expanduser("~/.cargo/bin/rust-analyzer"), # cargo install
os.path.expanduser("~/.local/bin/rust-analyzer"), # User local bin
]
)
for path in common_paths:
if path and os.path.isfile(path) and os.access(path, os.X_OK):
return path
# Last resort: check system PATH (can pick up incorrect aliases, hence checked last)
path_result = shutil.which("rust-analyzer")
if path_result and os.path.isfile(path_result) and os.access(path_result, os.X_OK):
return path_result
# Provide helpful error message with all searched locations
searched = [p for p in common_paths if p]
install_instructions = [
" - Rustup: rustup component add rust-analyzer",
" - Cargo: cargo install rust-analyzer",
]
if is_windows:
install_instructions.extend(
[
" - Scoop: scoop install rust-analyzer",
" - Chocolatey: choco install rust-analyzer",
" - Standalone: Download from https://github.com/rust-lang/rust-analyzer/releases",
]
)
else:
install_instructions.extend(
[
" - Homebrew (macOS): brew install rust-analyzer",
" - System package manager (Linux): apt/dnf/pacman install rust-analyzer",
]
)
raise RuntimeError(
"rust-analyzer is not installed or not in PATH.\n"
"Searched locations:\n" + "\n".join(f" - {p}" for p in searched) + "\n"
"Please install rust-analyzer via:\n" + "\n".join(install_instructions)
)
for path in common_paths:
if path and os.path.isfile(path) and os.access(path, os.X_OK):
return path
def _get_or_install_core_dependency(self) -> str:
return self._ensure_rust_analyzer_installed()
# Last resort: check system PATH (can pick up incorrect aliases, hence checked last)
path_result = shutil.which("rust-analyzer")
if path_result and os.path.isfile(path_result) and os.access(path_result, os.X_OK):
return path_result
# Provide helpful error message with all searched locations
searched = [p for p in common_paths if p]
install_instructions = [
" - Rustup: rustup component add rust-analyzer",
" - Cargo: cargo install rust-analyzer",
]
if is_windows:
install_instructions.extend(
[
" - Scoop: scoop install rust-analyzer",
" - Chocolatey: choco install rust-analyzer",
" - Standalone: Download from https://github.com/rust-lang/rust-analyzer/releases",
]
)
else:
install_instructions.extend(
[
" - Homebrew (macOS): brew install rust-analyzer",
" - System package manager (Linux): apt/dnf/pacman install rust-analyzer",
]
)
raise RuntimeError(
"rust-analyzer is not installed or not in PATH.\n"
"Searched locations:\n" + "\n".join(f" - {p}" for p in searched) + "\n"
"Please install rust-analyzer via:\n" + "\n".join(install_instructions)
)
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path]
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
"""
Creates a RustAnalyzer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
"""
rustanalyzer_executable_path = self._ensure_rust_analyzer_installed()
log.info(f"Using rust-analyzer at: {rustanalyzer_executable_path}")
super().__init__(
config,
repository_root_path,
ProcessLaunchInfo(cmd=rustanalyzer_executable_path, cwd=repository_root_path),
None,
"rust",
solidlsp_settings,
)
@@ -181,6 +184,9 @@ class RustAnalyzer(SolidLanguageServer):
self.initialize_searcher_command_available = threading.Event()
self.resolve_main_method_available = threading.Event()
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in ["target"]
@@ -715,6 +721,5 @@ class RustAnalyzer(SolidLanguageServer):
"completionItem": {"labelDetailsSupport": True},
}
self.server.notify.initialized({})
self.completions_available.set()
self.server_ready.wait()
src/solidlsp/language_servers/scala_language_server.py
+147 -4
View File
@@ -7,6 +7,7 @@ import os
import pathlib
import shutil
import subprocess
from enum import Enum
from overrides import override
@@ -23,16 +24,99 @@ if not PlatformUtils.get_platform_id().value.startswith("win"):
log = logging.getLogger(__name__)
# Default configuration constants
DEFAULT_METALS_VERSION = "1.6.4"
DEFAULT_CLIENT_NAME = "Serena"
DEFAULT_ON_STALE_LOCK = "auto-clean"
DEFAULT_LOG_MULTI_INSTANCE_NOTICE = True
class StaleLockMode(Enum):
"""Mode for handling stale Metals H2 database locks."""
AUTO_CLEAN = "auto-clean"
"""Automatically remove stale lock files (default, recommended)."""
WARN = "warn"
"""Log a warning but proceed; may result in degraded experience."""
FAIL = "fail"
"""Raise an error and refuse to start."""
def _get_scala_settings(solidlsp_settings: SolidLSPSettings) -> dict[str, object]:
"""
Extract Scala-specific settings with defaults applied.
Returns a dictionary with keys:
- metals_version: str
- client_name: str
- on_stale_lock: StaleLockMode
- log_multi_instance_notice: bool
"""
from solidlsp.ls_config import Language
defaults: dict[str, object] = {
"metals_version": DEFAULT_METALS_VERSION,
"client_name": DEFAULT_CLIENT_NAME,
"on_stale_lock": StaleLockMode.AUTO_CLEAN,
"log_multi_instance_notice": DEFAULT_LOG_MULTI_INSTANCE_NOTICE,
}
if not solidlsp_settings.ls_specific_settings:
return defaults
scala_settings = solidlsp_settings.get_ls_specific_settings(Language.SCALA)
# Parse stale lock mode with validation
on_stale_lock_str = scala_settings.get("on_stale_lock", DEFAULT_ON_STALE_LOCK)
try:
on_stale_lock = StaleLockMode(on_stale_lock_str)
except ValueError:
log.warning(f"Invalid on_stale_lock value '{on_stale_lock_str}', using '{DEFAULT_ON_STALE_LOCK}'")
on_stale_lock = StaleLockMode.AUTO_CLEAN
return {
"metals_version": scala_settings.get("metals_version", DEFAULT_METALS_VERSION),
"client_name": scala_settings.get("client_name", DEFAULT_CLIENT_NAME),
"on_stale_lock": on_stale_lock,
"log_multi_instance_notice": scala_settings.get("log_multi_instance_notice", DEFAULT_LOG_MULTI_INSTANCE_NOTICE),
}
class ScalaLanguageServer(SolidLanguageServer):
"""
Provides Scala specific instantiation of the LanguageServer class. Contains various configurations and settings specific to Scala.
Provides Scala specific instantiation of the LanguageServer class.
Contains various configurations and settings specific to Scala.
Configurable options in ls_specific_settings (in serena_config.yml):
ls_specific_settings:
scala:
# Stale lock handling: auto-clean | warn | fail
on_stale_lock: 'auto-clean'
# Log notice when another Metals instance is detected
log_multi_instance_notice: true
# Metals version to bootstrap (default: DEFAULT_METALS_VERSION)
metals_version: '1.6.4'
# Client identifier sent to Metals (default: DEFAULT_CLIENT_NAME)
client_name: 'Serena'
Multi-instance support:
Metals uses H2 AUTO_SERVER mode (enabled by default) to support multiple
concurrent instances sharing the same database. Running Serena's Metals
alongside VS Code's Metals is designed to work. The only issue is stale
locks from crashed processes, which this class can detect and clean up.
"""
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
"""
Creates a ScalaLanguageServer instance. This class is not meant to be instantiated directly. Use LanguageServer.create() instead.
Creates a ScalaLanguageServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
# Check for stale locks before setting up dependencies (fail-fast)
self._check_metals_db_status(repository_root_path, solidlsp_settings)
scala_lsp_executable_path = self._setup_runtime_dependencies(config, solidlsp_settings)
super().__init__(
config,
@@ -42,6 +126,62 @@ class ScalaLanguageServer(SolidLanguageServer):
solidlsp_settings,
)
def _check_metals_db_status(self, repository_root_path: str, solidlsp_settings: SolidLSPSettings) -> None:
"""
Check the Metals H2 database status and handle stale locks.
This method is called before setting up runtime dependencies to fail-fast
if there's a stale lock that the user has configured to fail on.
"""
from pathlib import Path
from solidlsp.ls_exceptions import MetalsStaleLockError
from solidlsp.util.metals_db_utils import (
MetalsDbStatus,
check_metals_db_status,
cleanup_stale_lock,
)
project_path = Path(repository_root_path)
status, lock_info = check_metals_db_status(project_path)
# Get settings using the shared helper function
settings = _get_scala_settings(solidlsp_settings)
on_stale_lock: StaleLockMode = settings["on_stale_lock"] # type: ignore[assignment]
log_multi_instance_notice: bool = settings["log_multi_instance_notice"] # type: ignore[assignment]
if status == MetalsDbStatus.ACTIVE_INSTANCE:
if log_multi_instance_notice and lock_info:
log.info(
f"Another Metals instance detected (PID: {lock_info.pid}). "
"This is fine - Metals supports multiple instances via H2 AUTO_SERVER. "
"Both instances will share the database and Bloop build server."
)
elif status == MetalsDbStatus.STALE_LOCK:
lock_path = lock_info.lock_path if lock_info else project_path / ".metals" / "metals.mv.db.lock.db"
lock_path_str = str(lock_path)
if on_stale_lock == StaleLockMode.AUTO_CLEAN:
log.info(f"Stale Metals lock detected, cleaning up: {lock_path_str}")
cleanup_success = cleanup_stale_lock(lock_path)
if not cleanup_success:
log.warning(
f"Failed to clean up stale lock at {lock_path_str}. "
"Metals may fall back to in-memory database (degraded experience)."
)
elif on_stale_lock == StaleLockMode.WARN:
log.warning(
f"Stale Metals lock detected at {lock_path_str}. "
"A previous Metals process may have crashed. "
"Metals will fall back to in-memory database (degraded experience). "
"Consider removing the lock file manually or setting on_stale_lock='auto-clean'."
)
elif on_stale_lock == StaleLockMode.FAIL:
raise MetalsStaleLockError(lock_path_str)
@override
def is_ignored_dirname(self, dirname: str) -> bool:
return super().is_ignored_dirname(dirname) or dirname in [
@@ -57,7 +197,10 @@ class ScalaLanguageServer(SolidLanguageServer):
"""
assert shutil.which("java") is not None, "JDK is not installed or not in PATH."
metals_version = "1.6.4"
# Get settings using the shared helper function
settings = _get_scala_settings(solidlsp_settings)
metals_version: str = settings["metals_version"] # type: ignore[assignment]
client_name: str = settings["client_name"] # type: ignore[assignment]
metals_home = os.path.join(cls.ls_resources_dir(solidlsp_settings), "metals-lsp")
os.makedirs(metals_home, exist_ok=True)
@@ -98,7 +241,7 @@ class ScalaLanguageServer(SolidLanguageServer):
"--java-opt",
"-Xms100m",
"--java-opt",
"-Dmetals.client=Serena",
f"-Dmetals.client={client_name}",
artifact,
"-o",
metals_executable,
src/solidlsp/language_servers/solargraph.py
-2
View File
@@ -320,7 +320,6 @@ class Solargraph(SolidLanguageServer):
if params.get("type") == "ServiceReady" and params.get("message") == "Service is ready.":
log.info("Solargraph service is ready.")
self.analysis_complete.set()
self.completions_available.set()
def execute_client_command_handler(params: dict) -> list:
return []
@@ -364,4 +363,3 @@ class Solargraph(SolidLanguageServer):
log.warning("Timeout waiting for Solargraph analysis completion, proceeding anyway")
# Fallback: assume analysis is complete after timeout
self.analysis_complete.set()
self.completions_available.set()
src/solidlsp/language_servers/sourcekit_lsp.py
-6
View File
@@ -2,7 +2,6 @@ import logging
import os
import pathlib
import subprocess
import threading
import time
from overrides import override
@@ -53,7 +52,6 @@ class SourceKitLSP(SolidLanguageServer):
super().__init__(
config, repository_root_path, ProcessLaunchInfo(cmd="sourcekit-lsp", cwd=repository_root_path), "swift", solidlsp_settings
)
self.server_ready = threading.Event()
self.request_id = 0
self._did_sleep_before_requesting_references = False
self._initialization_timestamp: float | None = None
@@ -332,10 +330,6 @@ class SourceKitLSP(SolidLanguageServer):
assert "definitionProvider" in capabilities, "definitionProvider capability missing"
self.server.notify.initialized({})
self.completions_available.set()
self.server_ready.set()
self.server_ready.wait()
# Mark initialization timestamp for smarter delay calculation
self._initialization_timestamp = time.time()
src/solidlsp/language_servers/taplo_server.py
+96 -96
View File
@@ -11,18 +11,16 @@ import platform
import shutil
import socket
import stat
import threading
import urllib.request
from typing import Any
# Download timeout in seconds (prevents indefinite hangs)
DOWNLOAD_TIMEOUT_SECONDS = 120
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_utils import PathUtils
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
@@ -120,116 +118,120 @@ class TaploServer(SolidLanguageServer):
Creates a TaploServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
taplo_executable_path = self._setup_runtime_dependencies(solidlsp_settings)
super().__init__(
config,
repository_root_path,
ProcessLaunchInfo(cmd=f"{taplo_executable_path} lsp stdio", cwd=repository_root_path),
None,
"toml",
solidlsp_settings,
)
self.server_ready = threading.Event()
@classmethod
def _setup_runtime_dependencies(cls, solidlsp_settings: SolidLSPSettings) -> str:
"""
Setup runtime dependencies for Taplo and return the command to start the server.
"""
# First check if taplo is already installed system-wide
system_taplo = shutil.which("taplo")
if system_taplo:
log.info(f"Using system-installed Taplo at: {system_taplo}")
return system_taplo
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
# Setup local installation directory
taplo_dir = os.path.join(cls.ls_resources_dir(solidlsp_settings), "taplo")
os.makedirs(taplo_dir, exist_ok=True)
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
def _get_or_install_core_dependency(self) -> str:
"""
Setup runtime dependencies for Taplo and return the command to start the server.
"""
# First check if taplo is already installed system-wide
system_taplo = shutil.which("taplo")
if system_taplo:
log.info(f"Using system-installed Taplo at: {system_taplo}")
return system_taplo
_, executable_name = _get_taplo_download_url()
taplo_executable = os.path.join(taplo_dir, executable_name)
# Setup local installation directory
taplo_dir = os.path.join(self._ls_resources_dir, "taplo")
os.makedirs(taplo_dir, exist_ok=True)
if os.path.exists(taplo_executable) and os.access(taplo_executable, os.X_OK):
log.info(f"Using cached Taplo at: {taplo_executable}")
return taplo_executable
_, executable_name = _get_taplo_download_url()
taplo_executable = os.path.join(taplo_dir, executable_name)
# Download and install Taplo
log.info(f"Taplo not found. Downloading version {TAPLO_VERSION}...")
cls._download_taplo(taplo_dir, taplo_executable)
if os.path.exists(taplo_executable) and os.access(taplo_executable, os.X_OK):
log.info(f"Using cached Taplo at: {taplo_executable}")
return taplo_executable
if not os.path.exists(taplo_executable):
raise FileNotFoundError(
f"Taplo executable not found at {taplo_executable}. "
"Installation may have failed. Try installing manually: cargo install taplo-cli --locked"
)
# Download and install Taplo
log.info(f"Taplo not found. Downloading version {TAPLO_VERSION}...")
self._download_taplo(taplo_dir, taplo_executable)
return taplo_executable
@classmethod
def _download_taplo(cls, install_dir: str, executable_path: str) -> None:
"""Download and extract Taplo binary with SHA256 verification."""
# TODO: consider using existing download utilities in SolidLSP instead of the custom logic here
download_url, _ = _get_taplo_download_url()
archive_filename = os.path.basename(download_url)
try:
log.info(f"Downloading Taplo from: {download_url}")
archive_path = os.path.join(install_dir, archive_filename)
# Download the archive with timeout to prevent indefinite hangs
old_timeout = socket.getdefaulttimeout()
try:
socket.setdefaulttimeout(DOWNLOAD_TIMEOUT_SECONDS)
urllib.request.urlretrieve(download_url, archive_path)
finally:
socket.setdefaulttimeout(old_timeout)
# Verify SHA256 checksum
expected_hash = TAPLO_SHA256_CHECKSUMS.get(archive_filename)
if expected_hash:
if not _verify_sha256(archive_path, expected_hash):
os.remove(archive_path)
raise RuntimeError(
f"SHA256 checksum verification failed for {archive_filename}. "
"The downloaded file may be corrupted or tampered with. "
"Try installing manually: cargo install taplo-cli --locked"
)
log.info(f"SHA256 checksum verified for {archive_filename}")
else:
log.warning(
f"No SHA256 checksum available for {archive_filename}. "
"Skipping verification - consider installing manually: cargo install taplo-cli --locked"
if not os.path.exists(taplo_executable):
raise FileNotFoundError(
f"Taplo executable not found at {taplo_executable}. "
"Installation may have failed. Try installing manually: cargo install taplo-cli --locked"
)
# Extract based on format
if archive_path.endswith(".gz") and not archive_path.endswith(".tar.gz"):
# Single file gzip
with gzip.open(archive_path, "rb") as f_in:
with open(executable_path, "wb") as f_out:
f_out.write(f_in.read())
elif archive_path.endswith(".zip"):
import zipfile
return taplo_executable
with zipfile.ZipFile(archive_path, "r") as zip_ref:
# Security: Validate paths to prevent zip slip vulnerability
for member in zip_ref.namelist():
member_path = os.path.normpath(os.path.join(install_dir, member))
if not member_path.startswith(os.path.normpath(install_dir)):
raise RuntimeError(f"Zip slip detected: {member} attempts to escape install directory")
zip_ref.extractall(install_dir)
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path, "lsp", "stdio"]
# Make executable on Unix systems
if os.name != "nt":
os.chmod(executable_path, os.stat(executable_path).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
@classmethod
def _download_taplo(cls, install_dir: str, executable_path: str) -> None:
"""Download and extract Taplo binary with SHA256 verification."""
# TODO: consider using existing download utilities in SolidLSP instead of the custom logic here
download_url, _ = _get_taplo_download_url()
archive_filename = os.path.basename(download_url)
# Clean up archive
os.remove(archive_path)
log.info(f"Taplo installed successfully at: {executable_path}")
try:
log.info(f"Downloading Taplo from: {download_url}")
archive_path = os.path.join(install_dir, archive_filename)
except Exception as e:
log.error(f"Failed to download Taplo: {e}")
raise RuntimeError(
f"Failed to download Taplo from {download_url}. Try installing manually: cargo install taplo-cli --locked"
) from e
# Download the archive with timeout to prevent indefinite hangs
old_timeout = socket.getdefaulttimeout()
try:
socket.setdefaulttimeout(DOWNLOAD_TIMEOUT_SECONDS)
urllib.request.urlretrieve(download_url, archive_path)
finally:
socket.setdefaulttimeout(old_timeout)
# Verify SHA256 checksum
expected_hash = TAPLO_SHA256_CHECKSUMS.get(archive_filename)
if expected_hash:
if not _verify_sha256(archive_path, expected_hash):
os.remove(archive_path)
raise RuntimeError(
f"SHA256 checksum verification failed for {archive_filename}. "
"The downloaded file may be corrupted or tampered with. "
"Try installing manually: cargo install taplo-cli --locked"
)
log.info(f"SHA256 checksum verified for {archive_filename}")
else:
log.warning(
f"No SHA256 checksum available for {archive_filename}. "
"Skipping verification - consider installing manually: cargo install taplo-cli --locked"
)
# Extract based on format
if archive_path.endswith(".gz") and not archive_path.endswith(".tar.gz"):
# Single file gzip
with gzip.open(archive_path, "rb") as f_in:
with open(executable_path, "wb") as f_out:
f_out.write(f_in.read())
elif archive_path.endswith(".zip"):
import zipfile
with zipfile.ZipFile(archive_path, "r") as zip_ref:
# Security: Validate paths to prevent zip slip vulnerability
for member in zip_ref.namelist():
member_path = os.path.normpath(os.path.join(install_dir, member))
if not member_path.startswith(os.path.normpath(install_dir)):
raise RuntimeError(f"Zip slip detected: {member} attempts to escape install directory")
zip_ref.extractall(install_dir)
# Make executable on Unix systems
if os.name != "nt":
os.chmod(executable_path, os.stat(executable_path).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
# Clean up archive
os.remove(archive_path)
log.info(f"Taplo installed successfully at: {executable_path}")
except Exception as e:
log.error(f"Failed to download Taplo: {e}")
raise RuntimeError(
f"Failed to download Taplo from {download_url}. Try installing manually: cargo install taplo-cli --locked"
) from e
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -308,8 +310,6 @@ class TaploServer(SolidLanguageServer):
self.server.notify.initialized({})
log.info("Taplo server initialization complete")
self.server_ready.set()
self.completions_available.set()
def is_ignored_dirname(self, dirname: str) -> bool:
"""Define TOML-specific directories to ignore."""
src/solidlsp/language_servers/terraform_ls.py
-5
View File
@@ -1,7 +1,6 @@
import logging
import os
import shutil
import threading
from typing import cast
from overrides import override
@@ -166,7 +165,6 @@ class TerraformLS(SolidLanguageServer):
"terraform",
solidlsp_settings,
)
self.server_ready = threading.Event()
self.request_id = 0
@staticmethod
@@ -232,8 +230,5 @@ class TerraformLS(SolidLanguageServer):
assert "definitionProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# terraform-ls server is typically ready immediately after initialization
self.server_ready.set()
self.server_ready.wait()
src/solidlsp/language_servers/typescript_language_server.py
+1 -3
View File
@@ -269,7 +269,6 @@ class TypeScriptLanguageServer(SolidLanguageServer):
"""
if params.get("quiescent") == True:
self.server_ready.set()
self.completions_available.set()
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
@@ -302,11 +301,10 @@ class TypeScriptLanguageServer(SolidLanguageServer):
log.info("Timeout waiting for TypeScript server to become ready, proceeding anyway")
# Fallback: assume server is ready after timeout
self.server_ready.set()
self.completions_available.set()
@override
def _get_wait_time_for_cross_file_referencing(self) -> float:
return 1
return 2
@override
def _get_preferred_definition(self, definitions: list[ls_types.Location]) -> ls_types.Location:
src/solidlsp/language_servers/vts_language_server.py
-2
View File
@@ -186,7 +186,6 @@ class VtsLanguageServer(SolidLanguageServer):
"""
if params.get("quiescent") is True:
self.server_ready.set()
self.completions_available.set()
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
@@ -222,7 +221,6 @@ class VtsLanguageServer(SolidLanguageServer):
log.info("Timeout waiting for VTS server to become ready, proceeding anyway")
# Fallback: assume server is ready after timeout
self.server_ready.set()
self.completions_available.set()
@override
def _get_wait_time_for_cross_file_referencing(self) -> float:
src/solidlsp/language_servers/vue_language_server.py
-2
View File
@@ -633,7 +633,6 @@ class VueLanguageServer(SolidLanguageServer):
if "initialized" in message_text.lower() or "ready" in message_text.lower():
log.info("Vue language server ready signal detected")
self.server_ready.set()
self.completions_available.set()
def tsserver_request_notification_handler(params: list) -> None:
try:
@@ -683,7 +682,6 @@ class VueLanguageServer(SolidLanguageServer):
if not self.server_ready.wait(timeout=self.VUE_SERVER_READY_TIMEOUT):
log.info("Timeout waiting for Vue server ready signal, proceeding anyway")
self.server_ready.set()
self.completions_available.set()
else:
log.info("Vue server initialization complete")
src/solidlsp/language_servers/yaml_language_server.py
+46 -45
View File
@@ -7,14 +7,12 @@ import logging
import os
import pathlib
import shutil
import threading
from typing import Any
from solidlsp.language_servers.common import RuntimeDependency, RuntimeDependencyCollection
from solidlsp.ls import SolidLanguageServer
from solidlsp.ls import LanguageServerDependencyProvider, LanguageServerDependencyProviderSinglePath, SolidLanguageServer
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
@@ -47,56 +45,61 @@ class YamlLanguageServer(SolidLanguageServer):
Creates a YamlLanguageServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
yaml_lsp_executable_path = self._setup_runtime_dependencies(config, solidlsp_settings)
super().__init__(
config,
repository_root_path,
ProcessLaunchInfo(cmd=yaml_lsp_executable_path, cwd=repository_root_path),
None,
"yaml",
solidlsp_settings,
)
self.server_ready = threading.Event()
@classmethod
def _setup_runtime_dependencies(cls, config: LanguageServerConfig, solidlsp_settings: SolidLSPSettings) -> str:
"""
Setup runtime dependencies for YAML Language Server and return the command to start the server.
"""
# Verify both node and npm are installed
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
def _create_dependency_provider(self) -> LanguageServerDependencyProvider:
return self.DependencyProvider(self._custom_settings, self._ls_resources_dir)
deps = RuntimeDependencyCollection(
[
RuntimeDependency(
id="yaml-language-server",
description="yaml-language-server package (Red Hat)",
command="npm install --prefix ./ yaml-language-server@1.19.2",
platform_id="any",
),
]
)
class DependencyProvider(LanguageServerDependencyProviderSinglePath):
def _get_or_install_core_dependency(self) -> str:
"""
Setup runtime dependencies for YAML Language Server and return the command to start the server.
"""
# Verify both node and npm are installed
is_node_installed = shutil.which("node") is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which("npm") is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
# Install yaml-language-server if not already installed
yaml_ls_dir = os.path.join(cls.ls_resources_dir(solidlsp_settings), "yaml-lsp")
yaml_executable_path = os.path.join(yaml_ls_dir, "node_modules", ".bin", "yaml-language-server")
# Handle Windows executable extension
if os.name == "nt":
yaml_executable_path += ".cmd"
if not os.path.exists(yaml_executable_path):
log.info(f"YAML Language Server executable not found at {yaml_executable_path}. Installing...")
deps.install(yaml_ls_dir)
log.info("YAML language server dependencies installed successfully")
if not os.path.exists(yaml_executable_path):
raise FileNotFoundError(
f"yaml-language-server executable not found at {yaml_executable_path}, something went wrong with the installation."
deps = RuntimeDependencyCollection(
[
RuntimeDependency(
id="yaml-language-server",
description="yaml-language-server package (Red Hat)",
command="npm install --prefix ./ yaml-language-server@1.19.2",
platform_id="any",
),
]
)
return f"{yaml_executable_path} --stdio"
# Install yaml-language-server if not already installed
yaml_ls_dir = os.path.join(self._ls_resources_dir, "yaml-lsp")
yaml_executable_path = os.path.join(yaml_ls_dir, "node_modules", ".bin", "yaml-language-server")
# Handle Windows executable extension
if os.name == "nt":
yaml_executable_path += ".cmd"
if not os.path.exists(yaml_executable_path):
log.info(f"YAML Language Server executable not found at {yaml_executable_path}. Installing...")
deps.install(yaml_ls_dir)
log.info("YAML language server dependencies installed successfully")
if not os.path.exists(yaml_executable_path):
raise FileNotFoundError(
f"yaml-language-server executable not found at {yaml_executable_path}, something went wrong with the installation."
)
return yaml_executable_path
def _create_launch_command(self, core_path: str) -> list[str]:
return [core_path, "--stdio"]
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
@@ -184,5 +187,3 @@ class YamlLanguageServer(SolidLanguageServer):
# YAML language server is ready immediately after initialization
log.info("YAML server initialization complete")
self.server_ready.set()
self.completions_available.set()
src/solidlsp/language_servers/zls.py
+1 -5
View File
@@ -8,7 +8,6 @@ import pathlib
import platform
import shutil
import subprocess
import threading
from overrides import override
@@ -100,7 +99,6 @@ class ZigLanguageServer(SolidLanguageServer):
self._setup_runtime_dependency()
super().__init__(config, repository_root_path, ProcessLaunchInfo(cmd="zls", cwd=repository_root_path), "zig", solidlsp_settings)
self.server_ready = threading.Event()
self.request_id = 0
@staticmethod
@@ -213,11 +211,9 @@ class ZigLanguageServer(SolidLanguageServer):
assert "referencesProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# ZLS server is ready after initialization
self.server_ready.set()
self.server_ready.wait()
# (no need to wait for an event)
# Open build.zig if it exists to help ZLS understand project structure
build_zig_path = os.path.join(self.repository_root_path, "build.zig")
src/solidlsp/ls.py
+142 -59
View File
@@ -18,13 +18,14 @@ from typing import Self, Union, cast
import pathspec
from sensai.util.pickle import getstate, load_pickle
from sensai.util.string import ToStringMixin
from serena.text_utils import MatchedConsecutiveLines
from serena.util.file_system import match_path
from solidlsp import ls_types
from solidlsp.ls_config import Language, LanguageServerConfig
from solidlsp.ls_exceptions import SolidLSPException
from solidlsp.ls_handler import SolidLanguageServerHandler
from solidlsp.ls_process import LanguageServerProcess
from solidlsp.ls_types import UnifiedSymbolInformation
from solidlsp.ls_utils import FileUtils, PathUtils, TextUtils
from solidlsp.lsp_protocol_handler import lsp_types
@@ -67,14 +68,90 @@ class LSPFileBuffer:
This class is used to store the contents of an open LSP file in memory.
"""
def __init__(self, uri: str, contents: str, encoding: str, version: int, language_id: str, ref_count: int) -> None:
def __init__(
self,
abs_path: Path,
uri: str,
encoding: str,
version: int,
language_id: str,
ref_count: int,
language_server: "SolidLanguageServer",
open_in_ls: bool = True,
) -> None:
self.abs_path = abs_path
self.language_server = language_server
self.uri = uri
self.contents = contents
self._read_file_modified_date: float | None = None
self._contents: str | None = None
self.version = version
self.language_id = language_id
self.ref_count = ref_count
self.encoding = encoding
self._content_hash: str | None = None
self._is_open_in_ls = False
if open_in_ls:
self._open_in_ls()
def _open_in_ls(self) -> None:
"""
Open the file in the language server if it is not already open.
"""
if self._is_open_in_ls:
return
self._is_open_in_ls = True
self.language_server.server.notify.did_open_text_document(
{
LSPConstants.TEXT_DOCUMENT: { # type: ignore
LSPConstants.URI: self.uri,
LSPConstants.LANGUAGE_ID: self.language_id,
LSPConstants.VERSION: 0,
LSPConstants.TEXT: self.contents,
}
}
)
def close(self) -> None:
if self._is_open_in_ls:
self.language_server.server.notify.did_close_text_document(
{
LSPConstants.TEXT_DOCUMENT: { # type: ignore
LSPConstants.URI: self.uri,
}
}
)
def ensure_open_in_ls(self) -> None:
"""Ensure that the file is opened in the language server."""
self._open_in_ls()
@property
def contents(self) -> str:
file_modified_date = self.abs_path.stat().st_mtime
# if contents are cached, check if they are stale (file modification since last read) and invalidate if so
if self._contents is not None:
assert self._read_file_modified_date is not None
if file_modified_date > self._read_file_modified_date:
self._contents = None
if self._contents is None:
self._read_file_modified_date = file_modified_date
self._contents = FileUtils.read_file(str(self.abs_path), self.encoding)
self._content_hash = None
return self._contents
@contents.setter
def contents(self, new_contents: str) -> None:
"""
Sets new contents for the file buffer (in-memory change only).
Persistence of the change to disk must be handled separately.
:param new_contents: the new contents to set
"""
self._contents = new_contents
self._content_hash = None
@property
def content_hash(self) -> str:
@@ -87,7 +164,7 @@ class LSPFileBuffer:
return self.contents.split("\n")
class SymbolBody:
class SymbolBody(ToStringMixin):
"""
Representation of the body of a symbol, which allows the extraction of the symbol's text
from the lines of the file it is defined in.
@@ -104,14 +181,21 @@ class SymbolBody:
self._end_line = end_line
self._end_col = end_col
def _tostring_excludes(self) -> list[str]:
return ["_lines"]
def get_text(self) -> str:
# extract relevant lines
symbol_body = "\n".join(self._lines[self._start_line : self._end_line + 1])
# remove leading indentation
# remove leading content from the first line
symbol_body = symbol_body[self._start_col :]
# TODO: handle end_col properly (this was never implemented)
# remove trailing content from the last line
last_line = self._lines[self._end_line]
trailing_length = len(last_line) - self._end_col
if trailing_length > 0:
symbol_body = symbol_body[: -(len(last_line) - self._end_col)]
return symbol_body
@@ -190,13 +274,12 @@ class LanguageServerDependencyProvider(ABC):
self._ls_resources_dir = ls_resources_dir
@abstractmethod
def create_launch_command(self) -> list[str] | str:
def create_launch_command(self) -> list[str]:
"""
Creates the launch command for this language server, potentially downloading and installing dependencies
beforehand.
:return: the launch command as a list containing the executable and its arguments (preferred for robustness)
or the entire command in a single string.
:return: the launch command as a list containing the executable and its arguments
"""
def create_launch_command_env(self) -> dict[str, str]:
@@ -228,7 +311,7 @@ class LanguageServerDependencyProviderSinglePath(LanguageServerDependencyProvide
:return: the core dependency's path (e.g. executable, jar, etc.)
"""
def create_launch_command(self) -> Union[str, list[str]]:
def create_launch_command(self) -> list[str]:
path = self._custom_settings.get("ls_path", None)
if path is not None:
core_path = path
@@ -237,11 +320,10 @@ class LanguageServerDependencyProviderSinglePath(LanguageServerDependencyProvide
return self._create_launch_command(core_path)
@abstractmethod
def _create_launch_command(self, core_path: str) -> list[str] | str:
def _create_launch_command(self, core_path: str) -> list[str]:
"""
:param core_path: path to the core dependency
:return: the launch command as a list containing the executable and its arguments (preferred for robustness)
or the entire command in a single string.
:return: the launch command as a list containing the executable and its arguments
"""
@@ -412,7 +494,6 @@ class SolidLanguageServer(ABC):
self._load_document_symbols_cache()
self.server_started = False
self.completions_available = threading.Event()
if config.trace_lsp_communication:
def logging_fn(source: str, target: str, msg: StringDict | str) -> None:
@@ -428,7 +509,7 @@ class SolidLanguageServer(ABC):
self._dependency_provider = self._create_dependency_provider()
process_launch_info = self._create_process_launch_info()
log.debug(f"Creating language server instance with {language_id=} and process launch info: {process_launch_info}")
self.server = SolidLanguageServerHandler(
self.server = LanguageServerProcess(
process_launch_info,
language=self.language,
determine_log_level=self._determine_log_level,
@@ -614,70 +695,72 @@ class SolidLanguageServer(ABC):
return self.language_id
@contextmanager
def open_file(self, relative_file_path: str) -> Iterator[LSPFileBuffer]:
def open_file(self, relative_file_path: str, open_in_ls: bool = True) -> Iterator[LSPFileBuffer]:
"""
Open a file in the Language Server. This is required before making any requests to the Language Server.
:param relative_file_path: The relative path of the file to open.
:param open_in_ls: whether to open the file in the language server, sending the didOpen notification.
Set this to False to read the local file buffer without notifying the LS; the file can
be opened in the LS later by calling the `ensure_open_in_ls` method on the returned LSPFileBuffer.
"""
if not self.server_started:
log.error("open_file called before Language Server started")
raise SolidLSPException("Language Server not started")
absolute_file_path = str(PurePath(self.repository_root_path, relative_file_path))
uri = pathlib.Path(absolute_file_path).as_uri()
absolute_file_path = Path(self.repository_root_path, relative_file_path)
uri = absolute_file_path.as_uri()
if uri in self.open_file_buffers:
assert self.open_file_buffers[uri].uri == uri
assert self.open_file_buffers[uri].ref_count >= 1
fb = self.open_file_buffers[uri]
assert fb.uri == uri
assert fb.ref_count >= 1
self.open_file_buffers[uri].ref_count += 1
yield self.open_file_buffers[uri]
self.open_file_buffers[uri].ref_count -= 1
fb.ref_count += 1
if open_in_ls:
fb.ensure_open_in_ls()
yield fb
fb.ref_count -= 1
else:
contents = FileUtils.read_file(absolute_file_path, self._encoding)
version = 0
language_id = self._get_language_id_for_file(relative_file_path)
self.open_file_buffers[uri] = LSPFileBuffer(
uri=uri, contents=contents, encoding=self._encoding, version=version, language_id=language_id, ref_count=1
fb = LSPFileBuffer(
abs_path=absolute_file_path,
uri=uri,
encoding=self._encoding,
version=version,
language_id=language_id,
ref_count=1,
language_server=self,
open_in_ls=open_in_ls,
)
self.server.notify.did_open_text_document(
{
LSPConstants.TEXT_DOCUMENT: { # type: ignore
LSPConstants.URI: uri,
LSPConstants.LANGUAGE_ID: language_id,
LSPConstants.VERSION: 0,
LSPConstants.TEXT: contents,
}
}
)
yield self.open_file_buffers[uri]
self.open_file_buffers[uri].ref_count -= 1
self.open_file_buffers[uri] = fb
yield fb
fb.ref_count -= 1
if self.open_file_buffers[uri].ref_count == 0:
self.server.notify.did_close_text_document(
{
LSPConstants.TEXT_DOCUMENT: { # type: ignore
LSPConstants.URI: uri,
}
}
)
self.open_file_buffers[uri].close()
del self.open_file_buffers[uri]
@contextmanager
def _open_file_context(self, relative_file_path: str, file_buffer: LSPFileBuffer | None = None) -> Iterator[LSPFileBuffer]:
def _open_file_context(
self, relative_file_path: str, file_buffer: LSPFileBuffer | None = None, open_in_ls: bool = True
) -> Iterator[LSPFileBuffer]:
"""
Internal context manager to open a file, optionally reusing an existing file buffer.
:param relative_file_path: the relative path of the file to open.
:param file_buffer: an optional existing file buffer to reuse.
:param open_in_ls: whether to open the file in the language server, sending the didOpen notification.
Set this to False to read the local file buffer without notifying the LS; the file can
be opened in the LS later by calling the `ensure_open_in_ls` method on the returned LSPFileBuffer.
"""
if file_buffer is not None:
if open_in_ls:
file_buffer.ensure_open_in_ls()
yield file_buffer
else:
with self.open_file(relative_file_path) as fb:
with self.open_file(relative_file_path, open_in_ls=open_in_ls) as fb:
yield fb
def insert_text_at_position(self, relative_file_path: str, line: int, column: int, text_to_be_inserted: str) -> ls_types.Position:
@@ -865,14 +948,14 @@ class SolidLanguageServer(ABC):
log.error("request_references called before Language Server started")
raise SolidLSPException("Language Server not started")
if not self._has_waited_for_cross_file_references:
# Some LS require waiting for a while before they can return cross-file references.
# This is a workaround for such LS that don't have a reliable "finished initializing" signal.
sleep(self._get_wait_time_for_cross_file_referencing())
self._has_waited_for_cross_file_references = True
t0 = perf_counter() if _debug_enabled else 0.0
with self.open_file(relative_file_path):
if not self._has_waited_for_cross_file_references:
# Some LS require waiting for a while before they can return cross-file references.
# This is a workaround for such LS that don't have a reliable "finished initializing" signal.
# The waiting has to happen after at least one file was opened in the ls
sleep(self._get_wait_time_for_cross_file_referencing())
self._has_waited_for_cross_file_references = True
t0 = perf_counter() if _debug_enabled else 0.0
try:
response = self._send_references_request(relative_file_path, line=line, column=column)
except Exception as e:
@@ -1023,7 +1106,6 @@ class SolidLanguageServer(ABC):
num_retries = 0
while response is None or (response["isIncomplete"] and num_retries < 30): # type: ignore
self.completions_available.wait()
response = self.server.send.completion(completion_params)
if isinstance(response, list):
response = {"items": response, "isIncomplete": False}
@@ -1152,7 +1234,7 @@ class SolidLanguageServer(ABC):
where the parent attribute will be the file symbol which in turn may have a package symbol as parent.
If you need a symbol tree that contains file symbols as well, you should use `request_full_symbol_tree` instead.
"""
with self._open_file_context(relative_file_path, file_buffer) as file_data:
with self._open_file_context(relative_file_path, file_buffer, open_in_ls=False) as file_data:
# check if the desired result is cached
cache_key = relative_file_path
file_hash_and_result = self._document_symbols_cache.get(cache_key)
@@ -1170,6 +1252,7 @@ class SolidLanguageServer(ABC):
log.debug("perf: document_symbols_cache STALE path=%s", relative_file_path)
# no cached result: request the root symbols from the language server
file_data.ensure_open_in_ls()
root_symbols = self._request_document_symbols(relative_file_path, file_data)
if root_symbols is None:
@@ -2142,7 +2225,7 @@ class SolidLanguageServer(ABC):
return self
@property
def handler(self) -> SolidLanguageServerHandler:
def handler(self) -> LanguageServerProcess:
"""Access the underlying language server handler.
Useful for advanced operations like sending custom commands
src/solidlsp/ls_config.py
+17 -2
View File
@@ -41,6 +41,7 @@ class Language(str, Enum):
RUBY = "ruby"
DART = "dart"
CPP = "cpp"
CPP_CCLS = "cpp_ccls"
PHP = "php"
R = "r"
PERL = "perl"
@@ -87,6 +88,10 @@ class Language(str, Enum):
"""Solargraph language server for Ruby (legacy, experimental).
Use Language.RUBY (ruby-lsp) for better performance and modern LSP features.
"""
PHP_PHPACTOR = "php_phpactor"
"""Phpactor language server for PHP (instead of Intelephense, which is the default).
Requires PHP 8.1+ on the system. Fully open-source (MIT license).
"""
MARKDOWN = "markdown"
"""Marksman language server for Markdown (experimental).
Must be explicitly specified as the main language, not auto-detected.
@@ -120,10 +125,12 @@ class Language(str, Enum):
self.PYTHON_JEDI,
self.CSHARP_OMNISHARP,
self.RUBY_SOLARGRAPH,
self.PHP_PHPACTOR,
self.MARKDOWN,
self.YAML,
self.TOML,
self.GROOVY,
self.CPP_CCLS,
}
def __str__(self) -> str:
@@ -170,13 +177,13 @@ class Language(str, Enum):
return FilenameMatcher("*.rb", "*.erb")
case self.RUBY_SOLARGRAPH:
return FilenameMatcher("*.rb")
case self.CPP:
case self.CPP | self.CPP_CCLS:
return FilenameMatcher("*.cpp", "*.h", "*.hpp", "*.c", "*.hxx", "*.cc", "*.cxx")
case self.KOTLIN:
return FilenameMatcher("*.kt", "*.kts")
case self.DART:
return FilenameMatcher("*.dart")
case self.PHP:
case self.PHP | self.PHP_PHPACTOR:
return FilenameMatcher("*.php")
case self.R:
return FilenameMatcher("*.R", "*.r", "*.Rmd", "*.Rnw")
@@ -304,10 +311,18 @@ class Language(str, Enum):
from solidlsp.language_servers.clangd_language_server import ClangdLanguageServer
return ClangdLanguageServer
case self.CPP_CCLS:
from solidlsp.language_servers.ccls_language_server import CCLS
return CCLS
case self.PHP:
from solidlsp.language_servers.intelephense import Intelephense
return Intelephense
case self.PHP_PHPACTOR:
from solidlsp.language_servers.phpactor import PhpactorServer
return PhpactorServer
case self.PERL:
from solidlsp.language_servers.perl_language_server import PerlLanguageServer
src/solidlsp/ls_exceptions.py
+24 -2
View File
@@ -24,7 +24,7 @@ class SolidLSPException(Exception):
:return: True if the exception is caused by the language server having terminated as indicated
by the causing exception being an instance of LanguageServerTerminatedException.
"""
from .ls_handler import LanguageServerTerminatedException
from .ls_process import LanguageServerTerminatedException
return isinstance(self.cause, LanguageServerTerminatedException)
@@ -32,7 +32,7 @@ class SolidLSPException(Exception):
"""
:return: the affected language for the case where the exception is caused by the language server having terminated
"""
from .ls_handler import LanguageServerTerminatedException
from .ls_process import LanguageServerTerminatedException
if isinstance(self.cause, LanguageServerTerminatedException):
return self.cause.language
@@ -50,3 +50,25 @@ class SolidLSPException(Exception):
s += " "
s += f"(caused by {self.cause})"
return s
class MetalsStaleLockError(SolidLSPException):
"""
Raised when a stale Metals H2 database lock is detected and the user
has configured fail-on-stale-lock behavior.
A stale lock occurs when a previous Metals process crashed without
cleaning up its lock file, which can prevent proper AUTO_SERVER
coordination with new instances.
"""
def __init__(self, lock_path: str, message: str | None = None) -> None:
self.lock_path = lock_path
if message is None:
message = (
f"Stale Metals lock file detected at {lock_path}. "
"A previous Metals process may have crashed. "
"To resolve: remove the lock file manually, or set "
"on_stale_lock='auto-clean' in ls_specific_settings.scala."
)
super().__init__(message)
src/solidlsp/ls_handler.py → src/solidlsp/ls_process.py
+26 -43
View File
@@ -22,7 +22,6 @@ from solidlsp.lsp_protocol_handler.lsp_types import ErrorCodes
from solidlsp.lsp_protocol_handler.server import (
ENCODING,
LSPError,
MessageType,
PayloadLike,
ProcessLaunchInfo,
StringDict,
@@ -93,11 +92,10 @@ class Request(ToStringMixin):
raise e
class SolidLanguageServerHandler:
class LanguageServerProcess:
"""
This class provides the implementation of Python client for the Language Server Protocol.
A class that launches the language server and communicates with it
using the Language Server Protocol (LSP).
Represents a language server process and provides methods for communicating with it using the
Language Server Protocol (LSP).
It provides methods for sending requests, responses, and notifications to the server
and for registering handlers for requests and notifications from the server.
@@ -117,7 +115,7 @@ class SolidLanguageServerHandler:
that handle requests from the server.
on_notification_handlers: A dictionary that maps method names to callback functions
that handle notifications from the server.
logger: An optional function that takes two strings (source and destination) and
_trace_log_fn: An optional function that takes two strings (source and destination) and
a payload dictionary, and logs the communication between the client and the server.
tasks: A dictionary that maps task ids to asyncio.Task objects that represent
the asynchronous tasks created by the handler.
@@ -152,7 +150,7 @@ class SolidLanguageServerHandler:
self._pending_requests: dict[Any, Request] = {}
self.on_request_handlers: dict[str, Callable[[Any], Any]] = {}
self.on_notification_handlers: dict[str, Callable[[Any], None]] = {}
self.logger = logger
self._trace_log_fn = logger
self.tasks: dict[int, Any] = {}
self.task_counter = 0
self.loop = None
@@ -301,19 +299,19 @@ class SolidLanguageServerHandler:
Perform the shutdown sequence for the client, including sending the shutdown request to the server and notifying it of exit
"""
self._is_shutting_down = True
self._log("Sending shutdown request to server")
log.info("Sending shutdown request to server")
self.send.shutdown()
self._log("Received shutdown response from server")
self._log("Sending exit notification to server")
log.info("Received shutdown response from server")
log.info("Sending exit notification to server")
self.notify.exit()
self._log("Sent exit notification to server")
log.info("Sent exit notification to server")
def _log(self, message: str | StringDict) -> None:
def _trace(self, src: str, dest: str, message: str | StringDict) -> None:
"""
Create a log message
Traces LS communication by logging the message with the source and destination of the message
"""
if self.logger is not None:
self.logger("client", "logger", message)
if self._trace_log_fn is not None:
self._trace_log_fn(src, dest, message)
def _read_bytes_from_process(self, process, stream, num_bytes) -> bytes: # type: ignore
"""Read exactly num_bytes from process stdout"""
@@ -402,18 +400,17 @@ class SolidLanguageServerHandler:
try:
self._receive_payload(json.loads(body))
except OSError as ex:
self._log(f"malformed {ENCODING}: {ex}")
log.error(f"Error processing payload: {ex}", exc_info=ex)
except UnicodeDecodeError as ex:
self._log(f"malformed {ENCODING}: {ex}")
log.error(f"Decoding error for encoding={ENCODING}: {ex}")
except json.JSONDecodeError as ex:
self._log(f"malformed JSON: {ex}")
log.error(f"JSON decoding error: {ex}")
def _receive_payload(self, payload: StringDict) -> None:
"""
Determine if the payload received from server is for a request, response, or notification and invoke the appropriate handler
"""
if self.logger:
self.logger("server", "client", payload)
self._trace("ls", "solidlsp", payload)
try:
if "method" in payload:
if "id" in payload:
@@ -423,9 +420,9 @@ class SolidLanguageServerHandler:
elif "id" in payload:
self._response_handler(payload)
else:
self._log(f"Unknown payload type: {payload}")
log.error(f"Unknown payload type: {payload}")
except Exception as err:
self._log(f"Error handling server payload: {err}")
log.error(f"Error handling server payload: {err}")
def send_notification(self, method: str, params: dict | None = None) -> None:
"""
@@ -443,7 +440,6 @@ class SolidLanguageServerHandler:
"""
Send error response to the given request id to the server with the given error
"""
# Use lock to prevent race conditions on tasks and task_counter
self._send_payload(make_error_response(request_id, err))
def _cancel_pending_requests(self, exception: Exception) -> None:
@@ -473,15 +469,14 @@ class SolidLanguageServerHandler:
self._send_payload(make_request(method, request_id, params))
self._log(f"Waiting for response to request {method} with params:\n{params}")
log.debug("Waiting for response to request %s with params:\n%s", method, params)
result = request.get_result(timeout=self._request_timeout)
log.debug("Completed: %s", request)
self._log("Processing result")
if result.is_error():
raise SolidLSPException(f"Error processing request {method} with params:\n{params}", cause=result.error) from result.error
self._log(f"Returning non-error result, which is:\n{result.payload}")
log.debug("Returning result:\n%s", result.payload)
return result.payload
def _send_payload(self, payload: StringDict) -> None:
@@ -490,7 +485,7 @@ class SolidLanguageServerHandler:
"""
if not self.process or not self.process.stdin:
return
self._log(payload)
self._trace("solidlsp", "ls", payload)
msg = create_message(payload)
# Use lock to prevent concurrent writes to stdin that cause buffer corruption
@@ -500,8 +495,7 @@ class SolidLanguageServerHandler:
self.process.stdin.flush()
except (BrokenPipeError, ConnectionResetError, OSError) as e:
# Log the error but don't raise to prevent cascading failures
if self.logger:
self.logger("client", "logger", f"Failed to write to stdin: {e}")
log.error(f"Failed to write to stdin: {e}")
return
def on_request(self, method: str, cb: Callable[[Any], Any]) -> None:
@@ -569,23 +563,12 @@ class SolidLanguageServerHandler:
params = response.get("params")
handler = self.on_notification_handlers.get(method)
if not handler:
self._log(f"unhandled {method}")
log.warning("Unhandled method '%s'", method)
return
try:
handler(params)
except asyncio.CancelledError:
return
except Exception as ex:
if (not self._is_shutting_down) and self.logger:
self.logger(
"client",
"logger",
str(
{
"type": MessageType.error,
"message": str(ex),
"method": method,
"params": params,
}
),
)
if not self._is_shutting_down:
log.error("Error handling notification for method '%s': %s", method, ex, exc_info=ex)
src/solidlsp/ls_request.py
+2 -2
View File
@@ -3,11 +3,11 @@ from typing import TYPE_CHECKING, Any, Union
from solidlsp.lsp_protocol_handler import lsp_types
if TYPE_CHECKING:
from .ls_handler import SolidLanguageServerHandler
from .ls_process import LanguageServerProcess
class LanguageServerRequest:
def __init__(self, handler: "SolidLanguageServerHandler"):
def __init__(self, handler: "LanguageServerProcess"):
self.handler = handler
def _send_request(self, method: str, params: Any | None = None) -> Any:
src/solidlsp/ls_utils.py
+3
View File
@@ -414,6 +414,9 @@ class PlatformUtils:
class SymbolUtils:
@staticmethod
def symbol_tree_contains_name(roots: list[UnifiedSymbolInformation], name: str) -> bool:
"""
Check if any symbol in the tree has a name matching the given name.
"""
for symbol in roots:
if symbol["name"] == name:
return True
src/solidlsp/util/metals_db_utils.py
+280
View File
@@ -0,0 +1,280 @@
"""
Utilities for detecting and managing Scala Metals H2 database state.
This module provides functions to detect existing Metals LSP instances by checking
the H2 database lock file, and to clean up stale locks from crashed processes.
Metals uses H2 AUTO_SERVER mode (enabled by default) to support multiple concurrent
instances sharing the same database. However, if a Metals process crashes without
proper cleanup, it can leave a stale lock file that prevents proper AUTO_SERVER
coordination, causing new instances to fall back to in-memory database mode.
"""
from __future__ import annotations
import logging
import re
from dataclasses import dataclass
from enum import Enum
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
pass
log = logging.getLogger(__name__)
class MetalsDbStatus(Enum):
"""Status of the Metals H2 database for a project."""
NO_DATABASE = "no_database"
"""No .metals directory or database exists (fresh project)."""
NO_LOCK = "no_lock"
"""Database exists but no lock file (safe to start)."""
ACTIVE_INSTANCE = "active_instance"
"""Lock held by a running process (will share via AUTO_SERVER)."""
STALE_LOCK = "stale_lock"
"""Lock held by a dead process (needs cleanup)."""
@dataclass
class MetalsLockInfo:
"""Information extracted from an H2 database lock file."""
pid: int | None
"""Process ID that holds the lock, if parseable."""
port: int | None
"""TCP port for AUTO_SERVER connection, if parseable."""
lock_path: Path
"""Path to the lock file."""
is_stale: bool
"""True if the owning process is no longer running."""
raw_content: str
"""Raw content of the lock file for debugging."""
def parse_h2_lock_file(lock_path: Path) -> MetalsLockInfo | None:
"""
Parse an H2 database lock file to extract connection information.
The H2 lock file format varies by version but typically contains
server connection information. Common formats include:
- Text format: "server:localhost:9092" or similar
- Binary format with embedded PID
Args:
lock_path: Path to the .lock.db file
Returns:
MetalsLockInfo if the file can be parsed, None if file doesn't exist
or is completely unparsable.
"""
if not lock_path.exists():
return None
try:
# Try reading as text first (most common for H2 AUTO_SERVER)
content = lock_path.read_text(encoding="utf-8", errors="replace")
except OSError as e:
log.debug(f"Could not read lock file {lock_path}: {e}")
return None
pid: int | None = None
port: int | None = None
# Try to extract port from common H2 lock file formats
# Format 1: "server:localhost:PORT"
server_match = re.search(r"server:[\w.]+:(\d+)", content, re.IGNORECASE)
if server_match:
port = int(server_match.group(1))
# Format 2: Look for standalone port numbers (H2 uses ports in 9000+ range typically)
if port is None:
port_match = re.search(r"\b(9\d{3})\b", content)
if port_match:
port = int(port_match.group(1))
# Try to extract PID - H2 may embed this in various formats
pid_match = re.search(r"pid[=:]?\s*(\d+)", content, re.IGNORECASE)
if pid_match:
pid = int(pid_match.group(1))
# Check if the process is still alive
is_stale = False
if pid is not None:
is_stale = not is_metals_process_alive(pid)
elif port is not None:
# If we have a port but no PID, try to find a Metals process using that port
is_stale = not _is_port_in_use_by_metals(port)
else:
# Can't determine - assume stale if lock exists but we can't parse it
# and no Metals processes are running for this project
log.debug(f"Could not parse PID or port from lock file: {lock_path}")
is_stale = True # Conservative: treat unparsable as stale
return MetalsLockInfo(
pid=pid,
port=port,
lock_path=lock_path,
is_stale=is_stale,
raw_content=content[:200], # Truncate for logging
)
def is_metals_process_alive(pid: int) -> bool:
"""
Check if a process with the given PID is alive and is a Metals process.
Args:
pid: Process ID to check
Returns:
True if the process exists and appears to be a Metals LSP server.
"""
try:
import psutil
proc = psutil.Process(pid)
if not proc.is_running():
return False
# Check if this is actually a Metals process
cmdline = " ".join(proc.cmdline()).lower()
return _is_metals_cmdline(cmdline)
except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
return False
except Exception as e:
log.debug(f"Error checking process {pid}: {e}")
return False
def _is_metals_cmdline(cmdline: str) -> bool:
"""Check if a command line string appears to be a Metals LSP server."""
cmdline_lower = cmdline.lower()
# Metals is a Scala/Java application
if "java" not in cmdline_lower:
return False
# Look for Metals-specific identifiers
return any(
marker in cmdline_lower
for marker in [
"metals",
"org.scalameta",
"-dmetals.client",
]
)
def _is_port_in_use_by_metals(port: int) -> bool:
"""Check if the given port is in use by a Metals process."""
try:
import psutil
for conn in psutil.net_connections(kind="tcp"):
if conn.laddr.port == port and conn.status == "LISTEN":
try:
proc = psutil.Process(conn.pid)
cmdline = " ".join(proc.cmdline()).lower()
if _is_metals_cmdline(cmdline):
return True
except (psutil.NoSuchProcess, psutil.AccessDenied):
pass
return False
except (psutil.AccessDenied, OSError) as e:
# On some systems, net_connections requires elevated privileges
log.debug(f"Could not check port {port}: {e}")
return False
def check_metals_db_status(project_path: Path) -> tuple[MetalsDbStatus, MetalsLockInfo | None]:
"""
Check the status of the Metals H2 database for a project.
This function determines whether it's safe to start a new Metals instance
and whether any cleanup is needed.
Args:
project_path: Path to the project root directory
Returns:
A tuple of (status, lock_info) where lock_info is populated for
ACTIVE_INSTANCE and STALE_LOCK statuses.
"""
metals_dir = project_path / ".metals"
db_path = metals_dir / "metals.mv.db"
lock_path = metals_dir / "metals.mv.db.lock.db"
if not metals_dir.exists():
log.debug(f"No .metals directory found at {metals_dir}")
return MetalsDbStatus.NO_DATABASE, None
if not db_path.exists():
log.debug(f"No Metals database found at {db_path}")
return MetalsDbStatus.NO_DATABASE, None
if not lock_path.exists():
log.debug(f"Metals database exists but no lock file at {lock_path}")
return MetalsDbStatus.NO_LOCK, None
# Lock file exists - parse it to determine status
lock_info = parse_h2_lock_file(lock_path)
if lock_info is None:
# Lock file exists but couldn't be read - treat as stale
log.warning(f"Could not read lock file at {lock_path}, treating as stale")
return MetalsDbStatus.STALE_LOCK, MetalsLockInfo(
pid=None,
port=None,
lock_path=lock_path,
is_stale=True,
raw_content="<unreadable>",
)
if lock_info.is_stale:
log.debug(f"Stale Metals lock detected: {lock_info}")
return MetalsDbStatus.STALE_LOCK, lock_info
else:
log.debug(f"Active Metals instance detected: {lock_info}")
return MetalsDbStatus.ACTIVE_INSTANCE, lock_info
def cleanup_stale_lock(lock_path: Path) -> bool:
"""
Remove a stale H2 database lock file.
This should only be called when we've verified the owning process is dead.
Removing a lock file from a running process could cause database corruption.
Args:
lock_path: Path to the .lock.db file to remove
Returns:
True if cleanup succeeded, False otherwise.
"""
if not lock_path.exists():
log.debug(f"Lock file already removed: {lock_path}")
return True
try:
lock_path.unlink()
log.info(f"Cleaned up stale Metals lock file: {lock_path}")
return True
except PermissionError as e:
log.warning(f"Permission denied removing stale lock file {lock_path}: {e}")
return False
except OSError as e:
log.warning(f"Could not remove stale lock file {lock_path}: {e}")
return False
test/conftest.py
+35 -3
View File
@@ -1,5 +1,7 @@
import logging
import os
import platform
import shutil as _sh
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
@@ -34,8 +36,17 @@ class LanguageParamRequest:
param: Language
_LANGUAGE_REPO_ALIASES: dict[Language, Language] = {
Language.CPP_CCLS: Language.CPP,
Language.PHP_PHPACTOR: Language.PHP,
Language.PYTHON_JEDI: Language.PYTHON,
Language.RUBY_SOLARGRAPH: Language.RUBY,
}
def get_repo_path(language: Language) -> Path:
return Path(__file__).parent / "resources" / "repos" / language / "test_repo"
repo_language = _LANGUAGE_REPO_ALIASES.get(language, language)
return Path(__file__).parent / "resources" / "repos" / repo_language / "test_repo"
def _create_ls(
@@ -52,7 +63,11 @@ def _create_ls(
gitignore_parser = GitignoreParser(str(repo_path))
for spec in gitignore_parser.get_ignore_specs():
ignored_paths.extend(spec.patterns)
config = LanguageServerConfig(code_language=language, ignored_paths=ignored_paths, trace_lsp_communication=trace_lsp_communication)
config = LanguageServerConfig(
code_language=language,
ignored_paths=ignored_paths,
trace_lsp_communication=trace_lsp_communication,
)
effective_solidlsp_dir = solidlsp_dir if solidlsp_dir is not None else SerenaPaths().serena_user_home_dir
return SolidLanguageServer.create(
config,
@@ -102,7 +117,7 @@ def start_default_ls_context(language: Language) -> Iterator[SolidLanguageServer
def _create_default_project(language: Language) -> Project:
repo_path = str(get_repo_path(language))
return Project.load(repo_path)
return Project.load(repo_path, serena_config=None)
@pytest.fixture(scope="session")
@@ -195,6 +210,8 @@ is_ci = os.getenv("CI") == "true" or os.getenv("GITHUB_ACTIONS") == "true"
Flag indicating whether the tests are running in the GitHub CI environment.
"""
is_windows = platform.system() == "Windows"
def _determine_disabled_languages() -> list[Language]:
"""
@@ -212,6 +229,21 @@ def _determine_disabled_languages() -> list[Language]:
if not clojure_tests_enabled:
result.append(Language.CLOJURE)
# Disable CPP_CCLS tests if ccls is not available
ccls_tests_enabled = _sh.which("ccls") is not None
if not ccls_tests_enabled:
result.append(Language.CPP_CCLS)
# Disable CPP (clangd) tests if clangd is not available
clangd_tests_enabled = _sh.which("clangd") is not None
if not clangd_tests_enabled:
result.append(Language.CPP)
# Disable PHP_PHPACTOR tests if php is not available
php_tests_enabled = _sh.which("php") is not None
if not php_tests_enabled:
result.append(Language.PHP_PHPACTOR)
al_tests_enabled = True
if not al_tests_enabled:
result.append(Language.AL)
test/resources/repos/cpp/test_repo/a.cpp
+6
View File
@@ -0,0 +1,6 @@
#include "b.hpp"
int main() {
int x = add(3, 4);
return x;
}
test/resources/repos/cpp/test_repo/b.cpp
+5
View File
@@ -0,0 +1,5 @@
#include "b.hpp"
int add(int a, int b) {
return a + b;
}
test/resources/repos/cpp/test_repo/b.hpp
+3
View File
@@ -0,0 +1,3 @@
#pragma once
int add(int a, int b);
test/resources/repos/cpp/test_repo/compile_commands.json
+12
View File
@@ -0,0 +1,12 @@
[
{
"directory": ".",
"command": "g++ -std=c++17 -I . -c a.cpp",
"file": "a.cpp"
},
{
"directory": ".",
"command": "g++ -std=c++17 -I . -c b.cpp",
"file": "b.cpp"
}
]
test/serena/config/test_global_ignored_paths.py
+261
View File
@@ -0,0 +1,261 @@
import os
import shutil
import tempfile
from pathlib import Path
from serena.config.serena_config import ProjectConfig, RegisteredProject, SerenaConfig
from serena.project import Project
from solidlsp.ls_config import Language
def _create_test_project(
project_root: Path,
project_ignored_paths: list[str] | None = None,
global_ignored_paths: list[str] | None = None,
) -> Project:
"""Helper to create a Project with the given ignored paths configuration."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=project_ignored_paths or [],
ignore_all_files_in_gitignore=False,
)
serena_config: SerenaConfig | None = None
if global_ignored_paths:
serena_config = SerenaConfig(gui_log_window=False, web_dashboard=False, ignored_paths=global_ignored_paths)
return Project(
project_root=str(project_root),
project_config=config,
serena_config=serena_config,
)
class TestGlobalIgnoredPaths:
"""Tests for system-global ignored_paths feature."""
def setup_method(self) -> None:
self.test_dir = tempfile.mkdtemp()
self.project_path = Path(self.test_dir)
# Create some test files and directories
(self.project_path / "main.py").write_text("print('hello')")
os.makedirs(self.project_path / "node_modules" / "pkg", exist_ok=True)
(self.project_path / "node_modules" / "pkg" / "index.js").write_text("module.exports = {}")
os.makedirs(self.project_path / "build", exist_ok=True)
(self.project_path / "build" / "output.js").write_text("compiled")
os.makedirs(self.project_path / "src", exist_ok=True)
(self.project_path / "src" / "app.py").write_text("def app(): pass")
(self.project_path / "debug.log").write_text("log data")
def teardown_method(self) -> None:
shutil.rmtree(self.test_dir)
def test_global_ignored_paths_are_applied(self) -> None:
"""Global ignored_paths from SerenaConfig are respected by Project.is_ignored_path()."""
project = _create_test_project(
self.project_path,
global_ignored_paths=["node_modules"],
)
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg" / "index.js"))
assert not project.is_ignored_path(str(self.project_path / "src" / "app.py"))
def test_additive_merge_of_global_and_project_patterns(self) -> None:
"""Global + project patterns are merged additively (both applied)."""
project = _create_test_project(
self.project_path,
project_ignored_paths=["build"],
global_ignored_paths=["node_modules"],
)
# Global pattern should be applied
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg" / "index.js"))
# Project pattern should also be applied
assert project.is_ignored_path(str(self.project_path / "build" / "output.js"))
# Non-ignored files should not be affected
assert not project.is_ignored_path(str(self.project_path / "src" / "app.py"))
def test_empty_global_ignored_paths_has_no_effect(self) -> None:
"""Empty global ignored_paths (default) has no effect on existing behavior."""
project = _create_test_project(
self.project_path,
project_ignored_paths=["build"],
global_ignored_paths=[],
)
# Project pattern still works
assert project.is_ignored_path(str(self.project_path / "build" / "output.js"))
# Non-ignored files still accessible
assert not project.is_ignored_path(str(self.project_path / "node_modules" / "pkg" / "index.js"))
def test_default_global_ignored_paths_backward_compatible(self) -> None:
"""Project created without global_ignored_paths parameter works as before."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=["build"],
ignore_all_files_in_gitignore=False,
)
project = Project(
project_root=str(self.project_path),
project_config=config,
)
assert project.is_ignored_path(str(self.project_path / "build" / "output.js"))
assert not project.is_ignored_path(str(self.project_path / "node_modules" / "pkg" / "index.js"))
def test_duplicate_patterns_across_global_and_project(self) -> None:
"""Duplicate patterns across global and project do not cause errors."""
project = _create_test_project(
self.project_path,
project_ignored_paths=["node_modules", "build"],
global_ignored_paths=["node_modules", "build"],
)
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg" / "index.js"))
assert project.is_ignored_path(str(self.project_path / "build" / "output.js"))
assert not project.is_ignored_path(str(self.project_path / "src" / "app.py"))
def test_glob_patterns_in_global_ignored_paths(self) -> None:
"""Global ignored_paths support gitignore-style glob patterns."""
project = _create_test_project(
self.project_path,
global_ignored_paths=["*.log"],
)
assert project.is_ignored_path(str(self.project_path / "debug.log"))
assert not project.is_ignored_path(str(self.project_path / "main.py"))
class TestRegisteredProjectGlobalIgnoredPaths:
"""RegisteredProject.get_project_instance() correctly passes global patterns to Project."""
def setup_method(self) -> None:
self.test_dir = tempfile.mkdtemp()
self.project_path = Path(self.test_dir).resolve()
(self.project_path / "main.py").write_text("print('hello')")
os.makedirs(self.project_path / "node_modules", exist_ok=True)
(self.project_path / "node_modules" / "pkg.js").write_text("module")
def teardown_method(self) -> None:
shutil.rmtree(self.test_dir)
def test_get_project_instance_passes_global_ignored_paths(self) -> None:
"""RegisteredProject.get_project_instance() passes global_ignored_paths to Project."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=[],
ignore_all_files_in_gitignore=False,
)
serena_config = SerenaConfig(gui_log_window=False, web_dashboard=False, ignored_paths=["node_modules"])
registered = RegisteredProject(
project_root=str(self.project_path),
project_config=config,
)
project = registered.get_project_instance(serena_config=serena_config)
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg.js"))
def test_get_project_instance_without_global_ignored_paths(self) -> None:
"""RegisteredProject without global_ignored_paths defaults to empty."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=[],
ignore_all_files_in_gitignore=False,
)
registered = RegisteredProject(
project_root=str(self.project_path),
project_config=config,
)
project = registered.get_project_instance(serena_config=None)
assert not project.is_ignored_path(str(self.project_path / "node_modules" / "pkg.js"))
def test_from_project_root_passes_global_ignored_paths(self) -> None:
"""RegisteredProject.from_project_root() threads global_ignored_paths to Project."""
# Create a minimal project.yml so from_project_root can load config
serena_dir = self.project_path / ".serena"
serena_dir.mkdir(exist_ok=True)
(serena_dir / "project.yml").write_text(
'project_name: "test_project"\nlanguages: ["python"]\nignored_paths: []\nignore_all_files_in_gitignore: false\n'
)
serena_config = SerenaConfig(gui_log_window=False, web_dashboard=False, ignored_paths=["node_modules"])
registered = RegisteredProject.from_project_root(
str(self.project_path),
)
project = registered.get_project_instance(serena_config=serena_config)
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg.js"))
def test_from_project_instance_passes_global_ignored_paths(self) -> None:
"""RegisteredProject.from_project_instance() threads global_ignored_paths to Project."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=[],
ignore_all_files_in_gitignore=False,
)
serena_config = SerenaConfig(gui_log_window=False, web_dashboard=False, ignored_paths=["node_modules"])
project = Project(
project_root=str(self.project_path),
project_config=config,
serena_config=serena_config,
)
registered = RegisteredProject.from_project_instance(project)
# The registered project already has a project_instance, so get_project_instance() returns it directly
retrieved = registered.get_project_instance(serena_config=serena_config)
assert retrieved.is_ignored_path(str(self.project_path / "node_modules" / "pkg.js"))
class TestGlobalIgnoredPathsWithGitignore:
"""Global ignored_paths combined with ignore_all_files_in_gitignore produces correct three-way merge."""
def setup_method(self) -> None:
self.test_dir = tempfile.mkdtemp()
self.project_path = Path(self.test_dir).resolve()
# Create test files
(self.project_path / "main.py").write_text("print('hello')")
os.makedirs(self.project_path / "node_modules", exist_ok=True)
(self.project_path / "node_modules" / "pkg.js").write_text("module")
os.makedirs(self.project_path / "dist", exist_ok=True)
(self.project_path / "dist" / "bundle.js").write_text("bundled")
os.makedirs(self.project_path / "build", exist_ok=True)
(self.project_path / "build" / "output.js").write_text("compiled")
# Create .gitignore that ignores dist/
(self.project_path / ".gitignore").write_text("dist/\n")
def teardown_method(self) -> None:
shutil.rmtree(self.test_dir)
def test_three_way_merge_global_project_and_gitignore(self) -> None:
"""Global patterns, project patterns, and .gitignore patterns are all applied together."""
config = ProjectConfig(
project_name="test_project",
languages=[Language.PYTHON],
ignored_paths=["build"],
ignore_all_files_in_gitignore=True,
)
serena_config = SerenaConfig(gui_log_window=False, web_dashboard=False, ignored_paths=["node_modules"])
project = Project(
project_root=str(self.project_path),
project_config=config,
serena_config=serena_config,
)
# Global pattern: node_modules
assert project.is_ignored_path(str(self.project_path / "node_modules" / "pkg.js"))
# Project pattern: build
assert project.is_ignored_path(str(self.project_path / "build" / "output.js"))
# Gitignore pattern: dist/
assert project.is_ignored_path(str(self.project_path / "dist" / "bundle.js"))
# Non-ignored file
assert not project.is_ignored_path(str(self.project_path / "main.py"))
class TestSerenaConfigIgnoredPaths:
"""Config loading with ignored_paths in serena_config.yml works correctly."""
def test_serena_config_default_ignored_paths(self) -> None:
"""SerenaConfig defaults to empty ignored_paths."""
config = SerenaConfig(gui_log_window=False, web_dashboard=False)
assert config.ignored_paths == []
def test_serena_config_with_ignored_paths(self) -> None:
"""SerenaConfig can be created with explicit ignored_paths."""
config = SerenaConfig(
gui_log_window=False,
web_dashboard=False,
ignored_paths=["node_modules", "*.log", "build"],
)
assert config.ignored_paths == ["node_modules", "*.log", "build"]
test/serena/test_cli_project_commands.py
+3 -3
View File
@@ -315,8 +315,8 @@ class TestFindProjectRoot:
finally:
os.chdir(original_cwd)
def test_falls_back_to_cwd_when_no_markers(self, temp_project_dir):
"""Test falls back to CWD when no markers exist within boundary."""
def test_falls_back_to_none_when_no_markers(self, temp_project_dir):
"""Test falls back to None when no markers exist within boundary."""
subdir = os.path.join(temp_project_dir, "src")
os.makedirs(subdir)
@@ -324,7 +324,7 @@ class TestFindProjectRoot:
try:
os.chdir(subdir)
result = find_project_root(root=temp_project_dir)
assert os.path.samefile(result, subdir)
assert result is None
finally:
os.chdir(original_cwd)
test/serena/test_jetbrains_plugin_client.py
+20
View File
@@ -0,0 +1,20 @@
import pytest
from serena.constants import REPO_ROOT
from serena.jetbrains.jetbrains_plugin_client import JetBrainsPluginClient
class TestSerenaJetBrainsPluginClient:
@pytest.mark.parametrize(
"serena_path, plugin_path",
[
(REPO_ROOT, REPO_ROOT),
("/home/user/project", "/home/user/project"),
("/home/user/project", "//wsl.localhost/Ubuntu-24.04/home/user/project"),
("/home/user/project", "//wsl$/Ubuntu/home/user/project"),
("/home/user/project", "//wsl$/Ubuntu/home/user/project"),
("/mnt/c/Users/user/projects/my-app", "/workspaces/serena/C:/Users/user/projects/my-app"),
],
)
def test_path_matching(self, serena_path, plugin_path) -> None:
assert JetBrainsPluginClient._paths_match(serena_path, plugin_path)
test/serena/test_serena_agent.py
+36 -8
View File
@@ -15,7 +15,7 @@ from serena.project import Project
from serena.tools import SUCCESS_RESULT, FindReferencingSymbolsTool, FindSymbolTool, ReplaceContentTool, ReplaceSymbolBodyTool
from solidlsp.ls_config import Language
from solidlsp.ls_types import SymbolKind
from test.conftest import get_repo_path, language_tests_enabled
from test.conftest import get_repo_path, is_ci, language_tests_enabled
from test.solidlsp import clojure as clj
@@ -36,6 +36,7 @@ def serena_config():
Language.CLOJURE,
Language.FSHARP,
Language.POWERSHELL,
Language.CPP_CCLS,
]:
repo_path = get_repo_path(language)
if repo_path.exists():
@@ -118,13 +119,20 @@ class TestSerenaAgent:
pytest.param(Language.CSHARP, "Calculator", "Class", "Program.cs", marks=pytest.mark.csharp),
pytest.param(Language.FSHARP, "Calculator", "Module", "Calculator.fs", marks=pytest.mark.fsharp),
pytest.param(Language.POWERSHELL, "function Greet-User ()", "Function", "main.ps1", marks=pytest.mark.powershell),
pytest.param(Language.CPP_CCLS, "add", "Function", "b.cpp", marks=pytest.mark.cpp),
],
indirect=["serena_agent"],
)
def test_find_symbol(self, serena_agent: SerenaAgent, symbol_name: str, expected_kind: str, expected_file: str):
# skip flaky tests in CI
# TODO: Revisit the flaky tests and re-enable once the LS issues are resolved #1039
flaky_languages = {Language.FSHARP, Language.RUST}
if set(serena_agent.get_active_lsp_languages()).intersection(flaky_languages) and is_ci:
pytest.skip("Test is flaky and thus skipped in CI environment.")
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply_ex(name_path_pattern=symbol_name, include_info=True)
result = find_symbol_tool.apply(name_path_pattern=symbol_name, include_info=True)
symbols = json.loads(result)
assert any(
@@ -186,15 +194,22 @@ class TestSerenaAgent:
pytest.param(Language.CSHARP, "Calculator", "Program.cs", "Program.cs", marks=pytest.mark.csharp),
pytest.param(Language.FSHARP, "add", "Calculator.fs", "Program.fs", marks=pytest.mark.fsharp),
pytest.param(Language.POWERSHELL, "function Greet-User ()", "main.ps1", "main.ps1", marks=pytest.mark.powershell),
pytest.param(Language.CPP_CCLS, "add", "b.cpp", "a.cpp", marks=pytest.mark.cpp),
],
indirect=["serena_agent"],
)
def test_find_symbol_references(self, serena_agent: SerenaAgent, symbol_name: str, def_file: str, ref_file: str) -> None:
# skip flaky tests in CI
# TODO: Revisit the flaky tests and re-enable once the LS issues are resolved #1039
flaky_languages = {Language.TYPESCRIPT}
if set(serena_agent.get_active_lsp_languages()).intersection(flaky_languages) and is_ci:
pytest.skip("Test is flaky and thus skipped in CI environment.")
agent = serena_agent
# Find the symbol location first
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply_ex(name_path_pattern=symbol_name, relative_path=def_file)
result = find_symbol_tool.apply(name_path_pattern=symbol_name, relative_path=def_file)
time.sleep(1)
symbols = json.loads(result)
@@ -203,12 +218,26 @@ class TestSerenaAgent:
# Now find references
find_refs_tool = agent.get_tool(FindReferencingSymbolsTool)
result = find_refs_tool.apply_ex(name_path=def_symbol["name_path"], relative_path=def_symbol["relative_path"])
result = find_refs_tool.apply(name_path=def_symbol["name_path"], relative_path=def_symbol["relative_path"])
def contains_ref_with_relative_path(refs, relative_path):
"""
Checks for reference to relative path, regardless of output format (grouped an ungrouped)
"""
if isinstance(refs, list):
for ref in refs:
if contains_ref_with_relative_path(ref, relative_path):
return True
elif isinstance(refs, dict):
if relative_path in refs:
return True
for value in refs.values():
if contains_ref_with_relative_path(value, relative_path):
return True
return False
refs = json.loads(result)
assert any(
ref["relative_path"] == ref_file for ref in refs
), f"Expected to find reference to {symbol_name} in {ref_file}. refs={refs}"
assert contains_ref_with_relative_path(refs, ref_file), f"Expected to find reference to {symbol_name} in {ref_file}. refs={refs}"
@pytest.mark.parametrize(
"serena_agent,name_path,substring_matching,expected_symbol_name,expected_kind,expected_file",
@@ -446,7 +475,6 @@ class TestSerenaAgent:
needle = r'console.log("WebSocketManager initializing\nStatus OK");'
repl = r'console.log("WebSocketManager initialized\nAll systems go!");'
replace_content_tool = serena_agent.get_tool(ReplaceContentTool)
mode: Literal["literal", "regex"]
with project_file_modification_context(serena_agent, relative_path):
result = replace_content_tool.apply(
needle=re.escape(needle) if mode == "regex" else needle,
test/serena/test_set_modes.py
+75
View File
@@ -0,0 +1,75 @@
"""Tests for SerenaAgent.set_modes() to verify that mode switching works correctly."""
import logging
from serena.agent import SerenaAgent
from serena.config.serena_config import ModeSelectionDefinition, SerenaConfig
class TestSetModes:
"""Test that set_modes correctly changes active modes."""
def _create_agent(self, modes: ModeSelectionDefinition | None = None) -> SerenaAgent:
config = SerenaConfig(gui_log_window=False, web_dashboard=False, log_level=logging.ERROR)
return SerenaAgent(serena_config=config, modes=modes)
def test_set_modes_changes_active_modes(self) -> None:
"""Test that calling set_modes actually changes the active modes."""
agent = self._create_agent(modes=ModeSelectionDefinition(default_modes=["editing", "interactive"]))
initial_mode_names = sorted(m.name for m in agent.get_active_modes())
assert "editing" in initial_mode_names
assert "interactive" in initial_mode_names
# Switch to planning mode
agent.set_modes(["planning", "interactive"])
new_mode_names = sorted(m.name for m in agent.get_active_modes())
assert "planning" in new_mode_names
assert "interactive" in new_mode_names
assert "editing" not in new_mode_names
def test_set_modes_overrides_config_defaults(self) -> None:
"""Test that set_modes takes precedence over config defaults."""
config = SerenaConfig(gui_log_window=False, web_dashboard=False, log_level=logging.ERROR)
config.default_modes = ["editing", "interactive"]
agent = SerenaAgent(serena_config=config)
# Verify config defaults are active
initial_mode_names = [m.name for m in agent.get_active_modes()]
assert "editing" in initial_mode_names
# Switch modes — should override config defaults
agent.set_modes(["planning", "one-shot"])
new_mode_names = [m.name for m in agent.get_active_modes()]
assert "planning" in new_mode_names
assert "one-shot" in new_mode_names
assert "editing" not in new_mode_names
def test_set_modes_persists_after_repeated_calls(self) -> None:
"""Test that set_modes result persists (modes don't revert)."""
agent = self._create_agent(modes=ModeSelectionDefinition(default_modes=["editing"]))
agent.set_modes(["planning"])
mode_names_1 = [m.name for m in agent.get_active_modes()]
assert "planning" in mode_names_1
# Call get_active_modes again — should still be planning
mode_names_2 = [m.name for m in agent.get_active_modes()]
assert mode_names_1 == mode_names_2
def test_set_modes_can_switch_back(self) -> None:
"""Test that modes can be switched back to original after switching away."""
agent = self._create_agent(modes=ModeSelectionDefinition(default_modes=["editing", "interactive"]))
# Switch away
agent.set_modes(["planning", "one-shot"])
assert "planning" in [m.name for m in agent.get_active_modes()]
# Switch back
agent.set_modes(["editing", "interactive"])
mode_names = [m.name for m in agent.get_active_modes()]
assert "editing" in mode_names
assert "interactive" in mode_names
assert "planning" not in mode_names
test/serena/test_symbol.py
+23 -1
View File
@@ -2,7 +2,8 @@ from unittest.mock import MagicMock
import pytest
from serena.symbol import LanguageServerSymbolRetriever, NamePathComponent, NamePathMatcher
from serena.jetbrains.jetbrains_types import SymbolDTO, SymbolDTOKey
from serena.symbol import LanguageServerSymbol, LanguageServerSymbolRetriever, NamePathComponent, NamePathMatcher
from solidlsp import SolidLanguageServer
from solidlsp.ls_config import Language
@@ -242,6 +243,27 @@ class TestLanguageServerSymbolRetriever:
assert "Create a new user and store it" in create_user_method_symbol_info
class TestSymbolDictTypes:
@staticmethod
def check_key_type(dict_type: type, key_type: type):
"""
:param dict_type: a TypedDict type
:param key_type: the corresponding key type (Literal[...]) that the dict should have for keys
"""
dict_type_keys = dict_type.__annotations__.keys()
assert len(dict_type_keys) == len(
key_type.__args__ # type: ignore
), f"Expected {len(key_type.__args__)} keys in {dict_type}, but got {len(dict_type_keys)}" # type: ignore
for expected_key in key_type.__args__: # type: ignore
assert expected_key in dict_type_keys, f"Expected key '{expected_key}' not found in {dict_type}"
def test_ls_symbol_dict_type(self):
self.check_key_type(LanguageServerSymbol.OutputDict, LanguageServerSymbol.OutputDictKey)
def test_jb_symbol_dict_type(self):
self.check_key_type(SymbolDTO, SymbolDTOKey)
def _make_mock_symbols(count: int, *, relative_path: str = "test_repo/services.py") -> list[MagicMock]:
symbols: list[MagicMock] = []
for i in range(count):
test/serena/test_symbol_editing.py
+1 -1
View File
@@ -218,7 +218,7 @@ class EditingTest(ABC):
with open(file_path, encoding="utf-8") as f:
return f.read()
def run_test(self, content_after_ground_truth: str | SnapshotAssertion) -> None:
def run_test(self, content_after_ground_truth: SnapshotAssertion) -> None:
with self._setup() as symbol_retriever:
content_before = self._read_file(self.rel_path)
code_editor = LanguageServerCodeEditor(symbol_retriever)
test/solidlsp/cpp/__init__.py
View File
Whitespace-only changes.
test/solidlsp/cpp/test_cpp_basic.py
+144
View File
@@ -0,0 +1,144 @@
"""
Basic tests for C/C++ language server integration (clangd and ccls).
This module tests both Language.CPP (clangd) and Language.CPP_CCLS (ccls)
using the same test repository. Tests are skipped if the respective language
server is not available.
"""
import os
import pathlib
import shutil
import pytest
from solidlsp import SolidLanguageServer
from solidlsp.ls_config import Language
from solidlsp.ls_utils import SymbolUtils
def _ccls_available() -> bool:
return shutil.which("ccls") is not None
_cpp_servers: list[Language] = [Language.CPP]
if _ccls_available():
_cpp_servers.append(Language.CPP_CCLS)
@pytest.mark.cpp
@pytest.mark.skipif(not _cpp_servers, reason="No C++ language server (clangd or ccls) available")
class TestCppLanguageServer:
"""Tests for C/C++ language servers (clangd and ccls)."""
@pytest.mark.parametrize("language_server", _cpp_servers, indirect=True)
def test_find_symbol(self, language_server: SolidLanguageServer) -> None:
"""Test that symbol tree contains expected functions."""
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "add"), "Function 'add' not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "main"), "Function 'main' not found in symbol tree"
@pytest.mark.parametrize("language_server", _cpp_servers, indirect=True)
def test_get_document_symbols(self, language_server: SolidLanguageServer) -> None:
"""Test document symbols for a.cpp."""
file_path = os.path.join("a.cpp")
symbols = language_server.request_document_symbols(file_path).get_all_symbols_and_roots()
# Flatten nested structure if needed
symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols
names = [s.get("name") for s in symbol_list]
assert "main" in names, f"Expected 'main' in document symbols, got: {names}"
@pytest.mark.parametrize("language_server", _cpp_servers, indirect=True)
def test_find_referencing_symbols_across_files(self, language_server: SolidLanguageServer) -> None:
"""Test finding references to 'add' function across files."""
# Locate 'add' in b.cpp
file_path = os.path.join("b.cpp")
symbols = language_server.request_document_symbols(file_path).get_all_symbols_and_roots()
symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols
add_symbol = None
for sym in symbol_list:
if sym.get("name") == "add":
add_symbol = sym
break
assert add_symbol is not None, "Could not find 'add' function symbol in b.cpp"
sel_start = add_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
ref_files = [ref.get("relativePath", "") for ref in refs]
assert any("a.cpp" in ref_file for ref_file in ref_files), f"Should find reference in a.cpp, {refs=}"
# Verify second call returns same results (stability check)
def _ref_key(ref: dict) -> tuple:
rp = ref.get("relativePath", "")
rng = ref.get("range") or {}
s = rng.get("start") or {}
e = rng.get("end") or {}
return (
rp,
s.get("line", -1),
s.get("character", -1),
e.get("line", -1),
e.get("character", -1),
)
refs2 = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert sorted(map(_ref_key, refs2)) == sorted(map(_ref_key, refs)), "Reference results should be stable across calls"
@pytest.mark.parametrize("language_server", _cpp_servers, indirect=True)
@pytest.mark.xfail(
strict=True,
reason=("Both clangd and ccls do not support cross-file references for newly created files that were never opened by the LS."),
)
def test_find_references_in_newly_written_file(self, language_server: SolidLanguageServer) -> None:
# Create a new file that references the 'add' function from b.cpp
new_file_path = os.path.join("temp_new_file.cpp")
new_file_abs_path = os.path.join(language_server.repository_root_path, new_file_path)
try:
# Write the new file with a reference to add()
with open(new_file_abs_path, "w", encoding="utf-8") as f:
f.write(
"""
#include "b.hpp"
int use_add() {
int result = add(5, 3);
return result;
}
"""
)
# Open the new file so clangd knows about it
with language_server.open_file(new_file_path):
# Request document symbols to ensure the file is fully loaded by clangd
new_file_symbols = language_server.request_document_symbols(new_file_path).get_all_symbols_and_roots()
assert new_file_symbols, "New file should have symbols"
# Verify the file stays in open_file_buffers after the context exits
uri = pathlib.Path(new_file_abs_path).as_uri()
assert uri in language_server.open_file_buffers, "File should remain in open_file_buffers"
# Find the 'add' symbol in b.cpp
b_file_path = os.path.join("b.cpp")
symbols = language_server.request_document_symbols(b_file_path).get_all_symbols_and_roots()
symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols
add_symbol = None
for sym in symbol_list:
if sym.get("name") == "add":
add_symbol = sym
break
assert add_symbol is not None, "Could not find 'add' function symbol in b.cpp"
# Request references for 'add'
sel_start = add_symbol["selectionRange"]["start"]
refs = language_server.request_references(b_file_path, sel_start["line"], sel_start["character"])
ref_files = [ref.get("relativePath", "") for ref in refs]
# Should find reference in the newly written file
assert any(
"temp_new_file.cpp" in ref_file for ref_file in ref_files
), f"Should find reference in newly written temp_new_file.cpp, {ref_files=}"
finally:
# Clean up the new file
if os.path.exists(new_file_abs_path):
os.remove(new_file_abs_path)
test/solidlsp/csharp/test_csharp_basic.py
+56 -9
View File
@@ -56,6 +56,7 @@ class TestCSharpLanguageServer:
# Handle nested symbol structure
symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols
for sym in symbol_list:
# Symbol names are normalized to base form (e.g., "Add" not "Add(int, int) : int")
if sym.get("name") == "Add":
add_symbol = sym
break
@@ -82,13 +83,13 @@ class TestCSharpLanguageServer:
# Check that we have the Person class
assert any(s.get("name") == "Person" and s.get("kind") == 5 for s in symbols)
# Check for properties and methods
# Check for properties and methods (names are normalized to base form)
symbol_names = [s.get("name") for s in symbols]
assert "Name" in symbol_names
assert "Age" in symbol_names
assert "Email" in symbol_names
assert "ToString" in symbol_names
assert "IsAdult" in symbol_names
assert "Name" in symbol_names, "Name property not found"
assert "Age" in symbol_names, "Age property not found"
assert "Email" in symbol_names, "Email property not found"
assert "ToString" in symbol_names, "ToString method not found"
assert "IsAdult" in symbol_names, "IsAdult method not found"
@pytest.mark.parametrize("language_server", [Language.CSHARP], indirect=True)
def test_find_referencing_symbols_across_files(self, language_server: SolidLanguageServer) -> None:
@@ -102,6 +103,7 @@ class TestCSharpLanguageServer:
subtract_symbol = None
for sym in symbol_list:
# Symbol names are normalized to base form (e.g., "Subtract" not "Subtract(int, int) : int")
if sym.get("name") == "Subtract":
subtract_symbol = sym
break
@@ -112,21 +114,66 @@ class TestCSharpLanguageServer:
sel_start = subtract_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"] + 1)
# Should find references in both Program.cs and Models/Person.cs
# Should find references where the method is called
ref_files = cast(list[str], [ref.get("relativePath", "") for ref in refs])
print(f"Found references: {refs}")
print(f"Reference files: {ref_files}")
# Check that we have references from both files
assert any("Program.cs" in ref_file for ref_file in ref_files), "Should find reference in Program.cs"
# Check that we have reference in Models/Person.cs where Calculator.Subtract is called
# Note: New Roslyn version doesn't include the definition itself as a reference (more correct behavior)
assert any(
os.path.join("Models", "Person.cs") in ref_file for ref_file in ref_files
), "Should find reference in Models/Person.cs where Calculator.Subtract is called"
assert len(refs) > 0, "Should find at least one reference"
# check for a second time, since the first call may trigger initialization and change the state of the LS
refs_second_call = language_server.request_references(file_path, sel_start["line"], sel_start["character"] + 1)
assert refs_second_call == refs, "Second call to request_references should return the same results"
@pytest.mark.parametrize("language_server", [Language.CSHARP], indirect=True)
def test_hover_includes_type_information(self, language_server: SolidLanguageServer) -> None:
"""Test that hover information is available and includes type information."""
file_path = os.path.join("Models", "Person.cs")
# Open the file first
language_server.open_file(file_path)
# Test 1: Hover over the Name property (line 6, column 23 - on "Name")
# Source: public string Name { get; set; }
hover_info = language_server.request_hover(file_path, 6, 23)
# Verify hover returns content
assert hover_info is not None, "Hover should return information for Name property"
assert isinstance(hover_info, dict), "Hover should be a dict"
assert "contents" in hover_info, "Hover should have contents"
contents = hover_info["contents"]
assert isinstance(contents, dict), "Hover contents should be a dict"
assert "value" in contents, "Hover contents should have value"
hover_text = contents["value"]
# Verify the hover contains property signature with type
assert "string" in hover_text, f"Hover should include 'string' type, got: {hover_text}"
assert "Name" in hover_text, f"Hover should include 'Name' property name, got: {hover_text}"
# Test 2: Hover over the IsAdult method (line 22, column 21 - on "IsAdult")
# Source: public bool IsAdult()
hover_method = language_server.request_hover(file_path, 22, 21)
# Verify method hover returns content
assert hover_method is not None, "Hover should return information for IsAdult method"
assert isinstance(hover_method, dict), "Hover should be a dict"
assert "contents" in hover_method, "Hover should have contents"
contents = hover_method["contents"]
assert isinstance(contents, dict), "Hover contents should be a dict"
assert "value" in contents, "Hover contents should have value"
method_hover_text = contents["value"]
# Verify the hover contains method signature with return type
assert "bool" in method_hover_text, f"Hover should include 'bool' return type, got: {method_hover_text}"
assert "IsAdult" in method_hover_text, f"Hover should include 'IsAdult' method name, got: {method_hover_text}"
@pytest.mark.csharp
class TestCSharpSolutionProjectOpening:
test/solidlsp/csharp/test_csharp_dotnet_install.py
+290
View File
@@ -0,0 +1,290 @@
"""Tests for C# language server .NET runtime installation using official install scripts."""
import tempfile
from pathlib import Path
from unittest.mock import Mock, patch
import pytest
from solidlsp.language_servers.csharp_language_server import CSharpLanguageServer
from solidlsp.settings import SolidLSPSettings
@pytest.mark.csharp
class TestDotNetInstallScript:
"""Test .NET runtime installation using Microsoft's official install scripts."""
@patch("solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server")
@patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id")
@patch("solidlsp.language_servers.csharp_language_server.platform.system")
@patch("solidlsp.language_servers.csharp_language_server.subprocess.run")
@patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve")
@patch("solidlsp.language_servers.csharp_language_server.shutil.which")
def test_install_dotnet_uses_bash_script_on_linux(
self, mock_which, mock_urlretrieve, mock_subprocess, mock_platform, mock_platform_id, mock_ensure_ls
):
"""Test that Linux uses bash install script."""
from solidlsp.ls_utils import PlatformId
mock_platform.return_value = "Linux"
mock_platform_id.return_value = PlatformId.LINUX_x64
mock_which.return_value = None # No system dotnet
mock_ensure_ls.return_value = "/fake/server.dll"
with tempfile.TemporaryDirectory() as temp_dir:
def mock_retrieve(url, path):
# Create the file that urlretrieve would create
Path(path).touch()
# Also create the dotnet executable after script download
if "dotnet-install.sh" in str(path):
dotnet_dir = Path(temp_dir) / "dotnet-runtime-10.0"
dotnet_dir.mkdir(parents=True, exist_ok=True)
dotnet_exe = dotnet_dir / "dotnet"
dotnet_exe.touch()
dotnet_exe.chmod(0o755)
mock_urlretrieve.side_effect = mock_retrieve
mock_subprocess.return_value = Mock(returncode=0, stdout="", stderr="")
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
_ = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Verify bash script was downloaded
assert mock_urlretrieve.called
script_url = mock_urlretrieve.call_args[0][0]
assert script_url == "https://dot.net/v1/dotnet-install.sh"
@patch("solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server")
@patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id")
@patch("solidlsp.language_servers.csharp_language_server.platform.system")
@patch("solidlsp.language_servers.csharp_language_server.subprocess.run")
@patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve")
@patch("solidlsp.language_servers.csharp_language_server.shutil.which")
def test_install_dotnet_uses_powershell_script_on_windows(
self, mock_which, mock_urlretrieve, mock_subprocess, mock_platform, mock_platform_id, mock_ensure_ls
):
"""Test that Windows uses PowerShell install script."""
from solidlsp.ls_utils import PlatformId
mock_platform.return_value = "Windows"
mock_platform_id.return_value = PlatformId.WIN_x64
mock_which.return_value = None # No system dotnet
mock_ensure_ls.return_value = "/fake/server.dll"
with tempfile.TemporaryDirectory() as temp_dir:
def mock_retrieve(url, path):
# Create the file that urlretrieve would create
Path(path).touch()
# Also create the dotnet executable after script download
if "dotnet-install.ps1" in str(path):
dotnet_dir = Path(temp_dir) / "dotnet-runtime-10.0"
dotnet_dir.mkdir(parents=True, exist_ok=True)
dotnet_exe = dotnet_dir / "dotnet.exe"
dotnet_exe.touch()
mock_urlretrieve.side_effect = mock_retrieve
mock_subprocess.return_value = Mock(returncode=0, stdout="", stderr="")
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
_ = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Verify PowerShell script was downloaded
assert mock_urlretrieve.called
script_url = mock_urlretrieve.call_args[0][0]
assert script_url == "https://dot.net/v1/dotnet-install.ps1"
def test_uses_system_dotnet_10_if_available(self):
"""Test that system .NET 10 is used if available instead of downloading."""
from solidlsp.ls_utils import PlatformId
with patch("solidlsp.language_servers.csharp_language_server.shutil.which") as mock_which:
with patch("solidlsp.language_servers.csharp_language_server.subprocess.run") as mock_subprocess:
with patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id") as mock_platform_id:
with patch(
"solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server"
) as mock_ensure_ls:
mock_which.return_value = "/usr/bin/dotnet"
mock_ensure_ls.return_value = "/fake/server.dll"
mock_platform_id.return_value = PlatformId.LINUX_x64
# Mock dotnet --list-runtimes output showing .NET 10
mock_result = Mock()
mock_result.stdout = "Microsoft.NETCore.App 10.0.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]"
mock_result.returncode = 0
mock_subprocess.return_value = mock_result
with tempfile.TemporaryDirectory() as temp_dir:
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Should have used system dotnet
assert dependency_provider._dotnet_path == "/usr/bin/dotnet"
def test_runtime_dependencies_no_longer_include_dotnet_downloads(self):
"""Test that _RUNTIME_DEPENDENCIES no longer includes manual .NET download entries."""
from solidlsp.language_servers.csharp_language_server import _RUNTIME_DEPENDENCIES
# Check that DotNetRuntime entries don't exist (we use install script instead)
dotnet_deps = [dep for dep in _RUNTIME_DEPENDENCIES if dep.id == "DotNetRuntime"]
# After our changes, there should be no DotNetRuntime entries
# (we'll use the install script instead)
assert len(dotnet_deps) == 0, "DotNetRuntime dependencies should be removed in favor of install script"
@patch("solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server")
@patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id")
@patch("solidlsp.language_servers.csharp_language_server.platform.system")
@patch("solidlsp.language_servers.csharp_language_server.subprocess.run")
@patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve")
@patch("solidlsp.language_servers.csharp_language_server.shutil.which")
def test_cached_dotnet_is_reused(self, mock_which, mock_urlretrieve, mock_subprocess, mock_platform, mock_platform_id, mock_ensure_ls):
"""Test that cached .NET installation is reused without re-downloading."""
from solidlsp.ls_utils import PlatformId
mock_platform.return_value = "Linux"
mock_platform_id.return_value = PlatformId.LINUX_x64
mock_which.return_value = None
mock_ensure_ls.return_value = "/fake/server.dll"
with tempfile.TemporaryDirectory() as temp_dir:
# Pre-create the cached dotnet executable
dotnet_dir = Path(temp_dir) / "dotnet-runtime-10.0"
dotnet_dir.mkdir(parents=True, exist_ok=True)
dotnet_exe = dotnet_dir / "dotnet"
dotnet_exe.touch()
dotnet_exe.chmod(0o755)
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Should use cached dotnet without downloading install script
assert dependency_provider._dotnet_path == str(dotnet_exe)
assert not mock_urlretrieve.called, "Should not download install script when cached dotnet exists"
def test_rejects_dotnet_9_and_installs_dotnet_10(self):
"""Test that .NET 9 is rejected and .NET 10 install is triggered."""
from solidlsp.ls_utils import PlatformId
with patch("solidlsp.language_servers.csharp_language_server.shutil.which") as mock_which:
with patch("solidlsp.language_servers.csharp_language_server.subprocess.run") as mock_subprocess:
with patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id") as mock_platform_id:
with patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve") as mock_urlretrieve:
with patch("solidlsp.language_servers.csharp_language_server.platform.system") as mock_platform:
with patch(
"solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server"
) as mock_ensure_ls:
mock_which.return_value = "/usr/bin/dotnet"
mock_ensure_ls.return_value = "/fake/server.dll"
mock_platform_id.return_value = PlatformId.LINUX_x64
mock_platform.return_value = "Linux"
# Mock dotnet --list-runtimes output showing only .NET 9
mock_result = Mock()
mock_result.stdout = "Microsoft.NETCore.App 9.0.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]"
mock_result.returncode = 0
call_count = 0
def subprocess_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count == 1:
# First call: dotnet --list-runtimes returns .NET 9
return mock_result
else:
# Second call: install script execution
return Mock(returncode=0, stdout="", stderr="")
mock_subprocess.side_effect = subprocess_side_effect
with tempfile.TemporaryDirectory() as temp_dir:
def mock_retrieve(url, path):
Path(path).touch()
if "dotnet-install.sh" in str(path):
dotnet_dir = Path(temp_dir) / "dotnet-runtime-10.0"
dotnet_dir.mkdir(parents=True, exist_ok=True)
dotnet_exe = dotnet_dir / "dotnet"
dotnet_exe.touch()
dotnet_exe.chmod(0o755)
mock_urlretrieve.side_effect = mock_retrieve
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Should have installed .NET 10 (not used system .NET 9)
assert dependency_provider._dotnet_path == str(Path(temp_dir) / "dotnet-runtime-10.0" / "dotnet")
assert mock_urlretrieve.called, "Should download install script when .NET 9 found"
def test_accepts_dotnet_11_and_higher(self):
"""Test that .NET 11+ versions are accepted for forward compatibility."""
from solidlsp.ls_utils import PlatformId
for version in ["11.0.1", "12.0.0", "15.3.2"]:
with patch("solidlsp.language_servers.csharp_language_server.shutil.which") as mock_which:
with patch("solidlsp.language_servers.csharp_language_server.subprocess.run") as mock_subprocess:
with patch("solidlsp.language_servers.common.PlatformUtils.get_platform_id") as mock_platform_id:
with patch(
"solidlsp.language_servers.csharp_language_server.CSharpLanguageServer.DependencyProvider._ensure_language_server"
) as mock_ensure_ls:
mock_which.return_value = "/usr/bin/dotnet"
mock_ensure_ls.return_value = "/fake/server.dll"
mock_platform_id.return_value = PlatformId.LINUX_x64
# Mock dotnet --list-runtimes output showing .NET 11+
mock_result = Mock()
mock_result.stdout = f"Microsoft.NETCore.App {version} [/usr/share/dotnet/shared/Microsoft.NETCore.App]"
mock_result.returncode = 0
mock_subprocess.return_value = mock_result
with tempfile.TemporaryDirectory() as temp_dir:
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Should use system dotnet for .NET 11+
assert dependency_provider._dotnet_path == "/usr/bin/dotnet", f"Should accept .NET {version}"
test/solidlsp/csharp/test_csharp_nuget_download.py
+118
View File
@@ -0,0 +1,118 @@
"""Tests for C# language server NuGet package download from NuGet.org."""
import tempfile
from unittest.mock import patch
import pytest
from solidlsp.language_servers.common import RuntimeDependency
from solidlsp.language_servers.csharp_language_server import CSharpLanguageServer
from solidlsp.settings import SolidLSPSettings
@pytest.mark.csharp
class TestNuGetOrgDownload:
"""Test downloading Roslyn language server packages from NuGet.org."""
def test_download_nuget_package_uses_direct_url(self):
"""Test that _download_nuget_package uses the URL from RuntimeDependency directly."""
with tempfile.TemporaryDirectory() as temp_dir:
# Create a RuntimeDependency with a NuGet.org URL
test_dependency = RuntimeDependency(
id="TestPackage",
description="Test package from NuGet.org",
package_name="roslyn-language-server.linux-x64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.linux-x64/5.5.0-2.26078.4",
platform_id="linux-x64",
archive_type="nupkg",
binary_name="Microsoft.CodeAnalysis.LanguageServer.dll",
extract_path="content/LanguageServer/linux-x64",
)
# Mock the dependency provider
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Mock urllib.request.urlretrieve to capture the URL being used
with patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve") as mock_retrieve:
with patch("solidlsp.language_servers.csharp_language_server.SafeZipExtractor"):
try:
dependency_provider._download_nuget_package(test_dependency)
except Exception:
# Expected to fail since we're mocking, but we want to check the URL
pass
# Verify that urlretrieve was called with the NuGet.org URL
assert mock_retrieve.called, "urlretrieve should be called"
called_url = mock_retrieve.call_args[0][0]
assert called_url == test_dependency.url, f"Should use URL from RuntimeDependency: {test_dependency.url}"
assert "nuget.org" in called_url, "Should use NuGet.org URL"
assert "azure" not in called_url.lower(), "Should not use Azure feed"
def test_runtime_dependencies_use_nuget_org_urls(self):
"""Test that _RUNTIME_DEPENDENCIES are configured with NuGet.org URLs."""
from solidlsp.language_servers.csharp_language_server import _RUNTIME_DEPENDENCIES
# Check language server dependencies
lang_server_deps = [dep for dep in _RUNTIME_DEPENDENCIES if dep.id == "CSharpLanguageServer"]
assert len(lang_server_deps) == 6, "Should have 6 language server platform variants"
for dep in lang_server_deps:
# Verify package name uses roslyn-language-server
assert dep.package_name is not None, f"Package name should be set for {dep.platform_id}"
assert dep.package_name.startswith(
"roslyn-language-server."
), f"Package name should start with 'roslyn-language-server.' but got: {dep.package_name}"
# Verify version is the newer NuGet.org version
assert dep.package_version == "5.5.0-2.26078.4", f"Should use NuGet.org version 5.5.0-2.26078.4, got: {dep.package_version}"
# Verify URL points to NuGet.org
assert dep.url is not None, f"URL should be set for {dep.platform_id}"
assert "nuget.org" in dep.url, f"URL should point to nuget.org, got: {dep.url}"
assert "azure" not in dep.url.lower(), f"URL should not point to Azure feed, got: {dep.url}"
def test_download_method_does_not_call_azure_feed(self):
"""Test that the new download method does not attempt to access Azure feed."""
with tempfile.TemporaryDirectory() as temp_dir:
test_dependency = RuntimeDependency(
id="TestPackage",
description="Test package",
package_name="roslyn-language-server.linux-x64",
package_version="5.5.0-2.26078.4",
url="https://www.nuget.org/api/v2/package/roslyn-language-server.linux-x64/5.5.0-2.26078.4",
platform_id="linux-x64",
archive_type="nupkg",
binary_name="test.dll",
)
mock_settings = SolidLSPSettings()
custom_settings = SolidLSPSettings.CustomLSSettings({})
dependency_provider = CSharpLanguageServer.DependencyProvider(
custom_settings=custom_settings,
ls_resources_dir=temp_dir,
solidlsp_settings=mock_settings,
repository_root_path="/fake/repo",
)
# Mock urllib.request.urlopen to track if Azure feed is accessed
with patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlopen") as mock_urlopen:
with patch("solidlsp.language_servers.csharp_language_server.urllib.request.urlretrieve"):
with patch("solidlsp.language_servers.csharp_language_server.SafeZipExtractor"):
try:
dependency_provider._download_nuget_package(test_dependency)
except Exception:
pass
# Verify that urlopen was NOT called (no service index lookup)
assert not mock_urlopen.called, "Should not call urlopen for Azure service index lookup"
test/solidlsp/fsharp/test_fsharp_basic.py
+11 -7
View File
@@ -2,6 +2,7 @@ import os
import tempfile
import threading
from pathlib import Path
from typing import Any
from unittest.mock import Mock, patch
import pytest
@@ -10,6 +11,7 @@ from solidlsp import SolidLanguageServer
from solidlsp.language_servers.fsharp_language_server import FSharpLanguageServer
from solidlsp.ls_config import Language
from solidlsp.ls_utils import SymbolUtils
from test.conftest import is_ci
@pytest.mark.fsharp
@@ -87,6 +89,7 @@ class TestFSharpLanguageServer:
for expected in expected_symbols:
assert expected in symbol_names, f"{expected} not found in Person.fs symbols"
@pytest.mark.skipif(is_ci, reason="Test is flaky") # TODO: Re-enable if the LS can be made more reliable #1039
@pytest.mark.parametrize("language_server", [Language.FSHARP], indirect=True)
def test_find_referencing_symbols_across_files(self, language_server: SolidLanguageServer) -> None:
"""Test finding references to Calculator functions across files."""
@@ -122,6 +125,7 @@ class TestFSharpLanguageServer:
# We should get at least some definitions
assert len(definitions) >= 0, "Should get definitions (even if empty for complex cases)"
@pytest.mark.skipif(is_ci, reason="Test is flaky") # TODO: Re-enable if the LS can be made more reliable #1039
@pytest.mark.parametrize("language_server", [Language.FSHARP], indirect=True)
def test_hover_information(self, language_server: SolidLanguageServer) -> None:
"""Test hover information functionality."""
@@ -140,14 +144,14 @@ class TestFSharpLanguageServer:
file_path = os.path.join("Program.fs")
# Use threading for cross-platform timeout (signal.SIGALRM is Unix-only)
result = [None]
exception = [None]
result: dict[str, Any] = dict(value=None)
exception: dict[str, Any] = dict(value=None)
def run_completion():
try:
result[0] = language_server.request_completions(file_path, 15, 10)
result["value"] = language_server.request_completions(file_path, 15, 10)
except Exception as e:
exception[0] = e
exception["value"] = e
thread = threading.Thread(target=run_completion, daemon=True)
thread.start()
@@ -158,10 +162,10 @@ class TestFSharpLanguageServer:
# The important thing is that the language server doesn't crash
return
if exception[0]:
raise exception[0]
if exception["value"]:
raise exception["value"]
assert isinstance(result[0], list), "Completions should be a list"
assert isinstance(result["value"], list), "Completions should be a list"
@pytest.mark.parametrize("language_server", [Language.FSHARP], indirect=True)
def test_diagnostics(self, language_server: SolidLanguageServer) -> None:
test/solidlsp/groovy/test_groovy_basic.py
+7 -3
View File
@@ -12,13 +12,14 @@ from solidlsp.settings import SolidLSPSettings
@pytest.mark.groovy
class TestGroovyLanguageServer:
language_server: SolidLanguageServer | None = None
test_repo_path: Path = Path(__file__).parent.parent.parent / "resources" / "repos" / "groovy" / "test_repo"
@classmethod
def setup_class(cls):
"""
Set up test class with Groovy test repository.
"""
cls.test_repo_path = Path(__file__).parent.parent.parent / "resources" / "repos" / "groovy" / "test_repo"
if not cls.test_repo_path.exists():
pytest.skip("Groovy test repository not found")
@@ -56,10 +57,11 @@ class TestGroovyLanguageServer:
"""
Clean up language server.
"""
if hasattr(cls, "language_server"):
if cls.language_server is not None:
cls.language_server.stop()
def test_find_symbol(self) -> None:
assert self.language_server is not None
symbols = self.language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "Main"), "Main class not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Utils"), "Utils class not found in symbol tree"
@@ -67,6 +69,7 @@ class TestGroovyLanguageServer:
assert SymbolUtils.symbol_tree_contains_name(symbols, "ModelUser"), "ModelUser class not found in symbol tree"
def test_find_referencing_class_symbols(self) -> None:
assert self.language_server is not None
file_path = os.path.join("src", "main", "groovy", "com", "example", "Utils.groovy")
refs = self.language_server.request_references(file_path, 3, 6)
assert any("Main.groovy" in ref.get("relativePath", "") for ref in refs), "Utils should be referenced from Main.groovy"
@@ -93,6 +96,7 @@ class TestGroovyLanguageServer:
assert len(model_user_refs) >= 1, f"Model should be referenced from ModelUser.groovy at least 1 time, found {len(model_user_refs)}"
def test_overview_methods(self) -> None:
assert self.language_server is not None
symbols = self.language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "Main"), "Main missing from overview"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Utils"), "Utils missing from overview"
Loaded 100 of 110 files, more files were not shown because too many files have changed in this diff. Show more