tiennm99/serena
1
0 Fork 0
mirror of https://github.com/tiennm99/serena.git synced 2026-10-16 06:14:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into evaluation

Browse Source
This commit is contained in:
Michael Panchenko committed 2026-04-13 09:51:20 +02:00
2 parents 99925447ea 1789967968
commit 685715dd02
33 files changed
+993 -264
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.gitignore
+1 -1
View File
@@ -216,7 +216,7 @@ tmp/
.vscode/
# Claude settings
.claude/settings.local.json
/.claude
# Elixir
/test/resources/repos/elixir/test_repo/deps
CHANGELOG.md
+19 -1
View File
@@ -2,11 +2,29 @@
Status of the `main` branch. Changes prior to the next official version change will appear here.
* General:
- Support environment variable `SERENA_USAGE_REPORTING` (set to `false` to disable usage reporting)
* Language Servers:
- Add mSL (mIRC Scripting Language) support (custom pygls-based language server; symbols, references, definitions)
# 1.1.1
* General:
- Enable cert verification for HTTPS request to oraios-software.de #1320
* JetBrains:
- `JetBrainsRenameTool` can now also rename occurrences in comments and text.
* Language Servers:
- Fix Dart LSP returning only symbol name as body instead of full method body.
# 1.1.0
* General:
- Add `serena init` and `serena setup` commands
- **Major**: Add commands for hooks and documentation of recommended setup. Consider setting up the [recommended hooks](https://oraios.github.io/serena/02-usage/030_clients.html) !
- Add `serena init` and `serena setup` commands
- Rework installation instructions, switching to releases on pypi for distribution. Please update your mcp startup commands!
- Add minimal usage data collection on startup (only Serena version, language backend, OS, dashboard enabled status; no personally identifiable information)
- Fix: git commit id in Serena version strings was incorrect
README.md
+11 -4
View File
@@ -7,10 +7,17 @@
Serena is the IDE for your coding agent.
</h3>
<div align="center">
<a href="https://discord.com/invite/cVUNQmnV4r"><img src="https://img.shields.io/badge/discord-join-5865F2?style=flat-square&labelColor=0a0e14&logo=discord&logoColor=5865F2" alt="discord"></a>
<a href="https://github.com/oraios/serena/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-b0e8ff?style=flat-square&labelColor=0a0e14" alt="license"></a>
</div>
<br>
* Serena provides essential **semantic code retrieval, editing and refactoring tools** that are akin to an IDE's capabilities,
operating at the symbol level and exploiting relational structure.
* It integrates with any client/LLM via the model context protocol (**MCP**).
Serena's **agent-first tool design** involves robust high-level abstractions, distinguishing it from
approaches that rely on low-level concepts like line numbers or primitive search patterns.
@@ -19,7 +26,7 @@ more complex codebases.
> [!IMPORTANT]
> Do not install Serena via an MCP or plugin marketplace! They contain outdated and suboptimal installation commands.
> Instead, follow our [Quick Start](quick-start) instructions.
> Instead, follow our [Quick Start](#quick-start) instructions.
## What Our "End Users" Say
@@ -58,7 +65,7 @@ To connect the Serena MCP server to your client, you either
* provide the client with a launch command that allows it to start the MCP server, or
* start the Serena MCP server yourself in HTTP mode and provide the client with the URL.
See the [Quick Start](quick-start) section below for information on how to get started.
See the [Quick Start](#quick-start) section below for information on how to get started.
## Programming Language Support & Semantic Analysis Capabilities
@@ -84,7 +91,7 @@ Serena incorporates a powerful abstraction layer for the integration of language
The underlying language servers are typically open-source projects or at least freely available for use.
When using Serena's language server backend, we provide **support for over 40 programming languages**, including
AL, Ansible, Bash, C#, C/C++, Clojure, Crystal, Dart, Elixir, Elm, Erlang, Fortran, F#, GLSL, Go, Groovy, Haskell, Haxe, HLSL, Java, JavaScript, Julia, Kotlin, Lean 4, Lua, Luau, Markdown, MATLAB, Nix, OCaml, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Solidity, Swift, TOML, TypeScript, WGSL, YAML, and Zig.
AL, Ansible, Bash, C#, C/C++, Clojure, Crystal, Dart, Elixir, Elm, Erlang, Fortran, F#, GLSL, Go, Groovy, Haskell, Haxe, HLSL, Java, JavaScript, Julia, Kotlin, Lean 4, Lua, Luau, Markdown, MATLAB, mSL, Nix, OCaml, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Solidity, Swift, TOML, TypeScript, WGSL, YAML, and Zig.
### The Serena JetBrains Plugin
docs/01-about/020_programming-languages.md
+3
View File
@@ -84,6 +84,9 @@ Some languages require additional installations or setup steps, as noted.
* **Luau**
* **Markdown**
(must explicitly enable language `markdown`, primarily useful for documentation-heavy projects)
* **mSL** (mIRC Scripting Language)
(auto-installed; no external dependencies required — uses a custom pygls-based LSP server shipped with Serena;
supports document symbols, workspace symbols, references, and go-to-definition for aliases, events, menus, dialogs, and CTCP handlers in `.mrc` files)
* **Nix**
(requires nixd installation)
* **OCaml**
docs/02-usage/010_installation.md
+6
View File
@@ -43,3 +43,9 @@ To update Serena to the latest version, run:
To keep informed about updates, make sure you regularly open [Serena's Dashboard](060_dashboard),
where we will announce releases along with the new features and improvements they bring.
:::
## Uninstalling Serena
Serena can be uninstalled with the following command:
uv tool uninstall serena-agent
docs/02-usage/030_clients.md
+6 -17
View File
@@ -40,10 +40,15 @@ A key mechanism for this is to use the appropriate [context](#contexts) when sta
(clients-common-pitfalls)=
### Common Pitfalls
**Discoverability of the `serena`**.
**Discoverability of the `serena` command**.
Your client may not find the `serena` CLI command, even if it is on your system PATH.
In this case, a workaround is to provide the full path to the `serena` executable.
**Serena's tools not being used**.
With some clients, you may experience that Serena's tools are not being used.
This is mainly due to problems in the client itself (like a poorly implemented tool discovery). To counteract this,
Serena comes with a set of commands that can be used in _hooks_. See the sections on hooks for Claude Code and VSCode below.
**Environment Variables**.
Some language servers may require additional environment variables to be set (e.g. F# on macOS with Homebrew),
which you may need to explicitly add to the MCP server configuration.
@@ -117,22 +122,6 @@ Confirm that Claude Code is connected to Serena by running the `/mcp` command an
If Serena fails to start fast enough, you should set `MCP_TIMEOUT` to a sufficiently high value
(e.g. by adding `export MCP_TIMEOUT=60000` to your shell profile)
**Maximum Token Efficiency.** To maximize token efficiency, you may want to use Claude Code's
*on-demand tool loading* feature, which is supported since at least v2.0.74 of Claude Code.
This feature avoids sending all tool descriptions to Claude upon startup, thus saving tokens.
Instead, Claude will search for tools as needed (but there are no guarantees that it will
search optimally, of course).
To enable this feature, set the environment variable `ENABLE_TOOL_SEARCH=true`.
Depending on your shell, you can also set this on a per-session basis, e.g. using
```shell
ENABLE_TOOL_SEARCH=true claude
```
in bash/zsh, or using
```shell
set ENABLE_TOOL_SEARCH=true && claude
```
in Windows CMD to launch Claude Code.
**Hooks.**
Due to recent changes (especially dynamic tool loading) in Claude Code, the agent will often fail to make proper use
of Serena's tools, either by failing to load them in the beginning or by forgetting the instructions in a long session
docs/02-usage/050_configuration.md
+9 -1
View File
@@ -472,7 +472,7 @@ Supported settings:
| Setting | Default | Description |
|---|---|---|
| `ls_path` | managed install or build | Override the `shader-language-server` executable path. |
| `version` | `1.3.0` | Override the bundled version Serena downloads, or builds from source on macOS, when `ls_path` is not set. |
| `version` | `1.3.1` | Override the bundled version Serena downloads, or builds from source on macOS, when `ls_path` is not set. |
#### Haxe
@@ -863,3 +863,11 @@ prompts:
```
It is advisable to use the default prompt as a starting point and modify it to suit your needs.
### Usage Reporting
On startup, Serena reports anonymous usage data to help us understand Serena usage.
Specifically, we collect the Serena version, the operating system & language backend being used as well as the dashboard enabled status.
No personally identifiable information or project-specific information is collected.
If you want to opt out of usage reporting, set the environment variable `SERENA_USAGE_REPORTING` to `false`.
lessons_learned.md
-90
View File
@@ -1,90 +0,0 @@
# Lessons Learned
In this document we briefly collect what we have learned while developing and using Serena,
what works well and what doesn't.
## What Worked
### Separate Tool Logic From MCP Implementation
MCP is just another protocol, one should let the details of it creep into the application logic.
The official docs suggest using function annotations to define tools and prompts. While that may be
useful for small projects to get going fast, it is not wise for more serious projects. In Serena,
all tools are defined independently and then converted to instances of `MCPTool` using our `make_tool`
function.
### Autogenerated PromptFactory
Prompt templates are central for most LLM applications, so one needs good representations of them in the code,
while at the same time they often need to be customizable and exposed to users. In Serena we address these conflicting
needs by defining prompt templates (in jinja format) in separate yamls that users can easily modify and by autogenerated
a `PromptFactory` class with meaningful method and parameter names from these yamls. The latter is committed to our code.
We separated out the generation logic into the [interprompt](/src/interprompt/README.md) subpackage that can be used as a library.
### Tempfiles and Snapshots for Testing of Editing Tools
We test most aspects of Serena by having a small "project" for each supported language in `tests/resources`.
For the editing tools, which would change the code in these projects, we use tempfiles to copy over the code.
The pretty awesome [syrupy](https://github.com/syrupy-project/syrupy) pytest plugin helped in developing
snapshot tests.
### Dashboard and GUI for Logging
It is very useful to know what the MCP Server is doing. We collect and display logs in a GUI or a web dashboard,
which helps a lot in seeing what's going on and in identifying any issues.
### Unrestricted Bash Tool
We know it's not particularly safe to permit unlimited shell commands outside a sandbox, but we did quite some
evaluations and so far... nothing bad has happened. Seems like the current versions of the AI overlords rarely want to execute `sudo rm - rf /`.
Still, we are working on a safer approach as well as better integration with sandboxing.
### Multilspy
The [multilspy](https://github.com/microsoft/multilspy/) project helped us a lot in getting started and stands at the core of Serena.
Many more well known python implementations of language servers were subpar in code quality and design (for example, missing types).
### Developing Serena with Serena
We clearly notice that the better the tool gets, the easier it is to make it even better
## Prompting
### Shouting and Emotive Language May Be Needed
When developing the `ReplaceRegexTool` we were initially not able to make Claude 4 (in Claude Desktop) use wildcards to save on output tokens. Neither
examples nor explicit instructions helped. It was only after adding
```
IMPORTANT: REMEMBER TO USE WILDCARDS WHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
```
to the initial instructions and to the tool description that Claude finally started following the instructions.
## What Didn't Work
### Lifespan Handling by MCP Clients
The MCP technology is clearly very green. Even though there is a lifespan context in the MCP SDK,
many clients, including Claude Desktop, fail to properly clean up, leaving zombie processes behind.
We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running
and can terminate it there.
### Trusting Asyncio
Running multiple asyncio apps led to non-deterministic
event loop contamination and deadlocks, which were very hard to debug
and understand. We solved this with a large hammer, by putting all asyncio apps into a separate
process. It made the code much more complex and slightly enhanced RAM requirements, but it seems
like that was the only way to reliably overcome asyncio deadlock issues.
### Cross-OS Tkinter GUI
Different OS have different limitations when it comes to starting a window or dealing with Tkinter
installations. This was so messy to get right that we pivoted to a web-dashboard instead
### Editing Based on Line Numbers
Not only are LLMs notoriously bad in counting, but also the line numbers change after edit operations,
and LLMs are also often too dumb to understand that they should update the line numbers information they had
received before. We pivoted to string-matching and symbol-name based editing.
llms-install.md
-29
View File
@@ -1,29 +0,0 @@
# MCP Installation instructions
This document is mainly used as instructions for AI-assistants like Cline and others that
try to do an automatic install based on freeform instructions.
0. Make sure `uv` is installed. If not, install it using either `curl -LsSf https://astral.sh/uv/install.sh | sh` (macOS, Linux) or
`powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"` (Windows). Find the path to the `uv` executable,
you'll need it later.
1. Clone the repo with `git clone git@github.com:oraios/serena.git` and change into its dir (e.g., `cd serena`)
2. Check if `serena_config.yml` exists. If not, create it with `cp serena_config.template.yml serena_config.yml`. Read the instructions in the config.
3. In the config, check if the path to your project was added. If not, add it to the `projects` section
4. In your project, create a `.serena` if needed and check whether `project.yml` exists there.
5. If no `project.yml` was found, create it using `cp /path/to/serena/myproject.template.yml /path/to/your/project/.serena/project.yml`
6. Read the instructions in `project.yml`. Make sure the `project.yml` has the correct project language configured.
Remove the project_root entry there.
7. Finally, add the Serena MCP server config like this:
```json
{
"mcpServers": {
...
"serena": {
"command": "/abs/path/to/uv",
"args": ["run", "--directory", "/abs/path/to/serena", "serena-mcp-server", "/path/to/your/project/.serena/project.yml"]
}
}
}
```
news/20260111.html
+2 -2
View File
@@ -8,7 +8,7 @@
<li><strong>Extended symbol information:</strong> The `find_symbol` and `find_referencing_symbols` tools (and their JetBrains variants)
now can return more information about the symbol (specifically, docstrings and signatures).
</li>
<li><strong>Type hierarchy tool:</strong> A new tool &ndash; exclusive to <a href="https://oraios.github.io/serena/02-usage/025_jetbrains_plugin.html">JetBrains mode</a> &ndash;
<li><strong>Type hierarchy tool:</strong> A new tool &ndash; exclusive to <a target="_blank" href="https://oraios.github.io/serena/02-usage/025_jetbrains_plugin.html">JetBrains mode</a> &ndash;
that can fetch a symbol's hierarchy, i.e. superclasses and subclasses, which is very useful in many situations.
</li>
<li><strong>Compact overviews:</strong> The `get_symbols_overview` tools now return a much more compact
@@ -17,6 +17,6 @@
</li>
</ul>
<p>
For more detailed information, see our <a href="https://github.com/oraios/serena/blob/main/CHANGELOG.md">changelog</a>.
For more detailed information, see our <a target="_blank" href="https://github.com/oraios/serena/blob/main/CHANGELOG.md">changelog</a>.
</p>
</div>
news/20260303.html
+1 -1
View File
@@ -7,6 +7,6 @@
into topics and sharing them across projects.
</p>
<p>
For more detailed information, see the <a href="https://oraios.github.io/serena/02-usage/045_memories.html">documentation</a>.
For more detailed information, see the <a target="_blank" href="https://oraios.github.io/serena/02-usage/045_memories.html">documentation</a>.
</p>
</div>
news/20260330.html
+1 -1
View File
@@ -8,6 +8,6 @@
to other projects.
</p>
<p>
Find out more about this feature in our <a href="https://oraios.github.io/serena/02-usage/040_workflow.html#reading-from-external-projects">documentation</a>.
Find out more about this feature in our <a target="_blank" href="https://oraios.github.io/serena/02-usage/040_workflow.html#reading-from-external-projects">documentation</a>.
</p>
</div>
news/20260411.html
+8 -8
View File
@@ -2,21 +2,21 @@
<h3>Serena v1.1: Hooks and New Release Mechanism</h3>
<p class="date">April 11, 2026</p>
<p>
<b>Installation.</b> Serena has matured and we are switching to a PyPI-based release and installation mechanism.
The recommended uv-tool-based installation decreases startup times and facilitates CLI usage.
See <a href="https://oraios.github.io/serena/02-usage/010_installation.html">updated installation instructions</a>
for more details.
<b>Installation.</b> Serena has matured, and we are switching to a PyPI-based release and installation mechanism.
The recommended uv tool-based installation decreases startup times and facilitates CLI usage.
See our <a target="_blank" href="https://oraios.github.io/serena/02-usage/010_installation.html">updated installation instructions</a>
for more details,
and update your <a target="_blank" href="https://oraios.github.io/serena/02-usage/030_clients.html">client configurations</a> after having switched
to the new installation method.
</p>
<p>
<b>Improving Tool Usage via Hooks.</b> To address issues with agents not using Serena's tools (agent drift), we now provide the `serena-hooks` commands and a recommended
setup for Claude Code and VSCode.
Please read the <a href="https://oraios.github.io/serena/02-usage/030_clients.html">updated documentation</a> for more details.
Please read the <a target="_blank" href="https://oraios.github.io/serena/02-usage/030_clients.html">updated documentation</a> for more details.
We <i>strongly recommend using the hooks</i>, since recent updates in Claude Code and VSCode resulted in
reduced performance, which our hook commands compensate for.
</p>
<p>
<b>Community.</b> We opened a <a href="https://discord.com/invite/cVUNQmnV4r">Discord server</a> for our community.
<b>Community.</b> We opened a <a target="_blank" href="https://discord.com/invite/cVUNQmnV4r">Discord server</a> for our community.
</p>
</div>
pyproject.toml
+8 -2
View File
@@ -4,14 +4,17 @@ requires = ["hatchling"]
[project]
name = "serena-agent"
version = "1.1.0"
description = ""
version = "1.1.1"
description = "A powerful MCP toolkit for coding, providing semantic retrieval and editing capabilities - the IDE for your agent"
authors = [{ name = "Oraios AI", email = "info@oraios-ai.de" }]
readme = "README.md"
requires-python = ">=3.11, <3.15"
classifiers = [
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.13",
]
dependencies = [
"requests==2.33.0",
@@ -38,6 +41,8 @@ dependencies = [
"beautifulsoup4==4.14.2",
"oraios-pywebview==6.2",
"pystray==0.19.5",
"pygls==2.1.1", # used to implement an msl language server
"lsprotocol==2025.0.0",
# Transitive deps pinned for security (dependabot alerts).
# Exact pins because uvx installs from git, ignoring the lock file.
"urllib3==2.6.3",
@@ -344,6 +349,7 @@ markers = [
"lean4: language server running for Lean 4",
"solidity: language server running for Solidity (uses @nomicfoundation/solidity-language-server)",
"ansible: language server running for Ansible (uses @ansible/ansible-language-server)",
"msl: language server running for mSL (mIRC Scripting Language)",
]
[tool.codespell]
scripts/bump_version.py
+1
View File
@@ -44,6 +44,7 @@ def bump_version(major: bool, minor: bool, patch: bool, target_version: str | No
if dry_run:
click.echo(f"Dry run complete. Version would be bumped to {new_version}")
else:
os.system("uv lock")
click.echo(f"Bumped version to {new_version}")
os.system("git add -u")
os.system(f'git commit -m "Release v{new_version}"')
src/serena/__init__.py
+1 -1
View File
@@ -1,4 +1,4 @@
__version__ = "1.1.0"
__version__ = "1.1.1"
import logging
src/serena/agent.py
+4 -5
View File
@@ -410,17 +410,16 @@ class SerenaAgent:
self._send_usage_info()
def _send_usage_info(self) -> None:
is_ci = os.getenv("CI") == "true" or os.getenv("GITHUB_ACTIONS") == "true"
if is_ci:
if os.getenv("CI") == "true" or os.getenv("GITHUB_ACTIONS") == "true" or os.getenv("SERENA_USAGE_REPORTING") == "false":
return
params = {
params: dict[str, str | int] = {
"os": platform.system(),
"dashboard": int(self.serena_config.web_dashboard),
"version": self.version,
"backend": self._language_backend.value,
}
try:
requests.get("https://www.oraios-software.de/serena_usage.php", params=params, timeout=1, verify=False)
requests.get("https://oraios-software.de/serena_usage.php", params=params, timeout=1)
except Exception as e:
log.debug(f"Failed to send usage info: {e}")
@@ -720,7 +719,7 @@ class SerenaAgent:
if self._language_backend == LanguageBackend.LSP:
languages_str = ", ".join([lang.value for lang in proj.project_config.languages])
msg += f"\nProgramming languages: {languages_str}."
msg += "File encoding: {proj.project_config.encoding}."
msg += f"File encoding: {proj.project_config.encoding}."
include_memories = self._active_tools.contains_tool_class(ReadMemoryTool)
if include_memories:
src/serena/task_executor.py
+1 -1
View File
@@ -80,7 +80,7 @@ class TaskExecutor:
:param timeout: the maximum time to wait in seconds; if None, use the task's own timeout
(which may be None to wait indefinitely)
:return: True if the task is done, False if the timeout was reached
:return: the result of the task
"""
return self.future.result(timeout=timeout)
src/serena/tools/jetbrains_tools.py
+19 -2
View File
@@ -512,16 +512,33 @@ class JetBrainsRenameTool(Tool, ToolMarkerSymbolicEdit, ToolMarkerOptional):
Renames a symbol, file or directory throughout the codebase using the JetBrains backend.
"""
def apply(self, relative_path: str, new_name: str, name_path: str | None = None) -> str:
def apply(
self,
relative_path: str,
new_name: str,
name_path: str | None = None,
rename_in_comments: bool = False,
rename_in_text_occurrences: bool = False,
) -> str:
"""
Renames a symbol, file or directory throughout the codebase.
Note: renaming in comments/text is on a best-effort basis by the IDE; if the symbol name is non-unique, further
verification is recommended.
:param relative_path: if `name_path` is passed, the relative path of the file containing the symbol.
Otherwise, the path to the directory or file to rename.
:param new_name: the new name
:param name_path: the name path of the symbol to rename or None if renaming a file or directory.
:param rename_in_comments: whether to also rename occurrences in comments. Default True.
:param rename_in_text_occurrences: whether to also rename occurrences in text. Default True.
:return: a status message
"""
code_editor = JetBrainsCodeEditor(self.project)
result = code_editor.rename_symbol(name_path=name_path, relative_path=relative_path, new_name=new_name)
result = code_editor.rename_symbol(
name_path=name_path,
relative_path=relative_path,
new_name=new_name,
rename_in_comments=rename_in_comments,
rename_in_text_occurrences=rename_in_text_occurrences,
)
return self._to_json(result)
src/serena/tools/tools_base.py
-1
View File
@@ -430,7 +430,6 @@ tool_packages = ["serena.tools"]
@singleton
class ToolRegistry:
_deleted_tools: list[str] = [
"think_about_collected_information",
"think_about_collected_information",
"prepare_for_new_conversation",
"summarize_changes",
src/solidlsp/language_servers/dart_language_server.py
+7 -1
View File
@@ -123,7 +123,13 @@ class DartLanguageServer(SolidLanguageServer):
"""
root_uri = pathlib.Path(repository_absolute_path).as_uri()
initialize_params = {
"capabilities": {},
"capabilities": {
"textDocument": {
"documentSymbol": {
"hierarchicalDocumentSymbolSupport": True,
}
}
},
"initializationOptions": {
"onlyAnalyzeProjectsWithOpenFiles": False,
"closingLabels": False,
src/solidlsp/language_servers/hlsl_language_server.py
+4 -95
View File
@@ -7,9 +7,6 @@ import logging
import os
import pathlib
import shutil
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
from typing import Any, cast
import psutil
@@ -18,11 +15,9 @@ from overrides import override
from solidlsp.ls import (
LanguageServerDependencyProvider,
LanguageServerDependencyProviderSinglePath,
LSPFileBuffer,
SolidLanguageServer,
)
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.ls_exceptions import SolidLSPException
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.settings import SolidLSPSettings
@@ -31,13 +26,13 @@ from .common import RuntimeDependency, RuntimeDependencyCollection
log = logging.getLogger(__name__)
# GitHub release version to download when not installed locally
_DEFAULT_VERSION = "1.3.0"
_DEFAULT_VERSION = "1.3.1"
_GITHUB_RELEASE_BASE = "https://github.com/antaalt/shader-sense/releases/download"
_HLSL_ALLOWED_HOSTS = ("github.com", "release-assets.githubusercontent.com", "objects.githubusercontent.com")
_HLSL_SHA256_BY_ASSET = {
"shader-language-server-x86_64-pc-windows-msvc.zip": "a945b000c296cdeebb9ee2d4452cec2a0f26544dd076bb08bfdcade2278296a6",
"shader-language-server-x86_64-unknown-linux-gnu.zip": "8c0a7b36f51cc58593762db3592ae13e21ca3cb982b2526cfaaf7c82e92ca089",
"shader-language-server-aarch64-pc-windows-msvc.zip": "cdbd7b41e71cf6040d5cdb7e211ba4b76671a404ee0f7add281d72d3ab8dfa65",
"shader-language-server-x86_64-pc-windows-msvc.zip": "49081c5547ddde1b8b3b17295282a80ddacbca1d6f5dcd834e2788c02bafa997",
"shader-language-server-x86_64-unknown-linux-gnu.zip": "61710df7ca17a2d063b598936c57c56c49fbf837707a1aa886f9b0193a35be3c",
"shader-language-server-aarch64-pc-windows-msvc.zip": "a3b3799affe2cad27652e788376b46fe76e1a6c2ce45946a486dcb26c9091412",
}
@@ -277,92 +272,6 @@ class HlslLanguageServer(SolidLanguageServer):
log.debug(f"Error cleaning up shader-language-server process tree: {e}")
super().stop(shutdown_timeout)
@contextmanager
def open_file(self, relative_file_path: str, open_in_ls: bool = True) -> Iterator[LSPFileBuffer]:
"""Open a file for LSP, preserving on-disk CRLF line endings.
Workaround for an upstream bug in shader-language-server
(antaalt/shader-sense) where `watch_main_file` replaces an already-cached
module's content without re-parsing the tree-sitter tree. When a file is
first pulled into the server's cache via an `#include` from another
shader (where it's read via `std::fs::read_to_string`, preserving CRLF),
and then later opened directly via `textDocument/didOpen` with the
client-normalized LF text, the stored tree still references byte offsets
into the longer CRLF content. The next symbol query slices the new
(shorter) content with stale offsets and panics with
`byte index N is out of bounds` in `shader-sense/src/symbols/symbol_parser.rs`.
The root-cause fix belongs upstream (the server should call
`update_module` instead of assigning `content` raw). Until then, we
ensure the text we send in `didOpen` matches byte-for-byte what the
server reads from disk by preloading the file buffer with a
CRLF-preserving read before the LSP notification is sent.
This is the only place in Serena that overrides `open_file`; the fix is
deliberately scoped to the HLSL language server. It mirrors the base
class logic in `SolidLanguageServer.open_file` verbatim except for the
buffer construction branch, where creation is deferred (`open_in_ls=False`)
so the buffer's contents can be preloaded before `ensure_open_in_ls` runs.
"""
if not self.server_started:
log.error("open_file called before Language Server started")
raise SolidLSPException("Language Server not started")
absolute_file_path = Path(self.repository_root_path, relative_file_path)
uri = absolute_file_path.as_uri()
if uri in self.open_file_buffers:
fb = self.open_file_buffers[uri]
assert fb.uri == uri
assert fb.ref_count >= 1
fb.ref_count += 1
if open_in_ls:
fb.ensure_open_in_ls()
yield fb
fb.ref_count -= 1
else:
version = 0
language_id = self._get_language_id_for_file(relative_file_path)
# Defer the didOpen so we can preload CRLF-preserved content first.
fb = LSPFileBuffer(
abs_path=absolute_file_path,
uri=uri,
encoding=self._encoding,
version=version,
language_id=language_id,
ref_count=1,
language_server=self,
open_in_ls=False,
)
self._preload_crlf_content(fb)
self.open_file_buffers[uri] = fb
if open_in_ls:
fb.ensure_open_in_ls()
yield fb
fb.ref_count -= 1
if self.open_file_buffers[uri].ref_count == 0:
self.open_file_buffers[uri].close()
del self.open_file_buffers[uri]
def _preload_crlf_content(self, fb: LSPFileBuffer) -> None:
"""Populate an LSPFileBuffer with a CRLF-preserving read of its backing file.
Python's default text-mode open applies universal-newlines translation
(CRLF -> LF), which would desync the client's `didOpen` text from the
server-side `std::fs::read_to_string` view that parsed the dependency
tree. Passing `newline=""` disables the translation so bytes match.
"""
with open(fb.abs_path, encoding=fb.encoding, newline="") as f:
raw = f.read()
# Set the buffer's cached state directly: the contents, the mtime
# (required by the contents property's staleness check), and clear the
# hash so it's recomputed against the new bytes.
fb._contents = raw
fb._read_file_modified_date = fb.abs_path.stat().st_mtime
fb._content_hash = None
@override
def is_ignored_dirname(self, dirname: str) -> bool:
"""Ignore Unity-specific directories that contain no user-authored shaders."""
src/solidlsp/language_servers/msl_language_server.py
+107
View File
@@ -0,0 +1,107 @@
"""
Provides mSL (mIRC Scripting Language) specific instantiation of the LanguageServer class.
Uses a custom Python-based LSP server (pygls) for parsing .mrc files.
The LSP server script is shipped as ``msl_lsp_server.py`` alongside this module
and launched as a subprocess using the current Python interpreter.
"""
import logging
import os
import pathlib
import sys
import threading
from solidlsp.ls import (
SolidLanguageServer,
)
from solidlsp.ls_config import LanguageServerConfig
from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams
from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo
from solidlsp.settings import SolidLSPSettings
log = logging.getLogger(__name__)
_MSL_LSP_SCRIPT = os.path.join(os.path.dirname(__file__), "msl_lsp_server.py")
class MslLanguageServer(SolidLanguageServer):
"""
Provides mSL (mIRC Scripting Language) specific instantiation of the LanguageServer class.
Uses a Python-based LSP server for parsing .mrc files (aliases, events, menus, dialogs).
"""
def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings):
"""
Creates an MslLanguageServer instance. This class is not meant to be instantiated directly.
Use LanguageServer.create() instead.
"""
process_launch_info = ProcessLaunchInfo(cmd=[sys.executable, _MSL_LSP_SCRIPT], cwd=repository_root_path)
super().__init__(
config,
repository_root_path,
process_launch_info=process_launch_info,
language_id="msl",
solidlsp_settings=solidlsp_settings,
)
self.server_ready = threading.Event()
@staticmethod
def _get_initialize_params(repository_absolute_path: str) -> InitializeParams:
"""Returns the initialize params for the mSL Language Server."""
root_uri = pathlib.Path(repository_absolute_path).as_uri()
initialize_params = {
"locale": "en",
"capabilities": {
"textDocument": {
"synchronization": {"didSave": True, "dynamicRegistration": True},
"documentSymbol": {
"dynamicRegistration": True,
"hierarchicalDocumentSymbolSupport": True,
"symbolKind": {"valueSet": list(range(1, 27))},
},
"hover": {"dynamicRegistration": True, "contentFormat": ["markdown", "plaintext"]},
"references": {"dynamicRegistration": True},
"definition": {"dynamicRegistration": True},
},
"workspace": {
"workspaceFolders": True,
"symbol": {"dynamicRegistration": True},
},
},
"processId": os.getpid(),
"rootPath": repository_absolute_path,
"rootUri": root_uri,
"workspaceFolders": [{"uri": root_uri, "name": os.path.basename(repository_absolute_path)}],
}
return initialize_params # type: ignore
def _start_server(self) -> None:
"""Starts the mSL Language Server."""
def window_log_message(msg: dict) -> None:
log.info(f"LSP: window/logMessage: {msg}")
self.server_ready.set()
def do_nothing(params: dict) -> None:
pass
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
log.info("Starting mSL server process")
self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
log.info("Sending initialize request to mSL LSP server")
init_response = self.server.send.initialize(initialize_params)
log.debug(f"Received initialize response: {init_response}")
self.server.notify.initialized({})
# Wait briefly for server readiness
if not self.server_ready.wait(timeout=2.0):
log.info("Timeout waiting for mSL server ready signal, proceeding anyway")
self.server_ready.set()
log.info("mSL server initialization complete")
src/solidlsp/language_servers/msl_lsp_server.py
+439
View File
@@ -0,0 +1,439 @@
"""mSL (mIRC Scripting Language) Language Server.
A minimal LSP implementation for mIRC scripting language (.mrc files).
Provides document symbols, hover, references, go-to-definition, and workspace symbols
for aliases, events, menus, dialogs, and CTCP handlers.
Launched as a subprocess by MslLanguageServer. Communicates via stdio.
"""
import logging
import os
import pathlib
import re
from lsprotocol import types as lsp
from pygls.lsp.server import LanguageServer
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
server = LanguageServer("msl-lsp", "0.1.0")
# mSL top-level construct patterns
ALIAS_PATTERN = re.compile(
r"^[ \t]*alias\s+(?:-l\s+)?([a-zA-Z_][\w.]*)\s*(?:\{|$)",
re.MULTILINE | re.IGNORECASE,
)
EVENT_PATTERN = re.compile(
r"^[ \t]*on\s+(\*|\d+):(\w+):([^{]*?)(?:\{|$)",
re.MULTILINE | re.IGNORECASE,
)
RAW_EVENT_PATTERN = re.compile(
r"^[ \t]*raw\s+(\d+):([^{]*?)(?:\{|$)",
re.MULTILINE | re.IGNORECASE,
)
MENU_PATTERN = re.compile(
r"^[ \t]*menu\s+([^\s{]+)\s*\{",
re.MULTILINE | re.IGNORECASE,
)
DIALOG_PATTERN = re.compile(
r"^[ \t]*dialog\s+(-l\s+)?([a-zA-Z_][\w]*)\s*\{",
re.MULTILINE | re.IGNORECASE,
)
CTCP_PATTERN = re.compile(
r"^[ \t]*ctcp\s+(\*|\d+):(\w+):([^{]*?)(?:\{|$)",
re.MULTILINE | re.IGNORECASE,
)
def _get_line_col(text: str, pos: int) -> tuple[int, int]:
lines = text[:pos].split("\n")
return len(lines) - 1, len(lines[-1]) if lines else 0
def _find_block_end(text: str, start: int) -> int:
count, i = 0, start
while i < len(text):
ch = text[i]
if ch == ";" and count > 0:
while i < len(text) and text[i] != "\n":
i += 1
continue
if ch == "{":
count += 1
elif ch == "}":
count -= 1
if count == 0:
return i
i += 1
return len(text) - 1
def _make_symbol(
name: str,
kind: lsp.SymbolKind,
detail: str,
text: str,
match_start: int,
match_end: int,
match_text: str,
) -> lsp.DocumentSymbol:
sl, sc = _get_line_col(text, match_start)
bs = text.find("{", match_start)
if bs != -1:
el, ec = _get_line_col(text, _find_block_end(text, bs))
else:
el, ec = _get_line_col(text, match_end)
return lsp.DocumentSymbol(
name=name,
kind=kind,
range=lsp.Range(lsp.Position(sl, 0), lsp.Position(el, ec + 1)),
selection_range=lsp.Range(lsp.Position(sl, sc), lsp.Position(sl, sc + len(match_text))),
detail=detail,
)
def parse_symbols(text: str) -> list[lsp.DocumentSymbol]:
"""Parse mSL source code and return document symbols."""
symbols: list[lsp.DocumentSymbol] = []
for m in ALIAS_PATTERN.finditer(text):
symbols.append(_make_symbol(m.group(1), lsp.SymbolKind.Function, "alias", text, m.start(), m.end(), m.group(0)))
for m in EVENT_PATTERN.finditer(text):
pat = m.group(3).strip().rstrip(":")
name = f"on {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "")
symbols.append(_make_symbol(name, lsp.SymbolKind.Event, f"event:{m.group(2)}", text, m.start(), m.end(), m.group(0)))
for m in RAW_EVENT_PATTERN.finditer(text):
pat = m.group(2).strip().rstrip(":")
name = f"raw {m.group(1)}" + (f":{pat}" if pat else "")
symbols.append(_make_symbol(name, lsp.SymbolKind.Event, "raw event", text, m.start(), m.end(), m.group(0)))
for m in MENU_PATTERN.finditer(text):
symbols.append(_make_symbol(f"menu {m.group(1)}", lsp.SymbolKind.Module, "menu", text, m.start(), m.end(), m.group(0)))
for m in DIALOG_PATTERN.finditer(text):
symbols.append(_make_symbol(f"dialog {m.group(2)}", lsp.SymbolKind.Class, "dialog", text, m.start(), m.end(), m.group(0)))
for m in CTCP_PATTERN.finditer(text):
pat = m.group(3).strip().rstrip(":")
name = f"ctcp {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "")
symbols.append(_make_symbol(name, lsp.SymbolKind.Event, "ctcp event", text, m.start(), m.end(), m.group(0)))
symbols.sort(key=lambda s: s.range.start.line)
return symbols
def _get_workspace_roots() -> list[str]:
"""Get workspace root paths from the server's workspace object.
After initialization, pygls populates ``server.workspace`` with the
root URI/path and any workspace folders sent by the client.
"""
from pygls.uris import to_fs_path
roots: list[str] = []
try:
ws = server.workspace
except (RuntimeError, AttributeError):
return roots
# workspace.folders is a dict of {uri_string: WorkspaceFolder}
if hasattr(ws, "folders") and ws.folders:
for folder_uri in ws.folders:
fs_path = to_fs_path(folder_uri)
if fs_path:
# resolve() normalizes drive letter casing on Windows
fs_path = str(pathlib.Path(fs_path).resolve())
if os.path.isdir(fs_path):
roots.append(fs_path)
# Fall back to root_path
if not roots and hasattr(ws, "root_path") and ws.root_path:
root = str(pathlib.Path(ws.root_path).resolve())
if os.path.isdir(root):
roots.append(root)
return roots
def _path_to_uri(path: str) -> str:
"""Convert a filesystem path to a file URI."""
return pathlib.Path(path).as_uri()
def _get_all_mrc_files() -> list[tuple[str, str, str]]:
"""Scan workspace roots for all .mrc files.
Returns list of (uri, file_path, source_text) tuples.
Prefers content from already-opened documents over disk reads.
"""
results: list[tuple[str, str, str]] = []
seen_paths: set[str] = set()
# First, include all files currently open in the workspace
try:
for uri, doc in server.workspace.text_documents.items():
if uri.endswith(".mrc"):
# Resolve the filesystem path from the URI
from pygls.uris import to_fs_path
file_path = to_fs_path(uri) or uri
norm = os.path.normcase(os.path.normpath(file_path))
seen_paths.add(norm)
results.append((uri, file_path, doc.source))
except (RuntimeError, AttributeError):
pass
# Then scan workspace roots for any .mrc files not yet opened
for root in _get_workspace_roots():
for dirpath, _dirnames, filenames in os.walk(root):
for fname in filenames:
if not fname.endswith(".mrc"):
continue
full_path = os.path.join(dirpath, fname)
norm = os.path.normcase(os.path.normpath(full_path))
if norm in seen_paths:
continue
seen_paths.add(norm)
try:
with open(full_path, encoding="utf-8", errors="replace") as f:
source = f.read()
uri = _path_to_uri(full_path)
results.append((uri, full_path, source))
except OSError:
pass
return results
def _find_symbol_at_position(text: str, line: int, character: int) -> str | None:
"""Find the symbol name at the given position in the text."""
lines = text.split("\n")
if line >= len(lines):
return None
line_text = lines[line]
# Check if position is within an alias definition
for m in ALIAS_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
return m.group(1)
# Check if position is within an event definition
for m in EVENT_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
pat = m.group(3).strip().rstrip(":")
return f"on {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "")
# Check if position is within a raw event
for m in RAW_EVENT_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
pat = m.group(2).strip().rstrip(":")
return f"raw {m.group(1)}" + (f":{pat}" if pat else "")
# Check if position is within a menu
for m in MENU_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
return f"menu {m.group(1)}"
# Check if position is within a dialog
for m in DIALOG_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
return f"dialog {m.group(2)}"
# Check if position is within a CTCP handler
for m in CTCP_PATTERN.finditer(text):
sl, _ = _get_line_col(text, m.start())
if sl == line:
return f"ctcp {m.group(1)}:{m.group(2)}"
# Fall back: extract word at position (for alias call sites)
if character < len(line_text):
pos = character
# Skip leading $ if cursor is on it (e.g., $format.coins)
if pos < len(line_text) and line_text[pos] == "$":
pos += 1
start = pos
while start > 0 and (line_text[start - 1].isalnum() or line_text[start - 1] in "_."):
start -= 1
end = pos
while end < len(line_text) and (line_text[end].isalnum() or line_text[end] in "_."):
end += 1
word = line_text[start:end]
if word:
return word
return None
def _get_symbol_detail(text: str, symbol_name: str) -> str | None:
"""Get detail/documentation text for a symbol by finding its definition and extracting context."""
for sym in parse_symbols(text):
if sym.name == symbol_name:
# Extract the definition line and a few lines of body for hover
text_lines = text.split("\n")
start_line = sym.range.start.line
end_line = min(sym.range.end.line, start_line + 5)
snippet_lines = text_lines[start_line : end_line + 1]
snippet = "\n".join(snippet_lines)
return f"```msl\n{snippet}\n```\n\n**Kind**: {sym.detail}"
return None
def _build_call_pattern(alias_name: str) -> re.Pattern[str]:
"""Build a regex that matches calls to an alias (both as command and as $identifier)."""
escaped = re.escape(alias_name)
return re.compile(
rf"(?<![.\w])(?:\$)?{escaped}(?![.\w])",
re.MULTILINE | re.IGNORECASE,
)
@server.feature(lsp.TEXT_DOCUMENT_DID_OPEN)
def did_open(params: lsp.DidOpenTextDocumentParams) -> None:
pass
@server.feature(lsp.TEXT_DOCUMENT_DID_CHANGE)
def did_change(params: lsp.DidChangeTextDocumentParams) -> None:
pass
@server.feature(lsp.TEXT_DOCUMENT_DID_CLOSE)
def did_close(params: lsp.DidCloseTextDocumentParams) -> None:
pass
@server.feature(lsp.TEXT_DOCUMENT_DOCUMENT_SYMBOL)
def document_symbol(params: lsp.DocumentSymbolParams) -> list[lsp.DocumentSymbol]:
"""Return document symbols for the given document."""
try:
doc = server.workspace.get_text_document(params.text_document.uri)
return parse_symbols(doc.source)
except Exception as e:
logger.error(f"Error: {e}")
return []
@server.feature(lsp.TEXT_DOCUMENT_HOVER)
def hover(params: lsp.HoverParams) -> lsp.Hover | None:
"""Return hover information for the symbol at the given position."""
try:
doc = server.workspace.get_text_document(params.text_document.uri)
symbol_name = _find_symbol_at_position(doc.source, params.position.line, params.position.character)
if not symbol_name:
return None
# Search all files for the symbol definition to get its detail
for _uri, _path, source in _get_all_mrc_files():
detail = _get_symbol_detail(source, symbol_name)
if detail:
return lsp.Hover(
contents=lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=detail),
)
# Fallback: return the symbol name itself
return lsp.Hover(
contents=lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=f"**{symbol_name}**"),
)
except Exception as e:
logger.error(f"Error in hover: {e}")
return None
@server.feature(lsp.TEXT_DOCUMENT_REFERENCES)
def references(params: lsp.ReferenceParams) -> list[lsp.Location]:
"""Find all references to the symbol at the given position across the workspace."""
try:
doc = server.workspace.get_text_document(params.text_document.uri)
symbol_name = _find_symbol_at_position(
doc.source,
params.position.line,
params.position.character,
)
if not symbol_name:
return []
results: list[lsp.Location] = []
# For alias symbols, search for call sites across all .mrc files
# For events/menus/dialogs, only return the definition location
is_alias = not any(symbol_name.startswith(prefix) for prefix in ("on ", "raw ", "menu ", "dialog ", "ctcp "))
for uri, _path, source in _get_all_mrc_files():
if is_alias:
call_pattern = _build_call_pattern(symbol_name)
for m in call_pattern.finditer(source):
ref_line, ref_col = _get_line_col(source, m.start())
results.append(
lsp.Location(
uri=uri,
range=lsp.Range(
lsp.Position(ref_line, ref_col),
lsp.Position(ref_line, ref_col + len(m.group(0))),
),
)
)
else:
# For non-alias symbols, find the definition
for sym in parse_symbols(source):
if sym.name == symbol_name:
results.append(lsp.Location(uri=uri, range=sym.range))
return results
except Exception as e:
logger.error(f"Error finding references: {e}")
return []
@server.feature(lsp.TEXT_DOCUMENT_DEFINITION)
def definition(params: lsp.DefinitionParams) -> list[lsp.Location]:
"""Go to definition of the symbol at the given position."""
try:
doc = server.workspace.get_text_document(params.text_document.uri)
symbol_name = _find_symbol_at_position(
doc.source,
params.position.line,
params.position.character,
)
if not symbol_name:
return []
results: list[lsp.Location] = []
for uri, _path, source in _get_all_mrc_files():
for sym in parse_symbols(source):
if sym.name == symbol_name:
results.append(lsp.Location(uri=uri, range=sym.selection_range))
return results
except Exception as e:
logger.error(f"Error finding definition: {e}")
return []
@server.feature(lsp.WORKSPACE_SYMBOL)
def workspace_symbol(params: lsp.WorkspaceSymbolParams) -> list[lsp.SymbolInformation]:
"""Search for symbols across the workspace."""
query = params.query.lower()
results = []
for uri, _path, source in _get_all_mrc_files():
for sym in parse_symbols(source):
if query in sym.name.lower():
results.append(
lsp.SymbolInformation(
name=sym.name,
kind=sym.kind,
location=lsp.Location(uri=uri, range=sym.range),
container_name=sym.detail,
)
)
return results
if __name__ == "__main__":
server.start_io()
src/solidlsp/ls_config.py
+12
View File
@@ -88,6 +88,12 @@ class Language(str, Enum):
Requires MATLAB R2021b or later and Node.js.
Set MATLAB_PATH environment variable or configure matlab_path in ls_specific_settings.
"""
MSL = "msl"
"""mIRC Scripting Language (mSL) language server.
Supports .mrc files used in mIRC and AdiIRC IRC clients.
Uses a custom LSP server based on pygls. Automatically sets up
a virtual environment with pygls dependencies on first use.
"""
# Experimental or deprecated Language Servers
TYPESCRIPT_VTS = "typescript_vts"
"""Use the typescript language server through the natively bundled vscode extension via https://github.com/yioneko/vtsls"""
@@ -321,6 +327,8 @@ class Language(str, Enum):
return FilenameMatcher("*.sol")
case self.ANSIBLE:
return FilenameMatcher("*.yaml", "*.yml")
case self.MSL:
return FilenameMatcher("*.mrc")
case _:
raise ValueError(f"Unhandled language: {self}")
@@ -544,6 +552,10 @@ class Language(str, Enum):
from solidlsp.language_servers.ansible_language_server import AnsibleLanguageServer
return AnsibleLanguageServer
case self.MSL:
from solidlsp.language_servers.msl_language_server import MslLanguageServer
return MslLanguageServer
case _:
raise ValueError(f"Unhandled language: {self}")
test/conftest.py
+1
View File
@@ -259,6 +259,7 @@ _LANGUAGE_PYTEST_MARKERS: dict[Language, list[MarkDecorator | Mark]] = {
Language.JAVA: [pytest.mark.java],
Language.KOTLIN: [pytest.mark.kotlin, pytest.mark.skipif(is_ci, reason="Kotlin LSP JVM crashes on restart in CI")],
Language.LEAN4: [pytest.mark.lean4, pytest.mark.skipif(_sh.which("lean") is None, reason="Lean is not installed")],
Language.MSL: [pytest.mark.msl],
Language.PHP: [pytest.mark.php],
Language.PHP_PHPACTOR: [pytest.mark.php],
Language.POWERSHELL: [pytest.mark.powershell],
test/resources/repos/msl/test_repo/main.mrc
+38
View File
@@ -0,0 +1,38 @@
; Main game module - tests alias, event, and menu parsing
alias greet {
msg $chan Hello, $1 $+ ! Welcome aboard.
}
alias -l calculate.doubloons {
var %base = 100
var %bonus = $rand(1,50)
return $calc(%base + %bonus)
}
on *:TEXT:!hello*:#: {
greet $nick
}
on *:JOIN:#: {
.timer 1 3 greet $nick
var %coins = $format.coins(100)
msg $chan You have %coins
}
raw 319:*: {
; Handle channel list numeric
echo -a Channels: $3-
}
menu channel {
Pirates Game
.Start Game:/start
.End Game:/stop
}
alias show.player.info {
if ($is.admin($nick)) {
msg $chan $nick is an admin with $format.coins(500)
}
}
test/resources/repos/msl/test_repo/utils.mrc
+26
View File
@@ -0,0 +1,26 @@
; Utility aliases and handlers
alias format.coins {
return $bytes($1,b) doubloons
}
alias is.admin {
if ($1 isop $chan) return $true
return $false
}
alias welcome.message {
greet $1
msg $chan You have $format.coins(100) to start with!
}
dialog settings {
title "Game Settings"
size -1 -1 200 150
edit "", 1, 10 10 180 20
button "Save", 2, 60 40 80 25
}
ctcp *:VERSION:*: {
ctcpreply $nick VERSION PiratesIRC v1.0
}
test/serena/test_serena_agent.py
+3
View File
@@ -47,6 +47,7 @@ def serena_config():
Language.CPP_CCLS,
Language.HAXE,
Language.LEAN4,
Language.MSL,
]:
repo_path = get_repo_path(language)
if repo_path.exists():
@@ -199,6 +200,7 @@ class TestSerenaAgent:
pytest.param(Language.CPP_CCLS, "add", "Function", "b.cpp", marks=pytest.mark.cpp),
pytest.param(Language.HAXE, "Main", "Class", "Main.hx", marks=pytest.mark.haxe),
pytest.param(Language.LEAN4, "add", "Method", "Helper.lean", marks=pytest.mark.lean4),
pytest.param(Language.MSL, "greet", "Function", "main.mrc", marks=pytest.mark.msl),
],
indirect=["serena_agent"],
)
@@ -307,6 +309,7 @@ class TestSerenaAgent:
marks=pytest.mark.haxe,
),
pytest.param(Language.LEAN4, "add", "Helper.lean", "Main.lean", marks=pytest.mark.lean4),
pytest.param(Language.MSL, "format.coins", "utils.mrc", "main.mrc", marks=pytest.mark.msl),
],
indirect=["serena_agent"],
)
test/solidlsp/dart/test_dart_basic.py
+49
View File
@@ -375,3 +375,52 @@ class TestDartLanguageServer:
f"Found malformed symbols: {[format_symbol_for_assert(sym) for sym in malformed_symbols]}",
pytrace=False,
)
@pytest.mark.parametrize("language_server", [Language.DART], indirect=True)
def test_symbol_body_contains_full_method(self, language_server: SolidLanguageServer) -> None:
"""Test that document symbols return the full method body range, not just the identifier.
Regression test: when hierarchicalDocumentSymbolSupport was not declared in client
capabilities, the Dart LSP returned SymbolInformation[] (flat format) where
location.range only covered the identifier name (single line), causing find_symbol
with include_body=True to return just the symbol name instead of its implementation.
"""
file_path = os.path.join("lib", "main.dart")
symbols = language_server.request_document_symbols(file_path).get_all_symbols_and_roots()
symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols
# Find the 'add' method — defined across multiple lines in main.dart:
# int add(int a, int b) {
# final result = a + b;
# _history.add('$a + $b = $result');
# return result;
# }
add_symbol = None
for sym in symbol_list:
if sym.get("name") == "add":
add_symbol = sym
break
if sym.get("name") == "Calculator" and "children" in sym:
for child in sym["children"]:
if child.get("name") == "add":
add_symbol = child
break
if add_symbol:
break
assert add_symbol is not None, "Could not find 'add' method symbol in main.dart"
# The body range must span multiple lines (not just the identifier line).
# With hierarchicalDocumentSymbolSupport declared, the Dart LSP returns
# DocumentSymbol[] where range covers the full method body.
body_start = add_symbol["location"]["range"]["start"]["line"]
body_end = add_symbol["location"]["range"]["end"]["line"]
assert body_end > body_start, (
f"Expected multi-line body range for 'add' method, got start={body_start}, end={body_end}. "
f"This likely means hierarchicalDocumentSymbolSupport is not declared in client capabilities."
)
# The body text must contain the method implementation, not just the name.
if add_symbol.get("body"):
body_text = add_symbol["body"].get_text()
assert "return result" in body_text, f"Expected method body to contain implementation, got: {body_text!r}"
test/solidlsp/msl/__init__.py
View File
Whitespace-only changes.
test/solidlsp/msl/test_msl_basic.py
+161
View File
@@ -0,0 +1,161 @@
"""
Basic integration tests for the mSL (mIRC Scripting Language) language server.
Tests validate document symbols, references, and definitions for aliases, events,
raw events, menus, dialogs, and CTCP handlers using the mSL test repository.
"""
import os
import pytest
from solidlsp import SolidLanguageServer
from solidlsp.ls_config import Language
from test.solidlsp.conftest import format_symbol_for_assert, has_malformed_name, request_all_symbols
pytestmark = [pytest.mark.msl]
class TestMslDocumentSymbols:
"""Test document symbol retrieval for mSL constructs."""
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_ls_is_running(self, language_server: SolidLanguageServer) -> None:
"""Test that the language server starts successfully."""
assert language_server.is_running()
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_document_symbols_main(self, language_server: SolidLanguageServer) -> None:
"""Test that document symbols are returned for the main file."""
doc_symbols = language_server.request_document_symbols("main.mrc")
all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots()
symbol_names = [s.get("name") for s in all_symbols if s.get("name")]
assert "greet" in symbol_names, f"greet alias not found. Found: {symbol_names}"
assert "calculate.doubloons" in symbol_names, f"calculate.doubloons alias not found. Found: {symbol_names}"
assert "show.player.info" in symbol_names, f"show.player.info alias not found. Found: {symbol_names}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_document_symbols_events(self, language_server: SolidLanguageServer) -> None:
"""Test that event handlers, raw events, and menus are detected in the main file."""
doc_symbols = language_server.request_document_symbols("main.mrc")
all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots()
symbol_names = [s.get("name") for s in all_symbols if s.get("name")]
# Check for on *:TEXT and on *:JOIN events
on_events = [n for n in symbol_names if n.startswith("on ")]
assert len(on_events) >= 2, f"Expected at least 2 event handlers. Found: {on_events}"
# Check for raw event
raw_events = [n for n in symbol_names if n.startswith("raw ")]
assert len(raw_events) >= 1, f"Expected at least 1 raw event handler. Found: {raw_events}"
# Check for menu
menus = [n for n in symbol_names if n.startswith("menu ")]
assert len(menus) >= 1, f"Expected at least 1 menu. Found: {menus}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_document_symbols_utils(self, language_server: SolidLanguageServer) -> None:
"""Test that document symbols are returned for the utils file."""
doc_symbols = language_server.request_document_symbols("utils.mrc")
all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots()
symbol_names = [s.get("name") for s in all_symbols if s.get("name")]
assert "format.coins" in symbol_names, f"format.coins alias not found. Found: {symbol_names}"
assert "is.admin" in symbol_names, f"is.admin alias not found. Found: {symbol_names}"
assert "welcome.message" in symbol_names, f"welcome.message alias not found. Found: {symbol_names}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_document_symbols_dialog_and_ctcp(self, language_server: SolidLanguageServer) -> None:
"""Test that dialog and CTCP handler definitions are detected."""
doc_symbols = language_server.request_document_symbols("utils.mrc")
all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots()
symbol_names = [s.get("name") for s in all_symbols if s.get("name")]
assert "dialog settings" in symbol_names, f"dialog settings not found. Found: {symbol_names}"
# Check for CTCP handler
ctcp_events = [n for n in symbol_names if n.startswith("ctcp ")]
assert len(ctcp_events) >= 1, f"Expected at least 1 ctcp handler. Found: {ctcp_events}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_find_symbol(self, language_server: SolidLanguageServer) -> None:
"""Test that the full symbol tree contains expected symbols from both files."""
from solidlsp.ls_utils import SymbolUtils
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "greet"), "greet not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "format.coins"), "format.coins not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "show.player.info"), "show.player.info not found in symbol tree"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_bare_symbol_names(self, language_server: SolidLanguageServer) -> None:
"""Test that symbol names do not contain unexpected formatting characters."""
all_symbols = request_all_symbols(language_server)
malformed_symbols = []
for s in all_symbols:
# mSL symbols can contain periods (e.g., calculate.doubloons) and
# colons/spaces in event names (e.g., "on *:TEXT"), so allow those
if has_malformed_name(s, period_allowed=True, colon_allowed=True, whitespace_allowed=True):
malformed_symbols.append(s)
if malformed_symbols:
pytest.fail(
f"Found malformed symbols: {[format_symbol_for_assert(sym) for sym in malformed_symbols]}",
pytrace=False,
)
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_find_references_within_file(self, language_server: SolidLanguageServer) -> None:
"""Test that references to 'greet' are found within main.mrc."""
file_path = "main.mrc"
all_symbols, _ = language_server.request_document_symbols(file_path).get_all_symbols_and_roots()
greet_symbol = next((s for s in all_symbols if s.get("name") == "greet"), None)
assert greet_symbol is not None, "Could not find 'greet' symbol in main.mrc"
sel_start = greet_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert refs, f"Expected non-empty references for greet but got {refs=}"
actual_locations = [
{
"uri_suffix": os.path.basename(ref.get("relativePath", ref.get("uri", ""))),
"line": ref["range"]["start"]["line"],
}
for ref in refs
]
# greet is called on line 13 (0-indexed) in main.mrc: `greet $nick`
call_site = {"uri_suffix": "main.mrc", "line": 13}
assert call_site in actual_locations, f"Expected reference to greet at line 13 in main.mrc, got {actual_locations}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_find_references_across_files(self, language_server: SolidLanguageServer) -> None:
"""Test that references to 'format.coins' are found across main.mrc and utils.mrc."""
# format.coins is defined in utils.mrc but called in both main.mrc and utils.mrc
file_path = "utils.mrc"
all_symbols, _ = language_server.request_document_symbols(file_path).get_all_symbols_and_roots()
fc_symbol = next((s for s in all_symbols if s.get("name") == "format.coins"), None)
assert fc_symbol is not None, "Could not find 'format.coins' symbol in utils.mrc"
sel_start = fc_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert refs, f"Expected non-empty references for format.coins but got {refs=}"
actual_locations = [
{
"uri_suffix": os.path.basename(ref.get("relativePath", ref.get("uri", ""))),
"line": ref["range"]["start"]["line"],
}
for ref in refs
]
# Verify cross-file: at least one reference is in main.mrc (different file from definition)
main_refs = [loc for loc in actual_locations if loc["uri_suffix"] == "main.mrc"]
assert len(main_refs) >= 1, f"Expected at least 1 reference in main.mrc, got {main_refs}"
@pytest.mark.parametrize("language_server", [Language.MSL], indirect=True)
def test_workspace_symbol(self, language_server: SolidLanguageServer) -> None:
"""Test that workspace symbol search returns results."""
result = language_server.request_workspace_symbol("greet")
assert result is not None, "Workspace symbol search returned None"
assert len(result) > 0, "Workspace symbol search returned no results"
assert any("greet" in str(s.get("name", "")) for s in result), f"Expected at least one result containing 'greet', got {result}"
uv.lock
Generated
+45 -1
View File
@@ -243,6 +243,19 @@ filecache = [
{ name = "filelock" },
]
[[package]]
name = "cattrs"
version = "26.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "attrs" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/a0/ec/ba18945e7d6e55a58364d9fb2e46049c1c2998b3d805f19b703f14e81057/cattrs-26.1.0.tar.gz", hash = "sha256:fa239e0f0ec0715ba34852ce813986dfed1e12117e209b816ab87401271cdd40", size = 495672, upload-time = "2026-02-18T22:15:19.406Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/80/56/60547f7801b97c67e97491dc3d9ade9fbccbd0325058fd3dfcb2f5d98d90/cattrs-26.1.0-py3-none-any.whl", hash = "sha256:d1e0804c42639494d469d08d4f26d6b9de9b8ab26b446db7b5f8c2e97f7c3096", size = 73054, upload-time = "2026-02-18T22:15:17.958Z" },
]
[[package]]
name = "certifi"
version = "2026.2.25"
@@ -1291,6 +1304,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/b4/de/88b3be5c31b22333b3ca2f6ff1de4e863d8fe45aaea7485f591970ec1d3e/linkify_it_py-2.1.0-py3-none-any.whl", hash = "sha256:0d252c1594ecba2ecedc444053db5d3a9b7ec1b0dd929c8f1d74dce89f86c05e", size = 19878, upload-time = "2026-03-01T07:48:46.098Z" },
]
[[package]]
name = "lsprotocol"
version = "2025.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "attrs" },
{ name = "cattrs" },
]
sdist = { url = "https://files.pythonhosted.org/packages/e9/26/67b84e6ec1402f0e6764ef3d2a0aaf9a79522cc1d37738f4e5bb0b21521a/lsprotocol-2025.0.0.tar.gz", hash = "sha256:e879da2b9301e82cfc3e60d805630487ac2f7ab17492f4f5ba5aaba94fe56c29", size = 74896, upload-time = "2025-06-17T21:30:18.156Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7b/f0/92f2d609d6642b5f30cb50a885d2bf1483301c69d5786286500d15651ef2/lsprotocol-2025.0.0-py3-none-any.whl", hash = "sha256:f9d78f25221f2a60eaa4a96d3b4ffae011b107537facee61d3da3313880995c7", size = 76250, upload-time = "2025-06-17T21:30:19.455Z" },
]
[[package]]
name = "markdown-it-py"
version = "3.0.0"
@@ -2143,6 +2169,20 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/d3/c622950d87a2ffd1654208733b5bd1c5645930014abed8f4c0d74863988b/pydata_sphinx_theme-0.15.4-py3-none-any.whl", hash = "sha256:2136ad0e9500d0949f96167e63f3e298620040aea8f9c74621959eda5d4cf8e6", size = 4640157, upload-time = "2024-06-25T19:28:42.383Z" },
]
[[package]]
name = "pygls"
version = "2.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "attrs" },
{ name = "cattrs" },
{ name = "lsprotocol" },
]
sdist = { url = "https://files.pythonhosted.org/packages/da/2e/7bbe061d175c0baddde8fc9edb908a4c31ba5d9165b8c68e3439c3a9f138/pygls-2.1.1.tar.gz", hash = "sha256:1da03ba9053201bb337dcdd8d121df70feb2a91e1a0dcc74de5da79755b1a201", size = 55091, upload-time = "2026-03-25T11:19:10.541Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fd/1a/208293b6c350f5abea6941d5606080d4a492644052504f5312e5de30a902/pygls-2.1.1-py3-none-any.whl", hash = "sha256:510a6dea2476177230c7d851125e5948efdf3fdb9ebfd8543fc434972f8faed4", size = 68975, upload-time = "2026-03-25T11:19:11.374Z" },
]
[[package]]
name = "pygments"
version = "2.19.2"
@@ -2911,7 +2951,7 @@ wheels = [
[[package]]
name = "serena-agent"
version = "1.1.0"
version = "1.1.1"
source = { editable = "." }
dependencies = [
{ name = "anthropic" },
@@ -2924,12 +2964,14 @@ dependencies = [
{ name = "fortls" },
{ name = "jinja2" },
{ name = "joblib" },
{ name = "lsprotocol" },
{ name = "mcp" },
{ name = "oraios-pywebview" },
{ name = "overrides" },
{ name = "pathspec" },
{ name = "psutil" },
{ name = "pydantic" },
{ name = "pygls" },
{ name = "pyright" },
{ name = "pystray" },
{ name = "python-dotenv" },
@@ -2998,6 +3040,7 @@ requires-dist = [
{ name = "jinja2", marker = "extra == 'dev'", specifier = "==3.1.6" },
{ name = "joblib", specifier = "==1.5.1" },
{ name = "jupyter-book", marker = "extra == 'dev'", specifier = "==1.0.4.post1" },
{ name = "lsprotocol", specifier = "==2025.0.0" },
{ name = "mcp", specifier = "==1.26.0" },
{ name = "mypy", marker = "extra == 'dev'", specifier = "==1.17.0" },
{ name = "nbconvert", marker = "extra == 'dev'", specifier = "==7.17.0" },
@@ -3009,6 +3052,7 @@ requires-dist = [
{ name = "psutil", specifier = "==7.0.0" },
{ name = "pyasn1", marker = "extra == 'dev'", specifier = "==0.6.3" },
{ name = "pydantic", specifier = "==2.12.5" },
{ name = "pygls", specifier = "==2.1.1" },
{ name = "pyinstrument", marker = "extra == 'dev'", specifier = "==5.1.1" },
{ name = "pyjwt", marker = "extra == 'dev'", specifier = "==2.12.0" },
{ name = "pyright", specifier = "==1.1.403" },