From ab67ecab3b64f4fd91f4c92c73281c8babe87b87 Mon Sep 17 00:00:00 2001 From: Pirates IRC <98669745+PiratesIRC@users.noreply.github.com> Date: Mon, 13 Apr 2026 01:47:51 -0500 Subject: [PATCH] feat: add mSL (mIRC Scripting Language) support (#1315) * feat: add mSL (mIRC Scripting Language) support Add language server support for mIRC Scripting Language (.mrc files). mSL is used in mIRC and AdiIRC IRC clients for scripting bots, games, and automation. The implementation uses a custom Python-based LSP server (pygls) that parses aliases, events, menus, dialogs, and CTCP handlers. Dependencies (pygls, lsprotocol) are installed in an isolated venv on first use. Includes test repo, integration tests, and documentation updates. * style: apply ruff formatting to msl_language_server.py * fix: remove INITIALIZE handler that crashes pygls 2.x pygls 2.x handles the initialize request internally. Overriding it via @server.feature(lsp.INITIALIZE) causes the LSP subprocess to crash. Removing the handler lets pygls auto-advertise capabilities based on registered features (document_symbol, workspace_symbol). * fix: update embedded LSP script for pygls 2.x pygls 2.x moved LanguageServer from pygls.server to pygls.lsp.server. Update import and bump requirement from pygls>=1.3.0 to pygls>=2.0.0. * refactor: ship mSL LSP as package module, not runtime disk write Per maintainer feedback: the mSL LSP is a Python script, so it should ship as a module inside the package rather than being written to disk at runtime. Changes: - Extract embedded LSP script to msl_lsp_server.py (proper module) - Simplify MslLanguageServer: remove venv creation, disk writes, _create_msl_lsp_files(). DependencyProvider now just returns sys.executable and launches the sibling module directly. - Add pygls>=2.0.0 and lsprotocol>=2023.0.0 to pyproject.toml deps * fix: regenerate uv.lock after upstream v1.1.0 merge * feat(msl): add references, definitions, and expand test coverage - Add textDocument/references handler for cross-file alias reference finding - Add textDocument/definition handler for go-to-definition support - Handle $ prefix on cursor position for mSL identifiers - Fix pre-existing mypy error in agent.py (dict[str, object] -> dict[str, str | int]) - Add MSL to _LANGUAGE_PYTEST_MARKERS in conftest.py - Add MSL parametrized cases to test_find_symbol_stable and test_find_symbol_references_stable - Expand test repo with cross-file reference examples (main.mrc <-> utils.mrc) - Add within-file and cross-file reference tests to test_msl_basic.py - Update docs and CHANGELOG to reflect expanded capabilities Co-Authored-By: Claude Opus 4.6 (1M context) * style: apply ruff formatting to msl_lsp_server.py Co-Authored-By: Claude Opus 4.6 (1M context) * fix(msl): add hover handler, fix cross-file references via filesystem scan Root causes of 4 CI test failures: - textDocument/hover was not implemented (test_find_symbol_stable needs it) - references/workspace_symbol/definition only searched opened documents, missing files not explicitly opened by the client - pygls.uris.to_fs_path returns lowercase drive letters on Windows, causing URI mismatch with the framework's repository path comparison Fixes: - Add textDocument/hover handler returning definition snippets in Markdown - Add _get_all_mrc_files() to scan workspace filesystem for all .mrc files - Read workspace roots from server.workspace (pygls 2.x API) instead of broken monkey-patching of server.lsp - Use pathlib.Path.resolve() to normalize drive letter casing on Windows - Simplify __main__ to just server.start_io() All 12 tests pass locally (10 solidlsp + 2 serena_agent integration). Co-Authored-By: Claude Opus 4.6 (1M context) * MSL language: removed unnecessary Dep. provider * Changelog * deps --------- Co-authored-by: Claude Opus 4.6 (1M context) Co-authored-by: Michael Panchenko --- CHANGELOG.md | 4 + README.md | 2 +- docs/01-about/020_programming-languages.md | 3 + pyproject.toml | 3 + .../language_servers/msl_language_server.py | 107 +++++ .../language_servers/msl_lsp_server.py | 439 ++++++++++++++++++ src/solidlsp/ls_config.py | 12 + test/conftest.py | 1 + test/resources/repos/msl/test_repo/main.mrc | 38 ++ test/resources/repos/msl/test_repo/utils.mrc | 26 ++ test/serena/test_serena_agent.py | 3 + test/solidlsp/msl/__init__.py | 0 test/solidlsp/msl/test_msl_basic.py | 161 +++++++ uv.lock | 44 ++ 14 files changed, 842 insertions(+), 1 deletion(-) create mode 100644 src/solidlsp/language_servers/msl_language_server.py create mode 100644 src/solidlsp/language_servers/msl_lsp_server.py create mode 100644 test/resources/repos/msl/test_repo/main.mrc create mode 100644 test/resources/repos/msl/test_repo/utils.mrc create mode 100644 test/solidlsp/msl/__init__.py create mode 100644 test/solidlsp/msl/test_msl_basic.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 12eb66de..35f4a3d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,9 @@ Status of the `main` branch. Changes prior to the next official version change w * General: - Support environment variable `SERENA_USAGE_REPORTING` (set to `false` to disable usage reporting) +* Language Servers: + - Add mSL (mIRC Scripting Language) support (custom pygls-based language server; symbols, references, definitions) + # 1.1.1 * General: @@ -16,6 +19,7 @@ Status of the `main` branch. Changes prior to the next official version change w * Language Servers: - Fix Dart LSP returning only symbol name as body instead of full method body. + # 1.1.0 * General: diff --git a/README.md b/README.md index 5f07841b..f1b513d1 100644 --- a/README.md +++ b/README.md @@ -71,7 +71,7 @@ Serena incorporates a powerful abstraction layer for the integration of language The underlying language servers are typically open-source projects or at least freely available for use. When using Serena's language server backend, we provide **support for over 40 programming languages**, including -AL, Ansible, Bash, C#, C/C++, Clojure, Crystal, Dart, Elixir, Elm, Erlang, Fortran, F#, GLSL, Go, Groovy, Haskell, Haxe, HLSL, Java, JavaScript, Julia, Kotlin, Lean 4, Lua, Luau, Markdown, MATLAB, Nix, OCaml, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Solidity, Swift, TOML, TypeScript, WGSL, YAML, and Zig. +AL, Ansible, Bash, C#, C/C++, Clojure, Crystal, Dart, Elixir, Elm, Erlang, Fortran, F#, GLSL, Go, Groovy, Haskell, Haxe, HLSL, Java, JavaScript, Julia, Kotlin, Lean 4, Lua, Luau, Markdown, MATLAB, mSL, Nix, OCaml, Perl, PHP, PowerShell, Python, R, Ruby, Rust, Scala, Solidity, Swift, TOML, TypeScript, WGSL, YAML, and Zig. ### The Serena JetBrains Plugin diff --git a/docs/01-about/020_programming-languages.md b/docs/01-about/020_programming-languages.md index 33fcd54d..999782ee 100644 --- a/docs/01-about/020_programming-languages.md +++ b/docs/01-about/020_programming-languages.md @@ -84,6 +84,9 @@ Some languages require additional installations or setup steps, as noted. * **Luau** * **Markdown** (must explicitly enable language `markdown`, primarily useful for documentation-heavy projects) +* **mSL** (mIRC Scripting Language) + (auto-installed; no external dependencies required — uses a custom pygls-based LSP server shipped with Serena; + supports document symbols, workspace symbols, references, and go-to-definition for aliases, events, menus, dialogs, and CTCP handlers in `.mrc` files) * **Nix** (requires nixd installation) * **OCaml** diff --git a/pyproject.toml b/pyproject.toml index c6daf2c9..8894c98f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -38,6 +38,8 @@ dependencies = [ "beautifulsoup4==4.14.2", "oraios-pywebview==6.2", "pystray==0.19.5", + "pygls==2.1.1", # used to implement an msl language server + "lsprotocol==2025.0.0", # Transitive deps pinned for security (dependabot alerts). # Exact pins because uvx installs from git, ignoring the lock file. "urllib3==2.6.3", @@ -344,6 +346,7 @@ markers = [ "lean4: language server running for Lean 4", "solidity: language server running for Solidity (uses @nomicfoundation/solidity-language-server)", "ansible: language server running for Ansible (uses @ansible/ansible-language-server)", + "msl: language server running for mSL (mIRC Scripting Language)", ] [tool.codespell] diff --git a/src/solidlsp/language_servers/msl_language_server.py b/src/solidlsp/language_servers/msl_language_server.py new file mode 100644 index 00000000..e2596a05 --- /dev/null +++ b/src/solidlsp/language_servers/msl_language_server.py @@ -0,0 +1,107 @@ +""" +Provides mSL (mIRC Scripting Language) specific instantiation of the LanguageServer class. +Uses a custom Python-based LSP server (pygls) for parsing .mrc files. + +The LSP server script is shipped as ``msl_lsp_server.py`` alongside this module +and launched as a subprocess using the current Python interpreter. +""" + +import logging +import os +import pathlib +import sys +import threading + +from solidlsp.ls import ( + SolidLanguageServer, +) +from solidlsp.ls_config import LanguageServerConfig +from solidlsp.lsp_protocol_handler.lsp_types import InitializeParams +from solidlsp.lsp_protocol_handler.server import ProcessLaunchInfo +from solidlsp.settings import SolidLSPSettings + +log = logging.getLogger(__name__) + +_MSL_LSP_SCRIPT = os.path.join(os.path.dirname(__file__), "msl_lsp_server.py") + + +class MslLanguageServer(SolidLanguageServer): + """ + Provides mSL (mIRC Scripting Language) specific instantiation of the LanguageServer class. + Uses a Python-based LSP server for parsing .mrc files (aliases, events, menus, dialogs). + """ + + def __init__(self, config: LanguageServerConfig, repository_root_path: str, solidlsp_settings: SolidLSPSettings): + """ + Creates an MslLanguageServer instance. This class is not meant to be instantiated directly. + Use LanguageServer.create() instead. + """ + process_launch_info = ProcessLaunchInfo(cmd=[sys.executable, _MSL_LSP_SCRIPT], cwd=repository_root_path) + super().__init__( + config, + repository_root_path, + process_launch_info=process_launch_info, + language_id="msl", + solidlsp_settings=solidlsp_settings, + ) + self.server_ready = threading.Event() + + @staticmethod + def _get_initialize_params(repository_absolute_path: str) -> InitializeParams: + """Returns the initialize params for the mSL Language Server.""" + root_uri = pathlib.Path(repository_absolute_path).as_uri() + initialize_params = { + "locale": "en", + "capabilities": { + "textDocument": { + "synchronization": {"didSave": True, "dynamicRegistration": True}, + "documentSymbol": { + "dynamicRegistration": True, + "hierarchicalDocumentSymbolSupport": True, + "symbolKind": {"valueSet": list(range(1, 27))}, + }, + "hover": {"dynamicRegistration": True, "contentFormat": ["markdown", "plaintext"]}, + "references": {"dynamicRegistration": True}, + "definition": {"dynamicRegistration": True}, + }, + "workspace": { + "workspaceFolders": True, + "symbol": {"dynamicRegistration": True}, + }, + }, + "processId": os.getpid(), + "rootPath": repository_absolute_path, + "rootUri": root_uri, + "workspaceFolders": [{"uri": root_uri, "name": os.path.basename(repository_absolute_path)}], + } + return initialize_params # type: ignore + + def _start_server(self) -> None: + """Starts the mSL Language Server.""" + + def window_log_message(msg: dict) -> None: + log.info(f"LSP: window/logMessage: {msg}") + self.server_ready.set() + + def do_nothing(params: dict) -> None: + pass + + self.server.on_notification("window/logMessage", window_log_message) + self.server.on_notification("textDocument/publishDiagnostics", do_nothing) + + log.info("Starting mSL server process") + self.server.start() + initialize_params = self._get_initialize_params(self.repository_root_path) + + log.info("Sending initialize request to mSL LSP server") + init_response = self.server.send.initialize(initialize_params) + log.debug(f"Received initialize response: {init_response}") + + self.server.notify.initialized({}) + + # Wait briefly for server readiness + if not self.server_ready.wait(timeout=2.0): + log.info("Timeout waiting for mSL server ready signal, proceeding anyway") + self.server_ready.set() + + log.info("mSL server initialization complete") diff --git a/src/solidlsp/language_servers/msl_lsp_server.py b/src/solidlsp/language_servers/msl_lsp_server.py new file mode 100644 index 00000000..e87d3d16 --- /dev/null +++ b/src/solidlsp/language_servers/msl_lsp_server.py @@ -0,0 +1,439 @@ +"""mSL (mIRC Scripting Language) Language Server. + +A minimal LSP implementation for mIRC scripting language (.mrc files). +Provides document symbols, hover, references, go-to-definition, and workspace symbols +for aliases, events, menus, dialogs, and CTCP handlers. + +Launched as a subprocess by MslLanguageServer. Communicates via stdio. +""" + +import logging +import os +import pathlib +import re + +from lsprotocol import types as lsp +from pygls.lsp.server import LanguageServer + +logging.basicConfig(level=logging.INFO) +logger = logging.getLogger(__name__) + +server = LanguageServer("msl-lsp", "0.1.0") + +# mSL top-level construct patterns +ALIAS_PATTERN = re.compile( + r"^[ \t]*alias\s+(?:-l\s+)?([a-zA-Z_][\w.]*)\s*(?:\{|$)", + re.MULTILINE | re.IGNORECASE, +) +EVENT_PATTERN = re.compile( + r"^[ \t]*on\s+(\*|\d+):(\w+):([^{]*?)(?:\{|$)", + re.MULTILINE | re.IGNORECASE, +) +RAW_EVENT_PATTERN = re.compile( + r"^[ \t]*raw\s+(\d+):([^{]*?)(?:\{|$)", + re.MULTILINE | re.IGNORECASE, +) +MENU_PATTERN = re.compile( + r"^[ \t]*menu\s+([^\s{]+)\s*\{", + re.MULTILINE | re.IGNORECASE, +) +DIALOG_PATTERN = re.compile( + r"^[ \t]*dialog\s+(-l\s+)?([a-zA-Z_][\w]*)\s*\{", + re.MULTILINE | re.IGNORECASE, +) +CTCP_PATTERN = re.compile( + r"^[ \t]*ctcp\s+(\*|\d+):(\w+):([^{]*?)(?:\{|$)", + re.MULTILINE | re.IGNORECASE, +) + + +def _get_line_col(text: str, pos: int) -> tuple[int, int]: + lines = text[:pos].split("\n") + return len(lines) - 1, len(lines[-1]) if lines else 0 + + +def _find_block_end(text: str, start: int) -> int: + count, i = 0, start + while i < len(text): + ch = text[i] + if ch == ";" and count > 0: + while i < len(text) and text[i] != "\n": + i += 1 + continue + if ch == "{": + count += 1 + elif ch == "}": + count -= 1 + if count == 0: + return i + i += 1 + return len(text) - 1 + + +def _make_symbol( + name: str, + kind: lsp.SymbolKind, + detail: str, + text: str, + match_start: int, + match_end: int, + match_text: str, +) -> lsp.DocumentSymbol: + sl, sc = _get_line_col(text, match_start) + bs = text.find("{", match_start) + if bs != -1: + el, ec = _get_line_col(text, _find_block_end(text, bs)) + else: + el, ec = _get_line_col(text, match_end) + return lsp.DocumentSymbol( + name=name, + kind=kind, + range=lsp.Range(lsp.Position(sl, 0), lsp.Position(el, ec + 1)), + selection_range=lsp.Range(lsp.Position(sl, sc), lsp.Position(sl, sc + len(match_text))), + detail=detail, + ) + + +def parse_symbols(text: str) -> list[lsp.DocumentSymbol]: + """Parse mSL source code and return document symbols.""" + symbols: list[lsp.DocumentSymbol] = [] + + for m in ALIAS_PATTERN.finditer(text): + symbols.append(_make_symbol(m.group(1), lsp.SymbolKind.Function, "alias", text, m.start(), m.end(), m.group(0))) + + for m in EVENT_PATTERN.finditer(text): + pat = m.group(3).strip().rstrip(":") + name = f"on {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "") + symbols.append(_make_symbol(name, lsp.SymbolKind.Event, f"event:{m.group(2)}", text, m.start(), m.end(), m.group(0))) + + for m in RAW_EVENT_PATTERN.finditer(text): + pat = m.group(2).strip().rstrip(":") + name = f"raw {m.group(1)}" + (f":{pat}" if pat else "") + symbols.append(_make_symbol(name, lsp.SymbolKind.Event, "raw event", text, m.start(), m.end(), m.group(0))) + + for m in MENU_PATTERN.finditer(text): + symbols.append(_make_symbol(f"menu {m.group(1)}", lsp.SymbolKind.Module, "menu", text, m.start(), m.end(), m.group(0))) + + for m in DIALOG_PATTERN.finditer(text): + symbols.append(_make_symbol(f"dialog {m.group(2)}", lsp.SymbolKind.Class, "dialog", text, m.start(), m.end(), m.group(0))) + + for m in CTCP_PATTERN.finditer(text): + pat = m.group(3).strip().rstrip(":") + name = f"ctcp {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "") + symbols.append(_make_symbol(name, lsp.SymbolKind.Event, "ctcp event", text, m.start(), m.end(), m.group(0))) + + symbols.sort(key=lambda s: s.range.start.line) + return symbols + + +def _get_workspace_roots() -> list[str]: + """Get workspace root paths from the server's workspace object. + + After initialization, pygls populates ``server.workspace`` with the + root URI/path and any workspace folders sent by the client. + """ + from pygls.uris import to_fs_path + + roots: list[str] = [] + try: + ws = server.workspace + except (RuntimeError, AttributeError): + return roots + + # workspace.folders is a dict of {uri_string: WorkspaceFolder} + if hasattr(ws, "folders") and ws.folders: + for folder_uri in ws.folders: + fs_path = to_fs_path(folder_uri) + if fs_path: + # resolve() normalizes drive letter casing on Windows + fs_path = str(pathlib.Path(fs_path).resolve()) + if os.path.isdir(fs_path): + roots.append(fs_path) + + # Fall back to root_path + if not roots and hasattr(ws, "root_path") and ws.root_path: + root = str(pathlib.Path(ws.root_path).resolve()) + if os.path.isdir(root): + roots.append(root) + + return roots + + +def _path_to_uri(path: str) -> str: + """Convert a filesystem path to a file URI.""" + return pathlib.Path(path).as_uri() + + +def _get_all_mrc_files() -> list[tuple[str, str, str]]: + """Scan workspace roots for all .mrc files. + + Returns list of (uri, file_path, source_text) tuples. + Prefers content from already-opened documents over disk reads. + """ + results: list[tuple[str, str, str]] = [] + seen_paths: set[str] = set() + + # First, include all files currently open in the workspace + try: + for uri, doc in server.workspace.text_documents.items(): + if uri.endswith(".mrc"): + # Resolve the filesystem path from the URI + from pygls.uris import to_fs_path + + file_path = to_fs_path(uri) or uri + norm = os.path.normcase(os.path.normpath(file_path)) + seen_paths.add(norm) + results.append((uri, file_path, doc.source)) + except (RuntimeError, AttributeError): + pass + + # Then scan workspace roots for any .mrc files not yet opened + for root in _get_workspace_roots(): + for dirpath, _dirnames, filenames in os.walk(root): + for fname in filenames: + if not fname.endswith(".mrc"): + continue + full_path = os.path.join(dirpath, fname) + norm = os.path.normcase(os.path.normpath(full_path)) + if norm in seen_paths: + continue + seen_paths.add(norm) + try: + with open(full_path, encoding="utf-8", errors="replace") as f: + source = f.read() + uri = _path_to_uri(full_path) + results.append((uri, full_path, source)) + except OSError: + pass + + return results + + +def _find_symbol_at_position(text: str, line: int, character: int) -> str | None: + """Find the symbol name at the given position in the text.""" + lines = text.split("\n") + if line >= len(lines): + return None + line_text = lines[line] + + # Check if position is within an alias definition + for m in ALIAS_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + return m.group(1) + + # Check if position is within an event definition + for m in EVENT_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + pat = m.group(3).strip().rstrip(":") + return f"on {m.group(1)}:{m.group(2)}" + (f":{pat}" if pat else "") + + # Check if position is within a raw event + for m in RAW_EVENT_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + pat = m.group(2).strip().rstrip(":") + return f"raw {m.group(1)}" + (f":{pat}" if pat else "") + + # Check if position is within a menu + for m in MENU_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + return f"menu {m.group(1)}" + + # Check if position is within a dialog + for m in DIALOG_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + return f"dialog {m.group(2)}" + + # Check if position is within a CTCP handler + for m in CTCP_PATTERN.finditer(text): + sl, _ = _get_line_col(text, m.start()) + if sl == line: + return f"ctcp {m.group(1)}:{m.group(2)}" + + # Fall back: extract word at position (for alias call sites) + if character < len(line_text): + pos = character + # Skip leading $ if cursor is on it (e.g., $format.coins) + if pos < len(line_text) and line_text[pos] == "$": + pos += 1 + start = pos + while start > 0 and (line_text[start - 1].isalnum() or line_text[start - 1] in "_."): + start -= 1 + end = pos + while end < len(line_text) and (line_text[end].isalnum() or line_text[end] in "_."): + end += 1 + word = line_text[start:end] + if word: + return word + return None + + +def _get_symbol_detail(text: str, symbol_name: str) -> str | None: + """Get detail/documentation text for a symbol by finding its definition and extracting context.""" + for sym in parse_symbols(text): + if sym.name == symbol_name: + # Extract the definition line and a few lines of body for hover + text_lines = text.split("\n") + start_line = sym.range.start.line + end_line = min(sym.range.end.line, start_line + 5) + snippet_lines = text_lines[start_line : end_line + 1] + snippet = "\n".join(snippet_lines) + return f"```msl\n{snippet}\n```\n\n**Kind**: {sym.detail}" + return None + + +def _build_call_pattern(alias_name: str) -> re.Pattern[str]: + """Build a regex that matches calls to an alias (both as command and as $identifier).""" + escaped = re.escape(alias_name) + return re.compile( + rf"(? None: + pass + + +@server.feature(lsp.TEXT_DOCUMENT_DID_CHANGE) +def did_change(params: lsp.DidChangeTextDocumentParams) -> None: + pass + + +@server.feature(lsp.TEXT_DOCUMENT_DID_CLOSE) +def did_close(params: lsp.DidCloseTextDocumentParams) -> None: + pass + + +@server.feature(lsp.TEXT_DOCUMENT_DOCUMENT_SYMBOL) +def document_symbol(params: lsp.DocumentSymbolParams) -> list[lsp.DocumentSymbol]: + """Return document symbols for the given document.""" + try: + doc = server.workspace.get_text_document(params.text_document.uri) + return parse_symbols(doc.source) + except Exception as e: + logger.error(f"Error: {e}") + return [] + + +@server.feature(lsp.TEXT_DOCUMENT_HOVER) +def hover(params: lsp.HoverParams) -> lsp.Hover | None: + """Return hover information for the symbol at the given position.""" + try: + doc = server.workspace.get_text_document(params.text_document.uri) + symbol_name = _find_symbol_at_position(doc.source, params.position.line, params.position.character) + if not symbol_name: + return None + + # Search all files for the symbol definition to get its detail + for _uri, _path, source in _get_all_mrc_files(): + detail = _get_symbol_detail(source, symbol_name) + if detail: + return lsp.Hover( + contents=lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=detail), + ) + + # Fallback: return the symbol name itself + return lsp.Hover( + contents=lsp.MarkupContent(kind=lsp.MarkupKind.Markdown, value=f"**{symbol_name}**"), + ) + except Exception as e: + logger.error(f"Error in hover: {e}") + return None + + +@server.feature(lsp.TEXT_DOCUMENT_REFERENCES) +def references(params: lsp.ReferenceParams) -> list[lsp.Location]: + """Find all references to the symbol at the given position across the workspace.""" + try: + doc = server.workspace.get_text_document(params.text_document.uri) + symbol_name = _find_symbol_at_position( + doc.source, + params.position.line, + params.position.character, + ) + if not symbol_name: + return [] + + results: list[lsp.Location] = [] + + # For alias symbols, search for call sites across all .mrc files + # For events/menus/dialogs, only return the definition location + is_alias = not any(symbol_name.startswith(prefix) for prefix in ("on ", "raw ", "menu ", "dialog ", "ctcp ")) + + for uri, _path, source in _get_all_mrc_files(): + if is_alias: + call_pattern = _build_call_pattern(symbol_name) + for m in call_pattern.finditer(source): + ref_line, ref_col = _get_line_col(source, m.start()) + results.append( + lsp.Location( + uri=uri, + range=lsp.Range( + lsp.Position(ref_line, ref_col), + lsp.Position(ref_line, ref_col + len(m.group(0))), + ), + ) + ) + else: + # For non-alias symbols, find the definition + for sym in parse_symbols(source): + if sym.name == symbol_name: + results.append(lsp.Location(uri=uri, range=sym.range)) + + return results + except Exception as e: + logger.error(f"Error finding references: {e}") + return [] + + +@server.feature(lsp.TEXT_DOCUMENT_DEFINITION) +def definition(params: lsp.DefinitionParams) -> list[lsp.Location]: + """Go to definition of the symbol at the given position.""" + try: + doc = server.workspace.get_text_document(params.text_document.uri) + symbol_name = _find_symbol_at_position( + doc.source, + params.position.line, + params.position.character, + ) + if not symbol_name: + return [] + + results: list[lsp.Location] = [] + for uri, _path, source in _get_all_mrc_files(): + for sym in parse_symbols(source): + if sym.name == symbol_name: + results.append(lsp.Location(uri=uri, range=sym.selection_range)) + return results + except Exception as e: + logger.error(f"Error finding definition: {e}") + return [] + + +@server.feature(lsp.WORKSPACE_SYMBOL) +def workspace_symbol(params: lsp.WorkspaceSymbolParams) -> list[lsp.SymbolInformation]: + """Search for symbols across the workspace.""" + query = params.query.lower() + results = [] + for uri, _path, source in _get_all_mrc_files(): + for sym in parse_symbols(source): + if query in sym.name.lower(): + results.append( + lsp.SymbolInformation( + name=sym.name, + kind=sym.kind, + location=lsp.Location(uri=uri, range=sym.range), + container_name=sym.detail, + ) + ) + return results + + +if __name__ == "__main__": + server.start_io() diff --git a/src/solidlsp/ls_config.py b/src/solidlsp/ls_config.py index ab1a4038..ff137894 100644 --- a/src/solidlsp/ls_config.py +++ b/src/solidlsp/ls_config.py @@ -88,6 +88,12 @@ class Language(str, Enum): Requires MATLAB R2021b or later and Node.js. Set MATLAB_PATH environment variable or configure matlab_path in ls_specific_settings. """ + MSL = "msl" + """mIRC Scripting Language (mSL) language server. + Supports .mrc files used in mIRC and AdiIRC IRC clients. + Uses a custom LSP server based on pygls. Automatically sets up + a virtual environment with pygls dependencies on first use. + """ # Experimental or deprecated Language Servers TYPESCRIPT_VTS = "typescript_vts" """Use the typescript language server through the natively bundled vscode extension via https://github.com/yioneko/vtsls""" @@ -321,6 +327,8 @@ class Language(str, Enum): return FilenameMatcher("*.sol") case self.ANSIBLE: return FilenameMatcher("*.yaml", "*.yml") + case self.MSL: + return FilenameMatcher("*.mrc") case _: raise ValueError(f"Unhandled language: {self}") @@ -544,6 +552,10 @@ class Language(str, Enum): from solidlsp.language_servers.ansible_language_server import AnsibleLanguageServer return AnsibleLanguageServer + case self.MSL: + from solidlsp.language_servers.msl_language_server import MslLanguageServer + + return MslLanguageServer case _: raise ValueError(f"Unhandled language: {self}") diff --git a/test/conftest.py b/test/conftest.py index ce5761c6..4d3f1154 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -259,6 +259,7 @@ _LANGUAGE_PYTEST_MARKERS: dict[Language, list[MarkDecorator | Mark]] = { Language.JAVA: [pytest.mark.java], Language.KOTLIN: [pytest.mark.kotlin, pytest.mark.skipif(is_ci, reason="Kotlin LSP JVM crashes on restart in CI")], Language.LEAN4: [pytest.mark.lean4, pytest.mark.skipif(_sh.which("lean") is None, reason="Lean is not installed")], + Language.MSL: [pytest.mark.msl], Language.PHP: [pytest.mark.php], Language.PHP_PHPACTOR: [pytest.mark.php], Language.POWERSHELL: [pytest.mark.powershell], diff --git a/test/resources/repos/msl/test_repo/main.mrc b/test/resources/repos/msl/test_repo/main.mrc new file mode 100644 index 00000000..04385ee1 --- /dev/null +++ b/test/resources/repos/msl/test_repo/main.mrc @@ -0,0 +1,38 @@ +; Main game module - tests alias, event, and menu parsing + +alias greet { + msg $chan Hello, $1 $+ ! Welcome aboard. +} + +alias -l calculate.doubloons { + var %base = 100 + var %bonus = $rand(1,50) + return $calc(%base + %bonus) +} + +on *:TEXT:!hello*:#: { + greet $nick +} + +on *:JOIN:#: { + .timer 1 3 greet $nick + var %coins = $format.coins(100) + msg $chan You have %coins +} + +raw 319:*: { + ; Handle channel list numeric + echo -a Channels: $3- +} + +menu channel { + Pirates Game + .Start Game:/start + .End Game:/stop +} + +alias show.player.info { + if ($is.admin($nick)) { + msg $chan $nick is an admin with $format.coins(500) + } +} diff --git a/test/resources/repos/msl/test_repo/utils.mrc b/test/resources/repos/msl/test_repo/utils.mrc new file mode 100644 index 00000000..c7a12266 --- /dev/null +++ b/test/resources/repos/msl/test_repo/utils.mrc @@ -0,0 +1,26 @@ +; Utility aliases and handlers + +alias format.coins { + return $bytes($1,b) doubloons +} + +alias is.admin { + if ($1 isop $chan) return $true + return $false +} + +alias welcome.message { + greet $1 + msg $chan You have $format.coins(100) to start with! +} + +dialog settings { + title "Game Settings" + size -1 -1 200 150 + edit "", 1, 10 10 180 20 + button "Save", 2, 60 40 80 25 +} + +ctcp *:VERSION:*: { + ctcpreply $nick VERSION PiratesIRC v1.0 +} diff --git a/test/serena/test_serena_agent.py b/test/serena/test_serena_agent.py index 01cd29d1..be078eae 100644 --- a/test/serena/test_serena_agent.py +++ b/test/serena/test_serena_agent.py @@ -47,6 +47,7 @@ def serena_config(): Language.CPP_CCLS, Language.HAXE, Language.LEAN4, + Language.MSL, ]: repo_path = get_repo_path(language) if repo_path.exists(): @@ -199,6 +200,7 @@ class TestSerenaAgent: pytest.param(Language.CPP_CCLS, "add", "Function", "b.cpp", marks=pytest.mark.cpp), pytest.param(Language.HAXE, "Main", "Class", "Main.hx", marks=pytest.mark.haxe), pytest.param(Language.LEAN4, "add", "Method", "Helper.lean", marks=pytest.mark.lean4), + pytest.param(Language.MSL, "greet", "Function", "main.mrc", marks=pytest.mark.msl), ], indirect=["serena_agent"], ) @@ -307,6 +309,7 @@ class TestSerenaAgent: marks=pytest.mark.haxe, ), pytest.param(Language.LEAN4, "add", "Helper.lean", "Main.lean", marks=pytest.mark.lean4), + pytest.param(Language.MSL, "format.coins", "utils.mrc", "main.mrc", marks=pytest.mark.msl), ], indirect=["serena_agent"], ) diff --git a/test/solidlsp/msl/__init__.py b/test/solidlsp/msl/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/test/solidlsp/msl/test_msl_basic.py b/test/solidlsp/msl/test_msl_basic.py new file mode 100644 index 00000000..92a4cd41 --- /dev/null +++ b/test/solidlsp/msl/test_msl_basic.py @@ -0,0 +1,161 @@ +""" +Basic integration tests for the mSL (mIRC Scripting Language) language server. + +Tests validate document symbols, references, and definitions for aliases, events, +raw events, menus, dialogs, and CTCP handlers using the mSL test repository. +""" + +import os + +import pytest + +from solidlsp import SolidLanguageServer +from solidlsp.ls_config import Language +from test.solidlsp.conftest import format_symbol_for_assert, has_malformed_name, request_all_symbols + +pytestmark = [pytest.mark.msl] + + +class TestMslDocumentSymbols: + """Test document symbol retrieval for mSL constructs.""" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_ls_is_running(self, language_server: SolidLanguageServer) -> None: + """Test that the language server starts successfully.""" + assert language_server.is_running() + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_document_symbols_main(self, language_server: SolidLanguageServer) -> None: + """Test that document symbols are returned for the main file.""" + doc_symbols = language_server.request_document_symbols("main.mrc") + all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots() + + symbol_names = [s.get("name") for s in all_symbols if s.get("name")] + assert "greet" in symbol_names, f"greet alias not found. Found: {symbol_names}" + assert "calculate.doubloons" in symbol_names, f"calculate.doubloons alias not found. Found: {symbol_names}" + assert "show.player.info" in symbol_names, f"show.player.info alias not found. Found: {symbol_names}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_document_symbols_events(self, language_server: SolidLanguageServer) -> None: + """Test that event handlers, raw events, and menus are detected in the main file.""" + doc_symbols = language_server.request_document_symbols("main.mrc") + all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots() + + symbol_names = [s.get("name") for s in all_symbols if s.get("name")] + # Check for on *:TEXT and on *:JOIN events + on_events = [n for n in symbol_names if n.startswith("on ")] + assert len(on_events) >= 2, f"Expected at least 2 event handlers. Found: {on_events}" + # Check for raw event + raw_events = [n for n in symbol_names if n.startswith("raw ")] + assert len(raw_events) >= 1, f"Expected at least 1 raw event handler. Found: {raw_events}" + # Check for menu + menus = [n for n in symbol_names if n.startswith("menu ")] + assert len(menus) >= 1, f"Expected at least 1 menu. Found: {menus}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_document_symbols_utils(self, language_server: SolidLanguageServer) -> None: + """Test that document symbols are returned for the utils file.""" + doc_symbols = language_server.request_document_symbols("utils.mrc") + all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots() + + symbol_names = [s.get("name") for s in all_symbols if s.get("name")] + assert "format.coins" in symbol_names, f"format.coins alias not found. Found: {symbol_names}" + assert "is.admin" in symbol_names, f"is.admin alias not found. Found: {symbol_names}" + assert "welcome.message" in symbol_names, f"welcome.message alias not found. Found: {symbol_names}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_document_symbols_dialog_and_ctcp(self, language_server: SolidLanguageServer) -> None: + """Test that dialog and CTCP handler definitions are detected.""" + doc_symbols = language_server.request_document_symbols("utils.mrc") + all_symbols, root_symbols = doc_symbols.get_all_symbols_and_roots() + + symbol_names = [s.get("name") for s in all_symbols if s.get("name")] + assert "dialog settings" in symbol_names, f"dialog settings not found. Found: {symbol_names}" + # Check for CTCP handler + ctcp_events = [n for n in symbol_names if n.startswith("ctcp ")] + assert len(ctcp_events) >= 1, f"Expected at least 1 ctcp handler. Found: {ctcp_events}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_find_symbol(self, language_server: SolidLanguageServer) -> None: + """Test that the full symbol tree contains expected symbols from both files.""" + from solidlsp.ls_utils import SymbolUtils + + symbols = language_server.request_full_symbol_tree() + assert SymbolUtils.symbol_tree_contains_name(symbols, "greet"), "greet not found in symbol tree" + assert SymbolUtils.symbol_tree_contains_name(symbols, "format.coins"), "format.coins not found in symbol tree" + assert SymbolUtils.symbol_tree_contains_name(symbols, "show.player.info"), "show.player.info not found in symbol tree" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_bare_symbol_names(self, language_server: SolidLanguageServer) -> None: + """Test that symbol names do not contain unexpected formatting characters.""" + all_symbols = request_all_symbols(language_server) + malformed_symbols = [] + for s in all_symbols: + # mSL symbols can contain periods (e.g., calculate.doubloons) and + # colons/spaces in event names (e.g., "on *:TEXT"), so allow those + if has_malformed_name(s, period_allowed=True, colon_allowed=True, whitespace_allowed=True): + malformed_symbols.append(s) + if malformed_symbols: + pytest.fail( + f"Found malformed symbols: {[format_symbol_for_assert(sym) for sym in malformed_symbols]}", + pytrace=False, + ) + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_find_references_within_file(self, language_server: SolidLanguageServer) -> None: + """Test that references to 'greet' are found within main.mrc.""" + file_path = "main.mrc" + all_symbols, _ = language_server.request_document_symbols(file_path).get_all_symbols_and_roots() + greet_symbol = next((s for s in all_symbols if s.get("name") == "greet"), None) + assert greet_symbol is not None, "Could not find 'greet' symbol in main.mrc" + + sel_start = greet_symbol["selectionRange"]["start"] + refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"]) + + assert refs, f"Expected non-empty references for greet but got {refs=}" + + actual_locations = [ + { + "uri_suffix": os.path.basename(ref.get("relativePath", ref.get("uri", ""))), + "line": ref["range"]["start"]["line"], + } + for ref in refs + ] + + # greet is called on line 13 (0-indexed) in main.mrc: `greet $nick` + call_site = {"uri_suffix": "main.mrc", "line": 13} + assert call_site in actual_locations, f"Expected reference to greet at line 13 in main.mrc, got {actual_locations}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_find_references_across_files(self, language_server: SolidLanguageServer) -> None: + """Test that references to 'format.coins' are found across main.mrc and utils.mrc.""" + # format.coins is defined in utils.mrc but called in both main.mrc and utils.mrc + file_path = "utils.mrc" + all_symbols, _ = language_server.request_document_symbols(file_path).get_all_symbols_and_roots() + fc_symbol = next((s for s in all_symbols if s.get("name") == "format.coins"), None) + assert fc_symbol is not None, "Could not find 'format.coins' symbol in utils.mrc" + + sel_start = fc_symbol["selectionRange"]["start"] + refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"]) + + assert refs, f"Expected non-empty references for format.coins but got {refs=}" + + actual_locations = [ + { + "uri_suffix": os.path.basename(ref.get("relativePath", ref.get("uri", ""))), + "line": ref["range"]["start"]["line"], + } + for ref in refs + ] + + # Verify cross-file: at least one reference is in main.mrc (different file from definition) + main_refs = [loc for loc in actual_locations if loc["uri_suffix"] == "main.mrc"] + assert len(main_refs) >= 1, f"Expected at least 1 reference in main.mrc, got {main_refs}" + + @pytest.mark.parametrize("language_server", [Language.MSL], indirect=True) + def test_workspace_symbol(self, language_server: SolidLanguageServer) -> None: + """Test that workspace symbol search returns results.""" + result = language_server.request_workspace_symbol("greet") + assert result is not None, "Workspace symbol search returned None" + assert len(result) > 0, "Workspace symbol search returned no results" + assert any("greet" in str(s.get("name", "")) for s in result), f"Expected at least one result containing 'greet', got {result}" diff --git a/uv.lock b/uv.lock index 17190a91..c89fecb4 100644 --- a/uv.lock +++ b/uv.lock @@ -243,6 +243,19 @@ filecache = [ { name = "filelock" }, ] +[[package]] +name = "cattrs" +version = "26.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "attrs" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a0/ec/ba18945e7d6e55a58364d9fb2e46049c1c2998b3d805f19b703f14e81057/cattrs-26.1.0.tar.gz", hash = "sha256:fa239e0f0ec0715ba34852ce813986dfed1e12117e209b816ab87401271cdd40", size = 495672, upload-time = "2026-02-18T22:15:19.406Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/80/56/60547f7801b97c67e97491dc3d9ade9fbccbd0325058fd3dfcb2f5d98d90/cattrs-26.1.0-py3-none-any.whl", hash = "sha256:d1e0804c42639494d469d08d4f26d6b9de9b8ab26b446db7b5f8c2e97f7c3096", size = 73054, upload-time = "2026-02-18T22:15:17.958Z" }, +] + [[package]] name = "certifi" version = "2026.2.25" @@ -1291,6 +1304,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b4/de/88b3be5c31b22333b3ca2f6ff1de4e863d8fe45aaea7485f591970ec1d3e/linkify_it_py-2.1.0-py3-none-any.whl", hash = "sha256:0d252c1594ecba2ecedc444053db5d3a9b7ec1b0dd929c8f1d74dce89f86c05e", size = 19878, upload-time = "2026-03-01T07:48:46.098Z" }, ] +[[package]] +name = "lsprotocol" +version = "2025.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "attrs" }, + { name = "cattrs" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e9/26/67b84e6ec1402f0e6764ef3d2a0aaf9a79522cc1d37738f4e5bb0b21521a/lsprotocol-2025.0.0.tar.gz", hash = "sha256:e879da2b9301e82cfc3e60d805630487ac2f7ab17492f4f5ba5aaba94fe56c29", size = 74896, upload-time = "2025-06-17T21:30:18.156Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7b/f0/92f2d609d6642b5f30cb50a885d2bf1483301c69d5786286500d15651ef2/lsprotocol-2025.0.0-py3-none-any.whl", hash = "sha256:f9d78f25221f2a60eaa4a96d3b4ffae011b107537facee61d3da3313880995c7", size = 76250, upload-time = "2025-06-17T21:30:19.455Z" }, +] + [[package]] name = "markdown-it-py" version = "3.0.0" @@ -2143,6 +2169,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e7/d3/c622950d87a2ffd1654208733b5bd1c5645930014abed8f4c0d74863988b/pydata_sphinx_theme-0.15.4-py3-none-any.whl", hash = "sha256:2136ad0e9500d0949f96167e63f3e298620040aea8f9c74621959eda5d4cf8e6", size = 4640157, upload-time = "2024-06-25T19:28:42.383Z" }, ] +[[package]] +name = "pygls" +version = "2.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "attrs" }, + { name = "cattrs" }, + { name = "lsprotocol" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/da/2e/7bbe061d175c0baddde8fc9edb908a4c31ba5d9165b8c68e3439c3a9f138/pygls-2.1.1.tar.gz", hash = "sha256:1da03ba9053201bb337dcdd8d121df70feb2a91e1a0dcc74de5da79755b1a201", size = 55091, upload-time = "2026-03-25T11:19:10.541Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fd/1a/208293b6c350f5abea6941d5606080d4a492644052504f5312e5de30a902/pygls-2.1.1-py3-none-any.whl", hash = "sha256:510a6dea2476177230c7d851125e5948efdf3fdb9ebfd8543fc434972f8faed4", size = 68975, upload-time = "2026-03-25T11:19:11.374Z" }, +] + [[package]] name = "pygments" version = "2.19.2" @@ -2924,12 +2964,14 @@ dependencies = [ { name = "fortls" }, { name = "jinja2" }, { name = "joblib" }, + { name = "lsprotocol" }, { name = "mcp" }, { name = "oraios-pywebview" }, { name = "overrides" }, { name = "pathspec" }, { name = "psutil" }, { name = "pydantic" }, + { name = "pygls" }, { name = "pyright" }, { name = "pystray" }, { name = "python-dotenv" }, @@ -2998,6 +3040,7 @@ requires-dist = [ { name = "jinja2", marker = "extra == 'dev'", specifier = "==3.1.6" }, { name = "joblib", specifier = "==1.5.1" }, { name = "jupyter-book", marker = "extra == 'dev'", specifier = "==1.0.4.post1" }, + { name = "lsprotocol", specifier = "==2025.0.0" }, { name = "mcp", specifier = "==1.26.0" }, { name = "mypy", marker = "extra == 'dev'", specifier = "==1.17.0" }, { name = "nbconvert", marker = "extra == 'dev'", specifier = "==7.17.0" }, @@ -3009,6 +3052,7 @@ requires-dist = [ { name = "psutil", specifier = "==7.0.0" }, { name = "pyasn1", marker = "extra == 'dev'", specifier = "==0.6.3" }, { name = "pydantic", specifier = "==2.12.5" }, + { name = "pygls", specifier = "==2.1.1" }, { name = "pyinstrument", marker = "extra == 'dev'", specifier = "==5.1.1" }, { name = "pyjwt", marker = "extra == 'dev'", specifier = "==2.12.0" }, { name = "pyright", specifier = "==1.1.403" },