Commit Graph
3216 Commits
Author SHA1 Message Date
Ivan 29d07d4f6b fix(erlang): address functions by name/arity without the name path separator
Erlang LS identifies functions, types and parameterised macros as
`name/arity`, but `/` separates the components of a Serena name path, so
`create_user/4` was parsed as "symbol `4` nested inside `create_user`" and
could never match -- not even via the name path Serena itself reported for
the symbol. Browsing still worked, but find_referencing_symbols,
replace_symbol_body and insert_after_symbol were unusable on Erlang
functions.

Normalize the name to `create_user#4` instead. `#` cannot occur in an
unquoted Erlang atom, so it can never collide with a real name, unlike `@`.
The arity is kept rather than stripped because it is part of a function's
identity in Erlang: create_order/3 and create_order/2 are different
functions and may coexist in one module.

Also document the "no `/` in symbol names" rule on _normalize_symbol_name,
which is what the next backend for such a language needs to know.
2026-08-03 01:37:36 +02:00
Dominik Jain 9fc8033ece Introduce DownloadedDependency abstraction with URL-keyed hash database
* Add DownloadedDependency abstraction to facilitate handling of downloaded
  and hash-verified dependencies.
  In CI, missing hashes raise an exception.
  For users, unverified downloads log a warning.
* Add DownloadedDependencyHashDatabase which manages a URL-keyed database
  of sha256 hashes. It can be conveniently updated incrementally through a
  script (src/solidlsp/resources/downloaded_dependency_hashes.json),
  provided that language server implementations provide DownloadedDependency
  instances statically

Applied in EclipseJDTLS:
* Replace the untyped nested-dict runtime dependency descriptors with DownloadedDependency instances
* Move the JDTLS version-pinning constants into DependencyProvider
* Replace the platform dependency dicts with the VSCodeJavaConfig dataclass

Utils:
 * Add FileUtils.ArchiveType to type archive types that were previously plain str
 * Add is_running_in_ci() to ls_utils

Updated memory on adding a language server accordingly.
2026-08-03 01:34:24 +02:00
Tyce Herrman c7dbe3bd11 ci: use Node 24 for Node-backed test batches 2026-08-01 16:14:38 +02:00
arimu1 f4ecb1a5bd fix(ls): re-raise LS termination from pull-diagnostics fallback
request_text_document_diagnostics swallowed all SolidLSPException from
textDocument/diagnostic and fell back to published diagnostics. When the
process had already died, that bypassed tools_base restart-and-retry and
silently returned empty diagnostics.

Re-raise when is_language_server_terminated(); keep the published-
diagnostics fallback for other pull-diagnostics failures.

Fixes #1770
2026-08-01 15:58:07 +02:00
Dominik Jain c958aa0235 Guard actions in __del__ (obj must be initialised) 2026-08-01 15:57:03 +02:00
Ivan c74373e29a Implement _create_base_initialize_params for ErlangLanguageServer
ErlangLanguageServer was left behind when the initialize-params refactor
moved server-specific capabilities into _create_base_initialize_params.
It still built its initialize dict inline in _start_server, so the class
never satisfied the abstract method declared on SolidLanguageServer and
could not be instantiated at all:

  Can't instantiate abstract class ErlangLanguageServer without an
  implementation for abstract method '_create_base_initialize_params'

Move the capabilities block into the required method, following the
pattern used by the other backends, and let _start_server build its
params via _create_initialize_params(). processId/rootPath/rootUri are
dropped from the literal since the builder supplies them.
2026-08-01 11:44:32 +02:00
Dominik Jain 97bc0980a2 Further improve docstring of find symbol tools 2026-07-31 16:06:02 +02:00
Dominik Jain a47df023f5 Report language server status in GetCurrentConfigTool result
Resolves #1782
2026-07-31 16:06:02 +02:00
Dr. Dominik Jain dd12fa032f Merge pull request #1779 from TyceHerrman/agent/rust-analyzer-lightweight
Rust: reduce rust-analyzer memory churn
2026-07-30 10:43:59 +02:00
Dominik Jain 7155dc596b Remove debug code
Fixes #1783
2026-07-30 10:15:31 +02:00
Dominik Jain 1f133f62cf Re-enable cargo.autoreload
autoreload=false was likely to leave project metadata stale after manifest or workspace changes #1779
2026-07-30 00:27:09 +02:00
Tyce Herrman 3ede3cbc33 Docs: use SessionEnd for Codex hook cleanup (#1733)
* Docs: use SessionEnd for Codex hook cleanup

* Docs: note legacy Codex Stop issue
2026-07-29 23:36:41 +02:00
Amir Fathi 19046bba66 fix(fsharp): correct selectionRange for module declarations (#925)
FsAutoComplete reports selectionRange for a `module <Name>` declaration
pointing at the `module` keyword itself, not at `<Name>`. Since
LanguageServerSymbol.line/.column (src/serena/symbol.py) build hover and
reference lookups from selectionRange.start, hovering a module returns
the LSP server's docs for the generic keyword instead of the module's
own, as reported in #925 by the F# team and confirmed by dsyme
(F# tooling) and MischaPanch in the issue thread.

This mirrors the fix already shipped for fortls in
fortran_language_server.py: a request_document_symbols override that
recovers the real identifier column by matching the declaration line
with a regex, scoped to Module symbols only since other kinds (functions,
types, open) already select correctly.

Verified with a synthetic-payload unit test (module declaration,
nested-with-equals, recursive module, an already-correct range left
untouched, and a non-Module symbol left untouched), since F# language
server tests are unconditionally disabled in this repo
(test/conftest.py: "F# language server is currently unreliable"), so no
live-FsAutoComplete regression test can run here or on CI. Confirmed the
new test fails on main (AttributeError / unfixed selectionRange) and
passes on this branch.

Fixes #925
2026-07-29 17:47:25 +02:00
Dominik Jain e6e25d3f86 Report project activation errors to the client in the system prompt
SerenaAgent now tracks an initial/startup project activation error and appends it
to the system prompt, instead of the failure being visible only in the log.
This applies both to a failed activation of an explicitly given project and to a
failed --project-from-cwd auto-detection.

Also fix the --project-from-cwd help text, which falsely promised a fallback to the CWD
Resolves #1773
2026-07-29 17:39:06 +02:00
Tyce Herrman 580a7b479e Rust: stabilize lightweight diagnostics 2026-07-29 09:45:49 -04:00
Michael Panchenko c839f77e26 Bump timeout
(cherry picked from commit 8d0608939f1e4719cd9e12cbaab341af723d4a71)
2026-07-29 09:45:49 -04:00
Michael Panchenko 23957a55e4 Bump timeout for rust diagnostics
(cherry picked from commit c42dcf29a9849e1fe935e5039aebd24e7b1f6cac)
2026-07-29 09:45:49 -04:00
davidcforbes 3be211c662 rust-analyzer: lightweight profile (disable cachePriming + cargo.autoreload)
serena's symbol tools (find_symbol, find_referencing_symbols, …) only need the
symbol index, but the LSP is launched with VS Code's full heavyweight defaults.
On a large, actively-built Rust workspace these drove rust-analyzer to 40+ GB
RSS and triggered a full re-index on every external `cargo build` (target/
churn). Disabling the two eager/auto re-indexing drivers keeps symbol
navigation working at a fraction of the memory:

- cachePriming.enable: True -> False  (no eager whole-workspace+deps index)
- cargo.autoreload:    True -> False  (no re-index storm on target/ changes)

checkOnSave is intentionally LEFT enabled: serena's get_diagnostics_for_file
relies on flycheck (`cargo check`) for Rust diagnostics, and disabling it makes
that tool return empty (test/solidlsp/rust/test_rust_diagnostics.py fails with
[]). checkOnSave only spawns transient `cargo check` child processes and is not
a driver of rust-analyzer's resident RSS, so keeping it on preserves
diagnostics at no meaningful memory cost.

Observed on a rustback (multi-crate + rustic_core fork) workspace: repeated
40-44 GB rust-analyzer processes under serena with no VS Code present. Suggest
upstream expose a "lightweight"/symbol-only RA profile or a config toggle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit e09b9b623e321f72e64b6ba3de6708ff519f524f)
2026-07-29 09:45:48 -04:00
Dominik Jain a61765ecc8 Fix: Serena PyPI version check triggered by callback on main thread could delay agent startup
Closes #1774
2026-07-29 15:24:22 +02:00
Dominik Jain 21f4b3fe58 Add section on trusted projects 2026-07-29 13:53:03 +02:00
Dominik Jain 466960b12f Remove unmaintainable parts of security documentation 2026-07-29 13:13:06 +02:00
Kyle HutchinsonandClaude Sonnet 5 32308a871c Fix: get_diagnostics_for_file crashed with SolidLSPException for any Ansible file with at least one finding
`ansible-language-server` doesn't implement `textDocument/documentSymbol`
and the request used to map diagnostics onto owning symbols just threw. `AnsibleLanguageServer` now
overrides that request to return `None` directly, so diagnostics fall back to being grouped under
the file-level path #1758

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-29 01:47:27 +02:00
Dominik Jain 6c1c965370 find_symbol: Change tool description to improve tool search results
in clients that load tools dynamically
2026-07-28 17:50:22 +02:00
Dominik Jain 9a9d07e83d Improve system prompt (OOP principles, etc.) 2026-07-27 13:41:00 +02:00
Bortlesboat 026b007bb8 fix(hooks): accept raw control characters in payloads 2026-07-26 18:47:18 +02:00
xormania 31aef83786 Docs: correct stale references in the language-addition guide
Four items in .serena/memories/adding_new_language_support_guide.md are
wrong against the current source. Two are references that no longer
resolve:

- the subprocess helper lives at solidlsp.util.subprocess_util and is a
  module-level function, not a method on a class; the guide named
  solidlsp.utils.subprocess_utils, which does not exist
- pytest markers are declared under [tool.pytest.ini_options].markers in
  pyproject.toml; there is no pytest.ini in the repository

and two are formatting defects:

- the Documentation section used "### 5" while every other step uses
  "## Step N:", so it nested under Step 4 in rendered output
- a stray double colon in the "Implementation Pointers" label

Promoting Documentation to a top-level step also completes the Overview
list, which stopped at four.

Step 5 gains src/serena/resources/project.template.yml, which the
checklist omitted even though a new language server has to appear in its
commented list. The instruction uses `uv run` because the script imports
solidlsp and fails under an ambient interpreter, and it notes the
trailing spaces the script pads each line with, which the committed list
does not carry.
2026-07-26 15:09:54 +02:00
xormania 7d08146436 Docs: add the language servers missing from the supported-language lists
Several backends that are registered and tested were absent from the
user-facing lists: six from the README (pascal, qml, rego, systemverilog,
terraform, vue), six from the Language Support docs page (matlab,
powershell, rego, systemverilog, terraform, toml), and two from the
commented language-server list in the project template
(python_basedpyright, qml).

The drift is long-standing and runs in both directions: matlab,
powershell and toml reached the README but never the docs page; pascal,
vue and qml reached the docs page but never the README; terraform, rego
and systemverilog reached neither, the oldest of them since 2025-07-06.
The two template omissions are the most recent, from #1705 and #1635.
The guide's Documentation step does ask for the README and the docs page,
so most of this was missed rather than unspecified; it does not mention
the project template at all.

The setup notes on the new docs-page entries name the versions Serena
actually pins, taken from the corresponding language-server modules:
MATLAB extension 1.3.9, PowerShell Editor Services 4.4.0 and
PSScriptAnalyzer 1.25.0, Verible v0.0-4051-g9fdb4057, terraform-ls
0.36.5, and Taplo 0.10.0. The PowerShell entry mentions both automatic
installations, since enabling that backend also runs Save-Module against
the user's configured PowerShell repository.

The new Regal link points at open-policy-agent/regal, the current
canonical repository; StyraInc/regal still redirects there, and the
adapter and the CI workflow have not been changed.

The template block is refreshed from scripts/print_language_list.py,
which is why its column widths change: python_basedpyright is longer than
any identifier the previous list contained. The pointer directly below it
still named the Language enum, renamed to LanguageServerId in d2cf18dd.
2026-07-26 11:40:23 +02:00
Dr. Dominik Jain 66b34dc8eb Merge pull request #1750 from AmirF194/fix/1490-kls-pdeathsig
fix(ls): don't orphan the language server process when Serena is killed
2026-07-26 11:38:33 +02:00
Dominik Jain aeb73c789d Move changelog entries to their proper place, rewording 2026-07-26 11:34:42 +02:00
Amir Fathi 06b47d064a Merge branch 'main' into fix/1490-kls-pdeathsig 2026-07-26 02:24:58 +00:00
Youcef Hanaia 34342a9d00 Chore: Replace subprocess.run with subprocess_run util function (#1754) 2026-07-25 20:56:08 +02:00
Dominik Jain 33b042ad06 Add script for visualising the memory graph of a project 2026-07-24 22:30:05 +02:00
Dominik Jain 68884f1190 Add callout on security advisories 2026-07-24 19:07:43 +02:00
Dominik Jain cbc7f286b3 Refactoring: Fully internalise the pdeathsig machinery into LanguageServerSubprocessLauncher 2026-07-24 18:19:03 +02:00
Amir Fathi ce685c8979 fix(ls): reify the pdeathsig launch ritual into LanguageServerSubprocessLauncher
The previous fix for #1490 depended on four fragments being combined
correctly by hand at the _start() call site: convert_shell_cmd's exec
prefix, the preexec_fn registration, routing through
popen_preserving_pdeathsig, and the Linux/own-session condition.
Nothing enforced their co-occurrence.

Reify "launch a subprocess whose lifecycle is tied to the parent
process (on Linux)" as LanguageServerSubprocessLauncher, a lazily
constructed singleton with a single public method:

    launch(process_launch_info: ProcessLaunchInfo, start_new_session: bool) -> Popen[bytes]

Internalizes the libc/prctl preexec_fn, the exec prefixing, the
platform/own-session gate, and the dedicated long-lived spawner
thread (created lazily, only for own-session launches on Linux). The
singleton is required because the spawner thread it owns must be the
one thread in the process performing pdeathsig-registering fork()
calls; launch() takes start_new_session per call so the instance
itself carries no configuration state.

convert_shell_cmd loses the use_exec flag it gained for this fix: its
other callers (solidlsp.language_servers.common, serena.agent) never
used it, and it is invalid on Windows. The exec prefix now lives
entirely inside the launcher, applied to convert_shell_cmd's plain
output, so it can never be requested without the machinery it depends
on.

StdioLanguageServer._start() now calls
LanguageServerSubprocessLauncher.instance().launch(...) instead of
assembling the four fragments itself.

Adapted test/solidlsp/test_pdeathsig.py: the two existing end-to-end
regression tests (short-lived calling thread, SIGKILLed driver) are
unchanged in behavior since they exercise _start()/start() end-to-end
and now transparently go through the new launcher. Added three
targeted tests: convert_shell_cmd's reverted signature, the
launcher's singleton identity, and that its spawner thread is not
created for a non-own-session launch.
2026-07-24 14:05:26 +00:00
Dominik Jain 140ffaf3ff Fix: Tool calls exceeding the timeout blocked the task executor indefinitely;
tasks are now cancelled on timeout and the executor recovers

Previously, Tool.apply_ex issued its task without a timeout, so a hung tool
would block the TaskExecutor's dispatcher thread forever: the caller received
a TimeoutError after tool_timeout, but no further tasks were ever processed.

Changes:
- Tool.apply_ex now passes the configured tool timeout to issue_task, bounding
  the dispatcher's wait as well as the caller's.
- TaskExecutor.Task.result gained cancel_on_timeout (default True): on timeout,
  the task is cancelled, preventing execution of tasks still waiting in the
  queue and discarding the result of tasks already running.
- Task.wait_until_done no longer takes a timeout parameter; it uses the task's
  own timeout and returns whether the task completed. The dispatcher logs a
  warning and continues with the next task if the timeout was reached.
- TaskExecutor.execute_task now propagates its timeout to result() instead of
  waiting indefinitely, thus also benefiting from cancel-on-timeout.
- Corrected the docstring of Task.result to match the actual semantics.
2026-07-24 13:23:06 +02:00
Amir Fathi 81d19bdd27 fix(ls): keep the PDEATHSIG registration alive across the calling thread's own exit
Tool calls and language server startup dispatch Popen() onto a short-lived
thread (e.g. LanguageServerManager's per-server StartLSThread), which
terminates moments after Popen() returns. PR_SET_PDEATHSIG ties its
registration to that specific calling thread, not the process, so the
language server received SIGTERM and died within milliseconds of a normal
startup on Linux CI, independent of any real SIGKILL scenario.

Route Popen() calls that set preexec_fn=set_pdeathsig_on_parent_exit through
one dedicated, permanently-running daemon thread instead, so the registration
stays tied to a thread that only ever dies alongside the whole process. Added
a regression test that starts a language server on a short-lived thread and
checks it survives past that thread's exit.
2026-07-24 01:00:41 +00:00
Tyce Herrman d86fd6c0c8 Support nixd ls_path and config_path 2026-07-24 02:22:23 +02:00
Amir Fathi 61ca9ba5de TypeScript: make the tsserver progress-start grace configurable, raise the default
find_referencing_symbols/request_references could return incomplete results on
large TypeScript projects. _wait_for_indexing_start_or_completion waits only
_INDEXING_START_GRACE_S (hardcoded 2.0s) for tsserver to *start* emitting
$/progress after opening files; if nothing has appeared by then, it assumes no
indexing is needed and proceeds immediately. On a large project, tsserver can
take longer than that just to resolve the project graph before it even creates
the first progress token, so the first cross-file reference query can race a
project that is still loading.

Every caller of _wait_for_indexing_start_or_completion (the base TypeScript
path and the Svelte companion) always calls expect_indexing() first, so the
"no progress observed" branch is reached specifically when indexing is already
expected; it cannot distinguish "nothing to do" from "not started yet",
because the two look identical from the client for as long as tsserver stays
quiet. That ambiguity cannot be resolved from client-observable signals alone,
so this makes the grace an ls_specific_settings knob (`indexing_start_grace`),
the same way indexing_timeout and server_ready_timeout already are, and raises
the default from 2.0s to 5.0s.

Fixes #1586
2026-07-24 00:27:36 +02:00
Amir Fathi 3ac66bb8c1 fix(ls): don't orphan the language server process when Serena is killed
On Linux, a SIGKILLed Serena (or an OOM kill, or a crash) leaves its language server process
running: it is launched in its own session so our graceful-shutdown code can clean up the whole
tree, but that code never runs if Serena itself dies uncleanly, and no signal from our process
group reaches a detached session either.

Fix via prctl(PR_SET_PDEATHSIG) at spawn time. Since the process runs under shell=True, the
registration alone only protects the intermediate shell; prefixing the command with `exec` makes
the shell replace itself with the language server so the same PID (and PDEATHSIG registration)
carries through.

Fixes #1490
2026-07-23 22:12:44 +00:00
Dr. Dominik Jain 05468d9685 Merge pull request #1734 from arimu1/fix/1732-glob-match-single-asterisk
fix(util): ensure bare * and ? in non-** glob patterns do not match /
2026-07-23 23:32:54 +02:00
Dominik Jain 5b5f8332ce Add subprocess_run util function (to replace direct calls of subprocess.run) 2026-07-23 23:13:09 +02:00
Dominik Jain ad2598e9f3 Move helper functions into GlobMatcher 2026-07-23 22:48:27 +02:00
Dominik Jain 954d8554fd Make GlobMatcher with cached regex patterns the only glob matching mechanism 2026-07-23 22:48:27 +02:00
Dominik Jain 7033e90306 Unify glob matching to always use regex translation 2026-07-23 15:28:48 +02:00
Dominik Jain 33125287dc Remove obsolete glob pattern support in search_text, removing duplicate glob translation 2026-07-23 14:33:06 +02:00
Dominik Jain 5e8662b283 JetBrainsFindSymbolTool: Disallow wildcard-only search, delegating to overview tool if file
Resolves oraios/serena-jetbrains-plugin-public#8
2026-07-23 13:50:38 +02:00
arimu1 52cf8552a0 test(util): update exclude glob to data/*.log for slash-aware matching
Bare `*.log` no longer crosses `/` after #1732, so the exclude glob in
the search_files parametrization is anchored to data/ to keep excluding
data/b.log. Matches the repo convention that `**` is required to cross
directory boundaries (see test_search_files_glob_patterns).
2026-07-23 12:22:05 +07:00
arimu1 70d32f2453 fix(util): ensure bare * and ? in non-** glob patterns do not match / (#1732) 2026-07-23 12:22:05 +07:00
Tyce Herrman ac256f3630 Fix project CLI language server aliases 2026-07-23 01:35:39 +02:00