FileUtils.read_file has two return paths with different newline semantics.
The primary path, open(file_path, encoding=encoding).read(), uses the default
newline=None and therefore applies universal newlines, so CR never reaches
callers. The charset_normalizer fallback decodes the raw bytes instead
(match.raw.decode(match.encoding)), which performs no newline translation, so
any file that fails to decode with the project's configured encoding re-enters
Serena carrying its CR characters.
Normalize to LF at the fallback's return, matching the primary path.
read_file determined line indices via Python's splitlines, which breaks on
\f, \v, \x1c-\x1e, \x85, U+2028 and U+2029 in addition to the LSP-compliant
line breaks counted by delete_lines/replace_lines. In a file containing any
of those characters the indices reported by read_file referred to different
physical lines than the editing tools operate on, so an edit made after the
required read_file could silently hit the wrong line.
Use TextUtils.split_lines, as already applied to search_text and
MatchedConsecutiveLines.from_file_contents. Files whose line breaks are all
LSP-compliant are decomposed exactly as before.
delete_lines/replace_lines address the position one line past the last line
(line == number of lines, col 0). get_index_from_line_col cannot resolve that
position when the file has no trailing newline (no closing newline to count),
so delete_text_between_positions raised InvalidTextLocationError and the delete
crashed on a very common file shape.
Clamp that past-EOF end position to end-of-file in delete_text_between_positions,
mirroring the guard insert_text_at_position already has. A file with no trailing
newline now deletes its last line identically to one with a trailing newline.
Adds TextUtils unit tests and a CHANGELOG entry.
stepping line by line instead of character by character,
delegating the scan for line breaks to native string search
(~15-100x speedup for a full pass over typical source files)
* Add helper class TextStepper to facilitate computations
* Reimplement TextUtils methods based on TextStepper
* Adhere to LSP definition of newlines being defined by either "\n", "\r\n" or "\r" on its own.
Previously, only "\n" was considered.
* Add util method TextUtils.split_lines, allowing usages of Python's splitlines to be replaced
The cpu matrix job had no timeout-minutes, so a hung job runs until
GitHub's default 6-hour limit. This is not hypothetical: on run
29064865197 (2026-07-10), the native (ubuntu-latest) job hung and ran
from 02:36:34 to 08:36:50 before GitHub cancelled it at exactly the
6-hour cap, blocking the PR's status the whole time and burning a
runner for 6 hours.
Healthy runs are far below the new limit: across the 8 most recent
successful runs of this workflow, the slowest job was 16 minutes
(niche on ubuntu-latest), with catch-all on windows-latest typically
11-14 minutes. 60 minutes leaves ~4x headroom for cache-cold runs
(language-server downloads, npm installs) while cutting the worst-case
hang from 6 hours to 1.
Part of the timeout-hardening discussed in #1695.
When auto-generating a Serena project configuration at startup, compute the languages
in a background thread and update them later to avoid delays of the MCP server startup.
Where completeness of the language list matters (i.e. language server initialisation),
a corresponding event is awaited to ensure completeness.
Perl::LanguageServer only indexes files whose extension is in its
`perl.fileFilter` and skips directories in `perl.ignoreDirs`. Both were
hardcoded in PerlLanguageServer, so projects whose Perl source uses
non-standard extensions (e.g. .cgi / .psgi web handlers, which can
dominate a mature codebase) had no way to make those files visible to
symbol/reference queries without forking Serena (#1449).
Surface both lists through `ls_specific_settings["perl"]` (keys
`file_filter` and `ignore_dirs`), defaulting to the existing values so
behaviour is unchanged for current projects. The resolution lives in a
pure static method `_resolve_filter_settings` so the config plumbing is
covered by unit tests that don't require a Perl runtime.
Keep Serena's symbol index in sync with the LS (#1449, per review):
`find_symbol` and symbol indexing are driven by
`Language.PERL.get_source_fn_matcher()`, so adding an extension (e.g.
.cgi) to `file_filter` alone would leave the matcher untouched and the
extension's symbols invisible. Since that matcher is now a @cache'd
per-language singleton, add a `FilenameMatcher.add_extensions()` mutator
and call it from `PerlLanguageServer` so the configured extensions also
reach the symbol index, the ignore checks, and language composition.
Avoid cross-project leakage (per review): because the matcher is a
process-wide singleton, extensions added for one project would otherwise
persist into the next. `FilenameMatcher` now snapshots its initial
configuration and exposes `reset()`, and `SolidLanguageServer.__init__`
resets the matcher right after `self.language` is determined, so every
activation starts from the language's defaults before the subclass
re-applies its own `file_filter`.
This complements 860c5841 (which added .t to the default list as a
surface-level fix) by addressing the root configurability gap that the
maintainer acknowledged in #1449.
Hanging was never reproduced, this is preemptive hardening
- Add a 120s timeout to the server_ready wait (previously waited with no
timeout, so a server that crashed or never sent experimental/serverStatus
with quiescent=true would hang startup forever). On timeout we log a
warning and proceed, matching kotlin/matlab/powershell/etc.
- Harden the language/status and experimental/serverStatus notification
handlers to use dict.get() instead of raising KeyError on unexpected
payloads (a KeyError in the quiescent handler could itself prevent
server_ready from ever being set, feeding the hang).
- Relax the over-strict completionProvider capability check: an exact-dict
equality assertion crashed initialization whenever rust-analyzer changed
or added a completionProvider field. These values are not used downstream,
so a mismatch is now logged as a warning instead of asserted.
Language servers can report locations of generated files that are not on
disk. JDTLS does this for Lombok-generated types, reporting e.g.
target/classes/test_repo/LombokModel$LombokModelBuilder.class while
indexing; is_ignored_path then raised FileNotFoundError, turning an
intermittent indexing state into a hard test failure
(test_bare_symbol_names[java]; Tests run #4240, job jvm (macos-latest)).
Changes:
- is_ignored_path classifies missing paths with the usual lexical rules
instead of raising: ignored-dirname components, the unsupported-extension
rule, then the configured pathspec. A missing path is treated as a file
if its name has a suffix and as a directory otherwise (so the leaf of a
missing suffixless path is also checked against ignored dirnames).
Behavior for existing paths is unchanged: the checks were reordered
(ignored-dirname before unsupported-extension), but each check is pure,
so the outcomes are identical.
- SymbolLocationRequest.convert_location_item skips locations whose
absolute path does not exist (log + return None), so phantom build
artifacts never reach consumers regardless of ignore configuration.
- New unit tests (test double, no language server process; they run in
the catch-all CI batch) pin the missing-path classification — including
the exact Lombok .class path shape from the CI failure — and the
skipping of symbol locations that are not on disk.
Notes:
- The removed raise arrived verbatim with the multilspy import (2f607ff2)
and was never a deliberate guard.
- The Project layer's equivalent check already tolerates missing paths
(9c19d5f7), but returns False unconditionally. The LS layer instead
classifies lexically so that LS-reported build artifacts still classify
as ignored: with #1645 having removed the JDTLS build-dirname ignores,
it is the unsupported-extension rule that catches *.class paths.
Make the Svelte companion TypeScript server raise on readiness and
indexing timeouts instead of silently proceeding with a partial index,
turning flaky/wrong cross-file results into clear failures:
- TypeScriptLanguageServer: timeout handling factored into overridable
hooks (_handle_server_ready_timeout, _handle_project_indexing_timeout);
the base server keeps its historical permissive behavior, so plain
TypeScript setups are unchanged.
- SvelteTypeScriptServer: both overrides raise TimeoutError; only the
indexing-timeout error includes describe_indexing_state(), instead of
serving requests from a cold or partially indexed program.
- Configurable server_ready_timeout / indexing_timeout via
ls_specific_settings.
- SvelteLanguageServer: file-open failures during companion preparation
raise SvelteCompanionPreparationError with a count-first file listing
(capped at 10), chained to the first underlying error so the cause is
visible without DEBUG logging.
Apply the same fail-loudly principle to the test fixture bootstrap:
- Svelte test repo: npm ci with a committed package-lock.json, removing
dependency-resolution non-determinism from CI runs.
- Run svelte-kit sync explicitly during fixture setup and fail loudly if
.svelte-kit/tsconfig.json cannot be generated. The fixture's "prepare"
script masks sync failures ("svelte-kit sync || echo ''"), and a known
npm bug (npm/cli#4828) can silently skip platform-specific optional
dependencies; the result is unresolvable $lib path aliases and
cross-file tests failing with partial results instead of a clear,
actionable error.
- codespell: skip third-party fixture repositories, using the anchored
pattern ./test/resources/repos (codespell matches skip globs against
the ./-prefixed paths it walks, so an anchor-less pattern never
matches).
Also add direct regression tests for the timeout-policy split
(test/solidlsp/test_typescript_timeout_policy.py: base stays
permissive, companion raises, settings plumbing and precedence, and
start-or-completion indexing-wait semantics). The tests use no mocking or
patching of any kind; a hand-written call-recording fake drives the companion
guard against real file discovery under a pytest tmp_path. They spawn no LS
process and avoid external readiness races; one wait test uses a real Timer
with a 30s ceiling.
The configuration docs state that the TypeScript table lists both knobs; the
Svelte table lists indexing_timeout, with readiness inheritance documented in
prose.
The Nomic solidity-language-server validates asynchronously: document
changes are debounced (~250ms), validation runs a forge/solc compile,
and completion is signalled with the custom notification
custom/validation-job-status -- emitted on every completion path
(success, validation failure, job errors; verified in the bundled 0.8.4
package, which publishes the document's diagnostics before the signal).
Serena ignored the signal ("Unhandled method
'custom/validation-job-status'") and waited a fixed 2.5s for a
non-empty publishDiagnostics, returning [] whenever validation finished
later. Warm environments fit the window; slow or cold ones (macOS and
Windows CI runners, which download solc 0.8.20 on every run -- the CI
cache only covers ~/.serena/language_servers/static, not
~/.local/share/svm) often do not. This is the root cause of the
empty-diagnostics failures of test_file_diagnostics[solidity]: xfailed
on macOS since its introduction (bd7e64e8) and failing on Windows on
Tests run #4236, job catch-all (windows-latest). Shrinking the wait
reproduces the exact failure signature (AssertionError: []) on Linux.
Changes:
- Handle custom/validation-job-status (threading.Event + last payload).
The payload carries no document URI, so the event is global to the
server instance.
- request_text_document_diagnostics keeps the 2.5s non-empty publish
wait as the fast path. When it misses, it awaits the completion
signal against a 60s deadline (aligned with the indexing wait) and
takes the generation-advancing publication for the request URI,
accepting an empty result as the validated answer; a completion that
brings no publication for the URI (e.g. one belonging to another
document) re-arms the wait until the deadline. If the signal never
arrives, a warning is logged and the previous empty-result behavior
applies, bounded by the deadline.
- Remove the darwin xfail from test_file_diagnostics; add regression
tests for the fallback delivery (fast path forced to miss), for the
never-signalled bound, and for re-arming on a spurious completion
belonging to another document.
- CHANGELOG entry under Language Servers.
Design notes:
- Mirrors #1639 (TypeScript/VTS: event-based indexing wait replacing a
fixed sleep).
- The simpler alternative -- accepting empty publications and raising
the fixed wait -- was rejected: without the completion signal, an
early clear-on-open publication would be taken as the answer, and
every clean file would stall for the full wait.
- Supersedes the quarantine approach of draft PR #1674 (broadening the
xfail to non-Linux); whichever lands second resolves the trivial
conflict on the xfail block.
Ehsan