`search_text` mapped the match's exclusive end index to a line number via
`get_line_from_index`, which resolves an index at a line start to the following
line. A match ending with a newline was therefore attributed one line too many,
and `context_lines_after` was shifted by the same amount.
FileUtils.read_file has two return paths with different newline semantics.
The primary path, open(file_path, encoding=encoding).read(), uses the default
newline=None and therefore applies universal newlines, so CR never reaches
callers. The charset_normalizer fallback decodes the raw bytes instead
(match.raw.decode(match.encoding)), which performs no newline translation, so
any file that fails to decode with the project's configured encoding re-enters
Serena carrying its CR characters.
Normalize to LF at the fallback's return, matching the primary path.
- read via `ReadFileTool`
- searched via `SearchForPatternTool`
when using plugin version 2023.3.3+ (yet unreleased)
This is enabled via JetBrainsFileProxy (which reads the file through the IDE)
read_file determined line indices via Python's splitlines, which breaks on
\f, \v, \x1c-\x1e, \x85, U+2028 and U+2029 in addition to the LSP-compliant
line breaks counted by delete_lines/replace_lines. In a file containing any
of those characters the indices reported by read_file referred to different
physical lines than the editing tools operate on, so an edit made after the
required read_file could silently hit the wrong line.
Use TextUtils.split_lines, as already applied to search_text and
MatchedConsecutiveLines.from_file_contents. Files whose line breaks are all
LSP-compliant are decomposed exactly as before.
delete_lines/replace_lines address the position one line past the last line
(line == number of lines, col 0). get_index_from_line_col cannot resolve that
position when the file has no trailing newline (no closing newline to count),
so delete_text_between_positions raised InvalidTextLocationError and the delete
crashed on a very common file shape.
Clamp that past-EOF end position to end-of-file in delete_text_between_positions,
mirroring the guard insert_text_at_position already has. A file with no trailing
newline now deletes its last line identically to one with a trailing newline.
Adds TextUtils unit tests and a CHANGELOG entry.
stepping line by line instead of character by character,
delegating the scan for line breaks to native string search
(~15-100x speedup for a full pass over typical source files)
* Add helper class TextStepper to facilitate computations
* Reimplement TextUtils methods based on TextStepper
* Adhere to LSP definition of newlines being defined by either "\n", "\r\n" or "\r" on its own.
Previously, only "\n" was considered.
* Add util method TextUtils.split_lines, allowing usages of Python's splitlines to be replaced
The cpu matrix job had no timeout-minutes, so a hung job runs until
GitHub's default 6-hour limit. This is not hypothetical: on run
29064865197 (2026-07-10), the native (ubuntu-latest) job hung and ran
from 02:36:34 to 08:36:50 before GitHub cancelled it at exactly the
6-hour cap, blocking the PR's status the whole time and burning a
runner for 6 hours.
Healthy runs are far below the new limit: across the 8 most recent
successful runs of this workflow, the slowest job was 16 minutes
(niche on ubuntu-latest), with catch-all on windows-latest typically
11-14 minutes. 60 minutes leaves ~4x headroom for cache-cold runs
(language-server downloads, npm installs) while cutting the worst-case
hang from 6 hours to 1.
Part of the timeout-hardening discussed in #1695.
When auto-generating a Serena project configuration at startup, compute the languages
in a background thread and update them later to avoid delays of the MCP server startup.
Where completeness of the language list matters (i.e. language server initialisation),
a corresponding event is awaited to ensure completeness.
Perl::LanguageServer only indexes files whose extension is in its
`perl.fileFilter` and skips directories in `perl.ignoreDirs`. Both were
hardcoded in PerlLanguageServer, so projects whose Perl source uses
non-standard extensions (e.g. .cgi / .psgi web handlers, which can
dominate a mature codebase) had no way to make those files visible to
symbol/reference queries without forking Serena (#1449).
Surface both lists through `ls_specific_settings["perl"]` (keys
`file_filter` and `ignore_dirs`), defaulting to the existing values so
behaviour is unchanged for current projects. The resolution lives in a
pure static method `_resolve_filter_settings` so the config plumbing is
covered by unit tests that don't require a Perl runtime.
Keep Serena's symbol index in sync with the LS (#1449, per review):
`find_symbol` and symbol indexing are driven by
`Language.PERL.get_source_fn_matcher()`, so adding an extension (e.g.
.cgi) to `file_filter` alone would leave the matcher untouched and the
extension's symbols invisible. Since that matcher is now a @cache'd
per-language singleton, add a `FilenameMatcher.add_extensions()` mutator
and call it from `PerlLanguageServer` so the configured extensions also
reach the symbol index, the ignore checks, and language composition.
Avoid cross-project leakage (per review): because the matcher is a
process-wide singleton, extensions added for one project would otherwise
persist into the next. `FilenameMatcher` now snapshots its initial
configuration and exposes `reset()`, and `SolidLanguageServer.__init__`
resets the matcher right after `self.language` is determined, so every
activation starts from the language's defaults before the subclass
re-applies its own `file_filter`.
This complements 860c5841 (which added .t to the default list as a
surface-level fix) by addressing the root configurability gap that the
maintainer acknowledged in #1449.
Hanging was never reproduced, this is preemptive hardening
- Add a 120s timeout to the server_ready wait (previously waited with no
timeout, so a server that crashed or never sent experimental/serverStatus
with quiescent=true would hang startup forever). On timeout we log a
warning and proceed, matching kotlin/matlab/powershell/etc.
- Harden the language/status and experimental/serverStatus notification
handlers to use dict.get() instead of raising KeyError on unexpected
payloads (a KeyError in the quiescent handler could itself prevent
server_ready from ever being set, feeding the hang).
- Relax the over-strict completionProvider capability check: an exact-dict
equality assertion crashed initialization whenever rust-analyzer changed
or added a completionProvider field. These values are not used downstream,
so a mismatch is now logged as a warning instead of asserted.
Language servers can report locations of generated files that are not on
disk. JDTLS does this for Lombok-generated types, reporting e.g.
target/classes/test_repo/LombokModel$LombokModelBuilder.class while
indexing; is_ignored_path then raised FileNotFoundError, turning an
intermittent indexing state into a hard test failure
(test_bare_symbol_names[java]; Tests run #4240, job jvm (macos-latest)).
Changes:
- is_ignored_path classifies missing paths with the usual lexical rules
instead of raising: ignored-dirname components, the unsupported-extension
rule, then the configured pathspec. A missing path is treated as a file
if its name has a suffix and as a directory otherwise (so the leaf of a
missing suffixless path is also checked against ignored dirnames).
Behavior for existing paths is unchanged: the checks were reordered
(ignored-dirname before unsupported-extension), but each check is pure,
so the outcomes are identical.
- SymbolLocationRequest.convert_location_item skips locations whose
absolute path does not exist (log + return None), so phantom build
artifacts never reach consumers regardless of ignore configuration.
- New unit tests (test double, no language server process; they run in
the catch-all CI batch) pin the missing-path classification — including
the exact Lombok .class path shape from the CI failure — and the
skipping of symbol locations that are not on disk.
Notes:
- The removed raise arrived verbatim with the multilspy import (2f607ff2)
and was never a deliberate guard.
- The Project layer's equivalent check already tolerates missing paths
(9c19d5f7), but returns False unconditionally. The LS layer instead
classifies lexically so that LS-reported build artifacts still classify
as ignored: with #1645 having removed the JDTLS build-dirname ignores,
it is the unsupported-extension rule that catches *.class paths.
Ehsan Barkhordar