`search_text` mapped the match's exclusive end index to a line number via
`get_line_from_index`, which resolves an index at a line start to the following
line. A match ending with a newline was therefore attributed one line too many,
and `context_lines_after` was shifted by the same amount.
FileUtils.read_file has two return paths with different newline semantics.
The primary path, open(file_path, encoding=encoding).read(), uses the default
newline=None and therefore applies universal newlines, so CR never reaches
callers. The charset_normalizer fallback decodes the raw bytes instead
(match.raw.decode(match.encoding)), which performs no newline translation, so
any file that fails to decode with the project's configured encoding re-enters
Serena carrying its CR characters.
Normalize to LF at the fallback's return, matching the primary path.
- read via `ReadFileTool`
- searched via `SearchForPatternTool`
when using plugin version 2023.3.3+ (yet unreleased)
This is enabled via JetBrainsFileProxy (which reads the file through the IDE)
read_file determined line indices via Python's splitlines, which breaks on
\f, \v, \x1c-\x1e, \x85, U+2028 and U+2029 in addition to the LSP-compliant
line breaks counted by delete_lines/replace_lines. In a file containing any
of those characters the indices reported by read_file referred to different
physical lines than the editing tools operate on, so an edit made after the
required read_file could silently hit the wrong line.
Use TextUtils.split_lines, as already applied to search_text and
MatchedConsecutiveLines.from_file_contents. Files whose line breaks are all
LSP-compliant are decomposed exactly as before.
delete_lines/replace_lines address the position one line past the last line
(line == number of lines, col 0). get_index_from_line_col cannot resolve that
position when the file has no trailing newline (no closing newline to count),
so delete_text_between_positions raised InvalidTextLocationError and the delete
crashed on a very common file shape.
Clamp that past-EOF end position to end-of-file in delete_text_between_positions,
mirroring the guard insert_text_at_position already has. A file with no trailing
newline now deletes its last line identically to one with a trailing newline.
Adds TextUtils unit tests and a CHANGELOG entry.
stepping line by line instead of character by character,
delegating the scan for line breaks to native string search
(~15-100x speedup for a full pass over typical source files)
* Add helper class TextStepper to facilitate computations
* Reimplement TextUtils methods based on TextStepper
* Adhere to LSP definition of newlines being defined by either "\n", "\r\n" or "\r" on its own.
Previously, only "\n" was considered.
* Add util method TextUtils.split_lines, allowing usages of Python's splitlines to be replaced
The cpu matrix job had no timeout-minutes, so a hung job runs until
GitHub's default 6-hour limit. This is not hypothetical: on run
29064865197 (2026-07-10), the native (ubuntu-latest) job hung and ran
from 02:36:34 to 08:36:50 before GitHub cancelled it at exactly the
6-hour cap, blocking the PR's status the whole time and burning a
runner for 6 hours.
Healthy runs are far below the new limit: across the 8 most recent
successful runs of this workflow, the slowest job was 16 minutes
(niche on ubuntu-latest), with catch-all on windows-latest typically
11-14 minutes. 60 minutes leaves ~4x headroom for cache-cold runs
(language-server downloads, npm installs) while cutting the worst-case
hang from 6 hours to 1.
Part of the timeout-hardening discussed in #1695.
When auto-generating a Serena project configuration at startup, compute the languages
in a background thread and update them later to avoid delays of the MCP server startup.
Where completeness of the language list matters (i.e. language server initialisation),
a corresponding event is awaited to ensure completeness.
Perl::LanguageServer only indexes files whose extension is in its
`perl.fileFilter` and skips directories in `perl.ignoreDirs`. Both were
hardcoded in PerlLanguageServer, so projects whose Perl source uses
non-standard extensions (e.g. .cgi / .psgi web handlers, which can
dominate a mature codebase) had no way to make those files visible to
symbol/reference queries without forking Serena (#1449).
Surface both lists through `ls_specific_settings["perl"]` (keys
`file_filter` and `ignore_dirs`), defaulting to the existing values so
behaviour is unchanged for current projects. The resolution lives in a
pure static method `_resolve_filter_settings` so the config plumbing is
covered by unit tests that don't require a Perl runtime.
Keep Serena's symbol index in sync with the LS (#1449, per review):
`find_symbol` and symbol indexing are driven by
`Language.PERL.get_source_fn_matcher()`, so adding an extension (e.g.
.cgi) to `file_filter` alone would leave the matcher untouched and the
extension's symbols invisible. Since that matcher is now a @cache'd
per-language singleton, add a `FilenameMatcher.add_extensions()` mutator
and call it from `PerlLanguageServer` so the configured extensions also
reach the symbol index, the ignore checks, and language composition.
Avoid cross-project leakage (per review): because the matcher is a
process-wide singleton, extensions added for one project would otherwise
persist into the next. `FilenameMatcher` now snapshots its initial
configuration and exposes `reset()`, and `SolidLanguageServer.__init__`
resets the matcher right after `self.language` is determined, so every
activation starts from the language's defaults before the subclass
re-applies its own `file_filter`.
This complements 860c5841 (which added .t to the default list as a
surface-level fix) by addressing the root configurability gap that the
maintainer acknowledged in #1449.
Dominik Jain