search_for_pattern resolved each match's line number by rescanning the
file from the beginning (TextUtils.get_line_from_index walks a
TextStepper from index 0), making coordinate resolution O(n) per match
and O(n*m) for m matches. search_text now precomputes line start
offsets once per file (O(n)) and resolves each match via binary search
(O(log n) per match).
The precomputed table mirrors TextStepper line semantics exactly:
\n, \r\n and bare \r are line separators, and an index pointing at the
\n of a \r\n pair resolves to (next line, 0) — the same rule
TextUtils.get_line_col_from_index implements, now pinned by regression
tests (including the mixed line-ending edge cases from the
insert_text_at_position fix).
On a synthetic 12k-line file (494 KB) with 723 matches, match-coordinate
resolution drops from ~1764 ms to ~2 ms (~1000x); the one-off
precompute costs ~1.1 ms. Small files regress nothing (100 lines, 5
matches: 70 µs -> 11 µs). Results are byte-for-byte identical to the
previous implementation.
Note: compatible with PR #1899 (interruptible agent regexes): that PR
touches the re.compile call site in search_text and the ContentReplacer
helpers, this PR touches the post-match line-coordinate resolution; the
two changes are complementary and rebase cleanly onto each other.
* perf(search_text): add reproducible benchmark script
scripts/profile_search_text.py regenerates the synthetic benchmark
content with a fixed seed (no real-world code), asserts both resolution
paths produce identical results, and prints the before/after timings
quoted in the PR description. Default: 12k lines / ~1830 matches;
optional CLI args scale the file down for quick runs.
* refactor(ls): encapsulate cached text coordinates in TextCoordinates abstraction
Address review feedback on the search_text line-lookup PR: the previously
added module-level helpers duplicated TextStepper's line-ending logic and
exposed raw offsets as a low-level data structure shared between two
functions, violating the project's software design principles (each
concern in exactly one home; dataclasses instead of tuples for simple
data storage).
Introduce in solidlsp/ls_utils.py:
- LineCol: frozen, kw-only dataclass for a 0-based line/column position
- TextCoordinates: encapsulates the cached table of line start offsets,
built via TextStepper (step_line + line_start_idx), with a single
public method line_col_at_index() resolving an index via binary
search, including the CRLF rule (an index pointing at the '\n' of a
'\r\n' pair denotes the beginning of the next line, column 0) and
InvalidTextLocationError for out-of-range indices
search_text now instantiates TextCoordinates once per file and resolves
each match's coordinates through the public method; the private helpers
(_compute_line_starts, _line_col_at_index) are removed.
Equivalence with TextUtils.get_line_col_from_index verified across
mixed line endings, empty content and boundary indices (0 mismatches).
Tests are behaviour-anchored: the implementation-coupled parametrized
test importing the private helpers is replaced by public-contract tests
of TextCoordinates (test/solidlsp/test_ls_utils.py) and absolute line
assertions through the public search_text API. The benchmark script
uses the public API as well.
Benchmark (synthetic 12k lines, ~723 matches, best of 5): coordinate
resolution ~1764 ms -> ~2 ms; precompute ~1.1 ms; small files unchanged.
Semantics are unchanged; results are identical to the previously
proposed implementation and to the upstream reference implementation.
* fix(test): rename test_ls_utils to test_text_coordinates to avoid basename collision
The new test file collided with the pre-existing upstream
test/solidlsp/util/test_ls_utils.py: pytest (prepend import mode, no
__init__.py in the test dirs) requires unique basenames, and collecting
both modules produced an import-file-mismatch error that aborted
collection with exit code 2 across every CI matrix job.
Rename to test_text_coordinates.py (unique, and matches the class under
test). Full-suite collection verified locally (2252 tests, 0 errors).
trusted_project_path_patterns is matched against the project's root path itself
(SerenaConfig.is_trusted_project_path, called from Project.is_trusted with project_root).
A '<project root>/**' entry therefore matches only paths below the root and trusts no
project at all, while reading like the natural way to name one project. The template's
example showed only parent-directory globs, so there was nothing to copy for the
single-project case.
Adds the bare-root form to the example, states which path a pattern is matched against,
and pins all three cases in tests so the documented example cannot silently go stale.
Requested by a maintainer in #2001.
The repository is licensed per component. SolidLSP (src/solidlsp,
test/solidlsp, test/resources) remains MIT-licensed and independently
reusable; the Serena application (src/serena, src/interprompt, scripts,
test/serena, docs) is licensed under GPL-3.0-or-later starting with the v2
licensing transition. The change is not retroactive: all releases and
commits up to v1.7.0 / 74c38a65 (tag mit-final) remain available under MIT.
Since MIT is GPL-compatible, a distribution combining both (such as the
serena-agent package) is as a whole subject to GPL-3.0-or-later, while the
SolidLSP files themselves stay MIT and can be extracted and used separately
under MIT terms. The distribution metadata therefore declares
GPL-3.0-or-later, with both license texts shipped alongside it.
Serena originally began under the GPL (v2) and was switched to MIT in
May 2025 following community requests. We consider that change a mistake;
the substantial changes in v2 make this the appropriate time to revert it.
We want the best version of Serena to remain free.
Changes:
* LICENSE is now the licensing overview; canonical license texts live in
LICENSES/ (MIT.txt is the previous LICENSE verbatim, GPL-3.0-or-later.txt
is the unmodified FSF text)
* pyproject.toml declares the PEP 639 license expression
"GPL-3.0-or-later" and bundles LICENSE and LICENSES/* as license files;
the deprecated MIT classifier is dropped and flake.nix declares gpl3Plus;
README has per-component license badges and a License section
* SPDX-License-Identifier headers in all Python sources under src/ and
scripts/, added by the new idempotent scripts/add_spdx_headers.py, which
gen_prompt_factory.py also uses to keep the header on the generated
module; existing third-party notices are preserved
* CLA.md: Contributor License Agreement (contributor retains copyright;
grants a perpetual, irrevocable license including relicensing under any
terms, incl. proprietary/commercial; patent grant; authority
representations), to be enforced repository-wide via cla-assistant.io
* CONTRIBUTING.md, PR template and a new docs page explain the licensing
boundary and the CLA workflow
* Apply object-oriented design
* Establish common interface via protocol LanguageServerIdLike
* Move all registration concerns to LanguageServerRegistry
`project health-check` runs three tools but only two of them can fail the check: an
exception from `FindReferencingSymbolsTool` was caught and logged as a warning, while
the verdict tested `find_symbol_data` alone. The command therefore printed
"Health check passed - All tools working correctly" and exited 0 on a project whose
reference search had just blown up - the one signal a health check exists to give.
Co-authored-by: Arkadiusz Gładki <biosmoothly@gmail.com>
* fix: disable automatic type acquisition in TypeScript backends
* fix: use shared initialization builder for VTS
Follow review guidance to apply shallow per-key overrides without restoring the ATA default after a user replaces the typescript block. Preserve the legacy settings alias, document the behavior, and cover canonical-key precedence.
---------
Co-authored-by: GuddXzy <267062192+GuddXzy@users.noreply.github.com>
Caching happened in the base implementation before subclass overrides could
post-process the result, so an override's work was either never cached or,
if it mutated in place, re-applied to cached output on every hit.
Restructure both symbol caches so that they wrap the subclass logic:
subclasses now post-process via dedicated hooks (`_request_raw_document_symbols`
and `_build_document_symbols_from_raw_symbols`), and the cached value is
the fully processed result. Cache versions are bumped where the cached
content changes as a consequence.
Two Serena instances activating the same project concurrently launched
their Kotlin LSP processes against the same on-disk index storage
location (both the modern launcher's --system-path arg and the legacy
initializationOptions.storagePath, hardcoded to None), so the second
instance's requests were repeatedly cancelled by the first instance's
server.
DependencyProvider now claims that storage directory with a
non-blocking file lock held for the process's lifetime. A single
instance, including across restarts, still gets the same directory
and keeps its cache; a second concurrent instance falls back to a
directory of its own instead of contending for the first one's.
Fixes#1966
insert_text_at_position returns the position of the end of the inserted text.
That position was computed by _get_updated_position_from_line_and_column_and_edit,
which steps the inserted text in isolation and adds the resulting line/column
delta to the start position.
An insertion is not independent of its surroundings. Under the LSP newline
definition adopted in 9208a3ae, a "\n" inserted directly after an existing "\r"
merges with it into a single "\r\n" sequence, so the text gains no line while the
isolated arithmetic still counts one:
insert_text_at_position("\r", 1, 0, "\n") -> ("\r\n", 2, 0); actual (1, 0)
insert_text_at_position("x\ry", 1, 0, "\n") -> ("x\r\ny", 2, 0); actual (1, 0)
insert_text_at_position("a\r", 2, 0, "X") -> ("a\r\nX", 2, 1); actual (1, 1)
The third case reaches the same fault through the one-line-past-EOF branch, which
prepends a "\n" that then merges with the trailing "\r". The converse interaction
was wrong too: inserting between the "\r" and the "\n" of a "\r\n" splits it, and
the isolated arithmetic reported (0, 2) -- a column on a line the split had just
emptied. In every case the resulting text was correct; only the position was wrong.
9208a3ae centralised line/column computation on TextStepper and reimplemented the
TextUtils methods on it. This helper was rebased onto TextStepper in that commit
but kept its original shape: it steps one string, and the string it steps is the
insertion rather than the result. Once a newline can be formed by two adjacent
characters, no computation over the inserted text alone can be correct.
The position is now determined from the resulting text via get_line_col_from_index,
the same function that resolves positions everywhere else, which removes the last
computation in TextUtils that did not go through the centralised path. The helper
had no other callers and is deleted, along with the line/col adjustment in the
past-EOF branch that existed only to feed it.
The column is clamped to the end of the text before both the splice and the
derivation. This is not a new tolerance: the splice already clamped implicitly
(text[:i] with i > len(text) yields the whole text), so an out-of-range column has
always appended successfully. Making the clamp explicit keeps the position derived
from the same index the splice used; without it, deriving from the result would
newly raise for input that previously worked.
Columns are offsets into the Python string (code points), which the docstring now
states. LSP columns are UTF-16 code units and nothing converts between the two at
the boundary -- a real defect, but a pre-existing and separate one: the previous
arithmetic did not preserve UTF-16 units either, since it added a code-point length
to a code-unit column and so produced a value in neither unit.
"\U0001f600" + "\U0001f600" at column 2
old (0, 3); UTF-16 (0, 4); code points (0, 2)
Seven regression tests cover the boundary, mid-document and past-EOF merges, the
newline split, an out-of-range column, a column past a non-BMP character, and an
unaffected "\n"-terminated control. The six that pin changed behaviour were each
verified to fail before the change; the control passes either way.
* fix(memories): stop losing memory content on an interrupted write
save_memory() and edit_memory() wrote the memory file with a plain
open(path, "w") followed by f.write(content). That truncates the file
before the new content is complete, so a crash, an OOM kill, or a
disk-full error partway through the write can leave the file empty or
holding a stale tail instead of either the old or the new content.
Add write_file_atomic() to util/file_system.py: it writes to a temp
file in the same directory, then os.replaces it into place, with the
same short PermissionError retry save_yaml() already uses for a
transient Windows sharing violation. save_memory() and edit_memory()
now go through it instead of the direct open().
CodeEditor._save_edited_file() is left out of this change: os.replace
makes the destination inherit the temp file's permissions rather than
the original's, and tempfile.mkstemp creates that temp file 0600, so
an executable source file would lose its +x bit on save. Memory files
are always plain text Serena itself creates, so this doesn't apply to
them.
Refs #1958
* fix(memories): preserve file mode in write_file_atomic
mkstemp() creates the temp file with mode 0600 regardless of umask, and
os.replace() carries that mode into the destination. save_memory() and
edit_memory() going through write_file_atomic() silently tightened an
existing memory file's permissions (e.g. 0644 -> 0600) on every save or
edit, dropping group/other read access.
Restore the target's existing mode before the replace, or fall back to
what a plain open(path, "w") would give a brand-new file (0666 masked
by the process umask) when there is nothing to preserve.
Refs #1958
* test(memories): skip POSIX permission-preservation tests on Windows
os.chmod does not set POSIX group/other bits on Windows, so
test_save_memory_preserves_existing_file_permissions and
test_edit_memory_preserves_existing_file_permissions assert a mode
the platform cannot produce. Matches the existing skipif convention
used elsewhere in this suite for POSIX-only filesystem behavior.
Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
---------
Signed-off-by: Amir Fathi <amirfathi.me@gmail.com>
The inputs had been pinned for a year.
Verified on aarch64-darwin: `nix build` succeeds and the resulting `bin/serena`
reports `Serena 1.7.1.dev0`. The Linux packages and devShell evaluate cleanly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nixpkgs now warns on every access to the `stdenv.is*` shorthands, which hide
the build/host/target distinction
(https://github.com/NixOS/nixpkgs/pull/518407). The nixpkgs pinned here
predates that change, but a future update will surface the warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DartLanguageServer registered handlers for $/analyzerStatus and
experimental/serverStatus but both were no-ops, so start() returned as
soon as initialized was sent instead of waiting for either signal, unlike
every other backend (pyright, basedpyright, rust-analyzer, clojure-lsp,
jedi, omnisharp, clangd) that receives the same notifications. A request
issued right after activation could return before the workspace scan
finished.
Gate on a threading.Event set by either notification, bounded by 60s,
mirroring pyright_server.py's analysis_complete pattern.
Codex's documented hook wiring only routes `serena-hooks remind` through
PreToolUse on the `Bash` matcher, so `ToolUseCounter.update()`'s
reset-on-Serena-tool-use branch is never reached there: it only fires when
`remind` itself is invoked for a `mcp__serena__*` tool name, which the
Bash-only matcher excludes. Reminder counters therefore never clear after a
successful Serena call, and an unrelated grep/read burst afterwards can trip
the deny threshold on state that should have been reset.
Add a `serena-hooks reset` command, wired to PostToolUse on Serena's own
tools, that resets the counters after a successful call. Gated on the call
having succeeded (`tool_response` carrying no `isError: true`, the MCP
`tools/call` result shape) so a failed Serena call does not mask a real
grep/read streak, per the issue's own acceptance criteria. The symbolic-tool
classification is shared with the existing PreToolUse hook via a small
extracted helper so both agree on what counts as a Serena tool.
Fixes#1852
_init_active_project_language_backend launched the configured IDE with a
fire-and-forget subprocess.Popen whenever no plugin server matched the
activating project, independently in every session. Two or more sessions
activating different projects at (nearly) the same time each decided
independently that no server was running and raced jetbrains_launch_command
against the same IDE config directory; only one IDE instance can hold that
directory's lock, so the rest failed with a native DirectoryLock modal that
never surfaces as a subprocess exit code, leaving Serena with no plugin
server and no indication why.
Add JetBrainsLaunchCoordinator.launch_and_wait_for_plugin_server, which
serializes launches on a file lock keyed by jetbrains_launch_command and
re-checks for a plugin server both before and after acquiring it, so a
session that loses the race waits for the winner's IDE instead of starting
its own. It also polls for the plugin server to become reachable after the
launch command exits, closing the second gap the issue names: the launcher
process exiting is not the same as the IDE being ready to serve.
Fixes#1864
Serena notifies language servers about files changed outside its own edit tools (a git checkout, another editor, a build step) by sending workspace/didChangeWatchedFiles from LanguageServerManager.poll_and_notify, whose docstring names this exact case: such edits are 'otherwise invisible to a warm language server, causing symbolic queries to answer from a stale index'.
ClojureLSP never declared the corresponding client capability, so clojure-lsp was not told those notifications are sent and need not treat them as a reason to invalidate its analysis. DefaultInitializeParamsBuilder does not supply a default capabilities block, so nothing filled this in on the server's behalf.
Relates to #1593, where find_symbol returned a body from a comment mentioning the symbol rather than its definition, at a position the symbol had previously occupied.
Serena notifies language servers about files changed outside its own edit tools (a git checkout, another editor, a build step) by sending workspace/didChangeWatchedFiles from LanguageServerManager.poll_and_notify, whose docstring names this exact case: such edits are 'otherwise invisible to a warm language server, causing symbolic queries to answer from a stale index'.
ClojureLSP never declared the corresponding client capability, so clojure-lsp was not told those notifications are sent and need not treat them as a reason to invalidate its analysis. DefaultInitializeParamsBuilder does not supply a default capabilities block, so nothing filled this in on the server's behalf.
Relates to #1593, where find_symbol returned a body from a comment mentioning the symbol rather than its definition, at a position the symbol had previously occupied.
_flush_deferred_workspace_scan sets _workspace_scan_flushed unconditionally after its
two completion() flush attempts, even when both raised. A later find_referencing_symbols
call then silently skips the flush and answers from a possibly-incomplete workspace AST
cache for the rest of the session, with no exception and no retry.
Fixes#1871
The Dart analysis server emits $/analyzerStatus while it analyzes a workspace.
The Dart adapter did not register this server-specific notification, so the
generic notification dispatcher logged every event as an unhandled-method
warning even though no client action is required.
Register the method with the adapter's existing no-op notification handler,
keeping the global behavior for genuinely unknown notifications unchanged. Add
a startup regression test that sends the notification through the production
dispatcher, and document the fix in the changelog.
Fixes#1855
typescript-language-server sends a $/progress "end" for the in-flight
token as part of tearing its connection down after tsserver aborts (e.g.
a V8 heap OOM). _wait_for_indexing_start_or_completion could not
distinguish that from a real completion, so find_referencing_symbols
logged "TypeScript cross-file indexing complete" and returned {} with
isError: false, indistinguishable from a symbol genuinely having no
references.
The crash is already observed independently via window/logMessage
(type Error, "[tsserver] Exited"), but the handler only logged it. Wire
that signal into a _crash_message flag and raise TypeScriptServerCrashedError
from every path that would otherwise report indexing as complete
(wait_for_indexing and both early-return branches of
_wait_for_indexing_start_or_completion), so a crash propagates as a tool
error instead of a silent empty result.
Fixes#1814
`dotenv` was added in 605bca19 together with the `from dotenv import
load_dotenv` call in `src/serena/agno.py`, six days after `python-dotenv` had
already been added in 276f6a84. The `dotenv` module that the code imports is
provided by `python-dotenv`. The separately named `dotenv` distribution on PyPI
is an unrelated, deprecated project by another author, republished as a
metadata-only stub whose only content is `Requires-Dist: python-dotenv`; its
wheel ships no Python module at all, so the dependency has never contributed
anything at runtime.
The redundancy survived the bulk pinning pass in 45cd8ca1, which mechanically
rewrote `dotenv>=0.9.9` to `dotenv==0.9.9`.
Dropping it removes a third-party PyPI name from the set of distributions
trusted at install time, with no behavioural change. Verified that `from dotenv
import load_dotenv` resolves with only `python-dotenv==1.2.2` installed.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Guillermo