diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c5bf317..12eb66de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,11 +5,22 @@ Status of the `main` branch. Changes prior to the next official version change w * General: - Support environment variable `SERENA_USAGE_REPORTING` (set to `false` to disable usage reporting) +# 1.1.1 + +* General: + - Enable cert verification for HTTPS request to oraios-software.de #1320 + +* JetBrains: + - `JetBrainsRenameTool` can now also rename occurrences in comments and text. + +* Language Servers: + - Fix Dart LSP returning only symbol name as body instead of full method body. + # 1.1.0 * General: - - Add `serena init` and `serena setup` commands - **Major**: Add commands for hooks and documentation of recommended setup. Consider setting up the [recommended hooks](https://oraios.github.io/serena/02-usage/030_clients.html) ! + - Add `serena init` and `serena setup` commands - Rework installation instructions, switching to releases on pypi for distribution. Please update your mcp startup commands! - Add minimal usage data collection on startup (only Serena version, language backend, OS, dashboard enabled status; no personally identifiable information) - Fix: git commit id in Serena version strings was incorrect diff --git a/README.md b/README.md index 6497f03c..5f07841b 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,17 @@ Serena is the IDE for your coding agent. +
+ discord + license +
+
+ + * Serena provides essential **semantic code retrieval, editing and refactoring tools** that are akin to an IDE's capabilities, operating at the symbol level and exploiting relational structure. * It integrates with any client/LLM via the model context protocol (**MCP**). - + Serena's **agent-first tool design** involves robust high-level abstractions, distinguishing it from approaches that rely on low-level concepts like line numbers or primitive search patterns. @@ -19,7 +26,7 @@ more complex codebases. > [!IMPORTANT] > Do not install Serena via an MCP or plugin marketplace! They contain outdated and suboptimal installation commands. -> Instead, follow our [Quick Start](quick-start) instructions. +> Instead, follow our [Quick Start](#quick-start) instructions. ## How Serena Works @@ -38,7 +45,7 @@ To connect the Serena MCP server to your client, you either * provide the client with a launch command that allows it to start the MCP server, or * start the Serena MCP server yourself in HTTP mode and provide the client with the URL. -See the [Quick Start](quick-start) section below for information on how to get started. +See the [Quick Start](#quick-start) section below for information on how to get started. ## Programming Language Support & Semantic Analysis Capabilities diff --git a/docs/02-usage/030_clients.md b/docs/02-usage/030_clients.md index 1d6cea12..8280d604 100644 --- a/docs/02-usage/030_clients.md +++ b/docs/02-usage/030_clients.md @@ -40,10 +40,15 @@ A key mechanism for this is to use the appropriate [context](#contexts) when sta (clients-common-pitfalls)= ### Common Pitfalls -**Discoverability of the `serena`**. +**Discoverability of the `serena` command**. Your client may not find the `serena` CLI command, even if it is on your system PATH. In this case, a workaround is to provide the full path to the `serena` executable. +**Serena's tools not being used**. +With some clients, you may experience that Serena's tools are not being used. +This is mainly due to problems in the client itself (like a poorly implemented tool discovery). To counteract this, +Serena comes with a set of commands that can be used in _hooks_. See the sections on hooks for Claude Code and VSCode below. + **Environment Variables**. Some language servers may require additional environment variables to be set (e.g. F# on macOS with Homebrew), which you may need to explicitly add to the MCP server configuration. @@ -117,22 +122,6 @@ Confirm that Claude Code is connected to Serena by running the `/mcp` command an If Serena fails to start fast enough, you should set `MCP_TIMEOUT` to a sufficiently high value (e.g. by adding `export MCP_TIMEOUT=60000` to your shell profile) -**Maximum Token Efficiency.** To maximize token efficiency, you may want to use Claude Code's -*on-demand tool loading* feature, which is supported since at least v2.0.74 of Claude Code. -This feature avoids sending all tool descriptions to Claude upon startup, thus saving tokens. -Instead, Claude will search for tools as needed (but there are no guarantees that it will -search optimally, of course). -To enable this feature, set the environment variable `ENABLE_TOOL_SEARCH=true`. -Depending on your shell, you can also set this on a per-session basis, e.g. using -```shell -ENABLE_TOOL_SEARCH=true claude -``` -in bash/zsh, or using -```shell -set ENABLE_TOOL_SEARCH=true && claude -``` -in Windows CMD to launch Claude Code. - **Hooks.** Due to recent changes (especially dynamic tool loading) in Claude Code, the agent will often fail to make proper use of Serena's tools, either by failing to load them in the beginning or by forgetting the instructions in a long session diff --git a/lessons_learned.md b/lessons_learned.md deleted file mode 100644 index b6d15a70..00000000 --- a/lessons_learned.md +++ /dev/null @@ -1,90 +0,0 @@ -# Lessons Learned - -In this document we briefly collect what we have learned while developing and using Serena, -what works well and what doesn't. - -## What Worked - -### Separate Tool Logic From MCP Implementation - -MCP is just another protocol, one should let the details of it creep into the application logic. -The official docs suggest using function annotations to define tools and prompts. While that may be -useful for small projects to get going fast, it is not wise for more serious projects. In Serena, -all tools are defined independently and then converted to instances of `MCPTool` using our `make_tool` -function. - -### Autogenerated PromptFactory - -Prompt templates are central for most LLM applications, so one needs good representations of them in the code, -while at the same time they often need to be customizable and exposed to users. In Serena we address these conflicting -needs by defining prompt templates (in jinja format) in separate yamls that users can easily modify and by autogenerated -a `PromptFactory` class with meaningful method and parameter names from these yamls. The latter is committed to our code. -We separated out the generation logic into the [interprompt](/src/interprompt/README.md) subpackage that can be used as a library. - -### Tempfiles and Snapshots for Testing of Editing Tools - -We test most aspects of Serena by having a small "project" for each supported language in `tests/resources`. -For the editing tools, which would change the code in these projects, we use tempfiles to copy over the code. -The pretty awesome [syrupy](https://github.com/syrupy-project/syrupy) pytest plugin helped in developing -snapshot tests. - -### Dashboard and GUI for Logging - -It is very useful to know what the MCP Server is doing. We collect and display logs in a GUI or a web dashboard, -which helps a lot in seeing what's going on and in identifying any issues. - -### Unrestricted Bash Tool - -We know it's not particularly safe to permit unlimited shell commands outside a sandbox, but we did quite some -evaluations and so far... nothing bad has happened. Seems like the current versions of the AI overlords rarely want to execute `sudo rm - rf /`. -Still, we are working on a safer approach as well as better integration with sandboxing. - -### Multilspy - -The [multilspy](https://github.com/microsoft/multilspy/) project helped us a lot in getting started and stands at the core of Serena. -Many more well known python implementations of language servers were subpar in code quality and design (for example, missing types). - -### Developing Serena with Serena - -We clearly notice that the better the tool gets, the easier it is to make it even better - -## Prompting - -### Shouting and Emotive Language May Be Needed - -When developing the `ReplaceRegexTool` we were initially not able to make Claude 4 (in Claude Desktop) use wildcards to save on output tokens. Neither -examples nor explicit instructions helped. It was only after adding - -``` -IMPORTANT: REMEMBER TO USE WILDCARDS WHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD! -``` - -to the initial instructions and to the tool description that Claude finally started following the instructions. - -## What Didn't Work - -### Lifespan Handling by MCP Clients - -The MCP technology is clearly very green. Even though there is a lifespan context in the MCP SDK, -many clients, including Claude Desktop, fail to properly clean up, leaving zombie processes behind. -We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running -and can terminate it there. - -### Trusting Asyncio - -Running multiple asyncio apps led to non-deterministic -event loop contamination and deadlocks, which were very hard to debug -and understand. We solved this with a large hammer, by putting all asyncio apps into a separate -process. It made the code much more complex and slightly enhanced RAM requirements, but it seems -like that was the only way to reliably overcome asyncio deadlock issues. - -### Cross-OS Tkinter GUI - -Different OS have different limitations when it comes to starting a window or dealing with Tkinter -installations. This was so messy to get right that we pivoted to a web-dashboard instead - -### Editing Based on Line Numbers - -Not only are LLMs notoriously bad in counting, but also the line numbers change after edit operations, -and LLMs are also often too dumb to understand that they should update the line numbers information they had -received before. We pivoted to string-matching and symbol-name based editing. \ No newline at end of file diff --git a/llms-install.md b/llms-install.md deleted file mode 100644 index b02711c7..00000000 --- a/llms-install.md +++ /dev/null @@ -1,29 +0,0 @@ -# MCP Installation instructions - -This document is mainly used as instructions for AI-assistants like Cline and others that -try to do an automatic install based on freeform instructions. - -0. Make sure `uv` is installed. If not, install it using either `curl -LsSf https://astral.sh/uv/install.sh | sh` (macOS, Linux) or - `powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"` (Windows). Find the path to the `uv` executable, - you'll need it later. -1. Clone the repo with `git clone git@github.com:oraios/serena.git` and change into its dir (e.g., `cd serena`) -2. Check if `serena_config.yml` exists. If not, create it with `cp serena_config.template.yml serena_config.yml`. Read the instructions in the config. -3. In the config, check if the path to your project was added. If not, add it to the `projects` section -4. In your project, create a `.serena` if needed and check whether `project.yml` exists there. -5. If no `project.yml` was found, create it using `cp /path/to/serena/myproject.template.yml /path/to/your/project/.serena/project.yml` -6. Read the instructions in `project.yml`. Make sure the `project.yml` has the correct project language configured. - Remove the project_root entry there. -7. Finally, add the Serena MCP server config like this: - -```json - { - "mcpServers": { - ... - "serena": { - "command": "/abs/path/to/uv", - "args": ["run", "--directory", "/abs/path/to/serena", "serena-mcp-server", "/path/to/your/project/.serena/project.yml"] - } - } - } - -``` diff --git a/news/20260111.html b/news/20260111.html index f8c75150..91fa5031 100644 --- a/news/20260111.html +++ b/news/20260111.html @@ -8,7 +8,7 @@
  • Extended symbol information: The `find_symbol` and `find_referencing_symbols` tools (and their JetBrains variants) now can return more information about the symbol (specifically, docstrings and signatures).
    • -
  • Type hierarchy tool: A new tool – exclusive to JetBrains mode – +
  • Type hierarchy tool: A new tool – exclusive to JetBrains mode – that can fetch a symbol's hierarchy, i.e. superclasses and subclasses, which is very useful in many situations.
  • Compact overviews: The `get_symbols_overview` tools now return a much more compact @@ -17,6 +17,6 @@

    • - For more detailed information, see our changelog. + For more detailed information, see our changelog.

    \ No newline at end of file diff --git a/news/20260303.html b/news/20260303.html index ca91377f..c42233d7 100644 --- a/news/20260303.html +++ b/news/20260303.html @@ -7,6 +7,6 @@ into topics and sharing them across projects.

    - For more detailed information, see the documentation. + For more detailed information, see the documentation.

    \ No newline at end of file diff --git a/news/20260330.html b/news/20260330.html index c78c4be7..3eca0574 100644 --- a/news/20260330.html +++ b/news/20260330.html @@ -8,6 +8,6 @@ to other projects.

    - Find out more about this feature in our documentation. + Find out more about this feature in our documentation.

    \ No newline at end of file diff --git a/news/20260411.html b/news/20260411.html index f337c07f..ce07de08 100644 --- a/news/20260411.html +++ b/news/20260411.html @@ -4,19 +4,19 @@

    Installation. Serena has matured and we are switching to a PyPI-based release and installation mechanism. The recommended uv-tool-based installation decreases startup times and facilitates CLI usage. - See updated installation instructions + See updated installation instructions for more details.

    Improving Tool Usage via Hooks. To address issues with agents not using Serena's tools (agent drift), we now provide the `serena-hooks` commands and a recommended setup for Claude Code and VSCode. - Please read the updated documentation for more details. + Please read the updated documentation for more details. We strongly recommend using the hooks, since recent updates in Claude Code and VSCode resulted in reduced performance, which our hook commands compensate for.

    - Community. We opened a Discord server for our community. + Community. We opened a Discord server for our community.

    \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index e1638adb..c6daf2c9 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ requires = ["hatchling"] [project] name = "serena-agent" -version = "1.1.0" +version = "1.1.1" description = "" authors = [{ name = "Oraios AI", email = "info@oraios-ai.de" }] readme = "README.md" diff --git a/scripts/bump_version.py b/scripts/bump_version.py index 2fdd213d..e3426baa 100644 --- a/scripts/bump_version.py +++ b/scripts/bump_version.py @@ -44,6 +44,7 @@ def bump_version(major: bool, minor: bool, patch: bool, target_version: str | No if dry_run: click.echo(f"Dry run complete. Version would be bumped to {new_version}") else: + os.system("uv lock") click.echo(f"Bumped version to {new_version}") os.system("git add -u") os.system(f'git commit -m "Release v{new_version}"') diff --git a/src/serena/__init__.py b/src/serena/__init__.py index 00137aad..151d3432 100644 --- a/src/serena/__init__.py +++ b/src/serena/__init__.py @@ -1,4 +1,4 @@ -__version__ = "1.1.0" +__version__ = "1.1.1" import logging diff --git a/src/serena/agent.py b/src/serena/agent.py index 4174e0ae..99f14bd1 100644 --- a/src/serena/agent.py +++ b/src/serena/agent.py @@ -419,7 +419,7 @@ class SerenaAgent: "backend": self._language_backend.value, } try: - requests.get("https://www.oraios-software.de/serena_usage.php", params=params, timeout=1, verify=False) + requests.get("https://oraios-software.de/serena_usage.php", params=params, timeout=1) except Exception as e: log.debug(f"Failed to send usage info: {e}") diff --git a/src/serena/tools/jetbrains_tools.py b/src/serena/tools/jetbrains_tools.py index 6db26e0b..a94d5c99 100644 --- a/src/serena/tools/jetbrains_tools.py +++ b/src/serena/tools/jetbrains_tools.py @@ -512,16 +512,33 @@ class JetBrainsRenameTool(Tool, ToolMarkerSymbolicEdit, ToolMarkerOptional): Renames a symbol, file or directory throughout the codebase using the JetBrains backend. """ - def apply(self, relative_path: str, new_name: str, name_path: str | None = None) -> str: + def apply( + self, + relative_path: str, + new_name: str, + name_path: str | None = None, + rename_in_comments: bool = False, + rename_in_text_occurrences: bool = False, + ) -> str: """ Renames a symbol, file or directory throughout the codebase. + Note: renaming in comments/text is on a best-effort basis by the IDE; if the symbol name is non-unique, further + verification is recommended. :param relative_path: if `name_path` is passed, the relative path of the file containing the symbol. Otherwise, the path to the directory or file to rename. :param new_name: the new name :param name_path: the name path of the symbol to rename or None if renaming a file or directory. + :param rename_in_comments: whether to also rename occurrences in comments. Default True. + :param rename_in_text_occurrences: whether to also rename occurrences in text. Default True. :return: a status message """ code_editor = JetBrainsCodeEditor(self.project) - result = code_editor.rename_symbol(name_path=name_path, relative_path=relative_path, new_name=new_name) + result = code_editor.rename_symbol( + name_path=name_path, + relative_path=relative_path, + new_name=new_name, + rename_in_comments=rename_in_comments, + rename_in_text_occurrences=rename_in_text_occurrences, + ) return self._to_json(result) diff --git a/src/solidlsp/language_servers/dart_language_server.py b/src/solidlsp/language_servers/dart_language_server.py index 11f78106..f82627c6 100644 --- a/src/solidlsp/language_servers/dart_language_server.py +++ b/src/solidlsp/language_servers/dart_language_server.py @@ -123,7 +123,13 @@ class DartLanguageServer(SolidLanguageServer): """ root_uri = pathlib.Path(repository_absolute_path).as_uri() initialize_params = { - "capabilities": {}, + "capabilities": { + "textDocument": { + "documentSymbol": { + "hierarchicalDocumentSymbolSupport": True, + } + } + }, "initializationOptions": { "onlyAnalyzeProjectsWithOpenFiles": False, "closingLabels": False, diff --git a/test/solidlsp/dart/test_dart_basic.py b/test/solidlsp/dart/test_dart_basic.py index 0a5b69cf..7dab1fe9 100644 --- a/test/solidlsp/dart/test_dart_basic.py +++ b/test/solidlsp/dart/test_dart_basic.py @@ -375,3 +375,52 @@ class TestDartLanguageServer: f"Found malformed symbols: {[format_symbol_for_assert(sym) for sym in malformed_symbols]}", pytrace=False, ) + + @pytest.mark.parametrize("language_server", [Language.DART], indirect=True) + def test_symbol_body_contains_full_method(self, language_server: SolidLanguageServer) -> None: + """Test that document symbols return the full method body range, not just the identifier. + + Regression test: when hierarchicalDocumentSymbolSupport was not declared in client + capabilities, the Dart LSP returned SymbolInformation[] (flat format) where + location.range only covered the identifier name (single line), causing find_symbol + with include_body=True to return just the symbol name instead of its implementation. + """ + file_path = os.path.join("lib", "main.dart") + symbols = language_server.request_document_symbols(file_path).get_all_symbols_and_roots() + symbol_list = symbols[0] if symbols and isinstance(symbols[0], list) else symbols + + # Find the 'add' method — defined across multiple lines in main.dart: + # int add(int a, int b) { + # final result = a + b; + # _history.add('$a + $b = $result'); + # return result; + # } + add_symbol = None + for sym in symbol_list: + if sym.get("name") == "add": + add_symbol = sym + break + if sym.get("name") == "Calculator" and "children" in sym: + for child in sym["children"]: + if child.get("name") == "add": + add_symbol = child + break + if add_symbol: + break + + assert add_symbol is not None, "Could not find 'add' method symbol in main.dart" + + # The body range must span multiple lines (not just the identifier line). + # With hierarchicalDocumentSymbolSupport declared, the Dart LSP returns + # DocumentSymbol[] where range covers the full method body. + body_start = add_symbol["location"]["range"]["start"]["line"] + body_end = add_symbol["location"]["range"]["end"]["line"] + assert body_end > body_start, ( + f"Expected multi-line body range for 'add' method, got start={body_start}, end={body_end}. " + f"This likely means hierarchicalDocumentSymbolSupport is not declared in client capabilities." + ) + + # The body text must contain the method implementation, not just the name. + if add_symbol.get("body"): + body_text = add_symbol["body"].get_text() + assert "return result" in body_text, f"Expected method body to contain implementation, got: {body_text!r}" diff --git a/uv.lock b/uv.lock index b0f37c6c..17190a91 100644 --- a/uv.lock +++ b/uv.lock @@ -2911,7 +2911,7 @@ wheels = [ [[package]] name = "serena-agent" -version = "1.1.0" +version = "1.1.1" source = { editable = "." } dependencies = [ { name = "anthropic" },