tiennm99/serena
1
0 Fork 0
mirror of https://github.com/tiennm99/serena.git synced 2026-10-14 00:16:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into better-symbolic-editing

Browse Source
This commit is contained in:
Michael Panchenko committed 2025-06-01 22:35:45 +02:00
2 parents e8d5cefe9d 2ccb7e0e4c
commit 03dc1d62b9
5 files changed
+353 -7
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+23 -1
View File
@@ -14,6 +14,24 @@ Note how Serena's tools enable Claude to find and edit the right symbols.
https://github.com/user-attachments/assets/6eaa9aa1-610d-4723-a2d6-bf1e487ba753
<p align="center">
<em>Serena is under active development! See the latest updates, upcoming features, and lessons learned to stay up to date.</em>
</p>
<p align="center">
<a href="CHANGELOG.md">
<img src="https://img.shields.io/badge/Updates-1e293b?style=flat&logo=rss&logoColor=white&labelColor=1e293b" alt="Changelog" />
</a>
<a href="roadmap.md">
<img src="https://img.shields.io/badge/Roadmap-14532d?style=flat&logo=target&logoColor=white&labelColor=14532d" alt="Roadmap" />
</a>
<a href="lessons_learned.md">
<img src="https://img.shields.io/badge/Lessons-Learned-7c4700?style=flat&logo=readthedocs&logoColor=white&labelColor=7c4700" alt="Lessons Learned" />
</a>
</p>
### LLM Integration
Serena provides the necessary [tools](#full-list-of-tools) for coding workflows, but an LLM is required to do the actual work,
@@ -163,7 +181,11 @@ Serena can be used in various ways, below you will find instructions for selecte
```
You can now add Serena to your MCP client as described below for various clients and
[activate your first project]()
[activate your first project](#project-activation).
> In the default configuration, Serena will start a small dashboard on localhost that will display logs and allow shutting down the
> MCP server (since many clients fail to cleanup processes, leaving zombies behind). If you don't want that, simply set `web_dashboard` to `False`
> in your `serena_config.yml`.
### Configuration
config/modes/editing.yml
+71 -1
View File
@@ -4,4 +4,74 @@ prompt: |
to implement the requested changes to the code base while adhering to the project's code style and patterns.
Use symbolic editing tools whenever possible for precise code modifications.
If no editing task has yet been provided, wait for the user to provide one.
excluded_tools: []
Your primary tool for editing code is a regex-based replacement. You use other tools to find the relevant content and
then use your knowledge of the codebase to write the regex.
You are extremely good at regex, so you never need to check whether the replacement produced the correct result.
In particular, you know what to escape and what not to escape, and you know how to use wildcards.
Moreover, the replacement tool will fail if it can't perform the desired replacement, and this is all the feedback you need.
Your overall goal for replacement operations is to use relatively short regexes, since I want you to minimize the number
of output tokens. For replacements of larger chunks of code, this means you intelligently make use of wildcards for the middle part
and of characteristic snippets for the before/after parts that uniquely identify the chunk.
For small replacements, up to a single line, you follow the following rules:
1. If the snippet to be replaced is likely to be unique within the file, you perform the replacement by directly using the escaped version of the
original.
2. If the snippet is probably not unique, and you want to replace all occurrences, you use the `allow_multiple_occurrences` flag.
3. If the snippet is not unique, and you want to replace a specific occurence, you make use of the code surrounding the snippet
to extend the regex with content before/after such that the regex will have exactly one match.
4. You generally assume that a snippet is unique, knowing that the tool will return an error on multiple matches. You only read more file content
(for crafvarting a more specific regex) if such a failure unexpectedly occurs.
Examples:
1 Small replacement
You have read code like
```python
...
x = linear(x)
x = relu(x)
return x
...
```
and you want to replace `x = relu(x)` with `x = gelu(x)`.
You first try `replace_regex()` with the regex `x = relu\(x\)` and the replacement `x = gelu(x)`.
If this fails due to multiple matches, you will try `(linear\(x\)\s*)x = relu\(x\)(\s*return)` with the replacement `\1x = gelu(x)\2`.
2 Larger replacement
You have read code like
```python
def my_func():
...
# a comment before the snippet
x = add_fifteen(x)
# beginning of long section within my_func
....
# end of long section
call_subroutine(z)
call_second_subroutine(z)
```
and you want to replace the code starting with `x = add_fifteen(x)` until (including) `call_subroutine(z)`, but not `call_second_subroutine(z)`.
Initially, you assume that the the beginning and end of the chunk uniquely determine it within the file.
Therefore, you perform the replacement by using the regex `x = add_fifteen\(x\)\s*.*?call_subroutine\(z\)`
and the replacement being the new code you want to insert.
If this fails due to multiple matches, you will try to extend the regex with the content before/after the snippet and match groups.
The matching regex becomes:
`(before the snippet\s*)x = add_fifteen\(x\)\s*.*?call_subroutine\(z\)`
and the replacement includes the group as (schematically):
`\1<new_code>`
Generally, I remind you that you rely on the regex tool with providing you the correct feedback, no need for more verification!
IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
excluded_tools:
- replace_lines
- insert_at_line
- delete_lines
lessons_learned.md
+82
View File
@@ -0,0 +1,82 @@
# Lessons Learned
In this document we briefly collect what we have learned while developing and using Serena,
what works well and what doesn't.
## What Worked
### Separate Tool Logic From MCP Implementation
MCP is just another protocol, one should let the details of it creep into the application logic.
The official docs suggest using function annotations to define tools and prompts. While that may be
useful for small projects to get going fast, it is not wise for more serious projects. In Serena,
all tools are defined independently and then converted to instances of `MCPTool` using our `make_tool`
function.
### Autogenerated PromptFactory
Prompt templates are central for most LLM applications, so one needs good representations of them in the code,
while at the same time they often need to be customizable and exposed to users. In Serena we address these conflicting
needs by defining prompt templates (in jinja format) in separate yamls that users can easily modify and by autogenerated
a `PromptFactory` class with meaningful method and parameter names from these yamls. The latter is committed to our code.
We separated out the generation logic into the [interprompt](/src/interprompt/README.md) subpackage that can be used as a library.
### Tempfiles and Snapshots for Testing of Editing Tools
We test most aspects of Serena by having a small "project" for each supported language in `tests/resources`.
For the editing tools, which would change the code in these projects, we use tempfiles to copy over the code.
The pretty awesome [syrupy](https://github.com/syrupy-project/syrupy) pytest plugin helped in developing
snapshot tests.
### Dashboard and GUI for Logging
It is very useful to know what the MCP Server is doing. We collect and display logs in a GUI or a web dashboard,
which helps a lot in seeing what's going on and in identifying any issues.
### Unrestricted Bash Tool
We know it's not particularly safe to permit unlimited shell commands outside a sandbox, but we did quite some
evaluations and so far... nothing bad has happened. Seems like the current versions of the AI overlords rarely want to execute `sudo rm - rf /`.
Still, we are working on a safer approach as well as better integration with sandboxing.
### Multilspy
The [multilspy](https://github.com/microsoft/multilspy/) project helped us a lot in getting started and stands at the core of Serena.
Many more well known python implementations of language servers were subpar in code quality and design (for example, missing types).
### Developing Serena with Serena
We clearly notice that the better the tool gets, the easier it is to make it even better
## Prompting
### Shouting and Emotive Language May Be Needed
When developing the `ReplaceRegexTool` we were initially not able to make Claude 4 (in Claude Desktop) use wildcards to save on output tokens. Neither
examples nor explicit instructions helped. It was only after adding
```
IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
```
to the initial instructions and to the tool description that Claude finally started following the instructions.
## What Didn't Work
### Lifespan Handling by MCP Clients
The MCP technology is clearly very green. Even though there is a lifespan context in the MCP SDK,
many clients, including Claude Desktop, fail to properly clean up, leaving zombie processes behind.
We mitigate this through the GUI window and the dashboard, so the user sees whether Serena is running
and can terminate it there.
### Cross-OS Tkinter GUI
Different OS have different limitations when it comes to starting a window or dealing with Tkinter
installations. This was so messy to get right that we pivoted to a web-dashboard instead
### Editing Based on Line Numbers
Not only are LLMs notoriously bad in counting, but also the line numbers change after edit operations,
and LLMs are also often too dumb to understand that they should update the line numbers information they had
received before. We pivoted to string-matching and symbol-name based editing.
roadmap.md
+63
View File
@@ -0,0 +1,63 @@
# Roadmap
This document gives an overview of the ongoing and future development of Serena.
If you have a proposal or want to discuss something, feel free to open a discussion
on Github. For a summary of the past development, see the [changelog](/CHANGELOG.md).
Want to see us reach our goals faster? You can help out with an issue, start a discussion, or
inform us about funding opportunities so that we can devote more time to the project.
## Overall Goals
Serena has the potential to be the go-to tool for most LLM coding tasks, since it is
unique in its ability to be used as MCP Server in any kind of environment
while still being a capable agent. We want to achieve the following goals in terms of functionality:
1. Top performance (comparable to API-based coding agents) when used through official (free) clients like Claude Desktop.
1. Lowering API costs and potentially improving performance of coding clients (Claude Code, Codex, Cline, Roo, Cursor/Windsurf/VSCode etc).
1. Transparency and simplicity of use. Achieved through the dashboard/logging GUI.
1. Integrations with major frameworks that don't accept MCP. Usable as a library.
Apart from the functional goals, we have the goal of having great code design, so that Serena can be viewed
as a reference for how to implement MCP Servers. Such projects are an emerging technology, and
best practices are yet to be determined. We will share our experiences in [lessons learned](/lessons_learned.md).
## Immediate/Ongoing
- Support for projects using multiple programming languages.
- Evaluate whether `ReplaceLinesTool` can be removed in favor of a more reliable and performant editing approach.
- Generally experiment with various approaches to editing tools
- Manual evaluation on selected tasks from SWE-verified
- Manual evaluation of cost-lowering and performance when used within popular non-MCP agents
- Improvements in prompts, in particular giving examples and extending modes and contexts
## Upcoming
- Publishing Serena as a package that can also be used as library
- Use linting and type-hierarchy from the LSP in tools
- Tools for refactoring (rename, move) - speculative, maybe won't do this.
- Tracking edits and rolling them back with the dashboard
- Improve configurability and safety of shell tool. Maybe autogeneration of tools from a list of commands and descriptions.
- Transparent comparison with DesktopCommander and ...
- Automatic evaluation using OpenHands, submission to SWE-Bench
- Evaluation whether incorporating other MCPs increases performance or usability (memory bank is a candidate)
- More documentation and best practices
## Stretch
- Allow for sandboxing and parallel instances of Serena, maybe use openhands or codex for that
- Incorporate a verifier model or generally a second model (maybe for applying edits) as a tool.
- Building on the above, allow for the second model itself to be reachable through an MCP server, so it can be used for free
- Tracking edits performed with shell tools
## Beyond Serena
The technologies and approaches taken in Serena can be used for various research and service ideas. Some thought that we had are:
- PR and issue assistant working with GitHub, similar to how [OpenHands](https://github.com/All-Hands-AI/OpenHands)
and [qodo](https://github.com/qodo-ai/pr-agent) operate. Should be callable through @serena
- Tuning a coding LLM with Serena's tools with RL on one-shot tasks. We would need compute-funding for that
- Develop a web app to quantitatively compare the performance of various agents by scraping PRs and manually crafted metadata.
The main metric for coding agents should be *developer experience*, and that is hard to grasp and is poorly correlated with
performance on current benchmarks.
src/serena/agent.py
+114 -5
View File
@@ -6,6 +6,7 @@ import inspect
import json
import os
import platform
import re
import shutil
import sys
import traceback
@@ -19,6 +20,7 @@ from fnmatch import fnmatch
from functools import cached_property
from logging import Logger
from pathlib import Path
from types import TracebackType
from typing import TYPE_CHECKING, Any, Self, TypeVar, Union
import yaml
@@ -91,6 +93,7 @@ class ProjectConfig(ToStringMixin):
read_only: bool = False
ignore_all_files_in_gitignore: bool = True
initial_prompt: str = ""
encoding: str = "utf-8"
SERENA_DEFAULT_PROJECT_FILE = "project.yml"
@@ -540,7 +543,22 @@ class SerenaAgent:
# activate a project configuration (if provided or if there is only a single project available)
if project is not None:
self.activate_project_from_path_or_name(project)
try:
self.activate_project_from_path_or_name(project)
except Exception as e:
log.error(
f"Error activating project '{project}': {e}; Note that out-of-project configurations were migrated. "
"You should now pass either --project <project_name> or --project <project_root>."
)
def get_project_root(self) -> str:
"""
:return: the root directory of the active project (if any); raises a ValueError if there is no active project
"""
project = self.get_active_project()
if project is None:
raise ValueError("Cannot get project root if no project is active.")
return project.project_root
def get_exposed_tool_instances(self) -> list["Tool"]:
"""
@@ -788,10 +806,7 @@ class Component(ABC):
"""
:return: the root directory of the active project, raises a ValueError if no active project configuration is set
"""
project_config = self.agent.get_active_project()
if project_config is None:
raise ValueError("Cannot get project root if no active project configuration is set.")
return project_config.project_root
return self.agent.get_project_root()
@property
def prompt_factory(self) -> PromptFactory:
@@ -1324,6 +1339,100 @@ class InsertBeforeSymbolTool(Tool, ToolMarkerCanEdit):
return SUCCESS_RESULT
class EditedFileContext:
"""
Context manager for file editing.
Create the context, then use `set_updated_content` to set the new content, the original content
being provided in `original_content`.
When exiting the context without an exception, the updated content will be written back to the file.
"""
def __init__(self, relative_path: str, agent: SerenaAgent):
self._project = agent.get_active_project()
assert self._project is not None
self._abs_path = os.path.join(self._project.project_root, relative_path)
if not os.path.isfile(self._abs_path):
raise FileNotFoundError(f"File {self._abs_path} does not exist.")
with open(self._abs_path, encoding=self._project.project_config.encoding) as f:
self._original_content = f.read()
self._updated_content: str | None = None
def __enter__(self) -> Self:
return self
def get_original_content(self) -> str:
"""
:return: the original content of the file before any modifications.
"""
return self._original_content
def set_updated_content(self, content: str) -> None:
"""
Sets the updated content of the file, which will be written back to the file
when the context is exited without an exception.
:param content: the updated content of the file
"""
self._updated_content = content
def __exit__(self, exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: TracebackType | None) -> None:
if self._updated_content is not None and exc_type is None:
assert self._project is not None
with open(self._abs_path, "w", encoding=self._project.project_config.encoding) as f:
f.write(self._updated_content)
log.info(f"Updated content written to {self._abs_path}")
# Language servers should automatically detect the change and update its state accordingly.
# If they do not, we may have to add a call to notify it.
class ReplaceRegexTool(Tool, ToolMarkerCanEdit):
"""
Replaces content in a file by using regular expressions.
"""
def apply(
self,
relative_path: str,
regex: str,
repl: str,
allow_multiple_occurrences: bool = False,
) -> str:
r"""
Replaces one or more occurrences of the given regular expression.
This is the preferred way to replace content in a file whenever the symbol-level
tools are not appropriate.
Even large sections of code can be replaced by providing a concise regular expression of
the form "beginning.*?end-of-text-to-be-replaced".
Always try to use wildcards to avoid specifying the exact content of the code to be replaced,
especially if it spans several lines.
IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
:param relative_path: the relative path to the file
:param regex: a Python-style regular expression, matches of which will be replaced.
Dot matches all characters, multi-line matching is enabled.
:param repl: the string to replace the matched content with, which may contain
backreferences like \1, \2, etc.
:param allow_multiple_occurrences: if True, the regex may match multiple occurrences in the file
and all of them will be replaced.
If this is set to False and the regex matches multiple occurrences, an error will be returned
(and you may retry with a revised, more specific regex).
"""
with EditedFileContext(relative_path, self.agent) as context:
original_content = context.get_original_content()
updated_content, n = re.subn(regex, repl, original_content, flags=re.DOTALL | re.MULTILINE)
if n == 0:
return f"Error: No matches found for regex '{regex}' in file '{relative_path}'."
if not allow_multiple_occurrences and n > 1:
return (
f"Error: Regex '{regex}' matches {n} occurrences in file '{relative_path}'. "
"Please revise the regex to be more specific or enable allow_multiple_occurrences if this is expected."
)
context.set_updated_content(updated_content)
return SUCCESS_RESULT
class DeleteLinesTool(Tool, ToolMarkerCanEdit):
"""
Deletes a range of lines within a file.