tiennm99/serena
1
0 Fork 0
mirror of https://github.com/tiennm99/serena.git synced 2026-10-13 08:15:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feature/context-and-mode

Browse Source 
# Conflicts:
#	scripts/gen_prompt_factory.py
#	src/serena/agent.py
#	src/serena/llm/multilang_prompt.py
This commit is contained in:
Michael Panchenko committed 2025-05-23 17:42:37 +02:00
2 parents 3584ce47eb 8ed8fce620
commit f85aabb40e
79 files changed
+2658 -1582
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.github/workflows/pytest.yml
+44 -16
View File
@@ -1,13 +1,24 @@
name: Tests on Ubuntu
name: Tests on CI
on: [push, pull_request]
on:
pull_request:
push:
branches:
- main
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
cpu:
runs-on: ubuntu-latest
name: Tests on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
if: "!contains(github.event.head_commit.message, 'ci skip')"
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.11"]
steps:
- uses: actions/checkout@v3
@@ -15,22 +26,39 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: "${{ matrix.python-version }}"
# Include the appropriate package manager steps based on the selection
- uses: actions/setup-go@v5
with:
go-version: '>=1.17.0'
# Add Go bin directory to PATH for this workflow
# GITHUB_PATH is a special file that GitHub Actions uses to modify PATH
# Writing to this file adds the directory to the PATH for subsequent steps
- name: Install gopls
shell: bash
run: go install golang.org/x/tools/gopls@latest
- name: Install uv
shell: bash
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Cache uv virtualenv
id: cache-uv
uses: actions/cache@v3
with:
path: .venv
key: uv-venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('uv.lock') }}
- name: Create virtual environment
run: uv venv
shell: bash
run: |
if [ ! -d ".venv" ]; then
uv venv
fi
- name: Install dependencies
shell: bash
run: uv pip install -e ".[dev]"
- name: Test with pytest
run: uv run poe test
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
with:
token: ${{ secrets.CODECOV }}
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
shell: bash
# Currently java Language Server is not working on macos
run: |
if [ "${{ matrix.os }}" = "macos-latest" ]; then
uv run pytest -m "not java"
else
uv run poe test
fi
.gitignore
+1 -1
View File
@@ -138,7 +138,7 @@ pylint.html
# Serena-specific
/*.yml
!myproject.demo.yml
!*.template.yml
/agent-ui
# dynamic LS installations
/src/multilspy/language_servers/*/static
.vscode/settings.json
Vendored
+1 -1
View File
@@ -10,5 +10,5 @@
"pixi",
"sensai",
"vibing"
]
],
}
CHANGELOG.md
+14 -1
View File
@@ -1,7 +1,19 @@
# Latest
Changes prior to the next official version change will appear here.
# 2025-05-21
**Signficant improvement in symbol finding!**
* Serena core:
* `FindSymbolTool` now can look for symbols by specifying paths to them, not just the symbol name
* Language Servers:
* Fixed `gopls` initialization
* Symbols retrieved through the symbol tree or through overview methods now are linked to their parents
# 2025-05-19
* Serena core:
* Bugfix in `FindSymbolTool` (a bug fixed in LS)
* Fix in `ListDirTool`: Do not ignore files with extensions not understood by the language server, only skip ignored directories
@@ -17,6 +29,7 @@ Changes prior to the next official version change will appear here.
* Language Servers:
* Add further file extensions considered by the language servers for Python (.pyi), JavaScript (.jsx) and TypeScript (.tsx, .jsx)
* Updated multilspy, adding support for Kotlin, Dart and C/C++ and several improvements.
* Added support for PHP
# 2025-04-07
CONTRIBUTING.md
+7
View File
@@ -45,6 +45,13 @@ for you.
Note: For the Windows subsystem for Linux (WSL), you may need to adjust the path for the
volume.
## Running Tools Locally
The Serena tools (and in fact all Serena code) can be executed without an LLM, and also without
any MCP specifics (though you can use the mcp inspector, if you want).
An example script for running tools is provided in [scripts/demo_run_tools.py](scripts/demo_run_tools.py).
## Adding a New Supported Language
Serena interacts with code through language servers which are included in
LICENSE
+17 -335
View File
@@ -1,339 +1,21 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
MIT License
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Copyright (c) 2025 Oraios AI
Preamble
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
+39 -9
View File
@@ -48,16 +48,17 @@ Language servers provide support for a wide range of programming languages.
With Serena, we provide
* direct, out-of-the-box support for:
* Python
* Java (_Note_: startup is slow, initial startup especially so)
* TypeScript
* TypeScript/Javascript
* PhP
* Go (need to install go and gopls first)
* Rust
* C/C++
* Java (_Note_: startup is slow, initial startup especially so. There seem to be issues with java on macos)
* indirect support (may require some code changes/manual installation) for:
* Ruby (untested)
* Go (untested)
* C# (untested)
* Rust (untested)
* Kotlin (untested)
* Dart (untested)
* C/C++ (untested)
These languages are supported by the language server library [multilspy](https://github.com/microsoft/multilspy), which Serena uses under the hood.
But we did not explicitly test whether the support for these languages actually works.
@@ -150,9 +151,17 @@ Serena can be used in various ways, below you will find instructions for selecte
1. Install `uv` (instructions [here](https://docs.astral.sh/uv/getting-started/installation/))
2. Clone the repository to `/path/to/serena`.
3. Copy `serena_config.template.yml` to `serena_config.yml` and adjust settings.
4. Copy `myproject.template.yml` to `myproject.yml` and adjust the settings specific to your project.
(Add one such file for each project you want Serena to work on.)
5. If you want Serena to dynamically switch between projects, add the list of all project files
```shell
cp serena_config.template.yml serena_config.yml
```
4. Copy `project.template.yml` to `project.yml` and adjust the settings specific to your project
(add one such file for each project you want Serena to work on). We recommend that you copy
it to the `.serena` directory of your project, e.g.,
```shell
mkdir -p /myproject/.serena
cp project.template.yml /myproject/.serena/project.yml
```
6. If you want Serena to dynamically switch between projects, add the list of all project files
created in the previous step to the `projects` list in `serena_config.yml`.
> ⚠️ **Note:** Serena is under active development. We are continuously adding features, improving stability and the UX.
@@ -188,7 +197,17 @@ want to use Serena.
If you are using paths containing backslashes for paths on Windows
(note that you can also just use forward slashes), be sure to escape them correctly (`\\`).
That's it! Save the config and then restart Claude Desktop.
That's it! Save the config and then restart Claude Desktop.
#### Troubleshooting
Some client/OS/setup configurations were reported to cause issues when using Serena with the standard `stdio` protocol, where the MCP server is started by the client application.
If you experience such problems, you can start Serena in `sse` mode by running, e.g.,
```shell
uv run --directory /path/to/serena serena-mcp-server --transport sse --port 9121 --project-file /path/to/project.yml
```
(the `--project-file` option is optional). Then configure your client to connect to `http://localhost:9121`.
Note: on Windows and macOS there are official Claude Desktop applications by Anthropic, for Linux there is an [open-source
community version](https://github.com/aaddrick/claude-desktop-debian).
@@ -211,6 +230,17 @@ therefore needs to be configured with a launch command.
For more information on MCP servers with Claude Desktop, see [the official quick start guide](https://modelcontextprotocol.io/quickstart/user).
### Claude Code
Serena is a great way to make Claude Code both cheaper and more powerful! We are collecting
several examples for that and have heard very positive feedback so far. Claude Code users can
add serena with
```shell
claude mcp add serena -- /path/to/uv "run" --directory /path/to/serena serena-mcp-server --project-file /path/to/project.yml
```
### Other MCP Clients - Cline, Roo-Code, Cursor, Windsurf etc.
Being an MCP Server, Serena can be included in any MCP Client. The same config as above,
myproject.template.yml → project.template.yml
+1 -1
View File
@@ -2,7 +2,7 @@
# This is optional if this file is placed in the project directory under `.serena/project.yml`.
project_root: /path/to/project
# language of the project (csharp, python, rust, java, typescript, javascript, go, or ruby)
# language of the project (csharp, python, rust, java, typescript, javascript, go, cpp, or ruby)
# Special requirements:
# * csharp: Requires the presence of a .sln file in the project folder.
language: python
pyproject.toml
+12 -26
View File
@@ -40,7 +40,7 @@ dependencies = [
serena-mcp-server = "serena.mcp:start_mcp_server"
[project.license]
text = "GPL-2.0"
text = "MIT"
[project.optional-dependencies]
dev = [
@@ -50,7 +50,6 @@ dev = [
"mypy>=1.4.1",
"poethepoet>=0.20.0",
"pytest>=8.0.2",
"pytest-cov",
"ruff>=0.0.285",
"toml-sort>=0.24.2",
"types-pyyaml>=6.0.12.20241230",
@@ -106,7 +105,7 @@ exclude = "^build/|^docs/"
PYDEVD_DISABLE_FILE_VALIDATION = "1"
[tool.poe.tasks]
test = "pytest test --cov=serena --cov-report=xml --cov-report=term-missing --durations=0 -v --color=yes"
test = "pytest test --color=yes"
_black_check = "black --check --exclude src/multilspy/ src scripts test"
_ruff_check = "ruff check --exclude .venv/ --exclude src/multilspy/ src scripts test"
_black_format = "black --exclude .venv/|src/multilspy/ src scripts test"
@@ -115,27 +114,10 @@ lint = [
"_black_check",
"_ruff_check",
]
clean-nbs = "python docs/nbstripout.py"
format = [
"_ruff_format",
"_black_format"
]
_autogen_rst = "python docs/autogen_rst.py"
_sphinx_build = "sphinx-build -W -b html docs docs/_build"
_jb_generate_toc = "python docs/create_toc.py"
_jb_generate_config = "jupyter-book config sphinx docs/"
doc-clean = "rm -rf docs/_build"
doc-generate-files = [
"_autogen_rst",
"_jb_generate_toc",
"_jb_generate_config"
]
doc-spellcheck = "sphinx-build -W -b spelling docs docs/_build"
doc-build = [
"doc-generate-files",
"doc-spellcheck",
"_sphinx_build"
]
_mypy = "mypy src/serena"
type-check = [
"_mypy",
@@ -262,12 +244,16 @@ max-complexity = 20
"tests/**" = [
"D103"
]
"docs/**" = [
"D103"
]
"examples/**" = [
"D103"
]
"scripts/**" = [
"D103"
]
[tool.pytest.ini_options]
markers = [
"python: language server running for Python",
"go: language server running for Go",
"java: language server running for Java",
"rust: language server running for Rust",
"typescript: language server running for TypeScript",
"php: language server running for PHP",
]
scripts/demo_run_tools.py
+26
View File
@@ -0,0 +1,26 @@
"""
This script demonstrates how to use Serena's tools locally, useful
for testing or development. Here the config is pointing to the serena repo itself,
it assumes that you already have a serena_config.yml file setup.
"""
import json
from pathlib import Path
from pprint import pprint
from serena.agent import *
project_root = Path(__file__).parent.parent
if __name__ == "__main__":
agent = SerenaAgent(project_config=str(project_root / ".serena" / "project.yml"))
overview_tool = agent.get_tool(GetSymbolsOverviewTool)
find_symbol_tool = agent.get_tool(FindSymbolTool)
print("Getting an overview of the util package\n")
pprint(json.loads(overview_tool.apply("src/serena/util")))
print("\n\n")
print("Finding the symbol 'SerenaAgent'\n")
pprint(json.loads(find_symbol_tool.apply("SerenaAgent")))
scripts/mcp_server.py
+2 -4
View File
@@ -1,6 +1,4 @@
from serena.mcp import create_mcp_server
mcp = create_mcp_server()
from serena.mcp import start_mcp_server
if __name__ == "__main__":
mcp.run()
start_mcp_server()
src/multilspy/language_server.py
+146 -100
View File
@@ -14,35 +14,31 @@ import os
import pathlib
import pickle
import re
from site import abs_paths
import threading
from collections import defaultdict
from contextlib import asynccontextmanager, contextmanager
from copy import copy
from typing import Dict, List, Optional, Tuple, Union
from fnmatch import fnmatch
from pathlib import Path, PurePath
from typing import AsyncIterator, Dict, Iterator, List, Optional, Tuple, Union, cast
import pathspec
from serena.text_utils import LineType, MatchedConsecutiveLines, TextLine, search_files, search_text
from serena.text_utils import LineType, MatchedConsecutiveLines, TextLine, search_files
from . import multilspy_types
from .lsp_protocol_handler import lsp_types as LSPTypes
from .lsp_protocol_handler.lsp_constants import LSPConstants
from .lsp_protocol_handler.lsp_types import SymbolKind
from .lsp_protocol_handler.lsp_types import Definition, DefinitionParams, LocationLink, SymbolKind
from .lsp_protocol_handler.server import (
Error,
LanguageServerHandler,
ProcessLaunchInfo,
)
from .multilspy_config import Language, MultilspyConfig
from .multilspy_exceptions import MultilspyException
from .multilspy_utils import PathUtils, FileUtils, TextUtils
from pathlib import PurePath
from typing import AsyncIterator, Iterator, List, Dict, Optional, Union, Tuple
from .multilspy_logger import MultilspyLogger
from .multilspy_utils import FileUtils, PathUtils, TextUtils
from .type_helpers import ensure_all_methods_implemented
from .lsp_protocol_handler import lsp_types
# Serena dependencies
# We will need to watch out for circular imports, but it's probably better to not
@@ -128,7 +124,7 @@ class LanguageServer:
raise ValueError(
f"Asked to add gitignore content to the config for {repository_root_path=} but there already is a non-empty entry"
)
with open(gitignore_path) as f:
with open(gitignore_path, "r", encoding="utf-8") as f:
gitignore_file_content = f.read()
config.gitignore_file_content = gitignore_file_content
@@ -188,6 +184,10 @@ class LanguageServer:
from multilspy.language_servers.clangd_language_server.clangd_language_server import ClangdLanguageServer
return ClangdLanguageServer(config, logger, repository_root_path)
elif config.code_language == Language.PHP:
from multilspy.language_servers.intelephense.intelephense import Intelephense
return Intelephense(config, logger, repository_root_path)
else:
logger.log(f"Language {config.code_language} is not supported", logging.ERROR)
raise MultilspyException(f"Language {config.code_language} is not supported")
@@ -496,6 +496,9 @@ class LanguageServer:
)
return deleted_text
async def _send_definition_request(self, definition_params: DefinitionParams) -> Union[Definition, List[LocationLink], None]:
return await self.server.send.definition(definition_params)
async def request_definition(
self, relative_file_path: str, line: int, column: int
) -> List[multilspy_types.Location]:
@@ -519,19 +522,18 @@ class LanguageServer:
with self.open_file(relative_file_path):
# sending request to the language server and waiting for response
response = await self.server.send.definition(
{
LSPConstants.TEXT_DOCUMENT: {
LSPConstants.URI: pathlib.Path(
str(PurePath(self.repository_root_path, relative_file_path))
).as_uri()
},
LSPConstants.POSITION: {
LSPConstants.LINE: line,
LSPConstants.CHARACTER: column,
},
}
)
definition_params = cast(DefinitionParams, {
LSPConstants.TEXT_DOCUMENT: {
LSPConstants.URI: pathlib.Path(
str(PurePath(self.repository_root_path, relative_file_path))
).as_uri()
},
LSPConstants.POSITION: {
LSPConstants.LINE: line,
LSPConstants.CHARACTER: column,
},
})
response = await self._send_definition_request(definition_params)
ret: List[multilspy_types.Location] = []
if isinstance(response, list):
@@ -580,6 +582,16 @@ class LanguageServer:
return ret
# Some LS cause problems with this, so the call is isolated from the rest to allow overriding in subclasses
async def _send_references_request(self, relative_file_path: str, line: int, column: int) -> List[lsp_types.Location] | None:
return await self.server.send.references(
{
"textDocument": {"uri": PathUtils.path_to_uri(os.path.join(self.repository_root_path, relative_file_path))},
"position": {"line": line, "character": column},
"context": {"includeDeclaration": False},
}
)
async def request_references(
self, relative_file_path: str, line: int, column: int
) -> List[multilspy_types.Location]:
@@ -594,7 +606,7 @@ class LanguageServer:
:return: A list of locations where the symbol is referenced (excluding ignored directories)
"""
if not self.server_started:
self.logger.log(
"request_references called before Language Server started",
@@ -604,21 +616,23 @@ class LanguageServer:
with self.open_file(relative_file_path):
# sending request to the language server and waiting for response
response = await self.server.send.references(
{
"context": {"includeDeclaration": False},
"textDocument": {
"uri": pathlib.Path(os.path.join(self.repository_root_path, relative_file_path)).as_uri()
},
"position": {"line": line, "character": column},
}
)
try:
response = await self._send_references_request(relative_file_path, line=line, column=column)
except Exception as e:
# Catch LSP internal error (-32603) and raise a more informative exception
if isinstance(e, Error) and getattr(e, 'code', None) == -32603:
raise RuntimeError(
f"LSP internal error (-32603) when requesting references for {relative_file_path}:{line}:{column}. "
"This often occurs when requesting references for a symbol not referenced in the expected way. "
) from e
raise
if response is None:
return []
ret: List[multilspy_types.Location] = []
assert isinstance(response, list), f"Unexpected response from Language Server: {response}"
assert isinstance(response, list), f"Unexpected response from Language Server (expected list, got {type(response)}): {response}"
for item in response:
assert isinstance(item, dict)
assert isinstance(item, dict), f"Unexpected response from Language Server (expected dict, got {type(item)}): {item}"
assert LSPConstants.URI in item
assert LSPConstants.RANGE in item
@@ -793,8 +807,11 @@ class LanguageServer:
:param relative_file_path: The relative path of the file that has the symbols
:param include_body: whether to include the body of the symbols in the result.
:return: A list of symbols in the file, and a list of root symbols that represent the tree structure of the symbols.
Each symbol in hierarchy starting from the roots has a children attribute.
All symbols will have a location and a children attribute.
All symbols will have a location, a children, and a parent attribute,
where the parent attribute is None for root symbols.
Note that this is slightly different from the call to request_full_symbol_tree,
where the parent attribute will be the file symbol which in turn may have a package symbol as parent.
If you need a symbol tree that contains file symbols as well, you should use `request_full_symbol_tree` instead.
"""
self.logger.log(f"Requesting document symbols for {relative_file_path} for the first time", logging.DEBUG)
# TODO: it's kinda dumb to not use the cache if include_body is False after include_body was True once
@@ -847,24 +864,32 @@ class LanguageServer:
item["selectionRange"] = item["range"]
else:
item["selectionRange"] = item["location"]["range"]
item[LSPConstants.CHILDREN] = item.get(LSPConstants.CHILDREN, [])
children = item.get(LSPConstants.CHILDREN, [])
for child in children:
child["parent"] = item
item[LSPConstants.CHILDREN] = children
flat_all_symbol_list: List[multilspy_types.UnifiedSymbolInformation] = []
assert isinstance(response, list), f"Unexpected response from Language Server: {response}"
root_nodes: List[multilspy_types.UnifiedSymbolInformation] = []
for item in response:
if "range" not in item and "location" not in item:
if item["kind"] in [SymbolKind.File, SymbolKind.Module]:
for root_item in response:
if "range" not in root_item and "location" not in root_item:
if root_item["kind"] in [SymbolKind.File, SymbolKind.Module]:
...
turn_item_into_symbol_with_children(item)
item = cast(multilspy_types.UnifiedSymbolInformation, item)
root_nodes.append(item)
assert isinstance(item, dict)
assert LSPConstants.NAME in item
assert LSPConstants.KIND in item
# mutation is more convenient than creating a new dict,
# so we cast and rename the var after the mutating call to turn_item_into_symbol_with_children
# which turned and item into a "symbol"
turn_item_into_symbol_with_children(root_item)
root_symbol = cast(multilspy_types.UnifiedSymbolInformation, root_item)
root_symbol["parent"] = None
root_nodes.append(root_symbol)
assert isinstance(root_symbol, dict)
assert LSPConstants.NAME in root_symbol
assert LSPConstants.KIND in root_symbol
if LSPConstants.CHILDREN in item:
if LSPConstants.CHILDREN in root_symbol:
# TODO: l_tree should be a list of TreeRepr. Define the following function to return TreeRepr as well
def visit_tree_nodes_and_build_tree_repr(node: GenericDocumentSymbol) -> List[multilspy_types.UnifiedSymbolInformation]:
@@ -878,9 +903,9 @@ class LanguageServer:
l.extend(visit_tree_nodes_and_build_tree_repr(child))
return l
flat_all_symbol_list.extend(visit_tree_nodes_and_build_tree_repr(item))
flat_all_symbol_list.extend(visit_tree_nodes_and_build_tree_repr(root_symbol))
else:
flat_all_symbol_list.append(multilspy_types.UnifiedSymbolInformation(**item))
flat_all_symbol_list.append(multilspy_types.UnifiedSymbolInformation(**root_symbol))
result = flat_all_symbol_list, root_nodes
self.logger.log(f"Caching document symbols for {relative_file_path}", logging.DEBUG)
@@ -890,17 +915,19 @@ class LanguageServer:
async def request_full_symbol_tree(self, within_relative_path: str | None = None, include_body: bool = False) -> List[multilspy_types.UnifiedSymbolInformation]:
"""
Will go through all files in the project and build a tree of symbols. Note: this may be slow the first time it is called.
Will go through all files in the project or within a relative path and build a tree of symbols.
Note: this may be slow the first time it is called, especially if `within_relative_path` is not used to restrict the search.
For each file, a symbol of kind Module (3) will be created. For directories, a symbol of kind Package (4) will be created.
For each file, a symbol of kind File (2) will be created. For directories, a symbol of kind Package (4) will be created.
All symbols will have a children attribute, thereby representing the tree structure of all symbols in the project
that are within the repository.
All symbols except the root packages will have a parent attribute.
Will ignore directories starting with '.', language-specific defaults
and user-configured directories (e.g. from .gitignore).
:param within_relative_path: pass a relative path to only consider symbols within this path.
If a file is passed, only the symbols within this file will be considered.
If a directory is passed, all files within this directory will be considered.
If a file is passed, only the symbols within this file will be considered.
If a directory is passed, all files within this directory will be considered.
:param include_body: whether to include the body of the symbols in the result.
:return: A list of root symbols representing the top-level packages/modules in the project.
@@ -919,17 +946,17 @@ class LanguageServer:
return root_nodes
# Helper function to recursively process directories
async def process_directory(dir_path: str) -> List[multilspy_types.UnifiedSymbolInformation]:
abs_dir_path = self.repository_root_path if dir_path == "." else os.path.join(self.repository_root_path, dir_path)
async def process_directory(rel_dir_path: str) -> List[multilspy_types.UnifiedSymbolInformation]:
abs_dir_path = self.repository_root_path if rel_dir_path == "." else os.path.join(self.repository_root_path, rel_dir_path)
abs_dir_path = os.path.realpath(abs_dir_path)
if self.is_ignored_path(str(Path(abs_dir_path).relative_to(self.repository_root_path))):
self.logger.log(f"Skipping directory: {dir_path}\n(because it should be ignored)", logging.DEBUG)
self.logger.log(f"Skipping directory: {rel_dir_path}\n(because it should be ignored)", logging.DEBUG)
return []
result = []
try:
items = os.listdir(abs_dir_path)
contained_dir_or_file_names = os.listdir(abs_dir_path)
except OSError:
return []
@@ -947,20 +974,45 @@ class LanguageServer:
)
result.append(package_symbol)
for item in items:
item_path = os.path.join(abs_dir_path, item)
abs_item_path = os.path.join(self.repository_root_path, item_path)
rel_item_path = str(Path(abs_item_path).resolve().relative_to(self.repository_root_path))
if self.is_ignored_path(rel_item_path):
self.logger.log(f"Skipping item: {rel_item_path}\n(because it should be ignored)", logging.DEBUG)
for contained_dir_or_file_name in contained_dir_or_file_names:
contained_dir_or_file_abs_path = os.path.join(abs_dir_path, contained_dir_or_file_name)
contained_dir_or_file_rel_path = str(Path(contained_dir_or_file_abs_path).resolve().relative_to(self.repository_root_path))
if self.is_ignored_path(contained_dir_or_file_rel_path):
self.logger.log(f"Skipping item: {contained_dir_or_file_rel_path}\n(because it should be ignored)", logging.DEBUG)
continue
if os.path.isdir(abs_item_path):
child_symbols = await process_directory(item_path)
if os.path.isdir(contained_dir_or_file_abs_path):
child_symbols = await process_directory(contained_dir_or_file_rel_path)
package_symbol["children"].extend(child_symbols)
for child in child_symbols:
child["parent"] = package_symbol
elif os.path.isfile(contained_dir_or_file_abs_path):
_, file_root_nodes = await self.request_document_symbols(contained_dir_or_file_rel_path, include_body=include_body)
# Create file symbol, link with children
file_rel_path = str(Path(contained_dir_or_file_abs_path).resolve().relative_to(self.repository_root_path))
with self.open_file(file_rel_path) as file_data:
fileRange = self._get_range_from_file_content(file_data.contents)
file_symbol = multilspy_types.UnifiedSymbolInformation( # type: ignore
name=os.path.splitext(contained_dir_or_file_name)[0],
kind=multilspy_types.SymbolKind.File,
range=fileRange,
selectionRange=fileRange,
location=multilspy_types.Location(
uri=str(pathlib.Path(contained_dir_or_file_abs_path).as_uri()),
range=fileRange,
absolutePath=str(contained_dir_or_file_abs_path),
relativePath=str(Path(contained_dir_or_file_abs_path).resolve().relative_to(self.repository_root_path)),
),
children=file_root_nodes,
parent=package_symbol,
)
for child in file_root_nodes:
child["parent"] = file_symbol
elif os.path.isfile(abs_item_path):
_, root_nodes = await self.request_document_symbols(item_path, include_body=include_body)
# Link file symbol with package
package_symbol["children"].append(file_symbol)
# TODO: Not sure if this is actually still needed given recent changes to relative path handling
def fix_relative_path(nodes: List[multilspy_types.UnifiedSymbolInformation]):
@@ -976,33 +1028,13 @@ class LanguageServer:
if "children" in node:
fix_relative_path(node["children"])
fix_relative_path(root_nodes)
# Create file symbol
file_rel_path = str(Path(abs_item_path).resolve().relative_to(self.repository_root_path))
with self.open_file(file_rel_path) as file_data:
fileRange = self._get_range_from_file_content(file_data.contents)
file_symbol = multilspy_types.UnifiedSymbolInformation( # type: ignore
name=os.path.splitext(item)[0],
kind=multilspy_types.SymbolKind.File,
range=fileRange,
selectionRange=fileRange,
location=multilspy_types.Location(
uri=str(pathlib.Path(abs_item_path).as_uri()),
range=fileRange,
absolutePath=str(abs_item_path),
relativePath=str(Path(abs_item_path).resolve().relative_to(self.repository_root_path)),
),
children=root_nodes
)
package_symbol["children"].append(file_symbol)
fix_relative_path(file_root_nodes)
return result
# Start from the root or the specified directory
start_path = within_relative_path or "."
return await process_directory(start_path)
start_rel_path = within_relative_path or "."
return await process_directory(start_rel_path)
@staticmethod
def _get_range_from_file_content(file_content: str) -> multilspy_types.Range:
@@ -1444,11 +1476,11 @@ class LanguageServer:
column_condition = True
if strict:
line_condition = end["line"] >= line > start["line"]
if column is not None:
if column is not None and line == start["line"]:
column_condition = column > start["character"]
else:
line_condition = end["line"] >= line >= start["line"]
if column is not None:
if column is not None and line == start["line"]:
column_condition = column >= start["character"]
return line_condition and column_condition
@@ -1483,12 +1515,16 @@ class LanguageServer:
async def request_container_of_symbol(self, symbol: multilspy_types.UnifiedSymbolInformation, include_body: bool = False) -> multilspy_types.UnifiedSymbolInformation | None:
"""
Finds the container of the given symbol if there is one.
Finds the container of the given symbol if there is one. If the parent attribute is present, the parent is returned
without further searching.
:param symbol: The symbol to find the container of.
:param include_body: whether to include the body of the symbol in the result.
:return: The container of the given symbol or None if no container is found.
"""
assert "location" in symbol
if "parent" in symbol:
return symbol["parent"]
assert "location" in symbol, f"Symbol {symbol} has no location and no parent attribute"
return await self.request_containing_symbol(
symbol["location"]["relativePath"],
symbol["location"]["range"]["start"]["line"],
@@ -1543,7 +1579,7 @@ class LanguageServer:
@property
def _cache_path(self) -> Path:
return Path(self.repository_root_path) / ".serena" / "cache" / "document_symbols_cache.pkl"
return Path(self.repository_root_path) / ".serena" / "cache" / "document_symbols_cache_v20-05-25.pkl"
def save_cache(self):
if self._cache_has_changed:
@@ -1581,7 +1617,7 @@ class LanguageServer:
:param query: The query string to filter symbols by
:return Union[List[multilspy_types.UnifiedSymbolInformation], None]: A list of matching symbols
:return: A list of matching symbols
"""
response = await self.server.send.workspace_symbol({"query": query})
if response is None:
@@ -1720,11 +1756,21 @@ class SyncLanguageServer:
:return List[multilspy_types.Location]: A list of locations where the symbol is referenced
"""
result = asyncio.run_coroutine_threadsafe(
self.language_server.request_references(file_path, line, column), self.loop
).result(timeout=self.timeout)
try:
result = asyncio.run_coroutine_threadsafe(
self.language_server.request_references(file_path, line, column), self.loop
).result(timeout=self.timeout)
except Exception as e:
from multilspy.lsp_protocol_handler.server import Error
if isinstance(e, Error) and getattr(e, 'code', None) == -32603:
raise RuntimeError(
f"LSP internal error (-32603) when requesting references for {file_path}:{line}:{column}. "
"This often occurs when requesting references for a symbol not referenced in the expected way. "
) from e
raise
return result
def request_references_with_content(
self, relative_file_path: str, line: int, column: int, context_lines_before: int = 0, context_lines_after: int = 0
) -> List[MatchedConsecutiveLines]:
src/multilspy/language_servers/eclipse_jdtls/eclipse_jdtls.py
+2 -2
View File
@@ -165,7 +165,7 @@ class EclipseJDTLS(LanguageServer):
"""
platformId = PlatformUtils.get_platform_id()
with open(str(PurePath(os.path.dirname(__file__), "runtime_dependencies.json")), "r") as f:
with open(str(PurePath(os.path.dirname(__file__), "runtime_dependencies.json")), "r", encoding="utf-8") as f:
runtimeDependencies = json.load(f)
del runtimeDependencies["_description"]
@@ -264,7 +264,7 @@ class EclipseJDTLS(LanguageServer):
Returns the initialize parameters for the EclipseJDTLS server.
"""
# Look into https://github.com/eclipse/eclipse.jdt.ls/blob/master/org.eclipse.jdt.ls.core/src/org/eclipse/jdt/ls/core/internal/preferences/Preferences.java to understand all the options available
with open(str(PurePath(os.path.dirname(__file__), "initialize_params.json")), "r") as f:
with open(str(PurePath(os.path.dirname(__file__), "initialize_params.json")), "r", encoding="utf-8") as f:
d: InitializeParams = json.load(f)
del d["_description"]
src/multilspy/language_servers/gopls/gopls.py
+12 -16
View File
@@ -5,13 +5,16 @@ import os
import pathlib
import subprocess
from contextlib import asynccontextmanager
from typing import AsyncIterator
from typing import AsyncIterator, List
from overrides import override
from multilspy import multilspy_types
from multilspy.lsp_protocol_handler import lsp_types
from multilspy.multilspy_exceptions import MultilspyException
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.server import Error, ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.multilspy_config import MultilspyConfig
@@ -57,28 +60,21 @@ class Gopls(LanguageServer):
Check if required Go runtime dependencies are available.
Raises RuntimeError with helpful message if dependencies are missing.
"""
missing_deps = []
# Check for Go installation
go_version = cls._get_go_version()
if not go_version:
missing_deps.append(("Go", "https://golang.org/doc/install"))
raise RuntimeError("Go is not installed. Please install Go from https://golang.org/doc/install and make sure it is added to your PATH.")
# Check for gopls
gopls_version = cls._get_gopls_version()
if not gopls_version:
missing_deps.append(("gopls", "https://pkg.go.dev/golang.org/x/tools/gopls#section-readme"))
if missing_deps:
error_msg = "Missing required dependencies:\n"
for dep, install_url in missing_deps:
error_msg += f"- {dep}: Please install from {install_url}\n"
raise RuntimeError(error_msg)
raise RuntimeError(
"Found a Go version but gopls is not installed.\n"
"Please install gopls as described in https://pkg.go.dev/golang.org/x/tools/gopls#section-readme\n\n"
"After installation, make sure it is added to your PATH (it might be installed in a different location than Go)."
)
return True
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
# Check runtime dependencies before initializing
self.setup_runtime_dependency()
super().__init__(
@@ -95,7 +91,7 @@ class Gopls(LanguageServer):
"""
Returns the initialize params for the TypeScript Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
src/multilspy/language_servers/gopls/initialize_params.json
+10
View File
@@ -18,6 +18,16 @@
},
"definition": {
"dynamicRegistration": true
},
"documentSymbol": {
"dynamicRegistration": true,
"hierarchicalDocumentSymbolSupport": true,
"symbolKind": {
"valueSet": [
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13,
14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
]
}
}
},
"workspace": {
src/multilspy/language_servers/intelephense/initialize_params.json
+36
View File
@@ -0,0 +1,36 @@
{
"_description": "The parameters sent by the client when initializing the language server with the \"initialize\" request. More details at https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#initialize",
"processId": "os.getpid()",
"locale": "en",
"rootPath": "$rootPath",
"rootUri": "$rootUri",
"capabilities": {
"textDocument": {
"synchronization": {
"didSave": true,
"dynamicRegistration": true
},
"completion": {
"dynamicRegistration": true,
"completionItem": {
"snippetSupport": true
}
},
"definition": {
"dynamicRegistration": true
}
},
"workspace": {
"workspaceFolders": true,
"didChangeConfiguration": {
"dynamicRegistration": true
}
}
},
"workspaceFolders": [
{
"uri": "$uri",
"name": "$name"
}
]
}
src/multilspy/language_servers/intelephense/intelephense.py
+203
View File
@@ -0,0 +1,203 @@
"""
Provides PHP specific instantiation of the LanguageServer class using Intelephense.
"""
import asyncio
import json
import shutil
import logging
import os
import subprocess
import pathlib
from contextlib import asynccontextmanager
from time import sleep
from typing import AsyncIterator
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import DefinitionParams, InitializeParams
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import PlatformUtils, PlatformId
class Intelephense(LanguageServer):
"""
Provides PHP specific instantiation of the LanguageServer class using Intelephense.
"""
@override
def is_ignored_dirname(self, dirname: str) -> bool:
# For PHP projects, we should ignore:
# - vendor: third-party dependencies managed by Composer
# - node_modules: if the project has JavaScript components
# - cache: commonly used for caching
return super().is_ignored_dirname(dirname) or dirname in ["node_modules", "vendor", "cache"]
def setup_runtime_dependencies(self, logger: MultilspyLogger, config: MultilspyConfig) -> str:
"""
Setup runtime dependencies for Intelephense.
"""
platform_id = PlatformUtils.get_platform_id()
valid_platforms = [
PlatformId.LINUX_x64,
PlatformId.LINUX_arm64,
PlatformId.OSX,
PlatformId.OSX_x64,
PlatformId.OSX_arm64,
PlatformId.WIN_x64,
PlatformId.WIN_arm64,
]
assert platform_id in valid_platforms, f"Platform {platform_id} is not supported for multilspy PHP at the moment"
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
runtime_dependencies = d.get("runtimeDependencies", [])
intelephense_ls_dir = os.path.join(os.path.dirname(__file__), "static", "php-lsp")
# Verify both node and npm are installed
is_node_installed = shutil.which('node') is not None
assert is_node_installed, "node is not installed or isn't in PATH. Please install NodeJS and try again."
is_npm_installed = shutil.which('npm') is not None
assert is_npm_installed, "npm is not installed or isn't in PATH. Please install npm and try again."
# Install intelephense if not already installed
if not os.path.exists(intelephense_ls_dir):
os.makedirs(intelephense_ls_dir, exist_ok=True)
for dependency in runtime_dependencies:
# Windows doesn't support the 'user' parameter and doesn't have pwd module
if PlatformUtils.get_platform_id().value.startswith("win"):
subprocess.run(
dependency["command"],
shell=True,
check=True,
cwd=intelephense_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
)
else:
# On Unix-like systems, run as non-root user
import pwd
user = pwd.getpwuid(os.getuid()).pw_name
subprocess.run(
dependency["command"],
shell=True,
check=True,
user=user,
cwd=intelephense_ls_dir,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL
)
intelephense_executable_path = os.path.join(intelephense_ls_dir, "node_modules", ".bin", "intelephense")
assert os.path.exists(intelephense_executable_path), "intelephense executable not found. Please install intelephense and try again."
return f"{intelephense_executable_path} --stdio"
def __init__(self, config: MultilspyConfig, logger: MultilspyLogger, repository_root_path: str):
# Setup runtime dependencies before initializing
intelephense_cmd = self.setup_runtime_dependencies(logger, config)
super().__init__(
config,
logger,
repository_root_path,
ProcessLaunchInfo(cmd=intelephense_cmd, cwd=repository_root_path),
"php"
)
self.server_ready = asyncio.Event()
self.request_id = 0
def _get_initialize_params(self, repository_absolute_path: str) -> InitializeParams:
"""
Returns the initialize params for the TypeScript Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
d["processId"] = os.getpid()
assert d["rootPath"] == "$rootPath"
d["rootPath"] = repository_absolute_path
assert d["rootUri"] == "$rootUri"
d["rootUri"] = pathlib.Path(repository_absolute_path).as_uri()
assert d["workspaceFolders"][0]["uri"] == "$uri"
d["workspaceFolders"][0]["uri"] = pathlib.Path(repository_absolute_path).as_uri()
assert d["workspaceFolders"][0]["name"] == "$name"
d["workspaceFolders"][0]["name"] = os.path.basename(repository_absolute_path)
return d
@asynccontextmanager
async def start_server(self) -> AsyncIterator["Intelephense"]:
"""Start Intelephense server process"""
async def register_capability_handler(params):
return
async def window_log_message(msg):
self.logger.log(f"LSP: window/logMessage: {msg}", logging.INFO)
async def do_nothing(params):
return
self.server.on_request("client/registerCapability", register_capability_handler)
self.server.on_notification("window/logMessage", window_log_message)
self.server.on_notification("$/progress", do_nothing)
self.server.on_notification("textDocument/publishDiagnostics", do_nothing)
async with super().start_server():
self.logger.log("Starting Intelephense server process", logging.INFO)
await self.server.start()
initialize_params = self._get_initialize_params(self.repository_root_path)
self.logger.log(
"Sending initialize request from LSP client to LSP server and awaiting response",
logging.INFO,
)
init_response = await self.server.send.initialize(initialize_params)
self.logger.log(
"After sent initialize params",
logging.INFO,
)
# Verify server capabilities
assert "textDocumentSync" in init_response["capabilities"]
assert "completionProvider" in init_response["capabilities"]
assert "definitionProvider" in init_response["capabilities"]
self.server.notify.initialized({})
self.completions_available.set()
# Intelephense server is typically ready immediately after initialization
self.server_ready.set()
await self.server_ready.wait()
yield self
await self.server.shutdown()
await self.server.stop()
@override
# For some reason, the LS may need longer to process this, so we just retry
async def _send_references_request(self, relative_file_path: str, line: int, column: int):
# TODO: The LS doesn't return references contained in other files if it doesn't sleep. This is
# despite the LS having processed requests already. I don't know what causes this, but sleeping
# one second helps. It may be that sleeping only once is enough but that's hard to reliably test.
# May be related to the time it takes to read the files or something like that.
# The sleeping doesn't seem to be needed on all systems
sleep(1)
return await super()._send_references_request(relative_file_path, line, column)
@override
async def _send_definition_request(self, definition_params: DefinitionParams):
# TODO: same as above, also only a problem if the definition is in another file
sleep(1)
return await super()._send_definition_request(definition_params)
src/multilspy/language_servers/intelephense/runtime_dependencies.json
+10
View File
@@ -0,0 +1,10 @@
{
"_description": "Used to download the runtime dependencies for running intelephense. Obtained from https://www.npmjs.com/package/intelephense",
"runtimeDependencies": [
{
"id": "intelephense",
"description": "Intelephense package for Linux, OSX, and Windows. Both x64 and arm64 are supported.",
"command": "npm install --prefix ./ intelephense@1.14.4"
}
]
}
src/multilspy/language_servers/jedi_language_server/jedi_server.py
+1 -1
View File
@@ -43,7 +43,7 @@ class JediServer(LanguageServer):
"""
Returns the initialize params for the Jedi Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
src/multilspy/language_servers/kotlin_language_server/kotlin_language_server.py
+2 -2
View File
@@ -67,7 +67,7 @@ class KotlinLanguageServer(LanguageServer):
assert platform_id.value.startswith("win-") or platform_id.value.startswith("linux-") or platform_id.value.startswith("osx-"), "Only Windows, Linux and macOS platforms are supported for Kotlin in multilspy at the moment"
# Load dependency information
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -134,7 +134,7 @@ class KotlinLanguageServer(LanguageServer):
"""
Returns the initialize params for the Kotlin Language Server.
"""
with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), "r") as f:
with open(str(pathlib.PurePath(os.path.dirname(__file__), "initialize_params.json")), "r", encoding="utf-8") as f:
d: InitializeParams = json.load(f)
del d["_description"]
src/multilspy/language_servers/omnisharp/omnisharp.py
+3 -3
View File
@@ -118,7 +118,7 @@ class OmniSharp(LanguageServer):
"""
Returns the initialize params for the Omnisharp Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -145,7 +145,7 @@ class OmniSharp(LanguageServer):
platform_id = PlatformUtils.get_platform_id()
dotnet_version = PlatformUtils.get_dotnet_version()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -384,7 +384,7 @@ class OmniSharp(LanguageServer):
)
init_response = await self.server.send.initialize(initialize_params)
self.server.notify.initialized({})
with open(os.path.join(os.path.dirname(__file__), "workspace_did_change_configuration.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "workspace_did_change_configuration.json"), "r", encoding="utf-8") as f:
self.server.notify.workspace_did_change_configuration({
"settings": json.load(f)
})
src/multilspy/language_servers/rust_analyzer/rust_analyzer.py
+2 -2
View File
@@ -51,7 +51,7 @@ class RustAnalyzer(LanguageServer):
"""
platform_id = PlatformUtils.get_platform_id()
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -88,7 +88,7 @@ class RustAnalyzer(LanguageServer):
"""
Returns the initialize params for the Rust Analyzer Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
src/multilspy/language_servers/solargraph/solargraph.py
+2 -2
View File
@@ -56,7 +56,7 @@ class Solargraph(LanguageServer):
if platform_id in [PlatformId.WIN_x64, PlatformId.WIN_arm64, PlatformId.WIN_x86]:
which_cmd = "where"
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
@@ -100,7 +100,7 @@ class Solargraph(LanguageServer):
"""
Returns the initialize params for the Solargraph Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r", encoding="utf-8") as f:
d = json.load(f)
del d["_description"]
src/multilspy/language_servers/typescript_language_server/typescript_language_server.py
+20 -8
View File
@@ -4,22 +4,23 @@ Provides TypeScript specific instantiation of the LanguageServer class. Contains
import asyncio
import json
import shutil
import logging
import os
import subprocess
import pathlib
import shutil
import subprocess
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import AsyncIterator, Tuple
from time import sleep
from overrides import override
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.language_server import LanguageServer
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.lsp_protocol_handler.lsp_types import InitializeParams
from multilspy.lsp_protocol_handler.server import ProcessLaunchInfo
from multilspy.multilspy_config import MultilspyConfig
from multilspy.multilspy_utils import PlatformUtils, PlatformId
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.multilspy_utils import PlatformId, PlatformUtils
# Platform-specific imports
if os.name != 'nt': # Unix-like systems
@@ -82,7 +83,7 @@ class TypeScriptLanguageServer(LanguageServer):
]
assert platform_id in valid_platforms, f"Platform {platform_id} is not supported for multilspy javascript/typescript at the moment"
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "runtime_dependencies.json")) as f:
d = json.load(f)
del d["_description"]
@@ -132,7 +133,7 @@ class TypeScriptLanguageServer(LanguageServer):
"""
Returns the initialize params for the TypeScript Language Server.
"""
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json"), "r") as f:
with open(os.path.join(os.path.dirname(__file__), "initialize_params.json")) as f:
d = json.load(f)
del d["_description"]
@@ -222,3 +223,14 @@ class TypeScriptLanguageServer(LanguageServer):
await self.server.shutdown()
await self.server.stop()
@override
# For some reason, the LS may need longer to process this, so we just retry
async def _send_references_request(self, relative_file_path: str, line: int, column: int):
# TODO: The LS doesn't return references contained in other files if it doesn't sleep. This is
# despite the LS having processed requests already. I don't know what causes this, but sleeping
# one second helps. It may be that sleeping only once is enough but that's hard to reliably test.
# It may be that even this 1sec is not enough in larger TS projects, at some point we should find what
# causes this and solve it.
sleep(1)
return await super()._send_references_request(relative_file_path, line, column)
src/multilspy/lsp_protocol_handler/server.py
+2 -1
View File
@@ -38,6 +38,7 @@ from typing import Any, Dict, List, Optional, Union
from .lsp_requests import LspNotification, LspRequest
from .lsp_types import ErrorCodes
from ..multilspy_exceptions import MultilspyException
StringDict = Dict[str, Any]
PayloadLike = Union[List[StringDict], StringDict, None]
@@ -481,7 +482,7 @@ class LanguageServerHandler:
await self._send_payload(make_request(method, request_id, params))
await request.cv.wait()
if isinstance(request.error, Error):
raise request.error
raise MultilspyException(f"Could not process request {method} with params:\n{params}.\n Language server error: {request.error}") from request.error
return request.result
def _send_payload_sync(self, payload: StringDict) -> None:
src/multilspy/multilspy_config.py
+3
View File
@@ -37,6 +37,7 @@ class Language(str, Enum):
RUBY = "ruby"
DART = "dart"
CPP = "cpp"
PHP = "php"
def __str__(self) -> str:
return self.value
@@ -65,6 +66,8 @@ class Language(str, Enum):
return FilenameMatcher("*.kt", "*.kts")
case self.DART:
return FilenameMatcher("*.dart")
case self.PHP:
return FilenameMatcher("*.php")
case _:
raise ValueError(f"Unhandled language: {self}")
src/multilspy/multilspy_types.py
+10 -1
View File
@@ -199,7 +199,11 @@ class UnifiedSymbolInformation(TypedDict):
""" The name of the symbol containing this symbol. This information is for
user interface purposes (e.g. to render a qualifier in the user interface
if necessary). It can't be used to re-infer a hierarchy for the document
symbols. """
symbols.
Note: within Serena, the parent attribute was added and should be used instead.
Most LS don't provide containerName.
"""
detail: NotRequired[str]
""" More detail for this symbol, e.g the signature of a function. """
@@ -220,6 +224,11 @@ class UnifiedSymbolInformation(TypedDict):
Added to be compatible with `lsp_types.DocumentSymbol`,
since it is sometimes useful to have the children of the symbol as a user-facing feature."""
parent: NotRequired[UnifiedSymbolInformation | None]
"""The parent of the symbol, if there is any. Added with Serena, not part of the LSP.
All symbols except the root packages will have a parent.
"""
class MarkupKind(Enum):
"""Describes the content type that a client supports in various
src/multilspy/multilspy_utils.py
+21 -13
View File
@@ -17,6 +17,7 @@ from enum import Enum
from multilspy.multilspy_exceptions import MultilspyException
from pathlib import PurePath, Path
from multilspy.multilspy_logger import MultilspyLogger
from multilspy.multilspy_types import UnifiedSymbolInformation
class TextUtils:
@@ -90,7 +91,14 @@ class PathUtils:
parsed = urlparse(uri)
host = "{0}{0}{mnt}{0}".format(os.path.sep, mnt=parsed.netloc)
return os.path.normpath(os.path.join(host, url2pathname(unquote(parsed.path))))
@staticmethod
def path_to_uri(path: str) -> str:
"""
Converts a file path to a file URI (file:///...).
"""
return str(Path(path).absolute().as_uri())
@staticmethod
def is_glob_pattern(pattern: str) -> bool:
"""Check if a pattern contains glob-specific characters."""
@@ -120,22 +128,12 @@ class FileUtils:
if not os.path.exists(file_path):
logger.log(f"File read '{file_path}' failed: File does not exist.", logging.ERROR)
raise MultilspyException(f"File read '{file_path}' failed: File does not exist.")
encodings = ["utf-8-sig", "utf-16", "utf-8", "latin-1"]
try:
for encoding in encodings:
try:
with open(file_path, "r", encoding=encoding) as inp_file:
return inp_file.read()
except UnicodeError:
continue
# Try system default encoding as a last resort
with open(file_path, "r") as inp_file:
with open(file_path, "r", encoding="utf-8") as inp_file:
return inp_file.read()
except Exception as exc:
logger.log(f"File read '{file_path}' failed: {exc}", logging.ERROR)
logger.log(f"File read '{file_path}' failed to read with encoding 'utf-8': {exc}", logging.ERROR)
raise MultilspyException("File read failed.") from None
logger.log(f"File read '{file_path}' failed: Unsupported encoding.", logging.ERROR)
raise MultilspyException(f"File read '{file_path}' failed: Unsupported encoding.") from None
@staticmethod
def download_file(logger: MultilspyLogger, url: str, target_path: str) -> None:
@@ -315,3 +313,13 @@ class PlatformUtils:
except (FileNotFoundError, subprocess.CalledProcessError):
raise MultilspyException("dotnet or mono not found on the system")
class SymbolUtils:
@staticmethod
def symbol_tree_contains_name(roots: list[UnifiedSymbolInformation], name: str) -> bool:
for symbol in roots:
if symbol["name"] == name:
return True
if SymbolUtils.symbol_tree_contains_name(symbol["children"], name):
return True
return False
src/serena/__init__.py
+1 -1
View File
@@ -1,4 +1,4 @@
__version__ = "2025-04-07"
__version__ = "2025-05-21"
def serena_root_path() -> str:
src/serena/agent.py
+151 -77
View File
@@ -10,11 +10,12 @@ import sys
import traceback
from abc import ABC
from collections import defaultdict
from collections.abc import Callable, Generator, Iterable
from collections.abc import Callable, Generator, Iterable, Sequence
from copy import copy
from dataclasses import dataclass, field
from logging import Logger
from pathlib import Path
from typing import TYPE_CHECKING, Any, Self, TypeVar, Union, cast
from typing import TYPE_CHECKING, Any, Self, TypeVar, Union
import yaml
from sensai.util import logging
@@ -30,7 +31,6 @@ from serena.config import SerenaAgentContext, SerenaAgentMode
from serena.prompt_factory import PromptFactory, SerenaPromptFactory
from serena.symbol import SymbolLocation, SymbolManager
from serena.text_utils import search_files
from serena.util.class_decorators import singleton
from serena.util.file_system import scan_directory
from serena.util.inspection import iter_subclasses
from serena.util.shell import execute_shell_command
@@ -69,22 +69,40 @@ class SerenaConfigError(Exception):
pass
@dataclass
class ProjectConfig(ToStringMixin):
project_name: str
language: Language
project_root: str
ignored_paths: list[str] = field(default_factory=list)
excluded_tools: set[str] = field(default_factory=set)
read_only: bool = False
ignore_all_files_in_gitignore: bool = True
SERENA_MANAGED_DIR = ".serena"
SERENA_DEFAULT_PROJECT_FILE = "project.yml"
def __init__(self, config_dict: dict[str, Any], project_name: str, project_root: Path | None = None):
self.project_name: str = project_name
@classmethod
def rel_path_to_project_yml(cls) -> str:
return os.path.join(cls.SERENA_MANAGED_DIR, cls.SERENA_DEFAULT_PROJECT_FILE)
@classmethod
def from_config_dict(cls, config_dict: dict[str, Any], project_name: str, project_root: Path | None = None) -> Self:
"""
Create a ProjectConfig instance from a configuration dictionary
"""
try:
self.language: Language = Language(config_dict["language"].lower())
language = Language(config_dict["language"].lower())
except ValueError as e:
raise ValueError(f"Invalid language: {config_dict['language']}.\nValid languages are: {[l.value for l in Language]}") from e
if project_root is None:
project_root = Path(config_dict["project_root"])
self.project_root: str = str(project_root.resolve())
self.ignored_paths: list[str] = config_dict.get("ignored_paths", [])
self.excluded_tools: set[str] = set(config_dict.get("excluded_tools", []))
self.read_only: bool = config_dict.get("read_only", False)
project_root_str = str(project_root.resolve())
ignored_paths = config_dict.get("ignored_paths", [])
excluded_tools = set(config_dict.get("excluded_tools", []))
read_only = config_dict.get("read_only", False)
if "ignore_all_files_in_gitignore" not in config_dict:
raise SerenaConfigError(
@@ -92,7 +110,7 @@ class ProjectConfig(ToStringMixin):
"Please update your `.yml` configuration file for this project. "
"It is recommended to set this to `True`."
)
self.ignore_all_files_in_gitignore = config_dict["ignore_all_files_in_gitignore"]
ignore_all_files_in_gitignore = config_dict["ignore_all_files_in_gitignore"]
# Raise errors for deprecated keys
if "ignored_dirs" in config_dict:
@@ -101,9 +119,24 @@ class ProjectConfig(ToStringMixin):
"Note that you can also set `ignore_all_files_in_gitignore` to `True`, which will be enough for most cases."
)
return cls(
project_name=project_name,
language=language,
project_root=project_root_str,
ignored_paths=ignored_paths,
excluded_tools=excluded_tools,
read_only=read_only,
ignore_all_files_in_gitignore=ignore_all_files_in_gitignore,
)
@classmethod
def from_yml(cls, yml_path: Path) -> Self:
def _from_yml(cls, yml_path: Path) -> Self:
"""
Create a ProjectConfig instance from a YAML file
"""
log.info(f"Loading project configuration from {yml_path}")
if not yml_path.exists():
raise FileNotFoundError(f"Project file not found: {yml_path}")
try:
with open(yml_path, encoding="utf-8") as f:
config_dict = yaml.safe_load(f)
@@ -113,10 +146,23 @@ class ProjectConfig(ToStringMixin):
else:
project_root = None
project_name = yml_path.stem
return cls(config_dict, project_name=project_name, project_root=project_root)
return cls.from_config_dict(config_dict, project_name=project_name, project_root=project_root)
except Exception as e:
raise ValueError(f"Error loading project configuration from {yml_path}: {e}") from e
@classmethod
def from_path(cls, path: Path | str) -> Self:
"""
Create a ProjectConfig instance from a path. Can either be a path to a yaml file or a directory
containing `.serena/project.yml`.
"""
path = Path(path)
if not path.exists():
raise FileNotFoundError(f"Path not found: {path}")
if path.is_dir():
path = path / cls.SERENA_MANAGED_DIR / cls.SERENA_DEFAULT_PROJECT_FILE
return cls._from_yml(path)
def get_serena_managed_dir(self) -> str:
return os.path.join(self.project_root, self.SERENA_MANAGED_DIR)
@@ -124,18 +170,29 @@ class ProjectConfig(ToStringMixin):
return set(ToolRegistry.get_tool_class_by_name(tool_name) for tool_name in self.excluded_tools)
@singleton
@dataclass
class SerenaConfig:
"""
Handles user-defined Serena configuration based on the configuration file
"""
projects: dict[str, ProjectConfig] = field(default_factory=dict)
project_names: list[str] = field(default_factory=list)
gui_log_window_enabled: bool = False
gui_log_window_level: int = logging.INFO
enable_project_activation: bool = True
CONFIG_FILE = "serena_config.yml"
def __init__(self) -> None:
config_file = os.path.join(serena_root_path(), self.CONFIG_FILE)
@classmethod
def from_config_file(cls) -> "SerenaConfig":
"""
Static constructor to create SerenaConfig from the configuration file
"""
config_file = os.path.join(serena_root_path(), cls.CONFIG_FILE)
if not os.path.exists(config_file):
raise FileNotFoundError(f"Serena configuration file not found: {config_file}")
with open(config_file, encoding="utf-8") as f:
try:
log.info(f"Loading Serena configuration from {config_file}")
@@ -143,10 +200,13 @@ class SerenaConfig:
except Exception as e:
raise ValueError(f"Error loading Serena configuration from {config_file}: {e}") from e
# Create instance
instance = cls()
# read projects
self.projects: dict[str, ProjectConfig] = {}
if "projects" not in config_yaml:
raise SerenaConfigError("`projects` key not found in Serena configuration. Please update your `serena_config.yml` file.")
projects = {}
for project_config_path in config_yaml["projects"]:
project_config_path = Path(project_config_path)
if not project_config_path.is_absolute():
@@ -156,15 +216,18 @@ class SerenaConfig:
if not project_config_path.is_file():
raise FileNotFoundError(f"Project file not found: {project_config_path}")
log.info(f"Loading project configuration from {project_config_path}")
project_config = ProjectConfig.from_yml(project_config_path)
self.projects[project_config.project_name] = project_config
self.project_names = list(self.projects.keys())
project_config = ProjectConfig.from_path(project_config_path)
projects[project_config.project_name] = project_config
self.gui_log_window_enabled = config_yaml.get("gui_log_window", False)
self.gui_log_window_level = config_yaml.get("gui_log_level", logging.INFO)
self.enable_project_activation = config_yaml.get("enable_project_activation", True)
instance.projects = projects
instance.project_names = list(projects.keys())
instance.gui_log_window_enabled = config_yaml.get("gui_log_window", False)
instance.gui_log_window_level = config_yaml.get("gui_log_level", logging.INFO)
instance.enable_project_activation = config_yaml.get("enable_project_activation", True)
def get_project_configuration(self, project_name: str) -> ProjectConfig:
return instance
def get_project_configuration(self, project_name: str) -> "ProjectConfig":
if project_name not in self.projects:
raise ValueError(f"Project '{project_name}' not found in Serena configuration; valid project names: {self.project_names}")
return self.projects[project_name]
@@ -189,13 +252,15 @@ class LinesRead:
class SerenaAgent:
def __init__(
self,
project_file_path: str | None = None,
project_config: ProjectConfig | str | Path | None = None,
project_activation_callback: Callable[[], None] | None = None,
serena_config: SerenaConfig | None = None,
context: SerenaAgentContext | None = None,
modes: list[SerenaAgentMode] | None = None,
):
"""
:param project_file_path: the configuration file (.yml) of the project to load immediately;
:param project_config: the project to load immediately; may be a path to the project configuration file
or a configuration instance;
if None, do not load any project (must use project selection tool to activate a project).
If a project is provided, the corresponding language server will be started.
:param project_activation_callback: a callback function to be called when a project is activated.
@@ -203,9 +268,10 @@ class SerenaAgent:
The context may adjust prompts, tool availability, and tool descriptions.
:param modes: list of modes in which the agent is operating (they will be combined), None for default modes.
The modes may adjust prompts, tool availability, and tool descriptions.
:param serena_config: the Serena configuration or None to read the configuration from the default location.
"""
# obtain serena configuration
self.serena_config = SerenaConfig()
self.serena_config = serena_config or SerenaConfig.from_config_file()
# open GUI log window if enabled
self._gui_log_handler: Union["GuiLogViewerHandler", None] = None # noqa
@@ -266,20 +332,20 @@ class SerenaAgent:
self._gui_log_handler.log_viewer.set_tool_names(tool_names)
# activate a project configuration (if provided or if there is only a single project available)
project_config: ProjectConfig | None = None
if project_file_path is not None:
if not os.path.exists(project_file_path):
raise FileNotFoundError(f"Project file not found: {project_file_path}")
log.info(f"Loading project configuration from {project_file_path}")
project_config = ProjectConfig.from_yml(Path(project_file_path))
project_config_to_load: ProjectConfig | None = None
if project_config is not None:
if isinstance(project_config, (str, Path)):
project_config_to_load = ProjectConfig.from_path(project_config)
else:
project_config_to_load = project_config
else:
match len(self.serena_config.projects):
case 0:
raise RuntimeError(f"No projects found in {SerenaConfig.CONFIG_FILE} and no project file specified.")
case 1:
project_config = self.serena_config.get_project_configuration(self.serena_config.project_names[0])
if project_config is not None:
self.activate_project(project_config)
project_config_to_load = self.serena_config.get_project_configuration(self.serena_config.project_names[0])
if project_config_to_load is not None:
self.activate_project(project_config_to_load)
else:
if not self.serena_config.enable_project_activation:
raise ValueError("Tool-based project activation is disabled in the configuration but no project file was provided.")
@@ -774,7 +840,7 @@ class FindSymbolTool(Tool):
def apply(
self,
name: str,
name_path: str,
depth: int = 0,
within_relative_path: str | None = None,
include_body: bool = False,
@@ -784,48 +850,56 @@ class FindSymbolTool(Tool):
max_answer_chars: int = _DEFAULT_MAX_ANSWER_LENGTH,
) -> str:
"""
Retrieves information on all symbols/code entities, i.e. classes, methods, attributes, variables, etc.
with the given name.
The returned symbol location information can subsequently be used to edit the returned symbols
or to retrieve further information using other tools.
If you already anticipate that you will need to reference children of the symbol (like methods or fields contained in a class),
you can specify a depth > 0.
Retrieves information on all symbols/code entities (classes, methods, etc.) based on the given `name_path`,
which represents a pattern for the symbol's path within the symbol tree of a single file.
The returned symbol location can be used for edits or further queries.
Specify `depth > 0` to retrieve children (e.g., methods of a class).
:param name: the name of the symbols to find
:param depth: specifies the depth up to which descendants of the symbol are to be retrieved
(e.g. depth 1 will retrieve methods and attributes for the case where the symbol refers to a class).
Provide a non-zero depth if you intend to subsequently query symbols that are contained in the
retrieved symbol.
:param within_relative_path: pass a relative path to only consider symbols within this path.
If a file is passed, only the symbols within this file will be considered.
If a directory is passed, all files within this directory will be considered.
If None, the entire codebase will be considered.
:param include_body: whether to include the body of all symbols in the result. You should only use this
if you actually need the body of the symbol for the task at hand (for example, for a deep analysis
of the functionality or for an editing task).
:param include_kinds: an optional list of ints representing the LSP symbol kind.
If provided, only symbols of the given kinds will be included in the result.
Valid kinds:
1=file, 2=module, 3=namespace, 4=package, 5=class, 6=method, 7=property, 8=field, 9=constructor, 10=enum,
The matching behavior is determined by the structure of `name_path`, which can
either be a simple name (e.g. "method") or a name path like "class/method" (relative name path)
or "/class/method" (absolute name path). Note that the name path is not a path in the file system
but rather a path in the symbol tree **within a single file**. Thus, file or directory names should never
be included in the `name_path`. For restricting the search to a single file or directory,
the `within_relative_path` parameter should be used instead. The retrieved symbols' `name_path` attribute
will always be composed of symbol names, never file or directory names.
Key aspects of the name path matching behavior:
- Trailing slashes in `name_path` play no role and are ignored.
- The name of the retrieved symbols will match (either exactly or as a substring)
the last segment of `name_path`, while other segments will restrict the search to symbols that
have a desired sequence of ancestors.
- If there is no starting or intermediate slash in `name_path`, there is no
restriction on the ancestor symbols. For example, passing `method` will match
against symbols with name paths like `method`, `class/method`, `class/nested_class/method`, etc.
- If `name_path` contains a `/` but doesn't start with a `/`, the matching is restricted to symbols
with the same ancestors as the last segment of `name_path`. For example, passing `class/method` will match against
`class/method` as well as `nested_class/class/method` but not `method`.
- If `name_path` starts with a `/`, it will be treated as an absolute name path pattern, meaning
that the first segment of it must match the first segment of the symbol's name path.
For example, passing `/class` will match only against top-level symbols like `class` but not against `nested_class/class`.
Passing `/class/method` will match against `class/method` but not `nested_class/class/method` or `method`.
:param name_path: The name path pattern to search for, see above for details.
:param depth: Depth to retrieve descendants (e.g., 1 for class methods/attributes).
:param within_relative_path: Optional. Restrict search to this file or directory. If None, searches entire codebase.
:param include_body: If True, include the symbol's source code. Use judiciously.
:param include_kinds: Optional. List of LSP symbol kind integers to include. (e.g., 5 for Class, 12 for Function).
Valid kinds: 1=file, 2=module, 3=namespace, 4=package, 5=class, 6=method, 7=property, 8=field, 9=constructor, 10=enum,
11=interface, 12=function, 13=variable, 14=constant, 15=string, 16=number, 17=boolean, 18=array, 19=object,
20=key, 21=null, 22=enum member, 23=struct, 24=event, 25=operator, 26=type parameter
:param exclude_kinds: If provided, symbols of the given kinds will be excluded from the result.
Takes precedence over include_kinds.
:param substring_matching: whether to use substring matching for the symbol name.
If True, the symbol name will be matched if it contains the given name as a substring.
:param max_answer_chars: if the output is longer than this number of characters,
no content will be returned. Don't adjust unless there is really no other way to get the content
required for the task. Instead, if the output is too long, you should
make a stricter query.
:return: a list of symbols (with symbol locations) that match the given name in JSON format
:param exclude_kinds: Optional. List of LSP symbol kind integers to exclude. Takes precedence over `include_kinds`.
:param substring_matching: If True, use substring matching for the last segment of `name`.
:param max_answer_chars: Max characters for the JSON result. If exceeded, no content is returned.
:return: JSON string: a list of symbols (with locations) matching the name.
"""
include_kinds = cast(list[SymbolKind] | None, include_kinds)
exclude_kinds = cast(list[SymbolKind] | None, exclude_kinds)
parsed_include_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in include_kinds] if include_kinds else None
parsed_exclude_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in exclude_kinds] if exclude_kinds else None
symbols = self.symbol_manager.find_by_name(
name,
name_path,
include_body=include_body,
include_kinds=include_kinds,
exclude_kinds=exclude_kinds,
include_kinds=parsed_include_kinds,
exclude_kinds=parsed_exclude_kinds,
substring_matching=substring_matching,
within_relative_path=within_relative_path,
)
@@ -876,13 +950,13 @@ class FindReferencingSymbolsTool(Tool):
make a stricter query.
:return: a list of JSON objects with the symbols referencing the requested symbol
"""
include_kinds = cast(list[SymbolKind] | None, include_kinds)
exclude_kinds = cast(list[SymbolKind] | None, exclude_kinds)
parsed_include_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in include_kinds] if include_kinds else None
parsed_exclude_kinds: Sequence[SymbolKind] | None = [SymbolKind(k) for k in exclude_kinds] if exclude_kinds else None
symbols = self.symbol_manager.find_referencing_symbols(
SymbolLocation(relative_path, line, column),
include_body=include_body,
include_kinds=include_kinds,
exclude_kinds=exclude_kinds,
include_kinds=parsed_include_kinds,
exclude_kinds=parsed_exclude_kinds,
)
symbol_dicts = [s.to_dict(kind=True, location=True, depth=0, include_body=include_body) for s in symbols]
result = json.dumps(symbol_dicts)
src/serena/symbol.py
+496 -380
View File
@@ -1,380 +1,496 @@
import logging
import os
from collections.abc import Iterator, Sequence
from contextlib import contextmanager
from copy import copy
from dataclasses import asdict, dataclass
from typing import TYPE_CHECKING, Any, Self
from sensai.util.string import ToStringMixin
from multilspy import SyncLanguageServer
from multilspy.multilspy_types import Position, SymbolKind, UnifiedSymbolInformation
if TYPE_CHECKING:
from .agent import SerenaAgent
log = logging.getLogger(__name__)
@dataclass
class SymbolLocation:
"""
Represents the (start) location of a symbol identifier
"""
relative_path: str | None
"""
the relative path of the file containing the symbol; if None, the symbol is defined outside of the project's scope
"""
line: int | None
"""
the line number in which the symbol identifier is defined (if the symbol is a function, class, etc.);
may be None for some types of symbols (e.g. SymbolKind.File)
"""
column: int | None
"""
the column number in which the symbol identifier is defined (if the symbol is a function, class, etc.);
may be None for some types of symbols (e.g. SymbolKind.File)
"""
def __post_init__(self) -> None:
if self.relative_path is not None:
self.relative_path = self.relative_path.replace("/", os.path.sep)
def to_dict(self) -> dict[str, Any]:
return asdict(self)
def has_position_in_file(self) -> bool:
return self.relative_path is not None and self.line is not None and self.column is not None
class Symbol(ToStringMixin):
def __init__(self, s: UnifiedSymbolInformation) -> None:
self.s = s
def _tostring_includes(self) -> list[str]:
return []
def _tostring_additional_entries(self) -> dict[str, Any]:
return dict(name=self.name, kind=self.kind, num_children=len(self.s["children"]))
@property
def name(self) -> str:
return self.s["name"]
@property
def kind(self) -> str:
return SymbolKind(self.symbol_kind).name
@property
def symbol_kind(self) -> SymbolKind:
return self.s["kind"]
@property
def relative_path(self) -> str | None:
return self.s["location"]["relativePath"]
@property
def location(self) -> SymbolLocation:
"""
:return: the start location of the actual symbol identifier
"""
return SymbolLocation(relative_path=self.relative_path, line=self.line, column=self.column)
@property
def body_start_position(self) -> Position:
return self.s["location"]["range"]["start"]
@property
def body_end_position(self) -> Position:
return self.s["location"]["range"]["end"]
@property
def line(self) -> int | None:
if "selectionRange" in self.s:
return self.s["selectionRange"]["start"]["line"]
else:
# line is expected to be undefined for some types of symbols (e.g. SymbolKind.File)
return None
@property
def column(self) -> int | None:
if "selectionRange" in self.s:
return self.s["selectionRange"]["start"]["character"]
else:
# precise location is expected to be undefined for some types of symbols (e.g. SymbolKind.File)
return None
@property
def body(self) -> str | None:
return self.s.get("body")
def iter_children(self) -> Iterator["Symbol"]:
for c in self.s["children"]:
yield Symbol(c)
def find(
self,
name: str,
substring_matching: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
) -> list["Symbol"]:
result = []
def should_include(s: "Symbol") -> bool:
if not ((substring_matching and name in s.name) or name == s.name):
return False
if include_kinds is not None and s.symbol_kind not in include_kinds:
return False
if exclude_kinds is not None and s.symbol_kind in exclude_kinds:
return False
return True
def traverse(s: "Symbol") -> None:
if should_include(s):
result.append(s)
for c in s.iter_children():
traverse(c)
traverse(self)
return result
def to_dict(
self, kind: bool = False, location: bool = False, depth: int = 0, include_body: bool = False, include_children_body: bool = False
) -> dict[str, Any]:
"""
Convert the symbol to a dictionary.
:param kind: whether to include the kind of the symbol
:param location: whether to include the location of the symbol
:param depth: the depth of the symbol
:param include_body: whether to include the body of the top-level symbol.
:param include_children_body: whether to also include the body of the children.
Note that the body of the children is part of the body of the parent symbol,
so there is usually no need to set this to True unless you want process the output
and pass the children without passing the parent body to the LM.
:return: a dictionary representation of the symbol
"""
result: dict[str, Any] = {"name": self.name}
if kind:
result["kind"] = self.kind
if location:
result["location"] = self.location.to_dict()
if include_body:
if self.body is None:
log.warning("Requested body for symbol, but it is not present. The symbol might have been loaded with include_body=False.")
result["body"] = self.body
def add_children(s: Self) -> list[dict[str, Any]]:
children = []
for c in s.iter_children():
children.append(
c.to_dict(
kind=kind,
location=location,
depth=depth - 1,
include_body=include_children_body,
include_children_body=include_children_body,
)
)
return children
if depth > 0:
result["children"] = add_children(self)
return result
class SymbolManager:
def __init__(self, lang_server: SyncLanguageServer, agent: "SerenaAgent") -> None:
self.lang_server = lang_server
self.agent = agent
def _to_symbols(self, items: list[UnifiedSymbolInformation]) -> list[Symbol]:
return [Symbol(s) for s in items]
def find_by_name(
self,
name: str,
within_relative_path: str | None = None,
include_body: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
substring_matching: bool = False,
) -> list[Symbol]:
"""
Find all symbols that match the given name.
:param name: the name of the symbol to find
:param within_relative_path: pass a relative path to only consider symbols within this path.
If a file is passed, only the symbols within this file will be considered.
If a directory is passed, all files within this directory will be considered.
If None, the entire codebase will be considered.
:param include_body: whether to include the body of all symbols in the result.
Note: you can filter out the bodies of the children if you set include_children_body=False
in the to_dict method.
:param include_kinds: an optional sequence of ints representing the LSP symbol kind.
If provided, only symbols of the given kinds will be included in the result.
:param exclude_kinds: If provided, symbols of the given kinds will be excluded from the result.
Takes precedence over include_kinds.
:param substring_matching: whether to use substring matching for the symbol name.
If True, the symbol name will be matched if it contains the given name as a substring.
:return: a list of symbols that match the given name
"""
symbols: list[Symbol] = []
symbol_roots = self.lang_server.request_full_symbol_tree(within_relative_path=within_relative_path, include_body=include_body)
for root in symbol_roots:
symbols.extend(
Symbol(root).find(name, include_kinds=include_kinds, exclude_kinds=exclude_kinds, substring_matching=substring_matching)
)
return symbols
def get_document_symbols(self, relative_path: str) -> list[Symbol]:
symbol_dicts, roots = self.lang_server.request_document_symbols(relative_path, include_body=False)
symbols = [Symbol(s) for s in symbol_dicts]
return symbols
def find_by_location(self, location: SymbolLocation) -> Symbol | None:
if location.relative_path is None:
return None
symbol_dicts, roots = self.lang_server.request_document_symbols(location.relative_path, include_body=False)
for symbol_dict in symbol_dicts:
symbol = Symbol(symbol_dict)
if symbol.location == location:
return symbol
return None
def find_referencing_symbols(
self,
symbol_location: SymbolLocation,
include_body: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
) -> list[Symbol]:
"""
Find all symbols that reference the symbol at the given location.
:param symbol_location: the location of the symbol for which to find references
:param include_body: whether to include the body of all symbols in the result.
Note: you can filter out the bodies of the children if you set include_children_body=False
in the to_dict method.
:param include_kinds: an optional sequence of ints representing the LSP symbol kind.
If provided, only symbols of the given kinds will be included in the result.
:param exclude_kinds: If provided, symbols of the given kinds will be excluded from the result.
Takes precedence over include_kinds.
:return: a list of symbols that reference the given symbol
"""
if not symbol_location.has_position_in_file():
raise ValueError("Symbol location does not contain a valid position in a file")
assert symbol_location.relative_path is not None
assert symbol_location.line is not None
assert symbol_location.column is not None
symbol_dicts = self.lang_server.request_referencing_symbols(
relative_file_path=symbol_location.relative_path,
line=symbol_location.line,
column=symbol_location.column,
include_imports=False,
include_self=False,
include_body=include_body,
)
if include_kinds is not None:
symbol_dicts = [s for s in symbol_dicts if s["kind"] in include_kinds]
if exclude_kinds is not None:
symbol_dicts = [s for s in symbol_dicts if s["kind"] not in exclude_kinds]
return self._to_symbols(symbol_dicts)
@contextmanager
def _edited_file(self, relative_path: str) -> Iterator[None]:
with self.lang_server.open_file(relative_path) as file_buffer:
yield
root_path = self.lang_server.language_server.repository_root_path
abs_path = os.path.join(root_path, relative_path)
with open(abs_path, "w") as f:
f.write(file_buffer.contents)
self.agent.mark_file_modified(relative_path)
@contextmanager
def _edited_symbol_location(self, location: SymbolLocation) -> Iterator[Symbol]:
symbol = self.find_by_location(location)
if symbol is None:
raise ValueError("Symbol not found/has no defined location within a file")
assert location.relative_path is not None
with self._edited_file(location.relative_path):
yield symbol
def replace_body(self, location: SymbolLocation, body: str) -> None:
"""
Replace the body of the symbol at the given location with the given body
:param location: the location of the symbol to replace
:param body: the new body
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
assert location.relative_path is not None
self.lang_server.delete_text_between_positions(location.relative_path, symbol.body_start_position, symbol.body_end_position)
self.lang_server.insert_text_at_position(
location.relative_path, symbol.body_start_position["line"], symbol.body_start_position["character"], body
)
def insert_after(self, location: SymbolLocation, body: str) -> None:
"""
Appends content after the given symbol
:param location: the location of the symbol after which to add new lines
:param body: the body of the entity to append
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
pos = symbol.body_end_position
assert location.relative_path is not None
self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body)
def insert_before(self, location: SymbolLocation, body: str) -> None:
"""
Inserts content before the given symbol
:param location: the location of the symbol before which to add new lines
:param body: the body of the entity to insert
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
pos = copy(symbol.body_start_position)
assert location.relative_path is not None
self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body)
def insert_at_line(self, relative_path: str, line: int, content: str) -> None:
"""
Inserts content at the given line in the given file.
:param line: the 0-based index of the line to insert content at
:param content: the content to insert
"""
with self._edited_file(relative_path):
self.lang_server.insert_text_at_position(relative_path, line, 0, content)
def delete_lines(self, relative_path: str, start_line: int, end_line: int) -> None:
"""
Deletes lines in the given file.
:param start_line: the 0-based index of the first line to delete (inclusive)
:param end_line: the 0-based index of the last line to delete (inclusive)
"""
with self._edited_file(relative_path):
start_pos = Position(line=start_line, character=0)
end_pos = Position(line=end_line + 1, character=0)
self.lang_server.delete_text_between_positions(relative_path, start_pos, end_pos)
import logging
import os
from collections.abc import Iterator, Sequence
from contextlib import contextmanager
from copy import copy
from dataclasses import asdict, dataclass
from typing import TYPE_CHECKING, Any, Self
from sensai.util.string import ToStringMixin
from multilspy import SyncLanguageServer
from multilspy.multilspy_types import Position, SymbolKind, UnifiedSymbolInformation
if TYPE_CHECKING:
from .agent import SerenaAgent
log = logging.getLogger(__name__)
@dataclass
class SymbolLocation:
"""
Represents the (start) location of a symbol identifier
"""
relative_path: str | None
"""
the relative path of the file containing the symbol; if None, the symbol is defined outside of the project's scope
"""
line: int | None
"""
the line number in which the symbol identifier is defined (if the symbol is a function, class, etc.);
may be None for some types of symbols (e.g. SymbolKind.File)
"""
column: int | None
"""
the column number in which the symbol identifier is defined (if the symbol is a function, class, etc.);
may be None for some types of symbols (e.g. SymbolKind.File)
"""
def __post_init__(self) -> None:
if self.relative_path is not None:
self.relative_path = self.relative_path.replace("/", os.path.sep)
def to_dict(self) -> dict[str, Any]:
return asdict(self)
def has_position_in_file(self) -> bool:
return self.relative_path is not None and self.line is not None and self.column is not None
class Symbol(ToStringMixin):
_NAME_PATH_SEP = "/"
@staticmethod
def match_name_path(
name_path: str,
symbol_name_path_parts: list[str],
substring_matching: bool,
) -> bool:
"""
Checks if a given `name_path` matches a symbol's qualified name parts.
See docstring of `Symbol.find` for more details.
"""
assert name_path, "name_path must not be empty"
assert symbol_name_path_parts, "symbol_name_path_parts must not be empty"
name_path_sep = Symbol._NAME_PATH_SEP
is_absolute_pattern = name_path.startswith(name_path_sep)
pattern_parts = name_path.lstrip(name_path_sep).rstrip(name_path_sep).split(name_path_sep)
# filtering based on ancestors
if len(pattern_parts) > len(symbol_name_path_parts):
# can't possibly match if pattern has more parts than symbol
return False
if is_absolute_pattern and len(pattern_parts) != len(symbol_name_path_parts):
# for absolute patterns, the number of parts must match exactly
return False
if symbol_name_path_parts[-len(pattern_parts) : -1] != pattern_parts[:-1]:
# ancestors must match
return False
# matching the last part of the symbol name
name_to_match = pattern_parts[-1]
symbol_name = symbol_name_path_parts[-1]
if substring_matching:
return name_to_match in symbol_name
else:
return name_to_match == symbol_name
def __init__(self, symbol_root_from_ls: UnifiedSymbolInformation) -> None:
self.symbol_root = symbol_root_from_ls
def _tostring_includes(self) -> list[str]:
return []
def _tostring_additional_entries(self) -> dict[str, Any]:
return dict(name=self.name, kind=self.kind, num_children=len(self.symbol_root["children"]))
@property
def name(self) -> str:
return self.symbol_root["name"]
@property
def kind(self) -> str:
return SymbolKind(self.symbol_kind).name
@property
def symbol_kind(self) -> SymbolKind:
return self.symbol_root["kind"]
@property
def relative_path(self) -> str | None:
location = self.symbol_root.get("location")
if location:
return location.get("relativePath")
return None
@property
def location(self) -> SymbolLocation:
"""
:return: the start location of the actual symbol identifier
"""
return SymbolLocation(relative_path=self.relative_path, line=self.line, column=self.column)
@property
def body_start_position(self) -> Position | None:
location = self.symbol_root.get("location")
if location:
range_info = location.get("range")
if range_info:
start_pos = range_info.get("start")
if start_pos:
return start_pos
return None
@property
def body_end_position(self) -> Position | None:
location = self.symbol_root.get("location")
if location:
range_info = location.get("range")
if range_info:
end_pos = range_info.get("end")
if end_pos:
return end_pos
return None
@property
def line(self) -> int | None:
if "selectionRange" in self.symbol_root:
return self.symbol_root["selectionRange"]["start"]["line"]
else:
# line is expected to be undefined for some types of symbols (e.g. SymbolKind.File)
return None
@property
def column(self) -> int | None:
if "selectionRange" in self.symbol_root:
return self.symbol_root["selectionRange"]["start"]["character"]
else:
# precise location is expected to be undefined for some types of symbols (e.g. SymbolKind.File)
return None
@property
def body(self) -> str | None:
return self.symbol_root.get("body")
def get_name_path(self) -> str:
"""
Get the name path of the symbol (e.g. "class/method/inner_function").
"""
return self._NAME_PATH_SEP.join(self.get_name_path_parts())
def get_name_path_parts(self) -> list[str]:
"""
Get the parts of the name path of the symbol (e.g. ["class", "method", "inner_function"]).
"""
ancestors_within_file = list(self.iter_ancestors(up_to_symbol_kind=SymbolKind.File))
ancestors_within_file.reverse()
return [a.name for a in ancestors_within_file] + [self.name]
def iter_children(self) -> Iterator[Self]:
for c in self.symbol_root["children"]:
yield self.__class__(c)
def iter_ancestors(self, up_to_symbol_kind: SymbolKind | None = None) -> Iterator[Self]:
"""
Iterate over all ancestors of the symbol, starting with the parent and going up to the root or
the given symbol kind.
:param up_to_symbol_kind: if provided, iteration will stop *before* the first ancestor of the given kind.
A typical use case is to pass `SymbolKind.File` or `SymbolKind.Package`.
"""
parent = self.get_parent()
if parent is not None:
if up_to_symbol_kind is None or parent.symbol_kind != up_to_symbol_kind:
yield parent
yield from parent.iter_ancestors(up_to_symbol_kind=up_to_symbol_kind)
def get_parent(self) -> Self | None:
parent_root = self.symbol_root.get("parent")
if parent_root is None:
return None
return self.__class__(parent_root)
def find(
self,
name_path: str,
substring_matching: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
) -> list[Self]:
"""
Find all symbols within the symbol's subtree that match the given `name_path`.
The matching behavior is determined by the structure of `name_path`, which can
either be a simple name (e.g. "method") or a name path like "class/method" (relative name path)
or "/class/method" (absolute name path).
Key aspects of the name path matching behavior:
- Trailing slashes in `name_path` play no role and are ignored.
- The name of the retrieved symbols will match (either exactly or as a substring)
the last segment of `name_path`, while other segments will restrict the search to symbols that
have a desired sequence of ancestors.
- If there is no starting or intermediate slash in `name_path`, there is no
restriction on the ancestor symbols. For example, passing `method` will match
against symbols with name paths like `method`, `class/method`, `class/nested_class/method`, etc.
- If `name_path` contains a `/` but doesn't start with a `/`, the matching is restricted to symbols
with the same ancestors as the last segment of `name_path`. For example, passing `class/method` will match against
`class/method` as well as `nested_class/class/method` but not `method`.
- If `name_path` starts with a `/`, it will be treated as an absolute name path pattern, meaning
that the first segment of it must match the first segment of the symbol's name path.
For example, passing `/class` will match only against top-level symbols like `class` but not against `nested_class/class`.
Passing `/class/method` will match against `class/method` but not `nested_class/class/method` or `method`.
:param name_path: the name path to match against
:param substring_matching: whether to use substring matching (as opposed to exact matching)
of the last segment of `name_path` against the symbol name.
:param include_kinds: an optional sequence of ints representing the LSP symbol kind.
If provided, only symbols of the given kinds will be included in the result.
:param exclude_kinds: If provided, symbols of the given kinds will be excluded from the result.
"""
result = []
def should_include(s: "Symbol") -> bool:
if include_kinds is not None and s.symbol_kind not in include_kinds:
return False
if exclude_kinds is not None and s.symbol_kind in exclude_kinds:
return False
return Symbol.match_name_path(
name_path=name_path,
symbol_name_path_parts=s.get_name_path_parts(),
substring_matching=substring_matching,
)
def traverse(s: "Symbol") -> None:
if should_include(s):
result.append(s)
for c in s.iter_children():
traverse(c)
traverse(self)
return result
def to_dict(
self, kind: bool = False, location: bool = False, depth: int = 0, include_body: bool = False, include_children_body: bool = False
) -> dict[str, Any]:
"""
Convert the symbol to a dictionary.
:param kind: whether to include the kind of the symbol
:param location: whether to include the location of the symbol
:param depth: the depth of the symbol
:param include_body: whether to include the body of the top-level symbol.
:param include_children_body: whether to also include the body of the children.
Note that the body of the children is part of the body of the parent symbol,
so there is usually no need to set this to True unless you want process the output
and pass the children without passing the parent body to the LM.
:return: a dictionary representation of the symbol
"""
result: dict[str, Any] = {"name": self.name, "name_path": self.get_name_path()}
if kind:
result["kind"] = self.kind
if location:
result["location"] = self.location.to_dict()
if include_body:
if self.body is None:
log.warning("Requested body for symbol, but it is not present. The symbol might have been loaded with include_body=False.")
result["body"] = self.body
def add_children(s: Self) -> list[dict[str, Any]]:
children = []
for c in s.iter_children():
children.append(
c.to_dict(
kind=kind,
location=location,
depth=depth - 1,
include_body=include_children_body,
include_children_body=include_children_body,
)
)
return children
if depth > 0:
result["children"] = add_children(self)
return result
class SymbolManager:
def __init__(self, lang_server: SyncLanguageServer, agent: "SerenaAgent") -> None:
self.lang_server = lang_server
self.agent = agent
def _to_symbols(self, items: list[UnifiedSymbolInformation]) -> list[Symbol]:
return [Symbol(s) for s in items]
def find_by_name(
self,
name: str,
include_body: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
substring_matching: bool = False,
within_relative_path: str | None = None,
) -> list[Symbol]:
"""
Find all symbols that match the given name. See docstring of `Symbol.find` for more details.
The only parameter not mentioned there is `within_relative_path`, which can be used to restrict the search
to symbols within a specific file or directory.
"""
symbols: list[Symbol] = []
symbol_roots = self.lang_server.request_full_symbol_tree(within_relative_path=within_relative_path, include_body=include_body)
for root in symbol_roots:
symbols.extend(
Symbol(root).find(name, include_kinds=include_kinds, exclude_kinds=exclude_kinds, substring_matching=substring_matching)
)
return symbols
def get_document_symbols(self, relative_path: str) -> list[Symbol]:
symbol_dicts, roots = self.lang_server.request_document_symbols(relative_path, include_body=False)
symbols = [Symbol(s) for s in symbol_dicts]
return symbols
def find_by_location(self, location: SymbolLocation) -> Symbol | None:
if location.relative_path is None:
return None
symbol_dicts, roots = self.lang_server.request_document_symbols(location.relative_path, include_body=False)
for symbol_dict in symbol_dicts:
symbol = Symbol(symbol_dict)
if symbol.location == location:
return symbol
return None
def find_referencing_symbols(
self,
symbol_location: SymbolLocation,
include_body: bool = False,
include_kinds: Sequence[SymbolKind] | None = None,
exclude_kinds: Sequence[SymbolKind] | None = None,
) -> list[Symbol]:
"""
Find all symbols that reference the symbol at the given location.
:param symbol_location: the location of the symbol for which to find references
:param include_body: whether to include the body of all symbols in the result.
Note: you can filter out the bodies of the children if you set include_children_body=False
in the to_dict method.
:param include_kinds: an optional sequence of ints representing the LSP symbol kind.
If provided, only symbols of the given kinds will be included in the result.
:param exclude_kinds: If provided, symbols of the given kinds will be excluded from the result.
Takes precedence over include_kinds.
:return: a list of symbols that reference the given symbol
"""
if not symbol_location.has_position_in_file():
raise ValueError("Symbol location does not contain a valid position in a file")
assert symbol_location.relative_path is not None
assert symbol_location.line is not None
assert symbol_location.column is not None
symbol_dicts = self.lang_server.request_referencing_symbols(
relative_file_path=symbol_location.relative_path,
line=symbol_location.line,
column=symbol_location.column,
include_imports=False,
include_self=False,
include_body=include_body,
)
if include_kinds is not None:
symbol_dicts = [s for s in symbol_dicts if s["kind"] in include_kinds]
if exclude_kinds is not None:
symbol_dicts = [s for s in symbol_dicts if s["kind"] not in exclude_kinds]
return self._to_symbols(symbol_dicts)
@contextmanager
def _edited_file(self, relative_path: str) -> Iterator[None]:
with self.lang_server.open_file(relative_path) as file_buffer:
yield
root_path = self.lang_server.language_server.repository_root_path
abs_path = os.path.join(root_path, relative_path)
with open(abs_path, "w", encoding="utf-8") as f:
f.write(file_buffer.contents)
self.agent.mark_file_modified(relative_path)
@contextmanager
def _edited_symbol_location(self, location: SymbolLocation) -> Iterator[Symbol]:
symbol = self.find_by_location(location)
if symbol is None:
raise ValueError("Symbol not found/has no defined location within a file")
assert location.relative_path is not None
with self._edited_file(location.relative_path):
yield symbol
def replace_body(self, location: SymbolLocation, body: str) -> None:
"""
Replace the body of the symbol at the given location with the given body
:param location: the location of the symbol to replace
:param body: the new body
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
assert location.relative_path is not None
start_pos = symbol.body_start_position
end_pos = symbol.body_end_position
if start_pos is None or end_pos is None:
raise ValueError(f"Symbol at {location} does not have a defined body range.")
# At this point, start_pos and end_pos are guaranteed to be Position objects
self.lang_server.delete_text_between_positions(location.relative_path, start_pos, end_pos)
self.lang_server.insert_text_at_position(location.relative_path, start_pos["line"], start_pos["character"], body)
def insert_after(self, location: SymbolLocation, body: str) -> None:
"""
Appends content after the given symbol
:param location: the location of the symbol after which to add new lines
:param body: the body of the entity to append
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
pos = symbol.body_end_position
if pos is None:
raise ValueError(f"Symbol at {location} does not have a defined end position.")
# At this point, pos is guaranteed to be a Position object
assert location.relative_path is not None
self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body)
def insert_before(self, location: SymbolLocation, body: str) -> None:
"""
Inserts content before the given symbol
:param location: the location of the symbol before which to add new lines
:param body: the body of the entity to insert
"""
# make sure body always ends with at least one newline
if not body.endswith("\n"):
body += "\n"
with self._edited_symbol_location(location) as symbol:
original_start_pos = symbol.body_start_position
if original_start_pos is None:
raise ValueError(f"Symbol at {location} does not have a defined start position.")
# At this point, original_start_pos is guaranteed to be a Position object, so copying is safe.
pos = copy(original_start_pos)
assert location.relative_path is not None
self.lang_server.insert_text_at_position(location.relative_path, pos["line"], pos["character"], body)
def insert_at_line(self, relative_path: str, line: int, content: str) -> None:
"""
Inserts content at the given line in the given file.
:param line: the 0-based index of the line to insert content at
:param content: the content to insert
"""
with self._edited_file(relative_path):
self.lang_server.insert_text_at_position(relative_path, line, 0, content)
def delete_lines(self, relative_path: str, start_line: int, end_line: int) -> None:
"""
Deletes lines in the given file.
:param start_line: the 0-based index of the first line to delete (inclusive)
:param end_line: the 0-based index of the last line to delete (inclusive)
"""
with self._edited_file(relative_path):
start_pos = Position(line=start_line, character=0)
end_pos = Position(line=end_line + 1, character=0)
self.lang_server.delete_text_between_positions(relative_path, start_pos, end_pos)
test/__init__.py
+1
View File
@@ -0,0 +1 @@
test/conftest.py
+59 -12
View File
@@ -14,25 +14,72 @@ def resources_dir() -> Path:
return current_dir / "resources"
@pytest.fixture(scope="session")
def repo_path() -> Path:
return Path(__file__).parent / "resources" / "test_repo"
class LanguageParamRequest:
param: Language
@pytest.fixture(scope="session")
def language_server(repo_path: Path):
"""Create a SyncLanguageServer instance configured to use the test repository."""
config = MultilspyConfig(code_language=Language.PYTHON)
def get_repo_path(language: Language) -> Path:
return Path(__file__).parent / "resources" / "repos" / language / "test_repo"
def create_default_ls(language: Language) -> SyncLanguageServer:
config = MultilspyConfig(code_language=language)
repo_path = str(get_repo_path(language))
logger = MultilspyLogger()
return SyncLanguageServer.create(config, logger, repo_path)
# Create a language server instance
server = SyncLanguageServer.create(config, logger, str(repo_path))
# Start the server
@pytest.fixture(scope="session")
def repo_path(request: LanguageParamRequest) -> Path:
"""Get the repository path for a specific language.
This fixture requires a language parameter via pytest.mark.parametrize:
Example:
```
@pytest.mark.parametrize("repo_path", [Language.PYTHON], indirect=True)
def test_python_repo(repo_path):
assert (repo_path / "src").exists()
```
"""
if not hasattr(request, "param"):
raise ValueError("Language parameter must be provided via pytest.mark.parametrize")
language = request.param
return get_repo_path(language)
@pytest.fixture(scope="session")
def language_server(request: LanguageParamRequest):
"""Create a SyncLanguageServer instance configured for the specified language.
This fixture requires a language parameter via pytest.mark.parametrize:
Example:
```
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_python_server(language_server: SyncLanguageServer) -> None:
# Use the Python language server
pass
```
You can also test multiple languages in a single test:
```
@pytest.mark.parametrize("language_server", [Language.PYTHON, Language.TYPESCRIPT], indirect=True)
def test_multiple_languages(language_server: SyncLanguageServer) -> None:
# This test will run once for each language
pass
```
"""
if not hasattr(request, "param"):
raise ValueError("Language parameter must be provided via pytest.mark.parametrize")
language = request.param
server = create_default_ls(language)
server.start()
try:
yield server
finally:
# Ensure server is shut down
server.stop()
test/multilspy/go/test_go_basic.py
+33
View File
@@ -0,0 +1,33 @@
import os
import pytest
from multilspy import SyncLanguageServer
from multilspy.multilspy_config import Language
from multilspy.multilspy_utils import SymbolUtils
@pytest.mark.go
class TestGoLanguageServer:
@pytest.mark.parametrize("language_server", [Language.GO], indirect=True)
def test_find_symbol(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "main"), "main function not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Helper"), "Helper function not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "DemoStruct"), "DemoStruct not found in symbol tree"
@pytest.mark.parametrize("language_server", [Language.GO], indirect=True)
def test_find_referencing_symbols(self, language_server: SyncLanguageServer) -> None:
file_path = os.path.join("main.go")
symbols = language_server.request_document_symbols(file_path)
helper_symbol = None
for sym in symbols[0]:
if sym.get("name") == "Helper":
helper_symbol = sym
break
assert helper_symbol is not None, "Could not find 'Helper' function symbol in main.go"
sel_start = helper_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert any(
"main.go" in ref.get("relativePath", "") for ref in refs
), "main.go should reference Helper (tried all positions in selectionRange)"
test/multilspy/java/test_java_basic.py
+50
View File
@@ -0,0 +1,50 @@
import os
import pytest
from multilspy import SyncLanguageServer
from multilspy.multilspy_config import Language
from multilspy.multilspy_utils import SymbolUtils
@pytest.mark.java
class TestJavaLanguageServer:
@pytest.mark.parametrize("language_server", [Language.JAVA], indirect=True)
def test_find_symbol(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "Main"), "Main class not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Utils"), "Utils class not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Model"), "Model class not found in symbol tree"
@pytest.mark.parametrize("language_server", [Language.JAVA], indirect=True)
def test_find_referencing_symbols(self, language_server: SyncLanguageServer) -> None:
# Use correct Maven/Java file paths
file_path = os.path.join("src", "main", "java", "test_repo", "Utils.java")
refs = language_server.request_references(file_path, 4, 20)
assert any("Main.java" in ref.get("relativePath", "") for ref in refs), "Main should reference Utils.printHello"
# Dynamically determine the correct line/column for the 'Model' class name
file_path = os.path.join("src", "main", "java", "test_repo", "Model.java")
symbols = language_server.request_document_symbols(file_path)
model_symbol = None
for sym in symbols[0]:
if sym.get("name") == "Model" and sym.get("kind") == 5: # 5 = Class
model_symbol = sym
break
assert model_symbol is not None, "Could not find 'Model' class symbol in Model.java"
# Use selectionRange if present, otherwise fall back to range
if "selectionRange" in model_symbol:
sel_start = model_symbol["selectionRange"]["start"]
else:
sel_start = model_symbol["range"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert any(
"Main.java" in ref.get("relativePath", "") for ref in refs
), "Main should reference Model (tried all positions in selectionRange)"
@pytest.mark.parametrize("language_server", [Language.JAVA], indirect=True)
def test_overview_methods(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "Main"), "Main missing from overview"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Utils"), "Utils missing from overview"
assert SymbolUtils.symbol_tree_contains_name(symbols, "Model"), "Model missing from overview"
test/multilspy/php/test_php_basic.py
+135
View File
@@ -0,0 +1,135 @@
from pathlib import Path
import pytest
from multilspy.language_server import SyncLanguageServer
from multilspy.multilspy_config import Language
@pytest.mark.php
class TestPhpLanguageServer:
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_ls_is_running(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
"""Test that the language server starts and stops successfully."""
# The fixture already handles start and stop
assert language_server.is_running()
assert Path(language_server.language_server.repository_root_path).resolve() == repo_path.resolve()
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_find_definition_within_file(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
# In index.php:
# Line 9 (1-indexed): $greeting = greet($userName);
# Line 11 (1-indexed): echo $greeting;
# We want to find the definition of $greeting (defined on line 9)
# from its usage in echo $greeting; on line 11.
# LSP is 0-indexed: definition on line 8, usage on line 10.
# $greeting in echo $greeting; is at char 5 on line 11 (0-indexed: line 10, char 5)
# e c h o $ g r e e t i n g
# ^ char 5
definition_location_list = language_server.request_definition(str(repo_path / "index.php"), 10, 6) # cursor on 'g' in $greeting
assert definition_location_list, f"Expected non-empty definition_location_list but got {definition_location_list=}"
assert len(definition_location_list) == 1
definition_location = definition_location_list[0]
assert definition_location["uri"].endswith("index.php")
# Definition of $greeting is on line 10 (1-indexed) / line 9 (0-indexed), char 0
assert definition_location["range"]["start"]["line"] == 9
assert definition_location["range"]["start"]["character"] == 0
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_find_definition_across_files(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
definition_location_list = language_server.request_definition(str(repo_path / "index.php"), 12, 5) # helperFunction
assert definition_location_list, f"Expected non-empty definition_location_list but got {definition_location_list=}"
assert len(definition_location_list) == 1
definition_location = definition_location_list[0]
assert definition_location["uri"].endswith("helper.php")
assert definition_location["range"]["start"]["line"] == 2
assert definition_location["range"]["start"]["character"] == 0
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_find_definition_simple_variable(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
file_path = str(repo_path / "simple_var.php")
# In simple_var.php:
# Line 2 (1-indexed): $localVar = "test";
# Line 3 (1-indexed): echo $localVar;
# LSP is 0-indexed: definition on line 1, usage on line 2
# Find definition of $localVar (char 5 on line 3 / 0-indexed: line 2, char 5)
# $localVar in echo $localVar; (e c h o $ l o c a l V a r)
# ^ char 5
definition_location_list = language_server.request_definition(file_path, 2, 6) # cursor on 'l' in $localVar
assert definition_location_list, f"Expected non-empty definition_location_list but got {definition_location_list=}"
assert len(definition_location_list) == 1
definition_location = definition_location_list[0]
assert definition_location["uri"].endswith("simple_var.php")
assert definition_location["range"]["start"]["line"] == 1 # Definition of $localVar (0-indexed)
assert definition_location["range"]["start"]["character"] == 0 # $localVar (0-indexed)
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_find_references_within_file(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
index_php_path = str(repo_path / "index.php")
# In index.php (0-indexed lines):
# Line 9: $greeting = greet($userName); // Definition of $greeting
# Line 11: echo $greeting; // Usage of $greeting
# Find references for $greeting from its usage in "echo $greeting;" (line 11, char 6 for 'g')
references = language_server.request_references(index_php_path, 11, 6)
assert references
# Intelephense, when asked for references from usage, seems to only return the usage itself.
assert len(references) == 1, "Expected to find 1 reference for $greeting (the usage itself)"
expected_locations = [{"uri_suffix": "index.php", "line": 11, "character": 5}] # Usage: echo $greeting (points to $)
# Convert actual references to a comparable format and sort
actual_locations = sorted(
[
{
"uri_suffix": loc["uri"].split("/")[-1],
"line": loc["range"]["start"]["line"],
"character": loc["range"]["start"]["character"],
}
for loc in references
],
key=lambda x: (x["uri_suffix"], x["line"], x["character"]),
)
expected_locations = sorted(expected_locations, key=lambda x: (x["uri_suffix"], x["line"], x["character"]))
assert actual_locations == expected_locations
@pytest.mark.parametrize("language_server", [Language.PHP], indirect=True)
@pytest.mark.parametrize("repo_path", [Language.PHP], indirect=True)
def test_find_references_across_files(self, language_server: SyncLanguageServer, repo_path: Path) -> None:
helper_php_path = str(repo_path / "helper.php")
# In index.php (0-indexed lines):
# Line 13: helperFunction(); // Usage of helperFunction
# Find references for helperFunction from its definition
references = language_server.request_references(helper_php_path, 2, len("function "))
assert references, f"Expected non-empty references for helperFunction but got {references=}"
# Intelephense might return 1 (usage) or 2 (usage + definition) references.
# Let's check for at least the usage in index.php
# Definition is in helper.php, line 2, char 0 (based on previous findings)
# Usage is in index.php, line 13, char 0
actual_locations_comparable = []
for loc in references:
actual_locations_comparable.append(
{
"uri_suffix": loc["uri"].split("/")[-1],
"line": loc["range"]["start"]["line"],
"character": loc["range"]["start"]["character"],
}
)
usage_in_index_php = {"uri_suffix": "index.php", "line": 13, "character": 0}
assert usage_in_index_php in actual_locations_comparable, "Usage of helperFunction in index.php not found"
test/multilspy/test_basic.py → test/multilspy/python/test_python_basic.py
+48 -28
View File
@@ -7,60 +7,79 @@ like request_references using the test repository.
import os
import pytest
from multilspy.language_server import SyncLanguageServer
from multilspy.multilspy_config import Language
from serena.text_utils import LineType
@pytest.mark.python
class TestLanguageServerBasics:
"""Test basic functionality of the language server."""
def test_request_references_user_class(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_references_user_class(self, language_server: SyncLanguageServer) -> None:
"""Test request_references on the User class."""
# Get references to the User class in models.py
file_path = os.path.join("test_repo", "models.py")
# Line 31 contains the User class definition
references = language_server.request_references(file_path, 31, 6)
# Use selectionRange only
symbols = language_server.request_document_symbols(file_path)
user_symbol = next((s for s in symbols[0] if s.get("name") == "User"), None)
if not user_symbol or "selectionRange" not in user_symbol:
raise AssertionError("User symbol or its selectionRange not found")
sel_start = user_symbol["selectionRange"]["start"]
references = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert len(references) > 1, "User class should be referenced in multiple files (using selectionRange if present)"
# User class should be referenced in multiple files
assert len(references) > 0
# At least two references should be found (one for the class definition itself)
assert len(references) > 1
def test_request_references_item_class(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_references_item_class(self, language_server: SyncLanguageServer) -> None:
"""Test request_references on the Item class."""
# Get references to the Item class in models.py
file_path = os.path.join("test_repo", "models.py")
# Line 56 contains the Item class definition
references = language_server.request_references(file_path, 56, 6)
# Item class should be referenced in multiple places
assert len(references) > 0
# At least one reference should be in services.py (ItemService class)
# Use selectionRange only
symbols = language_server.request_document_symbols(file_path)
item_symbol = next((s for s in symbols[0] if s.get("name") == "Item"), None)
if not item_symbol or "selectionRange" not in item_symbol:
raise AssertionError("Item symbol or its selectionRange not found")
sel_start = item_symbol["selectionRange"]["start"]
references = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
services_references = [ref for ref in references if "services.py" in ref["uri"]]
assert len(services_references) > 0
assert len(services_references) > 0, "At least one reference should be in services.py (using selectionRange if present)"
def test_request_references_function_parameter(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_references_function_parameter(self, language_server: SyncLanguageServer) -> None:
"""Test request_references on a function parameter."""
# Get references to the id parameter in get_user method
file_path = os.path.join("test_repo", "services.py")
# Line 24 contains the get_user method with id parameter
references = language_server.request_references(file_path, 24, 16)
# Use selectionRange only
symbols = language_server.request_document_symbols(file_path)
get_user_symbol = next((s for s in symbols[0] if s.get("name") == "get_user"), None)
if not get_user_symbol or "selectionRange" not in get_user_symbol:
raise AssertionError("get_user symbol or its selectionRange not found")
sel_start = get_user_symbol["selectionRange"]["start"]
references = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert len(references) > 0, "id parameter should be referenced within the method (using selectionRange if present)"
# id parameter should be referenced within the method
assert len(references) > 0
def test_request_references_create_user_method(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_references_create_user_method(self, language_server: SyncLanguageServer) -> None:
# Get references to the create_user method in UserService
file_path = os.path.join("test_repo", "services.py")
# Line 15 contains the create_user method definition
references = language_server.request_references(file_path, 15, 9)
# Use selectionRange only
symbols = language_server.request_document_symbols(file_path)
create_user_symbol = next((s for s in symbols[0] if s.get("name") == "create_user"), None)
if not create_user_symbol or "selectionRange" not in create_user_symbol:
raise AssertionError("create_user symbol or its selectionRange not found")
sel_start = create_user_symbol["selectionRange"]["start"]
references = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert len(references) > 1, "Should get valid references for create_user (using selectionRange if present)"
# Verify that we get valid references
assert len(references) > 1
def test_retrieve_content_around_line(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_retrieve_content_around_line(self, language_server: SyncLanguageServer) -> None:
"""Test retrieve_content_around_line functionality with various scenarios."""
file_path = os.path.join("test_repo", "models.py")
@@ -166,7 +185,8 @@ class TestLanguageServerBasics:
else:
assert line.match_type == LineType.AFTER_MATCH
def test_search_files_for_pattern(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_search_files_for_pattern(self, language_server: SyncLanguageServer) -> None:
"""Test search_files_for_pattern with various patterns and glob filters."""
# Test 1: Search for class definitions across all files
class_pattern = r"class\s+\w+\s*(?:\([^{]*\)|:)"
test/multilspy/test_retrieval_with_ignored_dirs.py → test/multilspy/python/test_retrieval_with_ignored_dirs.py
+9 -1
View File
@@ -6,10 +6,14 @@ import pytest
from multilspy.language_server import SyncLanguageServer
from multilspy.multilspy_config import Language, MultilspyConfig
from multilspy.multilspy_logger import MultilspyLogger
from test.conftest import get_repo_path
# This mark will be applied to all tests in this module
pytestmark = pytest.mark.python
@pytest.fixture(scope="module")
def ls_with_ignored_dirs(repo_path: Path) -> Generator[SyncLanguageServer, None, None]:
def ls_with_ignored_dirs() -> Generator[SyncLanguageServer, None, None]:
"""Fixture to set up an LS for the python test repo with the 'scripts' directory ignored."""
config = MultilspyConfig(
code_language=Language.PYTHON,
@@ -17,6 +21,7 @@ def ls_with_ignored_dirs(repo_path: Path) -> Generator[SyncLanguageServer, None,
ignored_paths=["scripts", "custom_test"], # Configure the relative path to be ignored
)
logger = MultilspyLogger()
repo_path = get_repo_path(Language.PYTHON)
ls = SyncLanguageServer.create(config, logger, str(repo_path))
ls.start()
try:
@@ -25,6 +30,7 @@ def ls_with_ignored_dirs(repo_path: Path) -> Generator[SyncLanguageServer, None,
ls.stop()
@pytest.mark.parametrize("ls_with_ignored_dirs", [Language.PYTHON], indirect=True)
def test_symbol_tree_ignores_dir(ls_with_ignored_dirs: SyncLanguageServer):
"""Tests that request_full_symbol_tree ignores the configured directory."""
root = ls_with_ignored_dirs.request_full_symbol_tree()[0]
@@ -33,6 +39,7 @@ def test_symbol_tree_ignores_dir(ls_with_ignored_dirs: SyncLanguageServer):
assert children_names == {"test_repo", "examples"}
@pytest.mark.parametrize("ls_with_ignored_dirs", [Language.PYTHON], indirect=True)
def test_find_references_ignores_dir(ls_with_ignored_dirs: SyncLanguageServer):
"""Tests that find_references ignores the configured directory."""
# Location of Item, which is referenced in scripts
@@ -46,6 +53,7 @@ def test_find_references_ignores_dir(ls_with_ignored_dirs: SyncLanguageServer):
assert not any("scripts" in ref["relativePath"] for ref in references)
@pytest.mark.parametrize("repo_path", [Language.PYTHON], indirect=True)
def test_refs_and_symbols_with_glob_patterns(repo_path: Path) -> None:
"""Tests that refs and symbols with glob patterns are ignored."""
config = MultilspyConfig(
test/multilspy/test_symbol_retrieval.py → test/multilspy/python/test_symbol_retrieval.py
+69 -42
View File
@@ -8,14 +8,20 @@ These tests focus on the following methods:
import os
import pytest
from multilspy.language_server import SyncLanguageServer
from multilspy.multilspy_config import Language
from multilspy.multilspy_types import SymbolKind
pytestmark = pytest.mark.python
class TestLanguageServerSymbols:
"""Test the language server's symbol-related functionality."""
def test_request_containing_symbol_function(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_containing_symbol_function(self, language_server: SyncLanguageServer) -> None:
"""Test request_containing_symbol for a function."""
# Test for a position inside the create_user method
file_path = os.path.join("test_repo", "services.py")
@@ -29,7 +35,8 @@ class TestLanguageServerSymbols:
if "body" in containing_symbol:
assert containing_symbol["body"].strip().startswith("def create_user(self")
def test_references_to_variables(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_references_to_variables(self, language_server: SyncLanguageServer) -> None:
"""Test request_referencing_symbols for a variable."""
file_path = os.path.join("test_repo", "variables.py")
# Line 75 contains the field status that is later modified
@@ -43,7 +50,8 @@ class TestLanguageServerSymbols:
assert "dataclass_instance" in ref_names
assert "second_dataclass" in ref_names
def test_request_containing_symbol_class(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_containing_symbol_class(self, language_server: SyncLanguageServer) -> None:
"""Test request_containing_symbol for a class."""
# Test for a position inside the UserService class but outside any method
file_path = os.path.join("test_repo", "services.py")
@@ -55,7 +63,8 @@ class TestLanguageServerSymbols:
assert containing_symbol["name"] == "UserService"
assert containing_symbol["kind"] == SymbolKind.Class
def test_request_containing_symbol_nested(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_containing_symbol_nested(self, language_server: SyncLanguageServer) -> None:
"""Test request_containing_symbol with nested scopes."""
# Test for a position inside a method which is inside a class
file_path = os.path.join("test_repo", "services.py")
@@ -80,7 +89,8 @@ class TestLanguageServerSymbols:
assert parent_symbol["name"] == "UserService"
assert parent_symbol["kind"] == SymbolKind.Class
def test_request_containing_symbol_none(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_containing_symbol_none(self, language_server: SyncLanguageServer) -> None:
"""Test request_containing_symbol for a position with no containing symbol."""
# Test for a position outside any function/class (e.g., in imports)
file_path = os.path.join("test_repo", "services.py")
@@ -90,15 +100,19 @@ class TestLanguageServerSymbols:
# Should return None or an empty dictionary
assert containing_symbol is None or containing_symbol == {}
def test_request_referencing_symbols_function(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_referencing_symbols_function(self, language_server: SyncLanguageServer) -> None:
"""Test request_referencing_symbols for a function."""
# Test referencing symbols for create_user function
file_path = os.path.join("test_repo", "services.py")
# Line 15 contains the create_user function definition
ref_symbols = language_server.request_referencing_symbols(file_path, 15, 9)
# Verify we get referencing symbols
assert len(ref_symbols) > 0
symbols = language_server.request_document_symbols(file_path)
create_user_symbol = next((s for s in symbols[0] if s.get("name") == "create_user"), None)
if not create_user_symbol or "selectionRange" not in create_user_symbol:
raise AssertionError("create_user symbol or its selectionRange not found")
sel_start = create_user_symbol["selectionRange"]["start"]
ref_symbols = language_server.request_referencing_symbols(file_path, sel_start["line"], sel_start["character"])
assert len(ref_symbols) > 0, "No referencing symbols found for create_user (selectionRange)"
# Verify the structure of referencing symbols
for symbol in ref_symbols:
@@ -108,45 +122,46 @@ class TestLanguageServerSymbols:
assert "start" in symbol["location"]["range"]
assert "end" in symbol["location"]["range"]
def test_request_referencing_symbols_class(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_referencing_symbols_class(self, language_server: SyncLanguageServer) -> None:
"""Test request_referencing_symbols for a class."""
# Test referencing symbols for User class
file_path = os.path.join("test_repo", "models.py")
# Line 31 contains the User class definition
ref_symbols = language_server.request_referencing_symbols(file_path, 31, 6)
# Verify we get referencing symbols
assert len(ref_symbols) > 0
# At least one reference should be from services.py
symbols = language_server.request_document_symbols(file_path)
user_symbol = next((s for s in symbols[0] if s.get("name") == "User"), None)
if not user_symbol or "selectionRange" not in user_symbol:
raise AssertionError("User symbol or its selectionRange not found")
sel_start = user_symbol["selectionRange"]["start"]
ref_symbols = language_server.request_referencing_symbols(file_path, sel_start["line"], sel_start["character"])
services_references = [
symbol
for symbol in ref_symbols
if "location" in symbol and "uri" in symbol["location"] and "services.py" in symbol["location"]["uri"]
]
assert len(services_references) > 0, "No referencing symbols from services.py for User (selectionRange)"
assert len(services_references) > 0
def test_request_referencing_symbols_parameter(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_referencing_symbols_parameter(self, language_server: SyncLanguageServer) -> None:
"""Test request_referencing_symbols for a function parameter."""
# Test referencing symbols for id parameter in get_user
file_path = os.path.join("test_repo", "services.py")
# Line 24 contains the get_user method with id parameter
ref_symbols = language_server.request_referencing_symbols(file_path, 24, 16)
# Verify we get referencing symbols
assert len(ref_symbols) > 0
# Verify the symbols include references within the method body
symbols = language_server.request_document_symbols(file_path)
get_user_symbol = next((s for s in symbols[0] if s.get("name") == "get_user"), None)
if not get_user_symbol or "selectionRange" not in get_user_symbol:
raise AssertionError("get_user symbol or its selectionRange not found")
sel_start = get_user_symbol["selectionRange"]["start"]
ref_symbols = language_server.request_referencing_symbols(file_path, sel_start["line"], sel_start["character"])
method_refs = [
symbol
for symbol in ref_symbols
if "location" in symbol and "range" in symbol["location"] and symbol["location"]["range"]["start"]["line"] > 24
if "location" in symbol and "range" in symbol["location"] and symbol["location"]["range"]["start"]["line"] > sel_start["line"]
]
assert len(method_refs) > 0, "No referencing symbols within method body for get_user (selectionRange)"
assert len(method_refs) > 0
def test_request_referencing_symbols_none(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_referencing_symbols_none(self, language_server: SyncLanguageServer) -> None:
"""Test request_referencing_symbols for a position with no symbol."""
# For positions with no symbol, the method might throw an error or return None/empty list
# We'll modify our test to handle this by using a try-except block
@@ -163,7 +178,8 @@ class TestLanguageServerSymbols:
pass
# Tests for request_defining_symbol
def test_request_defining_symbol_variable(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_defining_symbol_variable(self, language_server: SyncLanguageServer) -> None:
"""Test request_defining_symbol for a variable usage."""
# Test finding the definition of a symbol in the create_user method
file_path = os.path.join("test_repo", "services.py")
@@ -181,7 +197,8 @@ class TestLanguageServerSymbols:
if "location" in defining_symbol and "uri" in defining_symbol["location"]:
assert "services.py" in defining_symbol["location"]["uri"]
def test_request_defining_symbol_imported_class(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_defining_symbol_imported_class(self, language_server: SyncLanguageServer) -> None:
"""Test request_defining_symbol for an imported class."""
# Test finding the definition of the 'User' class used in the UserService.create_user method
file_path = os.path.join("test_repo", "services.py")
@@ -192,7 +209,8 @@ class TestLanguageServerSymbols:
assert defining_symbol is not None
assert defining_symbol.get("name") == "User"
def test_request_defining_symbol_method_call(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_defining_symbol_method_call(self, language_server: SyncLanguageServer) -> None:
"""Test request_defining_symbol for a method call."""
# Create an example file path for a file that calls UserService.create_user
examples_file_path = os.path.join("examples", "user_management.py")
@@ -216,7 +234,8 @@ class TestLanguageServerSymbols:
warnings.warn("Could not verify method call definition - file structure may differ from expected")
def test_request_defining_symbol_none(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_defining_symbol_none(self, language_server: SyncLanguageServer) -> None:
"""Test request_defining_symbol for a position with no symbol."""
# Test for a position with no symbol (e.g., whitespace or comment)
file_path = os.path.join("test_repo", "services.py")
@@ -226,7 +245,8 @@ class TestLanguageServerSymbols:
# Should return None for positions with no symbol
assert defining_symbol is None
def test_request_containing_symbol_variable(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_containing_symbol_variable(self, language_server: SyncLanguageServer) -> None:
"""Test request_containing_symbol where the symbol is a variable."""
# Test for a position inside a variable definition
file_path = os.path.join("test_repo", "services.py")
@@ -238,7 +258,8 @@ class TestLanguageServerSymbols:
assert containing_symbol["name"] == "user_var_str"
assert containing_symbol["kind"] == SymbolKind.Variable
def test_request_defining_symbol_nested_function(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_defining_symbol_nested_function(self, language_server: SyncLanguageServer) -> None:
"""Test request_defining_symbol for a nested function or closure."""
# Use the existing nested.py file which contains nested classes and methods
file_path = os.path.join("test_repo", "nested.py")
@@ -288,7 +309,8 @@ class TestLanguageServerSymbols:
assert defining_symbol.get("name") == "func_within_func"
assert defining_symbol.get("kind") == SymbolKind.Function.value
def test_symbol_methods_integration(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_symbol_methods_integration(self, language_server: SyncLanguageServer) -> None:
"""Test the integration between different symbol-related methods."""
# This test demonstrates using the various symbol methods together
# by finding a symbol and then checking its definition
@@ -336,7 +358,8 @@ class TestLanguageServerSymbols:
warnings.warn("Could not verify container hierarchy - implementation detail")
def test_symbol_tree_structure(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_symbol_tree_structure(self, language_server: SyncLanguageServer) -> None:
"""Test that the symbol tree structure is correctly built."""
# Get all symbols in the test file
repo_structure = language_server.request_full_symbol_tree()
@@ -363,7 +386,8 @@ class TestLanguageServerSymbols:
_, user_management_roots = language_server.request_document_symbols(os.path.join("examples", "user_management.py"))
assert user_management_roots == user_management_node["children"]
def test_symbol_tree_structure_subdir(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_symbol_tree_structure_subdir(self, language_server: SyncLanguageServer) -> None:
"""Test that the symbol tree structure is correctly built."""
# Get all symbols in the test file
examples_package_roots = language_server.request_full_symbol_tree(within_relative_path="examples")
@@ -383,7 +407,8 @@ class TestLanguageServerSymbols:
_, user_management_roots = language_server.request_document_symbols(os.path.join("examples", "user_management.py"))
assert user_management_roots == user_management_node["children"]
def test_request_dir_overview(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_dir_overview(self, language_server: SyncLanguageServer) -> None:
"""Test that request_dir_overview returns correct symbol information for files in a directory."""
# Get overview of the examples directory
overview = language_server.request_dir_overview("test_repo")
@@ -407,7 +432,8 @@ class TestLanguageServerSymbols:
for symbol in expected_symbols:
assert symbol in services_symbols
def test_request_document_overview(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_request_document_overview(self, language_server: SyncLanguageServer) -> None:
"""Test that request_document_overview returns correct symbol information for a file."""
# Get overview of the user_management.py file
overview = language_server.request_document_overview(os.path.join("examples", "user_management.py"))
@@ -416,7 +442,8 @@ class TestLanguageServerSymbols:
symbol_names = {s_info[0] for s_info in overview}
assert {"UserStats", "UserManager", "process_user_data", "main"}.issubset(symbol_names)
def test_containing_symbol_of_var_is_file(self, language_server: SyncLanguageServer):
@pytest.mark.parametrize("language_server", [Language.PYTHON], indirect=True)
def test_containing_symbol_of_var_is_file(self, language_server: SyncLanguageServer) -> None:
"""Test that the containing symbol of a variable is the file itself."""
# Get the containing symbol of a variable in a file
file_path = os.path.join("test_repo", "services.py")
test/multilspy/rust/test_rust_basic.py
+57
View File
@@ -0,0 +1,57 @@
import os
import pytest
from multilspy import SyncLanguageServer
from multilspy.multilspy_config import Language
from multilspy.multilspy_utils import SymbolUtils
@pytest.mark.rust
class TestRustLanguageServer:
@pytest.mark.parametrize("language_server", [Language.RUST], indirect=True)
def test_find_references_raw(self, language_server: SyncLanguageServer) -> None:
# Directly test the request_references method for the add function
file_path = os.path.join("src", "lib.rs")
symbols = language_server.request_document_symbols(file_path)
add_symbol = None
for sym in symbols[0]:
if sym.get("name") == "add":
add_symbol = sym
break
assert add_symbol is not None, "Could not find 'add' function symbol in lib.rs"
sel_start = add_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert any(
"main.rs" in ref.get("relativePath", "") for ref in refs
), "main.rs should reference add (raw, tried all positions in selectionRange)"
@pytest.mark.parametrize("language_server", [Language.RUST], indirect=True)
def test_find_symbol(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "main"), "main function not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "add"), "add function not found in symbol tree"
# Add more as needed based on test_repo
@pytest.mark.parametrize("language_server", [Language.RUST], indirect=True)
def test_find_referencing_symbols(self, language_server: SyncLanguageServer) -> None:
# Find references to 'add' defined in lib.rs, should be referenced from main.rs
file_path = os.path.join("src", "lib.rs")
symbols = language_server.request_document_symbols(file_path)
add_symbol = None
for sym in symbols[0]:
if sym.get("name") == "add":
add_symbol = sym
break
assert add_symbol is not None, "Could not find 'add' function symbol in lib.rs"
sel_start = add_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert any(
"main.rs" in ref.get("relativePath", "") for ref in refs
), "main.rs should reference add (tried all positions in selectionRange)"
@pytest.mark.parametrize("language_server", [Language.RUST], indirect=True)
def test_overview_methods(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "main"), "main missing from overview"
assert SymbolUtils.symbol_tree_contains_name(symbols, "add"), "add missing from overview"
test/multilspy/typescript/test_typescript_basic.py
+33
View File
@@ -0,0 +1,33 @@
import os
import pytest
from multilspy import SyncLanguageServer
from multilspy.multilspy_config import Language
from multilspy.multilspy_utils import SymbolUtils
@pytest.mark.typescript
class TestTypescriptLanguageServer:
@pytest.mark.parametrize("language_server", [Language.TYPESCRIPT], indirect=True)
def test_find_symbol(self, language_server: SyncLanguageServer) -> None:
symbols = language_server.request_full_symbol_tree()
assert SymbolUtils.symbol_tree_contains_name(symbols, "DemoClass"), "DemoClass not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "helperFunction"), "helperFunction not found in symbol tree"
assert SymbolUtils.symbol_tree_contains_name(symbols, "printValue"), "printValue method not found in symbol tree"
@pytest.mark.parametrize("language_server", [Language.TYPESCRIPT], indirect=True)
def test_find_referencing_symbols(self, language_server: SyncLanguageServer) -> None:
file_path = os.path.join("index.ts")
symbols = language_server.request_document_symbols(file_path)
helper_symbol = None
for sym in symbols[0]:
if sym.get("name") == "helperFunction":
helper_symbol = sym
break
assert helper_symbol is not None, "Could not find 'helperFunction' symbol in index.ts"
sel_start = helper_symbol["selectionRange"]["start"]
refs = language_server.request_references(file_path, sel_start["line"], sel_start["character"])
assert any(
"index.ts" in ref.get("relativePath", "") for ref in refs
), "index.ts should reference helperFunction (tried all positions in selectionRange)"
test/resources/repos/go/test_repo/main.go
+20
View File
@@ -0,0 +1,20 @@
package main
import "fmt"
func main() {
fmt.Println("Hello, Go!")
Helper()
}
func Helper() {
fmt.Println("Helper function called")
}
type DemoStruct struct {
Field int
}
func UsingHelper() {
Helper()
}
test/resources/repos/java/test_repo/pom.xml
+14
View File
@@ -0,0 +1,14 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>test_repo</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Java Test Repo</name>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
</properties>
</project>
test/resources/repos/java/test_repo/src/main/java/test_repo/Main.java
+13
View File
@@ -0,0 +1,13 @@
package test_repo;
public class Main {
public static void main(String[] args) {
Utils.printHello();
Model model = new Model("Cascade");
System.out.println(model.getName());
acceptModel(model);
}
public static void acceptModel(Model m) {
// Do nothing, just for LSP reference
}
}
test/resources/repos/java/test_repo/src/main/java/test_repo/Model.java
+13
View File
@@ -0,0 +1,13 @@
package test_repo;
public class Model {
private String name;
public Model(String name) {
this.name = name;
}
public String getName() {
return name;
}
}
test/resources/repos/java/test_repo/src/main/java/test_repo/ModelUser.java
+8
View File
@@ -0,0 +1,8 @@
package test_repo;
public class ModelUser {
public static void main(String[] args) {
Model model = new Model("Cascade");
System.out.println(model.getName());
}
}
test/resources/repos/java/test_repo/src/main/java/test_repo/Utils.java
+7
View File
@@ -0,0 +1,7 @@
package test_repo;
public class Utils {
public static void printHello() {
System.out.println("Hello from Utils!");
}
}
test/resources/repos/php/test_repo/helper.php
+7
View File
@@ -0,0 +1,7 @@
<?php
function helperFunction(): void {
echo "Helper function was called.";
}
?>
test/resources/repos/php/test_repo/index.php
+20
View File
@@ -0,0 +1,20 @@
<?php
require_once 'helper.php';
function greet(string $name): string {
return "Hello, " . $name . "!";
}
$userName = "PHP User";
$greeting = greet($userName);
echo $greeting;
helperFunction();
function useHelperFunction() {
helperFunction();
}
?>
test/resources/repos/php/test_repo/simple_var.php
+4
View File
@@ -0,0 +1,4 @@
<?php
$localVar = "test";
echo $localVar;
?>
test/resources/test_repo/.gitignore → test/resources/repos/python/test_repo/.gitignore
View File
File renamed without changes.
test/resources/test_repo/custom_test/__init__.py → test/resources/repos/python/test_repo/custom_test/__init__.py
View File
File renamed without changes.
test/resources/test_repo/custom_test/advanced_features.py → test/resources/repos/python/test_repo/custom_test/advanced_features.py
View File
File renamed without changes.
test/resources/test_repo/examples/__init__.py → test/resources/repos/python/test_repo/examples/__init__.py
View File
File renamed without changes.
test/resources/test_repo/examples/user_management.py → test/resources/repos/python/test_repo/examples/user_management.py
View File
File renamed without changes.
test/resources/test_repo/scripts/__init__.py → test/resources/repos/python/test_repo/scripts/__init__.py
View File
File renamed without changes.
test/resources/test_repo/scripts/run_app.py → test/resources/repos/python/test_repo/scripts/run_app.py
+1 -1
View File
@@ -44,7 +44,7 @@ def load_config(config_path: str) -> dict[str, Any]:
return {}
try:
with open(config_path) as f:
with open(config_path, encoding="utf-8") as f:
return json.load(f)
except json.JSONDecodeError:
logger.error(f"Invalid JSON in configuration file: {config_path}")
test/resources/test_repo/test_repo/__init__.py → test/resources/repos/python/test_repo/test_repo/__init__.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/complex_types.py → test/resources/repos/python/test_repo/test_repo/complex_types.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/models.py → test/resources/repos/python/test_repo/test_repo/models.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/name_collisions.py → test/resources/repos/python/test_repo/test_repo/name_collisions.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/nested.py → test/resources/repos/python/test_repo/test_repo/nested.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/nested_base.py → test/resources/repos/python/test_repo/test_repo/nested_base.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/overloaded.py → test/resources/repos/python/test_repo/test_repo/overloaded.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/services.py → test/resources/repos/python/test_repo/test_repo/services.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/utils.py → test/resources/repos/python/test_repo/test_repo/utils.py
View File
File renamed without changes.
test/resources/test_repo/test_repo/variables.py → test/resources/repos/python/test_repo/test_repo/variables.py
View File
File renamed without changes.
test/resources/repos/rust/test_repo/Cargo.lock
Generated
+7
View File
@@ -0,0 +1,7 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 3
[[package]]
name = "rsandbox"
version = "0.1.0"
test/resources/repos/rust/test_repo/Cargo.toml
+6
View File
@@ -0,0 +1,6 @@
[package]
name = "rsandbox"
version = "0.1.0"
edition = "2021"
[dependencies]
test/resources/repos/rust/test_repo/src/lib.rs
+9
View File
@@ -0,0 +1,9 @@
// This function returns the sum of 2 + 2
pub fn add() -> i32 {
let res = 2 + 2;
res
}
pub fn multiply() -> i32 {
2 * 3
}
test/resources/repos/rust/test_repo/src/main.rs
+8
View File
@@ -0,0 +1,8 @@
use rsandbox::add;
fn main() {
println!("Hello, World!");
println!("Good morning!");
println!("add result: {}", add());
println!("inserted line");
}
test/resources/repos/typescript/test_repo/index.ts
+16
View File
@@ -0,0 +1,16 @@
export class DemoClass {
value: number;
constructor(value: number) {
this.value = value;
}
printValue() {
console.log(this.value);
}
}
export function helperFunction() {
const demo = new DemoClass(42);
demo.printValue();
}
helperFunction();
test/resources/repos/typescript/test_repo/tsconfig.json
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"target": "ES2017",
"module": "commonjs",
"strict": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"skipLibCheck": true
},
"include": ["**/*.ts"]
}
test/resources/repos/typescript/test_repo/use_helper.ts
+8
View File
@@ -0,0 +1,8 @@
import {helperFunction} from "./index";
export function useHelper() {
helperFunction();
}
useHelper();
test/resources/test_repo/ignore_this_dir_with_postfix/ignored_module.py
-141
View File
@@ -1,141 +0,0 @@
"""
Example demonstrating user management with the test_repo module.
This example showcases:
- Creating and managing users
- Using various object types and relationships
- Type annotations and complex Python patterns
"""
import logging
from dataclasses import dataclass
from typing import Any
from test_repo.models import User, create_user_object
from test_repo.services import UserService
# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
@dataclass
class UserStats:
"""Statistics about user activity."""
user_id: str
login_count: int = 0
last_active_days: int = 0
engagement_score: float = 0.0
def is_active(self) -> bool:
"""Check if the user is considered active."""
return self.last_active_days < 30
class UserManager:
"""Example class demonstrating complex user management."""
def __init__(self, service: UserService):
self.service = service
self.active_users: dict[str, User] = {}
self.user_stats: dict[str, UserStats] = {}
def register_user(self, name: str, email: str, roles: list[str] | None = None) -> User:
"""Register a new user."""
logger.info(f"Registering new user: {name} ({email})")
user = self.service.create_user(name=name, email=email, roles=roles)
self.active_users[user.id] = user
self.user_stats[user.id] = UserStats(user_id=user.id)
return user
def get_user(self, user_id: str) -> User | None:
"""Get a user by ID."""
if user_id in self.active_users:
return self.active_users[user_id]
# Try to fetch from service
user = self.service.get_user(user_id)
if user:
self.active_users[user.id] = user
return user
def update_user_stats(self, user_id: str, login_count: int, days_since_active: int) -> None:
"""Update statistics for a user."""
if user_id not in self.user_stats:
self.user_stats[user_id] = UserStats(user_id=user_id)
stats = self.user_stats[user_id]
stats.login_count = login_count
stats.last_active_days = days_since_active
# Calculate engagement score based on activity
engagement = (100 - min(days_since_active, 100)) * 0.8
engagement += min(login_count, 20) * 0.2
stats.engagement_score = engagement
def get_active_users(self) -> list[User]:
"""Get all active users."""
active_user_ids = [user_id for user_id, stats in self.user_stats.items() if stats.is_active()]
return [self.active_users[user_id] for user_id in active_user_ids if user_id in self.active_users]
def get_user_by_email(self, email: str) -> User | None:
"""Find a user by their email address."""
for user in self.active_users.values():
if user.email == email:
return user
return None
# Example function demonstrating type annotations
def process_user_data(users: list[User], include_inactive: bool = False, transform_func: callable | None = None) -> dict[str, Any]:
"""Process user data with optional transformations."""
result: dict[str, Any] = {"users": [], "total": 0, "admin_count": 0}
for user in users:
if transform_func:
user_data = transform_func(user.to_dict())
else:
user_data = user.to_dict()
result["users"].append(user_data)
result["total"] += 1
if "admin" in user.roles:
result["admin_count"] += 1
return result
def main():
"""Main function demonstrating the usage of UserManager."""
# Initialize service and manager
service = UserService()
manager = UserManager(service)
# Register some users
admin = manager.register_user("Admin User", "admin@example.com", ["admin"])
user1 = manager.register_user("Regular User", "user@example.com", ["user"])
user2 = manager.register_user("Another User", "another@example.com", ["user"])
# Update some stats
manager.update_user_stats(admin.id, 100, 5)
manager.update_user_stats(user1.id, 50, 10)
manager.update_user_stats(user2.id, 10, 45) # Inactive user
# Get active users
active_users = manager.get_active_users()
logger.info(f"Active users: {len(active_users)}")
# Process user data
user_data = process_user_data(active_users, transform_func=lambda u: {**u, "full_name": u.get("name", "")})
logger.info(f"Processed {user_data['total']} users, {user_data['admin_count']} admins")
# Example of calling create_user directly
external_user = create_user_object(id="ext123", name="External User", email="external@example.org", roles=["external"])
logger.info(f"Created external user: {external_user.name}")
if __name__ == "__main__":
main()
test/serena/__init__.py
+1
View File
@@ -0,0 +1 @@
test/serena/test_serena_agent.py
+218
View File
@@ -0,0 +1,218 @@
import json
import os
import time
import pytest
from multilspy.multilspy_config import Language
from serena.agent import FindReferencingSymbolsTool, FindSymbolTool, ProjectConfig, SerenaAgent, SerenaConfig
from test.conftest import LanguageParamRequest, get_repo_path
@pytest.fixture
def serena_config():
return SerenaConfig()
@pytest.fixture
def serena_agent(request: LanguageParamRequest, serena_config) -> SerenaAgent:
language = Language(request.param)
repo = get_repo_path(language)
project_config = ProjectConfig(project_name=os.path.basename(str(repo)), language=language, project_root=str(repo))
return SerenaAgent(project_config=project_config, serena_config=serena_config)
class TestSerenaAgent:
@pytest.mark.parametrize(
"serena_agent,symbol_name,expected_kind,expected_file",
[
pytest.param(Language.PYTHON, "User", "Class", "models.py", marks=pytest.mark.python),
pytest.param(Language.GO, "Helper", "Function", "main.go", marks=pytest.mark.go),
pytest.param(Language.JAVA, "Model", "Class", "Model.java", marks=pytest.mark.java),
pytest.param(Language.RUST, "add", "Function", "lib.rs", marks=pytest.mark.rust),
pytest.param(Language.TYPESCRIPT, "DemoClass", "Class", "index.ts", marks=pytest.mark.typescript),
pytest.param(Language.PHP, "helperFunction", "Function", "helper.php", marks=pytest.mark.php),
],
indirect=["serena_agent"],
)
def test_find_symbol(self, serena_agent: SerenaAgent, symbol_name: str, expected_kind: str, expected_file: str):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(symbol_name)
symbols = json.loads(result)
assert any(
symbol_name in s["name"] and expected_kind.lower() in s["kind"].lower() and expected_file in s["location"]["relative_path"]
for s in symbols
), f"Expected to find {symbol_name} ({expected_kind}) in {expected_file} for {agent.project_config.language.name}"
@pytest.mark.parametrize(
"serena_agent,symbol_name,def_file,ref_file",
[
pytest.param(
Language.PYTHON,
"User",
os.path.join("test_repo", "models.py"),
os.path.join("test_repo", "services.py"),
marks=pytest.mark.python,
),
pytest.param(Language.GO, "Helper", "main.go", "main.go", marks=pytest.mark.go),
pytest.param(
Language.JAVA,
"Model",
os.path.join("src", "main", "java", "test_repo", "Model.java"),
os.path.join("src", "main", "java", "test_repo", "Main.java"),
marks=pytest.mark.java,
),
pytest.param(Language.RUST, "add", os.path.join("src", "lib.rs"), os.path.join("src", "main.rs"), marks=pytest.mark.rust),
pytest.param(Language.TYPESCRIPT, "helperFunction", "index.ts", "use_helper.ts", marks=pytest.mark.typescript),
pytest.param(Language.PHP, "helperFunction", "helper.php", "index.php", marks=pytest.mark.php),
],
indirect=["serena_agent"],
)
def test_find_symbol_references(self, serena_agent: SerenaAgent, symbol_name: str, def_file: str, ref_file: str) -> None:
agent = serena_agent
# Find the symbol location first
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(symbol_name, within_relative_path=def_file)
time.sleep(1)
symbols = json.loads(result)
# Find the definition location
def_symbol = symbols[0]
loc = def_symbol["location"]
# sel_start = def_symbol["location"]["selectionRange"]["start"]
# Now find references
find_refs_tool = agent.get_tool(FindReferencingSymbolsTool)
result = find_refs_tool.apply(def_file, loc["line"], loc["column"])
refs = json.loads(result)
assert any(
ref["location"]["relative_path"] == ref_file for ref in refs
), f"Expected to find reference to {symbol_name} in {ref_file} for {agent.project_config.language.name}. refs={refs}"
@pytest.mark.parametrize(
"serena_agent,name_path,substring_matching,expected_symbol_name,expected_kind,expected_file",
[
pytest.param(
Language.PYTHON,
"OuterClass/NestedClass",
False,
"NestedClass",
"Class",
os.path.join("test_repo", "nested.py"),
id="exact_qualname_class",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"OuterClass/NestedClass/find_me",
False,
"find_me",
"Method",
os.path.join("test_repo", "nested.py"),
id="exact_qualname_method",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"OuterClass/NestedCl", # Substring for NestedClass
True,
"NestedClass",
"Class",
os.path.join("test_repo", "nested.py"),
id="substring_qualname_class",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"OuterClass/NestedClass/find_m", # Substring for find_me
True,
"find_me",
"Method",
os.path.join("test_repo", "nested.py"),
id="substring_qualname_method",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"/OuterClass", # Absolute path
False,
"OuterClass",
"Class",
os.path.join("test_repo", "nested.py"),
id="absolute_qualname_class",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"/OuterClass/NestedClass/find_m", # Absolute path with substring
True,
"find_me",
"Method",
os.path.join("test_repo", "nested.py"),
id="absolute_substring_qualname_method",
marks=pytest.mark.python,
),
],
indirect=["serena_agent"],
)
def test_find_symbol_name_path(
self,
serena_agent: SerenaAgent,
name_path: str,
substring_matching: bool,
expected_symbol_name: str,
expected_kind: str,
expected_file: str,
):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(
name_path=name_path,
depth=0,
within_relative_path=None,
include_body=False,
include_kinds=None,
exclude_kinds=None,
substring_matching=substring_matching,
)
symbols = json.loads(result)
assert any(
expected_symbol_name == s["name"]
and expected_kind.lower() in s["kind"].lower()
and expected_file in s["location"]["relative_path"]
for s in symbols
), f"Expected to find {name_path} ({expected_kind}) in {expected_file} for {agent.project_config.language.name}. Symbols: {symbols}"
@pytest.mark.parametrize(
"serena_agent,name_path",
[
pytest.param(
Language.PYTHON,
"/NestedClass", # Absolute path, NestedClass is not top-level
id="absolute_path_non_top_level_no_match",
marks=pytest.mark.python,
),
pytest.param(
Language.PYTHON,
"/NoSuchParent/NestedClass", # Absolute path with non-existent parent
id="absolute_path_non_existent_parent_no_match",
marks=pytest.mark.python,
),
],
indirect=["serena_agent"],
)
def test_find_symbol_name_path_no_match(
self,
serena_agent: SerenaAgent,
name_path: str,
):
agent = serena_agent
find_symbol_tool = agent.get_tool(FindSymbolTool)
result = find_symbol_tool.apply(
name_path=name_path,
depth=0,
substring_matching=True,
)
symbols = json.loads(result)
assert (
not symbols
), f"Expected to find no symbols for {name_path} for {agent.project_config.language.name}. Symbols found: {symbols}"
test/serena/test_symbol.py
+162
View File
@@ -0,0 +1,162 @@
import pytest
from src.serena.symbol import Symbol
class TestSymbolNameMatching:
def _create_assertion_error_message(
self,
name_path_pattern: str,
symbol_name_path_parts: list[str],
is_substring_match: bool,
expected_result: bool,
actual_result: bool,
) -> str:
"""Helper to create a detailed error message for assertions."""
qnp_repr = "/".join(symbol_name_path_parts)
return (
f"Pattern '{name_path_pattern}' (substring: {is_substring_match}) vs "
f"Qualname parts {symbol_name_path_parts} (as '{qnp_repr}'). "
f"Expected: {expected_result}, Got: {actual_result}"
)
@pytest.mark.parametrize(
"name_path_pattern, symbol_name_path_parts, is_substring_match, expected",
[
# Exact matches, anywhere in the name (is_substring_match=False)
pytest.param("foo", ["foo"], False, True, id="'foo' matches 'foo' exactly (simple)"),
pytest.param("foo/", ["foo"], False, True, id="'foo/' matches 'foo' exactly (simple)"),
pytest.param("foo", ["bar", "foo"], False, True, id="'foo' matches ['bar', 'foo'] exactly (simple, last element)"),
pytest.param("foo", ["foobar"], False, False, id="'foo' does not match 'foobar' exactly (simple)"),
pytest.param(
"foo", ["bar", "foobar"], False, False, id="'foo' does not match ['bar', 'foobar'] exactly (simple, last element)"
),
pytest.param(
"foo", ["path", "to", "foo"], False, True, id="'foo' matches ['path', 'to', 'foo'] exactly (simple, last element)"
),
# Exact matches, absolute patterns (is_substring_match=False)
pytest.param("/foo", ["foo"], False, True, id="'/foo' matches ['foo'] exactly (absolute simple)"),
pytest.param("/foo", ["foo", "bar"], False, False, id="'/foo' does not match ['foo', 'bar'] (absolute simple, len mismatch)"),
pytest.param("/foo", ["bar"], False, False, id="'/foo' does not match ['bar'] (absolute simple, name mismatch)"),
pytest.param(
"/foo", ["bar", "foo"], False, False, id="'/foo' does not match ['bar', 'foo'] (absolute simple, position mismatch)"
),
# Substring matches, anywhere in the name (is_substring_match=True)
pytest.param("foo", ["foobar"], True, True, id="'foo' matches 'foobar' as substring (simple)"),
pytest.param("foo", ["bar", "foobar"], True, True, id="'foo' matches ['bar', 'foobar'] as substring (simple, last element)"),
pytest.param(
"foo", ["barfoo"], True, True, id="'foo' matches 'barfoo' as substring (simple)"
), # This was potentially ambiguous before
pytest.param("foo", ["baz"], True, False, id="'foo' does not match 'baz' as substring (simple)"),
pytest.param("foo", ["bar", "baz"], True, False, id="'foo' does not match ['bar', 'baz'] as substring (simple, last element)"),
pytest.param("foo", ["my_foobar_func"], True, True, id="'foo' matches 'my_foobar_func' as substring (simple)"),
pytest.param(
"foo",
["ClassA", "my_foobar_method"],
True,
True,
id="'foo' matches ['ClassA', 'my_foobar_method'] as substring (simple, last element)",
),
pytest.param("foo", ["my_bar_func"], True, False, id="'foo' does not match 'my_bar_func' as substring (simple)"),
# Substring matches, absolute patterns (is_substring_match=True)
pytest.param("/foo", ["foobar"], True, True, id="'/foo' matches ['foobar'] as substring (absolute simple)"),
pytest.param("/foo/", ["foobar"], True, True, id="'/foo/' matches ['foobar'] as substring (absolute simple, last element)"),
pytest.param("/foo", ["barfoobaz"], True, True, id="'/foo' matches ['barfoobaz'] as substring (absolute simple)"),
pytest.param(
"/foo", ["foo", "bar"], True, False, id="'/foo' does not match ['foo', 'bar'] as substring (absolute simple, len mismatch)"
),
pytest.param("/foo", ["bar"], True, False, id="'/foo' does not match ['bar'] (absolute simple, no substr)"),
pytest.param(
"/foo", ["bar", "foo"], True, False, id="'/foo' does not match ['bar', 'foo'] (absolute simple, position mismatch)"
),
pytest.param(
"/foo/", ["bar", "foo"], True, False, id="'/foo/' does not match ['bar', 'foo'] (absolute simple, position mismatch)"
),
],
)
def test_match_simple_name(self, name_path_pattern, symbol_name_path_parts, is_substring_match, expected):
"""Tests matching for simple names (no '/' in pattern)."""
result = Symbol.match_name_path(name_path_pattern, symbol_name_path_parts, is_substring_match)
error_msg = self._create_assertion_error_message(name_path_pattern, symbol_name_path_parts, is_substring_match, expected, result)
assert result == expected, error_msg
@pytest.mark.parametrize(
"name_path_pattern, symbol_name_path_parts, is_substring_match, expected",
[
# --- Relative patterns (suffix matching) ---
# Exact matches, relative patterns (is_substring_match=False)
pytest.param("bar/foo", ["bar", "foo"], False, True, id="R: 'bar/foo' matches ['bar', 'foo'] exactly"),
pytest.param("bar/foo", ["mod", "bar", "foo"], False, True, id="R: 'bar/foo' matches ['mod', 'bar', 'foo'] exactly (suffix)"),
pytest.param(
"bar/foo", ["bar", "foo", "baz"], False, False, id="R: 'bar/foo' does not match ['bar', 'foo', 'baz'] (pattern shorter)"
),
pytest.param("bar/foo", ["bar"], False, False, id="R: 'bar/foo' does not match ['bar'] (pattern longer)"),
pytest.param("bar/foo", ["baz", "foo"], False, False, id="R: 'bar/foo' does not match ['baz', 'foo'] (first part mismatch)"),
pytest.param("bar/foo", ["bar", "baz"], False, False, id="R: 'bar/foo' does not match ['bar', 'baz'] (last part mismatch)"),
pytest.param("bar/foo", ["foo"], False, False, id="R: 'bar/foo' does not match ['foo'] (pattern longer)"),
pytest.param(
"bar/foo", ["other", "foo"], False, False, id="R: 'bar/foo' does not match ['other', 'foo'] (first part mismatch)"
),
pytest.param(
"bar/foo", ["bar", "otherfoo"], False, False, id="R: 'bar/foo' does not match ['bar', 'otherfoo'] (last part mismatch)"
),
# Substring matches, relative patterns (is_substring_match=True)
pytest.param("bar/foo", ["bar", "foobar"], True, True, id="R: 'bar/foo' matches ['bar', 'foobar'] as substring"),
pytest.param(
"bar/foo", ["mod", "bar", "foobar"], True, True, id="R: 'bar/foo' matches ['mod', 'bar', 'foobar'] as substring (suffix)"
),
pytest.param("bar/foo", ["bar", "bazfoo"], True, True, id="R: 'bar/foo' matches ['bar', 'bazfoo'] as substring"),
pytest.param("bar/fo", ["bar", "foo"], True, True, id="R: 'bar/fo' matches ['bar', 'foo'] as substring"),
pytest.param("bar/foo", ["bar", "baz"], True, False, id="R: 'bar/foo' does not match ['bar', 'baz'] (last no substr)"),
pytest.param(
"bar/foo", ["baz", "foobar"], True, False, id="R: 'bar/foo' does not match ['baz', 'foobar'] (first part mismatch)"
),
pytest.param(
"bar/foo", ["bar", "my_foobar_method"], True, True, id="R: 'bar/foo' matches ['bar', 'my_foobar_method'] as substring"
),
pytest.param(
"bar/foo",
["mod", "bar", "my_foobar_method"],
True,
True,
id="R: 'bar/foo' matches ['mod', 'bar', 'my_foobar_method'] as substring (suffix)",
),
pytest.param(
"bar/foo",
["bar", "another_method"],
True,
False,
id="R: 'bar/foo' does not match ['bar', 'another_method'] (last no substr)",
),
pytest.param(
"bar/foo",
["other", "my_foobar_method"],
True,
False,
id="R: 'bar/foo' does not match ['other', 'my_foobar_method'] (first part mismatch)",
),
pytest.param("bar/f", ["bar", "foo"], True, True, id="R: 'bar/f' matches ['bar', 'foo'] as substring"),
# Exact matches, absolute patterns (is_substring_match=False)
pytest.param("/bar/foo", ["bar", "foo"], False, True, id="A: '/bar/foo' matches ['bar', 'foo'] exactly"),
pytest.param(
"/bar/foo", ["bar", "foo", "baz"], False, False, id="A: '/bar/foo' does not match ['bar', 'foo', 'baz'] (pattern shorter)"
),
pytest.param("/bar/foo", ["bar"], False, False, id="A: '/bar/foo' does not match ['bar'] (pattern longer)"),
pytest.param("/bar/foo", ["baz", "foo"], False, False, id="A: '/bar/foo' does not match ['baz', 'foo'] (first part mismatch)"),
pytest.param("/bar/foo", ["bar", "baz"], False, False, id="A: '/bar/foo' does not match ['bar', 'baz'] (last part mismatch)"),
# Substring matches (is_substring_match=True)
pytest.param("/bar/foo", ["bar", "foobar"], True, True, id="A: '/bar/foo' matches ['bar', 'foobar'] as substring"),
pytest.param("/bar/foo", ["bar", "bazfoo"], True, True, id="A: '/bar/foo' matches ['bar', 'bazfoo'] as substring"),
pytest.param("/bar/fo", ["bar", "foo"], True, True, id="A: '/bar/fo' matches ['bar', 'foo'] as substring"),
pytest.param("/bar/foo", ["bar", "baz"], True, False, id="A: '/bar/foo' does not match ['bar', 'baz'] (last no substr)"),
pytest.param(
"/bar/foo", ["baz", "foobar"], True, False, id="A: '/bar/foo' does not match ['baz', 'foobar'] (first part mismatch)"
),
],
)
def test_match_name_path_pattern_path_len_2(self, name_path_pattern, symbol_name_path_parts, is_substring_match, expected):
"""Tests matching for qualified names (e.g. 'module/class/func')."""
result = Symbol.match_name_path(name_path_pattern, symbol_name_path_parts, is_substring_match)
error_msg = self._create_assertion_error_message(name_path_pattern, symbol_name_path_parts, is_substring_match, expected, result)
assert result == expected, error_msg
uv.lock
Generated
+314 -354
View File
File diff suppressed because it is too large. Load diff