diff --git a/CHANGELOG.md b/CHANGELOG.md index 492ab00c..60597c85 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,14 +7,26 @@ Status of the `main` branch. Changes prior to the next official version change w The dict is forwarded to vtsls via `initializationOptions`, `workspace/didChangeConfiguration`, and `workspace/configuration` pulls. Enables Yarn PnP setups with `typescript.tsdk` pointing at the Yarn-generated SDK. + - `SvelteLanguageServer`: Fix diagnostics requests for TypeScript/JavaScript files incorrectly being + processed by the Svelte LS instead of the TypeScript LS. + - Improve quoting of arguments in shell executions - `typescript` / `typescript_vts`: No longer ignore directories named `coverage`. This was intended to skip coverage-report output, but matched by bare dirname and so also hid legitimate source directories named `coverage` (e.g. `src/routes/coverage/`) from symbol tools. Generated report dirs are already covered by gitignore. Fixes #1523. -* Dashboard: - - Fix: Host validation required a local host regardless of the listen address (regression introduced in v1.5.2), - preventing remote connections +* JetBrains: + - Add configuration option `jetbrains_launch_command`, allowing Serena to spawn IDE instances automatically + upon project activation + +* Dashboard: + - Make list of trusted hosts configurable, fixing host validation introduced in v1.5.2 allowing only + default local hostnames, effectively preventing remote connections + - Decouple configuration computation from the agent's task queue by introducing events for agent config/status updates. + This allows the dashboard to display the configuration while the project provided at startup is still initialising. #1064 + - Fix empty executions queue displaying "Loading..." + - Tray manager: Add NixOS-support for AppIndicator-based trays (e.g., most Wayland-trays) to the package in flake.nix. + - Fix: Wait for the subprocess that opens the browser window, preventing zombie processes #1488 # v1.5.3 (2026-05-26) diff --git a/docs/02-usage/025_jetbrains_plugin.md b/docs/02-usage/025_jetbrains_plugin.md index 4eb08688..73638445 100644 --- a/docs/02-usage/025_jetbrains_plugin.md +++ b/docs/02-usage/025_jetbrains_plugin.md @@ -84,7 +84,7 @@ language_backend: JetBrains Note that the file might not exist yet if you never executed Serena before. -**Per-Server Instance Configuration**. +**Per-Instance Configuration**. The configuration setting in the global config file can be overridden on a per-instance basis by providing the arguments `--language-backend JetBrains` when launching the Serena MCP server. @@ -121,8 +121,12 @@ You will also notice that your client will use the JetBrains-specific tools like Having installed the plugin in your IDE and having configured Serena to use the JetBrains backend, the general workflow is simple: + 1. Open the project you want to work on in your JetBrains IDE. - Note that the project must be appropriately set up in your IDE, i.e. symbol lookups for all relevant programming languages and frameworks should work in the IDE. + Note that the project must be appropriately set up in your IDE, i.e. symbol lookups for all relevant programming languages and frameworks should work in the IDE. + + You can optionally make Serena open an IDE instance for your project root folder automatically upon project activation, allowing you to skip this step for a project that was previously set up correctly. + To enable this, configure `jetbrains_launch_command` in [Serena's global configuration file](global-config) appropriately. 2. Activate the project's root folder as a project in Serena (see [Project Creation](project-creation-indexing) and [Project Activation](project-activation)). 3. Start using Serena's tools as usual. diff --git a/docs/02-usage/060_dashboard.md b/docs/02-usage/060_dashboard.md index ae11fd4d..50a1dc16 100644 --- a/docs/02-usage/060_dashboard.md +++ b/docs/02-usage/060_dashboard.md @@ -25,7 +25,7 @@ The dashboard is a web application, which is opened in one of three ways: * via a tray manager that aggregates multiple instances (on supported platforms) * via your default web browser (supported on all platforms) -Configure your preferred interface via the setting `web_dashoard_interface` in [Serena's global configuration file](global-config) (see below). +Configure your preferred interface via the setting `web_dashboard_interface` in [Serena's global configuration file](global-config) (see below). By default, the dashboard can be accessed at `http://localhost:24282/dashboard/index.html`, but a higher port may be used if the default port is unavailable/multiple instances are running. @@ -82,4 +82,4 @@ Via its menu, it furthermore allows you to The tool is mainly supported on Windows, but it may work on some Linux systems as well (depending on your desktop environment). -To enable the tool, set `gui_log_window` to `true` in Serena's [global configuration file](global-config). \ No newline at end of file +To enable the tool, set `gui_log_window` to `true` in Serena's [global configuration file](global-config). diff --git a/docs/02-usage/999_additional-usage.md b/docs/02-usage/999_additional-usage.md index 275fd190..d4da7ce6 100644 --- a/docs/02-usage/999_additional-usage.md +++ b/docs/02-usage/999_additional-usage.md @@ -3,30 +3,14 @@ ## Prompting Strategies We found that it is often a good idea to spend some time conceptualizing and planning a task -before actually implementing it, especially for non-trivial task. This helps both in achieving -better results and in increasing the feeling of control and staying in the loop. You can -make a detailed plan in one session, where Serena may read a lot of your code to build up the context, -and then continue with the implementation in another (potentially after creating suitable memories). - -## Running Out of Context - -For long and complicated tasks, or tasks where Serena has read a lot of content, you -may come close to the limits of context tokens. In that case, it is often a good idea to continue -in a new conversation. Serena has a dedicated tool to create a summary of the current state -of the progress and all relevant info for continuing it. You can request to create this summary and -write it to a memory. Then, in a new conversation, you can just ask Serena to read the memory and -continue with the task. In our experience, this worked really well. On the up-side, since in a -single session there is no summarization involved, Serena does not usually get lost (unlike some -other agents that summarize under the hood), and it is also instructed to occasionally check whether -it's on the right track. - -Serena instructs the LLM to be economical in general, so the problem of running out of context -should not occur too often, unless the task is very large or complicated. +before actually implementing it, especially for non-trivial tasks. +For very complex tasks, you can make a detailed plan in one session, +where Serena may read a lot of your code to build up the context, +and then continue with the implementation in another, +having persisted the plan in a memory or dedicated file. ## Serena and Git Worktrees [git-worktree](https://git-scm.com/docs/git-worktree) can be an excellent way to parallelize your work. More on this in [Anthropic: Run parallel Claude Code sessions with Git worktrees](https://docs.claude.com/en/docs/claude-code/common-workflows#run-parallel-claude-code-sessions-with-git-worktrees). -When it comes to serena AND git-worktree AND larger projects (that take longer to index), -the recommended way is to COPY your `$ORIG_PROJECT/.serena/cache` to `$GIT_WORKTREE/.serena/cache`. -Perform [pre-indexing of your project](indexing) to avoid having to re-index per each worktree you create. +Be sure to add the `.serena` folder to version control, such that your project-specific settings and memories are available across worktrees. diff --git a/flake.nix b/flake.nix index 9550e4ff..3cdd06f7 100644 --- a/flake.nix +++ b/flake.nix @@ -50,27 +50,79 @@ sourcePreference = "wheel"; # or sourcePreference = "sdist"; }; - pyprojectOverrides = final: prev: { - # Add setuptools for packages that need it during build - ruamel-yaml-clib = prev.ruamel-yaml-clib.overrideAttrs (old: { - nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ - final.setuptools - ]; - }); - proxy-tools = prev.proxy-tools.overrideAttrs (old: { - nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ - final.setuptools - ]; - }); - pywebview = prev.pywebview.overrideAttrs (old: { - nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ - final.setuptools - final.setuptools-scm - ]; - }); - }; - python = pkgs.python311; + pyprojectHacks = pkgs.callPackage pyproject-nix.build.hacks {}; + + pyprojectOverrides = final: prev: + { + # Add setuptools for packages that need it during build + ruamel-yaml-clib = prev.ruamel-yaml-clib.overrideAttrs (old: { + nativeBuildInputs = + (old.nativeBuildInputs or []) + ++ [ + final.setuptools + ]; + }); + proxy-tools = prev.proxy-tools.overrideAttrs (old: { + nativeBuildInputs = + (old.nativeBuildInputs or []) + ++ [ + final.setuptools + ]; + }); + pywebview = prev.pywebview.overrideAttrs (old: { + nativeBuildInputs = + (old.nativeBuildInputs or []) + ++ [ + final.setuptools + final.setuptools-scm + ]; + }); + } + // lib.optionalAttrs pkgs.stdenv.isLinux { + # pyproject-nix's virtualenv resolver follows pyproject-style + # passthru.dependencies, while these GI bindings are already packaged + # by nixpkgs' Python infrastructure. nixpkgsPrebuilt is the upstream + # adapter for that case: + # https://pyproject-nix.github.io/pyproject.nix/builders/hacks.html#using-prebuilt-packages-from-nixpkgs + pycairo = pyprojectHacks.nixpkgsPrebuilt { + from = python.pkgs.pycairo; + prev = { + passthru = { + dependencies = {}; + optional-dependencies = {}; + dependency-groups = {}; + }; + }; + }; + pygobject3 = pyprojectHacks.nixpkgsPrebuilt { + from = python.pkgs.pygobject3; + prev = { + passthru = { + dependencies = { + pycairo = []; + }; + optional-dependencies = {}; + dependency-groups = {}; + }; + }; + }; + + # pystray has an optional AppIndicator backend on Linux. Its uv lock + # metadata does not encode the GI dependency, so add the runtime edge + # here while keeping pystray itself lockfile-driven. + pystray = prev.pystray.overrideAttrs (old: { + passthru = + (old.passthru or {}) + // { + dependencies = + (old.passthru.dependencies or {}) + // { + pygobject3 = []; + }; + }; + }); + }; pythonSet = (pkgs.callPackage pyproject-nix.build.packages { @@ -88,20 +140,48 @@ packages = { serena-env = pythonSet.mkVirtualEnv "serena-env" workspace.deps.default; - serena = pkgs.runCommand "serena" { - meta = { - description = "A powerful coding agent toolkit providing semantic retrieval and editing capabilities (MCP server & Agno integration)"; - homepage = "https://oraios.github.io/serena"; - changelog = "https://github.com/oraios/serena/blob/main/CHANGELOG.md"; - mainProgram = "serena"; - license = pkgs.lib.licenses.mit; - platforms = lib.platforms.all; - }; - } '' - mkdir -p $out/bin - ln -s ${packages.serena-env}/bin/serena $out/bin/serena - ln -s ${packages.serena-env}/bin/serena-hooks $out/bin/serena-hooks - ''; + serena = pkgs.stdenv.mkDerivation { + name = "serena"; + dontUnpack = true; + dontWrapGApps = true; + nativeBuildInputs = + [pkgs.makeWrapper] + ++ lib.optionals pkgs.stdenv.isLinux [ + pkgs.wrapGAppsHook3 + pkgs.gobject-introspection + ]; + buildInputs = lib.optionals pkgs.stdenv.isLinux [ + pkgs.gtk3 + pkgs.libayatana-appindicator + ]; + installPhase = '' + runHook preInstall + + mkdir -p $out/bin + ${ + lib.optionalString (!pkgs.stdenv.isLinux) '' + ln -s ${packages.serena-env}/bin/serena $out/bin/serena + '' + } + ln -s ${packages.serena-env}/bin/serena-hooks $out/bin/serena-hooks + + runHook postInstall + ''; + # Run the GApps fixup phase so the wrapper gets GI_TYPELIB_PATH for + # Gtk and AyatanaAppIndicator3. This is the same pattern nixpkgs uses + # for pystray consumers such as plex-mpv-shim and jellyfin-mpv-shim. + preFixup = lib.optionalString pkgs.stdenv.isLinux '' + makeWrapper ${packages.serena-env}/bin/serena $out/bin/serena "''${gappsWrapperArgs[@]}" + ''; + meta = { + description = "A powerful coding agent toolkit providing semantic retrieval and editing capabilities (MCP server & Agno integration)"; + homepage = "https://oraios.github.io/serena"; + changelog = "https://github.com/oraios/serena/blob/main/CHANGELOG.md"; + mainProgram = "serena"; + license = pkgs.lib.licenses.mit; + platforms = lib.platforms.all; + }; + }; default = packages.serena; }; diff --git a/src/serena/agent.py b/src/serena/agent.py index ce4c1191..bef1b201 100644 --- a/src/serena/agent.py +++ b/src/serena/agent.py @@ -7,6 +7,7 @@ import multiprocessing import os import platform import signal +import subprocess import threading from collections import defaultdict from collections.abc import Callable, Iterator, Sequence @@ -40,6 +41,7 @@ from serena.config.serena_config import ( ToolInclusionDefinition, ) from serena.dashboard import SerenaDashboardAPI, SerenaDashboardTrayManager, SerenaDashboardViewer, open_url_in_browser +from serena.jetbrains import jetbrains_plugin_client from serena.ls_manager import LanguageServerManager from serena.memories.memory_manager import MemoryManager from serena.project import Project @@ -60,6 +62,7 @@ from serena.util.gui import system_has_usable_display from serena.util.inspection import iter_subclasses from serena.util.logging import MemoryLogHandler from solidlsp.ls_config import Language +from solidlsp.util import subprocess_util if TYPE_CHECKING: from serena.gui_log_viewer import GuiLogViewer @@ -544,6 +547,7 @@ class SerenaAgent: self._project_prompt_status = ProjectPromptProvisionStatus() self._session_mode_selection_definition = modes self.version = serena_version() + self._config_changed_callbacks: list[Callable[[], None]] = [] # obtain serena configuration using the decoupled factory function self.serena_config = serena_config or SerenaConfig.from_config_file() @@ -633,7 +637,7 @@ class SerenaAgent: # create executor for starting the language server and running tools in another thread # This executor is used to achieve linear task execution - self._task_executor = TaskExecutor("SerenaAgentTaskExecutor") + self._task_executor = TaskExecutor("SerenaAgentTaskExecutor", self._task_completion_callback) # Initialize the prompt factory self.prompt_factory = SerenaPromptFactory() @@ -659,17 +663,29 @@ class SerenaAgent: self._active_tools: AvailableTools self._update_active_tools() - # start the dashboard (web frontend), registering its log handler - # should be the last thing to happen in the initialization since the dashboard - # may access various parts of the agent + # create the dashboard backend (if enabled), which will register callback. + dashboard_api: SerenaDashboardAPI | None = None if self.serena_config.web_dashboard: - self._dashboard_thread, port = SerenaDashboardAPI( + dashboard_api = SerenaDashboardAPI( get_memory_log_handler(), tool_names, agent=self, tool_usage_stats=self._tool_usage_stats, host=self.serena_config.web_dashboard_listen_address, - ).run_in_thread() + trusted_hosts=self.serena_config.web_dashboard_trusted_hosts, + ) + + # propagate the initial config/state to listeners, particularly to the dashboard API. + # Note: The only ongoing task can be the LS initialisation, which does not change the config + # This is executed before starting the dashboard thread to ensure that the dashboard + # has the correct initial state before requests can come in. + self._on_config_changed() + + # start the dashboard backend thread and the dashboard frontend manager (if enabled). + # This should be the last thing to happen in the initialization since the dashboard + # may access various parts of the agent + if dashboard_api: + dashboard_thread, port = dashboard_api.run_in_thread() self._dashboard_manager = DashboardManager( port, self.serena_config.web_dashboard_listen_address, @@ -1076,6 +1092,27 @@ class SerenaAgent: """ return self._task_executor.execute_task(task, name=name, logged=logged, timeout=timeout) + def _task_completion_callback(self) -> None: + """ + Called after every task completion. Tasks potentially change the agent's state/configuration. + """ + self._on_config_changed() + + def _on_config_changed(self) -> None: + for callback in self._config_changed_callbacks: + try: + callback() + except Exception as e: + log.error(f"Error in config changed callback {callback}: {e}", exc_info=e) + + def register_config_changed_callback(self, callback: Callable[[], None]) -> None: + """ + Registers a callback to be called when the agent configuration has (potentially) changed. + + :param callback: the callback function to register + """ + self._config_changed_callbacks.append(callback) + def is_using_language_server(self) -> bool: """ :return: whether this agent uses language server-based code analysis @@ -1131,6 +1168,25 @@ class SerenaAgent: if self.get_language_backend().is_lsp(): self.issue_task(init_language_server_manager) + def init_jetbrains_ide() -> None: + assert self.serena_config.jetbrains_launch_command is not None + try: + client = jetbrains_plugin_client.JetBrainsPluginClient.from_project(project, log_warning=False) + log.info("Found Serena JetBrains Plugin server: %s", client) + except jetbrains_plugin_client.ServerNotFoundError: + log.info("Serena JetBrains Plugin server not found for project %s", project.project_name) + if self.serena_config.jetbrains_launch_command: + cmd = subprocess_util.convert_shell_cmd([self.serena_config.jetbrains_launch_command, project.project_root]) + log.info("Launching IDE with command: %s", cmd) + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True) + stdout, stderr = p.communicate() + if p.returncode != 0: + log.error(f"Failed to launch JetBrains IDE: {stderr.decode('utf-8')}") + + # for JetBrains mode, search for plugin server and spawn IDE if not found and enabled + if self.get_language_backend().is_jetbrains(): + self.issue_task(init_jetbrains_ide) + if self._project_activation_callback is not None: self._project_activation_callback() diff --git a/src/serena/config/serena_config.py b/src/serena/config/serena_config.py index ff63f523..03a18e72 100644 --- a/src/serena/config/serena_config.py +++ b/src/serena/config/serena_config.py @@ -720,7 +720,12 @@ class SerenaConfig(SharedConfig, ModeSelectionDefinitionWithBaseModes): web_dashboard_open_on_launch: bool = True web_dashboard_interface: str | None = None web_dashboard_listen_address: str = "127.0.0.1" + web_dashboard_trusted_hosts: list[str] = field(default_factory=lambda: ["127.0.0.1", "localhost"]) jetbrains_plugin_server_address: str = "127.0.0.1" + jetbrains_launch_command: str | None = None + """ + JetBrains IDE launch command, which can be used to auto-start an IDE instance on demand. + """ tool_timeout: float = DEFAULT_TOOL_TIMEOUT token_count_estimator: str = RegisteredTokenCountEstimator.CHAR_COUNT.name diff --git a/src/serena/dashboard.py b/src/serena/dashboard.py index f7c5c3cc..cac54bc5 100644 --- a/src/serena/dashboard.py +++ b/src/serena/dashboard.py @@ -198,21 +198,26 @@ class SerenaDashboardAPI: agent: "SerenaAgent", tool_usage_stats: ToolUsageStats | None = None, host: str = "127.0.0.1", + trusted_hosts: list[str] | None = None, ) -> None: self._memory_log_handler = memory_log_handler self._tool_names = tool_names self._agent = agent self._host = host self._app = Flask(__name__) - local_hosts = ["127.0.0.1", "localhost"] - if self._host in local_hosts: - self._app.config["TRUSTED_HOSTS"] = local_hosts + if trusted_hosts: + self._app.config["TRUSTED_HOSTS"] = trusted_hosts self._tool_usage_stats = tool_usage_stats self._loaded_news: dict[str, str] = {} self._news_ready = threading.Event() self._setup_routes() self._read_news = ReadNews.load() - # Fetch remote news in background on startup (non-blocking) + + # register callback for config changes + self._current_config_overview: dict[str, Any] | None = None + self._agent.register_config_changed_callback(self._on_agent_config_changed) + + # fetch remote news in background on startup (non-blocking) threading.Thread(target=self._fetch_news, daemon=True).start() @property @@ -277,8 +282,10 @@ class SerenaDashboardAPI: @self._app.route("/get_config_overview", methods=["GET"]) def get_config_overview() -> dict[str, Any]: - result = self._agent.execute_task(self._get_config_overview, logged=False) - return result.model_dump() + result = self._current_config_overview + if result is None: + raise ValueError("Config overview not yet available") + return result @self._app.route("/shutdown", methods=["PUT"]) def shutdown() -> dict[str, str]: @@ -473,7 +480,7 @@ class SerenaDashboardAPI: if self._tool_usage_stats is not None: self._tool_usage_stats.clear() - def _get_config_overview(self) -> ResponseConfigOverview: + def _compute_config_overview(self) -> ResponseConfigOverview: from serena.config.context_mode import SerenaAgentContext, SerenaAgentMode from serena.tools.tools_base import Tool @@ -601,6 +608,9 @@ class SerenaDashboardAPI: serena_version=self._agent.version, ) + def _on_agent_config_changed(self) -> None: + self._current_config_overview = self._compute_config_overview().model_dump() + def _get_available_languages(self) -> ResponseAvailableLanguages: from solidlsp.ls_config import Language @@ -803,13 +813,14 @@ def open_url_in_browser(url: str, use_subprocess: bool = False) -> None: if use_subprocess: # Use a subprocess to avoid any output from webbrowser.open being written to stdout try: - subprocess.Popen( + p = subprocess.Popen( [sys.executable, "-c", f"import webbrowser; webbrowser.open({url!r})"], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=False, ) + threading.Thread(target=p.wait, daemon=True).start() except Exception as e: # Subprocess creation can fail in rare cases (e.g. on some Linux systems; possibly subprocess/glibc bug) # See #1363 diff --git a/src/serena/jetbrains/jetbrains_plugin_client.py b/src/serena/jetbrains/jetbrains_plugin_client.py index c7b440ce..3cbd915f 100644 --- a/src/serena/jetbrains/jetbrains_plugin_client.py +++ b/src/serena/jetbrains/jetbrains_plugin_client.py @@ -105,7 +105,12 @@ class JetBrainsPluginClientManager: futures.append(future) return futures - def find_client(self, project_root: Path) -> "JetBrainsPluginClient": + def find_client(self, project_root: Path, log_warning: bool = True) -> "JetBrainsPluginClient": + """ + :param project_root: the project root path to find a plugin instance for + :param log_warning: whether to log a warning if no matching plugin instance is found + :return: the instance + """ plugin_paths_found = [] for future in self._submit_scan(): client = future.result() @@ -114,12 +119,13 @@ class JetBrainsPluginClientManager: elif client.project_root is not None: plugin_paths_found.append(client.project_root) - log.warning( - "Searched for Serena JetBrains plugin service for project at %s but found no matching service. " - "Found plugin instances for the following project paths: %s", - project_root, - plugin_paths_found, - ) + if log_warning: + log.warning( + "Searched for Serena JetBrains plugin service for project at %s but found no matching service. " + "Found plugin instances for the following project paths: %s", + project_root, + plugin_paths_found, + ) raise ServerNotFoundError( f"Found no Serena service in a JetBrains IDE instance for the project at {project_root}. " "STOP. Do not attempt any other tools or workarounds. Ask the user to open this folder as a project in a JetBrains IDE " @@ -219,7 +225,7 @@ class JetBrainsPluginClient(ToStringMixin): return ["_port", "project_root", "_plugin_version"] @classmethod - def from_project(cls, project: Project) -> Self: + def from_project(cls, project: Project, log_warning: bool = True) -> Self: resolved_path = Path(project.project_root).resolve() if cls._last_port is not None: @@ -227,7 +233,7 @@ class JetBrainsPluginClient(ToStringMixin): if client.matches(resolved_path): return client - client = JetBrainsPluginClientManager().find_client(resolved_path) + client = JetBrainsPluginClientManager().find_client(resolved_path, log_warning=log_warning) cls._last_port = client._port return client diff --git a/src/serena/resources/dashboard/dashboard.css b/src/serena/resources/dashboard/dashboard.css index ad658a97..fb2d70da 100644 --- a/src/serena/resources/dashboard/dashboard.css +++ b/src/serena/resources/dashboard/dashboard.css @@ -1012,6 +1012,11 @@ code, pre, kbd, samp, font-style: italic; } +.loaded { + text-align: center; + color: var(--text-muted); +} + .error-message { color: var(--log-error); text-align: center; diff --git a/src/serena/resources/dashboard/dashboard.js b/src/serena/resources/dashboard/dashboard.js index afa71865..43987b58 100644 --- a/src/serena/resources/dashboard/dashboard.js +++ b/src/serena/resources/dashboard/dashboard.js @@ -944,6 +944,7 @@ class Dashboard { displayActiveExecutionsQueue(executions) { if (!executions || executions.length === 0) { + this.$activeExecutionQueueDisplay.html('