Revamp mode selection

* Breaking change: Projects (project.yml) can no longer override `base_modes`.
   Instead, they can define `added_modes` to add modes on top of base and default modes.
   This allows `base_modes` to be fully applied for the computation of the base toolset
   as they are no longer dynamic.
 * Serena's default configuration now uses `interactive` and `editing` as `base_modes`
   instead of as `default_modes`.

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 Status of the `main` branch. Changes prior to the next official version change will appear here.
 
+* General:
+  - Breaking change in mode definitions: Projects (project.yml) can no longer override `base_modes`.
+    Instead, they can define `added_modes` to add modes on top of base and default modes.  
+    See updated [documentation on modes](https://oraios.github.io/serena/02-usage/050_configuration.html#modes).
+  - Serena's default configuration now uses `interactive` and `editing` as `base_modes` instead of as `default_modes`. 
+
 * Language Servers:
   - Java (`eclipse.jdt.ls`): Add upstream JDTLS mode for offline / restricted-network use. Setting both `jdtls_path` and `lombok_path` in `ls_specific_settings.java` makes Serena use an existing upstream JDTLS installation (e.g. `brew install jdtls`) and the system JDK 21+, skipping the ~500 MB vscode-java VSIX, Gradle, and IntelliCode downloads. New related setting `java_home` lets the user override the JDK used to launch JDTLS. Default behavior unchanged — the JDTLS workspace hash is preserved bit-for-bit for users on the default route, so existing project caches are reused without a one-time reindex; the launcher path is mixed into the hash only when `jdtls_path` is set, isolating upstream installations from the default workspace. #1415
 

docs/02-usage/050_configuration.md

@@ -118,32 +118,26 @@ Examples of built-in modes include:
 
 Find the concrete definitions of these modes [here](https://github.com/oraios/serena/tree/main/src/serena/resources/config/modes).
 
-Active modes are configured in (from lowest to highest precedence):
+The modes to be activated are configured in:
   * the global configuration file (`serena_config.yml`)
+     - defines `base_modes`, which are always included
+     - defines `default_modes`, which can be overridden by projects or command line parameters
   * the project configuration file (`project.yml`)
+     - defines `default_modes` (overriding the default modes in the global configuration)
+     - defines `added_modes`, which are added on top
   * at startup via command-line parameters
+     - can override default modes with `--mode`
+     - can define modes to be added on top with `--add-mode`
 
-The two former sources define both **base modes** and **default modes**.
-Ultimately, the active modes are the union of base modes and default modes (after applying all overrides).
-Command-line parameters override default modes but not base modes.
-Base modes should thus be used to define modes that you always want to be active, regardless of command-line parameters.
+Ultimately, the active modes are given by the union of 
+  * `base_modes` defined in the global configuration (always active)  
+  * `default_modes` (defined in the global configuration, optionally overridden by the project/CLI)
+  * `added_modes` (defined in the project configuration/via CLI parameters)
 
-Command-line parameters for overriding default modes:
-When launching the MCP sever, specify modes using `--mode <mode-name>`; multiple modes can be specified, e.g. `--mode planning --mode no-onboarding`.
-
-:::{important}
-By default, Serena activates the two modes `interactive` and `editing` (as defined in the global configuration).
-
-As soon as you start to specify modes via the command line, only the modes you explicitly specify will be active, however.
-Therefore, if you want to keep the default modes, you must specify them as well.  
-For example, to add mode `no-memories` to the default behaviour, specify
-```shell
---mode interactive --mode editing --mode no-memories
-```
-
-If you want to keep certain modes as always active, regardless of command-line parameters, 
-define them as *base modes* in the global or project configuration.
-:::
+So you should 
+ * define modes you definitely always want to use in `base_modes`,
+ * define modes that you typically want to use but sometimes want to override in `default_modes`,
+ * use `added_modes` to add modes that you need only for specific projects/sessions.
 
 :::{note}
 **Mode Compatibility**: While you can combine modes, some may be semantically incompatible (e.g., `interactive` and `one-shot`). 

src/serena/agent.py

@@ -31,6 +31,8 @@ from serena.config.context_mode import SerenaAgentContext, SerenaAgentMode
 from serena.config.serena_config import (
     LanguageBackend,
     ModeSelectionDefinition,
+    ModeSelectionDefinitionWithAddedModes,
+    ModeSelectionDefinitionWithBaseModes,
     NamedToolInclusionDefinition,
     RegisteredProject,
     SerenaConfig,
@@ -206,18 +208,36 @@ class ActiveModes:
     def __init__(self) -> None:
         self._configured_base_modes: Sequence[str] | None = None
         self._configured_default_modes: Sequence[str] | None = None
+        self._added_modes: set[str] = set()
+        self._dynamically_activated_mode_names: set[str] = set()
+        """
+        the subset of active mode names that are dynamically activated (not necessarily enabled after project change)
+        """
         self._active_mode_names: Sequence[str] = []
+        """
+        the full list of active mode names
+        """
 
     def apply(self, mode_selection: ModeSelectionDefinition) -> None:
+        log.debug("Applying mode selection definition %s", mode_selection)
+
         # apply overrides
-        log.debug("Applying mode selection: default_modes=%s, base_modes=%s", mode_selection.default_modes, mode_selection.base_modes)
-        if mode_selection.base_modes is not None:
-            self._configured_base_modes = mode_selection.base_modes
+        if isinstance(mode_selection, ModeSelectionDefinitionWithBaseModes):
+            if mode_selection.base_modes is not None:
+                self._configured_base_modes = mode_selection.base_modes
         if mode_selection.default_modes is not None:
             self._configured_default_modes = mode_selection.default_modes
         log.debug("Current mode selection: base_modes=%s, default_modes=%s", self._configured_base_modes, self._configured_default_modes)
 
-        self._active_mode_names = sorted(set(self._configured_base_modes or []) | set(self._configured_default_modes or []))
+        # apply added modes (if any)
+        if isinstance(mode_selection, ModeSelectionDefinitionWithAddedModes):
+            if mode_selection.added_modes:
+                log.debug("Adding modes: %s", mode_selection.added_modes)
+                self._added_modes.update(mode_selection.added_modes)
+                log.debug("Current added modes: %s", self._added_modes)
+
+        self._dynamically_activated_mode_names = set(self._configured_default_modes or []) | self._added_modes
+        self._active_mode_names = sorted(set(self._configured_base_modes or []) | self._dynamically_activated_mode_names)
 
     def get_mode_names(self) -> Sequence[str]:
         return self._active_mode_names
@@ -231,8 +251,8 @@ class ActiveModes:
     def get_modes(self) -> Sequence[SerenaAgentMode]:
         return [self.get_mode_instance(mode_name) for mode_name in self._active_mode_names]
 
-    def get_default_modes(self) -> Sequence[SerenaAgentMode]:
-        return [self.get_mode_instance(mode_name) for mode_name in self._configured_default_modes or []]
+    def get_dynamically_activated_modes(self) -> Sequence[SerenaAgentMode]:
+        return [self.get_mode_instance(mode_name) for mode_name in self._dynamically_activated_mode_names]
 
     def get_base_modes(self) -> Sequence[SerenaAgentMode]:
         return [self.get_mode_instance(mode_name) for mode_name in self._configured_base_modes or []]
@@ -511,8 +531,7 @@ class SerenaAgent:
         :param serena_config: the Serena configuration or None to read the configuration from the default location.
         :param context: the context in which the agent is operating, None for default context.
             The context may adjust prompts, tool availability, and tool descriptions.
-        :param modes: list of modes in which the agent is operating (they will be combined), None for default modes.
-            The modes may adjust prompts, tool availability, and tool descriptions.
+        :param modes: mode selection definition to apply for this session
         :param memory_log_handler: a MemoryLogHandler instance from which to read log messages; if None, a new one will be created
             if necessary.
         """
@@ -521,7 +540,7 @@ class SerenaAgent:
         self._gui_log_viewer: Optional["GuiLogViewer"] = None
         self._dashboard_manager: DashboardManager | None = None
         self._project_prompt_status = ProjectPromptProvisionStatus()
-        self._mode_overrides = modes
+        self._session_mode_selection_definition = modes
         self.version = serena_version()
 
         # obtain serena configuration using the decoupled factory function
@@ -712,9 +731,11 @@ class SerenaAgent:
         # * base modes: These cannot be changed, so they are fully applied
         for base_mode in modes.get_base_modes():
             tool_inclusion_definitions.append(base_mode)
-        # * default modes: When not in a single-project context, these modes are dynamic (can later be turned off),
-        #   so we consider only their inclusions (but not their exclusions, because these must not be hard)
-        for mode in modes.get_default_modes():
+        # * dynamically activated modes:
+        #    - When not in a single-project context, these modes can later be turned off,
+        #      so we consider only their inclusions (but not their exclusions, because these must not be hard).
+        #    - In a single-project context, we can consider them fully.
+        for mode in modes.get_dynamically_activated_modes():
             if is_single_project:
                 tool_inclusion_definitions.append(mode)
             else:
@@ -986,8 +1007,8 @@ class SerenaAgent:
         self._active_modes.apply(self.serena_config)
         if self._active_project:
             self._active_modes.apply(self._active_project.project_config)
-        if self._mode_overrides:
-            self._active_modes.apply(self._mode_overrides)
+        if self._session_mode_selection_definition:
+            self._active_modes.apply(self._session_mode_selection_definition)
         if log_message:
             active_mode_names = self._active_modes.get_mode_names()
             log.info(f"Active modes ({len(active_mode_names)}): {', '.join(active_mode_names)}")

src/serena/cli.py

@@ -23,6 +23,7 @@ from serena.config.context_mode import SerenaAgentContext, SerenaAgentMode
 from serena.config.serena_config import (
     LanguageBackend,
     ModeSelectionDefinition,
+    ModeSelectionDefinitionWithAddedModes,
     ProjectConfig,
     RegisteredProject,
     SerenaConfig,
@@ -37,7 +38,6 @@ from serena.constants import (
 )
 from serena.prompt_factory import SerenaPromptFactory
 from serena.util.cli_util import AutoRegisteringGroup
-from serena.util.dataclass import get_dataclass_default
 from serena.util.logging import MemoryLogHandler
 from solidlsp.ls_config import Language
 from solidlsp.ls_types import SymbolKind
@@ -46,15 +46,17 @@ from solidlsp.util.subprocess_util import subprocess_kwargs
 log = logging.getLogger(__name__)
 
 _MAX_CONTENT_WIDTH = 200
-_MODES_EXPLANATION = f"""\b\nBuilt-in mode names or paths to custom mode YAMLs with which to 
-override the default modes defined in the global Serena configuration or 
+_MODES_EXPLANATION = """\b\nBuilt-in mode names or paths to custom mode YAMLs with which to 
+override the default_modes defined in the global Serena configuration or 
 the active project.
 For details on mode configuration, see 
   https://oraios.github.io/serena/02-usage/050_configuration.html#modes.
-If no configuration changes were made, the base defaults are: 
-  {get_dataclass_default(SerenaConfig, "default_modes")}.
-Overriding them means that they no longer apply, so you will need to 
-re-specify them in addition to further modes if you want to keep them."""
+"""
+_ADD_MODES_EXPLANATION = """\b\nMode names or paths to custom mode YAMLs which shall
+be added on top of the other modes specified by the global/project configuration.
+For details on mode configuration, see 
+  https://oraios.github.io/serena/02-usage/050_configuration.html#modes.
+"""
 
 
 def find_project_root(root: str | Path | None = None) -> str | None:
@@ -228,13 +230,22 @@ class TopLevelCommands(AutoRegisteringGroup):
     )
     @click.option(
         "--mode",
-        "modes",
+        "default_modes",
         type=str,
         multiple=True,
         default=(),
         show_default=False,
         help=_MODES_EXPLANATION,
     )
+    @click.option(
+        "--add-mode",
+        "added_modes",
+        type=str,
+        multiple=True,
+        default=(),
+        show_default=False,
+        help=_ADD_MODES_EXPLANATION,
+    )
     @click.option(
         "--language-backend",
         type=click.Choice([lb.value for lb in LanguageBackend]),
@@ -300,7 +311,8 @@ class TopLevelCommands(AutoRegisteringGroup):
         project_file_arg: str | None,
         project_from_cwd: bool | None,
         context: str,
-        modes: Sequence[str],
+        default_modes: Sequence[str],
+        added_modes: Sequence[str],
         language_backend: str | None,
         transport: Literal["stdio", "sse", "streamable-http"],
         host: str,
@@ -346,11 +358,15 @@ class TopLevelCommands(AutoRegisteringGroup):
 
         project_file = project_file_arg or project
 
+        mode_selection_def: ModeSelectionDefinition | None = None
+        if default_modes or added_modes:
+            mode_selection_def = ModeSelectionDefinitionWithAddedModes(default_modes=default_modes or None, added_modes=added_modes or None)
+
         factory = SerenaMCPFactory(context=context, project=project_file, memory_log_handler=memory_log_handler)
         server = factory.create_mcp_server(
             host=host,
             port=port,
-            modes=modes,
+            mode_selection_def=mode_selection_def,
             language_backend=LanguageBackend.from_str(language_backend) if language_backend else None,
             enable_web_dashboard=enable_web_dashboard,
             open_web_dashboard=open_web_dashboard,

src/serena/config/serena_config.py

@@ -168,10 +168,22 @@ class NamedToolInclusionDefinition(ToolInclusionDefinition):
 
 @dataclass
 class ModeSelectionDefinition:
-    base_modes: Sequence[str] | None = None
     default_modes: Sequence[str] | None = None
 
 
+@dataclass
+class ModeSelectionDefinitionWithBaseModes(ModeSelectionDefinition):
+    base_modes: Sequence[str] | None = ("interactive", "editing")
+    """
+    the base modes to use, which are always guaranteed to be included
+    """
+
+
+@dataclass
+class ModeSelectionDefinitionWithAddedModes(ModeSelectionDefinition):
+    added_modes: Sequence[str] | None = None
+
+
 class LanguageBackend(Enum):
     LSP = "LSP"
     """
@@ -228,7 +240,7 @@ class LineEnding(Enum):
 
 
 @dataclass
-class SharedConfig(ModeSelectionDefinition, ToolInclusionDefinition, ToStringMixin):
+class SharedConfig(ToolInclusionDefinition, ToStringMixin):
     """Shared between SerenaConfig and ProjectConfig, the latter used to override values in the form
     (same as in ModeSelectionDefinition).
     The defaults here shall be none and should be set to the global default values in SerenaConfig.
@@ -255,7 +267,7 @@ Uses $projectDir and $projectFolderName as placeholders.
 
 
 @dataclass(kw_only=True)
-class ProjectConfig(SharedConfig):
+class ProjectConfig(SharedConfig, ModeSelectionDefinitionWithAddedModes):
     project_name: str
     languages: list[Language]
     ignored_paths: list[str] = field(default_factory=list)
@@ -471,6 +483,9 @@ class ProjectConfig(SharedConfig):
         excluded_tools = data["excluded_tools"] or []
         included_optional_tools = data["included_optional_tools"] or []
 
+        if "base_modes" in data and data["base_modes"] is not None:
+            log.warning("The base_modes setting in project.yml is deprecated and will be ignored.")
+
         return cls(
             project_name=data["project_name"],
             languages=languages,
@@ -486,7 +501,7 @@ class ProjectConfig(SharedConfig):
             encoding=data["encoding"],
             line_ending=line_ending,
             language_backend=language_backend,
-            base_modes=data["base_modes"],
+            added_modes=data["added_modes"],
             default_modes=data["default_modes"],
             symbol_info_budget=symbol_info_budget,
             ls_specific_settings=data.get("ls_specific_settings", {}),
@@ -677,7 +692,7 @@ class RegisteredProject(ToStringMixin):
 
 
 @dataclass(kw_only=True)
-class SerenaConfig(SharedConfig):
+class SerenaConfig(SharedConfig, ModeSelectionDefinitionWithBaseModes):
     """
     Holds the Serena agent configuration, which is typically loaded from a YAML configuration file
     (when instantiated via :method:`from_config_file`), which is updated when projects are added or removed.
@@ -731,7 +746,6 @@ class SerenaConfig(SharedConfig):
     """
     the language backend to use for code understanding features
     """
-    default_modes: Sequence[str] | None = ("interactive", "editing")
     line_ending: LineEnding = LineEnding.NATIVE
     symbol_info_budget: float = 10.0
     """

src/serena/mcp.py

@@ -3,7 +3,7 @@ The Serena Model Context Protocol (MCP) Server
 """
 
 import sys
-from collections.abc import AsyncIterator, Iterator, Sequence
+from collections.abc import AsyncIterator, Iterator
 from contextlib import asynccontextmanager
 from copy import deepcopy
 from dataclasses import dataclass
@@ -272,7 +272,7 @@ class SerenaMCPFactory:
         self,
         host: str = "127.0.0.1",
         port: int = 8000,
-        modes: Sequence[str] = (),
+        mode_selection_def: ModeSelectionDefinition | None = None,
         language_backend: LanguageBackend | None = None,
         enable_web_dashboard: bool | None = None,
         enable_gui_log_window: bool | None = None,
@@ -286,7 +286,7 @@ class SerenaMCPFactory:
 
         :param host: The host to bind to
         :param port: The port to bind to
-        :param modes: List of mode names or paths to mode files
+        :param mode_selection_def: the mode selection definition to apply
         :param language_backend: the language backend to use, overriding the configuration setting.
         :param enable_web_dashboard: Whether to enable the web dashboard. If not specified, will take the value from the serena configuration.
         :param enable_gui_log_window: Whether to enable the GUI log window. It currently does not work on macOS, and setting this to True will be ignored then.
@@ -318,9 +318,6 @@ class SerenaMCPFactory:
             if language_backend is not None:
                 config.language_backend = language_backend
 
-            mode_selection_def: ModeSelectionDefinition | None = None
-            if modes:
-                mode_selection_def = ModeSelectionDefinition(default_modes=modes)
             self.agent = self._create_serena_agent(config, mode_selection_def)
 
         except Exception as e:

src/serena/resources/project.template.yml

@@ -115,21 +115,21 @@ included_optional_tools: []
 # This cannot be combined with non-empty excluded_tools or included_optional_tools.
 fixed_tools: []
 
-# list of mode names to that are always to be included in the set of active modes
-# The full set of modes to be activated is base_modes + default_modes.
-# If the setting is undefined, the base_modes from the global configuration (serena_config.yml) apply.
-# Otherwise, this setting overrides the global configuration.
-# Set this to [] to disable base modes for this project.
-# Set this to a list of mode names to always include the respective modes for this project.
-base_modes:
-
-# list of mode names that are to be activated by default.
-# The full set of modes to be activated is base_modes + default_modes.
-# If the setting is undefined, the default_modes from the global configuration (serena_config.yml) apply.
+# list of mode names that are to be activated by default, overriding the setting in the global configuration.
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
 # Otherwise, this overrides the setting from the global configuration (serena_config.yml).
+# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
+# for this project.
 # This setting can, in turn, be overridden by CLI parameters (--mode).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
 default_modes:
 
+# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+added_modes:
+
 # initial prompt for the project. It will always be given to the LLM upon activating the project
 # (contrary to the memories, which are loaded on demand).
 initial_prompt: ""

src/serena/resources/serena_config.template.yml

@@ -112,19 +112,23 @@ included_optional_tools: []
 # This cannot be combined with non-empty excluded_tools or included_optional_tools.
 fixed_tools: []
 
-# list of mode names to that are always to be included in the set of active modes
-# The full set of modes to be activated is base_modes + default_modes.
-# If this is undefined, no base modes are included.
-# The project configuration (project.yml) may override this setting.
+# list of mode names to that are always to be included in the set of active modes.
+# The full set of modes to be activated is base_modes + default_modes + added_modes,
+# where added_modes can be defined by projects/CLI parameters.
+# If this is undefined/empty, no base modes are included.
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
 base_modes:
-
-# list of mode names that are to be activated by default.
-# The full set of modes to be activated is base_modes + default_modes.
-# These modes can be overridden by the project configuration (project.yml) or through the CLI (--mode).
-default_modes:
   - interactive
   - editing
 
+# list of mode names that are to be activated by default (but can be overridden by projects/CLI params).
+# The full set of modes to be activated is base_modes + default_modes + added_modes,
+# where added_modes are defined by projects/CLI parameters.
+# If this is undefined/empty, no default modes are defined.
+# These modes can be overridden by the project configuration (project.yml) or through the CLI (--mode).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+default_modes:
+
 # Used as default for tools where the apply method has a default maximal answer length.
 # Even though the value of the max_answer_chars can be changed when calling the tool, it may make sense to adjust this default
 # through the global configuration.