mirror of
https://github.com/tiennm99/serena.git
synced 2026-10-12 14:15:14 +00:00
Memory management:
* Add renaming with auto-updated references * More specific identifier naming (use term unmarked_memories) Improve memory documentation: * Explain reference concept * Explain rationale Add memory CLI commands * CRUD operations
This commit is contained in:
1 parent
be5b60bddb
commit
461b7ebd4b
7 files changed
+356
-55
No files matched your search
@@ -10,6 +10,20 @@ Status of the `main` branch. Changes prior to the next official version change w
|
||||
- Modal markup cleanup: extract shared CSS classes (`.modal-info`, `.modal-hint`, `.modal-prompt`, `.modal-field`, `.modal-input`, `.modal-select`, `.modal-textarea`, `.modal-actions`, `.btn-secondary`) and remove duplicated inline styles from all seven modals. Inputs and textareas get an accent-colored focus ring; the modal backdrop has a subtle blur.
|
||||
- Add Language: replace the native `<select>` with a filterable combobox — type to filter, keyboard navigation (Up/Down/Enter/Esc), substring highlight, click-outside to close. The typed value is validated against the available languages list before submission.
|
||||
|
||||
* Memories:
|
||||
- Memories can now reference each other using the `` `mem:NAME` `` convention. Renames
|
||||
propagate to all references automatically. See the [reference convention](https://oraios.github.io/serena/02-usage/045_memories.html#referencing-memories-from-other-memories).
|
||||
- New `serena memories` CLI command group: `list`, `read`, `write`, `check` (referential
|
||||
integrity report) and `auto-prefix-references` (heuristic rewrite of bare occurrences).
|
||||
See the [CLI subcommands](https://oraios.github.io/serena/02-usage/045_memories.html#cli-subcommands).
|
||||
- Onboarding now seeds a `memory_maintenance` memory describing the memory-style and
|
||||
routing conventions, and the agent is instructed to read it before writing any other
|
||||
memories. A `global/memory_maintenance` memory takes precedence over the per-project
|
||||
seed. See the [memory maintenance section](https://oraios.github.io/serena/02-usage/045_memories.html#the-memory-maintenance-memory).
|
||||
- Onboarding prompt rewritten to target a layout of `mem:core` (routing entry point),
|
||||
`mem:suggested_commands`, `mem:tech_stack`, `mem:conventions`, `mem:task_completion`,
|
||||
and optional per-module `mem:<module>/core` memories, using dense agent-notes style.
|
||||
|
||||
* Tools:
|
||||
- `edit_memory`: New `dotall` parameter (default `False`) controlling whether ``.`` matches
|
||||
newlines in regex mode. **Behavior change**: previously DOTALL was forced on; the new default
|
||||
|
||||
@@ -143,9 +143,17 @@ and to store memories, which it can then draw upon in future interactions.
|
||||
|
||||
In general, **memories** provide a way for Serena to store and retrieve
|
||||
information about the project, relevant conventions, and other relevant aspects.
|
||||
Memories may reference each other using the `` `mem:NAME` `` convention; references
|
||||
are kept in sync across renames, and a `serena memories check` command is available
|
||||
to report stale references.
|
||||
|
||||
For more information on this, including how to manage
|
||||
or disable these features, see [Memories & Onboarding](045_memories).
|
||||
During the first onboarding, Serena seeds a `memory_maintenance` memory describing the
|
||||
conventions (style, references, routing) that subsequent memories should follow, and the
|
||||
agent is instructed to read it before writing any project-specific memories.
|
||||
|
||||
For more information on this, including the target memory layout, the `mem:` reference
|
||||
convention, the `serena memories` CLI subcommands, and how to manage or disable these
|
||||
features, see [Memories & Onboarding](045_memories).
|
||||
|
||||
|
||||
## Preparing Your Project
|
||||
|
||||
@@ -7,7 +7,7 @@ agent's internal memory management (e.g., `AGENTS.md` files).
|
||||
## Memories
|
||||
|
||||
Memories are simple, human-readable Markdown files that both you and
|
||||
your agent can create, read, and edit.
|
||||
your agent can create, read, reference, and edit.
|
||||
|
||||
Serena differentiates between
|
||||
* **project-specific memories**, which are stored in the `.serena/memories/` directory within your project folder, and
|
||||
@@ -18,12 +18,76 @@ inferring appropriateness from the file name.
|
||||
When the agent starts working on a project, it receives the list of available memories.
|
||||
The agent should be instructed to update memories by the user when appropriate.
|
||||
|
||||
### Design Rationale
|
||||
|
||||
Serena's memory system is intentionally minimal. It was designed to satisfy the following
|
||||
criteria:
|
||||
|
||||
1. **Human-readable and editable.** Memories must remain directly readable and editable
|
||||
in any text editor. The agent is typically the day-to-day consumer, but a human author
|
||||
or reviewer must always be able to step in without going through the agent.
|
||||
2. **Versionable with the project.** Project memories live alongside the code and can
|
||||
be committed, reviewed in PRs, and reverted like any other repository artifact.
|
||||
3. **Progressive disclosure.** Agents receive the full memory *name list* up
|
||||
front as part of their initial instructions; any further routing is described inside
|
||||
the memory content itself - typically a `mem:core` entry point pointing at focused
|
||||
memories. The agent decides what to read based on names plus the routing it has
|
||||
already seen.
|
||||
4. **Prefer routing to search.** Given an intelligent agent and well-structured routing, search is
|
||||
unnecessary - and it adds noise: any retrieval method (lexical or semantic)
|
||||
produces both false positives and false negatives. Explicit, name-based routing
|
||||
decided by the agent is deterministic and avoids both error modes. Basic search via regex/grep
|
||||
is sufficient to complement the routing when needed and is available to any agent.
|
||||
5. **Prefer deliberate reads to triggers.** The agent decides what to read and when. The harness does
|
||||
not inject memory content on the agent's behalf.
|
||||
6. **Framework-agnostic.** The storage format is plain Markdown files in
|
||||
a simple directory layout. The only Serena-specific convention is the `mem:`
|
||||
prefix for references to memories, which does not prevent using the memory files outside
|
||||
of Serena.
|
||||
7. **Configurable and composable.** Two orthogonal memory scopes -
|
||||
[per-project](#memories) (committed alongside the code) and [global](global-memories)
|
||||
(shared across all your projects) - can be combined freely. Within either scope,
|
||||
regex patterns in the global or project configuration can mark subsets as read-only
|
||||
or [hide them entirely](#ignoring-memories) from the agent. This lets a project mix
|
||||
personal cross-project knowledge with checked-in project conventions, and selectively
|
||||
freeze either set, without custom plumbing.
|
||||
|
||||
Taken together, these criteria rule out several common alternatives:
|
||||
|
||||
- **Database-backed memory** (SQLite, graph databases, vector stores) is excluded by
|
||||
criteria 1, 4, and 6.
|
||||
- **`AGENTS.md` and similar single-file conventions** are excluded by criteria 3 and 5.
|
||||
- **Hooks and harness-internal memory systems** are excluded by criteria 5 and 6.
|
||||
|
||||
To our knowledge, no existing system satisfies our design goal, which is
|
||||
why Serena ships its own memory layer rather than reusing one.
|
||||
The closest existing approaches are in the family of Markdown-based personal knowledge
|
||||
management tools - **Obsidian**, **Logseq**, **Foam**.
|
||||
|
||||
### Organizing Memories
|
||||
|
||||
Memories can be organized into **topics** by using `/` in the memory name (e.g. `modules/frontend`).
|
||||
The structure is mapped to the file system, where topics correspond to subdirectories.
|
||||
The `list_memories` tool can filter by topic, allowing the agent to explore even large numbers of memories in a structured way.
|
||||
|
||||
(memory-references)=
|
||||
### Referencing Memories from Other Memories
|
||||
|
||||
Memories may reference each other. Serena recognizes a reference as a memory name prefixed with
|
||||
`mem:` and wrapped in backticks, for example `` `mem:auth/login` `` or `` `mem:suggested_commands` ``.
|
||||
This convention has two practical consequences:
|
||||
|
||||
- **Renames keep references intact.** When you rename or move a memory with the `rename_memory`
|
||||
tool, Serena rewrites every `` `mem:OLD_NAME` `` occurrence across all memories to point to
|
||||
the new name. References that do not use the `mem:` prefix will not be updated automatically.
|
||||
- **Integrity checks** (see [below](memory-cli)) report any `` `mem:NAME` `` whose target does
|
||||
not resolve to an existing memory, and propose similarly-named candidates as likely intended
|
||||
targets.
|
||||
|
||||
The full convention - including style, add/update thresholds, and how to structure routing across
|
||||
`core` memories - is shipped to every onboarded project as the `memory_maintenance` memory; see the
|
||||
[Onboarding section](onboarding) below.
|
||||
|
||||
(global-memories)=
|
||||
### Global Memories
|
||||
|
||||
@@ -49,7 +113,7 @@ global or project-level [configuration](050_configuration):
|
||||
ignored_memory_patterns: ["_archive/.*", "_episodes/.*"]
|
||||
```
|
||||
|
||||
Ignored memories are completely excluded — they cannot be accessed via `read_memory`, `write_memory`,
|
||||
Ignored memories are completely excluded - they cannot be accessed via `read_memory`, `write_memory`,
|
||||
or any other memory tool. To read an ignored memory file, use the `read_file` tool on the raw file path
|
||||
(e.g., `.serena/memories/_archive/2026-03/some-topic.md`).
|
||||
|
||||
@@ -66,8 +130,8 @@ viewing, creating, editing, and deleting memories while Serena is running.
|
||||
|
||||
By default, Serena performs an **onboarding process** when it encounters a project
|
||||
for the first time (i.e., when no project memories exist yet).
|
||||
The goal of the onboarding is for Serena to get familiar with the project —
|
||||
its structure, build system, testing setup, and other essential aspects —
|
||||
The goal of the onboarding is for Serena to get familiar with the project -
|
||||
its structure, build system, testing setup, and other essential aspects -
|
||||
and to store this knowledge as memories for future interactions.
|
||||
|
||||
In further project activations, Serena will check whether onboarding was already
|
||||
@@ -80,7 +144,32 @@ process if memories are found.
|
||||
performed (by checking if any memories exist).
|
||||
2. If no memories are found, Serena triggers the onboarding process, which
|
||||
reads key files and directories to understand the project.
|
||||
3. The gathered information is written into project-specific memory files (see above).
|
||||
3. Before any project memory is written, Serena materializes a project-local
|
||||
`memory_maintenance` memory (see below). The agent is then instructed to read it
|
||||
first and follow the conventions it describes.
|
||||
4. The gathered information is written into project-specific memory files following
|
||||
the onboarding prompt instructions and the conventions outlined in `memory_maintenance`.
|
||||
|
||||
(memory-maintenance-memory)=
|
||||
### The `memory_maintenance` Memory
|
||||
|
||||
To make memory conventions discoverable to both the LLM and the user, Serena seeds
|
||||
a `memory_maintenance` memory on first onboarding. The seed is copied from a template
|
||||
shipped with the Serena package and contains the dense agent-notes style, the
|
||||
`mem:` reference convention, the routing model around `core` memories, the
|
||||
add/update threshold, and the maintenance actions (rename / delete / split).
|
||||
|
||||
The seeding follows a strict precedence:
|
||||
|
||||
1. If you already maintain a `global/memory_maintenance` memory, Serena uses that
|
||||
and **does not** create a project-local copy. This is the recommended approach
|
||||
for teams that want one shared convention document across all projects.
|
||||
2. Otherwise, if the project already has a `memory_maintenance` memory, it is left
|
||||
untouched.
|
||||
3. Otherwise, the shipped template is written to `.serena/memories/memory_maintenance.md`.
|
||||
|
||||
Existing files are never overwritten - you can freely customize the project copy.
|
||||
To refresh from the shipped template, delete the existing memory first.
|
||||
|
||||
### Tips for Onboarding
|
||||
|
||||
@@ -92,6 +181,29 @@ process if memories are found.
|
||||
- **Review the results**: After onboarding, we recommend having a quick look at the
|
||||
generated memories and editing them or adding new ones as needed.
|
||||
|
||||
(memory-cli)=
|
||||
### CLI Subcommands
|
||||
|
||||
While the recommended way to manage memories is through the **MCP integration**,
|
||||
Serena also offers memory-related CLI commands.
|
||||
|
||||
The following commands have **no MCP tool counterpart** and are intended for human execution:
|
||||
|
||||
- `serena memories check` — referential-integrity report. By default reports only stale
|
||||
`` `mem:NAME` `` references whose target does not resolve. Pass `--include-unmarked` to
|
||||
also report bare occurrences of existing memory names found through a heuristic.
|
||||
- `serena memories auto-prefix-references` — heuristic rewrite of bare occurrences to add
|
||||
the `mem:` prefix; supports `--dry-run`.
|
||||
- `serena memories initialize` will seed the `memory_maintenance` memory for the project.
|
||||
|
||||
The remaining commands mirror the MCP tools, you can thus instruct your agent to manage memories with
|
||||
serena without having a running MCP server. Discover the full surface and per-command flags via:
|
||||
|
||||
```shell
|
||||
serena memories --help
|
||||
serena memories <subcommand> --help
|
||||
```
|
||||
|
||||
## Disabling Memories and Onboarding
|
||||
|
||||
If you do not require the functionality described in this section, you can selectively disable it.
|
||||
|
||||
+134
-8
@@ -9,7 +9,7 @@ import time
|
||||
from collections.abc import Iterator, Sequence
|
||||
from logging import Logger
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
from typing import TYPE_CHECKING, Any, Literal
|
||||
|
||||
import click
|
||||
from sensai.util import logging
|
||||
@@ -43,6 +43,9 @@ from solidlsp.ls_config import Language
|
||||
from solidlsp.ls_types import SymbolKind
|
||||
from solidlsp.util.subprocess_util import subprocess_kwargs
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from serena.project import MemoriesManager
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
_MAX_CONTENT_WIDTH = 200
|
||||
@@ -1124,14 +1127,43 @@ class MemoryCommands(AutoRegisteringGroup):
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _load_memories_manager(project: str):
|
||||
"""Load the active project and return its memories manager."""
|
||||
def _load_memories_manager(project: str) -> "MemoriesManager":
|
||||
"""
|
||||
Load the project at ``project`` and return its memories manager.
|
||||
|
||||
Requires the project to already be a Serena project (``.serena/project.yml`` must
|
||||
exist at the configured location). Never auto-creates the ``.serena`` directory —
|
||||
if no project configuration is found, the user is directed at ``serena project create``.
|
||||
"""
|
||||
from serena.project import Project
|
||||
|
||||
serena_config = SerenaConfig.from_config_file()
|
||||
proj = Project.load(os.path.abspath(project), serena_config=serena_config)
|
||||
project_path = os.path.abspath(project)
|
||||
try:
|
||||
proj = Project.load(project_path, serena_config=serena_config, autogenerate=False)
|
||||
except FileNotFoundError as e:
|
||||
raise click.UsageError(
|
||||
f"{e}\nNo Serena project found at {project_path}. Create one first with:\n serena project create {project_path}"
|
||||
) from e
|
||||
return proj.memories_manager
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"initialize",
|
||||
help=(
|
||||
"Initialize this project's memory layout by seeding the `memory_maintenance` memory. "
|
||||
"Requires the project to already exist as a Serena project (run `serena project create` first); "
|
||||
"this command does not create a `.serena` directory on its own. "
|
||||
"If a `global/memory_maintenance` memory exists, it takes precedence and no project copy is created."
|
||||
),
|
||||
context_settings={"max_content_width": _MAX_CONTENT_WIDTH},
|
||||
)
|
||||
@click.argument("project", type=click.Path(exists=True, file_okay=False, dir_okay=True), default=os.getcwd())
|
||||
def initialize(project: str) -> None:
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
name = manager.ensure_memory_maintenance_memory()
|
||||
click.echo(f"Memory maintenance memory ready: `mem:{name}`.")
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"list",
|
||||
@@ -1184,20 +1216,114 @@ class MemoryCommands(AutoRegisteringGroup):
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
click.echo(manager.save_memory(memory_name, content, is_tool_context=False))
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"delete",
|
||||
help="Delete a memory file. Use the `global/` prefix to address a global memory (e.g. `global/style_guide`).",
|
||||
context_settings={"max_content_width": _MAX_CONTENT_WIDTH},
|
||||
)
|
||||
@click.argument("memory_name", type=str)
|
||||
@click.argument("project", type=click.Path(exists=True, file_okay=False, dir_okay=True), default=os.getcwd())
|
||||
def delete(memory_name: str, project: str) -> None:
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
click.echo(manager.delete_memory(memory_name, is_tool_context=False))
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"rename",
|
||||
help=(
|
||||
"Rename or move a memory and update every `mem:OLD_NAME` reference across all memories. "
|
||||
"Use `/` in the name to organize into topics; use the `global/` prefix to address a "
|
||||
"global memory. Moving between project and global scope is supported."
|
||||
),
|
||||
context_settings={"max_content_width": _MAX_CONTENT_WIDTH},
|
||||
)
|
||||
@click.argument("old_name", type=str)
|
||||
@click.argument("new_name", type=str)
|
||||
@click.argument("project", type=click.Path(exists=True, file_okay=False, dir_okay=True), default=os.getcwd())
|
||||
def rename(old_name: str, new_name: str, project: str) -> None:
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
message, n_refs = manager.rename_memory_and_propagate_references(old_name, new_name, is_tool_context=False)
|
||||
click.echo(message)
|
||||
if n_refs > 0:
|
||||
click.echo(f"Updated {n_refs} `mem:` reference occurrence(s) across the memory graph.")
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"edit",
|
||||
help=(
|
||||
"Replace content matching a pattern in a memory. By default operates in literal "
|
||||
"(non-regex) mode and refuses to replace more than one occurrence; pass --mode regex "
|
||||
"to enable regex matching and --allow-multiple-occurrences to permit multiple hits."
|
||||
),
|
||||
context_settings={"max_content_width": _MAX_CONTENT_WIDTH},
|
||||
)
|
||||
@click.argument("memory_name", type=str)
|
||||
@click.argument("project", type=click.Path(exists=True, file_okay=False, dir_okay=True), default=os.getcwd())
|
||||
@click.option("--needle", type=str, required=True, help="The text to search for (literal by default; regex if --mode=regex).")
|
||||
@click.option("--repl", type=str, required=True, help="The replacement text (verbatim).")
|
||||
@click.option(
|
||||
"--mode",
|
||||
type=click.Choice(["literal", "regex"]),
|
||||
default="literal",
|
||||
show_default=True,
|
||||
help="Treat --needle as literal text or as a Python regex (MULTILINE flag is always on; DOTALL via --dotall).",
|
||||
)
|
||||
@click.option(
|
||||
"--allow-multiple-occurrences",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Permit and apply multiple matches; without this, multiple matches raise an error.",
|
||||
)
|
||||
@click.option(
|
||||
"--dotall",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Regex mode only: make `.` match newlines as well, enabling multi-line patterns. Use with care.",
|
||||
)
|
||||
def edit(
|
||||
memory_name: str,
|
||||
project: str,
|
||||
needle: str,
|
||||
repl: str,
|
||||
mode: Literal["literal", "regex"],
|
||||
allow_multiple_occurrences: bool,
|
||||
dotall: bool,
|
||||
) -> None:
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
click.echo(
|
||||
manager.edit_memory(
|
||||
memory_name,
|
||||
needle,
|
||||
repl,
|
||||
mode,
|
||||
allow_multiple_occurrences,
|
||||
is_tool_context=False,
|
||||
dotall=dotall,
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
@click.command(
|
||||
"check",
|
||||
help=(
|
||||
"Check referential integrity across all memories of the project (and global memories). "
|
||||
"Reports stale `mem:` references and unmarked-reference warnings split into high- and "
|
||||
"low-confidence groups. Read-only and never writes. Always exits 0."
|
||||
"Reports stale `mem:` references. With --include-unmarked, also reports bare occurrences "
|
||||
"of existing memory names that lack the `mem:` prefix (split into high- and low-confidence). "
|
||||
"Read-only and never writes. Always exits 0."
|
||||
),
|
||||
context_settings={"max_content_width": _MAX_CONTENT_WIDTH},
|
||||
)
|
||||
@click.argument("project", type=click.Path(exists=True, file_okay=False, dir_okay=True), default=os.getcwd())
|
||||
def check(project: str) -> None:
|
||||
@click.option(
|
||||
"--include-unmarked",
|
||||
is_flag=True,
|
||||
default=False,
|
||||
help="Also report potentially unmarked references (bare occurrences of memory names without the `mem:` prefix).",
|
||||
)
|
||||
def check(project: str, include_unmarked: bool) -> None:
|
||||
manager = MemoryCommands._load_memories_manager(project)
|
||||
report = manager.validate_referential_integrity()
|
||||
report = manager.validate_referential_integrity(include_unmarked=include_unmarked)
|
||||
click.echo(report.format())
|
||||
|
||||
@staticmethod
|
||||
|
||||
+58
-15
@@ -166,6 +166,9 @@ class MemoriesManager:
|
||||
# generic prose word like "repository" is a substring of "serena_repository_structure"
|
||||
# but their token Jaccard is only 1/3, so we reject it at this floor.
|
||||
_FUZZY_BARE_TOKEN_JACCARD_FLOOR: float = 0.5
|
||||
|
||||
_WORDS_TO_IGNORE_AS_MEMORY_NAME_CANDIDATES: frozenset[str] = frozenset({"core"})
|
||||
"""Names that we specifically mention in our instructions and that coincide with common words"""
|
||||
_VERSION_SUFFIX_PATTERN = re.compile(r"_(?:v\d+|old|new|legacy|bak)$", re.IGNORECASE)
|
||||
|
||||
@classmethod
|
||||
@@ -507,19 +510,19 @@ class MemoriesManager:
|
||||
Outcome of :meth:`MemoriesManager.validate_referential_integrity`.
|
||||
|
||||
:ivar stale_references: ``mem:NAME`` references whose target memory does not exist.
|
||||
:ivar high_confidence_warnings: bare references whose suspected target name is unlikely
|
||||
:ivar high_confidence_unmarked_memories: bare references whose suspected target name is unlikely
|
||||
to be coincidental prose (topic-path or sufficiently long).
|
||||
:ivar low_confidence_warnings: bare references whose suspected target name could plausibly
|
||||
:ivar low_confidence_unmarked_memories: bare references whose suspected target name could plausibly
|
||||
appear in ordinary prose (short, flat names).
|
||||
"""
|
||||
|
||||
stale_references: list["MemoriesManager.StaleReference"] = field(default_factory=list)
|
||||
high_confidence_warnings: list["MemoriesManager.UnmarkedReferenceWarning"] = field(default_factory=list)
|
||||
low_confidence_warnings: list["MemoriesManager.UnmarkedReferenceWarning"] = field(default_factory=list)
|
||||
high_confidence_unmarked_memories: list["MemoriesManager.UnmarkedReferenceWarning"] = field(default_factory=list)
|
||||
low_confidence_unmarked_memories: list["MemoriesManager.UnmarkedReferenceWarning"] = field(default_factory=list)
|
||||
|
||||
def is_clean(self) -> bool:
|
||||
""":return: True iff no stale references and no warnings of any confidence level were found."""
|
||||
return not (self.stale_references or self.high_confidence_warnings or self.low_confidence_warnings)
|
||||
return not (self.stale_references or self.high_confidence_unmarked_memories or self.low_confidence_unmarked_memories)
|
||||
|
||||
def format(self) -> str:
|
||||
""":return: a human-readable rendering suitable for CLI display."""
|
||||
@@ -543,8 +546,8 @@ class MemoriesManager:
|
||||
|
||||
# unmarked-reference warnings, split by confidence
|
||||
for label, warnings in (
|
||||
("High-confidence unmarked references", self.high_confidence_warnings),
|
||||
("Low-confidence unmarked references", self.low_confidence_warnings),
|
||||
("High-confidence unmarked references", self.high_confidence_unmarked_memories),
|
||||
("Low-confidence unmarked references", self.low_confidence_unmarked_memories),
|
||||
):
|
||||
if not warnings:
|
||||
continue
|
||||
@@ -714,6 +717,31 @@ class MemoriesManager:
|
||||
|
||||
return f"Memory renamed from {old_name} to {new_name}."
|
||||
|
||||
def rename_memory_and_propagate_references(self, old_name: str, new_name: str, is_tool_context: bool) -> tuple[str, int]:
|
||||
"""
|
||||
Renames a memory and updates every ``mem:OLD_NAME`` reference across all memories.
|
||||
|
||||
Memories whose content does not contain a reference to ``old_name`` are left
|
||||
untouched (no spurious mtime changes). Memories that do are rewritten via
|
||||
:meth:`save_memory`.
|
||||
|
||||
:param old_name: the current memory name (the source of the rename)
|
||||
:param new_name: the target memory name
|
||||
:param is_tool_context: forwarded to :meth:`save_memory` for read-only enforcement
|
||||
:return: a tuple of (rename message returned by :meth:`move_memory`, total number of
|
||||
``mem:`` reference occurrences rewritten across all memories).
|
||||
"""
|
||||
renaming_message = self.move_memory(old_name, new_name, is_tool_context=is_tool_context)
|
||||
|
||||
total_updates = 0
|
||||
for memory_name in self.list_memories().get_full_list():
|
||||
content = self.load_memory(memory_name)
|
||||
updated_content, n_replacements = self.rename_references_to_memory(content, old_name, new_name)
|
||||
if n_replacements > 0:
|
||||
self.save_memory(memory_name, updated_content, is_tool_context=is_tool_context)
|
||||
total_updates += n_replacements
|
||||
return renaming_message, total_updates
|
||||
|
||||
def edit_memory(
|
||||
self,
|
||||
name: str,
|
||||
@@ -760,7 +788,7 @@ class MemoriesManager:
|
||||
source_basename = source_memory.rsplit("/", 1)[-1]
|
||||
return suspected_name == source_basename
|
||||
|
||||
def validate_referential_integrity(self) -> "MemoriesManager.ReferentialIntegrityReport":
|
||||
def validate_referential_integrity(self, include_unmarked: bool = True) -> "MemoriesManager.ReferentialIntegrityReport":
|
||||
"""
|
||||
Scans every (non-ignored) memory's content for referential integrity issues.
|
||||
|
||||
@@ -773,7 +801,8 @@ class MemoriesManager:
|
||||
* **exact unmarked-reference warnings** — bare occurrences of an existing memory
|
||||
name that appear without the ``mem:`` prefix. Split into high-confidence (the
|
||||
suspected name contains a ``/`` or exceeds :attr:`_HIGH_CONFIDENCE_NAME_LENGTH`)
|
||||
and low-confidence groups.
|
||||
and low-confidence groups. Candidate names whose basename matches
|
||||
:attr:`WORDS_TO_IGNORE_AS_MEMORY_NAME_CANDIDATES` are skipped.
|
||||
* **fuzzy near-miss warnings** — long, distinctive bare tokens in a memory body
|
||||
that *do not* match an existing name exactly but similarity-match a
|
||||
high-confidence existing name AND share at least :attr:`_FUZZY_BARE_TOKEN_JACCARD_FLOOR`
|
||||
@@ -787,6 +816,8 @@ class MemoriesManager:
|
||||
|
||||
This method has no side effects.
|
||||
|
||||
:param include_unmarked: if False, skip the two unmarked-reference scans (exact and
|
||||
fuzzy near-miss) entirely; only stale references are reported.
|
||||
:return: a :class:`ReferentialIntegrityReport` summarizing all findings.
|
||||
"""
|
||||
report = MemoriesManager.ReferentialIntegrityReport()
|
||||
@@ -801,6 +832,10 @@ class MemoriesManager:
|
||||
high_confidence_names = [n for n in all_names if "/" in n or len(n) >= self._HIGH_CONFIDENCE_NAME_LENGTH]
|
||||
|
||||
for source_name in all_names:
|
||||
# content of this memory should not be analyzed regarding stale references
|
||||
if source_name == self.MEMORY_MAINTENANCE_NAME:
|
||||
continue
|
||||
|
||||
content = self.load_memory(source_name)
|
||||
if not content:
|
||||
# skip empty memories silently — no references can exist in empty content
|
||||
@@ -823,10 +858,16 @@ class MemoriesManager:
|
||||
)
|
||||
)
|
||||
|
||||
if not include_unmarked:
|
||||
continue
|
||||
|
||||
# exact unmarked references: bare occurrences of any existing memory name (except self)
|
||||
for candidate_name in all_names:
|
||||
if self._is_self_reference(source_name, candidate_name):
|
||||
continue
|
||||
# skip candidate names whose basename is an ambiguous common word
|
||||
if candidate_name.rsplit("/", 1)[-1] in self._WORDS_TO_IGNORE_AS_MEMORY_NAME_CANDIDATES:
|
||||
continue
|
||||
n = self._find_bare_occurrences(content, candidate_name)
|
||||
if n == 0:
|
||||
continue
|
||||
@@ -840,9 +881,9 @@ class MemoriesManager:
|
||||
actual_token=candidate_name,
|
||||
)
|
||||
if is_high_conf:
|
||||
report.high_confidence_warnings.append(warning)
|
||||
report.high_confidence_unmarked_memories.append(warning)
|
||||
else:
|
||||
report.low_confidence_warnings.append(warning)
|
||||
report.low_confidence_unmarked_memories.append(warning)
|
||||
|
||||
# fuzzy near-misses: long bare tokens that similarity-match a high-confidence existing name
|
||||
for token, n in self._iter_long_bare_tokens(content):
|
||||
@@ -855,6 +896,8 @@ class MemoriesManager:
|
||||
for existing in high_confidence_names:
|
||||
if self._is_self_reference(source_name, existing):
|
||||
continue
|
||||
if existing.rsplit("/", 1)[-1] in self._WORDS_TO_IGNORE_AS_MEMORY_NAME_CANDIDATES:
|
||||
continue
|
||||
score = self.compute_name_similarity(token, existing)
|
||||
if score < self.NAME_SIMILARITY_THRESHOLD:
|
||||
continue
|
||||
@@ -873,7 +916,7 @@ class MemoriesManager:
|
||||
best_name = existing
|
||||
if best_name is None:
|
||||
continue
|
||||
report.high_confidence_warnings.append(
|
||||
report.high_confidence_unmarked_memories.append(
|
||||
MemoriesManager.UnmarkedReferenceWarning(
|
||||
source_memory=source_name,
|
||||
suspected_name=best_name,
|
||||
@@ -933,11 +976,11 @@ class MemoriesManager:
|
||||
validation = self.validate_referential_integrity()
|
||||
|
||||
# combine the warning groups according to the include_flat_names policy
|
||||
warnings: list[MemoriesManager.UnmarkedReferenceWarning] = list(validation.high_confidence_warnings)
|
||||
warnings: list[MemoriesManager.UnmarkedReferenceWarning] = list(validation.high_confidence_unmarked_memories)
|
||||
if include_flat_names:
|
||||
warnings.extend(validation.low_confidence_warnings)
|
||||
warnings.extend(validation.low_confidence_unmarked_memories)
|
||||
else:
|
||||
report.skipped_flat.extend(validation.low_confidence_warnings)
|
||||
report.skipped_flat.extend(validation.low_confidence_unmarked_memories)
|
||||
|
||||
# apply replacements per source memory, grouping warnings to avoid re-reading content
|
||||
warnings_by_source: dict[str, list[MemoriesManager.UnmarkedReferenceWarning]] = {}
|
||||
|
||||
@@ -82,13 +82,11 @@ class RenameMemoryTool(Tool, ToolMarkerCanEdit):
|
||||
The "global" topic should only be used if explicitly instructed.
|
||||
References to other memories that are marked with the `mem:` prefix will be updated accordingly.
|
||||
"""
|
||||
renaming_message = self.memories_manager.move_memory(old_name, new_name, is_tool_context=True)
|
||||
for memory in self.memories_manager.list_memories().get_full_list():
|
||||
memory_content = self.memories_manager.load_memory(memory)
|
||||
updated_content, n_replacements = self.memories_manager.rename_references_to_memory(memory_content, old_name, new_name)
|
||||
if n_replacements > 0:
|
||||
log.info(f"Updated {n_replacements} references to memory {old_name} to {new_name}")
|
||||
self.memories_manager.save_memory(memory, updated_content, is_tool_context=True)
|
||||
renaming_message, n_references_updated = self.memories_manager.rename_memory_and_propagate_references(
|
||||
old_name, new_name, is_tool_context=True
|
||||
)
|
||||
if n_references_updated > 0:
|
||||
log.info(f"Updated {n_references_updated} references to memory {old_name} to {new_name}")
|
||||
return renaming_message
|
||||
|
||||
|
||||
|
||||
@@ -278,21 +278,21 @@ class TestValidateReferentialIntegrity:
|
||||
_write(fs_manager, "auth/login", "# notes")
|
||||
_write(fs_manager, "docs", "the auth/login process is documented elsewhere")
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
assert len(report.high_confidence_warnings) == 1
|
||||
warning = report.high_confidence_warnings[0]
|
||||
assert len(report.high_confidence_unmarked_memories) == 1
|
||||
warning = report.high_confidence_unmarked_memories[0]
|
||||
assert warning.suspected_name == "auth/login"
|
||||
assert warning.is_high_confidence is True
|
||||
assert warning.occurrences == 1
|
||||
assert report.low_confidence_warnings == []
|
||||
assert report.low_confidence_unmarked_memories == []
|
||||
|
||||
def test_low_confidence_unmarked_warning_for_short_flat_name(self, fs_manager: MemoriesManager) -> None:
|
||||
_write(fs_manager, "auth", "# notes")
|
||||
_write(fs_manager, "docs", "you must use auth here")
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
assert report.high_confidence_warnings == []
|
||||
assert len(report.low_confidence_warnings) == 1
|
||||
assert report.low_confidence_warnings[0].suspected_name == "auth"
|
||||
assert report.low_confidence_warnings[0].is_high_confidence is False
|
||||
assert report.high_confidence_unmarked_memories == []
|
||||
assert len(report.low_confidence_unmarked_memories) == 1
|
||||
assert report.low_confidence_unmarked_memories[0].suspected_name == "auth"
|
||||
assert report.low_confidence_unmarked_memories[0].is_high_confidence is False
|
||||
|
||||
def test_long_flat_name_is_high_confidence(self, fs_manager: MemoriesManager) -> None:
|
||||
# a flat name >= _HIGH_CONFIDENCE_NAME_LENGTH chars is unlikely to be coincidental prose
|
||||
@@ -301,7 +301,7 @@ class TestValidateReferentialIntegrity:
|
||||
_write(fs_manager, long_name, "# notes")
|
||||
_write(fs_manager, "docs", f"the {long_name} is documented here")
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
assert len(report.high_confidence_warnings) == 1
|
||||
assert len(report.high_confidence_unmarked_memories) == 1
|
||||
|
||||
def test_fuzzy_near_miss_rejects_substring_only_matches(self, fs_manager: MemoriesManager) -> None:
|
||||
"""
|
||||
@@ -328,8 +328,8 @@ class TestValidateReferentialIntegrity:
|
||||
)
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
assert report.is_clean(), (
|
||||
f"Expected no findings; got high={report.high_confidence_warnings}, "
|
||||
f"low={report.low_confidence_warnings}, stale={report.stale_references}"
|
||||
f"Expected no findings; got high={report.high_confidence_unmarked_memories}, "
|
||||
f"low={report.low_confidence_unmarked_memories}, stale={report.stale_references}"
|
||||
)
|
||||
|
||||
def test_long_flat_name_near_miss_is_high_confidence(self, fs_manager: MemoriesManager) -> None:
|
||||
@@ -350,12 +350,12 @@ class TestValidateReferentialIntegrity:
|
||||
_write(fs_manager, "docs", f"See {bare} for details")
|
||||
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
assert len(report.high_confidence_warnings) == 1, (
|
||||
assert len(report.high_confidence_unmarked_memories) == 1, (
|
||||
f"expected one high-confidence warning for the near-miss reference, "
|
||||
f"got {len(report.high_confidence_warnings)} "
|
||||
f"(high={report.high_confidence_warnings}, low={report.low_confidence_warnings})"
|
||||
f"got {len(report.high_confidence_unmarked_memories)} "
|
||||
f"(high={report.high_confidence_unmarked_memories}, low={report.low_confidence_unmarked_memories})"
|
||||
)
|
||||
warning = report.high_confidence_warnings[0]
|
||||
warning = report.high_confidence_unmarked_memories[0]
|
||||
assert warning.suspected_name == existing
|
||||
assert warning.source_memory == "docs"
|
||||
|
||||
@@ -367,7 +367,7 @@ class TestValidateReferentialIntegrity:
|
||||
# `auth/login` mentioning its own basename `login` should not produce a warning;
|
||||
# `auth` mentioned in `auth/login` content also shouldn't (it's the prefix), and only
|
||||
# actual matches matter — assert no warning concerns the source itself
|
||||
for w in report.high_confidence_warnings + report.low_confidence_warnings:
|
||||
for w in report.high_confidence_unmarked_memories + report.low_confidence_unmarked_memories:
|
||||
assert not (w.source_memory == "auth/login" and w.suspected_name in ("login", "auth/login"))
|
||||
|
||||
def test_marked_reference_does_not_produce_unmarked_warning(self, fs_manager: MemoriesManager) -> None:
|
||||
@@ -382,8 +382,8 @@ class TestValidateReferentialIntegrity:
|
||||
_write(fs_manager, "docs", "we have `mem:auth/login` plus a bare auth/login mention")
|
||||
report = fs_manager.validate_referential_integrity()
|
||||
# only the bare occurrence should be counted
|
||||
assert len(report.high_confidence_warnings) == 1
|
||||
assert report.high_confidence_warnings[0].occurrences == 1
|
||||
assert len(report.high_confidence_unmarked_memories) == 1
|
||||
assert report.high_confidence_unmarked_memories[0].occurrences == 1
|
||||
|
||||
def test_empty_memory_skipped(self, fs_manager: MemoriesManager) -> None:
|
||||
_write(fs_manager, "auth/login", "")
|
||||
@@ -397,7 +397,7 @@ class TestValidateReferentialIntegrity:
|
||||
_write(manager, "frozen/notes", "see bare auth/login mention")
|
||||
_write(manager, "auth/login", "# notes")
|
||||
report = manager.validate_referential_integrity()
|
||||
warnings = report.high_confidence_warnings
|
||||
warnings = report.high_confidence_unmarked_memories
|
||||
assert len(warnings) == 1
|
||||
assert warnings[0].source_memory == "frozen/notes"
|
||||
assert warnings[0].source_is_read_only is True
|
||||
|
||||
Reference in new issue
Block a user