Deleted ReplaceTextSpanTool. Extended prompting to make ReplaceRegexTool work

The editing mode now disabled line-based editing tools. These tools may be completely removed in the future

config/modes/editing.yml

@@ -4,4 +4,74 @@ prompt: |
   to implement the requested changes to the code base while adhering to the project's code style and patterns.
   Use symbolic editing tools whenever possible for precise code modifications.
   If no editing task has yet been provided, wait for the user to provide one.
-excluded_tools: []
+
+  Your primary tool for editing code is a regex-based replacement. You use other tools to find the relevant content and
+  then use your knowledge of the codebase to write the regex.
+  You are extremely good at regex, so you never need to check whether the replacement produced the correct result.
+  In particular, you know what to escape and what not to escape, and you know how to use wildcards.
+  Moreover, the replacement tool will fail if it can't perform the desired replacement, and this is all the feedback you need.
+  Your overall goal for replacement operations is to use relatively short regexes, since I want you to minimize the number
+  of output tokens. For replacements of larger chunks of code, this means you intelligently make use of wildcards for the middle part 
+  and of characteristic snippets for the before/after parts that uniquely identify the chunk.
+  
+  
+  For small replacements, up to a single line, you follow the following rules:
+
+    1. If the snippet to be replaced is likely to be unique within the file, you perform the replacement by directly using the escaped version of the 
+       original.
+    2. If the snippet is probably not unique, and you want to replace all occurrences, you use the `allow_multiple_occurrences` flag.
+    3. If the snippet is not unique, and you want to replace a specific occurence, you make use of the code surrounding the snippet
+       to extend the regex with content before/after such that the regex will have exactly one match.
+    4. You generally assume that a snippet is unique, knowing that the tool will return an error on multiple matches. You only read more file content
+       (for crafvarting a more specific regex) if such a failure unexpectedly occurs. 
+
+  Examples:
+
+  1 Small replacement
+  You have read code like
+    
+    ```python
+    ...
+    x = linear(x)
+    x = relu(x)
+    return x
+    ...
+    ```
+
+  and you want to replace `x = relu(x)` with `x = gelu(x)`.
+  You first try `replace_regex()` with the regex `x = relu\(x\)` and the replacement `x = gelu(x)`.
+  If this fails due to multiple matches, you will try `(linear\(x\)\s*)x = relu\(x\)(\s*return)` with the replacement `\1x = gelu(x)\2`.
+
+  2 Larger replacement
+
+  You have read code like
+
+  ```python
+  def my_func():
+    ...
+    # a comment before the snippet
+    x = add_fifteen(x)
+    # beginning of long section within my_func
+    ....
+    # end of long section
+    call_subroutine(z)
+    call_second_subroutine(z)
+  ```
+  and you want to replace the code starting with `x = add_fifteen(x)` until (including) `call_subroutine(z)`, but not `call_second_subroutine(z)`.
+  Initially, you assume that the the beginning and end of the chunk uniquely determine it within the file.
+  Therefore, you perform the replacement by using the regex `x = add_fifteen\(x\)\s*.*?call_subroutine\(z\)`
+  and the replacement being the new code you want to insert.
+
+  If this fails due to multiple matches, you will try to extend the regex with the content before/after the snippet and match groups. 
+  The matching regex becomes:
+  `(before the snippet\s*)x = add_fifteen\(x\)\s*.*?call_subroutine\(z\)` 
+  and the replacement includes the group as (schematically):
+  `\1<new_code>`
+
+  Generally, I remind you that you rely on the regex tool with providing you the correct feedback, no need for more verification!
+
+  IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
+excluded_tools: 
+ - replace_lines
+ - insert_at_line
+ - delete_lines

lessons_learned.md

@@ -48,6 +48,19 @@ Many more well known python implementations of language servers were subpar in c
 
 We clearly notice that the better the tool gets, the easier it is to make it even better
 
+## Prompting
+
+### Shouting and Emotive Language May Be Needed
+
+When developing the `ReplaceRegexTool` we were initially not able to make Claude 4 (in Claude Desktop) use wildcards to save on output tokens. Neither
+examples nor explicit instructions helped. It was only after adding 
+
+```
+IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
+```
+
+to the initial instructions and to the tool description that Claude finally started following the instructions.
+
 ## What Didn't Work
 
 ### Lifespan Handling by MCP Clients

src/serena/agent.py

@@ -1457,6 +1457,8 @@ class ReplaceRegexTool(Tool, ToolMarkerCanEdit):
         Always try to use wildcards to avoid specifying the exact content of the code to be replaced,
         especially if it spans several lines.
 
+        IMPORTANT: REMEMBER TO USE WILDCARDS WEHEN APPROPRIATE! I WILL BE VERY UNHAPPY IF YOU WRITE LONG REGEXES WITHOUT USING WILDCARDS INSTEAD!
+
         :param relative_path: the relative path to the file
         :param regex: a Python-style regular expression, matches of which will be replaced.
             Dot matches all characters, multi-line matching is enabled.
@@ -1481,40 +1483,6 @@ class ReplaceRegexTool(Tool, ToolMarkerCanEdit):
         return SUCCESS_RESULT
 
 
-class ReplaceTextSpanTool(Tool, ToolMarkerCanEdit):
-    """
-    Replaces text in a file, with the replaced section being demarcated by text snippets.
-    """
-
-    def apply(
-        self,
-        relative_path: str,
-        start_snippet: str,
-        end_snippet: str,
-        repl: str,
-    ) -> str:
-        r"""
-        Replaces the text starting with the given start snippet and ending with the end snippet.
-        The start and end snippets should serve only to delimit the text to be replaced.
-        They must not overlap and must uniquely identify the section to be replaced.
-        They should not be longer than necessary to demarcate the section to be replaced.
-
-        For example, to transform "before start long-middle end after" into "before new after",
-        one would set start_snippet="start" and end_snippet="end", and repl="new",
-        avoiding the need to specify the entire middle section.
-
-        :param relative_path: the relative path to the file
-        :param start_snippet: the text snippet marking the beginning of the section to be replaced.
-        :param end_snippet: the text snippet marking the end of the section to be replaced.
-        """
-        return self.agent.get_tool(ReplaceRegexTool).apply(
-            relative_path,
-            re.escape(start_snippet) + r".*?" + re.escape(end_snippet),
-            repl,
-            allow_multiple_occurrences=False,
-        )
-
-
 class DeleteLinesTool(Tool, ToolMarkerCanEdit):
     """
     Deletes a range of lines within a file.