fix(admin): second review pass

Correctness
- /api/remote never recorded source.created, so URL, GitHub and connector
  sources had a source.deleted with no matching creation. All three creation
  paths now go through one _audit_source_created helper.
- The prompt-cache rate divided cached tokens by a whole bucket's prompt
  tokens. A bucket is a day and mixes calls whose provider reports a cache
  breakdown with calls whose provider does not, so filtering buckets in the
  client could not separate them and the rate was understated by however much
  traffic ran on a non-reporting provider. The denominator is now computed in
  SQL over the reporting rows.
- The outcome pill matched values nothing writes. Guardrails emit triggered /
  not_evaluated and the device feed emits dispatched; the map had blocked /
  denied / allowed, so a guardrail that fired rendered neutral grey -- the one
  signal the merged feed exists to surface. Fixtures were seeding the
  fictional values, so the tests passed on it too.
- Stream duration_ms timed the consumer. stream_token_usage is a generator,
  so start-to-exhaustion includes the agent loop's tool handling and the SSE
  client's pace; a slow browser recorded ~30s for a sub-second call. It now
  accumulates only the time spent inside next().

Safety
- Activity filters failed open: an unknown facet or unparseable timestamp was
  dropped, and no filter means every row, so a typo widened an audit view and
  on the export streamed the full history. Both are now a 400.
- The search term was interpolated into an ILIKE pattern, so "100%" matched
  everything and "q1_report" matched more than it should. Escaped.
- 0034 set actor_id NOT NULL with no default. A previous-release process
  inserting mid-rollout would raise, and in admin/routes.py that insert shares
  the request transaction, so a role grant beside it would roll back too.

Noise and dead code
- The per-user panel is a security panel: data-plane events file under the
  actor, so an active account's routine deletes pushed a denied login out of
  the 20-row window. It now excludes them; the Activity tab shows everything.
- device_audit_log had no created_at-leading index, so the merged feed
  sequentially scanned that branch every page (migration 0036).
- conversation.deleted_all no longer records when nothing was deleted, and
  agent.updated no longer records an empty field list.
- Dropped by_model from /admin/usage (no consumer; an extra aggregate per page
  load), the duplicate filter surface on AuthEventsRepository that nothing
  called, and the unreachable FLOW_LABELS.schedule entry.
- Type hints on record_event's conn and the remaining unannotated helpers.

docsgpt/storage/db/base_repository.py

@@ -65,3 +65,21 @@ def row_to_dict(row: Any) -> dict:
         out["_id"] = out["id"]
 
     return out
+
+
+def like_escape(term: str) -> str:
+    """Escape LIKE/ILIKE metacharacters so ``term`` matches literally.
+
+    A search box takes a substring, not a pattern. Interpolated raw, ``%``
+    matches everything and ``_`` matches any single character, so searching
+    for ``100%`` or ``q1_report`` silently returns the wrong rows.
+
+    Callers must pair this with ``ESCAPE '\\'`` on the comparison.
+
+    Args:
+        term: The user-supplied substring.
+
+    Returns:
+        ``term`` with ``\\``, ``%`` and ``_`` backslash-escaped.
+    """
+    return term.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")

docsgpt/storage/db/models.py

@@ -250,9 +250,11 @@ token_usage_table = Table(
     # Added in ``0033_quotas``. USD cost of the call at write time; 0 for
     # unpriced and bring-your-own models.
     Column("cost", Numeric(12, 8), nullable=False, server_default="0"),
-    # Added in ``0035_token_usage_latency``. Wall-clock for the call, and time
-    # to the first streamed chunk. ``ttft_ms`` is NULL for non-streaming calls
-    # and for streams that failed before yielding -- "no first token", not 0.
+    # Added in ``0035_token_usage_latency``. Time spent waiting on the
+    # provider (a stream excludes consumer backpressure; see ``usage.py``),
+    # and time to the first streamed chunk. ``ttft_ms`` is NULL for
+    # non-streaming calls and for streams that failed before yielding --
+    # "no first token", not 0.
     Column("duration_ms", Integer),
     Column("ttft_ms", Integer),
 )

docsgpt/storage/db/repositories/activity.py

@@ -25,6 +25,7 @@ from typing import Any, Iterator, Optional, Sequence
 from sqlalchemy import Connection, text
 
 from docsgpt.audit_events import category_case_sql
+from docsgpt.storage.db.base_repository import like_escape
 
 
 # Columns every branch of the union produces, in order.
@@ -197,12 +198,18 @@ class ActivityRepository:
             clauses.append("created_at <= :until")
             params["until"] = until
         if search:
+            # ``search`` is a literal substring, not a pattern: an operator
+            # looking for "100%" or "q1_report" must not have those characters
+            # read as wildcards.
             clauses.append(
-                "(actor_id ILIKE :search OR target_id ILIKE :search "
-                "OR event ILIKE :search OR ip ILIKE :search "
-                "OR COALESCE(outcome, '') ILIKE :search OR detail::text ILIKE :search)"
+                "(actor_id ILIKE :search ESCAPE '\\' "
+                "OR target_id ILIKE :search ESCAPE '\\' "
+                "OR event ILIKE :search ESCAPE '\\' "
+                "OR ip ILIKE :search ESCAPE '\\' "
+                "OR COALESCE(outcome, '') ILIKE :search ESCAPE '\\' "
+                "OR detail::text ILIKE :search ESCAPE '\\')"
             )
-            params["search"] = f"%{search}%"
+            params["search"] = f"%{like_escape(search)}%"
         # Always a WHERE, even with no filters, so the keyset cursor below can
         # append " AND ..." without sniffing the string it was handed.
         where = "WHERE " + " AND ".join(clauses or ["TRUE"])
@@ -259,7 +266,10 @@ class ActivityRepository:
         return [dict(row._mapping) for row in result.fetchall()]
 
     def _page_after(
-        self, cursor: Optional[tuple], limit: int, filters: dict
+        self,
+        cursor: Optional[tuple[datetime, str, str]],
+        limit: int,
+        filters: dict,
     ) -> list[dict]:
         """One keyset page strictly older than ``cursor``.
 
@@ -332,7 +342,7 @@ class ActivityRepository:
             Row dicts carrying :data:`ACTIVITY_COLUMNS`.
         """
         emitted = 0
-        cursor: Optional[tuple] = None
+        cursor: Optional[tuple[datetime, str, str]] = None
         while emitted < max_rows:
             page = self._page_after(
                 cursor, min(chunk_size, max_rows - emitted), filters

docsgpt/storage/db/repositories/admin_stats.py

@@ -17,7 +17,7 @@ from docsgpt.storage.db.base_repository import row_to_dict
 from docsgpt.storage.db.repositories.token_usage import TokenUsageRepository
 
 
-def _round_ms(value) -> Optional[int]:
+def _round_ms(value: Optional[float]) -> Optional[int]:
     """Round a percentile to whole milliseconds, preserving NULL as None."""
     return int(round(float(value))) if value is not None else None
 

docsgpt/storage/db/repositories/auth_events.py

@@ -88,18 +88,44 @@ class AuthEventsRepository:
         )
         return row_to_dict(result.fetchone())
 
-    def list_recent(self, user_id: str, limit: int = 50) -> list[dict]:
-        """Return the newest events for ``user_id``, newest first."""
+    def list_recent(
+        self,
+        user_id: str,
+        limit: int = 50,
+        *,
+        exclude_events_like: Optional[Sequence[str]] = None,
+    ) -> list[dict]:
+        """Return the newest events for ``user_id``, newest first.
+
+        Args:
+            user_id: The subject whose trail to read.
+            limit: How many rows to return.
+            exclude_events_like: Event-name prefixes to drop, e.g.
+                ``("source.", "agent.")``. Data-plane events are filed under
+                the actor, so on an active account they would otherwise push
+                a denied login or a role grant out of a short window within
+                minutes.
+
+        Returns:
+            Matching rows, newest first.
+        """
+        clauses = ["user_id = :user_id"]
+        params: dict = {"user_id": user_id, "limit": limit}
+        for index, prefix in enumerate(exclude_events_like or ()):
+            key = f"excl_{index}"
+            clauses.append(f"event NOT LIKE :{key}")
+            params[key] = f"{prefix}%"
+        where = " AND ".join(clauses)
         result = self._conn.execute(
             text(
-                """
+                f"""
                 SELECT * FROM auth_events
-                WHERE user_id = :user_id
+                WHERE {where}
                 ORDER BY created_at DESC
                 LIMIT :limit
                 """
             ),
-            {"user_id": user_id, "limit": limit},
+            params,
         )
         return [row_to_dict(row) for row in result.fetchall()]
 
@@ -108,41 +134,18 @@ class AuthEventsRepository:
         event: Optional[str],
         user_id: Optional[str],
         since: Any,
-        *,
-        events: Optional[Sequence[str]] = None,
-        actor_id: Optional[str] = None,
-        until: Any = None,
-        search: Optional[str] = None,
     ) -> tuple[str, dict]:
         clauses: list[str] = []
         params: dict = {}
         if event:
             clauses.append("event = :event")
             params["event"] = event
-        if events:
-            clauses.append("event = ANY(:events)")
-            params["events"] = list(events)
         if user_id:
             clauses.append("(user_id = :user_id OR target_id = :user_id)")
             params["user_id"] = user_id
-        if actor_id:
-            clauses.append("actor_id = :actor_id")
-            params["actor_id"] = actor_id
         if since is not None:
             clauses.append("created_at >= :since")
             params["since"] = since
-        if until is not None:
-            clauses.append("created_at <= :until")
-            params["until"] = until
-        if search:
-            # Substring match across the human-meaningful columns plus the
-            # serialized metadata, so "ci-deploy-key" finds the PAT it named.
-            clauses.append(
-                "(user_id ILIKE :search OR actor_id ILIKE :search "
-                "OR target_id ILIKE :search OR event ILIKE :search "
-                "OR ip ILIKE :search OR metadata::text ILIKE :search)"
-            )
-            params["search"] = f"%{search}%"
         where = ("WHERE " + " AND ".join(clauses)) if clauses else ""
         return where, params
 
@@ -150,25 +153,18 @@ class AuthEventsRepository:
         self,
         *,
         event: Optional[str] = None,
-        events: Optional[Sequence[str]] = None,
         user_id: Optional[str] = None,
-        actor_id: Optional[str] = None,
         since=None,
-        until=None,
-        search: Optional[str] = None,
         limit: int = 50,
         offset: int = 0,
     ) -> list[dict]:
-        """Global audit feed (admin), newest first; see :meth:`_filter_clauses`."""
-        where, params = self._filter_clauses(
-            event,
-            user_id,
-            since,
-            events=events,
-            actor_id=actor_id,
-            until=until,
-            search=search,
-        )
+        """Single-journal audit feed for ``GET /api/admin/audit``, newest first.
+
+        The admin UI reads the merged feed instead; richer filtering lives on
+        :class:`~docsgpt.storage.db.repositories.activity.ActivityRepository`
+        rather than being implemented twice here.
+        """
+        where, params = self._filter_clauses(event, user_id, since)
         params.update({"limit": int(limit), "offset": int(offset)})
         result = self._conn.execute(
             text(
@@ -183,37 +179,14 @@ class AuthEventsRepository:
         self,
         *,
         event: Optional[str] = None,
-        events: Optional[Sequence[str]] = None,
         user_id: Optional[str] = None,
-        actor_id: Optional[str] = None,
         since=None,
-        until=None,
-        search: Optional[str] = None,
     ) -> int:
         """Total matching the same filters as :meth:`list_all` (for pagination)."""
-        where, params = self._filter_clauses(
-            event,
-            user_id,
-            since,
-            events=events,
-            actor_id=actor_id,
-            until=until,
-            search=search,
-        )
+        where, params = self._filter_clauses(event, user_id, since)
         return int(
             self._conn.execute(
                 text(f"SELECT count(*) FROM auth_events {where}"), params
             ).scalar()
             or 0
         )
-
-    def event_names(self) -> list[str]:
-        """Distinct event names present in the table, alphabetically.
-
-        Feeds the admin filter's event picker, so operators pick from what the
-        instance actually recorded instead of typing a name from memory.
-        """
-        result = self._conn.execute(
-            text("SELECT DISTINCT event FROM auth_events ORDER BY event")
-        )
-        return [row[0] for row in result.fetchall()]

docsgpt/storage/db/repositories/token_usage.py

@@ -306,7 +306,18 @@ class TokenUsageRepository:
                        COALESCE(SUM(tu.cost), 0) AS cost,
                        -- NULL cache bins mean "provider did not report", so
                        -- they stay out of the sum rather than reading as 0.
-                       SUM(tu.cached_tokens) AS cached_tokens
+                       SUM(tu.cached_tokens) AS cached_tokens,
+                       -- The matching denominator: prompt tokens from the
+                       -- rows that reported. Dividing ``cached_tokens`` by
+                       -- ``prompt_tokens`` instead would understate the hit
+                       -- rate by however much traffic ran on a provider that
+                       -- reports nothing, since those rows are in one sum but
+                       -- not the other.
+                       COALESCE(
+                           SUM(tu.prompt_tokens)
+                               FILTER (WHERE tu.cached_tokens IS NOT NULL),
+                           0
+                       ) AS cache_eligible_prompt_tokens
                        {group_select}
                 FROM token_usage tu
                 {join}
@@ -328,6 +339,9 @@ class TokenUsageRepository:
                     if row._mapping["cached_tokens"] is not None
                     else None
                 ),
+                "cache_eligible_prompt_tokens": int(
+                    row._mapping["cache_eligible_prompt_tokens"]
+                ),
                 **(
                     {"group_key": row._mapping["group_key"]}
                     if group_by is not None