fix: accurate per-call token usage and oversized-context guards

Token accounting:
- Drain each tool round's provider stream to exhaustion before running
  tools and recursing, so the usage decorator persists exactly one
  token_usage row per LLM call, at call end. Previously every round's
  generator was abandoned mid-iteration and flushed together at request
  teardown, writing N near-identical rows stamped with the final
  round's provider counts (duplicate billing).
- Consume the Chat Completions include_usage terminal chunk (it arrives
  after finish_reason and was never read) so streamed calls record
  provider-exact token counts instead of tiktoken estimates.
- Claim provider-reported usage once per call (_last_usage_claimed) so
  a late-finalized generator can never adopt another call's counts.

Oversized-context guards:
- Enforce Responses API function_call/function_call_output pairing in
  the input builder (drop unpaired items; bypassed for store-mode
  previous_response_id chaining where calls are matched server-side).
- Hard pre-send context gate: shrink oversized tool results and refuse
  payloads that cannot fit the model's window before dispatch, so a
  hopeless request is never sent or billed.
- Cap a single tool result entering the LLM context
  (TOOL_RESULT_MAX_TOKENS, default 20000); the tool journal and
  persistence keep the full result. Applied on the resume/continuation
  path too.
- Skip the fallback attempt when the payload cannot fit the fallback
  model's context window (10% estimation slack).
- Compression: never save a compression point that does not reduce
  tokens; bound oversized verbatim fields kept after a compression
  point (COMPRESSION_RECENT_FIELD_MAX_TOKENS, default 8000).

Robustness fixes from review:
- Google parallel function calls: complete index-less ToolCalls are no
  longer merged into one another (dict arguments raised TypeError on
  +=; second call could execute with the first call's arguments).
- Trailing-frame failures after a delivered answer no longer error the
  stream or restream the whole answer from the fallback
  (_stream_reached_finish).
- In-memory compression falls back to minimal pruning when the summary
  is not smaller than the original.
- keep<=0 guard in the middle-truncation helpers (a tiny cap returned
  marker + full text).

Frontend: tooltip on the Analytics tokens stat card explaining that
agent tool loops re-send conversation context on every step.

tests/llm/handlers/test_context_gate.py

@@ -0,0 +1,206 @@
+"""Pre-send context gate and per-tool-result cap.
+
+Covers the two guards that keep tool-loop payloads inside the model's
+context window: ``_bound_tool_response_for_llm`` (one giant tool result
+must not enter the message array uncapped) and
+``BaseAgent._enforce_context_window`` (an over-window payload is shrunk
+or refused *before* the usage decorators run).
+"""
+
+from unittest.mock import Mock, patch
+
+import pytest
+
+from application.agents.base import BaseAgent
+from application.llm.handlers.base import (
+    LLMHandler,
+    LLMResponse,
+    ToolCall,
+    _bound_tool_response_for_llm,
+)
+
+
+class GateHandler(LLMHandler):
+    def parse_response(self, response):
+        kind = response.get("kind")
+        if kind == "tool_call":
+            return LLMResponse(
+                content="",
+                tool_calls=[ToolCall(id="1", name="t", arguments="{}", index=0)],
+                finish_reason="tool_calls",
+                raw_response=None,
+            )
+        return LLMResponse(
+            content="", tool_calls=[], finish_reason="stop", raw_response=None
+        )
+
+    def create_tool_message(self, tool_call, result):
+        return {
+            "role": "tool",
+            "tool_call_id": tool_call.id,
+            "content": result if isinstance(result, str) else str(result),
+        }
+
+    def _iterate_stream(self, response):
+        yield from response
+
+
+class MockAgent(BaseAgent):
+    def _gen_inner(self, query, log_context=None):
+        yield {"answer": "test"}
+
+
+@pytest.fixture
+def agent():
+    a = MockAgent(
+        endpoint="test",
+        llm_name="openai",
+        model_id="gpt-4o",
+        api_key="test-key",
+    )
+    a.llm = Mock()
+    return a
+
+
+@pytest.mark.unit
+class TestBoundToolResponse:
+    def test_small_result_passes_through_unchanged(self):
+        result = {"status": "ok", "data": "small"}
+        assert _bound_tool_response_for_llm(result) is result
+
+    def test_oversized_result_is_truncated(self, monkeypatch):
+        monkeypatch.setattr(
+            "application.core.settings.settings.TOOL_RESULT_MAX_TOKENS",
+            30,
+            raising=False,
+        )
+        big = "word " * 2000
+        bounded = _bound_tool_response_for_llm(big)
+        assert "tool result truncated" in bounded
+        assert len(bounded) < len(big)
+
+    def test_zero_cap_disables_truncation(self, monkeypatch):
+        monkeypatch.setattr(
+            "application.core.settings.settings.TOOL_RESULT_MAX_TOKENS",
+            0,
+            raising=False,
+        )
+        big = "word " * 2000
+        assert _bound_tool_response_for_llm(big) is big
+
+    def test_handle_tool_calls_bounds_the_llm_copy(self, monkeypatch):
+        """The message handed to the LLM is capped even though the executor
+        returned the full result (journal/persistence keep the original)."""
+        monkeypatch.setattr(
+            "application.core.settings.settings.TOOL_RESULT_MAX_TOKENS",
+            30,
+            raising=False,
+        )
+        handler = GateHandler()
+        big = "word " * 2000
+
+        def fake_executor(tools_dict, call):
+            yield {"type": "tool_call", "data": {}}
+            return big, "call_1"
+
+        mock_agent = Mock()
+        mock_agent._check_context_limit = Mock(return_value=False)
+        mock_agent._execute_tool_action = Mock(side_effect=fake_executor)
+        mock_agent.tool_executor.check_pause = Mock(return_value=None)
+
+        gen = handler.handle_tool_calls(
+            mock_agent,
+            [ToolCall(id="call_1", name="t", arguments="{}", index=0)],
+            {},
+            [],
+        )
+        while True:
+            try:
+                next(gen)
+            except StopIteration as e:
+                messages, pending = e.value
+                break
+
+        tool_messages = [m for m in messages if m.get("role") == "tool"]
+        assert len(tool_messages) == 1
+        assert "tool result truncated" in tool_messages[0]["content"]
+        assert len(tool_messages[0]["content"]) < len(big)
+        assert pending is None
+
+
+@pytest.mark.unit
+class TestEnforceContextWindow:
+    def test_within_window_returns_messages_untouched(self, agent):
+        messages = [{"role": "user", "content": "hello"}]
+        with patch(
+            "application.core.model_utils.get_token_limit", return_value=1000
+        ):
+            assert agent._enforce_context_window(messages) is messages
+
+    def test_over_window_shrinks_tool_messages(self, agent):
+        messages = [
+            {"role": "user", "content": "question"},
+            {"role": "tool", "tool_call_id": "1", "content": "word " * 2000},
+        ]
+        with patch(
+            "application.core.model_utils.get_token_limit", return_value=1000
+        ):
+            shrunk = agent._enforce_context_window(messages)
+        tool_content = shrunk[1]["content"]
+        assert "truncated to fit context limit" in tool_content
+        assert agent._calculate_current_context_tokens(shrunk) < 1000
+
+    def test_impossible_payload_raises_before_dispatch(self, agent):
+        # The bulk is NOT in tool messages, so shrinking cannot help.
+        messages = [{"role": "user", "content": "word " * 3000}]
+        with patch(
+            "application.core.model_utils.get_token_limit", return_value=100
+        ):
+            with pytest.raises(ValueError, match="exceeds the model's context window"):
+                agent._enforce_context_window(messages)
+
+    def test_streaming_loop_gates_before_next_round(self):
+        """handle_streaming must run the gate before dispatching the next
+        round's gen_stream."""
+        handler = GateHandler()
+        order = []
+
+        mock_agent = Mock()
+        mock_agent.context_limit_reached = False
+        mock_agent.tools = []
+        mock_agent.model_id = "m"
+        mock_agent._enforce_context_window = Mock(
+            side_effect=lambda msgs: (order.append("gate"), msgs)[1]
+        )
+        mock_agent.llm.gen_stream = Mock(
+            side_effect=lambda **kw: (order.append("dispatch"), [{"kind": "stop"}])[1]
+        )
+
+        def fake_tool_calls(a, calls, tools_dict, messages, **kwargs):
+            yield {"type": "tool_call", "data": {}}
+            return messages, None
+
+        with patch.object(handler, "handle_tool_calls", fake_tool_calls):
+            list(
+                handler.handle_streaming(
+                    mock_agent, [{"kind": "tool_call"}], {}, []
+                )
+            )
+
+        assert order == ["gate", "dispatch"]
+
+
+@pytest.mark.unit
+class TestTinyCapTruncation:
+    def test_tiny_cap_never_returns_more_than_original(self, monkeypatch):
+        """keep==0 used to produce marker + FULL text (text[-0:] is the
+        whole string) — a 'truncation' that grows the payload."""
+        monkeypatch.setattr(
+            "application.core.settings.settings.TOOL_RESULT_MAX_TOKENS",
+            1,
+            raising=False,
+        )
+        big = "word " * 500
+        bounded = _bound_tool_response_for_llm(big)
+        assert len(bounded) < len(big)
+        assert "truncated" in bounded

tests/llm/handlers/test_review_regressions.py

@@ -0,0 +1,263 @@
+"""Regression tests from the adversarial review of the stream-drain change.
+
+Covers: Google-style index-less parallel tool calls (complete payloads must
+never be merged), trailing-frame failures after finish_reason (must not fail
+or fallback-restream a delivered answer), and the in-memory compression
+path surviving the negative-savings ValueError via minimal pruning.
+"""
+
+from unittest.mock import Mock, patch
+
+import pytest
+
+from application.llm.handlers.base import LLMHandler, LLMResponse, ToolCall
+
+
+class ScriptedHandler(LLMHandler):
+    """Chunks are dicts carrying a prebuilt LLMResponse under 'resp'."""
+
+    def parse_response(self, response):
+        return response["resp"]
+
+    def create_tool_message(self, tool_call, result):
+        return {
+            "role": "tool",
+            "tool_call_id": tool_call.id,
+            "content": str(result),
+        }
+
+    def _iterate_stream(self, response):
+        yield from response
+
+
+def _agent(llm=None):
+    agent = Mock()
+    agent.llm = llm or Mock()
+    agent.tools = []
+    agent.model_id = "test-model"
+    agent.context_limit_reached = False
+    agent._check_context_limit = Mock(return_value=False)
+    agent._enforce_context_window = Mock(side_effect=lambda msgs: msgs)
+    # Explicit falsy values: auto-created Mock attributes are truthy and
+    # would trip the provider-finished check in the drain loop.
+    agent.llm._stream_reached_finish = False
+    agent.llm._fallback_llm = None
+    return agent
+
+
+@pytest.mark.unit
+class TestIndexlessParallelToolCalls:
+    def test_google_style_parallel_calls_stay_separate(self):
+        """Two complete index-less calls (dict arguments) must reach
+        handle_tool_calls as two distinct calls — the merge branch would
+        raise ``dict += dict`` or execute tool_B with tool_A's args."""
+        handler = ScriptedHandler()
+        chunk_a = {
+            "resp": LLMResponse(
+                content="",
+                tool_calls=[
+                    ToolCall(id="a", name="tool_A", arguments={"x": 1}, index=None)
+                ],
+                finish_reason="tool_calls",
+                raw_response=None,
+            )
+        }
+        chunk_b = {
+            "resp": LLMResponse(
+                content="",
+                tool_calls=[
+                    ToolCall(id="b", name="tool_B", arguments={"y": 2}, index=None)
+                ],
+                finish_reason="tool_calls",
+                raw_response=None,
+            )
+        }
+        stop_round = [
+            {
+                "resp": LLMResponse(
+                    content="done",
+                    tool_calls=[],
+                    finish_reason="stop",
+                    raw_response=None,
+                )
+            }
+        ]
+
+        received_calls = []
+
+        def fake_tool_calls(agent, calls, tools_dict, messages, **kwargs):
+            received_calls.extend(calls)
+            yield {"type": "tool_call", "data": {}}
+            return messages, None
+
+        llm = Mock()
+        llm.gen_stream = Mock(return_value=stop_round)
+        agent = _agent(llm)
+
+        with patch.object(handler, "handle_tool_calls", fake_tool_calls):
+            list(handler.handle_streaming(agent, [chunk_a, chunk_b], {}, []))
+
+        assert [(c.name, c.arguments) for c in received_calls] == [
+            ("tool_A", {"x": 1}),
+            ("tool_B", {"y": 2}),
+        ]
+
+    def test_indexed_string_deltas_still_merge(self):
+        """OpenAI-style indexed argument fragments keep concatenating."""
+        handler = ScriptedHandler()
+        frag1 = {
+            "resp": LLMResponse(
+                content="",
+                tool_calls=[ToolCall(id="c1", name="t", arguments='{"q":', index=0)],
+                finish_reason=None,
+                raw_response=None,
+            )
+        }
+        frag2 = {
+            "resp": LLMResponse(
+                content="",
+                tool_calls=[ToolCall(id="", name="", arguments='"x"}', index=0)],
+                finish_reason="tool_calls",
+                raw_response=None,
+            )
+        }
+        stop_round = [
+            {
+                "resp": LLMResponse(
+                    content="done",
+                    tool_calls=[],
+                    finish_reason="stop",
+                    raw_response=None,
+                )
+            }
+        ]
+
+        received_calls = []
+
+        def fake_tool_calls(agent, calls, tools_dict, messages, **kwargs):
+            received_calls.extend(calls)
+            yield {"type": "tool_call", "data": {}}
+            return messages, None
+
+        llm = Mock()
+        llm.gen_stream = Mock(return_value=stop_round)
+        agent = _agent(llm)
+
+        with patch.object(handler, "handle_tool_calls", fake_tool_calls):
+            list(handler.handle_streaming(agent, [frag1, frag2], {}, []))
+
+        assert len(received_calls) == 1
+        assert received_calls[0].arguments == '{"q":"x"}'
+
+
+class FailingTailStream:
+    """Yields chunks, then raises — models a reset in the trailing frames."""
+
+    def __init__(self, chunks, error):
+        self._chunks = chunks
+        self._error = error
+
+    def __iter__(self):
+        yield from self._chunks
+        raise self._error
+
+
+@pytest.mark.unit
+class TestTrailingFrameFailures:
+    def _stop_chunk(self, text="hello"):
+        return {
+            "resp": LLMResponse(
+                content=text,
+                tool_calls=[],
+                finish_reason="stop",
+                raw_response=None,
+            )
+        }
+
+    def test_failure_after_finish_is_swallowed(self):
+        handler = ScriptedHandler()
+        stream = FailingTailStream(
+            [self._stop_chunk()], RuntimeError("reset in trailing frame")
+        )
+        out = list(handler.handle_streaming(_agent(), stream, {}, []))
+        assert "hello" in out
+
+    def test_failure_before_finish_still_propagates(self):
+        handler = ScriptedHandler()
+        stream = FailingTailStream([], RuntimeError("early failure"))
+        with pytest.raises(RuntimeError, match="early failure"):
+            list(handler.handle_streaming(_agent(), stream, {}, []))
+
+    def test_final_round_failure_after_provider_finish_is_swallowed(self):
+        """The final answer round never surfaces a parseable 'stop' chunk
+        (content arrives as bare strings), so the swallow must key off the
+        LLM-level _stream_reached_finish flag."""
+        handler = ScriptedHandler()
+        agent = _agent()
+        agent.llm._stream_reached_finish = True
+        # Bare string content, no parseable finish chunk, then a trailing
+        # frame failure — the OpenAI chat final-round shape.
+        stream = FailingTailStream(
+            ["the full answer"], RuntimeError("reset in trailing frame")
+        )
+        out = list(handler.handle_streaming(agent, stream, {}, []))
+        assert "the full answer" in out
+
+    def test_fallback_served_stream_failure_after_finish_is_swallowed(self):
+        """When the fallback delivered the answer, its finish flag lives on
+        the fallback instance — the swallow must check it there too."""
+        handler = ScriptedHandler()
+        agent = _agent()
+        agent.llm._stream_reached_finish = False
+        agent.llm._fallback_llm = Mock()
+        agent.llm._fallback_llm._stream_reached_finish = True
+        stream = FailingTailStream(
+            ["fallback answer"], RuntimeError("reset in trailing frame")
+        )
+        out = list(handler.handle_streaming(agent, stream, {}, []))
+        assert "fallback answer" in out
+
+
+@pytest.mark.unit
+class TestInMemoryCompressionNegativeSavings:
+    def test_value_error_falls_back_to_minimal_pruning(self):
+        """The negative-savings ValueError from compress_conversation must
+        route to _prune_messages_minimal, not kill the tool loop."""
+        handler = ScriptedHandler()
+        agent = _agent()
+        agent.model_id = "gpt-4o"
+        agent.decoded_token = {"sub": "u1"}
+        messages = [
+            {"role": "user", "content": "q1"},
+            {"role": "assistant", "content": "a1"},
+        ]
+        pruned = [{"role": "system", "content": "pruned"}]
+
+        with patch.object(
+            handler,
+            "_build_conversation_from_messages",
+            return_value={"queries": [{"prompt": "q1", "response": "a1"}]},
+        ), patch(
+            "application.core.model_utils.get_provider_from_model_id",
+            return_value="openai",
+        ), patch(
+            "application.core.model_utils.get_api_key_for_provider",
+            return_value="k",
+        ), patch(
+            "application.llm.llm_creator.LLMCreator.create_llm",
+            return_value=Mock(),
+        ), patch(
+            "application.api.answer.services.compression.service."
+            "CompressionService.compress_conversation",
+            side_effect=ValueError(
+                "Compression did not reduce token count (10 → 20); "
+                "keeping original history"
+            ),
+        ), patch.object(
+            handler, "_prune_messages_minimal", return_value=pruned
+        ):
+            ok, rebuilt = handler._perform_in_memory_compression(agent, messages)
+
+        assert ok is True
+        assert rebuilt == pruned
+        assert agent.context_limit_reached is False

tests/llm/handlers/test_round_usage_persistence.py

@@ -0,0 +1,220 @@
+"""Per-round token-usage persistence in the streaming tool loop.
+
+Regression tests for the duplicate ``token_usage`` rows bug: each tool
+round's provider stream must be consumed to exhaustion *before* the next
+round starts, so the ``stream_token_usage`` ``finally`` persists one row
+per round with that round's own counts — never a late flush at request
+teardown that adopts the final round's shared ``_last_usage``.
+"""
+
+from types import SimpleNamespace
+from unittest.mock import Mock, patch
+
+from application import usage as usage_mod
+from application.llm.handlers.base import LLMHandler, LLMResponse, ToolCall
+
+
+class RecordingStream:
+    """Iterable stream that records whether it was fully consumed."""
+
+    def __init__(self, chunks):
+        self._chunks = list(chunks)
+        self.exhausted = False
+
+    def __iter__(self):
+        for chunk in self._chunks:
+            yield chunk
+        self.exhausted = True
+
+
+class RoundHandler(LLMHandler):
+    """Minimal concrete handler: chunk dicts drive the parse result."""
+
+    def parse_response(self, response):
+        kind = response.get("kind")
+        if kind == "tool_call":
+            return LLMResponse(
+                content="",
+                tool_calls=[ToolCall(id="1", name="t", arguments="{}", index=0)],
+                finish_reason="tool_calls",
+                raw_response=None,
+            )
+        if kind == "content":
+            return LLMResponse(
+                content="hello",
+                tool_calls=[],
+                finish_reason=None,
+                raw_response=None,
+            )
+        if kind == "stop":
+            return LLMResponse(
+                content="",
+                tool_calls=[],
+                finish_reason="stop",
+                raw_response=None,
+            )
+        # Terminal usage-only chunk: no content, no finish_reason.
+        return LLMResponse(
+            content="", tool_calls=[], finish_reason=None, raw_response=None
+        )
+
+    def create_tool_message(self, tool_call, result):
+        return {
+            "role": "tool",
+            "tool_call_id": tool_call.id,
+            "content": str(result),
+        }
+
+    def _iterate_stream(self, response):
+        yield from response
+
+
+def _agent(llm):
+    agent = Mock()
+    agent.llm = llm
+    agent.tools = []
+    agent.model_id = "test-model"
+    agent.context_limit_reached = False
+    agent._check_context_limit = Mock(return_value=False)
+    agent._enforce_context_window = Mock(side_effect=lambda msgs: msgs)
+    return agent
+
+
+class TestStreamDrainedBeforeNextRound:
+    def test_round_stream_exhausted_before_tool_handling(self):
+        """The round's generator must be fully consumed (usage chunk and
+        all) before handle_tool_calls runs — not abandoned mid-iteration."""
+        handler = RoundHandler()
+        round1 = RecordingStream([{"kind": "tool_call"}, {"kind": "usage"}])
+        round2 = RecordingStream([{"kind": "stop"}])
+
+        exhausted_at_tool_time = []
+
+        def fake_tool_calls(agent, calls, tools_dict, messages, **kwargs):
+            exhausted_at_tool_time.append(round1.exhausted)
+            yield {"type": "tool_call", "data": {}}
+            return messages, None
+
+        llm = Mock()
+        llm.gen_stream = Mock(return_value=round2)
+        agent = _agent(llm)
+
+        with patch.object(handler, "handle_tool_calls", fake_tool_calls):
+            list(handler.handle_streaming(agent, round1, {}, []))
+
+        assert exhausted_at_tool_time == [True]
+        assert round2.exhausted is True
+
+    def test_stop_round_drains_trailing_usage_chunk(self):
+        """A round that finishes with ``stop`` still consumes the trailing
+        usage-only chunk (Chat Completions include_usage arrives after the
+        finish_reason chunk)."""
+        handler = RoundHandler()
+        stream = RecordingStream(
+            [{"kind": "content"}, {"kind": "stop"}, {"kind": "usage"}]
+        )
+        agent = _agent(Mock())
+
+        out = list(handler.handle_streaming(agent, stream, {}, []))
+
+        assert stream.exhausted is True
+        assert "hello" in out
+
+
+class TestPerRoundUsageRows:
+    def test_one_row_per_round_with_own_counts(self):
+        """Two rounds → two persisted rows, each with that round's own
+        provider counts (not two copies of the final round's)."""
+        handler = RoundHandler()
+
+        class FakeLLM:
+            def __init__(self):
+                self.token_usage = {"prompt_tokens": 0, "generated_tokens": 0}
+                self.decoded_token = {"sub": "u1"}
+                self.user_api_key = None
+                self.agent_id = None
+                self._last_usage = None
+                self._last_usage_claimed = False
+                self.round = 0
+
+            def gen_stream(self, model, messages, tools=None, **kwargs):
+                wrapped = usage_mod.stream_token_usage(FakeLLM._raw_gen_stream)
+                return wrapped(self, model, messages, True, tools, **kwargs)
+
+            def _raw_gen_stream(self, model, messages, stream, tools, **kwargs):
+                self._last_usage = None
+                self._last_usage_claimed = False
+                self.round += 1
+                current = self.round
+                if current == 1:
+                    yield {"kind": "tool_call"}
+                else:
+                    yield {"kind": "content"}
+                    yield {"kind": "stop"}
+                # Provider reports usage in the terminal chunk, after the
+                # finish_reason — only a drained stream ever reaches this.
+                self._last_usage = {
+                    "prompt_tokens": 100 * current,
+                    "completion_tokens": 10 * current,
+                }
+                self._last_usage_claimed = False
+
+        llm = FakeLLM()
+        agent = _agent(llm)
+
+        def fake_tool_calls(a, calls, tools_dict, messages, **kwargs):
+            yield {"type": "tool_call", "data": {}}
+            return messages, None
+
+        rows = []
+        with patch.object(
+            usage_mod,
+            "_persist_call_usage",
+            side_effect=lambda llm_, cu: rows.append(dict(cu)),
+        ):
+            with patch.object(handler, "handle_tool_calls", fake_tool_calls):
+                first = llm.gen_stream(
+                    model="m", messages=[{"role": "user", "content": "hi"}]
+                )
+                list(handler.handle_streaming(agent, first, {}, []))
+
+        assert len(rows) == 2
+        assert rows[0] == {"prompt_tokens": 100, "generated_tokens": 10}
+        assert rows[1] == {"prompt_tokens": 200, "generated_tokens": 20}
+
+
+class TestPreferProviderUsageClaim:
+    def test_usage_claimed_only_once(self):
+        llm = SimpleNamespace(
+            _last_usage={"prompt_tokens": 7, "completion_tokens": 9},
+            _last_usage_claimed=False,
+        )
+        estimate = {"prompt_tokens": 42, "generated_tokens": 1}
+
+        first = usage_mod._prefer_provider_usage(llm, dict(estimate))
+        assert first == {"prompt_tokens": 7, "generated_tokens": 9}
+        assert llm._last_usage_claimed is True
+
+        second = usage_mod._prefer_provider_usage(llm, dict(estimate))
+        assert second == dict(estimate)
+
+    def test_fresh_usage_resets_claim(self):
+        llm = SimpleNamespace(
+            _last_usage={"prompt_tokens": 7, "completion_tokens": 9},
+            _last_usage_claimed=False,
+        )
+        usage_mod._prefer_provider_usage(llm, {"prompt_tokens": 1, "generated_tokens": 1})
+
+        # Provider records a new call's usage → unclaimed again.
+        llm._last_usage = {"prompt_tokens": 70, "completion_tokens": 90}
+        llm._last_usage_claimed = False
+
+        result = usage_mod._prefer_provider_usage(
+            llm, {"prompt_tokens": 2, "generated_tokens": 2}
+        )
+        assert result == {"prompt_tokens": 70, "generated_tokens": 90}
+
+    def test_missing_usage_keeps_estimate(self):
+        llm = SimpleNamespace(_last_usage=None, _last_usage_claimed=False)
+        estimate = {"prompt_tokens": 5, "generated_tokens": 3}
+        assert usage_mod._prefer_provider_usage(llm, dict(estimate)) == estimate

tests/llm/test_fallback.py

@@ -811,3 +811,124 @@ class TestRespondingProviderTracking:
         primary = _Google(fail_at=0, backup_models=["backup-model"])
         primary.gen(**CALL_ARGS)
         assert primary._responding_provider == "openai"
+
+
+# Tests — fallback payload size gate
+
+
+@pytest.mark.integration
+class TestFallbackPayloadSizeGate:
+    """A payload that cannot fit the fallback's context window must skip the
+    fallback attempt (it would be a guaranteed second rejection) and
+    propagate the primary's error instead."""
+
+    def _primary_with_backup(self, patch_model_utils, backup):
+        patch_model_utils(
+            get_provider=lambda mid, **_kw: "openai",
+            get_api_key=lambda p: "k",
+            create_llm=lambda type, **kw: backup,
+        )
+        return FakeLLM(fail_at=0, backup_models=["backup-model"])
+
+    BIG_ARGS = dict(
+        model="test-model",
+        messages=[{"role": "user", "content": "word " * 300}],
+    )
+
+    def test_gen_skips_fallback_when_payload_cannot_fit(
+        self, monkeypatch, patch_model_utils
+    ):
+        backup = FakeLLM(responses=["backup ok"])
+        primary = self._primary_with_backup(patch_model_utils, backup)
+        monkeypatch.setattr(
+            "application.core.model_utils.get_token_limit",
+            lambda mid, user_id=None: 10,
+        )
+        with pytest.raises(RuntimeError, match="primary model unavailable"):
+            primary.gen(**self.BIG_ARGS)
+        assert backup.gen_called is False
+
+    def test_stream_skips_fallback_when_payload_cannot_fit(
+        self, monkeypatch, patch_model_utils
+    ):
+        backup = FakeLLM(stream_chunks=["backup chunk"])
+        primary = self._primary_with_backup(patch_model_utils, backup)
+        monkeypatch.setattr(
+            "application.core.model_utils.get_token_limit",
+            lambda mid, user_id=None: 10,
+        )
+        with pytest.raises(RuntimeError, match="mid-stream failure"):
+            list(primary.gen_stream(**self.BIG_ARGS))
+        assert backup.gen_stream_called is False
+
+    def test_fallback_proceeds_when_payload_fits(
+        self, monkeypatch, patch_model_utils
+    ):
+        backup = FakeLLM(responses=["backup ok"])
+        primary = self._primary_with_backup(patch_model_utils, backup)
+        monkeypatch.setattr(
+            "application.core.model_utils.get_token_limit",
+            lambda mid, user_id=None: 100000,
+        )
+        assert primary.gen(**self.BIG_ARGS) == "backup ok"
+        assert backup.gen_called is True
+
+    def test_estimation_failure_never_blocks_fallback(
+        self, monkeypatch, patch_model_utils
+    ):
+        backup = FakeLLM(responses=["backup ok"])
+        primary = self._primary_with_backup(patch_model_utils, backup)
+
+        def boom(*a, **kw):
+            raise ValueError("estimator broken")
+
+        monkeypatch.setattr("application.usage._count_prompt_tokens", boom)
+        assert primary.gen(**self.BIG_ARGS) == "backup ok"
+        assert backup.gen_called is True
+
+
+# Tests — no fallback restream after the primary delivered its finish signal
+
+
+@pytest.mark.integration
+class TestNoRestreamAfterFinish:
+    """A trailing-frame failure (between the finish chunk and stream end)
+    must NOT restream the already-delivered answer from the fallback."""
+
+    class FinishThenFailLLM(FakeLLM):
+        def _raw_gen_stream(self, baseself, model, messages, stream, tools=None, **kwargs):
+            self.gen_stream_called = True
+            yield "the full answer"
+            # Provider marked the stream finished (finish_reason arrived)...
+            self._stream_reached_finish = True
+            # ...then the trailing usage/[DONE] frame dies.
+            raise RuntimeError("connection reset in trailing frame")
+
+    def test_trailing_frame_failure_skips_fallback(self, patch_model_utils):
+        backup = FakeLLM(stream_chunks=["fallback chunk"])
+        patch_model_utils(
+            get_provider=lambda mid, **_kw: "openai",
+            get_api_key=lambda p: "k",
+            create_llm=lambda type, **kw: backup,
+        )
+        primary = self.FinishThenFailLLM(backup_models=["backup-model"])
+
+        received = []
+        with pytest.raises(RuntimeError, match="trailing frame"):
+            for chunk in primary.gen_stream(**CALL_ARGS):
+                received.append(chunk)
+
+        assert received == ["the full answer"]
+        assert backup.gen_stream_called is False
+
+    def test_pre_finish_failure_still_falls_back(self, patch_model_utils):
+        backup = FakeLLM(stream_chunks=["fallback chunk"])
+        patch_model_utils(
+            get_provider=lambda mid, **_kw: "openai",
+            get_api_key=lambda p: "k",
+            create_llm=lambda type, **kw: backup,
+        )
+        primary = FakeLLM(fail_at=0, backup_models=["backup-model"])
+        out = list(primary.gen_stream(**CALL_ARGS))
+        assert out == ["fallback chunk"]
+        assert backup.gen_stream_called is True

tests/llm/test_openai_responses.py

@@ -126,15 +126,20 @@ def test_to_responses_input_reinjects_reasoning(monkeypatch):
         "encrypted_content": "enc", "summary": [],
     }
     llm._reasoning_for_calls = {"call_1": [reasoning_item]}
-    messages = [{
-        "role": "assistant",
-        "content": None,
-        "tool_calls": [{
-            "id": "call_1",
-            "type": "function",
-            "function": {"name": "t", "arguments": "{}"},
-        }],
-    }]
+    # The call must be paired with its output — unpaired calls are dropped
+    # (the API rejects them with "No tool output found for function call").
+    messages = [
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [{
+                "id": "call_1",
+                "type": "function",
+                "function": {"name": "t", "arguments": "{}"},
+            }],
+        },
+        {"role": "tool", "tool_call_id": "call_1", "content": "ok"},
+    ]
     items = llm._to_responses_input(messages)
     # Reasoning item is emitted immediately before its function call.
     assert items[0] == reasoning_item
@@ -254,15 +259,19 @@ def test_export_import_replays_encrypted_reasoning_on_fresh_instance(monkeypatch
 
     second = _make_llm(monkeypatch, _responses_caps())
     assert second.import_responses_state(first.export_responses_state()) is True
-    items = second._to_responses_input([{
-        "role": "assistant",
-        "content": None,
-        "tool_calls": [{
-            "id": "call_1",
-            "type": "function",
-            "function": {"name": "run", "arguments": "{}"},
-        }],
-    }])
+    # Paired with its output: unpaired calls are dropped by the builder.
+    items = second._to_responses_input([
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [{
+                "id": "call_1",
+                "type": "function",
+                "function": {"name": "run", "arguments": "{}"},
+            }],
+        },
+        {"role": "tool", "tool_call_id": "call_1", "content": "ok"},
+    ])
     assert items[0] == reasoning
     assert items[1]["type"] == "function_call"
 
@@ -982,3 +991,117 @@ def test_builtin_default_models_stay_chat_completions():
     catalogs = load_model_yamls([BUILTIN_MODELS_DIR])
     models = {m.id: m for c in catalogs for m in c.models}
     assert models["gpt-5.4-mini"].capabilities.api_flavor == "chat_completions"
+
+
+@pytest.mark.unit
+def test_to_responses_input_drops_unpaired_function_call(monkeypatch):
+    """A function_call with no matching output must not reach the provider
+    (it 400s with "No tool output found for function call ...")."""
+    llm = _make_llm(monkeypatch, _responses_caps())
+    messages = [
+        {"role": "user", "content": "hi"},
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [
+                {
+                    "id": "call_orphan",
+                    "type": "function",
+                    "function": {"name": "search", "arguments": "{}"},
+                },
+                {
+                    "id": "call_ok",
+                    "type": "function",
+                    "function": {"name": "read", "arguments": "{}"},
+                },
+            ],
+        },
+        {"role": "tool", "tool_call_id": "call_ok", "content": "result"},
+    ]
+    items = llm._to_responses_input(messages)
+    calls = [i for i in items if i.get("type") == "function_call"]
+    outputs = [i for i in items if i.get("type") == "function_call_output"]
+    assert [c["call_id"] for c in calls] == ["call_ok"]
+    assert [o["call_id"] for o in outputs] == ["call_ok"]
+
+
+@pytest.mark.unit
+def test_to_responses_input_drops_orphaned_output(monkeypatch):
+    """A function_call_output whose call was never emitted is dropped."""
+    llm = _make_llm(monkeypatch, _responses_caps())
+    messages = [
+        {"role": "user", "content": "hi"},
+        {"role": "tool", "tool_call_id": "call_ghost", "content": "result"},
+        {"role": "assistant", "content": "final"},
+    ]
+    items = llm._to_responses_input(messages)
+    assert not [i for i in items if i.get("type") == "function_call_output"]
+    assert items[-1]["role"] == "assistant"
+
+
+@pytest.mark.unit
+def test_to_responses_input_skips_reasoning_of_dropped_calls(monkeypatch):
+    """When every call on an assistant message is unpaired, its reasoning
+    items are suppressed too — a trailing reasoning item with no following
+    item is itself rejected by the Responses API."""
+    llm = _make_llm(monkeypatch, _responses_caps())
+    llm._reasoning_for_calls = {
+        "call_orphan": [{"type": "reasoning", "id": "rs_1", "summary": []}]
+    }
+    messages = [
+        {"role": "user", "content": "hi"},
+        {
+            "role": "assistant",
+            "content": None,
+            "responses_reasoning_items": [
+                {"type": "reasoning", "id": "rs_msg", "summary": []}
+            ],
+            "tool_calls": [{
+                "id": "call_orphan",
+                "type": "function",
+                "function": {"name": "search", "arguments": "{}"},
+            }],
+        },
+    ]
+    items = llm._to_responses_input(messages)
+    assert not [i for i in items if i.get("type") == "reasoning"]
+    assert not [i for i in items if i.get("type") == "function_call"]
+
+
+@pytest.mark.unit
+def test_to_responses_input_chained_keeps_bare_tool_output(monkeypatch):
+    """Store-mode chaining (previous_response_id) deliberately sends a bare
+    function_call_output whose call lives server-side — the pairing guard
+    must not drop it (that would 400 every chained tool round)."""
+    llm = _make_llm(monkeypatch, _responses_caps())
+    messages = [
+        {"role": "system", "content": "sys"},
+        {"role": "tool", "tool_call_id": "call_server_side", "content": "result"},
+    ]
+    items = llm._to_responses_input(messages, chained=True)
+    outputs = [i for i in items if i.get("type") == "function_call_output"]
+    assert [o["call_id"] for o in outputs] == ["call_server_side"]
+
+
+@pytest.mark.unit
+def test_to_responses_input_drops_out_of_order_pair_entirely(monkeypatch):
+    """An output that precedes its call is malformed history: BOTH sides
+    are dropped (keeping the call alone would produce the unpaired-call
+    400 the guard exists to prevent)."""
+    llm = _make_llm(monkeypatch, _responses_caps())
+    messages = [
+        {"role": "user", "content": "hi"},
+        {"role": "tool", "tool_call_id": "call_x", "content": "early result"},
+        {
+            "role": "assistant",
+            "content": None,
+            "tool_calls": [{
+                "id": "call_x",
+                "type": "function",
+                "function": {"name": "t", "arguments": "{}"},
+            }],
+        },
+    ]
+    items = llm._to_responses_input(messages)
+    assert not [i for i in items if i.get("type") == "function_call"]
+    assert not [i for i in items if i.get("type") == "function_call_output"]