Headless runs searched only the agent's primary source, so an agent whose
knowledge sat in its extra sources answered a schedule or webhook without
it. They now take the primary and every extra source through the same
owner-or-sponsor check a chat uses, shared as one helper, and retrieve
through the per-source dispatcher so each source keeps its own settings.
DeepSeek and every other OpenAI-compatible API run through OpenAILLM, so
spans and metrics reported them as openai. The provider now comes from the
endpoint: known API hosts map to their gen_ai.provider.name (deepseek,
azure.ai.openai, mistral_ai, ...), and an OpenAI client pointed anywhere
else is openai_compatible. LLM spans also record server.address.
The OTel replay and the trace INSERT ran in the stream's finally, so a slow
database held the SSE connection open after the last event. The trace is
still frozen when the stream ends, but written on a small writer pool.
Span.fail stored the exception message in span.error, which is stored and
exported whatever the content settings; a provider's content-filter error
can quote the prompt. span.error now holds the exception type, and the
message is a capture-gated preview, like tool errors already were. A
yielded stream error on the agent span is treated the same way.
With TRACES_ENABLED off, LLM calls record no GenAI metrics, and a streamed
answer is joined into a preview only when a span keeps it. Agent runs and
continuations build their invoke_agent span in one place, which now reads
the model from model_id. The trace panel shows a stream's model time next
to how long it was open. Adds missing type hints.
A failed trace-summary lookup now leaves the Logs page intact without
chips. Tool-call counts include only calls that ran, not their paused,
denied or skipped records. A local guardrail that fires unchanged on every
streamed segment is recorded once, so it cannot use up the span cap.
Tool results, denial comments and tool exception text now reach a span only
as a capture-gated preview; span.error is a fixed message, since it is
stored and exported regardless of content settings. A turn whose stream
yields an error event is recorded as failed. Search traces are listed by a
query copied into the small summary column, so the Logs timeline never
reads the spans JSONB. Adds GraphRAG span tests.
run_agent_headless records each unattended run under its endpoint; the
scheduler passes its run id and the webhook worker its task id so Logs rows
can find their trace, while the LLM's own request id stays untouched for
quota counts. /api/search and MCP search_docs record their retrieval, and a
graph build records every extraction call under one step.
StreamProcessor starts the trace and mints the request id before the
agent is built, so pre-fetch retrieval and compression are inside it and
side-channel LLM calls share the id. complete_stream activates it in the
SSE pump thread, binds the message and conversation, and writes it once
however the stream ends: paused, failed, abandoned or superseded (dropped).
user_logs rows now carry request_id and message_id.
The dispatcher and each retriever's search open retrieval spans recording
sources, chunk counts, top scores and a chunk preview. The shared query
embedding, every per-source vector search and the prescreen rerank get
their own spans; pool workers carry the trace so they nest correctly.
@log_activity opens the invoke_agent span for every agent run and names
the trace's activity; gen_continuation opens its own. ToolExecutor.execute
records each executed call with redacted argument and result previews, and
paused, denied, skipped and client-executed calls are recorded where they
are decided.
The token-usage wrappers open one span per decorated invocation, so a
primary attempt, its retry and a fallback appear as siblings with the
provider that ran. Stream spans start on the first pull, carry tokens,
cost, time to first token and cache hits, and feed the GenAI metrics.
Finished traces are replayed with their recorded timestamps and explicit
parents under the request's server span, using gen_ai.* attribute names.
Content is exported only when OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT
opts in. gen_ai.client token-usage and operation-duration histograms are
recorded per model call.
docsgpt.tracing records agent, LLM, tool, retrieval and embedding spans in
memory and writes the finished trace once. Container spans nest by a
per-thread stack so suspended generators cannot corrupt the tree; open
spans are cancelled at flush and previews are bounded and redacted.
arc53-machine