When code_executor or read_document is given an input that doesn't resolve to an
existing artifact, it now falls back to the caller's own chat attachments,
bridges the referenced one into a conversation-scoped artifact (idempotent,
server-computed size/sha256, quota-respected), and stages it. This is lazy —
plain chat uploads are never bridged, only a file a tool actually references —
and conversation-scoped only (workflow nodes bridge attachments up front).
The match is confined to the request's own attachments and re-verified against
the user-scoped attachments repo, so a model-supplied name/id can only ever reach
the caller's own files. Only the current request's attachments are reachable (the
attachments table has no conversation column); prior-turn files are a follow-up.
OpenAI-compatible clients send multimodal user turns as a `content` array of
typed parts. translate_request previously assigned the array straight to the
question, breaking the string-only retrieval / token-budgeting / history paths
(HTTP 500). Now:
- content_to_text() extracts text from content arrays for the question,
history and system prompt, so the string paths work unchanged.
- The full content array (text + image_url parts) is preserved as
`multimodal_content`, threaded to the agent and emitted as the final user
message so images reach the model. Token budgeting uses the text only.
The content array (incl. image_url) now reaches the LLM call intact; images
render for vision-capable models. A text-only upstream model will reject the
image_url variant, as expected.
- Stateless tool continuation. OpenAI-compatible clients (opencode, etc.)
resend the full messages array — system, user, assistant(tool_calls),
tool(results) — but no conversation_id, so the prior
"conversation_id required for tool continuation" 400 broke every tool call.
When no conversation_id is present, rebuild the agent + pending tool calls +
tool results directly from the resent messages
(StreamProcessor.build_continuation_from_messages) instead of loading
server-side pending_tool_state, and call gen_continuation.
- Forward OpenAI sampling params (temperature, max_tokens,
max_completion_tokens, top_p, frequency_penalty, presence_penalty, stop,
seed) from the request to the LLM gen call; the agent otherwise uses its
configured defaults.
Make the OpenAI-compatible Chat Completions endpoint honor per-request
Structured Outputs and keep it OpenAI-compatible.
- translate_request now forwards the request's `response_format` (json_schema)
or a `response_schema` convenience field to the agent as its json_schema,
overriding the agent-configured schema for that request.
- Honor `response_format.json_schema.strict` (default true); strict:false
passes the schema through without forcing additionalProperties:false /
all-required (OpenAI's lenient mode).
- Support `response_format {"type":"json_object"}` via the provider's native
JSON mode.
- Keep `content` clean: some models echo their reasoning into content as
stringified `{'type': 'thought', ...}` reprs when response_format is set.
Strip those from content and reroute them to `reasoning_content` (OpenAI
never puts reasoning in content), for both streaming and non-streaming.
Flow: translate_request -> StreamProcessor._configure_agent ->
Agent.json_schema / json_schema_strict / json_object -> _llm_gen ->
prepare_structured_output_format(schema, strict).
Route opted-in models to /v1/responses while keeping Chat Completions as
the default. The flavor is selected per-model in the catalog YAML and
threaded through ModelCapabilities (with a new reasoning_effort field).
- Translate chat-shaped messages/tools/structured-output into the
Responses input / text.format / flattened-tool shapes at the provider
boundary; stored history and intra-turn tool round-trips need no
migration.
- Normalize the typed SSE event stream back into the existing handler
contract (content / thought / tool-call chunks) so the streaming
accumulator and OpenAI handler stay unchanged.
- Carry encrypted reasoning items across the in-turn tool loop
(provider-local, keyed by call_id) for stateless reasoning continuity.
- OPENAI_RESPONSES_STORE opt-in enables server-side state and
previous_response_id chaining across turns, persisted in message
metadata.
- Set gpt-5.5 to api_flavor: responses, reasoning_effort: medium.
- Remove AzureOpenAILLM and the azure_openai provider/catalog.
These attributes were only set by StreamProcessor after agent creation,
causing an AttributeError in _perform_mid_execution_compression when
the context limit was hit through other code paths (e.g. worker).
Declaring them as None in init lets the handler fall through to
in-memory compression gracefully.
* feat: enhance modal functionality and configuration handling
- Updated WrapperModal to improve click outside detection for closing the modal.
- Refactored ToolConfig to utilize ConfigFieldSpec for better configuration management.
- Added validation and dynamic handling of configuration fields in ToolConfig.
- Introduced reconnect functionality for MCP tools in the Tools component.
- Enhanced user experience with improved error handling and loading states.
- Updated types for better type safety and clarity in configuration requirements.
* refactor: reorganize imports and improve conditional formatting
* fix: revert API_URL to use backend service name in docker-compose
* feat: add MCP auth status endpoint and integrate into user service and tools
* feat: implement logging for Brave, Postgres, and Telegram tools; add transport sanitization and credential extraction for MCP
---------
Co-authored-by: Alex <a@tushynski.me>
* (feat:memory) use fs/storage for files
* (feat:todo) artifact_id via sse
* (feat:notes) artifact id return
* (feat:artifact) add get endpoint, store todos with conv id
* (feat: artifacts) fe integration
* feat(artifacts): ui enhancements, notes as mkdwn
* chore(artifacts) updated artifact tests
* (feat:todo_tool) return all todo items
* (feat:tools) use specific tool names in bubble
* feat: add conversationId prop to artifact components in Conversation
* Revert "(feat:memory) use fs/storage for files"
This reverts commit d1ce3bea31a76374ea70ce65f1691e34e66bc683.
* (fix:fe) build fail
* feat: implement WorkflowAgent and GraphExecutor for workflow management and execution
* refactor: workflow schemas and introduce WorkflowEngine
- Updated schemas in `schemas.py` to include new agent types and configurations.
- Created `WorkflowEngine` class in `workflow_engine.py` to manage workflow execution.
- Enhanced `StreamProcessor` to handle workflow-related data.
- Added new routes and utilities for managing workflows in the user API.
- Implemented validation and serialization functions for workflows.
- Established MongoDB collections and indexes for workflows and related entities.
* refactor: improve WorkflowAgent documentation and update type hints in WorkflowEngine
* feat: workflow builder and managing in frontend
- Added new endpoints for workflows in `endpoints.ts`.
- Implemented `getWorkflow`, `createWorkflow`, and `updateWorkflow` methods in `userService.ts`.
- Introduced new UI components for alerts, buttons, commands, dialogs, multi-select, popovers, and selects.
- Enhanced styling in `index.css` with new theme variables and animations.
- Refactored modal components for better layout and styling.
- Configured TypeScript paths and Vite aliases for cleaner imports.
* feat: add workflow preview component and related state management
- Implemented WorkflowPreview component for displaying workflow execution.
- Created WorkflowPreviewSlice for managing workflow preview state, including queries and execution steps.
- Added WorkflowMiniMap for visual representation of workflow nodes and their statuses.
- Integrated conversation handling with the ability to fetch answers and manage query states.
- Introduced reusable Sheet component for UI overlays.
- Updated Redux store to include workflowPreview reducer.
* feat: enhance workflow execution details and state management in WorkflowEngine and WorkflowPreview
* feat: enhance workflow components with improved UI and functionality
- Updated WorkflowPreview to allow text truncation for better display of long names.
- Enhanced BaseNode with connectable handles and improved styling for better visibility.
- Added MobileBlocker component to inform users about desktop requirements for the Workflow Builder.
- Introduced PromptTextArea component for improved variable insertion and search functionality, including upstream variable extraction and context addition.
* feat(workflow): add owner validation and graph version support
* fix: ruff lint
---------
Co-authored-by: Alex <a@tushynski.me>
* feat: enhance API tool with body serialization and content type handling
* feat: enhance ToolConfig with import functionality and user action management
- Added ImportSpecModal to allow importing actions into the tool configuration.
- Implemented search functionality for user actions with expandable action details.
- Introduced method colors for better visual distinction of HTTP methods.
- Updated APIActionType and ParameterGroupType to include optional 'required' field.
- Refactored action rendering to improve usability and maintainability.
* feat: add base URL input to ImportSpecModal for action URL customization
* feat: update TestBaseAgentTools to include 'required' field for parameters
* feat: standardize API call timeout to DEFAULT_TIMEOUT constant
* feat: add import specification functionality and related translations for multiple languages
---------
Co-authored-by: Alex <a@tushynski.me>
* feat: Implement model registry and capabilities for multi-provider support
- Added ModelRegistry to manage available models and their capabilities.
- Introduced ModelProvider enum for different LLM providers.
- Created ModelCapabilities dataclass to define model features.
- Implemented methods to load models based on API keys and settings.
- Added utility functions for model management in model_utils.py.
- Updated settings.py to include provider-specific API keys.
- Refactored LLM classes (Anthropic, OpenAI, Google, etc.) to utilize new model registry.
- Enhanced utility functions to handle token limits and model validation.
- Improved code structure and logging for better maintainability.
* feat: Add model selection feature with API integration and UI component
* feat: Add model selection and default model functionality in agent management
* test: Update assertions and formatting in stream processing tests
* refactor(llm): Standardize model identifier to model_id
* fix tests
---------
Co-authored-by: Alex <a@tushynski.me>
* Update routes.py, added token and request limits to create/update agent operations
* added usage limit check to api endpoints
cannot create agents with usage limit right now that will be implemented
* implemented api limiting as either token limiting or request limiting modes
* minor typo & bug fix
- Implemented encryption utility for securely storing sensitive credentials.
- Added MCPServerModal component for managing MCP server configurations.
- Updated AddToolModal to include MCP server management.
- Enhanced localization with new strings for MCP server features.
- Introduced SVG icons for MCP tools in the frontend.
- Created a new settings section for MCP server configurations in the application.
- Added functionality to select agents in the Navigation component, allowing users to reset conversations and set the selected agent.
- Updated the MessageInput component to conditionally show source and tool buttons based on the selected agent.
- Modified the Conversation component to handle agent-specific queries and manage file uploads.
- Improved conversation fetching logic to include agent IDs and handle attachments.
- Introduced new types for conversation summaries and results to streamline API responses.
- Refactored Redux slices to manage selected agent state and improve overall state management.
- Enhanced error handling and loading states across components for better user experience.
Alex