2 Commits
Author SHA1 Message Date
thotam a5a853f461 feat(tools): image reference processing and native provider support (#1251)
* feat(tools): implement image reference processing and native provider support

- Support OpenAI image edits via both Multipart form-data and JSON payloads
- Automatically append reference image descriptions to prompt under [Reference Image Roles]
- Support downloading image URLs for Gemini native image generation
- Deduplicate reference images to optimize API request size
- Add unit tests for Codex, DashScope, MiniMax, BytePlus and local/remote path resolution

* fix(tools): SSRF-guard reference-image URL downloads in create_image

downloadImageBytes fetched caller-supplied ref_images[].url with a plain
http.Client and unbounded io.ReadAll — no SSRF validation, redirect
policy, or size cap, letting the gateway dial loopback/private/metadata
hosts or read arbitrarily large responses.

- Validate the URL via security.Validate and pin the resolved IP, then
  download through security.NewSafeClient (pinned dial, no redirects).
- Cap the response with a bounded read (refImageMaxBytes, 20 MB).
- Reject non-HTTP(S) reference URLs up front (file://, data:, gopher://)
  so provider-forwarded URLs stay HTTP(S)-only; document the trust
  boundary between gateway-side fetch and provider-forwarded URLs.
- Add regression tests: blocked loopback/private/metadata, unfollowed
  redirect, oversized response, and non-http(s) scheme rejection.
2026-06-21 18:27:51 +07:00
viettranx aa5158d4a2 feat(providers): add BytePlus ModelArk provider with Seedream/Seedance media gen
Add BytePlus ModelArk as a new OpenAI-compatible provider for Seed 2.0
models (chat, vision). Two provider types: standard API and Coding Plan
(separate base URLs, same auth).

Integrate Seedream image generation (sync API) and Seedance video
generation (async polling) into the builtin media tool chain, following
the established DashScope/Gemini patterns.

- Add WithAuthPrefix option to OpenAIProvider for future non-standard auth
- Add ProviderBytePlus/ProviderBytePlusCoding store constants and config
- Register provider from config.json and llm_providers DB table
- Add BytePlus to media chain routing, priority lists, and dispatch
- Create create_image_byteplus.go (Seedream, sync response)
- Create create_video_byteplus.go (Seedance, async poll with 5min timeout)
- Add BytePlus to web and desktop UI provider type dropdowns
- Update provider docs with BytePlus entries

Closes #686
2026-04-04 23:07:42 +07:00