fix(vision): file-ref mode + media pipeline fixes for image visibility (#511)

* fix(vision): file-ref mode + media pipeline fixes for image visibility

Replace inline base64 image loading with file path references when
read_image provider is configured. LLM calls read_image(path=...)
instead of receiving 25K-250K tokens of base64 per image.

Changes:
- Agent loop: skip base64 context storage in file-ref mode, only
  load historical images for inline fallback
- New enrichImagePaths() enriches ALL user messages with file paths
  (not just current turn) so historical images are accessible
- System prompt: imperative tool summaries ("REQUIRED when you see
  <media:image> tags") + dedicated "Media Files" section
- Slack: store mediaPaths in pending history + CollectMedia on mention
- Feishu: add CollectMedia for group context history media
- RC-1 fix: save enriched content (with media IDs/paths) to DB
  instead of raw req.Message

Closes #509

* fix(vision): resolve Claude review — Feishu early media + skip loadImages in file-ref mode

- Feishu: download media BEFORE mention gate (step 4) and store file
  paths in pending history via Media field. Reuse early-resolved
  MediaInfo at step 10 to avoid double-download. CollectMedia now
  returns actual paths instead of empty.
- Agent loop: guard loadImages() behind !deferToReadImageTool so
  file-ref mode avoids unnecessary disk I/O + base64 encoding.
This commit is contained in:
Duy /zuey/ authored and GitHub committed 2026-03-27 14:20:05 +07:00
1 parent 4dafd70c83
commit 231bc9684e
6 files changed
+159 -33

No files matched your search

+5
View File
@@ -31,6 +31,11 @@ Things like:
- Default speaker: Kitchen HomePod
```
## Media Files
When users send images, videos, audio, or documents, use the `read_*` tools to analyze them.
The `path` attribute in media tags points to the file — pass it directly to the tool.
## Why Separate?
Skills are shared. Your setup is yours. Keeping them apart means you can update skills without losing your notes, and share skills without leaking your infrastructure.