Extract input preparation and image conversion handling into look-at-input-preparer.ts.
Extract prompt construction and multimodal session execution into look-at-prompt.ts and look-at-session-runner.ts while keeping createLookAt stable.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
When Read tool is disabled (attachment mode), the agent prompt now
instructs to analyze the attached content directly instead of
trying to use Read. 3 new tests, 55 total pass.
Closes#3033
Sisyphus-authored fixes across 15 files:
- plugin-identity: align CONFIG_BASENAME with actual config file name
- add-plugin-to-opencode-config: handle legacy→canonical name migration
- plugin-detection tests: update expectations for new identity constants
- doctor/system: fix legacy name warning test assertions
- install tests: align with new plugin name
- chat-params tests: fix mock isolation
- model-capabilities tests: fix snapshot expectations
- image-converter: fix platform-dependent test assertions (Linux CI)
- example configs: expanded with more detailed comments
Full suite: 4484 pass, 0 fail, typecheck clean.
- Replace overly broad .includes('anthropic') with exact provider ID
matching against known Anthropic providers (anthropic, google-vertex-
anthropic, aws-bedrock-anthropic) in context-limit-resolver
- Add afterEach cleanup for vision-capable-models cache in look-at
tool tests to prevent cross-test state leakage
Extends conversion logic to handle Base64-encoded images (e.g., from clipboard).
Previously, unsupported formats like HEIC/RAW/PSD in Base64 form bypassed
the conversion check and caused failures at multimodal-looker agent.
Changes:
- Add convertBase64ImageToJpeg() function in image-converter.ts
- Save Base64 data to temp file, convert, read back as Base64
- Update tools.ts to check and convert Base64 images when needed
- Ensure proper cleanup of all temporary files
Testing:
- All tests pass (29/29)
- Verified with 1.7MB HEIC file converted from Base64
- Type checking passes
Adds automatic conversion of unsupported image formats (HEIC, HEIF, RAW, PSD)
to JPEG before sending to multimodal-looker agent.
Changes:
- Add image-converter.ts module with format detection and conversion
- Modify look_at tool to auto-convert unsupported formats
- Extend mime-type-inference.ts to support 15+ additional formats
- Use sips (macOS) and ImageMagick (Linux/Windows) for conversion
- Add proper cleanup of temporary files
Fixes#722
Testing:
- All existing tests pass (29/29)
- TypeScript type checking passes
- Verified HEIC to JPEG conversion on macOS
df0b9f76 regressed look_at from synchronous prompt (session.prompt) to
async prompt (session.promptAsync) + pollSessionUntilIdle polling. This
introduced a race condition where the poller fires before the server
registers the session as busy, causing it to return immediately with no
messages available.
Fix: restore promptSyncWithModelSuggestionRetry (blocking HTTP call) and
remove polling entirely. Catch prompt errors gracefully and still attempt
to fetch messages, since session.prompt may throw even on success.
- fix(delegate-task): return error on poll timeout instead of silent null
- fix(delegate-task): ensure toast and session cleanup on all error paths with try/finally
- fix(delegate-task): apply agent tool restrictions in sync-prompt-sender
- fix(plugin): add symmetric idle dedup to prevent double hook triggers
- fix(cli): replace regex-based JSONC editing with jsonc-parser in auth-plugins
- fix(cli): abort event stream after completion and restore no-timeout default
All changes verified with tests and typecheck.
session.prompt() may throw {} or JSON parse errors even when the server
successfully processes the request. Instead of crashing the tool, catch
all errors and proceed to fetch messages — if the response is available,
return it; otherwise return a clean error string.
PR #1620 migrated all prompt calls from session.prompt (blocking) to
session.promptAsync (fire-and-forget HTTP 204). This broke look_at which
needs the multimodal-looker response to be available immediately after
the prompt call returns.
Fix: add promptSyncWithModelSuggestionRetry() that uses session.prompt
(blocking) with model suggestion retry support. look_at now uses this
sync variant while all other callers keep using promptAsync.
- Add promptSyncWithModelSuggestionRetry to model-suggestion-retry.ts
- Switch look_at from promptWithModelSuggestionRetry to sync variant
- Add comprehensive tests for the new sync function
- No changes to other callers (delegate-task, background-agent)
Closes#704
Add support for base64-encoded image data in the look_at tool,
enabling analysis of clipboard/pasted images without requiring
a file path.
Changes:
- Add optional image_data parameter to LookAtArgs type
- Update validateArgs to accept either file_path or image_data
- Add inferMimeTypeFromBase64 function to detect image format
- Add try/catch around atob() to handle invalid base64 gracefully
- Update execute to handle both file path and data URL inputs
- Add comprehensive tests for image_data functionality
- Add skipNotification option to cancelTask method
- Apply skipNotification to background_cancel tool
- Prevents unwanted notifications when user cancels via tool
- fetchAvailableModels now falls back to client.model.list() when cache is empty
- provider-models cache empty → models.json → client API (3-tier fallback)
- look-at tool explicitly passes registered agent's model to session.prompt
- Ensures multimodal-looker uses correctly resolved model (e.g., gemini-3-flash-preview)
- Add comprehensive tests for fuzzy matching and fallback scenarios
When multimodal-looker agent returns empty/malformed response, the SDK
throws 'JSON Parse error: Unexpected EOF'. This commit adds try-catch
around session.prompt() to provide user-friendly error message with
troubleshooting guidance.
- Add error handling for JSON parse errors with detailed guidance
- Add error handling for generic prompt failures
- Add test cases for both error scenarios
- Add permission field to look_at and call_omo_agent session.create()
- Match pattern used in delegate_task and background-agent
- Add better error messages for Unauthorized failures
- Provide actionable guidance in error messages
This addresses potential session creation failures by ensuring
consistent session configuration across all tools that create
child sessions.
LLMs often call look_at with 'path' instead of 'file_path' parameter,
causing TypeError and infinite retry loops.
- Add normalizeArgs() to accept both 'path' and 'file_path'
- Add validateArgs() with clear error messages showing correct usage
- Add tests for normalization and validation
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Background tasks were defaulting to $HOME instead of the parent session's
working directory. This caused background agents to scan the entire home
directory instead of the project directory, leading to:
- High CPU/memory load from scanning unrelated files
- Permission errors on system directories
- Task failures and timeouts
The fix retrieves the parent session's directory before creating a new
background session and passes it via the query.directory parameter.
Files modified:
- manager.ts: Look up parent session directory in launch()
- call-omo-agent/tools.ts: Same fix for sync mode
- look-at/tools.ts: Same fix for look_at tool
- sisyphus-task/tools.ts: Same fix + interface update for directory prop
- index.ts: Pass directory to sisyphusTask factory
Fixes#276 - The look_at tool was constructing invalid file:// URLs on Windows
by using template literals. Now uses Node.js pathToFileURL() which correctly
handles backslashes, spaces, and the triple-slash prefix required on Windows.
🤖 GENERATED WITH ASSISTANCE OF [OhMyOpenCode](https://github.com/code-yeongyu/oh-my-opencode)
- Embed files directly in message parts using file:// URL format
- Remove dependency on Read tool for multimodal-looker agent
- Add inferMimeType helper for proper MIME type detection
- Disable read tool in agent tools config (no longer needed)
- Upgrade multimodal-looker model to gemini-3-flash
- Update all README docs to reflect gemini-3-flash change
Fixes#126🤖 GENERATED WITH ASSISTANCE OF [OhMyOpenCode](https://github.com/code-yeongyu/oh-my-opencode)
Add a new tool and agent for analyzing media files (PDFs, images, diagrams)
that require visual interpretation beyond raw text.
- Add `multimodal-looker` agent using Gemini 2.5 Flash model
- Add `look_at` tool that spawns multimodal-looker sessions
- Restrict multimodal-looker from calling task/call_omo_agent/look_at tools
Inspired by Sourcegraph Ampcode's look_at tool design.
🤖 GENERATED WITH ASSISTANCE OF [OhMyOpenCode](https://github.com/code-yeongyu/oh-my-opencode)
YeonGyu-Kim