Media

What it does: Gives agents the ability to process images, audio, video, and documents — analyze, transcribe, synthesize, generate, and extract. Who it’s for: Agents living in group chats where users share photos, voice notes, documents, and video clips. Instead of ignoring media content, your agent can understand and respond to it.

Quick Reference

Tool	What It Does	Provider Categories
`image_analyze`	Analyze images using AI vision (file, URL, base64, or attachment URL)	Vision (follows the agent’s main vision-capable model — no separate key needed; else the Anthropic / OpenAI / Google registry)
`image_generate`	Generate an image from a text prompt and deliver it to the current channel	Image gen (follows the agent’s main provider by default; a Codex/ChatGPT-login main needs no image key; OpenRouter / fal opt-in)
`video_generate`	Generate a short video from a text prompt and deliver it to the current channel	Video gen (follows the agent’s main provider by default — Google→Veo, xAI→Grok Imagine, reusing its key; or explicit `fal` + `FAL_KEY`)
`tts_synthesize`	Generate spoken audio from text — returns a workspace file path	TTS (OpenAI, ElevenLabs, Edge)
`transcribe_audio`	Convert speech to text	STT (OpenAI Whisper, Groq, Deepgram)
`describe_video`	Describe raw video content using vision	Vision (Gemini-only raw video)
`extract_document`	Extract text from PDFs, CSVs, and other documents	None (local extraction)

For per-provider setup, model defaults, and configuration, see the Media & Voice section.

Tool Details

image_analyze -- Analyze images with AI vision

Use image_analyze to have your agent look at an image and describe what it sees, answer questions about the content, or extract information like text in a screenshot.The tool supports multiple ways to provide an image:

Parameter	Type	Required	Description
`action`	string	Yes	Always `"analyze"`
`source_type`	string	No	How the image is provided: `"file"`, `"url"`, or `"base64"` (required unless `attachment_url` is used)
`source`	string	No	The image data — a file path, URL, or base64-encoded string (required unless `attachment_url` is used)
`prompt`	string	No	A specific question about the image (e.g., “What text is visible in this screenshot?”). Defaults to a general description.
`attachment_url`	string	No	Platform attachment URL from a message hint (e.g., `tg-file://...`, `discord://...`). When provided, overrides `source_type`/`source`.
`mime_type`	string	No	MIME type for base64 input (auto-detected if omitted)

Source types explained:

file — A path to an image file on the server (e.g., /tmp/screenshot.png)
url — A publicly accessible image URL
base64 — Raw image data encoded as a base64 string

For images attached to chat messages, use the attachment_url parameter instead of source_type/source. The attachment_url is provided in the message hint and resolves platform-specific attachment URLs automatically.Example usage: When a user sends a photo in a chat, the agent can analyze it:

"Analyze this screenshot and tell me what application is shown."

The agent calls image_analyze with attachment_url from the message hint, then responds with a description of the image content.

Use the prompt parameter to ask specific questions about images. Without a prompt, the agent generates a general description. With a prompt like “What error message is shown?”, the agent focuses on extracting that specific information.

Vision follows the agent’s main provider. When the agent’s main model is itself vision-capable (Claude, GPT-4o, Gemini, and other models that accept image input), image_analyze routes the image through that main model, reusing the agent’s existing credentials — so no separate vision key is required in that case. The model that reasons for the agent is the model that “sees” the image, over one completion call.The resolver tries the tiers in this locked order and logs which tier served (the path field on the completion log):

main-vision — the agent’s main model, when it accepts image input (reuses the main provider’s key / Codex OAuth profile). No separate vision key needed.
registry — the independent vision registry’s own OpenAI -> Anthropic -> Google chain (its own configured keys; unchanged for current setups). Used when the main model is not vision-capable, when an explicit defaultProvider is set (see below), or as the fallback if a main-vision attempt fails at runtime.
honest “unavailable” — when no tier can serve, the tool reports an unavailable result with an error kind and a hint naming the knob to set, never a crash or a misroute to the wrong provider.

An explicit integrations.media.vision.defaultProvider in config.yaml overrides main-first (explicit operator config wins) — the registry tier serves that provider directly.

tts_synthesize -- Generate spoken audio from text

Use tts_synthesize to convert text into spoken audio. The tool returns a workspace file path (under media/tts/) along with the MIME type and size. The agent must then send that file via the message tool’s attach action to deliver it to a chat — TTS itself does not auto-deliver.

Parameter	Type	Required	Description
`action`	string	Yes	Always `"synthesize"`
`text`	string	Yes	The text to speak
`voice`	string	No	Voice identifier (provider-specific, available voices depend on your configured provider)
`format`	string	No	Audio format: `"mp3"`, `"opus"`, or `"wav"` (default: resolved from config and channel; Telegram defaults to opus)

The TTS provider (OpenAI, ElevenLabs, or Edge TTS) is determined by your config.yaml — there is no per-call provider parameter.Supported providers (configured in config.yaml):

OpenAI — High-quality voices with natural intonation. Requires an OpenAI API key.
ElevenLabs — Wide selection of realistic voices with emotion control. Requires an ElevenLabs API key.
Edge TTS — Free text-to-speech using Microsoft Edge’s built-in voices. No API key required.

Returns: { filePath, mimeType, sizeBytes }.Example two-step usage:

tts_synthesize with the summary text -> obtain filePath.
message action attach with attachment_url: file://{filePath}, attachment_type: audio to deliver the audio file.

transcribe_audio -- Convert speech to text

Use transcribe_audio to convert an audio or voice message into written text. This is useful for processing voice messages that users send in chat.

Parameter	Type	Required	Description
`attachment_url`	string	Yes	The URL of the audio file to transcribe (from a message hint, e.g., `tg-file://...`, `discord://...`)
`language`	string	No	BCP-47 language hint to improve transcription accuracy (e.g., `"en"`, `"he"`, `"es"`)

Supported providers:

OpenAI Whisper — High accuracy across many languages
Groq — Fast transcription with Whisper models
Deepgram — Real-time transcription with speaker detection

The provider used depends on your configuration. The tool returns the transcribed text, which the agent can then use in its response.Supported audio formats: Most common audio formats are supported, including MP3, WAV, OGG, M4A, and WEBM. Voice messages from chat platforms (Discord, Telegram, WhatsApp) are automatically handled.Example usage: When a user sends a voice message, the agent automatically transcribes it and responds to the spoken content. This works especially well in chat platforms where voice messages are common — the agent can participate in voice-message conversations by reading the transcribed text and responding in text or with a TTS audio reply.

describe_video -- Describe video content

Use describe_video to have your agent watch a video and describe what happens in it. The tool sends the raw video to a Gemini vision model (the only provider that accepts raw video natively in this release — there is no frame extraction); see the routing note below.

Parameter	Type	Required	Description
`attachment_url`	string	Yes	The URL of the video to describe (e.g., `tg-file://...`, `discord://...`)
`prompt`	string	No	Custom analysis prompt to guide the description (defaults to a generic description prompt)

The tool sends the raw video to a video-capable vision model and returns a coherent description of the content. This is useful for understanding short clips, tutorials, or screen recordings shared in chat.How it works:

The video is downloaded from the provided URL.
The raw video buffer is sent to a video-capable vision provider for analysis.
The provider returns a description of the video content.

Raw video is Gemini-only. Unlike image_analyze, describe_video does not route through an arbitrary main model — only Gemini accepts the raw video buffer (there is no frame extraction in this release). So describe_video goes straight to the gemini-video tier (via the vision registry’s Google provider); if no video-capable provider is configured, the tool reports an honest “unavailable” result with a hint to configure Gemini, never a crash. A non-Gemini main model does not change this — the main-provider vision path applies to image_analyze, not raw video.Example usage: When a user shares a video clip, the agent can describe what is happening:

"What's going on in this video?"

The agent watches the video and responds with a description like: “The video shows a step-by-step tutorial for configuring a Discord bot, starting with the developer portal and ending with the bot joining a server.”

extract_document -- Extract text from documents

Use extract_document to pull readable text from documents. This is useful for processing PDFs, spreadsheets, and other files that users share in chat.

Parameter	Type	Required	Description
`attachment_url`	string	Yes	The URL of the document to extract text from
`max_chars`	number	No	Maximum number of characters to extract from the document

Supported formats:

PDF — Extracts text from PDF documents
CSV — Reads comma-separated data
TXT — Plain text files
Other text-based formats

The tool returns the extracted text content, which the agent can then summarize, answer questions about, or process further.Example usage:

"Read this PDF and give me a summary of the key points."

The agent extracts the text from the PDF, then uses that text to generate a summary. For spreadsheets (CSV), the data is returned in a structured format that the agent can analyze, filter, or summarize.

Document extraction works best with text-based PDFs. Scanned documents (images of text) may require the image_analyze tool instead, since the content is stored as images rather than selectable text.

image_generate -- Generate images from text prompts

Use image_generate to create images from text descriptions. The generated image is delivered directly to the current channel via the daemon.By default (integrations.media.imageGeneration.provider: auto) image generation follows the agent’s main provider and reuses its credentials — no image-specific key is needed when the main provider can generate images. If the agent’s main provider cannot generate images (e.g. Anthropic) and no explicit provider is set, the tool returns an honest “unavailable” with a hint to set integrations.media.imageGeneration.provider — it never silently routes to a different paid provider. To opt in to OpenRouter, set provider: openrouter and supply OPENROUTER_API_KEY.Codex (ChatGPT-login) agents generate images with no API key. When the agent’s main provider is openai-codex (provider: auto), or you set provider: openai-codex explicitly, image_generate reuses the agent’s existing ChatGPT/Codex OAuth credentials — there is no image-specific API key to set. The OAuth bearer is resolved per call, so an expired token refreshes automatically; if the agent is not logged in, the tool returns an honest “unavailable” with a comis auth login --provider openai-codex hint (it never silently routes elsewhere).Native OpenAI and Google (key-auth) follow-main paths are wired. A key-auth openai main (with OPENAI_API_KEY) generates via the OpenAI Images API (default gpt-image-1), and a google main (with GOOGLE_API_KEY) generates via Gemini (default gemini-2.5-flash-image) — both under provider: auto, reusing the same key the main provider uses (no image-specific key). You can also pin them explicitly with provider: openai / provider: google.

Provider (resolved main)	Images path	`provider` value	Default model	Credential
`openai`	OpenAI Images API	`auto` or `openai`	`gpt-image-1`	`OPENAI_API_KEY`
`openai-codex`	ChatGPT/Codex (Responses)	`auto` or `openai-codex`	`gpt-image-1`	ChatGPT/Codex OAuth (no key)
`google` / `google-vertex`	Gemini (`generateContent`)	`auto` or `google`	`gemini-2.5-flash-image`	`GOOGLE_API_KEY`
`openrouter`	OpenRouter (built-in)	`auto` or `openrouter`	`black-forest-labs/flux.2-pro`	`OPENROUTER_API_KEY`
(explicit only)	fal.ai	`fal`	provider preset	`FAL_KEY`
`anthropic` and other image-incapable mains	—	—	—	honest “unavailable” (never a silent reroute)

Parameter	Type	Required	Description
`prompt`	string	Yes	Text description of the image to generate
`size`	string	No	Provider-specific size. fal.ai uses presets (`square_hd`, `landscape_16_9`); OpenAI uses pixel dimensions (`1024x1024`, `1792x1024`). Omit for the provider default.
`model`	string	No	Override the provider’s default image model (e.g. `gpt-image-1`, `gemini-2.5-flash-image`). Rejected with an actionable hint listing the valid models if it is not valid for the active provider. Omit to use the default.
`reference_image`	string	No	A workspace file path, URL, or data-URI of a reference image for edit/img2img. The OpenAI path routes it to `images.edit`; the Google path adds it as an `inlineData` part. Omit for text-to-image.

A reference_image is resolved safely: a workspace file path is confined to the agent’s workspace (path-traversal blocked), a URL passes an SSRF guard before any fetch (and is size-capped), and a data-URI/base64 is decoded in place.The daemon-side handler applies rate limiting, safety checking, model validation, and provider execution before delivering the generated image to the current channel.Delivery, persistence, and degradation. A generated image is first persisted to the agent’s confined workspace under ~/.comis/workspace/media/photos/ (a UUID filename, size-capped) and then delivered from that durable path — never an ephemeral OS temp file. Delivery is capability-driven, not a hardcoded channel list: a channel whose adapter implements attachment sending receives the persisted file directly; a channel without it (currently only IRC) degrades to a bounded base64 result returned over RPC. The same base64 fallback is used when the request has no originating channel, or if persistence or channel delivery fails — so a generation is never lost, it is just delivered by the best means the channel supports.Cost ceiling. Beyond the maxPerHour count limit, an optional per-agent USD ceiling, integrations.media.imageGeneration.maxCostPerHourUsd, bounds spend: once an agent’s accumulated image-generation cost in the current hour reaches the ceiling, further requests are blocked with errorKind: quota_exceeded (and a hint naming the key) before the provider is called. Unset means no cost ceiling (the count limit still applies). See config-yaml.mdx.Observability. Each generation logs a single INFO completion line carrying the executing image provider, the agent’s main provider, the model, durationMs, costUsd, sizeBytes, and the mime type; every failure branch logs an errorKind + an actionable hint (never the raw provider message, a key, or a bearer). The image turn is reconstructable from observability: comis explain <sessionKey> surfaces the image block (provider / model / costUsd / outcome / delivered) from the session trajectory, and the cost also flows into the per-session cost rollups (comis fleet).

When following a Codex main provider, Comis identifies itself to the ChatGPT backend as the official Codex client — it sends the same first-party client headers (originator: codex_cli_rs, a Codex-shaped User-Agent, a ChatGPT-Account-ID decoded from your token, and per-request session-id/x-client-request-id identifiers) that the official Codex CLI sends. This is a first-party-client compatibility shim using your own ChatGPT/Codex OAuth credentials against the same endpoint the official Codex CLI uses — so Cloudflare does not 403 requests from non-residential IPs. It is authorized reuse of your own credentials, not detection evasion. If a future change to the ChatGPT backend rejects this image path, set integrations.media.imageGeneration.provider: openrouter (with OPENROUTER_API_KEY) as the explicit fallback.

video_generate -- Generate videos from text prompts

Use video_generate to create a short video clip from a text prompt. Every backend — native Google Veo, xAI Grok Imagine, and the FAL queue API — runs through one unified submit -> poll -> download contract.Async by default (the render outlives the turn). A video render takes 30 s-5 min — longer than a single agent turn and longer than the promptTimeout. So video_generate does not block: it submits the job, persists a durable job record, and returns a job handle ({ jobId, state: "submitted", estimatedCostUsd }) promptly. A background poller then drives the render to completion independent of the originating turn and across a daemon restart, downloads the clip before the (expiring) provider URL dies, persists it, and announces the finished video to the originating channel on its own. Use the video_status tool with the returned jobId to check progress in the meantime. Delivery is at-least-once: a clip is delivered once it completes; if the daemon restarts mid-render it resumes from the persisted job and still delivers — so a rare, bounded duplicate is possible, but a render is never lost and never redelivered indefinitely.Provider following (integrations.media.videoGeneration.provider: auto, the default). Video generation follows the agent’s main provider and reuses its credentials, mirroring image generation: a google main resolves to Google Veo and an xai main resolves to xAI Grok Imagine, each with the key the agent already uses (GOOGLE_API_KEY / XAI_API_KEY) — no video-specific key. Where the main provider has no video API (e.g. openai, anthropic), the tool returns an honest “unavailable” with a hint to set integrations.media.videoGeneration.provider (e.g. fal + FAL_KEY) — it never silently routes to a different paid provider.

Under provider: auto (the default), a google main generates via Google Veo (default veo-3.0-fast-generate-001, GA — generates audio by default) reusing GOOGLE_API_KEY, and an xai main generates via xAI Grok Imagine (grok-imagine-video, 480p/720p) reusing XAI_API_KEY (or a SuperGrok login) — no video-specific key. Where the main provider has no video API, the tool still returns an honest “unavailable” naming the opt-in path. The native-Veo path also has a FAL-hosted fallback: set provider: fal + model: fal-ai/veo3.1 to render Veo through the FAL queue instead (it needs a second credential, FAL_KEY, so it is not the follow-main default).

Parameter	Type	Required	Description
`prompt`	string	Yes	Text description of the video to generate
`duration`	number	No	Clip length in seconds (provider-validated; default from `defaultDurationSecs`, e.g. 8)
`aspect_ratio`	string	No	`16:9`, `9:16`, `1:1` (provider-validated; default from `defaultAspectRatio`)
`resolution`	string	No	`720p`, `1080p`, `4k` (provider-validated; default from `defaultResolution`)
`audio`	boolean	No	Generate audio when the backend supports it (raises the cost estimate)
`negative_prompt`	string	No	What to avoid (provider-dependent)
`seed`	number	No	Deterministic seed (provider-dependent)
`image_url`	string	No	A workspace file path, URL, or data-URI of a source image for image-to-video. When present, the tool renders an image-to-video clip on backends that support it — FAL swaps to its image-to-video endpoint, and Veo/Grok add the source image to the same model. SSRF-guarded + workspace-confined. Omit for text-to-video.
`model`	string	No	Override the backend’s default video model/endpoint (e.g. `fal-ai/veo3.1/fast`). Omit to use the default.

Per-backend options (validated against the active backend). The clip-shape params are validated against the active backend’s capability matrix before the render is submitted: an unsupported value (a resolution the backend cannot produce, a duration outside its range, or image_url on a text-to-video-only model) is rejected up front with a hint listing the valid set for the current backend — it is never silently routed to a wrong value or surfaced as a provider error. The concrete per-backend options today are:

Backend	Durations	Resolutions	Audio	Image-to-video
FAL (`fal-ai/veo3.1/fast`)	`4` / `6` / `8` s (snapped to the nearest)	`720p` / `1080p` / `4k`	yes	yes (swaps to the FAL image-to-video endpoint)
Veo (`veo-3.0-fast-generate-001`)	`4` / `6` / `8` s (snapped)	`720p` / `1080p` / `4k` (`1080p`/`4k` require an `8` s duration)	yes	yes
Grok (`grok-imagine-video`)	`1`-`15` s (clamped)	`480p` / `720p` only	no	yes

Because the active backend follows the agent’s main provider (or integrations.media.videoGeneration.provider), the video_generate tool description the agent sees is built at daemon startup from the active backend’s real options — so the agent is told the active provider’s durations/resolutions/aspect ratios and whether image-to-video is supported up front, rather than a static superset. Changing provider therefore needs a daemon restart for the description to refresh (the same boot-bound contract as provider/credential selection).

Image-to-video currently takes a single source image (image_url). Supplying multiple reference images (e.g. distinct first-frame / last-frame / style-reference roles) is a planned follow-up and is not yet available.

Cost ceiling (estimate-first). Video is dollars-per-clip and is already rendering once submitted, so unlike image generation the per-agent USD ceiling, integrations.media.videoGeneration.maxCostPerHourUsd, is gated on a worst-case pre-submit estimate (duration x per-second rate, with audio/4k as the upper bound): if the estimate would push the agent’s accumulated hourly spend over the ceiling, the request is blocked with errorKind: quota_exceeded (and a hint naming the key) before the provider is called. The maxPerHour count limit is retained and orthogonal. Unset means no cost ceiling.Delivery, persistence, and degradation (in the background poller). When the render completes, the poller downloads the clip to a buffer and persists it to the agent’s confined workspace under ~/.comis/workspace/media/videos/ (a UUID filename, size-capped) before any delivery decision — an expiring provider URL cannot orphan it. Delivery is capability-driven: a channel whose adapter implements attachment sending receives the persisted file directly (as a video attachment); a channel without it (currently only IRC) degrades to a notice + the persisted path. Because delivery happens off-turn from the persisted job, the finished clip reaches the channel that requested it even after the turn has ended (and after a daemon restart). The persisted file is not deleted after delivery, so generated clips survive for later inspection.Oversized-clip degrade (never silently dropped). A 4K/long clip can exceed a channel’s video-upload limit. Rather than drop it, the poller checks the persisted clip’s size against a per-channel limit before sending and degrades gracefully — it is never routed through the silent “attachment too large -> fallback text” path. On a channel that renders URLs (Discord, Slack, Telegram, LINE, WhatsApp, Signal, iMessage, Email) the poller sends a link message to the retained provider URL when one is available; otherwise (or on a notice-only channel) it sends a notice message naming the saved workspace path. Either way the clip stays persisted and recoverable, and the message always carries the saved path. The per-channel limit follows each platform’s documented bot/upload cap (e.g. WhatsApp ~16 MB, Discord ~25 MB, Telegram ~50 MB, Slack/Signal/iMessage ~100 MB, LINE ~200 MB) and is overridable. IRC, which has no attachment surface at all, degrades to a notice via the same capability gate.Observability. When the render completes the poller logs a single INFO completion line carrying the executing video provider, the model, costUsd, sizeBytes, durationMs, and the mime type; every failure branch (submit failure, a failed render, or a poll timeout) logs an errorKind + an actionable hint (never the raw provider message, a key, a bearer, or the Veo keyed download URL — a raw provider/channel error message is scrubbed before it rides a log line). For a provider-native render (Veo/Grok) the poller preserves the specific classified failure kind off-turn — a safety-policy block is logged (and persisted to the job’s error) as content_blocked, a quota/credits limit as quota_exceeded, and so on — rather than collapsing every terminal failure to a generic empty result, so a content-policy block is distinguishable from a transient empty render in the logs alone. Because the completion runs off-turn, the poller stamps the originating traceId (captured at submit) on its log lines so the later completion stitches back to the request.comis explain reconstruction (incl. background completion). A video turn is reconstructable from observability: comis explain <sessionKey> surfaces a videoGenerated block (provider, model, jobId, estimatedCostUsd, the reconciled costUsd, outcome, delivered) from the session trajectory — including a job that completed in the background after the originating turn ended: the submit record (written in-turn) and the off-turn completion record are stitched by the carried jobId/traceId on the originating session_key (the column added to the video_jobs store for exactly this). The reconciled cost also flows into the per-session cost rollups (comis fleet). Cost provenance is per-backend: FAL and Veo report no per-call actual cost, so the reconstructed cost is the pre-submit estimate; xAI Grok reports an actual cost (from its usage ticks) that is reconciled in. (A live comis explain against a real background-completed render is an operator check; the deterministic reconstruction path is the binding contract.)

video_status -- Check a video render's progress and result

Use video_status to check the status of a video-generation job by the jobId that video_generate returned. Because a render outlives the turn (and survives a daemon restart), video_status is how the agent polls progress while the background poller drives the job to completion — the finished clip is then delivered to the channel automatically (no video_status call is required for delivery; it is for visibility).

Parameter	Type	Required	Description
`job_id`	string	Yes	The job handle returned by `video_generate`.

The response reports the job’s durable state:

Field	Type	When present
`state`	`"pending"` \| `"done"` \| `"failed"`	Always.
`progress`	number (0-1)	While `pending`, when the provider reports it.
`mediaPath`	string	When `done` — the persisted workspace path of the delivered clip.
`costUsd`	number	When `done` — the reconciled actual cost.
`error`	string	When `failed` — an actionable, secret-free hint describing the failure and what to do (e.g. “xAI blocked the prompt by moderation. Revise the prompt and retry.” for a content block, or a timeout hint naming the `timeoutMs` knob). The matching machine-readable failure kind (`content_blocked`, `quota_exceeded`, `auth_required`, `job_timeout`, `empty_response`, …) is carried on the off-turn poller’s `errorKind` log field, not this string.

Agent-scoped. video_status is scoped to the calling agent: a jobId that belongs to a different agent returns a not-found result (state: "failed"), never another agent’s media path or cost. In a multi-agent daemon an agent can only see its own jobs.

Common Workflows

Here are some typical ways agents use media tools together:

Voice message handling — When a user sends a voice note, the agent uses transcribe_audio to convert it to text, processes the request, and optionally uses tts_synthesize to reply with audio.
Document Q&A — A user shares a PDF report. The agent uses extract_document to read it, then answers questions about the content.
Image moderation — In a group chat, the agent uses image_analyze to check shared images for content that violates community guidelines.
Accessibility — The agent uses describe_video and image_analyze to provide text descriptions of visual content for users who need them.

Provider Configuration

Media tools route to providers based on the credentials present in your secret store and the providers listed in config.yaml. None of the tools accept a provider parameter at call time — selection is config-driven, not per-call.

Capability	Available providers	Auto-selection order	Required env keys
Vision — `image_analyze`	Follows the agent’s main provider when it is vision-capable (Claude / GPT-4o / Gemini and other image-accepting models), reusing its key; else the independent vision registry (OpenAI / Anthropic / Google)	main-vision -> registry (OpenAI -> Anthropic -> Google, first available) -> honest unavailable; an explicit `defaultProvider` overrides main-first	None in the main-vision case (reuses the main provider’s key / Codex OAuth profile). The registry fallback uses `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY` (at least one).
Vision — `describe_video` (raw video)	Gemini only (raw video buffer; no frame extraction this release)	gemini-video -> honest unavailable	`GOOGLE_API_KEY` (the vision registry’s Google provider)
TTS (`tts_synthesize`)	OpenAI TTS, ElevenLabs, Edge	Configured order; `auto-tts` heuristic decides when to use voice	`OPENAI_API_KEY`, `ELEVENLABS_API_KEY` (Edge is free)
STT (`transcribe_audio`)	OpenAI Whisper (gpt-4o-mini-transcribe), Groq Whisper, Deepgram nova-3	Fallback chain in declared order	`OPENAI_API_KEY`, `GROQ_API_KEY`, `DEEPGRAM_API_KEY` (at least one)
Image generation (`image_generate`)	Follows the agent’s main provider (`auto`): native OpenAI (`gpt-image-1`), Google Gemini (`gemini-2.5-flash-image`), Codex (ChatGPT-login), and OpenRouter; explicit `openrouter` / `openai-codex` / `openai` / `google` / `fal`	`auto` = the agent’s main provider (reuses its key, or OAuth bearer for a Codex main); else the explicit `provider`	None in the `auto` case (reuses the main provider’s key): `OPENAI_API_KEY` for an `openai` main, `GOOGLE_API_KEY` for a `google` main, the OAuth bearer for a Codex main (no image key). `OPENROUTER_API_KEY` for the OpenRouter opt-in; `FAL_KEY` for explicit `fal`. Honest “unavailable” (not silent) if the main provider can’t generate images and no `provider` is set.
Video generation (`video_generate`)	Follows the agent’s main provider (`auto`): native Google Veo (`veo-3.0-fast-generate-001`) for a `google` main, native xAI Grok Imagine (`grok-imagine-video`) for an `xai` main; explicit `fal` (FAL queue, `fal-ai/veo3.1/fast`; also the FAL-hosted-Veo fallback via `model: fal-ai/veo3.1`)	`auto` = the agent’s main provider (reuses its key); else the explicit `provider` (`fal` / `google` / `xai`)	`FAL_KEY` for explicit `fal`; in the `auto` case reuses the main provider’s key (`GOOGLE_API_KEY` for Veo, `XAI_API_KEY` for Grok). Honest “unavailable” (not silent) if the main provider can’t generate video and no `provider` is set.
Document extraction (`extract_document`)	Local pdf.js + CSV / text decoders	n/a	None (uses FFmpeg for some audio metadata)

Cost notes

Media calls bill against your model providers, not Comis itself. Rough rules of thumb:

Vision is significantly more expensive per call than text generation — a single high-res image can cost 1-5K tokens of input. Use prompt to focus the analysis and avoid re-analyzing the same image across turns.
STT is generally cheap (Groq’s whisper-large-v3-turbo is the lowest-latency / lowest-cost option in the fallback chain).
TTS is priced per character. ElevenLabs is the priciest tier; Edge TTS is free but voice quality is more limited.
Image generation has the highest per-call cost. The image-gen rate limiter is conservative by default (~1 req/min on OpenAI, ~1 req/sec on FAL), and you can additionally cap spend per agent/hour with integrations.media.imageGeneration.maxCostPerHourUsd (requests over the ceiling are blocked with quota_exceeded). Each generation’s costUsd is recorded in the session trajectory (comis explain) and the cost rollups (comis fleet).

Comis applies graceful degradation: if no credentials are configured for a capability, the corresponding tool is omitted from the agent’s tool catalog rather than failing at call time. For image_analyze, vision follows the agent’s main provider first (reusing its key when the main model is vision-capable — no separate vision key needed), then the independent vision registry, then an honest “unavailable” result; an explicit integrations.media.vision.defaultProvider overrides main-first. See the Media & Voice section for detailed provider setup, the Vision page for image flow details, and Voice for TTS/STT options.

All media tools process content on your server. Files are not sent to third parties beyond the configured AI provider for analysis.

Enabling Media Tools

Media tools are enabled by default in the full tool policy profile. If you are using a restricted profile (like minimal or coding), you can enable specific media tools by adding them to your agent’s allow list:

skills:
  toolPolicy:
    profile: minimal
    allow:
      - image_analyze
      - transcribe_audio

See Tool Policy for more details on controlling which tools your agents can access.

Media & Voice

Detailed media provider setup and configuration

Vision

How AI vision processing works

Voice

Text-to-speech and transcription configuration

Agent Tools Overview

See all available agent tools

Overview

Tool Guides

Platform Actions

Quick Reference

Tool Details

Common Workflows

Provider Configuration

Cost notes

Enabling Media Tools

Media & Voice

Vision

Voice

Agent Tools Overview

​Quick Reference

​Tool Details

​Common Workflows

​Provider Configuration

​Cost notes

​Enabling Media Tools

​Related

Media & Voice

Vision

Voice

Agent Tools Overview

Quick Reference

Tool Details

Common Workflows

Provider Configuration

Cost notes

Enabling Media Tools

Related