Jarvis Tools — Canvas, Skills, HyperFrames & Search

Jarvis on the canvas

Spawn Jarvis from the Agents / Widgets rail. Each instance keeps its own message memory. Replies render as GitHub-flavored markdown. Tools only run through the model’s function-calling API — writing a tool name in the chat text does nothing. Voice wake-word and hold-to-talk use this same list; see Voice dock.

Prefer add_widgets to open several panes in one call. Call list_canvas_widgets before spawning a second Terminal — reuse an open CLI with type_terminal. Session tools default to the prompt session (the tab that received the request), not a tab the user switched to mid-turn.

JSON speech envelope and `play_tts`

A finished Jarvis (and voice) reply is a JSON object with two strings:

{ "text": "GitHub-flavored markdown for the chat widget.", "speech": "I have finished the layout." }

The client displays only text. speech is Piper for the completed result.
While work is in progress, call play_tts (aliases speak_tts, say, speak) for a short first-person update.
Never call play_tts just to announce that a tool is being used. Completion speech belongs in speech, not another tool call.
Spoken lines start with I am, I have, I will, or I did. One short sentence. Letters, numbers, spaces, commas, periods, and dashes only — no URLs, paths, or markdown.

Voices, Test Piper, and the :4123 fallback live on Piper TTS.

Canvas / sessions

Tool	What it does
`add_widgets`	Preferred create: open browser, terminal, YouTube, HyperFrames, video, CLI, or catalog widgets on the prompt session without rearranging existing ones.
`list_canvas_widgets`	List widgets on a session (instanceId, title, type, URL, layout, browser tabs).
`list_sessions`	List session tabs (id, name, active, promptSession, widget count, workspacePath).
`create_session`	Create a background session tab, then `add_widgets` onto it. Does not steal the visible tab unless `activate=true`.
`rename_session`	Rename a session tab.
`close_session`	Close a session and its widgets. Closing the last tab creates a fresh empty session.
`switch_session`	Target a session for later tools without changing the visible tab mid-turn.
`set_session_workspace`	Set or clear the folder path terminals, Files, and iframes use on that session.
`layout_widget`	Rearrange, resize, or tile existing widgets; `arrange=tile` + `all=true` cleans overlap.
`remove_widget`	Close widgets by instanceId, title, catalog id, or `all=true`.
`focus_widget`	Raise a widget and pan/zoom so it is fully in view.
`set_canvas_view`	Set pan and/or zoom (or reset the camera) on a session canvas.
`rename_widget`	Change a widget window title.
`screenshot_canvas`	Capture the viewport or entire canvas and optionally vision-analyze it.
`screenshot_widget`	Capture one widget (content or chrome) and optionally analyze it.

Skills

Packages live in C:\vibefuse\skills. Step-by-step authoring: Write a skill.

Tool	What it does
`list_skills`	List installed skills (name, description, slug, author).
`create_skill`	Create or overwrite a SKILL.md package; the Skills rail and Settings → Skills refresh immediately.
`update_skill`	Patch description, body, and/or supporting files.
`delete_skill`	Delete the skill folder and remove it from the rail.
`execute_skill`	Load the full SKILL.md and apply it to the current task (also runs when you click a skill).
`read_skill`	Return the full SKILL.md body.
`read_skill_file`	Read a supporting file under the skill folder (`references/`, `scripts/`, `assets/`).

Styling

Packs live in C:\vibefuse\styling and appear under Settings → Appearance.

Tool	What it does
`list_styling_packs`	List installed appearance packs (name, orb, particles, chrome).
`create_styling_pack`	Write manifest.json, theme.json, and orb.json and optionally activate the pack.
`update_styling_pack`	Patch tokens, particles, effects, CSS/JS, or orb and re-apply if active.
`apply_styling_pack`	Select a pack (or clear back to the built-in theme grid) and refresh the live UI.
`refresh_styling_pack`	Reload packs from disk and re-apply the current one.

Widgets

Custom packages go in C:\vibefuse\widgets. Guide: Build a widget.

Tool	What it does
`create_widget`	Author a new marketplace iframe package (manifest.json + index.html) and optionally spawn it.
`update_widget`	Update HTML, description, icon, or tags; version bump reloads open iframes.
`publish_widget`	Publish a local package as a Fuse Intelligence marketplace listing (signed-in license required).
`list_widget_catalog`	List every spawnable widget — built-ins and local packages.
`create_catalog_widget`	Open an existing catalog widget (does not write a new package). Use `create_widget` to author one.

Browser / media

Tool	What it does
`create_browser_widget`	Open a browser widget with a URL, domain, search, or several tabs. Prefer `add_widgets`; reuse an open page with `navigate_browser_widget`.
`create_youtube_widget`	Open a YouTube widget, optionally preloaded with a search or video URL.
`navigate_browser_widget`	Change the URL of an open browser/YouTube/saved-page; focuses an existing tab when that URL is already loaded.
`get_browser_html`	Read live HTML or a CSS-selected fragment. Prefer inspect/interact for clicks; screenshot to see pixels.
`execute_browser_js`	Evaluate JavaScript in the page and return a JSON-serializable result.
`scroll_browser`	Scroll the live page (direction, amount, or selector into view), then screenshot to verify.
`screenshot_browser`	Capture the webview and vision-describe it. Call after navigate/click/submit before saying a web task is done.
`inspect_widget`	List visible interactive controls (selectors, labels, rects) in any mounted widget, including browser pages.
`interact_widget`	Click, type, fill, or otherwise act on a control. Use inspect first. Do not use this for CLI/xterm input.

Terminal

Tool	What it does
`create_terminal`	Spawn a Terminal widget. Reuse an open Terminal/CLI with `type_terminal` instead of duplicating.
`create_cli_widget`	Spawn a CLI widget for terminal, Claude, Codex, Gemini, Cursor, Qwen, or another supported provider.
`read_terminal`	Read visible/scrollback text from a Terminal or CLI widget. Always call after type + wait.
`type_terminal`	Type into an open PTY and submit Enter (default). Workflow: list → type → wait → read.

Web

Search and scrape use the local SearXNG / Crawl4AI sidecars from Settings → Local Servers.

Tool	What it does
`web_search`	Search the public web via local SearXNG; returns titles, URLs, and snippets.
`firecrawl_search`	Same live search path; optional `scrape=true` pulls markdown per result via Crawl4AI.
`firecrawl_scrape`	Scrape one URL to clean markdown (Crawl4AI, with a fetch fallback).
`firecrawl_map`	Discover URLs on a site before scraping.

HyperFrames

Studio runs on http://127.0.0.1:4567 from Settings → Local Servers. Workspace C:\vibefuse\hyperframes. Product page: HyperFrames Studio. Call hyperframes_compose before render — a blank starter produces a black MP4.

Tool	What it does
`hyperframes_status`	Whether Studio is running, studio URL, HyperFrames dir, and the session workspace folder.
`hyperframes_open`	Start Studio if needed and place the HyperFrames widget on a session.
`create_hyperframes_widget`	Alias of `hyperframes_open`.
`hyperframes_compose`	Required before render: build a visible composition from the workspace + prompt (title, tagline, scene cards).
`hyperframes_init`	Scaffold a project under `C:\vibefuse\hyperframes`. Compose next — do not render the blank starter.
`hyperframes_list`	List projects plus the selected workspace and a suggested project name.
`hyperframes_inspect_workspace`	Read the canvas workspace (tree, README, package.json) so a video can match that project.
`hyperframes_write`	Write a file under the HyperFrames workspace. Prefer compose over hand-authoring a blank index.html.
`hyperframes_read`	Read a file from the HyperFrames workspace (relative path, e.g. `studio/index.html`).
`hyperframes_run`	Run an allowlisted CLI command (init, catalog, add, lint, check, snapshot, render, info, …).
`hyperframes_render`	Render the composed project to MP4. Refuses a blank composition.
`create_video_widget`	Open a Video widget on a local file (typically the MP4 from `hyperframes_render`).

Memory / LLM

Tool	What it does
`agent_memory`	Read, write, append, or delete lasting facts in `C:\vibefuse\MEMORY.md` (also injected into the system prompt).
`agent_task_list`	Manage the running task list in `C:\vibefuse\TASK_LIST.md` (pending / in_progress / done / blocked / cancelled).
`get_llm_settings`	Read the current Settings → Agent provider, endpoint, model, and whether a key is set.
`set_llm_settings`	Change Jarvis provider, endpoint, model, and/or API key (aliases `set_provider`, `configure_llm`).
`list_llm_models`	List models from the current (or just-configured) endpoint.
`list_subagent_providers`	List saved Settings → Agent profiles before spawning a background agent. Never invent API keys.
`spawn_subagent`	Delegate one task to a background Jarvis sub-agent (same tools by default). Returns an id immediately. Max 4 run at once.
`spawn_subagents`	Start several sub-agents in one call; extras queue.
`list_subagents`	List this session’s sub-agents (id, name, status, provider, model, result preview).
`get_subagent`	Read one sub-agent’s status and full result. Do not invent an answer while it is running.
`wait_subagent`	Wait until a sub-agent finishes (default 30s) and return its result.
`cancel_subagent`	Abort a running background sub-agent.

Other

Tool	What it does
`wait`	Pause milliseconds (or seconds) so CLI/SSH output can finish before read or screenshot.
`play_tts`	Speak a brief first-person Piper update while a task is in progress. See speech rules and Piper TTS.
`github`	Preferred GitHub tool using the PAT from Settings → Extensions (repos, issues, pulls, contents, search). Integers must be JSON numbers.
`list_google_drive_files`	List files in the signed-in Drive. Prefer this over MCP full-text search with `*` or an empty query.
`code_studio`	Drive an open Code Studio: list/open/edit/save files, commit, push, create/delete, select repo or folder.
`raspberry_pi`	Drive the Raspberry Pi widget (SSH left, VNC right). Do not use type_terminal or interact_widget on this widget.
`telegram_screenshot`	Capture the canvas or a widget and send a real Telegram photo in the current chat (after Extensions → Telegram).
`telegram_video`	Send a local video file (e.g. a HyperFrames MP4) into the current Telegram chat.

Connected MCP tools (Extensions + Skill Seekers)

After you Connect a card, Jarvis and voice also get that server’s tools. Names are prefixed by connection. Nothing here appears until the card is ready.

Source	Where you connect it	Jarvis names
Google Workspace / Drive	Settings → Extensions	`mcp_google-workspace_`, `mcp_google-drive_`, plus `list_google_drive_files`
GitHub / Copilot / Project Manager	Extensions	`github`, `mcp_github_`, `mcp_github-copilot_`, `mcp_github-project-manager_*`
Discord	Extensions	`mcp_discord_*` (omit guildId; default server is filled in)
Telegram	Extensions	Not MCP — the gateway uses the same Jarvis tools, plus `telegram_screenshot` / `telegram_video`
Linear	Extensions	`mcp_linear_*`
Custom stdio	Extensions → Add MCP Connection	`mcp_<server-name>_*` as listed by that process
Skill Seekers	Settings → Local Servers	`mcp_skill-seekers_*`. If the user asks whether a skill exists, `execute_skill` with `skillId=skill-seekers-lookup` first.

HyperFrames, Skill Seekers, and Crawl4AI are not Extensions cards — they start on Local Servers.