browser-widget-ai-agent

# Browser Widget AI Agent Skill

Control a VibeFuse **browser widget** on the canvas using AI tools to analyze web pages, discover URL routes, confirm results visually, and drive live pages (forms, fields, clicks).

---

## 1. Spawn the Browser Widget

Use `add_widgets` (preferred) or `create_browser_widget`.

```
add_widgets(session="<tab>", widgets=[{type:"browser", url:"https://example.com"}])
```

- If the user names a background tab, pass `session` = that tab name. Do not switch tabs unless asked.
- To open several tabs at once: `create_browser_widget(urls=["https://a.com","https://b.com"])`.

## 2. Analyze a Page (Routes & Structure)

Never assume what's on the page. Discover it.

### Step A — Read the live HTML
`get_browser_html(instanceId, maxChars=...)` returns raw markup. Use it to find:
- **Navigation links** — `<nav>`, `<header>`, `<ul class="menu">`, `<a href="...">`
- **Footer links** — `<footer>`, `<a href="...">`
- **Body content routes** — links inside `<main>`, `<article>`, cards, CTAs
- **Forms** — `<form>`, `<input>`, `<select>`, `<textarea>`, `<button>`

### Step B — Extract URL routes
Collect every distinct `href` from nav + footer + body. Build a route map:

| Source | Sample route |
|--------|-------------|
| Nav | `/pricing`, `/docs`, `/blog` |
| Footer | `/about`, `/contact`, `/privacy` |
| Body CTA | `/signup`, `/login` |

### Step C — Confirm visually with vision
`inspect_widget(instanceId, query="nav")` to see interactive controls, then
`screenshot_widget(instanceId, analyze=true, prompt="What routes are visible in the nav and footer?")`
to confirm the page actually renders those links (SPAs may render differently than raw HTML).

## 3. Drive the Page (Forms, Fields, Clicks)

Browser webviews receive **real trusted clicks and typing**, so React/SPA sites work.

### Discover controls first
`inspect_widget(instanceId, query="login")` → returns buttons, inputs, selects, checkboxes, etc.

### Interact
`interact_widget(instanceId, action="fill", selector="#email", text="user@example.com")`

Supported actions: `click`, `dblclick`, `hover`, `fill`, `select`, `check`, `toggle`, `submit`, `scroll`, `focus`, `press`, `wait`.

Examples:
- Fill a field: `interact_widget(instanceId, action="fill", label="Email", text="...")`
- Click a button: `interact_widget(instanceId, action="click", label="Sign in")`
- Select dropdown: `interact_widget(instanceId, action="select", label="Country", value="US")`
- Submit form: `interact_widget(instanceId, action="submit")`
- Press key: `interact_widget(instanceId, action="press", key="Enter")`

## 4. Verify After Every Action

Never assume it worked. After clicking/filling:
1. `inspect_widget(instanceId, query="<expected control>")` — did the UI change?
2. `screenshot_widget(instanceId, analyze=true, prompt="Did the form submit / did the page navigate?")` — visual confirmation.
3. `get_browser_html(instanceId)` — check the new URL/route in the markup.

## 5. Navigate Between Routes

`navigate_browser_widget(instanceId, url="https://example.com/pricing")` to change the URL.
Pass `newTab=true` to open another page as a tab inside the same widget.

## 6. Full Workflow Example — Analyze + Drive

1. `add_widgets` a browser widget at the site root.
2. `get_browser_html` → read nav + footer → extract routes.
3. `screenshot_widget(analyze=true)` → confirm routes render.
4. `navigate_browser_widget` to a discovered route (e.g. `/login`).
5. `inspect_widget(query="login")` → find the form fields.
6. `interact_widget(fill)` each field → `interact_widget(click)` submit.
7. `screenshot_widget(analyze=true)` → confirm success/error.
8. Report the real result (routes found, form outcome).

## Rules
- Always inspect/screenshot to verify — never assume a page state.
- Use `get_browser_html` for raw markup, not to drive the page.
- Use `interact_widget` (not HTML reads) to operate live pages.
- Report real results and errors; if a step fails, adjust and retry.