feat(skills): add dev-browser skill with Windows support (#1093)

* feat(skills): add dev-browser skill with Windows support * chore: trigger CI
2026-01-25 15:34:07 +09:00
parent 05904ca617
commit bccc943173
4 changed files with 782 additions and 1 deletions
@@ -0,0 +1,193 @@
+# Dev Browser Installation Guide
+
+This guide covers installation for all platforms: macOS, Linux, and Windows.
+
+## Prerequisites
+
+- [Node.js](https://nodejs.org) v18 or later with npm
+- Git (for cloning the skill)
+
+## Installation
+
+### Step 1: Clone the Skill
+
+```bash
+# Clone dev-browser to a temporary location
+git clone https://github.com/sawyerhood/dev-browser /tmp/dev-browser-skill
+
+# Copy to skills directory (adjust path as needed)
+# For oh-my-opencode: already bundled
+# For manual installation:
+mkdir -p ~/.config/opencode/skills
+cp -r /tmp/dev-browser-skill/skills/dev-browser ~/.config/opencode/skills/dev-browser
+
+# Cleanup
+rm -rf /tmp/dev-browser-skill
+```
+
+**Windows (PowerShell):**
+```powershell
+# Clone dev-browser to temp location
+git clone https://github.com/sawyerhood/dev-browser $env:TEMP\dev-browser-skill
+
+# Copy to skills directory
+New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.config\opencode\skills"
+Copy-Item -Recurse "$env:TEMP\dev-browser-skill\skills\dev-browser" "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+
+# Cleanup
+Remove-Item -Recurse -Force "$env:TEMP\dev-browser-skill"
+```
+
+### Step 2: Install Dependencies
+
+```bash
+cd ~/.config/opencode/skills/dev-browser
+npm install
+```
+
+**Windows (PowerShell):**
+```powershell
+cd "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+npm install
+```
+
+### Step 3: Start the Server
+
+#### Standalone Mode (New Browser Instance)
+
+**macOS/Linux:**
+```bash
+cd ~/.config/opencode/skills/dev-browser
+./server.sh &
+# Or for headless:
+./server.sh --headless &
+```
+
+**Windows (PowerShell):**
+```powershell
+cd "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+Start-Process -NoNewWindow -FilePath "node" -ArgumentList "server.js"
+# Or for headless:
+Start-Process -NoNewWindow -FilePath "node" -ArgumentList "server.js", "--headless"
+```
+
+**Windows (CMD):**
+```cmd
+cd %USERPROFILE%\.config\opencode\skills\dev-browser
+start /B node server.js
+```
+
+Wait for the `Ready` message before running scripts.
+
+#### Extension Mode (Use Existing Chrome)
+
+**macOS/Linux:**
+```bash
+cd ~/.config/opencode/skills/dev-browser
+npm run start-extension &
+```
+
+**Windows (PowerShell):**
+```powershell
+cd "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+Start-Process -NoNewWindow -FilePath "npm" -ArgumentList "run", "start-extension"
+```
+
+Wait for `Extension connected` message.
+
+## Chrome Extension Setup (Optional)
+
+The Chrome extension allows controlling your existing Chrome browser with all your logged-in sessions.
+
+### Installation
+
+1. Download `extension.zip` from [latest release](https://github.com/sawyerhood/dev-browser/releases/latest)
+2. Extract to a permanent location:
+   - **macOS/Linux:** `~/.dev-browser-extension`
+   - **Windows:** `%USERPROFILE%\.dev-browser-extension`
+3. Open Chrome → `chrome://extensions`
+4. Enable "Developer mode" (toggle in top right)
+5. Click "Load unpacked" → select the extracted folder
+
+### Usage
+
+1. Click the Dev Browser extension icon in Chrome toolbar
+2. Toggle to "Active"
+3. Start the extension relay server (see above)
+4. Use dev-browser scripts - they'll control your existing Chrome
+
+## Troubleshooting
+
+### Server Won't Start
+
+**Check Node.js version:**
+```bash
+node --version  # Should be v18+
+```
+
+**Check port availability:**
+```bash
+# macOS/Linux
+lsof -i :3000
+
+# Windows
+netstat -ano | findstr :3000
+```
+
+### Playwright Installation Issues
+
+If Chromium fails to install:
+```bash
+npx playwright install chromium
+```
+
+### Windows-Specific Issues
+
+**Execution Policy:**
+If PowerShell scripts are blocked:
+```powershell
+Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
+```
+
+**Path Issues:**
+Use forward slashes or escaped backslashes in paths:
+```powershell
+# Good
+cd "$env:USERPROFILE/.config/opencode/skills/dev-browser"
+# Also good
+cd "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+```
+
+### Extension Not Connecting
+
+1. Ensure extension is "Active" (click icon to toggle)
+2. Check relay server is running (`npm run start-extension`)
+3. Look for `Extension connected` message in console
+4. Try reloading the extension in `chrome://extensions`
+
+## Permissions
+
+To skip permission prompts in Claude Code, add to `~/.claude/settings.json`:
+
+```json
+{
+  "permissions": {
+    "allow": ["Skill(dev-browser:dev-browser)", "Bash(npx tsx:*)"]
+  }
+}
+```
+
+## Updating
+
+```bash
+cd ~/.config/opencode/skills/dev-browser
+git pull
+npm install
+```
+
+**Windows:**
+```powershell
+cd "$env:USERPROFILE\.config\opencode\skills\dev-browser"
+git pull
+npm install
+```
@@ -0,0 +1,155 @@
+# Data Scraping Guide
+
+For large datasets (followers, posts, search results), **intercept and replay network requests** rather than scrolling and parsing the DOM. This is faster, more reliable, and handles pagination automatically.
+
+## Why Not Scroll?
+
+Scrolling is slow, unreliable, and wastes time. APIs return structured data with pagination built in. Always prefer API replay.
+
+## Start Small, Then Scale
+
+**Don't try to automate everything at once.** Work incrementally:
+
+1. **Capture one request** - verify you're intercepting the right endpoint
+2. **Inspect one response** - understand the schema before writing extraction code
+3. **Extract a few items** - make sure your parsing logic works
+4. **Then scale up** - add pagination loop only after the basics work
+
+This prevents wasting time debugging a complex script when the issue is a simple path like `data.user.timeline` vs `data.user.result.timeline`.
+
+## Step-by-Step Workflow
+
+### 1. Capture Request Details
+
+First, intercept a request to understand URL structure and required headers:
+
+```typescript
+import { connect, waitForPageLoad } from "@/client.js";
+import * as fs from "node:fs";
+
+const client = await connect();
+const page = await client.page("site");
+
+let capturedRequest = null;
+page.on("request", (request) => {
+  const url = request.url();
+  // Look for API endpoints (adjust pattern for your target site)
+  if (url.includes("/api/") || url.includes("/graphql/")) {
+    capturedRequest = {
+      url: url,
+      headers: request.headers(),
+      method: request.method(),
+    };
+    fs.writeFileSync("tmp/request-details.json", JSON.stringify(capturedRequest, null, 2));
+    console.log("Captured request:", url.substring(0, 80) + "...");
+  }
+});
+
+await page.goto("https://example.com/profile");
+await waitForPageLoad(page);
+await page.waitForTimeout(3000);
+
+await client.disconnect();
+```
+
+### 2. Capture Response to Understand Schema
+
+Save a raw response to inspect the data structure:
+
+```typescript
+page.on("response", async (response) => {
+  const url = response.url();
+  if (url.includes("UserTweets") || url.includes("/api/data")) {
+    const json = await response.json();
+    fs.writeFileSync("tmp/api-response.json", JSON.stringify(json, null, 2));
+    console.log("Captured response");
+  }
+});
+```
+
+Then analyze the structure to find:
+
+- Where the data array lives (e.g., `data.user.result.timeline.instructions[].entries`)
+- Where pagination cursors are (e.g., `cursor-bottom` entries)
+- What fields you need to extract
+
+### 3. Replay API with Pagination
+
+Once you understand the schema, replay requests directly:
+
+```typescript
+import { connect } from "@/client.js";
+import * as fs from "node:fs";
+
+const client = await connect();
+const page = await client.page("site");
+
+const results = new Map(); // Use Map for deduplication
+const headers = JSON.parse(fs.readFileSync("tmp/request-details.json", "utf8")).headers;
+const baseUrl = "https://example.com/api/data";
+
+let cursor = null;
+let hasMore = true;
+
+while (hasMore) {
+  // Build URL with pagination cursor
+  const params = { count: 20 };
+  if (cursor) params.cursor = cursor;
+  const url = `${baseUrl}?params=${encodeURIComponent(JSON.stringify(params))}`;
+
+  // Execute fetch in browser context (has auth cookies/headers)
+  const response = await page.evaluate(
+    async ({ url, headers }) => {
+      const res = await fetch(url, { headers });
+      return res.json();
+    },
+    { url, headers }
+  );
+
+  // Extract data and cursor (adjust paths for your API)
+  const entries = response?.data?.entries || [];
+  for (const entry of entries) {
+    if (entry.type === "cursor-bottom") {
+      cursor = entry.value;
+    } else if (entry.id && !results.has(entry.id)) {
+      results.set(entry.id, {
+        id: entry.id,
+        text: entry.content,
+        timestamp: entry.created_at,
+      });
+    }
+  }
+
+  console.log(`Fetched page, total: ${results.size}`);
+
+  // Check stop conditions
+  if (!cursor || entries.length === 0) hasMore = false;
+
+  // Rate limiting - be respectful
+  await new Promise((r) => setTimeout(r, 500));
+}
+
+// Export results
+const data = Array.from(results.values());
+fs.writeFileSync("tmp/results.json", JSON.stringify(data, null, 2));
+console.log(`Saved ${data.length} items`);
+
+await client.disconnect();
+```
+
+## Key Patterns
+
+| Pattern                 | Description                                            |
+| ----------------------- | ------------------------------------------------------ |
+| `page.on('request')`    | Capture outgoing request URL + headers                 |
+| `page.on('response')`   | Capture response data to understand schema             |
+| `page.evaluate(fetch)`  | Replay requests in browser context (inherits auth)     |
+| `Map` for deduplication | APIs often return overlapping data across pages        |
+| Cursor-based pagination | Look for `cursor`, `next_token`, `offset` in responses |
+
+## Tips
+
+- **Extension mode**: `page.context().cookies()` doesn't work - capture auth headers from intercepted requests instead
+- **Rate limiting**: Add 500ms+ delays between requests to avoid blocks
+- **Stop conditions**: Check for empty results, missing cursor, or reaching a date/ID threshold
+- **GraphQL APIs**: URL params often include `variables` and `features` JSON objects - capture and reuse them