From f619e261c417947227d7b3fa00195333dafbf58a Mon Sep 17 00:00:00 2001 From: Kieran Klaassen Date: Thu, 18 Dec 2025 10:26:43 -0800 Subject: [PATCH] [2.14.0] Add /playwright-test command for browser testing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - New `/playwright-test` command for end-to-end browser tests on PR-affected pages - Uses Playwright MCP to navigate, snapshot, check console errors - Supports human-in-the-loop for OAuth/email/payment flows - Creates P1 todos for failures and retries until passing - Added Section 7 to `/workflows:review` - optional Playwright testing as subagent 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../.claude-plugin/plugin.json | 4 +- plugins/compound-engineering/CHANGELOG.md | 10 + plugins/compound-engineering/README.md | 5 +- .../commands/playwright-test.md | 248 ++++++++++++++++++ .../commands/workflows/review.md | 29 ++ 5 files changed, 292 insertions(+), 4 deletions(-) create mode 100644 plugins/compound-engineering/commands/playwright-test.md diff --git a/plugins/compound-engineering/.claude-plugin/plugin.json b/plugins/compound-engineering/.claude-plugin/plugin.json index e427568..27b0fa4 100644 --- a/plugins/compound-engineering/.claude-plugin/plugin.json +++ b/plugins/compound-engineering/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "compound-engineering", - "version": "2.13.0", - "description": "AI-powered development tools. 27 agents, 17 commands, 13 skills, 2 MCP servers for code review, research, design, and workflow automation.", + "version": "2.14.0", + "description": "AI-powered development tools. 27 agents, 18 commands, 13 skills, 2 MCP servers for code review, research, design, and workflow automation.", "author": { "name": "Kieran Klaassen", "email": "kieran@every.to", diff --git a/plugins/compound-engineering/CHANGELOG.md b/plugins/compound-engineering/CHANGELOG.md index 5426525..192a0b1 100644 --- a/plugins/compound-engineering/CHANGELOG.md +++ b/plugins/compound-engineering/CHANGELOG.md @@ -5,6 +5,16 @@ All notable changes to the compound-engineering plugin will be documented in thi The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [2.14.0] - 2025-12-18 + +### Added + +- **`/playwright-test` command** - Run end-to-end browser tests on pages affected by a PR or branch. Uses Playwright MCP to navigate pages, capture snapshots, check console errors, test interactions, and pause for human verification on OAuth/email/payment flows. Creates P1 todos for failures and retries until passing. + +### Changed + +- **`/workflows:review` command** - Added optional Playwright testing phase (Section 7). After review agents complete, offers to spawn `/playwright-test` as a subagent to verify affected pages in a real browser. + ## [2.13.0] - 2025-12-15 ### Added diff --git a/plugins/compound-engineering/README.md b/plugins/compound-engineering/README.md index b23f9da..4f958b7 100644 --- a/plugins/compound-engineering/README.md +++ b/plugins/compound-engineering/README.md @@ -7,8 +7,8 @@ AI-powered development tools that get smarter with every use. Make each unit of | Component | Count | |-----------|-------| | Agents | 27 | -| Commands | 17 | -| Skills | 12 | +| Commands | 18 | +| Skills | 13 | | MCP Servers | 2 | ## Agents @@ -95,6 +95,7 @@ Core workflow commands use `workflows:` prefix to avoid collisions with built-in | `/resolve_pr_parallel` | Resolve PR comments in parallel | | `/resolve_todo_parallel` | Resolve todos in parallel | | `/triage` | Triage and prioritize issues | +| `/playwright-test` | Run browser tests on PR-affected pages | ## Skills diff --git a/plugins/compound-engineering/commands/playwright-test.md b/plugins/compound-engineering/commands/playwright-test.md new file mode 100644 index 0000000..44472d1 --- /dev/null +++ b/plugins/compound-engineering/commands/playwright-test.md @@ -0,0 +1,248 @@ +--- +name: playwright-test +description: Run Playwright browser tests on pages affected by current PR or branch +argument-hint: "[PR number, branch name, or 'current' for current branch]" +--- + +# Playwright Test Command + +Run end-to-end browser tests on pages affected by a PR or branch changes using Playwright MCP. + +## Introduction + +QA Engineer specializing in browser-based end-to-end testing + +This command tests affected pages in a real browser, catching issues that unit tests miss: +- JavaScript integration bugs +- CSS/layout regressions +- User workflow breakages +- Console errors + +## Prerequisites + + +- Local development server running (e.g., `bin/dev`, `rails server`) +- Playwright MCP server connected +- Git repository with changes to test + + +## Main Tasks + +### 1. Determine Test Scope + + $ARGUMENTS + + + +**If PR number provided:** +```bash +gh pr view [number] --json files -q '.files[].path' +``` + +**If 'current' or empty:** +```bash +git diff --name-only main...HEAD +``` + +**If branch name provided:** +```bash +git diff --name-only main...[branch] +``` + + + +### 2. Map Files to Routes + + + +Map changed files to testable routes: + +| File Pattern | Route(s) | +|-------------|----------| +| `app/views/users/*` | `/users`, `/users/:id`, `/users/new` | +| `app/controllers/settings_controller.rb` | `/settings` | +| `app/javascript/controllers/*_controller.js` | Pages using that Stimulus controller | +| `app/components/*_component.rb` | Pages rendering that component | +| `app/views/layouts/*` | All pages (test homepage at minimum) | +| `app/assets/stylesheets/*` | Visual regression on key pages | +| `app/helpers/*_helper.rb` | Pages using that helper | + +Build a list of URLs to test based on the mapping. + + + +### 3. Verify Server is Running + + + +Before testing, verify the local server is accessible: + +``` +mcp__playwright__browser_navigate({ url: "http://localhost:3000" }) +mcp__playwright__browser_snapshot({}) +``` + +If server is not running, inform user: +```markdown +**Server not running** + +Please start your development server: +- Rails: `bin/dev` or `rails server` +- Node: `npm run dev` + +Then run `/playwright-test` again. +``` + + + +### 4. Test Each Affected Page + + + +For each affected route: + +**Step 1: Navigate and capture snapshot** +``` +mcp__playwright__browser_navigate({ url: "http://localhost:3000/[route]" }) +mcp__playwright__browser_snapshot({}) +``` + +**Step 2: Check for errors** +``` +mcp__playwright__browser_console_messages({ level: "error" }) +``` + +**Step 3: Verify key elements** +- Page title/heading present +- Primary content rendered +- No error messages visible +- Forms have expected fields + +**Step 4: Test critical interactions (if applicable)** +``` +mcp__playwright__browser_click({ element: "[description]", ref: "[ref]" }) +mcp__playwright__browser_snapshot({}) +``` + + + +### 5. Human Verification (When Required) + + + +Pause for human input when testing touches: + +| Flow Type | What to Ask | +|-----------|-------------| +| OAuth | "Please sign in with [provider] and confirm it works" | +| Email | "Check your inbox for the test email and confirm receipt" | +| Payments | "Complete a test purchase in sandbox mode" | +| SMS | "Verify you received the SMS code" | +| External APIs | "Confirm the [service] integration is working" | + +Use AskUserQuestion: +```markdown +**Human Verification Needed** + +This test touches the [flow type]. Please: +1. [Action to take] +2. [What to verify] + +Did it work correctly? +1. Yes - continue testing +2. No - describe the issue +``` + + + +### 6. Handle Failures + + + +When a test fails: + +1. **Document the failure:** + - Screenshot the error state + - Capture console errors + - Note the exact reproduction steps + +2. **Ask user how to proceed:** + ```markdown + **Test Failed: [route]** + + Issue: [description] + Console errors: [if any] + + How to proceed? + 1. Fix now - I'll help debug and fix + 2. Create todo - Add to todos/ for later + 3. Skip - Continue testing other pages + ``` + +3. **If "Fix now":** + - Investigate the issue + - Propose a fix + - Apply fix + - Re-run the failing test + +4. **If "Create todo":** + - Create `{id}-pending-p1-playwright-{description}.md` + - Continue testing + +5. **If "Skip":** + - Log as skipped + - Continue testing + + + +### 7. Test Summary + + + +After all tests complete, present summary: + +```markdown +## 🎭 Playwright Test Results + +**Test Scope:** PR #[number] / [branch name] +**Server:** http://localhost:3000 + +### Pages Tested: [count] + +| Route | Status | Notes | +|-------|--------|-------| +| `/users` | ✅ Pass | | +| `/settings` | ✅ Pass | | +| `/dashboard` | ❌ Fail | Console error: [msg] | +| `/checkout` | ⏭️ Skip | Requires payment credentials | + +### Console Errors: [count] +- [List any errors found] + +### Human Verifications: [count] +- OAuth flow: ✅ Confirmed +- Email delivery: ✅ Confirmed + +### Failures: [count] +- `/dashboard` - [issue description] + +### Created Todos: [count] +- `005-pending-p1-playwright-dashboard-error.md` + +### Result: [PASS / FAIL / PARTIAL] +``` + + + +## Quick Usage Examples + +```bash +# Test current branch changes +/playwright-test + +# Test specific PR +/playwright-test 847 + +# Test specific branch +/playwright-test feature/new-dashboard +``` diff --git a/plugins/compound-engineering/commands/workflows/review.md b/plugins/compound-engineering/commands/workflows/review.md index 5e8005a..b90bcd7 100644 --- a/plugins/compound-engineering/commands/workflows/review.md +++ b/plugins/compound-engineering/commands/workflows/review.md @@ -425,6 +425,35 @@ After creating all todo files, present comprehensive summary: ``` +### 7. Playwright Testing (Optional) + + +After presenting the Summary Report, ask the user: + +**"Want to run Playwright tests on the affected pages?"** +1. Yes - run `/playwright-test` +2. No - skip to next steps + + +#### If User Accepts: + +Spawn a subagent to run the tests (preserves main context): + +``` +Task general-purpose("Run /playwright-test for PR #[number]. Test all affected pages, check for console errors, handle failures by creating todos and fixing.") +``` + +The subagent will: +1. Identify pages affected by the PR +2. Navigate to each page and capture snapshots +3. Check for console errors +4. Test critical interactions +5. Pause for human verification on OAuth/email/payment flows +6. Create P1 todos for any failures +7. Fix and retry until all tests pass + +**Alternatively**, user can run standalone: `/playwright-test [PR number]` + ### Important: P1 Findings Block Merge Any **🔴 P1 (CRITICAL)** findings must be addressed before merging the PR. Present these prominently and ensure they're resolved before accepting the PR.