Playwright Test Agents: CLI vs MCP vs the Test Runner

• npx playwright test — version-controlled specs. This is the only release gate.
• @playwright/cli — token-cheap clicks and snapshots for Cursor, Claude Code, or Copilot while they edit the repo.
• Playwright MCP — a live accessibility tree for long exploratory loops. Do not run it as CI.

If Playwright tests fail in CI but pass locally, fix that before you give an agent permission to rewrite locators. An agent will paper over drift with weaker assertions.

Use this when a human has already written the acceptance criteria. The agent authors the spec; CI still runs the committed file.

# 1. Agent explores (dev machine only)
npx @playwright/cli@latest open https://staging.example.com/checkout
npx @playwright/cli@latest snapshot

# 2. Agent writes tests/checkout.spec.ts
# 3. Human reviews the PR — reject status-only asserts
# 4. CI runs the committed spec, not the agent session
npx playwright test tests/checkout.spec.ts --project=chromium

import { test, expect } from "@playwright/test";

test("checkout Pay now stays disabled until card is valid", async ({ page }) => {
  await page.goto("/checkout");
  const pay = page.getByRole("button", { name: "Pay now" });
  await expect(pay).toBeDisabled();
  await page.getByLabel("Card number").fill("4242424242424242");
  await page.getByLabel("Expiry").fill("12/30");
  await page.getByLabel("CVC").fill("123");
  await expect(pay).toBeEnabled();
});

• Reject any agent spec that only asserts page.toHaveURL or HTTP 200.
• Scope the agent to tests/**. Do not let it edit playwright.config.ts or GitHub workflows on the first week.
• Keep an audit line in the PR: which prompt, which files, who approved.
• If you need a black-box-free rollout, start from introduce AI into QA. Check where your team sits today with the free QA maturity assessment.

• Pass: Agent-authored spec is reviewed, merged, and the same file fails when you break the Pay now enablement rule.
• Fail: The agent session is green but CI never runs the file, or the agent changed expected text to match a bug.
• Cleanup: Delete MCP traces from the repo. Do not commit .playwright-cli session dumps.