How do I review a Playwright test an AI agent wrote?

Review an AI-generated test by checking that it actually asserts user-visible outcomes with web-first assertions, uses resilient semantic locators instead of brittle selectors, and fails when the application behavior is broken.

This answer is one piece of Chapter 31: Reviewing AI-Written Tests.

Why Agent-Written Tests Pass Review
The Assertion That Isn't One
Tests That Pass for the Wrong Reason
Setup That Silently Does Nothing
Reading a Diff You Didn't Write
Accept, Fix, or Throw Away

AI coding agents generate syntactically valid Playwright tests with ease, but they frequently introduce subtle defects that undermine test suite reliability.

When reviewing an AI-generated test, look for four common structural issues:

Assertion validity: Did the AI use web-first assertions like expect(locator).toHaveText(), or did it use passive reads like textContent() without an assertion?
Unasserted actions: Did the agent read values from the page into variables without ever passing them to expect()?
Brittle locators: Did the agent write fragile CSS or XPath selectors instead of semantic role-based locators like getByRole() and getByTestId()?
Verification under failure: Break the underlying behavior by commenting out the action. If the test still reports green, the agent generated an empty test.

review-ai-test.spec.js(https://practice.learnplayforge.com)

import { test, expect } from '@playwright/test';

// Bad agent-style test: reads cart-total into the void, asserts nothing
test('unasserted agent test (agent.spec.js)', async ({ page }) => {
  await page.goto('/catalog.html?mock=static&seed=42&autologin=1');
  const grinder = page.locator('article[data-sku="MRD-1003"]');
  await grinder.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByTestId('cart-total').textContent();
});

// Reviewed fix: verifies outcome with web-first toHaveText matcher (fixed.spec.js)
test('reviewed test with assertion (fixed.spec.js)', async ({ page }) => {
  await page.goto('/catalog.html?mock=static&seed=42&autologin=1');
  const grinder = page.locator('article[data-sku="MRD-1003"]');
  await grinder.getByRole('button', { name: 'Add to cart' }).click();
  await expect(page.getByTestId('cart-total')).toHaveText('₹11,900');
});

Output

$ npx playwright test agent.spec.js
1 passed (3.8s)

$ npx playwright test fixed.spec.js
1 passed (7.9s)

Test yourself

An AI agent writes a test ending with await page.getByTestId('cart-total').textContent(). What defect does this test contain?

Practised in the full course: the drill The typo that passes.

See the chapter

AI can write the test now. The job is knowing whether it checks anything. That's what the course teaches, one chapter at a time.

163 lessons · 226 drills · start free

How do I review a Playwright test an AI agent wrote?

Related questions