How can a passing test check nothing?

A test passes while checking nothing if it performs actions and reads DOM state without calling expect(), because the test runner marks any test that exits without throwing an unhandled exception as green.

This answer is one piece of Chapter 31: Reviewing AI-Written Tests.

Why Agent-Written Tests Pass Review
The Assertion That Isn't One
Tests That Pass for the Wrong Reason
Setup That Silently Does Nothing
Reading a Diff You Didn't Write
Accept, Fix, or Throw Away

One of the most dangerous defects in automated testing is a test that passes regardless of whether the application works. This false confidence happens when a test interacts with the page but never asserts on the resulting state.

In agent.spec.js, an AI agent wrote a test that navigates to the catalog, clicks 'Add to cart', and calls page.getByTestId('cart-total').textContent(). Because textContent() merely returns a Promise resolving to text rather than making an assertion, no error is thrown.

To prove this test checks nothing, broken.spec.js comments out the button click entirely. When executed, broken.spec.js still finishes with 1 passed (2.4s). The test runner only checks if an unhandled Error occurred; without an explicit expect() assertion, a broken feature passes every run.

broken.spec.js(https://practice.learnplayforge.com)

import { test, expect } from '@playwright/test';

// broken.spec.js: the action is commented out, yet the test still passes!
test('adding the grinder updates the cart total', async ({ page }) => {
  await page.goto('/catalog.html?mock=static&seed=42&autologin=1');
  const grinder = page.locator('article[data-sku="MRD-1003"]');
  // await grinder.getByRole('button', { name: 'Add to cart' }).click();
  await page.getByTestId('cart-total').textContent();
});

Output

Running 1 test using 1 worker

[1/1] broken.spec.js:3:1 · adding the grinder updates the cart total
  1 passed (2.4s)

Test yourself

Why does a test with no expect() assertions still report as '1 passed' in Playwright's test reporter?

Practised in the full course: the drill A read is not a check.

See the chapter

AI can write the test now. The job is knowing whether it checks anything. That's what the course teaches, one chapter at a time.

163 lessons · 226 drills · start free

How can a passing test check nothing?

Related questions