Playwright Regression and Smoke Testing in Action: Parallel Execution and Test Scaling

Anuradha Agarwal
Feb 25
12 min read

In the previous lesson, we automated a complete purchase workflow on our DemoShop test store using Playwright. One of the key improvements we introduced was the beforeAll hook, where we logged in once and saved the authenticated session using Playwright’s storageState.

This was an important step forward.

Instead of logging in before every test, Playwright reused the saved authentication state. This made our tests faster, cleaner, and more efficient. We also organized the workflow using test.step(), which gave us clear, structured reporting and made debugging easier.

At that point, we had a reliable end-to-end automation workflow.

So far, we have built a complete end-to-end Playwright test that performs a full business workflow:

open DemoShop
search for a product
apply a filter
add the product to the cart
complete checkout
verify the order in My Account
logout

This is a powerful test because it validates the application exactly the way a real user interacts with it.

At this stage, it might be assumed that adding more and more steps into this single test is the best approach.

But in real automation frameworks, this is not how test suites are designed.

Instead, automation is organized into two important categories:

Regression Tests: Small, Focused Validations

Regression tests verify individual features independently.

For example:

Does search return correct results?
Does the price filter work correctly?
Can a product be added to the cart?
Does the orders page display correctly?

Each regression test is small, focused, and easy to debug. If something fails, we immediately know which feature is broken.

Smoke Tests: Critical End-to-End Workflows

Smoke tests validate complete business workflows from start to finish.

For example:

Can a user successfully purchase a product?
Does checkout work correctly?
Does the order appear in the account?

These tests simulate real user journeys and ensure that the core functionality of the application is working.

Smoke tests are fewer in number but extremely important because they verify the system at a business level.

Effectively, this is our base version which we will be modifying step by step to align with goal of our current post

import { test, expect, BrowserContext } from "@playwright/test";

let webContext: BrowserContext;

// Test data
const baseURL = "https://qa-cart.com/";
const username = "anuradha.learn@gmail.com";
const password = "Play@1234#$";
const maxPrice = 25;
const storageStatePath = "state.json";

test.beforeAll(async ({ browser }) => {
  const myContext = await browser.newContext();
  const page = await myContext.newPage();

  await test.step("Authenticate once and save storageState", async () => {
    await page.goto(baseURL);

    await expect(page.getByRole("heading", { name: /login/i })).toBeVisible();

    await page.getByRole("textbox", { name: /username/i }).fill(username);
    await page.getByRole("textbox", { name: /password/i }).fill(password);
    await page.getByRole("button", { name: /log in/i }).click();

    await expect(page.getByRole("link", { name: /log out/i }).first()).toBeVisible();

    await myContext.storageState({ path: storageStatePath });
  });

  webContext = await browser.newContext({ storageState: storageStatePath });
  await myContext.close();
});

test.afterAll(async () => {
  await webContext?.close();
});

test("E2E: Search → Filter → Add to Cart → Checkout → Verify Order → Logout", async () => {
  const page = await webContext.newPage();

  // We'll capture these during the flow
  let productName = "";
  let orderId = "";

  await test.step("Open DemoShop", async () => {
    await page.goto(baseURL);
    await page.getByRole("link", { name: "DemoShop" }).click();
    await expect(page.getByRole("heading", { name: "DemoShop" })).toBeVisible();
  });

  await test.step('Search products with keyword "organic"', async () => {
    await page.getByRole("searchbox", { name: /search/i }).fill("organic");
    await page.getByRole("button", { name: /search/i }).click();

    await expect(page.getByRole("heading", { name: /search results.*organic/i })).toBeVisible();
  });

  await test.step(`Apply max price filter (${maxPrice}) and confirm filter applied`, async () => {
    const maxPriceInput = page.getByRole("textbox", { name: /maximum price/i });

    await maxPriceInput.clear();
    await maxPriceInput.fill(String(maxPrice));
    await maxPriceInput.press("Enter");

    await expect(page.getByRole("button", { name: /remove price up to/i })).toBeVisible();
    await expect(page.getByRole("alert")).toContainText(/\d+ results/i);
  });

  await test.step(`Validate all listed products have price <= ${maxPrice}`, async () => {
    const productGrid = page.locator("ul.products");
    await expect(productGrid).toBeVisible();

    const products = productGrid.locator(":scope > li.product");
    const countItems = await products.count();
    expect(countItems).toBeGreaterThan(0);

    for (let i = 0; i < countItems; i++) {
      const product = products.nth(i);

      const priceText = await product.locator("span.price bdi").first().innerText();
      const price = parseFloat(priceText.replace("$", "").trim());

      expect(price).toBeLessThanOrEqual(maxPrice);
    }
  });

  await test.step("Add first product to cart and capture selected product name", async () => {
    const productGrid = page.locator("ul.products");
    const products = productGrid.locator(":scope > li.product");
    const firstProduct = products.first();

    productName = (await firstProduct.locator(".woocommerce-loop-product__title").innerText()).trim();
    expect(productName).toBeTruthy();

    const addBtn = firstProduct.locator("a.add_to_cart_button");
    await addBtn.click();

    // Wait for state changes
    await expect(addBtn).not.toHaveClass(/loading/, { timeout: 15000 });
    await expect(addBtn).toHaveClass(/added/, { timeout: 15000 });

    const cartLink = page.getByRole("link", { name: /^View Shopping Cart/i });
    await expect(cartLink).toBeVisible();
  });

  await test.step("Open cart and verify selected product is present", async () => {
    await page.goto("https://qa-cart.com/mycart/");
    await expect(page).toHaveURL(/mycart/);

    const cartRows = page.locator("table.cart tr.cart_item");
    await expect(cartRows.first()).toBeVisible();

    const countRows = await cartRows.count();
    expect(countRows).toBeGreaterThan(0);

    let productFound = false;
    for (let i = 0; i < countRows; i++) {
      const cartProductName = (await cartRows.nth(i).locator("td.product-name").innerText()).trim();
      if (cartProductName.includes(productName)) {
        productFound = true;
        break;
      }
    }
    expect(productFound).toBeTruthy();
  });

  await test.step("Proceed to checkout and place the order", async () => {
    const checkoutLink = page.getByRole("link", { name: /proceed to checkout/i });
    await checkoutLink.scrollIntoViewIfNeeded();
    await expect(checkoutLink).toBeVisible();
    await checkoutLink.click();

    await expect(page).toHaveURL(/checkout/i, { timeout: 15000 });
    await expect(page.getByRole("heading", { name: /checkout/i })).toBeVisible({ timeout: 15000 });
    await expect(page.locator("form.checkout")).toBeVisible({ timeout: 15000 });

    const placeOrder = page.locator("#place_order");
    await expect(placeOrder).toBeVisible({ timeout: 15000 });
    await placeOrder.scrollIntoViewIfNeeded();
    await placeOrder.click();

    await expect(page.getByText(/your order has been received/i)).toBeVisible({ timeout: 15000 });

    orderId = (await page.locator("ul.order_details>li.order strong").innerText()).trim();
    expect(orderId).toBeTruthy();
  });

  await test.step("Go to My Account → Orders and verify the order exists", async () => {
    await page.getByRole("link", { name: /my account/i }).click();
    await expect(page.getByRole("heading", { name: /my account/i })).toBeVisible();

    await page.getByRole("link", { name: /^orders$/i }).first().click();
    await expect(page).toHaveURL(/orders/i);

    const ordersTable = page.locator("table.woocommerce-orders-table");
    await expect(ordersTable).toBeVisible();

    // Find and open matching order
    const orderRowLink = ordersTable.getByRole("link", { name: `View order number ${orderId}` });
    await expect(orderRowLink).toBeVisible();
    await orderRowLink.click();

    await expect(page).toHaveURL(new RegExp(`view-order.*${orderId}`), { timeout: 15000 });

    await expect(
      page.getByRole("heading", { name: new RegExp(`Order\\s*#${orderId}`, "i") })
    ).toBeVisible();

    await expect(page.getByRole("heading", { name: `Order #${orderId}` })).toBeVisible();
  });

  await test.step("Logout", async () => {
    await page.getByRole("link", { name: /my account/i }).click();
    await expect(page.getByRole("heading", { name: /my account/i })).toBeVisible();

    const logoutLink = page.getByRole("link", { name: /log out/i }).first();
    await expect(logoutLink).toBeVisible();
    await logoutLink.click();

    // Optional: verify we are back on login page
    await expect(page.getByRole("heading", { name: /login/i })).toBeVisible();
  });

  await page.close();
});

Extract “new authenticated page”

Add:

async function newAuthedPage() { 
 return await webContext.newPage();
}

Then inside your test, replace:

const page = await webContext.newPage();

with:

const page = await newAuthedPage();

Our existing E2E test is a perfect example of a Smoke test.

Instead of leaving it as a standalone test, we now organize it inside a Smoke group.

This makes the purpose of the test clear and prepares our suite for future expansion.

As we add more regression tests, Playwright will be able to run them efficiently while keeping the critical Smoke workflow separate and easy to monitor.

Wrap the Existing E2E Test Inside a Smoke Group

We use Playwright’s test.describe() function to create a logical group.

test.describe("Smoke", () => { 
 test("E2E: Search → Filter → Add to Cart → Checkout → Verify Order → Logout", async () => {  
  // existing full workflow remains unchanged  });
}
);

The test.describe() block groups related tests together.

In this case, it clearly indicates that this test belongs to the Smoke suite.

This improves:

test organization
report readability
execution control
framework scalability

When viewing Playwright reports, the structure becomes clearer:

Smoke 
└──
 E2E: Search → Filter → Add to Cart → Checkout → Verify Order → Logout

At this point, we are not changing the test itself. We are simply organizing it properly.

This prepares our automation suite for the next step, where we will add regression tests alongside the smoke workflow, creating a structured and scalable Playwright test framework.

Adding Regression Tests and Marking Them for Flexible Execution

In the previous step, we organized our complete end-to-end workflow inside a Smoke group. That test validates the full business journey—from searching for a product to placing an order and verifying it in the account.

This Smoke test gives us confidence that the most critical user workflow is working correctly.

However, relying only on a full end-to-end test is not enough for real automation suites.

When something fails in a long workflow, it becomes difficult to immediately identify the exact cause of the failure.

For example, if the Smoke test fails, we might ask:

Did search stop working?
Did the filter stop applying correctly?
Did the Add to Cart functionality break?
Or did checkout fail?

To answer these questions quickly, we introduce Regression tests.

Instead of testing everything in one long workflow, regression tests isolate and verify specific functionality.

For example, we create separate tests for:

Shop and search validation
Filter validation
Cart add and cart verification

This makes debugging easier, because if a regression test fails, we immediately know which feature is broken.

Each regression test should follow a simple structure:

Create a fresh page
Navigate to the required starting point
Perform a focused validation
Close the page

This ensures that regression tests remain independent, clean, and reliable.

Example: Regression Test for Search Validation

Example: Regression Test for Filter Validation

test(`@regression Filter: max price (${maxPrice}) limits product prices`, async () => {
    const page = await newAuthedPage();

    await page.goto(`${baseURL}/demoshop/`);

    await page.getByRole("searchbox", { name: /search/i }).fill("organic");
    await page.getByRole("button", { name: /search/i }).click();
    await expect(page.getByRole("heading", { name: /search results.*organic/i })).toBeVisible();

    const maxPriceInput = page.getByRole("textbox", { name: /maximum price/i });
    await maxPriceInput.clear();
    await maxPriceInput.fill(String(maxPrice));
    await maxPriceInput.press("Enter");

    await expect(page.getByRole("button", { name: /remove price up to/i })).toBeVisible();
    await expect(page.getByRole("alert")).toContainText(/\d+ results/i);

    const productGrid = page.locator("ul.products");
    await expect(productGrid).toBeVisible();

    const products = productGrid.locator(":scope > li.product");
    const count = await products.count();
    expect(count).toBeGreaterThan(0);

    for (let i = 0; i < count; i++) {
      const priceText = await products.nth(i).locator("span.price bdi").first().innerText();
      const price = parseFloat(priceText.replace("$", "").trim());
      expect(price).toBeLessThanOrEqual(maxPrice);
    }

    await page.close();
  });

This test verifies only the filter functionality.

Example: Regression Test for Cart Validation

test("@regression Cart: add first product and verify it appears in cart", async () => {
      const page = await newAuthedPage();
  
      await page.goto(`${baseURL}/shop/`);
  
      const productGrid = page.locator("ul.products");
      await expect(productGrid).toBeVisible();
  
      const firstProduct = productGrid.locator(":scope > li.product").first();
      const productName = (await firstProduct.locator(".woocommerce-loop-product__title").innerText()).trim();
      expect(productName).toBeTruthy();
  
      const addBtn = firstProduct.locator("a.add_to_cart_button");
      await addBtn.click();
  
      await expect(addBtn).not.toHaveClass(/loading/, { timeout: 15000 });
      await expect(addBtn).toHaveClass(/added/, { timeout: 15000 });
  
      await page.goto(`${baseURL}/mycart/`);
      await expect(page).toHaveURL(/mycart/);
  
      const cartRows = page.locator("table.cart tr.cart_item");
      await expect(cartRows.first()).toBeVisible();
  
      const countRows = await cartRows.count();
      expect(countRows).toBeGreaterThan(0);
  
      let found = false;
      for (let i = 0; i < countRows; i++) {
        const cartProductName = (await cartRows.nth(i).locator("td.product-name").innerText()).trim();
        if (cartProductName.includes(productName)) {
          found = true;
          break;
        }
      }
      expect(found).toBeTruthy();
  
      await page.close();
    });
  });

This test verifies only the cart functionality.

Marking Tests Using @regression and @smoke

You may have noticed that we added @regression in the test title:

test("@regression Cart: add product and verify cart row", async () => {

This is called test marking or tagging.

Marking tests allows us to run specific subsets of tests.

For example:

Smoke tests validate critical workflows:

test.describe("Smoke", () => {  test("@smoke E2E purchase workflow", async () => {    // full business workflow  });});

Regression tests validate individual features:

test("@regression Filter validation", async () => {

How to Run Smoke and Regression Tests Separately

Playwright provides the --grep option to run specific tests based on tags.

Run only Smoke tests:

npx playwright test --grep "@smoke"

Run only Regression tests:

npx playwright test --grep "@regression"

Run all tests:

npx playwright test

Parallel vs Serial Execution in Playwright - Making Tests Faster and More Reliable

So far, we have organized our test suite into two categories:

Regression tests, which validate individual features like search, filter, and cart
Smoke tests, which validate the complete business workflow from start to finish

At this point, an important question arises:

How should these tests run — one after another, or at the same time?

This brings us to two important execution modes in Playwright:

Parallel execution
Serial execution

Understanding when to use each is essential for building fast and reliable automation suites.

Parallel Execution: Faster Execution for Independent Tests

Regression tests are independent. Each test creates its own page, navigates to DemoShop, and performs a focused validation.

For example:

Search test validates only search
Filter test validates only filtering
Cart test validates only cart functionality

None of these tests depends on each other.

Because they are independent, Playwright can safely run them at the same time using parallel execution.

This significantly improves execution speed.

Instead of running tests like this:

Search Test → Filter Test → Cart Test

Playwright can run them like this:

Search Test
Filter Test    → running simultaneously
Cart Test

This makes regression testing much faster, especially when the suite grows.

How to Enable Parallel Execution

By default test present within the same file run in serial mode. Note that test files will run in parallel. Playwright allows parallel execution using test.describe.configure().

This tells Playwright that tests inside this group are safe to run in parallel.

Serial Execution: Stability for Dependent Workflows

Now let’s consider the Smoke test.

Our Smoke test performs a complete workflow:

open DemoShop
search product
apply filter
add to cart
place order
verify order

These steps are dependent on each other.

For example:

You cannot verify an order before placing it
You cannot place an order before adding a product to the cart

This workflow must run in sequence.

Fortunately, since our Smoke workflow is written as a single test, Playwright already executes it in order.

When to Use Serial Execution Explicitly

Later, as your framework grows, you may choose to split the Smoke workflow into multiple dependent tests.

Example:

test.describe.serial("Smoke workflow", () => {  test("Add product to cart", async () => { });  test("Checkout", async () => { });  test("Verify order", async () => { });});

The .serial mode ensures these tests run in strict order.

Here is an example parallel run

This means:

--grep "@regression" → run only tests tagged with @regression
--workers=3 → run tests in parallel using 3 worker processes

The key part here is workers = parallel execution.

What “workers” means in Playwright

A worker is a separate Node.js process that runs tests independently.

So when you specify:

--workers=3

Playwright creates 3 parallel test runners.

Each worker:

launches its own browser instance
runs its assigned tests independently
does not share memory with other workers

Without parallel execution

Suppose you have 6 regression tests:

Test 1 → 5 sec
Test 2 → 5 sec
Test 3 → 5 sec
Test 4 → 5 sec
Test 5 → 5 sec
Test 6 → 5 sec

Sequential execution (default workers=1):

Total time = 30 seconds

Example: With parallel execution (--workers=3)

Playwright splits tests across workers:

Worker 1 → Test 1, Test 4
Worker 2 → Test 2, Test 5
Worker 3 → Test 3, Test 6

Each worker runs in parallel:

Total time ≈ 10 seconds

Huge performance improvement.

Each worker has:

its own browser
its own context
its own test environment

Parallel Execution in Action: Worker Distribution Across Test Files

So far, our regression suite focused on DemoShop workflows such as search, filtering, and cart functionality.

Now let’s add another independent workflow — Account Edit — as a separate test file.

This reflects how real-world automation frameworks are structured, where different features are organized into different spec files.

Example file structure:

tests/
│
├── account-edit.spec.ts
├── demoshop-regression.spec.ts

The Account Edit test file contains validations related only to account management:

import { test, expect, BrowserContext } from "@playwright/test";

// --------------------
// Test data / config
// --------------------
const baseURL = "https://qa-cart.com";
const username = "anuradha.learn@gmail.com";
const password = "Play@1234#$";

const storageStatePath = "state.account.json";
let webContext: BrowserContext;

// --------------------
// Auth once (storageState) + reusable context
// --------------------
test.beforeAll(async ({ browser }) => {
  const loginContext = await browser.newContext();
  const page = await loginContext.newPage();

  await test.step("Authenticate once and save storageState", async () => {
        
      await page.goto(`${baseURL}`);
      await expect(page.getByRole("heading", { name: /login/i })).toBeVisible();
    
      await page.getByRole("textbox", { name: /username/i }).fill(username);
      await page.getByRole("textbox", { name: /password/i }).fill(password);
      await page.getByRole("button", { name: /log in/i }).click();
    
      await expect(page.getByRole("link", { name: /log out/i }).first()).toBeVisible();
    
    
    await loginContext.storageState({ path: storageStatePath });
  });

  webContext = await browser.newContext({ storageState: storageStatePath });
  await loginContext.close();
});

test.afterAll(async () => {
  await webContext?.close();
});

// Utility: always get a fresh page per test
async function newAuthedPage() {
  return await webContext.newPage();
}

// =====================================================
// Regression: Account Edit Workflow
// =====================================================
test.describe("Regression: Account Management", () => {

  test("@regression Account Edit: update fields and verify persistence", async () => {
    const page = await newAuthedPage();

    await test.step("Open My Account and go to Account details", async () => {
      await page.goto(baseURL);

      // We are already authenticated because of storageState
      await expect(page.getByRole("link", { name: /log out/i }).first()).toBeVisible();

      await page.getByRole("link", { name: "Account details", exact: true }).click();
      await expect(page).toHaveURL(/edit-account/);
    });

    await test.step("Update account fields and save changes", async () => {
      await page.getByRole("radio", { name: "Intermediate" }).check();
      await page.getByRole("checkbox", { name: "Playwright Automation" }).check();

      await page.getByRole("textbox", { name: "Preferred Start Date" }).fill("2026-04-15");

      await page.getByLabel("Preferred Automation Tool").selectOption("selenium");

      await page.getByRole("checkbox", { name: "Subscribe to AI Testing" }).check();

      await page.getByRole("button", { name: /save changes/i }).click();

      await expect(page.getByRole("alert")).toContainText(
        "Account details changed successfully."
      );
    });

    await test.step("Reload and verify values persisted", async () => {
      // Option A: reload the same page (simple and reliable)
      await page.reload();
      await expect(page).toHaveURL(/edit-account/);

      await expect(page.getByRole("radio", { name: "Intermediate" })).toBeChecked();
      await expect(page.getByRole("checkbox", { name: "Playwright Automation" })).toBeChecked();
      await expect(page.getByRole("textbox", { name: "Preferred Start Date" })).toHaveValue("2026-04-15");
      await expect(page.getByLabel("Preferred Automation Tool")).toHaveValue("selenium");
      await expect(page.getByRole("checkbox", { name: "Subscribe to AI Testing" })).toBeChecked();
    });

    await page.close();
  });

});

This test is completely independent of DemoShop workflows.

How Playwright Executes Separate Test Files in Parallel

Now let’s run regression tests using multiple workers:

npx playwright test --grep "@regression" --headed

Playwright distributes test files across workers automatically.

Example worker distribution:

Worker 1 → account-edit.spec.ts

Worker 2, Worker 3, Worker 4 → demoshop-regression.spec.ts - 3 regression

Each worker:

runs its assigned test file
launches its own browser
executes independently

You can visually observe multiple browser windows opening simultaneously — one per worker.

This confirms that Playwright is executing test files in parallel.

It is important to understand this key behaviour:

Tests inside the same file run in serial mode by default
Test files run in parallel across workers

This is why organizing tests into separate files improves parallel execution efficiency.

Login Now Runs Multiple Times

Currently, login is implemented inside each test file using beforeAll.

Example:

When tests run across multiple files, each worker executes this setup independently.

Example:

Worker 1 → login (account-edit.spec.ts)

Worker 2 → login (demoshop-regression.spec.ts)

Worker 3 → login (cart.spec.ts)

Worker 4 → login (checkout.spec.ts)

Authentication is repeated unnecessarily.

So far, we have used beforeAll to authenticate once and reuse the session using storageState. This worked well when tests were organized in a single file.

However, as our framework grows and tests are split across multiple files and executed using multiple workers, authentication starts happening multiple times — once per worker.

This is inefficient and unnecessary.

What we really want is a framework-level mechanism that allows us to authenticate once and reuse that session across all tests, files, and workers.

This is exactly what Playwright’s globalSetup provides. This would be covered in our next post.

ANURADHA AGARWAL

Playwright Regression and Smoke Testing in Action: Parallel Execution and Test Scaling

Regression Tests: Small, Focused Validations

Smoke Tests: Critical End-to-End Workflows

Extract “new authenticated page”

Wrap the Existing E2E Test Inside a Smoke Group

Adding Regression Tests and Marking Them for Flexible Execution

Example: Regression Test for Search Validation

Example: Regression Test for Filter Validation

Example: Regression Test for Cart Validation

Marking Tests Using @regression and @smoke

How to Run Smoke and Regression Tests Separately

Parallel vs Serial Execution in Playwright - Making Tests Faster and More Reliable

Parallel Execution: Faster Execution for Independent Tests

How to Enable Parallel Execution

Serial Execution: Stability for Dependent Workflows

When to Use Serial Execution Explicitly

What “workers” means in Playwright

Without parallel execution

Example: With parallel execution (--workers=3)

Parallel Execution in Action: Worker Distribution Across Test Files

How Playwright Executes Separate Test Files in Parallel

Login Now Runs Multiple Times

Recent Posts

Comments

Never Miss a Post. Subscribe Now!