I would like to test some apis which are tied directly into browser and not necessary have UI representation.

Simpliest test beeing something like:

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  const img = new Image();
  const asset = new DynamicAsset(img);

  expect(asset.element()).toEqual(img);
});

When using cypress, all the tests are run in browser context. That would work.

But in playwright following error occurs:

ReferenceError: Image is not defined

    at file:///Users/tomche/development/frontendara/amandes/e2e/assets/Dynamic.spec.js:6:15

And if using page.evaluate like:

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  await page.evaluate(() => {
    const img = new Image();
    const asset = new DynamicAsset(img);

    expect(asset.element()).toEqual(img);
  });
});

Error is:

page.evaluate: ReferenceError: expect is not defined

Which is happening since code evaluated in browser context doesn't have imports from upper scope.

Documentation seems to mention only things which are serialisable, but this code needs plex things to be verified which are not really mockable in node or serializable.

So is there a way to run assertions inside the browser similar to cypress/karma etc.?

I would like to test some apis which are tied directly into browser and not necessary have UI representation.

Simpliest test beeing something like:

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  const img = new Image();
  const asset = new DynamicAsset(img);

  expect(asset.element()).toEqual(img);
});

When using cypress, all the tests are run in browser context. That would work.

But in playwright following error occurs:

ReferenceError: Image is not defined

    at file:///Users/tomche/development/frontendara/amandes/e2e/assets/Dynamic.spec.js:6:15

And if using page.evaluate like:

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  await page.evaluate(() => {
    const img = new Image();
    const asset = new DynamicAsset(img);

    expect(asset.element()).toEqual(img);
  });
});

Error is:

page.evaluate: ReferenceError: expect is not defined

Which is happening since code evaluated in browser context doesn't have imports from upper scope.

Documentation seems to mention only things which are serialisable, but this code needs plex things to be verified which are not really mockable in node or serializable.

So is there a way to run assertions inside the browser similar to cypress/karma etc.?

• javascript
• testing
• cypress
• playwright

Share Follow edited Apr 20, 2022 at 7:17 T.Chmelevskij asked Mar 27, 2022 at 4:01 T.ChmelevskijT.Chmelevskij 2,1391919 silver badges3131 bronze badges 1

• Have you tried dynamic imports? Just spitballing... – Dois Commented Apr 19, 2022 at 15:38

Add a ment  | 

3 Answers 3

Sorted by: Reset to default 5

Pass expect into page.evaluate()

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  await page.evaluate((expect) => {
    const img = new Image();
    const asset = new DynamicAsset(img);

    expect(asset.element()).toEqual(img);
  }, expect);
});

Running full test code as updated, can get expect to work outside of evaluate()

(But note I have a different error message from original)

import { test, expect } from "@playwright/test";

test("creates Image element", async ({ page }) => {
  await page.goto(`http://localhost:3333/e2e/assets/Dynamic.spec.html`);

  const result = await page.evaluate(() => {
    const img = new Image();
    const asset = new DynamicAsset(img);
    return {img, asset}
  });

  expect(result.asset.element()).toEqual(result.img);
});

The golden rule of Playwright, Puppeteer and other backend-based browser automation libraries is: keep browser stuff in the browser, and keep backend stuff in the backend. The only way to move data back and forth between the browser process and the backend process is with serialization and deserialization (stringification and destringification if you prefer) over a network socket.

For OP's use case, the existing answers won't work because they attempt to send or return plex, non-serializable objects as strings through a network socket.

In the "return browser objects back to Node" solution, even if you could serialize the objects and reconstruct them in Node, identities would no longer match. Even if the identities did match, the whole point of testing is to deal with the JS running in a website in the browser. If it was a test you could perform using Node objects, there'd be little point to using Playwright. A simple Jest unit test in pure Node would be the way to go.

In the "pass a Playwright assertion into the browser", even if you could serialize everything the object would need to have to report an error back to Node, which it's not designed to do.

The correct approach to the scenario at hand is to perform the equality check in the browser, where the objects live, then return a simple, serializable boolean back to Node for the assertion:

import {expect, test} from "@playwright/test"; // ^1.39.0

test("creates Image element", async ({page}) => {
  await page.goto("http://localhost:3333/e2e/assets/Dynamic.spec.html");

  const isAssetElementEqualToImg = await page.evaluate(() => {
    const img = new Image();
    const asset = new DynamicAsset(img);
    return asset.element() === img;
  });
  expect(isAssetElementEqualToImg).toBe(true);
});

A custom error message can be useful for improving the readability of the assertion failure message:

test("creates Image element", async ({page}) => {
  await page.goto("http://localhost:3333/e2e/assets/Dynamic.spec.html");

  const eq = await page.evaluate(() => {
    const img = new Image();
    const asset = new DynamicAsset(img);
    return asset.element() === img;
  });
  expect(eq, "asset.element() === img was false").toBe(true);
});

This is fine for OP's specific scenario, since the predicate can be determined synchronously, but for other tests that involve real-time asynchronous work, you may want to poll or use a custom matcher to wait for the assertion to pass.

Here's a minimal example showing that this answer gives a false positive, as it treats any Node references as identical:

import {expect, test} from "@playwright/test";

const html = `<!DOCTYPE html>
<html>
<body>
<script>
img = new Image();
para = document.createElement("p");
</script>
</body>
</html>`;

// incorrect, trying to return Nodes from `evaluate`:
test("fail: an image should not be equal to a paragraph", async ({page}) => {
  test.fail(); // this test should fail but doesn't
  await page.setContent(html);
  const {img, para} = await page.evaluate(() => {
    return {img, para};
  });
  await expect(img).toEqual(para);
});

// the correct approach, paring in the browser environment:
test("an image should not be equal to a paragraph", async ({page}) => {
  await page.setContent(html);
  const eq = await page.evaluate(() => img === para);
  expect(eq).toBe(false);
});