Compare

PixelCI and the free tools you already run

Playwright’s screenshot assertions, jest-image-snapshot, BackstopJS, reg-suit and Loki are free and good at what they do. PixelCI compares any PNGs your tests write to a folder, whichever tool takes them. What changes is where the baselines live and how a change gets approved.

toHaveScreenshot() in your Playwright tests, with masks, animation freezing and pixel or ratio thresholds.

Baselines: PNGs committed next to the test, one per browser and platform (e.g. -chromium-darwin.png).

Approving a change: Run with --update-snapshots and commit the new PNGs; review happens in the pull request.

Playwright says to generate baselines in the same environment as CI, because rendering differs between platforms.

A Jest matcher that compares any image you give it, with pixelmatch or SSIM.

Baselines: PNGs in __image_snapshots__ next to the test, committed.

Approving a change: Run Jest with -u and commit.

A matcher only: capture, masking and review are up to you.

Captures configured URLs and scenarios with Playwright or Puppeteer, optionally in Docker, and writes an HTML report.

Baselines: Reference bitmaps in backstop_data, committed.

Approving a change: backstop approve copies the latest run into the references; commit them.

No GitHub or GitLab status integration; CI gets a JUnit report.

Compares a directory of images from any tool, publishes an HTML report, and comments on pull and merge requests.

Baselines: Stored in S3 or Google Cloud Storage per commit; a branch compares against the commit it branched from.

Approving a change: No approval step in the core; the GitHub plugin lets a reviewer turn the failed status green.

The closest open-source tool to PixelCI’s model: images in, baselines out of git.

Loki

MIT

Storybook stories rendered in Chrome (Docker or local) and on iOS and Android emulators.

Baselines: Reference images in .loki/reference, committed.

Approving a change: loki approve and commit.

Last release in August 2024.

Committed baselines

What committed baselines cost

  • Every approved change adds new PNGs to your repository history.
  • Baselines depend on the machine that rendered them; Playwright, for example, keeps one file per browser and platform and says to generate them in the same environment as CI.
  • When two branches change the same screenshot, Git can’t merge the two images: someone picks one and regenerates the other.
  • Approval is a commit. There’s no per-screen decision or reviewer comment to look back on.
PixelCI

What PixelCI adds

  • A review screen: each changed screen side by side with its baseline, approved or rejected one at a time, with a comment.
  • Baselines kept outside Git, per branch, with no PNGs in your history.
  • A change approved on one branch is recognised on every other branch it shows up on.
  • The CI job fails while changes need review and passes once they’re approved, on GitHub Actions and GitLab CI.
Switching

Write to a folder, upload the folder

Write your screenshots to a folder and upload the folder with the PixelCI CLI. Each file becomes a screen named after its path. If you use an assertion such as toHaveScreenshot(), which writes nothing when a test passes, switch it to page.screenshot({ path }).

// checkout.spec.ts
await page.screenshot({ path: 'screenshots/checkout.png', fullPage: true });
# GitHub Actions (needs permissions: id-token: write)
- uses: docker://ghcr.io/zyno-io/pixelci/cli:latest
  env:
    PIXELCI_API_URL: https://pixelci.example.com
    PIXELCI_APP_ID: <your-app-id>
    PIXELCI_IMAGES_PATH: ./screenshots

Setup guide

Price

Still free

The self-hosted edition is free for your organisation’s internal use. PixelCI Cloud has a free plan for 5,000 screenshots a month.

Keep your test tool. Change the review.

Write screenshots to a folder and point the PixelCI CLI at it.