By Max Schmitt, Published on 10/14/2020
End-to-end tests are important to ensure that there are no regressions in the functionality of a web application over time. As part of a modern development workflow, we would want to run them automatically in the CI before every code merge to the codebase (e.g. GitHub pull request) and deploy.
An end-to-end test is usually a programmatically executed real user flow in the app, typically executed in the cloud on headless browser instances as part of CI process. End-to-end tests help us to keep our app working and the users satisfied.
On this blog post, we will focus on Playwright, a Node.js library which makes it possible to automate Chromium, Firefox, and WebKit with a single API and Root Cause, a library to help us troubleshoot end-to-end tests.
One of the challenges of creating, running and maintaining end-to-end tests is that once changes are implemented, tests might fail and finding the root cause of the failure can be difficult. For this scenario, Testim has created an Open Source tool called Root Cause, that makes it easy to debug and analyze failures with a central interface even when they fail on the CI. It focuses on the following core needs:
For Jest, you only have to install it with
npm install @testim/root-cause-jest from NPM and adjust your main Jest configuration, for a full setup guide, see the official documentation on help.testim.io.
Once it's successfully integrated you can continue as normal with Playwright. So you can interact with pages to navigate to sites, click on elements, or call your custom logic. Root Cause internally will then record it in the background to provide a first-class debug experience if they are failing.
The results which Root Cause is recording gets saved automatically on your disk. You can view them by running
npx root-cause ls to list them and
npx root-cause show <result-id> to view a selected report. This will open the interface to see all the tests which were running, stack traces if there are failing tests, and additional metadata for network traffic, pages, or console as you can see here:
Root Cause viewer is also available as a hosted solution which gives you the flexibility of uploading your test results from your CI provider (e.g. GitHub Actions, or Travis CI) to them and act as a central interface of your project. This automatically will analyze them if multiple are failing for the same reason or give you statistically-based insights.
Root Cause is a great way to simplify troubleshooting your Playwright tests. You can get started using the OSS version on GitHub. For more information on the hosted Root Cause solution on Testim.io which can be used for free for most use cases. They also provide an example showcase repository on GitHub which demonstrates other integrations like with Mocha.
The Open Source project is hosted on GitHub: github.com/testimio/root-cause.