At rtCamp, our Quality Assurance team makes active contributions to the WordPress Core for both manual and automation testing processes. We endeavor to enhance the WordPress Core with our QA best practices, and the latest methodologies.
Recently, our QA team collaborated with the WordPress Core test team in a wide range of efforts, right from expediting “E2E migration from Jest to Playwright” to Manual testing for WordPress core tickets.
This post is to share our experience in WordPress Core testing and what we learned along the way.
Background
WordPress Core decided to deprecate two Gutenberg packages e2e-test-utils and e2e-tests to be replaced by the new Playwright packages e2e-test-utils-playwright and test/e2e.
We were curious about the reasoning behind the switch, and decided to get involved—from rtCamp’s QA team, Juhi participated in the initial discussions with the WordPress Core contributors.
We soon figured that there are some salient features that favor Playwright:
Auto-waiting mechanism
Auto-waiting mechanism waits for all the relevant checks to pass before performing the requested action. For example, if an element is taking some time to appear on the page, this mechanism allows the test to wait before performing any action.
Earlier, extra lines of code had to be written if we wanted the test to wait for some elements to appear, before performing an action.
Advanced selector support
Advanced selector support negates the need for complicated XPath selectors during execution of tests.
Extensive web browser support
Support for major web browsers like Chromium-based Chrome/Edge, WebKit-based Safari, and Quantum-based Firefox.
Dedicated test runner
The Playwright test runner functions with extensive configurations and contains several features for end-to-end testing such as utils, tooling, concurrency, reporting, assertions, and artifacts among others. These advanced configurations were not available in Jest, and we used to depend on third-party packages.
Trace Viewer
Playwright features a trace-viewer GUI tool, facilitating comprehensive debugging. It also optimizes the scripts’ execution time, as you get an overview of both passing and failing test cases. Traces get recorded and stored in a zip file. You can easily change the setting for trace-viewer in the test-configuration file in the root directory.
Debugging Tests
Debugging is a rather simple process in Playwright, providing more capabilities compared to other testing frameworks. It is possible to debug and record simultaneously, which helps in fixing the code or to identify the main cause of a failed test case. Debug and record feature was not available in Jest.
Customizability
Playwright can be integrated with other 3rd party NPM packages, extending the functionality of the framework.
It also supports integration with BDD (Behavior-driven development) using Cucumber – allowing tests to be written in a natural language format, easily understandable by both technical and non-technical stakeholders, making the tests more accessible.
Versatility
Playwright is a versatile framework which can fulfill most of the requirements of a QA team. Within a single framework, it enables managing all the aspects of the testing process that could be carried out with total automation. With Playwright, you can manage QA processes for:
- E2E testing
- API testing
- Visual testing
- A11Y (accessibility) testing
- Unit testing
Recently, we developed an API Playwright Utils framework, that provides a basic ready-to-use utils suite for test automation. Read about its implementation on our blog.
Our Contribution
Having learned the advantages, our team decided to contribute towards migrating Jest E2E tests to Playwright.
Making contributions to open-source software is a two-way process. By contributing to the migration process, we actually get better at effective documentation, working with very strict coding standards and gain a more in-depth understanding of the Playwright testing framework.
With the initial framework set up and comprehensive migration guidelines, our team was all set to facilitate the migration of test cases. You can get a glimpse of our team’s work in the following repositories:
- Migrate mentions tests to playwright
- Migrate Post Visibility Test to Playwright
- Migrate deprecated node matcher tests to playwright
- Migrate iframe-rendering test case
We effectively migrated the old Jest tests to the new Playwright tests and Utils for the WordPress Gutenberg repo, which resulted in accelerating the process. Until now, we have migrated over 25 test cases and working on new ones.
As migration to Playwright commenced, we simultaneously explored approaches to migrate our existing Jest-based WP E2E Framework. We also started a thread that could benefit 3rd parties currently using the old utils and mulling migration to Playwright.
Our conversation led to the ticket prerelease E2E Test Utils for Playwright, which has now been merged and the related NPM package released. However, before the package was released, we also submitted a patch to use these packages and began migrating the old test cases. Currently, our WP E2E framework comes ready with more than 25 WordPress test cases.
For manual testing, we helped triaging the tickets and reproducing the issues. We are happy to see that our test team has earned badges for making contributions.
QA expertise for WordPress
Our QA team has been instrumental in helping improve the code quality of the world’s leading CMS. We leverage our experience in Core contribution to offer QA services for enterprise-grade web applications.
Over time, we have built a full-fledged E2E testing suite that includes API testing, bug reporting, visual automation, accessibility, and performance testing. This helps us enable a bug-free web application, and a faster deployment cycle that leads to growth, consistency and predictable costs.
By contributing to Core testing objectives, we deepen our understanding of testing, and end up making our QA process even more robust and efficient.
Link: Quality Assurance Services | Checklist for cost-effective web accessibility
