In chapter 5, we automate design system testing to prevent UI bugs. This chapter dives into what characteristics of UI components warrant testing and potential pitfalls to avoid. We researched professional teams at Wave, BBC, and Salesforce to land on a test strategy that balances comprehensive coverage, straightforward setup, and low maintenance.
Fundamentals of UI component testing
Before we begin, let’s figure out what makes sense to test. Design systems are composed of UI components. Each UI component includes stories (permutations) that describe the intended look and feel given a set of inputs (props). Stories are then rendered by a browser or device for the end-user.
Whoa! As you can see, one component contains many states. Multiply the states by the number of design system components and you can see why keeping track of them all is a Sisyphean task. In reality, it’s unsustainable to review each experience by hand, especially as the design system grows.
All the more reason to set up automated testing now to save work in the future.
Prepare to test
I surveyed four front-end teams in a previous article about professional Storybook workflows. They agreed on these best practices for writing stories to make testing easy and comprehensive.
Design Systems for Developers is a guide for professional developers based on how the smartest teams engineer design systems at scale—like Shopify, Airbnb, and IBM. Read the rest of the guide here.
Articulate supported component states as stories to clarify which combinations of inputs yields a given state. Ruthlessly omit unsupported states to reduce noise.
Render components consistently to mitigate variability that can be triggered by randomized (Math.random) or relative (Date.now) inputs.
“The best kind of stories allow you to visualize all of the states your component could experience in the wild” – Tim Hingston, Tech lead at Apollo GraphQL
Visual test appearance
Design systems contain presentational UI components, which are inherently visual. Visual tests validate the visual aspects of the rendered UI.
Visual tests capture an image of every UI component in a consistent browser environment. New screenshots are automatically compared to previously accepted baseline screenshots. When there are visual differences, you get notified.
If you’re building a modern UI, visual testing saves your front-end team from time-consuming manual review and prevents expensive UI regressions. We’ll demo visual testing using Chromatic, an industrial-grade service by the Storybook maintainers.
First, go to ChromaticQA.com and sign up with your GitHub account.
From there, choose your design system repo. Behind the scenes, this will sync access permissions and instrument the PR checks.
Install the storybook-chromatic package via npm.
Make sure you import Chromatic in your Storybook configuration. Your
.storybook/config.js file should look like this:
Open up your command line and navigate to the
design-system directory. Then run your first test to establish your visual test baselines (you’ll need to use the app code that Chromatic supplies on the website).
Chromatic captured a baseline image of every story! Subsequent test runs will capture new images and compare them against these baselines. See how that works by tweaking a UI component and saving it. Go to the global styles (
src/shared/styles.js) and increase the font size.
Run the test command again.
Yikes! That small tweak resulted in a flood of UI changes.
Visual testing helps identify UI changes in Storybook. Review the changes to confirm whether they’re intentional (improvements) or unintentional (bugs). If you’re fond of the new font-size, go ahead and accept the changes and commit to git. Or perhaps the changes are too ostentatious, go ahead and undo them.
Create your own design system using InVision’s Design System Manager.
Let’s add visual testing to the continuous integration job. Open
.circleci/config.yml and add the test command.
git commit. Congratulations you just set up visual testing in CI!
Unit test functionality
Unit tests verify whether the UI code returns the correct output given a controlled input. They live alongside the component and help you validate specific functionality.
Everything is a component in modern view layers like React, Vue, and Angular. Components encapsulate diverse functionality from modest buttons to elaborate date pickers. The more intricate a component, the trickier it becomes to capture nuances using visual testing alone. That’s why we need unit tests.
For instance, our Link component is a little complicated when combined with systems that generate link URLs (“LinkWrappers” in ReactRouter, Gatsby, or Next.js). A mistake in the implementation can lead to links without a valid href value.
Visually, it isn’t possible to see if the
href attribute is there and points to the right location, which is why a unit test can be appropriate to avoid regressions.
Unit testing hrefs
Let’s add a unit test for our
Link component. create-react-app has set up a unit test environment for us already, so we can simply create a file
We can run the above unit test as part of our
yarn test command.
Earlier we configured our Circle config.js file to run
yarn test on every commit. Our contributors will now benefit from this unit test. The Link component will be robust to regressions.
Note: Watch out for too many unit tests which can make updates cumbersome. We recommend unit testing design systems in moderation.
“Our enhanced automated test suite has empowered our design systems team to move faster with more confidence.”
Dan Green-Leipciger, Senior software engineer at Wave
“Accessibility means all people, including those with disabilities, can understand, navigate, and interact with your app… Online [examples include] alternative ways to access content such as using the tab key and a screen reader to traverse a site,” writes developer Alex Wilson from T.Rowe Price.
Disabilities affect 15% of the population according to the World Health Organization. Design systems have an outsized impact on accessibility because they contain the building blocks of user interfaces. Improving accessibility of just one component means every instance of that component across your company benefits.
Get a head start on inclusive UI with Storybook’s Accessibility addon, a tool for verifying web accessibility standards (WCAG) in realtime.
Register the addon in
And add the
withA11y decorator to our
Once installed, you’ll see a new “Accessibility” tab in the Storybook addons panel.
This shows you accessibility levels of DOM elements (violations and passes). Click the “highlight results” checkbox to visualize violations in situ with the UI component.
From here, follow the addon’s accessibility recommendations.
Other testing strategies
Paradoxically, tests can save time but also bog down development velocity with maintenance. Be judicious about testing the right things – not everything. Even though software development has many test strategies, we discovered the hard way that some aren’t suited for design systems.
Snapshot tests (Jest)
This technique captures the code output of UI components and compares it to previous versions. Testing UI component markup ends up testing implementation details (code), not what the user experiences in the browser.
Diffing code snapshots is unpredictable and prone to false positives. At the component level, code snapshotting doesn’t account for global changes like design tokens, CSS, and 3rd party API updates (web fonts, Stripe forms, Google Maps, etc.). In practice, developers resort to “approving all” or ignoring snapshot tests altogether.
“Most component snapshot tests are just a worse version of screenshot tests. Test your outputs. Snapshot what gets rendered, not the underlying (volatile!) markup.”
Mark Dalgliesh, Front-end infrastructure at SEEK, CSS modules creator
End-to-end tests (Selenium, Cypress)
End-to-end tests traverse the component DOM to simulate the user flow. They’re best suited for verifying app flows like the signup or checkout process. The more complex functionality the more useful this testing strategy.
Design systems contain atomic components with relatively simple functionality. Validating user flows are often overkill for this task because the tests are time-consuming to create and brittle to maintain. However, in rare situations, components may benefit from end-to-end tests. For instance, validating complex UIs like datepickers or self-contained payment forms.
Drive adoption with documentation
A design system is not complete with tests alone. Since design systems serve stakeholders from across the organization, we need to teach others how to get the most from our well-tested UI components.
Continue reading Design Systems for Developers here.