Automating Accessibility Testing for a More Inclusive World

Accessibility testing, often referred to as a11y testing, has traditionally depended heavily on manual processes. However, the integration of automation is transforming our approach to achieving accessibility compliance. The term "a11y" is a numeronym for "accessibility," where the number 11 represents the count of letters between the "a" and the "y."

At Chase, our A11y efforts are driven by feedback from our customers with disabilities. Because we release a new version of the Chase Mobile® App every two weeks in addition to maintaining Chase.com without compromising accessibility, testing and automation are crucial.

Importance and Role of Accessibility Automation

Automated accessibility testing is crucial for several reasons. It has emerged as a powerful ally in the quest for digital accessibility and helps ensure digital content is inclusive and compliant with accessibility standards, such as Web Content Accessibility Guidelines (WCAG), and to reduce the risk of discrimination lawsuits. By leveraging automated tools and scripts, organizations can efficiently identify and rectify accessibility issues across various digital interfaces, including websites, mobile applications, and software platforms.

Test Accuracy and Precision

Automation can cover various aspects of accessibility, including keyboard navigation, semantic HTML structure, Accessible Rich Internet Applications (ARIA) attributes and more. This comprehensive coverage helps identify a wide range of accessibility issues. One of the primary advantages of accessibility automation is its ability to provide accurate and precise results. Automated tools can detect accessibility violations with a high degree of accuracy, offering detailed insights into compliance issues that might otherwise go unnoticed.

Benefits of Accessibility Automation

The benefits of a11y automation are vast. It saves time and resources, catches issues early in the development cycle, and provides consistent and scalable testing.

Our Technical Approach

Our approach to accessibility automation is grounded in creating a seamless and efficient process that ensures accessibility from the earliest stages of development. External accessibility testing tools and software, combined with our custom Unified Digital Framework (UDF) scripts, plays a critical role in this process by identifying both common and complex accessibility issues before they reach production.

In short, UDF scripts provide flexibility and depth in our accessibility testing, allowing us to go beyond out-of-the-box solutions and ensure our products meet both basic and advanced accessibility standards. They allow us to improve test coverage and ensure a more inclusive digital experience by developing customized checks for complex scenarios, dynamic content, and ARIA and enhanced semantic HTML testing.

By leveraging built-in accessibility rules, we can automate the detection of common accessibility issues such as missing document-title, empty-heading, html-has-lang, html-lang-valid, and meta-viewport without extensive manual testing. These are often the most frequent accessibility barriers that prevent users with disabilities from fully interacting with digital products. The built-in tool’s ability to check for these issues directly within our code allows us to fix them early and rapidly, contributing to a faster time-to-market and higher overall quality.

However, accessibility is more than just catching low-hanging fruit. That’s where our UDF scripts come in. These scripts allow us to extend the tool’s capabilities and address more nuanced accessibility concerns, such as ensuring smooth keyboard navigation and focus management, avoiding keyboard traps, and maintaining a clear semantic HTML structure. Our use of ARIA attributes ensures that dynamic content is properly announced to screen readers, making our web applications more inclusive.

This dual approach, combining the power of automated testing with the precision of customized UDF scripts, forms the backbone of our accessibility strategy. It allows us to confidently meet WCAG guidelines while improving the overall usability of our products, ensuring that accessibility is an integral part of our development lifecycle rather than an afterthought.

Challenges of Accessibility Automation

Despite its benefits, accessibility automation also presents unique challenges:

The Hybrid Approach: Automation and Manual Testing

To address these challenges, a hybrid approach combining automated testing and manual testing with expert audits is often the most effective strategy. Automation provides speed and scalability, while human intervention ensures accuracy and depth in accessibility evaluations.

A recent hybrid approach test scenario resulted in 52% of applicable checkpoints being tested using automation, with the remaining tested manually. As we continue to test, we’ll be able to evaluate our speed, scalability and effectiveness to increase our ratio of automated checkpoints.

Applying Automation Testing to Mobile Apps

For the Chase Mobile App, automated accessibility checks are powered to scan through hundreds of screens on both the Android and iOS apps focusing primarily on 5 WCAG checkpoints. A set of validations that may have previously required more than 5 hours of effort using several tools can now be performed within minutes, reducing our time to market significantly.

To achieve this, we leverage the capabilities of a third-party software development kit (SDK) through a single point integration on our UDF. This allows us to utilize and extend the tool capabilities across our entire suit of automated tests via a runtime command. WCAG, WCAG 2.0 level validations for Contrast, Accessibility Name, Focus Trap are automatically triggered as test scripts execute. Checkpoint validations are continuously and seamlessly performed as each test script flows from one screen to the other. A report is generated per test script, furnished with a screenshot and details that include the id of the element, nature and the severity of the non-compliance, and a detailed description of the WCAG checkpoint violation detected.

We are also able to reuse test scenarios depicting actual user stories that enable us to gauge accessibility of our apps from our customers’ point of view. Automation has provided us with an opportunity to build consistency in our testing practices.  

Future Directions

Based on progress achieved and the commitment to enhancing accessibility automation coverage, future directions for further improvements at Chase include:

By focusing on these future directions, the aim is to not only increase automation coverage but also enhance the overall effectiveness and impact of accessibility automation initiatives. This proactive approach contributes to creating more inclusive digital experiences for all users. 

Automation has revolutionized digital accessibility testing, offering efficiency, scalability, and enhanced coverage. While automation is not without its challenges, its integration with manual testing methodologies represents a holistic approach to achieving and maintaining accessibility compliance. As we embrace emerging technologies and methodologies, the goal of inclusive design and development remains within reach, enabling digital experiences to be accessible for all.

Like what you’re reading? Check out Next at Chase for more insights from one of banking’s most innovative organizations.