Manual testing should be part of your web accessibility strategy

Are you manually checking your website for accessibility issues? If not, you might be overlooking critical accessibility issues that affect your compliance with the Web Content Accessibility Guidelines (WCAG). There is an ongoing discussion in the field of accessibility on how much automated testing can cover, but most accessibility experts agree that automated testing alone only covers 30 to 40 % of accessibility issues.

Including manual testing as part of your web accessibility strategy creates a more balanced and comprehensive approach to accessibility compliance when paired with reliable automated testing. It is tempting to shy away from manual testing, as it can be time-consuming to comb through websites by hand. However, manual testing is an unavoidable method for catching accessibility issues that automated solutions are not able to detect.

Let’s dive into what manual testing is and how it can support your accessibility goals.

What is manual testing?

Manual accessibility testing involves checking for accessibility issues that cannot be detected by automated software and instead require human judgment. For example, checking that your images’ alt text is accurately describing your site’s images must be done manually.

Manual testing is most effective for checking compatibility with:

• Assistive technology, e.g., screen readers
• Keyboard navigation
• Browser plugins, e.g., HTML and JavaScript

Manual accessibility testing services

If you want to be sure you catch all critical accessibility issues, manual testing is best done by trained professionals who are knowledgeable about different disabilities and assistive technology. Fortunately, there are manual testing services available to help organizations ensure their sites are up to accessibility standards. These trained professionals use a variety of software and semi-automated tools to manually check your website, such as:

• JAWS and NVDA on Windows 10
• VoiceOver on iOS
• Navigating only using the keyboard
• Chrome, Firefox, and Internet Explorer
• Color contrast checker
• In-house manual accessibility testing team

In-house manual accessibility testing team 

For a holistic approach to accessibility, your organization could benefit from setting up an in-house team of manual accessibility testers. Having a team within your organization can help ensure accessibility issues are caught early on, as the team can more readily establish themselves in the design and development processes and advocate for accessibility consistently. This approach does entail an organization-wide effort to establish buy-in and budget for the team, however, in the long run, it is the most cost-effective and sustainable solution.

User testing with people with disabilities 

A highly effective way to check your website for accessibility issues is to set up user testing and focus groups including people with a range of disabilities. It is important to keep in mind that feedback from one person with a disability does not apply to all people with disabilities. A person with a disability does not necessarily know how other people with the same disability interact with the web. You should understand the range of disabilities you will be planning and testing for, as you might need assistive technology prepared for your tester.

How manual and automated testing work together

Finding the right balance between manual and automated accessibility testing depends on a few factors, such as your organization’s accessibility goals, the number of websites your organization owns, budget, and existing in-house accessibility knowledge. Here are some examples of how you can leverage the benefits of each testing approach:

Nip accessibility issues in the bud with early automated testing

If your team oversees multiple websites, use an automated testing solution to check that all code is up to accessibility standards before it is implemented. This can save hours of work compared to remediation. Early automated testing is also a good indicator as to how accessible your website is overall, which can help you in the early stages of planning. For example, if a website accessibility checker detects many issues on your website, you can set aside more time and resources to tackle the problem areas effectively.

Semi-automated testing

Assisted testing, or semi-automated testing, is the perfect example of manual and automated testing approaches working together to reach the most accurate results. The way assisted testing works is that the automated tool detects a potential accessibility issue. The automated tool then asks a series of deciding questions which someone must manually review. These questions help the automated tool diagnose the correct issue and guides you to resolve the issue.

Follow up with routine manual testing

A manual testing approach is essential to evaluating your website’s usability. Plan for regular user testing and you ensure a consistent flow of feedback from your target audience even as your website changes and evolves as websites do over time. Every time a web developer or content contributor change something on the website, it opens the possibility of new accessibility issues. Accessibility, therefore, is an ongoing process.

Track and report progress using automated tools

An automated testing solution with built-in reporting tools will save you the headache of tracking progress and creating reports. The right automated tool can track progress on accessibility issues, maintain an overview of task owners, and document your results as you go. Tracking your progress on accessibility issues is not only important for legal purposes, but it is also a great way to motivate and encourage your organization’s accessibility efforts.

Curious to learn more about how to foster an organization-wide commitment to accessibility and sustainable accessibility testing practices?

Our Siteimprove Frontier learning path, Accessibility for leadership, covers all the important aspects of accessibility testing. This learning path includes a course on “Accessibility testing in your organization: a holistic approach” that gives you valuable insight into manual testing and uncovers the benefits of an in-house manual testing team.