Conducting usability testing with people with disabilities

February 7, 2023

If you’re testing your products with people without disabilities, why shouldn’t you test them with someone who’s blind, deaf or dyslexic? They have an equal right to check the products on your website, add the ones they want to the cart and pay what they purchase, without needing the help of a friend or a family member. If they aren’t able to do that, they’ll simple turn to a competitor who offers a more accessible website.

Why conducting usability tests with people with disabilities?

When 15% of the world’s population lives with some kind of disability (Source: WHO) it doesn’t make sense to exclude them from the usability tests of product. It’s like you’re ignoring 15% of your target market. Not to mention the growing number of aging users, who develop vision, hearing, motor and cognitive limitations and will also benefit if a product is built without any barrriers. 

Even if the website was designed and developed in conformance with the accessibility guidelines, people with disabilities might still not be able to accomplish the necessary tasks on the website. Testing with a greater diversity of users will make all the difference for the website (and for the business). Not only are you making it available for a larger number of people with purchasing power, you’re also increasing the chance of turning them into customers.

Drawing on our experience conducting usability tests with users with disabilities, we’re sharing a few things to take into account when planning and conducting research with people with disabilities and make sure everything goes smoothly and you can collect valuable insights.

How to recruit users with disabilities?

When running usability tests with people without disabilities we know there are those who are tech-savvy and those who struggle a bit more when using digital products. When it comes to people with disabilities the same happens. People who are blind can be more or less comfortable using screen readers. It’s important to test with regular level users, not just experts in accessibility and assistive technology.

If you’re not working with a recruitment agency, there are a number of places where you can find people with disabilities willing to help you:

• Online communities for people with disabilities;
• Advocacy groups and charities; 
• Organisations and agencies who provide training and other services to people with disabilities;
• Seniors organisations and local senior centres. 

Don’t assume that one participant with a disability represents all

If you’re on a budget you’ll probably feel forced to invite people with different disabilities — one person who’s blind, one person who’s deaf, one person who has a motor disability and other with a cognitive disability. However you should be aware that that’s not a representation of everyone with similar disabilities. 

A blind person is not a representation of everyone with visual impairments. 

Photo by Mikhail Nilov in Pexels

As there’s different types of visual impairments, people use different interaction techniques, adaptive strategies and assistive technology configurations to navigate the web. While someone might use VoiceOver on Apple devices, someone using Windows will use NVDA for example, or they might even use a braille display. If a person has low vision they might set colour preferences for high contrast mode or set up a bigger font size.

While the ideal would be testing with several people with different visual impairments, reality is budget and time don’t always allow that to happen. 

You need to adjust the participants you recruit to the product you’re testing. If you’re testing a product without any audio information or speech functionalities then you won’t need to recruit users with auditory impairments. If you’re testing a university app and your user analysis reveals there will be a lot of people with dyslexia using it, then you should include people with this disability in your testing.

Also, don’t assume that people with disabilities can’t or won’t use your product. If you’re testing a website for a car brand, don’t exclude people who are blind from the usability testing just because they aren’t able to drive. They might be looking to buy a family car or as a gift for their children. 

Where to conduct the usability tests?

Depending on their disability, for some people it might be an issue getting to the research lab. A participant with mobility issues might feel more comfortable in their own home or a place they often go to, like a cafe or school. 

While conducting the tests in the lab makes it easier to record the session and having the project team joining in to observe, the best thing to do is finding out where the participant would prefer to do the test.

If you decide to conduct the usability tests in-house, you need to make sure your lab is accessible: 

• Check if the entrance of the lab and the building has any steps or if there’s a working elevator available;
• Check if the doors are wide enough for a wheelchair to fit in, as well as if it can fit under the table where the testing device is in (at least 70cm high);
• Check if the room is large enough to accommodate a service dog, a caretaker or a sign language interpreter;
• Check if the toilet is accessible.

Photo by Marcus Aurelius in Pexels

If you’re welcoming a participant with a visual impairment, offer to guide them or ask how they would like to be guided. Let them know if there are any obstacles (steps, furniture, plants) and what’s on the desk (devices, a bottle of water, napkins). 

You can also offer some assistance with booking a taxi to pick them up or meet them in front of the building to guide them to the office. 

Remote testing is can also be a good option to avoid any complications participants might experience with getting to the lab. 

If you’re working with people with disabilities, check these general rules of etiquette for interacting with people with disabilities. 

Which devices should the participants use?

While for people without disabilities is pretty much indifferent which device they use to complete the tasks, for a user with some form of disability that’s not the case. They normally have their computer and smartphone configured a certain way and with the assistive technology they normally use. 

Allowing the participant to use their own device will be less costly. There’s a multitude of assistive devices and softwares, and while some are free, acquiring some of them would represent a big investment. For example when it comes to screen readers, while one participant might use NVDA, which is free, other might use JAWS, a paid software ($90 a year or $900 one-time purchase). On a similar example, if a blind participant uses VoiceOver on Apple products, they might struggle if they have to complete the test in a Windows device.

Having the participant bringing their device will also be less time-consuming. Although you still have to allow some time at the beginning of the test for them to set everything up, it’ll probably be faster than having them setting up their preferred configurations in the lab devices. 

💡 Tip: If you’re testing with screen reader software, ask if the participant can slow down the speaking rate. This will make it easier for the moderator and anyone observing the test to understand where they user is at. You’ll be surprised with how fast they have it set! 

Is the website accessible enough to be tested?

Photo by Christina Morillo in Pexels

The last thing you want is having a person with a disability coming to your office just to realise the website is not accessible enough to be tested. 

From running the website through an automated tool or doing some manual checking yourself, here’s a few things you can do:

1. Try using a screen reader to see if the website is correctly read. Look for incorrect reading order or tags and images without helpful alt text, for example.
2. Navigate the website using a keyboard by clicking the tab key. If no element of the website becomes focused it won’t be possible for someone with a motor disability that prevents them to use the mouse to navigate the website.
3. Check colour contrast between the text and the background. If there’s a low contrast it will be harder to read by a participant with low vision.
4. Conduct an accessibility audit against WCAG 2.1 guidelines.
5. Conduct usability testing with users without disabilities first. This will help you to correct any obvious technical or usability issues with information architecture, navigation or content. If they represent a problem to users without disabilities, they are probably an ever bigger problem for a user with a disability to overcome.

Schedule the right amount of time for the testing

It’s only natural that people with certain disabilities take longer to complete the test tasks (specially if the website has severe issues) when comparing with a regular user.

The disability itself, medications, and the extra effort to use assistive technology are some of the factors that can influence the time it takes for the user to complete the test. You’ll want to avoid user fatigue. They might not be able to attend a long session or might need to take some breaks. People with tremor or loss of fine motor control might take more time to do basic tasks like clicking a button. A person with dyslexia might need more time to process textual information. 

There’s not a one-size-fits-all, you should take into consideration each participant and their particular needs. The best thing is to have a shorter task list and prepare additional tasks if there’s time and the participant doesn’t mind going through a couple more.

Besides planning more time for the test session, it’s also better to plan for more time before and after the test. Plan time at the beginning of the test for the participant to set up any assistive technologies and configure settings as they wish. Then plan more time in-between sessions as you might need additional time to pack the participants devices and help them leave the building. 

Moderating the test session

All documents you provide to participants such as the consent form, non-disclosure agreement, instructions and directions, tasks or forms should be in clear and simple language, specially for participants with cognitive disabilities. You should also consider the best way to send or hand out the documents, based on each participant ability. They might prefer to receive it by e-mail (accessible HTML, PDF, plain text), printed in large font or in braille, for example. 

At the beginning of the test, it’s very important to reinforce that it’s not the participant that’s being tested, but the website. They are probably doing it for the first time and assuring you’re not assessing their skills or abilities might make them feel more at ease.

Ideally you should use the same tasks for users with and without disabilities. However, while in a regular test the moderator should help the participant as little as possible, with users with disabilities it might be necessary to intervene a bit more.

Be prepared to adjust. When testing with users with intellectual disabilities you should be ready to explain the task a bit better to ensure they understand what they have to do. If you’re using rating scales, use plain language and rather than using a 7-point scale, keep it simpler with a maximum of 5-point. 

Bring the team to observe the testing

Photo by Tima Miroshnichenko in Pexels

There are many people who have never seen a blind person using a computer or smartphone with a screen reader. For people to design and develop with accessibility in mind they need to be aware of the specific needs of people with disabilities when navigating through a website. 

Calling different members of the team — designers, developers, content writers — will allow them to see how people with disabilities actually navigate the web, hearing first-hand about the issues they struggle with. For some it will certainly be a wake up call.

. . .

More usable and accessible websites for all users 

When you make your website or app accessible you’re not only improving the experience for people with disabilities. Elderly people, someone with a temporary or situational limitation and users without disabilities will benefit. 

Having a product that checks all accessibility guidelines is not enough. Testing it with real users, with real limitations, will uncover the real usability issues they struggle with. 

. . .

 📚 Useful resources: 

• Just Ask: Integrating Accessibility Throughout Design — a comprehensive book about accessibility, available online, with a useful chapter on usability testing.
• “Accessibility — Creating accessible digital experiences” — a white paper where we cover disabilities and barriers to web accessibility, best practices to build accessible websites, and more.
• “Web Accessibility” — an infographic we created to raise awareness for web accessibility to celebrate the International Day of Persons with Disabilities.