Testing times

How can you be sure that what you have now or what you want to create is what users want?

The most obvious answer  is arguably: “By asking them directly.”

But if it’s that easy, why is it often a battle to sell user testing? When we suggest user testing to our clients, we are often met with comments like “We haven’t got time before launch”,  “We don’t need to ask users to know what they want” or “It’ll only tell us what we already know”.

This seems a bit nonsensical to me. Why spend months of your life pouring blood, sweat and tears into something you think people want but never ask them if they actually want it?

The most exasperating thing is that user testing doesn’t have to be some big, complicated drawn-out process. In our experience, just ten minutes with a real user can yield all kinds of useful insight that can improve out clients’ bottom line. As Steve Krug says, any kind if testing is better than no testing.

So in an attempt to debunk some of the myths around user testing, here’s how we try to explain it to our clients…

A common approach

We have a tried and tested approach which seems to follow the industry standard for user testing:

  • Plan for testing
    Clarify the purpose and scale testing and identify areas where specific user feedback is required. In addition, clarify the test user profiles and the types of devices to be tested
  • Produce collatera
    Produce user profiles to help when recruiting test subjects and create test scripts to guide the sessions
  • Recruit users
    Either recruit suitable test users that match the defined profile from your own customer databases or via a third part market research recruitment company. In our experience, six users is enough to solicit good quality feedback via user testing
  • Test
    Test the website with the recruited users, and record onscreen interaction and verbal feedback, and consider the benefits of additional specialist recording techniques, such as eye tracking
  • Report
    Pull together and prioritise your observations and recommendations for improvements, citing video evidence where relevant.

Quantitative research

User testing is qualitative. You will get lots of useful information on what users do and why they do it, but it will only come from a small selection of users. It should be supported with some level of quantitative research so that you can validate that the opinions given by test users are reflective of common trends among your entire audience.

We recommend that some time should be set aside to review available user data so that you can identify common trends and behaviours. Common areas of useful analytics include audience demographics, content data, search and referrer data, browsing devices and operating systems, and conversion goals.

Methods of testing

There are three methods of user testing that we commonly employ, each with its own pros and cons.

Lab-based testing

Lab-based testing is the most comprehensive but is the most rigid and most expensive approach.

Testing takes place in a specialist usability testing lab with an observation room (we have our own usability testing lab in Leamington Spa). Test moderators sit with test users and ask them questions, and use specialist software and hardware to record interaction, facial expressions.

Many people can watch the testing sessions via a one-way mirror and on screen in the observation room and you have the opportunity to interview users after the sessions and ask more specific questions.

Guerrilla testing

Guerrilla testing doesn’t take place in a specialist lab and therefore is more flexible and cheaper. Moderators travel to a specified location – all they need is a quiet space, a connection to the internet and somewhere to plug in a laptop.

As with lab-based testing, moderators ask questions and use specialist software and hardware to record interaction, facial expressions and voice, and can record interaction on desktops and mobile devices.

However there is no real opportunity for many people to observe testing in a live environment or interview test users.

Remote testing

Remote testing is relatively cheap and quick but provides the least amount of user insight. User recruitment and testing is all handled online via a third party provider. Although you can vet users before testing and specify what you want them to test, you have no control over them and won’t have the opportunity to interrogate them during or after testing.

Once the remote test has been set up and launched, users will be automatically recruited. They will be asked to carry out a test from their own computer in their own time. On screen interaction and verbal feedback is recorded and packaged as a downloadable video.

So, get testing

There are many other possible approaches to user testing. The three described here are the ones we most commonly employ.

The most important thing to remember is just find an approach that works for you and do it. Any kind of user testing is better than no kind of user testing!