OPINION: Best user testing mimics real life


17 Jan 2015

Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Pin on PinterestShare on RedditEmail this to someone

Testing - be it of aeroplane evacuation procedures in the event of an emergency or of website usability - should mimic real-life scenarios for best results

Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Pin on PinterestShare on RedditEmail this to someone

If the corporate brochure is to be believed, researchers in pristine labs carry out perfectly designed research projects devoid of even a hint of bias.

They observe users on the latest technology who look just like the beautiful people in the persona definition picture from earlier in the project, and match the target customer demographic to the exact scintilla. 

These test subjects have the exact interests, social class, income, family background, gender, level of web use and expertise, technical prowess, professional and personal motivations as the company’s target customers. The icing on the cake is that the subjects are perfectly motivated to provide the researchers with as much time as they need to squeeze every last piece of information from them with the beautiful result that they provide the researchers with near-perfect results, free of noise and inaccuracies.

Meanwhile in the real world, people of all shapes, sizes, stress levels, foibles, preconceptions, expectations and web expertise on hardware of diverse ages and capability are using broadband connections of assorted speeds, seeking to fulfil tasks using websites.

And somewhat importantly it is the latter scenario rather than the former into which a website seeks to bring value.

The impact of this dynamic on user testing is that researchers need to work harder than ever before to recreate real-world scenarios in real-world contexts. How users are observed, more than ever, needs to be indicative of how they use websites in the wild.

Recreating real-life situations

We are fortunate to work with a client who assists sportspeople with game practice, by more accurately mimicking match play in practice play. The premise upon which this business is built is that the vast majority of game practice (particularly in solo sports, such as golf and tennis) is ineffective because it recreates only the technical context of match play, but crucially not the psychological context, which arguably has a greater bearing on performance. The business achieves this by recreating pressure, context, the potential for social embarrassment, competition and finality.

Using technology and scenario planning, they mimic as far as humanly possible what it must feel like to take a penalty at Wembley, serve for Championship point at Wimbledon or putt for the winning birdie at the US Open.

Learning, whether it’s from practice or observation, has to be based on experiences as close to the real-world environment as possible.

A leading American airline wanted to test its evacuation procedures and to do so simulated a crash in a plane full of test subjects. These subjects intently listened to the various announcements as the plane imitated a crash situation, and patiently and politely carried out the instructions delivered over the speakers in accordance with the wishes of the captain and cabin crew.

The conclusion from the test? That the safety procedures were exactly correct and appropriate for the situation.

The problem with that? The test bore no resemblance to real life.

Money matters

Aware of the shortcomings of the test, the airline designed another test, which this time fashioned the behavioural motivations of a crash situation, which frankly cannot be created via a simulated crash. Rather these motivations were recreated by getting a plane full of 20-something testosterone-filled American footballers into the cabin, and telling them that on the trigger of various announcements they were to leave the plane according to the instructions. Oh yes, and did we mention that the first man out got US$1,000, the next five got US$500 and the following 10 got US$100 each?

The outcomes of the test? Very different indeed to the previous test scenario. The safety procedures fell some way short of dealing with the human emotion involved in a real-life crash.

Testing and research are not possible from ivory towers. Empathy cannot be expressed from a safe distance. Smart designers, committed to human-centred design processes, are prepared to roll up their sleeves, find the right test subjects the hard way, and observe them in real-life situations. Because that’s the battleground where the fruits of their labour will stand or fall.

Gareth Dunlop

 Gareth Dunlop Gareth Dunlop owns and runs Fathom, a user-experience consultancy that helps ambitious organisations get the most from their website and internet marketing by viewing the world from the perspective of their customers. Specialist areas include user testing, usability and customer journey planning, web accessibility and integrated online marketing. Clients include Invest NI, Power NI, Three, Ordnance Survey Ireland, and Independent News and Media. Visit Fathom online.

Seat on fire image via Shutterstock