OPINION: All design is a hypothesis until it is put in front of a user

16 Aug 2014

The challenging dichotomy of interaction design is that while users are the ultimate arbiters of design quality they are consistently terrible at judging it without actually using it.

In other words, it is the ultimate ‘do what I do’ not ‘do what I say’ environment. Design quality cannot be assessed aesthetically.  

It is entirely possible for a site to be pretty and pretty awful simultaneously.

Wired magazine, a design and UX authority if ever there was one, recently ran an article, The Super-Slick UX of Virgin America’s New Booking Site, commending the new Virgin America website. With the site’s laudable and quite appropriate focus on mobile devices and on the usability of the printed-out board pass, Wired was gushing in its praise of this beautiful piece of design. The iconography, it commented, was particularly creative and effective.

“Instead of using route maps to depict the cities the airline services, Work & Co. created iconic buttons that play off each city’s landmarks and stereotypes. Lincoln represents DC, a terrier reps Boston, and a pair of gay walrus computer programmers represent San Francisco (of course).”

However, in the comments, Nick Aster wants to know “Why fetishize mobile? And they took away the route map!”


Wired felt “The typical multi-page flow has been reduced to a single screen designed to fit the mental map of the user rather than forcing them through a legacy path dictated by outmoded technology.” However, Marcy Sutton had a very different experience and a lot of anger! “Have you tried using a keyboard or a screen reader? The accessibility experience is dismal. It kills me that this gets a pass as a slick user experience. You’re leaving so many users in the dark. FIX IT NOW!”

The article claimed the redesign was more than prettying things up. “Beyond the cutesy gloss is the first radical rethinking of the flight booking experience in a decade”, but that wasn’t how Prime 1987 felt. “Yeah, Virgin Atlantic’s website looks superficially pretty but as soon as you try to go flexible and deep, the thing is a horror. I can’t imagine how much business they lose because of their lousy online capability.”

Nancy Gwen summed up the overall mood in the comments box. “It looks very pretty, but in actual use I have found it enormously frustrating – both on phone and laptop. Harder to make comparisons between the various flights. Everything seems to take me longer than the old way!”

Marcy, Nancy, Nick and Prime 1987 (from Transformers?) may or may not be representative of Virgin America customers more widely, however, at the time of writing there wasn’t a single trace of positive sentiment in the comments area, nor any response from Virgin America.

I’m not trying to make the specific point about who is right and who is wrong regarding the article in question, rather I’m making the more general point that what can appear new, groovy, cool or slick to the designer or even to the informed commentator, as in the case of Wired, still needs to be measured by the ultimate barometer of the opinion, which is what users think of it.

And crucially, that opinion musn’t be garnered by detached observation and judgment; it must be gleaned by direct use and behaviour. Until a design has been released into the wild and proves its mettle in the real world it must only be considered a hypothesis, to be proved or disproved like any other.

Gareth Dunlop

 Gareth Dunlop Gareth Dunlop owns and runs Fathom, a user-experience consultancy that helps ambitious organisations get the most from their website and internet marketing by viewing the world from the perspective of their customers. Specialist areas include user testing, usability and customer journey planning, web accessibility and integrated online marketing. Clients include Invest NI, Power NI, Merchant Hotel, Ordnance Survey Ireland, and Belfast City Council. Visit Fathom online.

Frustrated website user image via Shutterstock

Gareth Dunlop runs Fathom, a UX consultancy that helps organisations get the most from their digital products.