[IxDA Discuss] Measuring User Experience

[Katie Albers]

Judging by the methods you've proposed here as 
possible solutions, I'm going to assume that you 
actual mean how to *assess* User Experience, nor 
measure it. Don't get me wrong, I actually 
believe that a non-numeric method of examining 
user experience is a better guide to 
understanding a current state and planning 
improvement, but management does love its numbers.

The only way to measure user experience is with 
users...otherwise it's something else. That 
immediately leaves out heuristic evaluation. 
Heuristic evaluation is very good at developing 
reasonable expectations of how standards and 
expertise predict that something will or will not 
be used and can be used as a preliminary stage in 
development or as a guide for what to look for in 
researching User Experience, but it cannot, and 
is not designed to, "measure" user experience, 
nor to assess it.

The best ways of assessing user experience are by 
individual user testing, 
ethnographically-oriented studies (observing 
users as they use the software in the actual 
situations in which they would use it), and 
interviewing actual users (this becomes less and 
less useful as users become more accustomed to an 
application because they tend to start to take 
the problems for granted). I've also often found 
it extremely useful to take a couple of users out 
for a drink after work and get them talking about 
whatever I'm assessing...but I'm a consultant and 
I can get away with things like that.

Focus groups tend to foster groupthink, which I 
believe makes them useless in considering User 
Experience. Surveys often force those 
participating in them to answer things in ways 
which may be misleading, so while they can be 
useful, I recommend them as an adjunct and use 
them when I've got a series of questions and 
responses that reflect the way in which users 
have discussed the application in other research.

Content, usability and functionality will all 
normally show up automatically within the testing 
and studies I discussed above. It's worth 
remembering that as far as the user is concerned 
these are not really separate things, so you need 
to pay careful attention to exactly what they say 
and in what context and backcheck your 
conclusions with further conversation/inquiry 
into "why did you say that" and "are you looking 
for something you can't find" or "what did you 
expect it to do" and so forth.

Assessing "branding" is something of a 
double-edged sword: The user experience shapes 
the brand at least as much as the other way 
around. This is not something users normally 
notice, but if they find an application 
(software, website, hardware, whatever) obscure, 
difficult to use, lacking in things they expect 
to be there, poorly planned, ugly...It can have 
all the logos and approved colors, layouts and 
fonts you like, but that may, in fact be a bad 
thing, because you're making the brand look worse.

Good luck

Katie

At 3:54 PM +0200 4/16/08, Jorge Márquez wrote:
>Hello everyone,
>
>I was wondering which is the best practice to Measure User Experience, based
>on 4 key pilars: 1) Branding; 2) Usability ; 3) Content and 4)
>functionality.
>
>What would you recomend for this kind of studies: Survey, Focus group, user
>testing, heuristic evaluation.... etc..?
>
>Best regards,
>
>Jorge
>
>--
>Échale un vistazo a mi blog www.usandolo.com
>________________________________________________________________
>Welcome to the Interaction Design Association (IxDA)!
>To post to this list ....... discuss at ixda.org
>Unsubscribe ................ http://www.ixda.org/unsubscribe
>List Guidelines ............ http://www.ixda.org/guidelines
>List Help .................. http://www.ixda.org/help


-- 

----------------
Katie Albers
katie at firstthought.com