[IxDA Discuss] Hypotheses about tasks

[Peter Bagnall]

Robert,

I like the idea, but I think you're going to find this quite  
difficult as you've got it phrased.

1) Difficulty is going to be quite hard to define precisely. To test  
this hypothesis you're going to need a strict definition that you can  
measure in some way. Perhaps the simplest would be look at users  
perceptions of difficulty (say using a questionnaire), or success  
rate as an indicator. I'd also tend to measure the effect of number  
of steps and time per step separately - you'll learn more that way  
than you will by conflating them.

2) Again, definitions are the hardest bit. Time is nice and simple,  
so that's great, but how are you going to measure relative  
complexity? Likewise willingness to solve the problem is also going  
to be pretty hard to measure. The greater the error on measuring it  
the larger the number of test subjects you'll need to get a  
meaningful result.

3) It could be argued that this is a definition of difficulty (all  
other things being equal).

So generally I would say you need to simply your hypothesis - just  
try to test the relationship between two variables at a time, one of  
which you control. Clearly your users will vary in many ways, but you  
get round this my having a number of people and using the stats to  
get a sense of the variation - continue until you have high enough  
confidence that your result is not just luck!

The other thing is definitions. A hypothesis is not much use unless  
you can actually measure the things it relates to. Measuring things  
like "difficulty" is hard, because difficulty is not a simple concept  
- many things contribute to it, and something one person finds  
difficult may be easy to someone else - so you need to be very clear  
what you mean by it if it's going to be useful. You'll probably have  
to go for something that is a component of difficulty otherwise  
you'll have to invent some aggregate of the various factors which  
will be very hard to justify.

As an example, I did an experiment on the effects of "complexity" of  
software on "ease of use". I measured complexity as the number of  
available functions in the system (I used Word in it's default form  
and with most of its functions removed). I measured "ease of use" by  
assuming that people would complete faster with something that was  
easier to use. That's not a perfect definition of course, but I was  
very clear that's what I was measuring, so the result had some  
meaning - and I found a weak effect, although I would have needed to  
test with more people to be really confident of it. I think I had  
about a 10% chance of my result being sheer fluke, which is  
indicative, but hardly proof. Also I had no illusions that number of  
functions is the only thing that contributes to complexity, but it  
was one factor that I could measure.

At the end of the day you have to be clear about what you measure,  
and then what inferences you draw. If you measure number of functions  
as your way of getting a handle on complexity you can't then  
generalise it to other types of complexity (say, menu depth).

Hope that's helpful - I'll be very interested to see your results  
from this. Hoekman's law here we come ;-)

Cheers
--Pete


On 17 Aug 2006, at 23:16, Robert Hoekman, Jr. wrote:

> [Please voluntarily trim replies to include only relevant quoted  
> material.]
>
> OK - so after conversations with several people, all of whom I've been
> talking with separately, here are some hypotheses that seem to be  
> taking
> shape:
>
> 1) "The difficulty of completing a task is a function of the number  
> of steps
> involved in the task and the time it takes to perform each step."
>
> 2) "The time it takes to perform each step in a task is a function  
> of the
> relative complexity of the step and a person's willingness and/or  
> ability to
> understand and complete the step."
>
> 3) "The more difficult a task, the less likely it is to be completed."
>
> I think this is worth exploring, and apparently several other  
> people do as
> well, so I wanted to start a new thread to keep in all going in one  
> place
> instead of managing several different threads.
>
> The wording of these statements is the key source of debate so far.  
> Mainly,
> I'm not sure "difficulty" is the right word, but "cognitive load" is a
> little too technical. I'm erring on the side of simplicity.
>
> Thoughts?
>
> -r-
> ________________________________________________________________
> Welcome to the Interaction Design Association (IxDA)!
> To post to this list ....... discuss at ixda.org
> List Guidelines ............ http://listguide.ixda.org/
> List Help .................. http://listhelp.ixda.org/
> (Un)Subscription Options ... http://subscription-options.ixda.org/
> Announcements List ......... http://subscribe-announce.ixda.org/
> Questions .................. lists at ixda.org
> Home ....................... http://ixda.org/
> Resource Library ........... http://resources.ixda.org
>

--------------------------------------------------
A society grows great when old men plant trees whose shade
they know they shall never sit in.
      - Greek proverb

Peter Bagnall - http://people.surfaceeffect.com/pete/