This article was first published on 9 December 2004.
ISO 9241 says that usable software is composed of three elements: efficiency, effectiveness, and satisfation. However, there may be cases when these three are mutually exclusive. This article discusses an experiment where the provision of the greatest effectiveness and efficiency also provided the least satisfactory.
Much of the field of usability concerns providing users with what they want. This is often seen to result in ‘featuritis’, where an application becomes bloated with features that hardly anybody uses. However, research indicates that many users’ wishes are best not accounted for. How is it possible for the usability designer to know what should be taken into account and what should be ignored?
From my own research, I have found out that what people want and they can be shown to prefer might not be the best thing to provide them with. For an experiment of mine researching search engines, I presented people with short summaries of documents. The participants had to decide which summaries came from documents that were relevant to a particular question, and which were not. The summaries themselves were very basic, but made up similar to those found in common search engines.
The participants sometimes got summaries that had just the document title (the content within the title tags), and sometimes they got a short text snippet which showed where the search keywords were in the document (much like the little snippet of text you see with Google). The rest of the time, they saw both the title and the snippet. Note that there was (on average) just over twice the number of words in the snippet as in the title.
It seemed obvious to me that having both title and snippet would allow the most accurate decisions, because after all, this is the largest amount of information. The title would probably give the least accurate decisions because it had the lowest amount of information, and the snippet showed you where the search words appeared in the document. Clear?
Well that was not the case: I found that the titles gave the most accurate decisions and the snippet the worst. The reason appeared to be because the snippet showed searchers where their search terms were, but in doing so gives searchers a false impression that the document is more relevant to the search than it actually is. The titles did not differ according to what was being searched for, but I found that this was good because it allowed the searchers to do a more objective assessment of the documents contents.
However, a few people mentioned that they found the title-only condition to be very hard to judge even though it produced the most accurate judgements. They hated having to make decisions using just the title so much, and yet they produced the best decisions. It seems that having to think about things objectively is not so easy, but it is a much better way forward if we want to find what we need.
Clearly there was a case here where providing people with what they preferred to use resulted in less than optimal performance. In these cases, what should a practitioner decide is the best interface? Discussion with the users would probably be the best thing to do next step. Testing should illustrate whether the demands of a low satisfaction outweigh the benefits arising from the most effective and efficient interface, and it wouldn’t need to be an onorous task (maybe even a short questionnaire or ad-hoc discussion with typical users).
And the moral of this experiment? The searchers preferred having lots of information and said clearly that having less (and more objective) information is harder to deal with. However, information searches are best done by placing more emphasis not on the searcher, but on what stuff they have to wade through.
Machines are wonderful things: they can do so much to make our lives easier and more productive, but they cannot make our decisions for us. At least, not yet!
Give ‘em what they want! Or should you?
ISO 9241 says that usable software is composed of three elements: efficiency, effectiveness, and satisfation. However, there may be cases when these three are mutually exclusive. This article discusses an experiment where the provision of the greatest effectiveness and efficiency also provided the least satisfactory.
Much of the field of usability concerns providing users with what they want. This is often seen to result in ‘featuritis’, where an application becomes bloated with features that hardly anybody uses. However, research indicates that many users’ wishes are best not accounted for. How is it possible for the usability designer to know what should be taken into account and what should be ignored?
From my own research, I have found out that what people want and they can be shown to prefer might not be the best thing to provide them with. For an experiment of mine researching search engines, I presented people with short summaries of documents. The participants had to decide which summaries came from documents that were relevant to a particular question, and which were not. The summaries themselves were very basic, but made up similar to those found in common search engines.
The participants sometimes got summaries that had just the document title (the content within the title tags), and sometimes they got a short text snippet which showed where the search keywords were in the document (much like the little snippet of text you see with Google). The rest of the time, they saw both the title and the snippet. Note that there was (on average) just over twice the number of words in the snippet as in the title.
It seemed obvious to me that having both title and snippet would allow the most accurate decisions, because after all, this is the largest amount of information. The title would probably give the least accurate decisions because it had the lowest amount of information, and the snippet showed you where the search words appeared in the document. Clear?
Well that was not the case: I found that the titles gave the most accurate decisions and the snippet the worst. The reason appeared to be because the snippet showed searchers where their search terms were, but in doing so gives searchers a false impression that the document is more relevant to the search than it actually is. The titles did not differ according to what was being searched for, but I found that this was good because it allowed the searchers to do a more objective assessment of the documents contents.
However, a few people mentioned that they found the title-only condition to be very hard to judge even though it produced the most accurate judgements. They hated having to make decisions using just the title so much, and yet they produced the best decisions. It seems that having to think about things objectively is not so easy, but it is a much better way forward if we want to find what we need.
Clearly there was a case here where providing people with what they preferred to use resulted in less than optimal performance. In these cases, what should a practitioner decide is the best interface? Discussion with the users would probably be the best thing to do next step. Testing should illustrate whether the demands of a low satisfaction outweigh the benefits arising from the most effective and efficient interface, and it wouldn’t need to be an onorous task (maybe even a short questionnaire or ad-hoc discussion with typical users).
And the moral of this experiment? The searchers preferred having lots of information and said clearly that having less (and more objective) information is harder to deal with. However, information searches are best done by placing more emphasis not on the searcher, but on what stuff they have to wade through.
Machines are wonderful things: they can do so much to make our lives easier and more productive, but they cannot make our decisions for us. At least, not yet!
No comments:
Post a Comment