My thesis for download

For those interested, my Ph.D. thesis is available for download [link at archive.org, the WayBack Machine - thanks guys for storing it].

Summary

The problem of information search has been recognised for many years: the main issue is one of being able to filter out the masses of irrelevant information presented rather than accessing relevant information in the first place.

Because machines cannot consistently understand a users’ context, the user remains the ultimate arbiter of what documents are relevant to their concerns. The summaries presented by a Web search engine offer a way to quickly scan large amount of data, and being able to accurately judge the relevance of these summaries is important.

In the first two experiments, this thesis tested a custom scale and found it reliable and valid. The second experiment compared four methods of summary construction (initial mention, query-biased, keyword and title only), finding that the title alone enabled the most accurate discrimination. Experiment four partly replicated this finding, and models of negative modulation were proposed, tested and found to be present in some if not all summaries.

Experiment five tested relevance decision accuracy with participant generated materials to deal with the high variance of responses encountered in the earlier experiments, but this experiment needed more power to be reliable.

Experiments six and seven tested misleading and accurate summary elements, and found that they are combined as a modulation, though the user ascribes greater significance to the snippet than the title. The title however remained the better element for accurate judgements. These experiments also found evidence to support the idea of negative information scent, an extension to Pirolli and Card’s Information Foraging theory.