Industry-day

Abstract:
We present a new framework to conceptualize and operationalize the total user experience of search, by studying the entirety of a search journey from an utilitarian point of view.

Web search engines are widely perceived as “free”. But search requires time and effort: in reality there are many intermingled non-monetary costs (e.g. time costs, cognitive costs, interactivity costs) and the benefits may be marred by various impairments, such as misunderstanding and misinformation. This characterization of costs and benefits appears to be inherent to the human search for information within the pursuit of some larger task: most of the costs and impairments can be identified in interactions with any web search engine, interactions with public libraries, and even in interactions with ancient oracles. To emphasize this innate connection, we call these costs and benefits Delphic, in contrast to explicitly financial costs and benefits.

Our main thesis is that users’ satisfaction with a search engine mostly depends on their experience of Delphic cost and benefits, in other words on their utility. The consumer utility is correlatedwith classic measures of search engine quality, such as ranking, precision, recall, etc., but is not completely determined by them. To argue our thesis, we catalog the Delphic costs and benefits and show how the development of web search over the last quarter century, from its classic Information Retrieval roots to the integration of large Language Models and Generative AI, was driven to a great extent by the quest to decrease Delphic costs and increase Delphicbenefits.

We hope that the Delphic costs framework will engender new ideas and new research for evaluating and improving the web experience for everyone.

This talk reflects joint work with Preston McAfee and MarcNajork.