Discovering and Understanding Word Level User Intent in Web Search Queries

Rishiraj Saha Roy, Rahul Katare, Niloy Ganguly, Srivatsan Laxman, Monojit Choudhury

Abstract


Identifying and interpreting user intent are fundamental to semantic  search. In this paper, we investigate the association of intent with individualwords of a search query. We propose that words in queries can be classified as either content or intent, where content words represent the central topic of the query, while users add intent words to make their requirementsmore explicit. We argue that intelligent processing of intent words can be vital to improving result quality, and in this work we focus on intent word discovery and understanding. Our approach towards intent word detection is motivated by the hypotheses that query intent words satisfy
certain distributional properties in large query logs similar to function words in natural language corpora. Following this idea, we first prove the effectiveness of our corpus distributional features, namely, word co-occurrence counts and entropies, towards function word detection for five natural languages. Next, we show that reliable detection of intent words in queries is possible using these same features computed from query logs. To make the distinction between content and intent words more tangible,
we additionally provide operational definitions of content and intent words as those words that should match, and those that need not match, respectively, in the text of relevant documents. In addition to a standard evaluation against human annotations, we also provide an alternative validation of our ideas using clickthrough data. Concordance of the two orthogonal evaluation approaches provide further support to our original hypothesis of the existence of two distinct word classes in search queries. Finally, we provide a taxonomy of intent words derived through rigorous manual analysis of large query logs.

Full Text: PDF
Type of Paper: Research Paper
Keywords: Query understanding, Query intent, Intent words, Co-occurrence entropy
Show BibTex format: BibTeX