Beyond bag-of-words: Bigram-enhanced context-dependent term weights

While term independence is a widely held assumption in most of the established information retrieval approaches, it is clearly not true and various works in the past have investigated a relaxation of the assumption. One approach is to use n‐grams in document representation instead of unigrams. However, the majority of early works on n‐grams obtained only modest performance improvement. On the other hand, the use of information based on supporting terms or “contexts” of queries has been found to be promising. In particular, recent studies showed that using new context‐dependent term weights improved the performance of relevance feedback (RF) retrieval compared with using traditional bag‐of‐words BM25 term weights. Calculation of the new term weights requires an estimation of the local probability of relevance of each query term occurrence. In previous studies, the estimation of this probability was based on unigrams that occur in the neighborhood of a query term. We explore an integration of the n‐gram and context approaches by computing context‐dependent term weights based on a mixture of unigrams and bigrams. Extensive experiments are performed using the title queries of the Text Retrieval Conference (TREC)‐6, TREC‐7, TREC‐8, and TREC‐2005 collections, for RF with relevance judgment of either the top 10 or top 20 documents of an initial retrieval. We identify some crucial elements needed in the use of bigrams in our methods, such as proper inverse document frequency (IDF) weighting of the bigrams and noise reduction by pruning bigrams with large document frequency values. We show that enhancing context‐dependent term weights with bigrams is effective in further improving retrieval performance.