Search Engine Quality Evaluation Via Replaying Search Queries

In an example embodiment, traditional offline analysis of search engine quality is modified to provide multiple replays of queries against new versions of search engine algorithms. This helps to evaluate quality of new versions of search engine algorithms while reducing or eliminating parity issues without increasing network bandwidth utilization.