Improved user RTSE experience on the web through fast retrieval of social media content

Web 2.0 applications like Twitter and Facebook are producing real-time content continuously, at a very rapid pace, courtesy the growing interest of the masses in the social media. Real-time search engines (RTSE), which have the capacity to search social media content, like Topsy, Bing etc are currently in their initial stage, due to low coverage on web, lack of full access to APIs of social media websites and inconvenient user experience. In this paper, two issues related to fast retrieval of real-time content are presented; first issue is indexing and second being handling of real-time content spikes. Indexing has been discussed as a technique that causes overhead in terms of time consumed to display the results to the user and result spikes are discussed in context of bulk data being produced on the social media and its availability and readability to the user. This has greatly enhanced not only our learning in this area but also reduced the delays in generation of result substantially. Finally, based on these issues, current RTSEs are analyzed and recommendations of a framework are proposed. Standard practices of current RTSEs are introduced, and a framework for proposed recommendations is described in order to improve the efficiency of the system. The framework revolves around the use of indexes by the real-time search engines. It also discusses that how can indexes be avoided to increase the efficiency and throughput of the system for increased user satisfaction.