Collective Intelligence in Action

There's a great deal of wisdom in a crowd, but how do you listen to a thousand people talking at once? Identifying the wants, needs, and knowledge of internet users can be like listening to a mob.In the Web 2.0 era, leveraging the collective power of user contributions, interactions, and feedback is the key to market dominance. A new category of powerful programming techniques lets you discover the patterns, inter-relationships, and individual profiles—the collective intelligence—locked in the data people leave behind as they surf websites, post blogs, and interact with other users.Collective Intelligence in Action is a hands-on guidebook for implementing collective-intelligence concepts using Java. It is the first Java-based book to emphasize the underlying algorithms and technical implementation of vital data gathering and mining techniques like analyzing trends, discovering relationships, and making predictions. It provides a pragmatic approach to personalization by combining content-based analysis with collaborative approaches.About the TechnologyAbout the BookFollowing a running example in which you harvest and use information from blogs, you learn to develop software that you can embed in your own applications. The code examples are immediately reusable and give the Java developer a working collective intelligence toolkit.Along the way, you work with a number of APIs and open-source toolkits including text analysis and search using Lucene, web-crawling using Nutch, and applying machine learning algorithms using WEKA and the Java Data Mining (JDM) standard.What's InsideArchitecture for embedding intelligence in your applicationDeveloping metadata about the user and contentGather intelligence from tagging and build tag cloudsIntroduction to intelligent web crawling and NutchHarvesting information from the blogosphereBuild a text analysis toolkit leveraging LuceneBusiness intelligence and data mining for recommendations and promotionsLeveraging open-source data mining toolkit WEKA and the Java Data Mining (JDM) standardIncorporating intelligent search in your applicationBuilding a recommendation engine—finding related users and contentReal-world case studies of Amazon, Google News, and Netflix personalization.About the ReaderThis book assumes you have a basic level of Java coding skills.About the AuthorSatnam Alag, PhD, is currently the Vice President of Engineering at NextBio, a vertical search engine and a Web 2.0 collaboration application for the lif