Data consolidation solution for internal security needs

The threats of the 21st century are too complex, difficult and time consuming to discern with traditional intelligence practices that shun advances in information technology and rely heavily on human experts. Good information is fundamental to understand and respond to 21st century national security threats. Without comprehensive information, decision-makers operate with a limited understanding of the threat horizon or the best means to address it. Required information exists across a variety of proprietary and open sources, and the volume of data available that might potentially contain relevant facts is simply too large and the bandwidth of the trained analysts is limited. Such information must be available in time-critical situations to be able to quickly connect the dots across various related pieces of information. It is imperative that decision-makers are provided intelligent tools that can automatically extract new relevant information from data without being explicitly asked, leading to actionable intelligence. To overcome these challenges we propose an information collection, management and analysis framework to meet the ever-growing threats to national security. The proposed framework establishes a collaborative environment to semi-automatically generate actionable intelligence by ensuring that the right people have access to all inclusive information at the right time. The core of this framework is to create a single view of entity by correlating information from different sources, stored in different formats. These sources can be passport, immigration, driving license, FIR records, Telecom and Utility services. The correlation algorithm is able to handle varied amount of noise in the data such as syntactic and semantic variations, format changes, spelling error, incomplete data, regional and linguistics variation as well as addition or removal of fields. The framework can further exploit the consolidated view to discover relationships between entities thus expanding the reach for relevant information. The framework provides multiple avenues of interaction and the foresight needed to incorporate new sources of data as they arise in future.