A Solution to Query Processing Challenges Through Smart Query Processor for Big Data Analytics

Data generation and collection is a continuous process throughout the world. It is going to be difficult to handle various challenges that arise because of various categories of data collected through various sources. The journey of any data analytics work starts with preprocessing. This is the only biggest challenge that takes much time for data separation and categorization. Once this step completed, then by applying several tools, practitioners can process it and can move forward to the next step of centralization, indexing, etc. Many scholars are putting their efforts to get quick responses from the information and hence data analytics task is getting easier. Previous work focused on several challenging issues and provide solutions to issues on preprocessing and data management. Now proposed a big data processing framework Smart Query Processor—SQP which provides the solution to challenges in query processing and processes about 500 GB of data. This paper describes a novel approach using hybrid algorithms and got results in 5X times faster than existing approaches. Finally, compared the results with the previously published work achieved an accuracy of up to 95–96%. In the future, the work will be extend to process several TB of data on highly configured workstations available in the labs.