A Study on Big Data Engineering Using Cloud Data Warehouse

In the current smart, Internet of Things world, millions of devices are connected around us for effective communication. A huge volume of data is getting generated, varying from gigabytes to brontobytes with a variety of data formats, storing all the acquired data in an on‐premise Data warehouse has limitations due to storage capacity and homogeneous data formats compatibility. The internet revolution has brought us an entirely new communication and cloud model. We now have fast wireless connectivity for data transfer, messaging and web surfing, and we also perform extract, transform and Load (ETL) activities to the cloud data warehouse based on the client's requirements. The internet revolution has been the core inspiration for many new business ideas and models. The rise in online retailing business is strictly dependent on internet connectivity. Even consumers find online shopping convenient and time friendly. Internet communication is a must for urban life. Mobile internet helps us keep things handy and organized. We now can transfer or share a heavy size file from a portable handy device in a couple of minutes. A lot of businesses use internet for e‐commerce, media site of a company, social media, Customer relationship management, Employee productivity management, etc., and due to this evolution, business faces a lot of issues with the storing of a large amount of data and analysing it over time. This is where Data Warehouse was born. Data warehouse is a central repository which is built to in‐house the data acquired from various source systems in the enterprise business ecosystem. The intention is to support the decision makers with appropriate reporting facility to arrive at the right decisions to the right time. An on‐premise data warehouse is built to handle online analytical processing with moderate cost for moderate business enterprises. After a while the businesses faced issues in installing the components to store lot of data on‐premise and it was not cost‐effective. To solve this problem, cloud data warehouse was evolved. We explored the best possible solutions to handle the limitations of on‐premise data warehouse with comparative study.