Data Analytics

Big Data analytics explores a vast amount of data to uncover patterns, insights, correlations within data. Big Data analytics gives organizations opportunities and visibility of the organization. One of the fundamental sources of data analytics is log collection. This chapter focuses on Apache Flume and Fluentd. Flume has the concepts of events where each event is a unit of data flowing through Flume agents. Fluentd offers a buffering mechanism to overcome any issue or flakiness on downstream consumers. The chapter looks at different elements of Big Data set transferring and tools to use. Aggregating data provides efficiency when data is queried. Data aggregation enables engineers, analysts, and managers to get insights about data quicker. The chapter presents some of the common workflow management tools widely used for Big Data pipelines. Building effective Big Data pipelines require assessing the relative merits of data in the context of business. Interactive data visualization connects the dots with analytics.