Nowadays, a huge quantity of data are produced by means of multiple data sources. The existing tools and techniques are not capable of handling such voluminous data produced from a variety of sources. This continuous and varied generation of data requires advanced technologies for processing and storage, which seems to be a big challenge for data scientists. Some research studies are well defined in the area of streaming in big data. Streaming data are the real-time data or data in motion such as stock market data, sensor data, GPS data and twitter data. In stream processing, the data are not stored in databases instead it is processed and analyzed on the fly to get the value as soon as they are generated. There are a number of streaming frameworks proposed till date for big data applications that are used to pile up, evaluate and process the data that are generated and captured continuously. In this chapter, we provide an in-depth summary of various big data streaming approaches like Apache Storm, Apache Hive and Apache Samza. We also presented a comparative study regarding these streaming platforms.

Chapter Contents:

• 5.1 Introduction
• 5.2 Real-time data processing topology
• 5.2.1 Choosing the platform
• 5.2.2 Entry points
• 5.2.3 Data processing infrastructure
• 5.2.3.1 Formality layer
• 5.2.3.2 Filtering layer
• 5.2.3.3 Analytics layer
• 5.2.3.4 Storing layer
• 5.3 Streaming processing
• 5.4 Stream mining
• 5.4.1 Clustering
• 5.4.2 Classification
• 5.4.3 Frequent
• 5.4.4 Outlier and anomaly detection
• 5.5 Lambda architecture
• 5.6 Stream processing approach for big data
• 5.6.1 Apache Spark
• 5.6.2 Apache Flink
• 5.6.3 Apache Samza
• 5.6.4 Apache Storm
• 5.6.5 Apache Flume
• 5.6.6 Apache Kafka
• 5.7 Evaluation of data streaming processing approaches
• 5.8 Conclusion
• Acknowledgment
• References

Inspec keywords: social networking (online); Big Data; Global Positioning System

Other keywords: Apache Hive; Apache Samza; stream processing; twitter data; Apache Storm; GPS data; Big Data analytics; real-time data processing

Subjects: Data handling techniques; Information networks