Introduction to big data

Introduction to big data

For access to this article, please select a purchase option:

Buy chapter PDF
(plus tax if applicable)
Buy Knowledge Pack
10 chapters for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
Big Data and Software Defined Networks — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

The amount of data generated during the last few years has been unprecedented. This is not only due to the prevalence of online social networks and the ubiquitous devices connected to the Internet but also as the result of the advances in technology across other fields, for instance, whole genome sequencing. Hence, it is fair to say that we are living in the era of big data. Big data refers to large datasets or data flows that have outpaced our capability to store and process and cannot be analyzed by traditional means. In the presence of these challenges, traditional platforms fail to show the expected performance, and thus, new systems for storing and processing large-scale data are crucial to emerge. In this chapter, we explore some of the new trends of technology for handling big data.

Chapter Contents:

  • 5.1 Big data platforms: challenges and requirements
  • 5.2 How to store big data?
  • 5.2.1 Distributed file systems
  • GFS and HDFS
  • 5.2.2 Messaging systems
  • Kafka
  • 5.2.3 NoSQL databases
  • Dynamo
  • BigTable and HBase
  • 5.3 How to process big data?
  • 5.3.1 Batch data processing platforms
  • MapReduce
  • Spark
  • 5.3.2 Streaming data processing platforms
  • Spark Streaming
  • Storm
  • Flink
  • 5.3.3 Graph data processing platforms
  • Pregel
  • GraphLab
  • PowerGraph
  • GraphX
  • 5.3.4 Structured data processing platforms
  • Hive
  • Spark SQL
  • 5.4 Concluding remarks
  • References

Inspec keywords: Big Data

Other keywords: ubiquitous devices; Big Data; data flows; data storage; Internet; genome sequencing; online social networks; large-scale data processing

Subjects: Data handling techniques

Preview this chapter:
Zoom in

Introduction to big data, Page 1 of 2

| /docserver/preview/fulltext/books/pc/pbpc015e/PBPC015E_ch5-1.gif /docserver/preview/fulltext/books/pc/pbpc015e/PBPC015E_ch5-2.gif

Related content

This is a required field
Please enter a valid email address