http://iet.metastore.ingenta.com
1887

SDN helps Big Data to optimize access to data

SDN helps Big Data to optimize access to data

For access to this article, please select a purchase option:

Buy chapter PDF
£10.00
(plus tax if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
Big Data and Software Defined Networks — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

This chapter introduces the state of the art in the emerging area of combining high performance computing (HPC) with Big Data Analysis. To understand the new area, the chapter first surveys the existing approaches to integrating HPC with Big Data. Next, the chapter introduces several optimization solutions that focus on how to minimize the data transfer time from computation-intensive applications to analysis intensive applications as well as minimizing the end-to-end time-to-solution. The solutions utilize Software Defined Network (SDN) to adaptively use both high speed interconnect network and high performance parallel file systems to optimize the application performance. A computational framework called DataBroker is designed and developed to enable a tight integration of HPC with data analysis. Multiple types of experiments have been conducted to show different performance issues in both message passing and parallel file systems and to verify the effectiveness of the proposed research approaches.

Chapter Contents:

  • 14.1 Introduction
  • 14.2 State of the art and related work
  • 14.3 Performance analysis of message passing and parallel file system I/O
  • 14.4 Analytical modeling-based end-to-end time optimization
  • 14.4.1 The problem
  • 14.4.2 The traditional method
  • 14.4.3 Improved version of the traditional method
  • 14.4.4 The fully asynchronous pipeline method
  • 14.4.5 Microbenchmark for the analytical model
  • 14.5 Design and implementation of DataBroker for the fully asynchronous method
  • 14.6 Experiments with synthetic and real applications
  • 14.6.1 Synthetic and real-world applications
  • 14.6.2 Accuracy of the analytical model
  • 14.6.3 Performance speedup
  • 14.7 Open issues and challenges
  • 14.8 Conclusion
  • Acknowledgments
  • References

Inspec keywords: software defined networking; Big Data; parallel processing; storage management; data analysis

Other keywords: DataBroker; application performance optimization; computation-intensive applications; data transfer time minimization; analysis intensive applications; high performance computing; SDN; data access optimization; high speed interconnect network; Big Data Analysis; computational framework; message passing; high performance parallel file systems; HPC; software defined network

Subjects: Parallel software; Computer communications; File organisation; Data handling techniques; Computer networks and techniques

Preview this chapter:
Zoom in
Zoomout

SDN helps Big Data to optimize access to data, Page 1 of 2

| /docserver/preview/fulltext/books/pc/pbpc015e/PBPC015E_ch14-1.gif /docserver/preview/fulltext/books/pc/pbpc015e/PBPC015E_ch14-2.gif

Related content

content/books/10.1049/pbpc015e_ch14
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address