Peter Pietzuch
Imperial College London
6 February 2014, 04:15 pm
S2|02 Raum C110, Robert-Piloty-Gebäude, Hochschulstr. 10, 64289 Darmstadt
“Adapting the Data Centre Network for Big Data Processing”
Abstract:
Distributed platforms for processing “big data”, such as Hadoop, Spark and Storm, use extensively network resources in data centres when handling large volumes of data. Their performance is therefore often network-bound due to their communication patterns: data is distributed over many servers, which leads to network contention caused by many-to-few, high-bandwidth traffic flows between servers. In this talk, I argue that this is one instance of a class of problems that can be solved by adapting the data centre network to application specific requirements. We term this approach “network-as-a-service” (NaaS) because it allows applications to customise the service that they receive from the network. As an example of the NaaS approach, I describe how an in-network processing service can aggregate data along network paths according to application-specific functions. This alleviates hotspots at the network edge and thus improves the performance of big data platforms.
Bio:
Peter researches on the design and engineering of scalable, reliable and secure large-scale software systems with a particular interest in data management issues. Focus topics include event processing, peer-to-peer systems, and global sensing applications. Peter obtained his PhD from University of Cambridge (UK) and was a post-doctoral fellow in the Systems Research group at Harvard University (Cambridge, MA). He is currently heading the Largescale Distributed Systems group in the Department of Computing at Imperial College London (UK) as a Senior Lecturer.
