IBAS: An Infinite Buffer Abstraction for Streaming
Loading...
Date
2019-04-26
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Recent technological trends have resulted in the creation, manipulation and storage of vast amounts of information. Effectively dealing with these data volumes requires scalable data computing capable of functioning at high performance and making use of large distributed computing systems. One of the most prominent frameworks for tackling this problem is MapReduce, which abstracts the process of parallelizing and processing a computation across a large dataset. Current open-source implementations of MapReduce, however, are lacking in key aspects: performance, efficiency, and size (or ‘bloat’). To overcome this challenge, new abstractions are necessary in order to provide the lightweight scalability and speed required. In particular, this research examines algorithms for a new infinite-stream abstraction called IBAS, which is no longer limited by the size of the virtual memory supported by the CPU or operating system. Besides having unlimited size, this architecture eliminates the kernel overhead needed for page remapping, which should make it faster than previous streaming abstractions under many streaming conditions.
Description
Keywords
Streaming, Windows, MapReduce, Abstraction