From NetSysLab

Jump to: navigation, search

An unprecedented growth in data generation is taking place. Data is accumulated for larger dynamic systems, capturing events at increasingly fine granularity, and with processing requirements that approach real-time. This explosion in data growth and processing demands has not been matched by improved algorithmic or infrastructure techniques, with research still focusing on analyzing data post mortem. To keep up, data-analytics pipelines need to be viable at massive scale, and switch away from static, post-processing focused offline scenarios to support fully online analysis.

The challenges this work aims to tackle are twofold. First, the feasibility of online massive scale analytics for dynamic graph processing must be shown. Performance, scalability, and programmer efficiency (complexity hiding) are all important success metrics to this end. Secondly, the characteristics and capabilities of dynamic queries that react to underlying data changes must be exposed. This means creating novel versions of common static algorithms, which can operate in a dynamic environment.


Scott Sallinen
Matei Ripeanu


[2] Graph Colouring as a Challenge Problem for Dynamic Graph Processing on Distributed Systems, Scott Salinnen, Keita Iwabuchi, Suraj Poudel, Roger Pearce, Matei Ripeanu, IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2016), Salt Lake City, UT November 2016 (acceptance rate: 82/446=18.3%) pdf slides
[1] Systems for Near Real-Time Analysis of Large-Scale Dynamic Graphs, Luis M. Vaquero, Felix Cuadrado, Matei Ripeanu, Technical Report arXiv:1410.1903, October 2014.

Related Projects