Pregel

Pregel: a system for large-scale graph processing. Many practical computing problems concern large graphs. Standard examples include the Web graph and various social networks. The scale of these graphs - in some cases billions of vertices, trillions of edges - poses challenges to their efficient processing. In this paper we present a computational model suitable for this task. Programs are expressed as a sequence of iterations, in each of which a vertex can receive messages sent in the previous iteration, send messages to other vertices, and modify its own state and that of its outgoing edges or mutate graph topology. This vertex-centric approach is flexible enough to express a broad set of algorithms. The model has been designed for efficient, scalable and fault-tolerant implementation on clusters of thousands of commodity computers, and its implied synchronicity makes reasoning about programs easier. Distribution-related details are hidden behind an abstract API. The result is a framework for processing large graphs that is expressive and easy to program.

This software is also peer reviewed by journal TOMS.


References in zbMATH (referenced in 33 articles )

Showing results 1 to 20 of 33.
Sorted by year (citations)

1 2 next

  1. Becchetti, Luca; Clementi, Andrea E.; Natale, Emanuele; Pasquale, Francesco; Trevisan, Luca: Find your place: simple distributed algorithms for community detection (2020)
  2. Li, Qi; Zhong, Jiang; Cao, Zehong; Li, Xue: Optimizing streaming graph partitioning via a heuristic greedy method and caching strategy (2020)
  3. Montealegre, P.; Perez-Salazar, S.; Rapaport, I.; Todinca, I.: Graph reconstruction in the congested clique (2020)
  4. Alsinet, Teresa; Argelich, Josep; Béjar, Ramón; Cemeli, Joel: A distributed argumentation algorithm for mining consistent opinions in weighted Twitter discussions (2019)
  5. Aydin, Kevin; Bateni, Mohammadhossein; Mirrokni, Vahab: Distributed balanced partitioning via linear embedding (2019)
  6. Brandt, Sebastian; Wattenhofer, Roger: Approximating small balanced vertex separators in almost linear time (2019)
  7. Das, Ariyam; Zaniolo, Carlo: A case for stale synchronous distributed model for declarative recursive computation (2019)
  8. Joana M. F. da Trindade, Konstantinos Karanasos, Carlo Curino, Samuel Madden, Julian Shun: Kaskade: Graph Views for Efficient Graph Analytics (2019) arXiv
  9. Marquer, Yoann; Gava, Frédéric: Axiomatization and characterization of BSP algorithms (2019)
  10. Arleo, Alessio; Didimo, Walter; Liotta, Giuseppe; Montecchiani, Fabrizio: GiVip: a visual profiler for distributed graph processing systems (2018)
  11. R. Brisaboa, Nieves; Caro, Diego; Fariña, Antonio; Andrea Rodriguez, M.: Using compressed suffix-arrays for a compact representation of temporal-graphs (2018)
  12. Wang, Hongzhi; Li, Ning; Li, Jianzhong; Gao, Hong: Parallel algorithms for flexible pattern matching on big graphs (2018)
  13. Ahmed, Aly; Thomo, Alex: Computing source-to-target shortest paths for complex networks in RDBMS (2017)
  14. Broß, Jan; Gog, Simon; Hauck, Matthias; Paradies, Marcus: Fast construction of compressed web graphs (2017)
  15. Cui, Huanqing; Niu, Jian; Zhou, Chuanai; Shu, Minglei: A multi-threading algorithm to detect and remove cycles in vertex- and arc-weighted digraph (2017)
  16. Fegaras, Leonidas: An algebra for distributed Big Data analytics (2017)
  17. Hong, Jihye; Park, Kisung; Han, Yongkoo; Rasel, Mostofa Kamal; Vonvou, Dawanga; Lee, Young-Koo: Disk-based shortest path discovery using distance index over large dynamic graphs (2017)
  18. Lluch Lafuente, Alberto; Loreti, Michele; Montanari, Ugo: Asynchronous distributed execution of fixpoint-based computational fields (2017)
  19. Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, Ion Stoica: Ray: A Distributed Framework for Emerging AI Applications (2017) arXiv
  20. Arleo, Alessio; Didimo, Walter; Liotta, Giuseppe; Montecchiani, Fabrizio: A distributed multilevel force-directed algorithm (2016)

1 2 next