GraphChi
GraphChi: Large-Scale Graph Computation on Just a PC. Current systems for graph computation require a dis- tributed computing cluster to handle very large real-world problems, such as analysis on social networks or the web graph. While distributed computational resources have be- come more accessible, developing distributed graph algo- rithms still remains challenging, especially to non-experts. In this work, we present GraphChi, a disk-based system for computing efficiently on graphs with billions of edges. By using a well-known method to break large graphs into small parts, and a novel parallel sliding windows method, GraphChi is able to execute several advanced data mining, graph mining, and machine learning algorithms on very large graphs, using just a single consumer-level computer. We further extend GraphChi to support graphs that evolve over time, and demonstrate that, on a single computer, GraphChi can process over one hundred thousand graph updates per second, while simultaneously performing com- putation. We show, through experiments and theoretical analysis, that GraphChi performs well on both SSDs and rotational hard drives. By repeating experiments reported for existing dis- tributed systems, we show that, with only fraction of the resources, GraphChi can solve the same problems in very reasonable time. Our work makes large-scale graph com- putation available to anyone with a modern PC
Keywords for this software
References in zbMATH (referenced in 5 articles )
Showing results 1 to 5 of 5.
Sorted by year (- Lee, Dongha; Oh, Jinoh; Yu, Hwanjo: \textscOCam: out-of-core coordinate descent algorithm for matrix completion (2020)
- Ahmed, Aly; Thomo, Alex: Computing source-to-target shortest paths for complex networks in RDBMS (2017)
- Broß, Jan; Gog, Simon; Hauck, Matthias; Paradies, Marcus: Fast construction of compressed web graphs (2017)
- Zhao, Junzhou; Wang, Pinghui; Lui, John C. S.; Towsley, Don; Guan, Xiaohong: I/O-efficient calculation of (H)-group closeness centrality over disk-resident graphs (2017)
- Sonobe, Tomohiro: An efficient Monte Carlo approach to compute PageRank for large graphs on a single PC (2016)