CYCLADES: Conflict-free Asynchronous Machine Learning. We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent conflict-free nature and cache locality, our multi-core implementation of CYCLADES consistently outperforms HOGWILD!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to the HOGWILD! implementation of SGD, and up to 5x gains over asynchronous implementations of variance reduction algorithms.
Keywords for this software
References in zbMATH (referenced in 3 articles )
Showing results 1 to 3 of 3.
- Kaya, Kamer; Öztoprak, Figen; Birbil, Ş. İlker; Cemgil, A. Taylan; Şimşekli, Umut; Kuru, Nurdan; Koptagel, Hazal; Öztürk, M. Kaan: A framework for parallel second order incremental optimization algorithms for solving partially separable problems (2019)
- Leblond, Rémi; Pedregosa, Fabian; Lacoste-Julien, Simon: Improved asynchronous parallel optimization analysis for stochastic incremental methods (2018)
- Zhang, Junyu; Liu, Haoyang; Wen, Zaiwen; Zhang, Shuzhong: A sparse completely positive relaxation of the modularity maximization for community detection (2018)