DeepFlow: Large Displacement Optical Flow with Deep Matching. Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox and Malik, our approach, termed Deep Flow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable asset for integration into an energy minimization framework for optical flow estimation. Deep Flow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset.
Keywords for this software
References in zbMATH (referenced in 10 articles )
Showing results 1 to 10 of 10.
- Pengpeng Liu, Irwin King, Michael R.Lyu, Jia Xu: DDFlow: Learning Optical Flow with Unlabeled Data Distillation (2019) arXiv
- Wang, Song; Wang, Zengfu: Optical flow estimation with occlusion detection (2019)
- Kuzmin, A. I.: Learning the regularization operator for the optical flow problem (2018)
- Nakahata, Mateus T.; Thomaz, Lucas A.; da Silva, Allan F.; da Silva, Eduardo A. B.; Netto, Sergio L.: Anomaly detection with a moving camera using spatio-temporal codebooks (2018)
- Rodríguez, Mariano; Delon, Julie; Morel, Jean-Michel: Covering the space of tilts. Application to affine invariant image comparison (2018)
- E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox: FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (2016) arXiv
- Estellers, V.; Soatto, S.: Detecting occlusions as an inverse problem (2016)
- Fortun, Denis; Bouthemy, Patrick; Kervrann, Charles: A variational aggregation framework for patch-based optical flow estimation (2016)
- Demetz, Oliver; Hafner, David; Weickert, Joachim: Morphologically invariant matching of structures with the complete rank transform (2015)
- Fortun, Denis; Bouthemy, Patrick; Kervrann, Charles: Sparse aggregation framework for optical flow estimation (2015)