HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. While research on articulated human motion and pose estimation has progressed rapidly in the last few years, there has been no systematic quantitative evaluation of competing methods to establish the current state of the art. We present data obtained using a hardware system that is able to capture synchronized video and ground-truth 3D motion. The resulting HumanEva datasets contain multiple subjects performing a set of predefined actions with a number of repetitions. On the order of 40,000 frames of synchronized motion capture and multi-view video (resulting in over one quarter million image frames in total) were collected at 60 Hz with an additional 37,000 time instants of pure motion capture data. A standard set of error measures is defined for evaluating both 2D and 3D pose estimation and tracking algorithms. We also describe a baseline algorithm for 3D articulated tracking that uses a relatively standard Bayesian framework with optimization in the form of Sequential Importance Resampling and Annealed Particle Filtering. In the context of this baseline algorithm we explore a variety of likelihood functions, prior models of human motion and the effects of algorithm parameters. Our experiments suggest that image observation models and motion priors play important roles in performance, and that in a multi-view laboratory environment, where initialization is available, Bayesian filtering tends to perform well. The datasets and the software are made available to the research community. This infrastructure will support the development of new articulated motion and pose estimation algorithms, will provide a baseline for the evaluation and comparison of new methods, and will help establish the current state of the art in human pose estimation and tracking.

References in zbMATH (referenced in 18 articles )

Showing results 1 to 18 of 18.
Sorted by year (citations)

  1. Escalera, Sergio; Athitsos, Vassilis; Guyon, Isabelle: Challenges in multimodal gesture recognition (2016) ioport
  2. Elhoseiny, Mohamed; Elgammal, Ahmed: Generalized twin Gaussian processes using Sharma-Mittal divergence (2015)
  3. Eslami, S.M.Ali; Heess, Nicolas; Williams, Christopher K.I.; Winn, John: The shape Boltzmann machine: a strong model of object shape (2014)
  4. Gonczarek, Adam; Tomczak, Jakub M.: Manifold regularized particle filter for articulated human motion tracking (2014)
  5. Ma, Zhigang; Yang, Yi; Nie, Feiping; Sebe, Nicu; Yan, Shuicheng; Hauptmann, Alexander G.: Harnessing lab knowledge for real-world action recognition (2014)
  6. Ren, Carl Yuheng; Prisacariu, Victor; Reid, Ian: Regressing local to global shape properties for online segmentation and tracking (2014) ioport
  7. Yamada, Makoto; Sigal, Leonid; Chang, Yi: Domain adaptation for structured regression (2014)
  8. Ellis, Chris; Masood, Syed Zain; Tappen, Marshall F.; Laviola, Joseph J.jun.; Sukthankar, Rahul: Exploring the trade-off between accuracy and observational latency in action recognition (2013) ioport
  9. Li, Yi; Sun, Zhengxing: Articulated human motion tracking using sequential immune genetic algorithm (2013) ioport
  10. Zhang, Dianyong; Miao, Zhenjiang; Chen, Shengyong; Wan, Lili: Optimization and soft constraints for human shape and pose estimation based on a 3D morphable model (2013) ioport
  11. Han, Hong; Tong, Minglei; Chen, Zhichao; Fan, Youjian: Variable structure multiple model for articulated human motion tracking from monocular video sequences (2012) ioport
  12. Hofmann, M.; Gavrila, D.M.: Multi-view 3D human pose estimation in complex environment (2012) ioport
  13. Rogez, Grégory; Rihan, Jonathan; Orrite-Uruñuela, Carlos; Torr, Philip H.S.: Fast human pose detection using randomized hierarchical cascades of rejectors (2012) ioport
  14. Schwarz, Loren Arthur; Mateus, Diana; Navab, Nassir: Recognizing multiple human activities and tracking full-body pose in unconstrained environments (2012) ioport
  15. Sigal, Leonid; Isard, Michael; Haussecker, Horst; Black, Michael J.: Loose-limbed people: estimating 3D human pose and motion using non-parametric belief propagation (2012)
  16. Tsai, David; Flagg, Matthew; Nakazawa, Atsushi; Rehg, James M.: Motion coherent tracking using multi-label MRF optimization (2012) ioport
  17. Yao, Angela; Gall, Juergen; Van Gool, Luc: Coupled action recognition and pose estimation from multiple views (2012)
  18. Sigal, Leonid; Balan, Alexandru O.; Black, Michael J.: HumanEva: synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion (2010) ioport