TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation. Image auto-annotation is an important open problem in computer vision. For this task we propose TagProp, a discriminatively trained nearest neighbor model. Tags of test images are predicted using a weighted nearest-neighbor model to exploit labeled training images. Neighbor weights are based on neighbor rank or distance. TagProp allows the integration of metric learning by directly maximizing the log-likelihood of the tag predictions in the training set. In this manner, we can optimally combine a collection of image similarity metrics that cover different aspects of image content, such as local shape descriptors, or global color histograms. We also introduce a word specific sigmoidal modulation of the weighted neighbor tag predictions to boost the recall of rare words. We investigate the performance of different variants of our model and compare to existing work. We present experimental results for three challenging data sets. On all three, TagProp makes a marked improvement as compared to the current state-of-the-art.

References in zbMATH (referenced in 17 articles )

Showing results 1 to 17 of 17.
Sorted by year (citations)

  1. Ge, Hongwei; Yan, Zehang; Dou, Jing; Wang, Zhen; Wang, ZhiQiang: A semisupervised framework for automatic image annotation based on graph embedding and multiview nonnegative matrix factorization (2018)
  2. Li, Yaoyi; Lu, Hongtao: On multi-modal fusion learning in constraint propagation (2018)
  3. Wu, Baoyuan; Jia, Fan; Liu, Wei; Ghanem, Bernard; Lyu, Siwei: Multi-label learning with missing labels using mixed dependency graphs (2018)
  4. Xue, Zhe; Li, Guorong; Huang, Qingming: Joint multi-view representation and image annotation via optimal predictive subspace learning (2018)
  5. Liu, Weiwei; Tsang, Ivor W.: Making decision trees feasible in ultrahigh feature and label dimensions (2017)
  6. Verma, Yashaswi; Jawahar, C. V.: Image annotation by propagating labels from semantic neighbourhoods (2017)
  7. Zang, Miao; Xu, Huimin; Zhang, Yongmei: Kernel-based multiview joint sparse coding for image annotation (2017)
  8. Hamid Amiri, S.; Jamzad, Mansour: Efficient multi-modal fusion on supergraph for scalable image annotation (2015)
  9. Gong, Yunchao; Ke, Qifa; Isard, Michael; Lazebnik, Svetlana: A multi-view embedding space for modeling Internet images, tags, and their semantics (2014) ioport
  10. Guillaumin, Matthieu; Küttel, Daniel; Ferrari, Vittorio: ImageNet auto-annotation with segmentation propagation (2014) ioport
  11. Wang, Mei; Xia, Xiaoling; Le, Jiajin; Zhou, Xiangdong: Effective automatic image annotation via integrated discriminative and generative models (2014) ioport
  12. Li, Zechao; Liu, Jing; Xu, Changsheng; Lu, Hanqing: MLRank: multi-correlation learning to rank for image annotation (2013)
  13. McAuley, Julian J.; Ramisa, Arnau; Caetano, Tibério S.: Optimization of robust loss functions for weakly-labeled image taxonomies (2013)
  14. Guillaumin, Matthieu; Mensink, Thomas; Verbeek, Jakob; Schmid, Cordelia: Face recognition from caption-based supervision (2012)
  15. Miao, Xu; Rao, Rajesh P. N.: Fast structured prediction using large margin sigmoid belief networks (2012)
  16. Dimitrovski, Ivica; Kocev, Dragi; Loskovska, Suzana; Džeroski, Sašo: Hierarchical annotation of medical images (2011) ioport
  17. Weston, Jason; Bengio, Samy; Usunier, Nicolas: Large scale image annotation: learning to rank with joint word-image embeddings (2010)