SimpleMKL

SimpleMKL. Multiple kernel learning (MKL) aims at simultaneously learning a kernel and the associated predictor in supervised learning settings. For the support vector machine, an efficient and general multiple kernel learning algorithm, based on semi-infinite linear programming, has been recently proposed. This approach has opened new perspectives since it makes MKL tractable for large-scale problems, by iteratively using existing support vector machine code. However, it turns out that this iterative algorithm needs numerous iterations for converging towards a reasonable solution. In this paper, we address the MKL problem through a weighted 2-norm regularization formulation with an additional constraint on the weights that encourages sparse kernel combinations. Apart from learning the combination, we solve a standard SVM optimization problem, where the kernel is defined as a linear combination of multiple kernels. We propose an algorithm, named SimpleMKL, for solving this MKL problem and provide a new insight on MKL algorithms based on mixed-norm regularization by showing that the two approaches are equivalent. We show how SimpleMKL can be applied beyond binary classification, for problems like regression, clustering (one-class classification) or multiclass classification. Experimental results show that the proposed algorithm converges rapidly and that its efficiency compares favorably to other MKL algorithms. Finally, we illustrate the usefulness of MKL for some regressors based on wavelet kernels and on some model selection problems related to multiclass classification problems.


References in zbMATH (referenced in 65 articles )

Showing results 1 to 20 of 65.
Sorted by year (citations)

1 2 3 4 next

  1. Aziznejad, Shayan; Unser, Michael: Multikernel regression with sparsity constraint (2021)
  2. Han, Biao; Shang, Chao; Huang, Dexian: Multiple kernel learning-aided robust optimization: learning algorithm, computational tractability, and usage in multi-stage decision-making (2021)
  3. Wang, Peiyan; Cai, Dongfeng: Multiple kernel learning by empirical target kernel (2020)
  4. Won, Daehan; Manzour, Hasan; Chaovalitwongse, Wanpracha: Convex optimization for group feature selection in networked data (2020)
  5. Shen, Yanning; Chen, Tianyi; Giannakis, Georgios B.: Random feature-based online multi-kernel learning in environments with unknown dynamics (2019)
  6. Tang, Jingjing; Tian, Yingjie; Liu, Dalian; Kou, Gang: Coupling privileged kernel method for multi-view learning (2019)
  7. Suzuki, Taiji: Learning theory of multiple kernel learning (2018)
  8. Tang, Jingjing; Tian, Yingjie; Liu, Xiaohui; Li, Dewei; Lv, Jia; Kou, Gang: Improved multi-view privileged support vector machine (2018)
  9. Yan, Caixia; Luo, Minnan; Liu, Huan; Li, Zhihui; Zheng, Qinghua: Top-(k) multi-class SVM using multiple features (2018)
  10. Zhao, Jinwei; Hei, Xinhong; Shi, Zhenghao; Dong, Longlei; Liu, Yu; Yan, Ruiping; Li, Xiuxiu: Regression learning based on incomplete relationships between attributes (2018)
  11. Lan, Liang; Zhang, Kai; Ge, Hancheng; Cheng, Wei; Liu, Jun; Rauber, Andreas; Li, Xiao-Li; Wang, Jun; Zha, Hongyuan: Low-rank decomposition meets kernel learning: a generalized Nyström method (2017)
  12. Liu, Weiwei; Tsang, Ivor W.: Making decision trees feasible in ultrahigh feature and label dimensions (2017)
  13. Wang, Xiaoming; Huang, Zengxi; Du, Yajun: Improving localized multiple kernel learning via radius-margin bound (2017)
  14. Xu, Lixiang; Chen, Xiu; Niu, Xin; Zhang, Cheng; Luo, Bin: A multiple attributes convolution kernel with reproducing property (2017)
  15. Christmann, Andreas; Dumpert, Florian; Xiang, Dao-Hong: On extension theorems and their connection to universal consistency in machine learning (2016)
  16. Gondzio, Jacek; González-Brevis, Pablo; Munari, Pedro: Large-scale optimization with the primal-dual column generation method (2016)
  17. Niu, Guo; Ma, Zhengming; Liu, Shuyu: A multikernel-like learning algorithm based on data probability distribution (2016)
  18. Qi, Chengming; Wang, Yuping; Tian, Wenjie; Wang, Qun: Multiple kernel boosting framework based on information measure for classification (2016)
  19. Gaüzère, Benoit; Grenier, Pierre-Anthony; Brun, Luc; Villemin, Didier: Treelet kernel incorporating cyclic, stereo and inter pattern information in chemoinformatics (2015)
  20. Jawanpuria, Pratik; Nath, Jagarlapudi Saketha; Ramakrishnan, Ganesh: Generalized hierarchical kernel learning (2015)

1 2 3 4 next