Kernel-based algorithms have achieved great success in diverse applications. Rather than requesting the user select a specific kernel, which is crucial to the success of these algorithms, multiple kernel learning (MKL) provides a flexible strategy by combining multiple and heterogeneous features in terms of their discriminative power. Specifically, MKL seeks to learn linear or nonlinear combination of base kernels that maximizes certain performance measure, such as a support vector machines (SVM)-type objective, kernel target alignment, Fisher discriminative analysis, and Bayesian framework.

The objective of this research is to train SVM-based localized multiple kernel learning (LMKL), utilizing the alternating optimization between the standard SVM solvers with the local combination of base kernels and the sample-specific kernel weights. The advantage of alternating optimization developed from the state-of-the-art MKL is the SVM-tied overall complexity and the simultaneous optimization on both the kernel weights and the classifier. Unfortunately, in LMKL, the sample-specific character makes the updating of kernel weights a difficult quadratic nonconvex problem.

In this research, it is started from the new primal-dual equivalence. The canonical objective on which the state-of-the-art methods are based is first decomposed into an ensemble of objectives corresponding to each sample, namely, sample-wise objectives. And then, the associated sample-wise alternating optimization method is conducted. And the localized kernel weights can be independently obtained by solving their exclusive sample-wise objectives, either linear programming (for l₁-norm) or with closed-form solutions (for l_p-norm).

At test time, the learnt kernel weights for the training data are deployed based on the nearest-neighbor rule. Hence, to guarantee their generality among the test part, the neighborhood information is introduced and incorporated into the empirical loss when deriving the sample-wise objectives. Extensive experiments on four benchmark machine learning datasets and two real-world computer vision datasets demonstrate the effectiveness and efficiency of the proposed algorithm.

This research was supported in part by the National Natural Scientific Foundation of China under Grant 11174235, and the Opening Project of State Key Laboratory of Acoustics under Grant SKLA201201.

The research entitled “Localized Multiple Kernel Learning via Sample-wise Alternating Optimization” has been published on IEEE Transactions on Cybernetics (Vol. 44, No. 1, pp.137-148,January 2014).