Support Measure Machines

We have got a paper at NIPS this year.

Learning from Distributions via Support Measure Machines (Spotlight)
K. Muandet, K. Fukumizu, F. Dinuzzo, B. Schoelkopf

Abstract This paper presents a kernel-based discriminative learning framework on probability measures. Rather than relying on large collections of vectorial training examples, our framework learns using a collection of probability distributions that have been constructed to meaningfully represent training data. By representing these probability distributions as mean embeddings in the reproducing kernel Hilbert space (RKHS), we are able to apply many standard kernel-based learning techniques in straightforward fashion. To accomplish this, we construct a generalization of the support vector machine (SVM) called a support measure machine (SMM). Our analyses of SMMs provides several insights into their relationship to traditional SVMs. Based on such insights, we propose a flexible SVM (Flex-SVM) that places different kernel functions on each training example. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework.

This is joint work with Kenji Fukumizu (The Institute for Statistical Mathematics, Japan), Francesco Dinuzzo (MPI-IS), and my supervisor Prof. Bernhard Schoelkopf (MPI-IS). Parts of this work were done while Kenji was visiting us in summer 2011.

The arXiv manuscript can be found here. This is not up-to-date version, but will give a basic idea of this work.

2 thoughts on “Support Measure Machines

  1. Rishabh Mehrotra

    Congratz on the NIPS accept. Just went through the arXiv version, very interesting concept. I saw results on Hand writing and scene categorization. Do you have any results on some text application of the same? Just curious.

    1. admin Post author

      We didn't do any experiment on the text/document classification. That would also be very interesting as the documents can be seen as probability distributions over the topics, e.g., as in the topic modelling literatures.

      I will be very happy to discuss about it in detail if you have an idea. Thanks.


Leave a Reply

Your email address will not be published. Required fields are marked *