probability model in kernlab::ksvm

Question

For classification tasks in kernlab::ksvm, the default SVM used is C-svm (LIBSVM, Chang & Lin), which calculates a binary classification task. This can be extended to multiclass problems by calculating multiple 1 vs many binary classifiers and aggregating the results. Native multiclass classification is supported via spoc-svm (Crammer, Singer) and via kbb-svm (Weston, Watkins).

These are supported in kernlab via the type parameter in ksvm (see ?kernlab::ksvm):

ksvm(..., type= "C-svc", ...)
ksvm(..., type= "spoc-svc", ...)
ksvm(..., type= "kbb-svc", ...)

However, prediction probabilities are only available via C-svm. Why? Is this an error in the implementation?

library(kernlab)
data(iris)
# default - C-svc allows for a prob.model
k1 <- ksvm(Species ~. ,data= iris, 
           type= "C-svc",
           prob.model=TRUE,
           kernel= "rbfdot", C= 1)

p1 <- predict(k1, iris, type= "probabilities") # works

#### non default, doesn't work:
k1 <- ksvm(Species ~. ,data= iris, 
           type= "spoc-svc",
           prob.model=TRUE,
           kernel= "rbfdot", C= 1)

p1 <- predict(k1, iris, type= "probabilities")
Error in .local(object, ...) : 
  ksvm object contains no probability model. Make sure you set the paramater    prob.model in ksvm during training.

k1 <- ksvm(Species ~. ,data= iris, 
           type= "kbb-svc",
           prob.model=TRUE,
           kernel= "rbfdot", C= 1)

p1 <- predict(k1, iris, type= "probabilities")
Error in .local(object, ...) : 
  ksvm object contains no probability model. Make sure you set the paramater prob.model in ksvm during training.

The documentation does not note this or provide any guidelines. As you can see, the parameter prob.model has been specified in the function call. At a minimum, this appears to be a problematic error message.

score 1 · Accepted Answer · answered Jun 28 '16 at 12:20

1

kernlab currently does not support probability estimation for types other than C-svc, nu-svc and C-bsvc (check the code).

if(type == "probabilities")
{ 
  if(is.null(prob.model(object)[[1]]))
    stop("ksvm object contains no probability model. Make sure you set the paramater prob.model in ksvm during training.")

  if(type(object)=="C-svc"||type(object)=="nu-svc"||type(object)=="C-bsvc")
    {
        [...]
    }
  else
    stop("probability estimates only supported for C-svc, C-bsvc and nu-svc")
}

The problem is native multiclass solutions lack the binary probabilities that go as input into couple. Actually, coding your own solution wouldn't be that hard.

answered Jun 28 '16 at 12:20

catastrophic-failure

3,759
1
24
43

Where would one start looking in order to code multiclass SVM probability models please? – GMSL Jul 28 '20 at 13:13
Can't edit my previous because it has been > 5 mins. It seems like the best way to go is the implementation by Wu _et al._, _J Mach Learn Res_, 5, 2004 http://www.jmlr.org/papers/volume5/wu04a/wu04a.pdf – GMSL Jul 28 '20 at 13:24

probability model in kernlab::ksvm

1 Answers1