1

I am following the example on this page : Example of 10-fold SVM classification in MATLAB.

Basically, i am following the example to execute my classification. The problem i face is that pred is always postive. It is not able to detect negative data.

clear all;
clc;
load('C:\Users\HP\Documents\MATLAB\TrainLabel');
load('C:\Users\HP\Documents\MATLAB\TrainVec');
cvFolds = crossvalind('Kfold', TrainLabel, 10);  
cp = classperf(TrainLabel);   
for i = 1:10                                   
    testIdx = (cvFolds == i);                   
    trainIdx = ~testIdx;                             
%     Model = svmtrain(TrainVec(trainIdx,:), TrainLabel(trainIdx),'showplot',true); 
    Model = svmtrain(TrainVec(trainIdx,:), TrainLabel(trainIdx), ...              
     'Autoscale',true, 'Showplot',false, 'Method','QP', ...              
     'BoxConstraint',2e-1, 'Kernel_Function','rbf', 'RBF_Sigma',1);
    pred = svmclassify(Model, TrainVec(testIdx,:),'Showplot',false);      
    cp = classperf(cp, pred, testIdx);
end 
cp.CorrectRate 
cp.CountingMatrix

The values for pred is [1;1;1;1;1;1] but my correctrate is 0.65(65%) and the TrainLabel is <60x1 double> and TrainVec is <60x5900 double>.

Two more qns:

  1. must the values of TrainLabel be 0 and 1? is it ok if it is -1 or 1

  2. TrainVec is such that each feature from a image is placed in a row. The feature from the next image is placed in the next row. Is this correct? or must each of features be placed in a different column?

Need some help on this... thanks

Community
  • 1
  • 1
user999450
  • 249
  • 5
  • 14

2 Answers2

4

You just have to many features.

You are trying to find a separating 5899-dimensional hyperplane using only 60 training points. That simply is not going to work because of the Curse of dimensionality (aka. the Hughes effect).

You need to extract relevant features first, and work on only those. This is called Feature Extraction.

One easy way of doing this is using pcacov to transform your data using Principle Component Analysis, then keep only a certain fraction (use the third, EXPLAINED result to keep the k PC's explaining a certrain level of variance, like 98%). That'll cut short the dimensionality of your problem, and very likely improve your results.

Do remember to transform all your data, not just the training set.

Aside of that, your approach seems correct to me. Different samples go in different rows, with their features spanning the columns.

The label vector can be whatever you'd want:

Y is a grouping variable, i.e., it can be a categorical, numeric, or logical vector; a cell vector of strings; or a character matrix with each row representing a class label (see help for groupingvariable)

jpjacobs
  • 9,359
  • 36
  • 45
  • First of all, thanks for replying.. I really appreciate that.. One thing i would like to ask, is my code written properly besides the PCA part? – user999450 Jan 11 '12 at 15:19
  • I would say that the problem is not that he has too many feature, but that he has to few exemplars... – Oli Jan 11 '12 at 18:33
  • I guess he means training samples. @Oli: Well, I have to disagree on that. I think there is no practical example whatsoever that has 5900 actually discerning features. If you would need to use them all, then you indeed need a lot more training instances. But if it turns out that you can extract like 10 discerning features, you might be OK with 60 training samples. – jpjacobs Jan 12 '12 at 08:17
  • must the values of TrainLabel be 0 and 1? is it ok if it is -1 or 1 – user999450 Jan 12 '12 at 15:57
  • Can be either, as long as you're using then consistently. See the last quote of my answer. – jpjacobs Jan 12 '12 at 15:59
  • Hey, i just tried with more samples, the pred label is still all 1... it is still not correct. what else might be the cause? any idea? – user999450 Jan 13 '12 at 06:26
  • Like I said: reduce the number of features (along the column dimension of your data). Use only those that you need. Which ones are useful to you entirely depends on the application. For images you could for example extract histograms, and use them to classify badly exposed images, or using color-histograms classify them according to a predominant color, ... Bottom line: less, but better features is always better and faster than just having a huge amount of then – jpjacobs Jan 13 '12 at 07:57
  • thanks a lot... Now i have more images and i have also reduced my features to 1495... – user999450 Jan 17 '12 at 13:45
  • but the problem still remains... the features are lesser but the pred label remains the same. – user999450 Jan 17 '12 at 14:23
  • Try reducing the number of features to 100 or less. – jpjacobs Jan 18 '12 at 07:41
  • finally found it... need to scale it.. if you didn't scale it... it will never work... – user999450 Jan 19 '12 at 17:56
-1

Scale the values to between 0 to 1.This will solve the problem

user999450
  • 249
  • 5
  • 14
  • 1
    This is plain wrong. No way you're getting away with 5900 features. Aside of that take a look at the `doc svmtrain`: it says: " **autoscale**: Boolean specifying whether svmtrain automatically centers the data points at their mean, and scales them to have unit standard deviation, before training. Default: true". So it should not matter whether you scale or not – jpjacobs Jan 20 '12 at 08:00