2

I'm having problem in in finding the right parameters for the information gain, if I don't have any discrete values and thus I first need to discretize these points into intervals.

What I have:

I'm doing image processing, where my features have a possible range 0-255. With some training data I can define some intervals (which only define "is object or is not object"). If goods are the number of intervals for for a matching point and bads is labeled for its environment. I'll calculate it this way with

formula

information gain for this case:

formula

where

formula

Results and idea:

For some reason I end up with a negative IG which is quiet nonsense but I don't see the error. Another idea was instead of counting the object-matching intervals forgood, count the samples in good that fit into any good-interval.

Has anyone an idea?

user3085931
  • 1,757
  • 4
  • 29
  • 55

1 Answers1

0

I don't see what you have there as before and after (or P and Q) distributions.

Have you changed anything to go from one situation to another? It's unclear.

Look at What is "entropy and information gain"?

It seems good+bad represent the whole distribution.

So you need to have something change to go from one (good, bad) to another (good, bad).

Then you apply the formula correctly - or follow the example

Your formula seems to be messed up.

Community
  • 1
  • 1
gpasch
  • 2,672
  • 3
  • 10
  • 12
  • well if I hadn't seen stuff like this I wouldn't have a solution either. The thing is I don't see where I went wrong – user3085931 Dec 14 '15 at 10:00
  • I think you are trying to trainan image recognition/processing sytem right? The two situations I'm referring to are: you trained your system and got performance A: good=X, bad=Y, then trained again and got B: good=Z, bad=W. I you can verify you have these two situations (A, B) to compare we can go further – gpasch Dec 14 '15 at 10:31
  • 1
    to put a close ito it: IG=[Entropy(X/(X+Y))+Entropy(Y/(X+Y))]-[Entropy(Z/(Z+W))+Entropy(W/(Z+W))] - you cant get info gain with only one run of the experiment - can you get moving by standing still?? – gpasch Dec 14 '15 at 19:01
  • **Solution**: If anyone has a similiar problem. I was missing out to wander around the whole range from 0 - 255 to find the best split position for the information gain to a particular feature. – user3085931 Dec 21 '15 at 21:56