How can I calculate high precision double value use java in Machine learning?

Question

I was using Logistic regression to train my data. In this process, I need calculate a sigmoid function coded like:

private double sigmoid(double x) {
    return 1.0 / (1 + Math.pow(Math.E, -1 * x));
}

So if my x is large than 36, it always return 1.0, and I will get Infinity in log(1 - sigmoid(x)).

I guess java is not a good language at machine-learning, I just need apply in this project.

I also know Java's precision principle through this question, But I still want to know is there anyway to solve this problem.

Consider using `BigDecimal` for high precision values. Also this question may be helpful to you: https://stackoverflow.com/questions/13784056/calculating-sigmoid-value-of-a-bigdecimal-in-java — d.j.brown, Jul 01 '17 at 11:26
This is floating point specific (not language specific). One trick is to use a different scale, say a logarithmic one or such - if possible. Here probably not. — Joop Eggen, Jul 01 '17 at 12:58
sorry, my meaning is java maybe is not good at machine-learning. — fmyblack, Jul 01 '17 at 15:09

Thomas Jungblut · Answer 1 · 2017-07-01T16:31:24.067

1

While it is a precision issue, usually the way to deal with exploding gradients and ensure numerical stability is to clip the input of the function:

final double CLIP = 30d;
input = Math.min(CLIP, Math.max(-CLIP, input)));
return 1.0 / (1.0 + Math.exp(-input))

You can also clip the output to [0, 1]:

double output = 1.0 / (1.0 + Math.exp(-input))
return Math.min(0d, Math.max(1d, output)));

edited Jul 01 '17 at 16:31

answered Jul 01 '17 at 12:31

Thomas Jungblut

20,854
6
68
91

I am not sure if I am understanding correctly, sigmoid function always return value between 0 and 1, so it can't do any change. Do your meaning is like `Math.min(CLIP, Math.max(-CLIP, input))` ? – fmyblack Jul 01 '17 at 16:01
true, sigmoid output can be clipped between 0 and 1, or the input can be clipped with a higher decimal. – Thomas Jungblut Jul 01 '17 at 16:29

score 0 · Answer 2 · answered Jul 01 '17 at 12:52

this works perfectly for me

tested 20 , 40 , 49

import java.math.BigDecimal;
import java.math.MathContext;

/**
 *
 * @author tawfik
 */
public class Mathic {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
       BigDecimal sigmoid = sigmoid(37);
        System.out.println(sigmoid);
    }
    static private BigDecimal sigmoid(double x) {
    return new BigDecimal(1.0).divide(new BigDecimal(1).add(new BigDecimal(Math.pow(Math.E, -1 * x))),MathContext.DECIMAL128);
}
}

score 0 · Answer 3 · answered Jul 01 '17 at 17:41

There are other ways of writing the sigmoid function that are more numerically stable

private double sigmoid(double x) {
    if (x > 0) {
        return 1.0 / (1 + Math.exp(-x));
    }
    else {
        z = Math.exp(x);
        return z / (1 + z);
    }
}

How can I calculate high precision double value use java in Machine learning?

3 Answers3