java.util.Random makes 100 times faster computation?

Question

I am totally lost for words, i have no idea what's going on...

//import java.util.Random;

public class TestInProgress {
    public static void main(String[] args) {
        long start, end;
        long average;

//      Random random= new Random();

    for(int j=0; j<5; j++) {

        average= 0L;
        for(int i=0; i<100; i++) {

            start= System.nanoTime();
                double temp= fastComputeSigm(0.0);
            end= System.nanoTime();
            average+= (end - start);
        }

        average= average/ 100L;
        System.out.println("AVERAGE FASTCOMPUTE: "+average);

        average= 0L;
        for(int i=0; i<100; i++) {

            start= System.nanoTime();
                double temp= Math.exp(0.0)/(1.0 + Math.exp(0.0));
            end= System.nanoTime();

            average+= (end - start);
        }

        average= average / 100L;
        System.out.println("AVERAGE SIGMOID: "+average);
    }
}

static double fastComputeSigm(double value) {
    return 0.0;
}

}

The most surprising is the output:

AVERAGE FASTCOMPUTE: 98 AVERAGE SIGMOID: 38625 AVERAGE FASTCOMPUTE: 106 AVERAGE SIGMOID: 65 AVERAGE FASTCOMPUTE: 299 AVERAGE SIGMOID: 201 AVERAGE FASTCOMPUTE: 36 AVERAGE SIGMOID: 65 AVERAGE FASTCOMPUTE: 53 AVERAGE SIGMOID: 57

see that? I uncommented Random (and its import) What would you expect? output:

AVERAGE FASTCOMPUTE: 90 AVERAGE SIGMOID: 324 AVERAGE FASTCOMPUTE: 131 AVERAGE SIGMOID: 73 AVERAGE FASTCOMPUTE: 217 AVERAGE SIGMOID: 36 AVERAGE FASTCOMPUTE: 53 AVERAGE SIGMOID: 12 AVERAGE FASTCOMPUTE: 53 AVERAGE SIGMOID: 69

I tried this on eclipse Oxygen.3a Release (4.7.3a) I'm using java 10.0.2 My computer has a Intel Core i5-7300HQ with those cache memories : L1 256, L2 1024, L3 6M I run on windows 10

note: I tried to make the forloop with j<1,2,3,... several times no helping the first sigmoid always has absurd time length

edit: tried launching it with windows commandline (javac and java), same result

Writing Java benchmarks correctly is difficult due to things like the Just In Time compiler. Use a proper benchmarking tool - see the link in the first comment. — greg-449, Nov 03 '18 at 17:59
Removed the eclipse label - this is reproducible with Java 8 / macOS / i7 — David Soroko, Nov 04 '18 at 13:40

David Soroko · Accepted Answer · 2018-11-04T14:45:27.817

In general comments pointing you at the trickiness of micro-benchmarking are correct and you should be using JMH. In this particular case it looks like your timing / benchmarking code introduces latency that wasn't there in the first place.

Here is a compacted version of your code that demonstrates that the issue is triggered not by the use of Random but of the use of System.nanoTime() (which Random() calls internally)

public static void main(String[] args) {
    long start, end, average;
    double temp = 0;

    // start = System.nanoTime();

    average = 0L;
    for (int i = 0; i < 100; i++) {
        start = System.nanoTime();
        temp += Math.exp(0.0) / (1.0 + Math.exp(0.0));
        end = System.nanoTime();

        average += (end - start);
        // System.out.println(i + " " + (end - start));
    }
    System.out.println("AVERAGE: " + (average / 100));
    System.out.println("temp: " + temp);
}

When I uncomment start = System.nanoTime() I observer about x100 speedup - at this point it is important to remember that the units here are nanoseconds so we are very sensitive to whatever it is the runtime is doing in the background. If you uncomment System.out.println(i + " " + (end - start)); you will see an occasional hiccup that is responsible for the entire slowdown. While it may be interesting to chase the reason for that hiccup the following version of the code indicates that it is due to the measurement rather than the core functionality so you may want to make sure that this is something you want to spend your own time on [1]:

public static void main(String[] args) {
    double temp = 0;

    long start = System.nanoTime();

    for (int i = 0; i < 100; i++) {
        temp += Math.exp(0.0) / (1.0 + Math.exp(0.0));
    }

    System.out.println("AVERAGE: " + ((System.nanoTime() - start) / 100));
    System.out.println("temp: " + temp);
}

The AVERAGE values here are similar to the previous version with start = System.nanoTime() uncommented.

[1] If you are interested in digging deeper into this particular behavior, try timing your test while running with the -Xint VM option which disables the compiler.

**CODE 1:** Whether I uncomment or not `long start= System.nanoTime()`, the average doesn't change, but yeah println(string) does adds an extra. **CODE 2:** Strangely enough this version that I thought more compact gives me 2-4 times bigger result. code1: (without println)`AVERAGE: 18x-24x temp: 50.0`code2 and code1 with println: `AVERAGE: 4xx-8xx temp: 50.0` — Sheed, Nov 04 '18 at 15:11
So what I just showed kinda contradict your statement "the code indicates that it is due to the measurement rather than the core functionality". Nonetheless it doesn't take away that my micro-benchmark isn't on point. — Sheed, Nov 04 '18 at 15:13

java.util.Random makes 100 times faster computation?

1 Answers1