2

I'm attempting to profile some methods to see which implementations are faster. I've run into a snag as java appears to be cheating the profiles by not processing all the information, each time it's requested.

public void Profile() {
    double[] testArray = new double[100000000];
    Math math = new Math();
    long start = System.nanoTime();
    for (int i = 1; i <= 100000000; i++) {
        testArray[i - 1] = math.Divide(i);
    }
    long stop = System.nanoTime();
    System.out.println("math.divide(): " + TimeUnit.NANOSECONDS.toMillis(stop - start));
    System.out.println(testArray[(int) (java.lang.Math.random() * (100000000 - 0))]);
}

public class Math {
    public double Divide(int i) {
        int dividend = i;
        int divisor = 12;
        return dividend / (double) divisor;
    }
}

if I don't assign the the Math.Divide() value to the array, the entire for loop returns in 2 ms, when I do, it returns in ~200 ms, which still seems too fast (considering an identical implementation in C# takes a minimum of 1.8 seconds).

ultimately my questions are: is there a more accurate way to profile a method? and how can I override java so that it stops skipping work it considers redundant.

Tsung-Ting Kuo
  • 1,171
  • 6
  • 16
  • 21
user2835725
  • 154
  • 6
  • 11
    This is no "cheat"; this is called the JIT. Welcome to the JVM world. – fge Dec 01 '15 at 00:35
  • 1
    @fge the problem is that the JIT is not processing all the information I need it to, too properly collect the profiling data I need. c#'s JIT doesn't do this, and I cant seem to find a compilation option to stop it. – user2835725 Dec 01 '15 at 00:37
  • 3
    http://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java Why would you want to stop these optimizations? – Paul Boddington Dec 01 '15 at 00:37
  • 4
    And? If you don't assign the result of an operation to a variable, what use is this operation? This is pure logic, and the JIT detects that. Ultimately it means your .Divide() is elided entirely. Moreoever: you should rename the method to something else, and not use Math since there exists java.util.Math already. – fge Dec 01 '15 at 00:39
  • 2
    @PaulBoddington because the point of the code is to understand the time it takes for the method to execute. the JIT sees the repeated executions as a waste, but they need to be individual and repeatedly calculated to get a meaningful execution time. doing it once and just caching the result or only doing the single needed calculation does not meet the objectives of the for loop. – user2835725 Dec 01 '15 at 00:41
  • 1
    Again: welcome to the JVM world and the JIT. If you want to do micro benchmarcks correctly, have a look at [JMH](http://openjdk.java.net/projects/code-tools/jmh/) – fge Dec 01 '15 at 00:43
  • http://stackoverflow.com/questions/12176175/can-jit-be-prevented-from-optimising-away-method-calls and http://stackoverflow.com/questions/16929845/can-jit-be-prevented-from-optimising-method. Those are _cheats_. – Sotirios Delimanolis Dec 01 '15 at 00:43
  • @user2835725 You're right - that wasn't a very sensible question. Still, the link I provided is useful. – Paul Boddington Dec 01 '15 at 00:45
  • 2
    Well, if you don't want "cheats" like the compiler making things fast for you, you can always try `-Xint`... and go out for a coffee break. – vanza Dec 01 '15 at 00:49

3 Answers3

1

As already mentioned in the comments, that is no cheat at all. That's a feature of the JVM and that feature is called the JIT. Among other optimizations, it will perform visibility checks and, if you perform an operation which result you never use, it will just avoid running the code altogether. Just like in your first example.

In other words: that's not a bug, that's a feature. A real feature.

If you want to correctly do micro benchmarks then use JMH. Here is a link showing sample codes, and note how many examples there are to "work around" various optimizations that the JVM would otherwise perform so that the benchmark environment remain stable: the JVM (HotSpot, at least) is very, very good at optimizations.

fge
  • 119,121
  • 33
  • 254
  • 329
0

I first met the problem of optimization of benchmarks in Fortran, in the early 1980's and I did not get the impression it was a new problem then.

There are two solutions.

The most reliable one is to benchmark code you really care about, code where you will be happy if the language implementation has a clever way to make it faster. Presumably, you are writing your divide function because you care about integer divide performance in some program. Run that program as your benchmark. That does the division in exactly the context that matters to you, permitting relevant optimizations but without irrelevant ones.

The second best is to really, really carefully prevent optimizations that stop the code you are measuring from running the way you want. My basic strategy for that has been to make sure that every step I want to measure ultimately affects a value in a human-readable display. I don't trust your random trick, because a sufficiently clever JVM, after inlining the divide function, might notice that only one division result is going to be displayed, and just calculate that one after flipping the random number generator. Instead, I would output, for example, the exclusive-or of all the array elements. The JMH described in @fge's answer looks like a better, more automatic, way of doing this.

Community
  • 1
  • 1
Patricia Shanahan
  • 25,849
  • 4
  • 38
  • 75
0

This optimization is used to overcome the inherent problem of decoding instructions twice (class file to jvm, jvm to cpu). The JIT is described here...

It is possible to disable the JIT I'm told, though I've never tried it, and this forum discusses it: Oracle: Disabling JIT

JIT compiler can have bugs from codes that optimize the bytecode. During the phase of converting bytecode to optimal binary code, you have a chance to hit some unexpected behaviors and you got JVM crahsed.

In that case, you should disable JIT compilation for some specific methods or entire JVM. "-Xint" option forces your entire JVM run in interpreter mode. Compiler command file(.hotspot_compiler file) provides "exclude" command to disable JIT compilation for specific methods. exclude java/util/Arrays mergeSort

Out of curiosity, and sorry to go computer science 101 on you, but have you calculated big-O on them first?

Sean
  • 126
  • 1
  • 4