Wrong time of method execution

Question

I want to test the time of adding and getting item in simple and generic hashmap:

public void TestHashGeneric(){


        Map hashsimple = new HashMap();             

        long startTime = System.currentTimeMillis();



        for (int i = 0; i < 100000; i++) {
            hashsimple.put("key"+i, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" );
        }

        for (int i = 0; i < 100000; i++) {
            String ret =(String)hashsimple.get("key"+i);
        }
        long endTime =System.currentTimeMillis();

        System.out.println("Hash Time " + (endTime - startTime) + " millisec");

        Map<String,String> hm = new HashMap<String,String>();

        startTime =  System.currentTimeMillis();



        for (int i = 0; i < 100000; i++) {
            hm.put("key"+i, "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" );
        }

        for (int i = 0; i < 100000; i++) {
            String ret = hm.get("key"+i);
        }
        endTime = System.currentTimeMillis();

        System.out.println("Hash generic Time " + (endTime - startTime) + " millisec");         

    }

The problem is that I get different time if I change places between hashmap's code section! if i put the loops (with the time print ofcourse) of generic below the simple I get better time for generic and if i put simple below the generic I get better time for simple!

Same happens if I use different methods for this.

Please edit your question so that it makes more sense - at the moment it's extremely hard to understand. Please read http://tinyurl.com/so-hints — Jon Skeet, Dec 19 '11 at 07:33
Question is unclear. Are you saying that you have two supposedly completely unrelated pieces of code, and the one that runs second runs faster (at the other way around when you switch their order)? Could be JIT at work among other things. Maybe "warm up" both pieces and do the timings after that is done. — Thilo, Dec 19 '11 at 07:36
But I can tell you this: Generics have zero impact on runtime performance. They are a purely compile-time thing. — Thilo, Dec 19 '11 at 07:37

score 1 · Answer 1 · answered Dec 19 '11 at 07:34

The JIT will compile and optimise your program while it is running, so the 2nd run will always be faster.

You should make the following modifications:

Run both tests untimed first, then re-run them timed so that you don't get affected by the JIT.
You should use System.nanoTime() as this is more accurate for timing (you should never get a diff of 0).
You should also test against some empty methods, as you are also timing the string concatenation operation in each loop.

Also note that in Java generic types are erased, so there should be no runtime difference at all.

score 0 · Answer 2 · answered Dec 19 '11 at 07:34

The Java Runtime is pretty sophistocated - it does some learning/optimisation while it's running. You might be able to get the test you want by "warming up" the JVM first. Try calling TestHashGeneric() twice and see what the second set of results gives you.

You've also got twice as much stuff in memory on the second run. There's all sorts of variables that can affect this test.

score 0 · Answer 3 · edited May 23 '17 at 11:57

0

This is not the correct way to perform micro-benchmarks as it is very involved (Why?). I suggest you use Caliper framework for this.

edited May 23 '17 at 11:57

Community

1
1

answered Dec 19 '11 at 07:34

Aravind Yarram

78,777
46
231
327

score 0 · Answer 4 · answered Dec 19 '11 at 08:39

When you have a single loop which reaches the compile threshold (about 10K) the whole method is compiled. This can make either the first loop appear faster (as it is optimised correctly as the second loop has no counter information) or the second loop appear after (as its compiled before it is even started)

This simplest way to fix this is to place each test in its own method and they will be compiled and optimised independently. (The order can still matter but is less important) I would still suggest running the test a number of times to see how the results vary.

Wrong time of method execution

4 Answers4