MethodHandle performance

Question

I wrote a little benchmark that tests performance of java.lang.invoke.MethodHandle, java.lang.reflect.Method and direct calls of methods.

I read that MethodHandle.invoke() performance almost the same as direct calls. But my test results show another: MethodHandle invoke about three times slower than reflection. What is my problem? May be this is result of some JIT optimisations?

public class Main {
    public static final int COUNT = 100000000;
    static TestInstance test = new TestInstance();

    static void testInvokeDynamic() throws NoSuchMethodException, IllegalAccessException {
        int [] ar = new int[COUNT];

        MethodHandles.Lookup lookup = MethodHandles.lookup();
        MethodType mt = MethodType.methodType(int.class);

        MethodHandle handle = lookup.findStatic(TestInstance.class, "publicStaticMethod", mt) ;

        try {
            long start = System.currentTimeMillis();

            for (int i=0; i<COUNT; i++) {
                ar[i] = (int)handle.invokeExact();
            }

            long stop = System.currentTimeMillis();

            System.out.println(ar);

            System.out.println("InvokeDynamic time: " + (stop - start));
        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    static void testDirect() {
        int [] ar = new int[COUNT];

        try {
            long start = System.currentTimeMillis();

            for (int i=0; i<COUNT; i++) {
                ar[i] = TestInstance.publicStaticMethod();
            }

            long stop = System.currentTimeMillis();

            System.out.println(ar);

            System.out.println("Direct call time: " + (stop - start));
        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    static void testReflection() throws NoSuchMethodException {
        int [] ar = new int[COUNT];

        Method method = test.getClass().getMethod("publicStaticMethod");

        try {
            long start = System.currentTimeMillis();

            for (int i=0; i<COUNT; i++) {
                ar[i] = (int)method.invoke(test);
            }

            long stop = System.currentTimeMillis();

            System.out.println(ar);

            System.out.println("Reflection time: " + (stop - start));
        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    static void testReflectionAccessible() throws NoSuchMethodException {
        int [] ar = new int[COUNT];

        Method method = test.getClass().getMethod("publicStaticMethod");
        method.setAccessible(true);

        try {
            long start = System.currentTimeMillis();

            for (int i=0; i<COUNT; i++) {
                ar[i] = (int)method.invoke(test);
            }

            long stop = System.currentTimeMillis();

            System.out.println(ar);

            System.out.println("Reflection accessible time: " + (stop - start));
        } catch (Throwable throwable) {
            throwable.printStackTrace();
        }
    }

    public static void main(String ... args) throws NoSuchMethodException, InvocationTargetException, IllegalAccessException, InterruptedException {
        Thread.sleep(5000);

        Main.testDirect();
        Main.testInvokeDynamic();
        Main.testReflection();
        Main.testReflectionAccessible();

        System.out.println("\n___\n");

        System.gc();
        System.gc();

        Main.testDirect();
        Main.testInvokeDynamic();
        Main.testReflection();
        Main.testReflectionAccessible();
    }
}

Environment: java version "1.7.0_11" Java(TM) SE Runtime Environment (build 1.7.0_11-b21) Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode) OS - Windows 7 64

Answer 1

Looks like this was indirectly answered by @AlekseyShipilev in reference to a different query. In the following link How can I improve performance of Field.set (perhap using MethodHandles)?

If you read through you will see additional benchmarks that show similar findings. It is likely that direct calls can simply be optimized by JIT in ways that According to the findings above, the difference is:

• MethodHandle.invoke =~195ns
• MethodHandle.invokeExact =~10ns
• Direct calls = 1.266ns

So - direct calls will still be faster, but MH is very fast. For most use-cases this should be sufficient and is certainly faster than the old reflection framework (btw - according to the findings above, reflection is also significantly faster under java8 vm)

If this difference is significant in your system, i would suggest finding different patterns rather than direct reflection which will support direct calls.

Answer 2

It appears others have seen similar results: http://vanillajava.blogspot.com/2011/08/methodhandle-performance-in-java-7.html

Here's someone else's: http://andrewtill.blogspot.com/2011/08/using-method-handles.html

I ran that second one and saw they were about the same speed even fixing that test to have a warmup. However, I fixed it so it wasn't creating an args array every time. At the default count, it resulted in the same result: methodhandles were a little faster. But I did a count of 10000000 (default*10) and reflection went much faster.

So, I would recommend testing with parameters. I wonder if MethodHandles more efficiently deal with parameters? Also, check changing the count--how many iterations.

@meriton's comment on the question links to his work and looks very helpful: Calling a getter in Java though reflection: What's the fastest way to repeatedly call it (performance and scalability wise)?

Answer 3

If the publicStaticMethod was a simple implementation like returning a constant, It is very much possible that direct call was in-lined by JIT compiler. This may not be possible with methodHandles.

RE http://vanillajava.blogspot.com/2011/08/methodhandle-performance-in-java-7.html example, as mentioned that comments its not great implementation. if you change the type casting to int (instead of Integer) in the calculation loop, the results are closer to direct method call.

With convoluted implementation of ( creating and calling a future task which returns a random int) gave benchmark with closer numbers where MethodStatic was max ~10% slower than direct method. So you might be seeing 3 times slower performance due to JIT optimizations