Static final variable initialization (in Java) incorrect during Kotlin CI Tests

Question

I manage an open source project and have a user reporting a situation which I think is impossible according to Java's order of initialization of static variables in classes. The value of a static final class variable is incorrect, apparently resulting from different results of a dependency's static method based on its own static final variable.

I'd like to understand what's happening in order to figure the best workaround. At the moment, I am baffled.

The problem

The main entry point for my project is the class SystemInfo which has the following constructor:

public SystemInfo() {
    if (getCurrentPlatform().equals(PlatformEnum.UNKNOWN)) {
        throw new UnsupportedOperationException(NOT_SUPPORTED + Platform.getOSType());
    }
}

When run by itself, the problem doesn't reproduce; but when run as part of many tests being executed a larger build (mvn install) it is consistently reproducible, implying the problem is likely associated with multithreading or multiple forks. (To clarify: I mean the simultaneous initialization of static members in two different classes, and the various JVM-internal locking/synchronization mechanisms associated with this process.)

They receive the following result:

java.lang.UnsupportedOperationException: Operating system not supported: JNA Platform type 2

This exception implies two things are true when SystemInfo instantiation begins:

• The result of getCurrentPlatform() is the enum value PlatformEnum.UNKNOWN
• The result of Platform.getOSType() is 2

However, this situation should be impossible; a value of 2 would return WINDOWS, and unknown would return a value other than 2. Since both variables are both static and final they should never simultaneously reach this state.

(User's) MCRE

I have tried to reproduce this on my own and failed, and am relying on a report from a user executing tests in their Kotlin-based (kotest) framework.

The user's MCRE simply invokes this constructor as part of a larger number of tests, running on the Windows operating system:

public class StorageOnSystemJava {
    public StorageOnSystemJava(SystemInfo info) {
    }
}

class StorageOnSystemJavaTest {
    @Test
    void run() {
        new StorageOnSystemJava(new SystemInfo());
    }
}

Underlying code

The getCurrentPlatform() method simply returns the value of this static final variable.

public static PlatformEnum getCurrentPlatform() {
    return currentPlatform;
}

This is a static final variable populated as the very first line in the class (so it should be the first thing initialized):

private static final PlatformEnum currentPlatform = queryCurrentPlatform();

where

private static PlatformEnum queryCurrentPlatform() {
    if (Platform.isWindows()) {
        return WINDOWS;
    } else if (Platform.isLinux()) {
        // other Platform.is*() checks here
    } else {
        return UNKNOWN; // The exception message shows the code reaches this point
    }
}

This means that during class initialization, all of the Platform.is*() checks returned false.

However, as indicated above this should not have happened. These are calls to JNA's Platform class static methods. The first check, which should have returned true (and does, if called in the constructor or anywhere in code after instantiation) is:

public static final boolean isWindows() {
    return osType == WINDOWS || osType == WINDOWSCE;
}

Where osType is a static final variable defined thus:

public static final int WINDOWS = 2;

private static final int osType;

static {
    String osName = System.getProperty("os.name");
    if (osName.startsWith("Linux")) {
        // other code
    }
    else if (osName.startsWith("Windows")) {
        osType = WINDOWS; // This is the value being assigned, showing the "2" in the exception
    }
    // other code
}

From my understanding of the order of initialization, Platform.isWindows() should always return true (on a Windows OS). I do not understand how it could possibly return false when called from my own code's static variable initialization. I've tried both the static method, and a static initialization block immediately following the variable declaration.

Expected order of initialization

1. User calls the SystemInfo constructor
2. SystemInfo class initialization begins ("T is a class and an instance of T is created.")
3. The static final currentPlatform variable is encountered by the initializer (first line of class)
4. The initializer calls the static method queryCurrentPlatform() to obtain a result (same result if the value is assigned in a static block immediately following the static variable declaration)
5. The Platform.isWindows() static method is called
6. The Platform class is initialized ("T is a class and a static method of T is invoked.")
7. The Platform class sets the osType value to 2 as part of initialization
8. When Platform initialization is complete, the static method isWindows() returns true
9. The queryCurrentPlatform() sees the true result and sets the currentPlatform variable value (This is not happening as expected!)
10. After SystemInfo class initialization is complete, its constructor executes, showing the conflicting values and throwing the exception.

Workarounds

Some workarounds stop the problem, but I don't understand why they do:

• Performing the Platform.isWindows() check anytime during the instantiation process (including the constructor) properly returns true and assigns the enum appropriately.

  • This includes lazy instantiation of the currentPlatform variable (removing the final keyword), or ignoring the enum and directly calling JNA's Platform class.

• Moving the first call to the static method getCurrentPlatform() out of the constructor.

These workarounds imply a possible root cause is associated with executing static methods of multiple classes during class initialization. Specifically:

• During initialization, the Platform.isWindows() check apparently returns false because code reaches the else block
• After initialization (during instantiation), the Platform.isWindows() check returns true. (Since it is based on a static final value it should not ever return different results.)

Research

I've thoroughly reviewed multiple tutorials about Java clearly showing the initialization order, as well as these other SO questions and the linked Java Language Specs:

• Java static class initialization
• in what order are static blocks and static variables in a class executed?
• In what order are the different parts of a class initialized when a class is loaded in the JVM?

Answer 1

It's not multithreading, because the JVM prevents other threads from accessing a class while it is being initialized. This behavior is mandated by the Java Language Specification, section 12.4.2, step 2:

If the Class object for C indicates that initialization is in progress for C by some other thread, then release LC and block the current thread until informed that the in-progress initialization has completed, at which time repeat this step.

It is exceedingly unlikely for a JVM to have a bug in this area, since it would cause repeated initializer execution, which would be very noticeable.

However, a static final field can appear to have a changing value if:

• there is a cyclic dependency among initializers

  Same section, step 3 writes:

  If the Class object for C indicates that initialization is in progress for C by the current thread, then this must be a recursive request for initialization. Release LC and complete normally.

  Therefore, a recursive initialization may allow a thread to read a static final field before it is assigned. This can only happen if class initializers create a cyclic dependency among initializers.

• somebody (ab)uses reflection to reassign the static final field

• the class is loaded by more than one class loader

  In this case, each class has its own copy of the static field, and may initialize it differently.

• if the field is a compile time constant expression, and the code was compiled at different times

  The spec mandates that compile time constant expressions are inlined by the compiler. If different classes are compiled at different times, the value being inlined may have been different. (In your case, the expression is not compile time constant; I only mention this possiblity for the sake of future visitors).

From the evidence you have given, it is impossible to say which of these apply. That's why I recommend further investigation.

Answer 2

DISCLAIMER: I'm writing this as an answer because I don't know how to make it fit into a comment. If it doesn't help you let me know, and I'll delete it.

---

Let's start with a small recap which given the question quality, I'm sure you're aware of already:

• A field which is static to a class, means that it only exists once for any instance. No matter how many instances of the class you create, the field will always point to the same memory address.
• A field which is final means that once initialized, its value cannot change any longer.

So, when you mix these two into a static final field, it means:

• You only have one value for that field, no matter how many instances
• Once the value is assigned, it doesn't change any longer

So, my suspect is not that there is any thread-safety issue (I don't think you're running your tests in parallel, so I guess no two-threads would simultaneously work on these objects, right?), but rather that a previous test of your test suite has initialized the variables differently and, being they run into the same JVM, they don't change any longer their values.

Take this very simple test example.

I have one very basic class:

public final class SomeClass {

    private static final boolean FILE_EXISTS;

    static {
        FILE_EXISTS = new File("test").exists();
    }

    public SomeClass() {
        System.out.println("File exists? " + FILE_EXISTS);
    }

}

The above class simply has a static final boolean saying whether a certain file named test exists into the working directory. As you can see, the field is initialized once (final) and will be the same for each instance.

Now, let's run these two very simple tests:

@Test
public void test_some_class() throws IOException {
    System.out.println("Running test_some_class");
    File testFile = new File("test");
    if (testFile.exists()) {
        System.out.println("Deleting file: " + testFile.delete());
    } else {
        System.out.println("Could create the file test: " + testFile.createNewFile());
    }
    SomeClass instance1 = new SomeClass();
}

@Test
public void other_test_some_class() {
    System.out.println("Running other_test_some_class");
    SomeClass instance2 = new SomeClass();
}

In the first test, I check if file test exists. If it does exists, I'll delete it. Else, I'll create it. Then, I'll initialize a new SomeClass().

In the second test, I simply initialize a new SomeClass().

This is the output of my tests run together:

Running other_test_some_class //<-- JUnit picks the second test to start
File exists? false //<-- The constructor of SomeClass() prints the static final variable: file doesn't exist
Running test_some_class //<-- JUnit continues running the first test
Could create the file test: true //<-- it is able to create the file
File exists? false //<-- yet, the initializer of new SomeClass() still prints false

The reason why it prints false, even though we clearly created the test file before initializing new SomeClass(), is that the field FILE_EXISTS is static (hence shared across all instances) and final (hence initialized once, lasting forever).

So if you wonder why private static final int osType; has a value which returns you UNKNOWN when you run mvn install but not when you run the single test, I'd simply look what test, in your full test suite, has already initialized it with a value that you don't expect.

Solution

There are two types of solution for this, and they depend on your production code.

It may be that, functionally, you actually need this field to be maybe final to the instance of the class, but not static. If that's the case, you should just declare it final to the class (once initialized, it doesn't change, but still you have one different value per instance).

Or, it may be that you really need the field to be static final in production, but not during tests as you initialize a new test context each time. If that's the case, you should configure your test plugin to reuseForks = false (meaning a fresh JVM fork is created for each test class, and that guarantees you that each test class will start with a fresh memory for your static final fields):

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>${maven.surefire.plugin.version}</version>
            <configuration>
                <forkCount>1</forkCount>
                <reuseForks>false</reuseForks>
            </configuration>
        </plugin>