Access all Environment properties as a Map or Properties object

Question

I am using annotations to configure my spring environment like this:

@Configuration
...
@PropertySource("classpath:/config/default.properties")
...
public class GeneralApplicationConfiguration implements WebApplicationInitializer 
{
    @Autowired
    Environment env;
}

This leads to my properties from default.properties being part of the Environment. I want to use the @PropertySource mechanism here, because it already provides the possibility to overload properties through several fallback layers and different dynamic locations, based on the environment settings (e.g. config_dir location). I just stripped the fallback to make the example easier.

However, my problem now is that I want to configure for example my datasource properties in default.properties. You can pass the settings to the datasource without knowing in detail what settings the datasource expects using

Properties p = ...
datasource.setProperties(p);

However, the problem is, the Environment object is neither a Properties object nor a Map nor anything comparable. From my point of view it is simply not possible to access all values of the environment, because there is no keySet or iterator method or anything comparable.

Properties p <=== Environment env?

Am I missing something? Is it possible to access all entries of the Environment object somehow? If yes, I could map the entries to a Map or Properties object, I could even filter or map them by prefix - create subsets as a standard java Map ... This is what I would like to do. Any suggestions?

Answer 1

You need something like this, maybe it can be improved. This is a first attempt:

...
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.AbstractEnvironment;
import org.springframework.core.env.Environment;
import org.springframework.core.env.MapPropertySource;
...

@Configuration
...
@org.springframework.context.annotation.PropertySource("classpath:/config/default.properties")
...
public class GeneralApplicationConfiguration implements WebApplicationInitializer 
{
    @Autowired
    Environment env;

    public void someMethod() {
        ...
        Map<String, Object> map = new HashMap();
        for(Iterator it = ((AbstractEnvironment) env).getPropertySources().iterator(); it.hasNext(); ) {
            PropertySource propertySource = (PropertySource) it.next();
            if (propertySource instanceof MapPropertySource) {
                map.putAll(((MapPropertySource) propertySource).getSource());
            }
        }
        ...
    }
...

Basically, everything from the Environment that's a MapPropertySource (and there are quite a lot of implementations) can be accessed as a Map of properties.

Answer 2

This is an old question, but the accepted answer has a serious flaw. If the Spring Environment object contains any overriding values (as described in Externalized Configuration), there is no guarantee that the map of property values it produces will match those returned from the Environment object. I found that simply iterating through the PropertySources of the Environment did not, in fact, give any overriding values. Instead it produced the original value, the one that should have been overridden.

Here is a better solution. This uses the EnumerablePropertySources of the Environment to iterate through the known property names, but then reads the actual value out of the real Spring environment. This guarantees that the value is the one actually resolved by Spring, including any overriding values.

Properties props = new Properties();
MutablePropertySources propSrcs = ((AbstractEnvironment) springEnv).getPropertySources();
StreamSupport.stream(propSrcs.spliterator(), false)
        .filter(ps -> ps instanceof EnumerablePropertySource)
        .map(ps -> ((EnumerablePropertySource) ps).getPropertyNames())
        .flatMap(Arrays::<String>stream)
        .forEach(propName -> props.setProperty(propName, springEnv.getProperty(propName)));

Answer 3

I had the requirement to retrieve all properties whose key starts with a distinct prefix (e.g. all properties starting with "log4j.appender.") and wrote following Code (using streams and lamdas of Java 8).

public static Map<String,Object> getPropertiesStartingWith( ConfigurableEnvironment aEnv,
                                                            String aKeyPrefix )
{
    Map<String,Object> result = new HashMap<>();

    Map<String,Object> map = getAllProperties( aEnv );

    for (Entry<String, Object> entry : map.entrySet())
    {
        String key = entry.getKey();

        if ( key.startsWith( aKeyPrefix ) )
        {
            result.put( key, entry.getValue() );
        }
    }

    return result;
}

public static Map<String,Object> getAllProperties( ConfigurableEnvironment aEnv )
{
    Map<String,Object> result = new HashMap<>();
    aEnv.getPropertySources().forEach( ps -> addAll( result, getAllProperties( ps ) ) );
    return result;
}

public static Map<String,Object> getAllProperties( PropertySource<?> aPropSource )
{
    Map<String,Object> result = new HashMap<>();

    if ( aPropSource instanceof CompositePropertySource)
    {
        CompositePropertySource cps = (CompositePropertySource) aPropSource;
        cps.getPropertySources().forEach( ps -> addAll( result, getAllProperties( ps ) ) );
        return result;
    }

    if ( aPropSource instanceof EnumerablePropertySource<?> )
    {
        EnumerablePropertySource<?> ps = (EnumerablePropertySource<?>) aPropSource;
        Arrays.asList( ps.getPropertyNames() ).forEach( key -> result.put( key, ps.getProperty( key ) ) );
        return result;
    }

    // note: Most descendants of PropertySource are EnumerablePropertySource. There are some
    // few others like JndiPropertySource or StubPropertySource
    myLog.debug( "Given PropertySource is instanceof " + aPropSource.getClass().getName()
                 + " and cannot be iterated" );

    return result;

}

private static void addAll( Map<String, Object> aBase, Map<String, Object> aToBeAdded )
{
    for (Entry<String, Object> entry : aToBeAdded.entrySet())
    {
        if ( aBase.containsKey( entry.getKey() ) )
        {
            continue;
        }

        aBase.put( entry.getKey(), entry.getValue() );
    }
}

Note that the starting point is the ConfigurableEnvironment which is able to return the embedded PropertySources (the ConfigurableEnvironment is a direct descendant of Environment). You can autowire it by:

@Autowired
private ConfigurableEnvironment  myEnv;

If you not using very special kinds of property sources (like JndiPropertySource, which is usually not used in spring autoconfiguration) you can retrieve all properties held in the environment.

The implementation relies on the iteration order which spring itself provides and takes the first found property, all later found properties with the same name are discarded. This should ensure the same behaviour as if the environment were asked directly for a property (returning the first found one).

Note also that the returned properties are not yet resolved if they contain aliases with the ${...} operator. If you want to have a particular key resolved you have to ask the Environment directly again:

myEnv.getProperty( key );

Answer 4

The original question hinted that it would be nice to be able to filter all the properties based on a prefix. I have just confirmed that this works as of Spring Boot 2.1.1.RELEASE, for Properties or Map<String,String>. I'm sure it's worked for while now. Interestingly, it does not work without the prefix = qualification, i.e. I do not know how to get the entire environment loaded into a map. As I said, this might actually be what OP wanted to begin with. The prefix and the following '.' will be stripped off, which might or might not be what one wants:

@ConfigurationProperties(prefix = "abc")
@Bean
public Properties getAsProperties() {
    return new Properties();
}

@Bean
public MyService createService() {
    Properties properties = getAsProperties();
    return new MyService(properties);
}

Postscript: It is indeed possible, and shamefully easy, to get the entire environment. I don't know how this escaped me:

@ConfigurationProperties
@Bean
public Properties getProperties() {
    return new Properties();
}

Answer 5

As this Spring's Jira ticket, it is an intentional design. But the following code works for me.

public static Map<String, Object> getAllKnownProperties(Environment env) {
    Map<String, Object> rtn = new HashMap<>();
    if (env instanceof ConfigurableEnvironment) {
        for (PropertySource<?> propertySource : ((ConfigurableEnvironment) env).getPropertySources()) {
            if (propertySource instanceof EnumerablePropertySource) {
                for (String key : ((EnumerablePropertySource) propertySource).getPropertyNames()) {
                    rtn.put(key, propertySource.getProperty(key));
                }
            }
        }
    }
    return rtn;
}

Answer 6

Spring won't allow to decouple via java.util.Properties from Spring Environment.

But Properties.load() still works in a Spring boot application:

Properties p = new Properties();
try (InputStream is = getClass().getResourceAsStream("/my.properties")) {
    p.load(is);
}

Answer 7

The other answers have pointed out the solution for the majority of cases involving PropertySources, but none have mentioned that certain property sources are unable to be casted into useful types.

One such example is the property source for command line arguments. The class that is used is SimpleCommandLinePropertySource. This private class is returned by a public method, thus making it extremely tricky to access the data inside the object. I had to use reflection in order to read the data and eventually replace the property source.

If anyone out there has a better solution, I would really like to see it; however, this is the only hack I have gotten to work.

Answer 8

Working with Spring Boot 2, I needed to do something similar. Most of the answers above work fine, just beware that at various phases in the app lifecycles the results will be different.

For example, after a ApplicationEnvironmentPreparedEvent any properties inside application.properties are not present. However, after a ApplicationPreparedEvent event they are.

Answer 9

For Spring Boot, the accepted answer will overwrite duplicate properties with lower priority ones. This solution will collect the properties into a SortedMap and take only the highest priority duplicate properties.

final SortedMap<String, String> sortedMap = new TreeMap<>();
for (final PropertySource<?> propertySource : env.getPropertySources()) {
    if (!(propertySource instanceof EnumerablePropertySource))
        continue;
    for (final String name : ((EnumerablePropertySource<?>) propertySource).getPropertyNames())
        sortedMap.computeIfAbsent(name, propertySource::getProperty);
}

Answer 10

I though I'd add one more way. In my case I supply this to com.hazelcast.config.XmlConfigBuilder which only needs java.util.Properties to resolve some properties inside the Hazelcast XML configuration file, i.e. it only calls getProperty(String) method. So, this allowed me to do what I needed:

@RequiredArgsConstructor
public class SpringReadOnlyProperties extends Properties {

  private final org.springframework.core.env.Environment delegate;

  @Override
  public String getProperty(String key) {
    return delegate.getProperty(key);
  }

  @Override
  public String getProperty(String key, String defaultValue) {
    return delegate.getProperty(key, defaultValue);
  }

  @Override
  public synchronized String toString() {
    return getClass().getName() + "{" + delegate + "}";
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    if (!super.equals(o)) return false;
    SpringReadOnlyProperties that = (SpringReadOnlyProperties) o;
    return delegate.equals(that.delegate);
  }

  @Override
  public int hashCode() {
    return Objects.hash(super.hashCode(), delegate);
  }

  private void throwException() {
    throw new RuntimeException("This method is not supported");
  }

  //all methods below throw the exception

  * override all methods *
}

P.S. I ended up not using this specifically for Hazelcast because it only resolves properties for XML file but not at runtime. Since I also use Spring, I decided to go with a custom org.springframework.cache.interceptor.AbstractCacheResolver#getCacheNames. This resolves properties for both situations, at least if you use properties in cache names.

Answer 11

To get ONLY properties, defined in my hibernate.properteies file:

@PropertySource(SomeClass.HIBERNATE_PROPERTIES)
public class SomeClass {

  public static final String HIBERNATE_PROPERTIES = "hibernate.properties";

  @Autowired
  private Environment env;

  public void someMethod() {
    final Properties hibProps = asProperties(HIBERNATE_PROPERTIES);
  }

  private Properties asProperties(String fileName) {
    return StreamSupport.stream(
      ((AbstractEnvironment) env).getPropertySources().spliterator(), false)
      .filter(ps -> ps instanceof ResourcePropertySource)
      .map(ps -> (ResourcePropertySource) ps)
      .filter(rps -> rps.getName().contains(fileName))
      .collect(
        Properties::new,
        (props, rps) -> props.putAll(rps.getSource()),
        Properties::putAll);
  }
}

Answer 12

A little helper to analyze the sources of a property, which sometimes drive me crazy . I used this discussion to write SpringConfigurableEnvironment.java on github.

It could be used in a test:

@SpringBootTest
public class SpringConfigurableEnvironmentTest {
    @Autowired
    private ConfigurableEnvironment springEnv;

    @Test
    public void testProperties() {
        SpringConfigurableEnvironment properties = new SpringConfigurableEnvironment(springEnv);
        SpringConfigurableEnvironment.PropertyInfo info = properties.get("profile.env");
        assertEquals("default", properties.get(info.getValue());
        assertEquals(
           "Config resource 'class path resource [application.properties]' via location 'optional:classpath:/'", 
           info.getSourceList.get(0));
    }
}

Answer 13

All answers above have pretty much covers everything, but be aware of overridden values from environment variables. They may have different key values.

For example, if a user override my.property[1].value using environment variable MY_PROPERTY[1]_VALUE, iterating through EnumerablePropertySources.getPropertyNames() would give you both my.property[1].value and MY_PROPERTY[1]_VALUE key values.

What even worse is that if my.property[1].value is not defined in applications.conf (or applications.yml), a MY_PROPERTY[1]_VALUE in environment variables would not give you my.property[1].value but only MY_PROPERTY[1]_VALUE key value from EnumerablePropertySources.getPropertyNames().

So it is developers' job to cover the those properties from environment variables. Unfortunately, there is no one-on-one mapping between environment variables schema vs the normal schema, see the source code of SystemEnvironmentPropertySource. For example, MY_PROPERTY[1]_VALUE could be either my.property[1].value or my-property[1].value