implement the ability to plugin fallback values #236

andponlin · 2025-12-05T08:18:52Z

The config system currently obtains fallback values from system properties and env-vars. This is not always desirable. This PR provides a way to make the fallbacks pluggable so that a project can opt to provide a fallback in a different way.

- rename ConfigurationFallbacks to ConfigurationFallback - Change List<ConfigurationFallbacks> fallbacks to single ConfigurationFallback fallback - Make DefaultFallback final - Rename method Configuration.Builder.withFallbacks() to fallback()

rbygrave · 2025-12-06T23:02:35Z

Hi @andponlin I've push a commit that changes this from using a List<ConfigurationFallbacks> to a single ConfigurationFallback (plus some refactor renames).

Can you review those changes I've made and see if you are happy with them? Thanks, Rob.

andponlin · 2025-12-07T05:10:48Z

avaje-config/src/main/java/io/avaje/config/ConfigServiceLoader.java

      } else if (spi instanceof ModificationEventRunner) {
        _eventRunner = (ModificationEventRunner) spi;
+      } else if (spi instanceof ConfigurationFallback) {
+        _fallback = (ConfigurationFallback) spi;


It's probably a good idea to either fail if there are two instances of ConfigurationFallback configured; otherwise the behaviour may become unpredictable.

I think I have a use case where for Tests, it would be useful to use an alternative ConfigurationFallback.

In this case the test configuration would have values like ssm://test/myapp/some_key_of_secret

... and if a value had that special prefix it would load the value from the AWS secret store. You need aws single signon to be able to do this of course. This is to assist 2 scenarios which is (1) running locally against Dev/Test and (2) Integration tests

I think the override feature would actually support this and ideally this is only done for Testing, so the override implementation is only in maven test scope.

@rbygrave ; I think it may be good to look into throwing in this scenario because otherwise the behaviour could be confusing and difficult to track down.

I think I have a use case where for Tests, it would be useful to use an alternative ConfigurationFallback.

So in this case there would be an instance of ConfigurationFallback from the main classpath and an instance from the test classpath? It may be worth logging to stderr (at least) that this is happening because in other situation a clash will be hard to comprehend.

andponlin · 2025-12-07T05:12:36Z

Can you review those changes I've made and see if you are happy with them? Thanks, Rob.

Afternoon @rbygrave ; thanks for the review on this and totally happy with the change to have a single ConfigurationFallback configured. I have made another sugestion here but otherwise all good to me. 👍

rbygrave · 2025-12-07T18:42:50Z

currently obtains fallback values from system properties and env-vars. This is not always desirable.

What is the actual use case @andponlin for this? What case are we hitting where the current behaviour isn't desirable and what are we replacing the system properties/env vars with in that case?

andponlin · 2025-12-07T19:04:07Z

What case are we hitting where the current behaviour isn't desirable and what are we replacing the system properties/env vars with in that case?

What I would ultimately like to achieve is that env-vars and system properties (in that order) are overrides for config keys and not the other way around such that they are fallbacks. I want to be able to override a value via env-var regardless if it is configured via configuration files or not. I also want to use a different mapping from the configuration key to the env-var name.

In my case, I would implement ConfigurationFallback to prevent a fallback being possible at all; it always returns null. I initially implement this PR as a flag to disable fallbacks but then maybe some people would prefer to have fallbacks for just the system property or just env-vars or maybe they have some other source of fallbacks or maybe they want to read from a differently formatted env-var name. This means with the ConfigurationFallback is more flexible and general for handling such cases.

To implement the override behaviour, I am currently implementing ConfigurationPlugin and then iterating the keys and replacing the values but this is not working if there is a configuration key for which there is no values configured in a file. I may need to likewise suggest an SPI interface to be able to allow for dynamic overrides.

Does this make sense? 🤔

SentryMan · 2025-12-07T20:59:50Z

What I would ultimately like to achieve is that env-vars and system properties (in that order) are overrides for config keys and not the other way around such that they are fallbacks. I want to be able to override a value via env-var regardless if it is configured via configuration files or not. I also want to use a different mapping from the configuration key to the env-var name.

To borrow a phrase from my stance on social media, "I don't follow". How does disabling fallback translate to env variables being treated as the override. Is it possible to provide a more concrete example?

rbygrave · 2025-12-07T21:01:53Z

ultimately like to achieve is that env-vars and system properties (in that order) are overrides for config keys

Right, makes sense.

In my case, I would implement ConfigurationFallback to prevent a fallback being possible at all;

Yeah.

My gut is suggesting that this PR doesn't actually resolve the issue for your use case? The use case of ... "env-vars and system properties (in that order) are overrides" ... actually needs a different PR in order to achieve that ... and then if we do that do we actually need this pluggable ConfigurationFallback? Maybe but maybe not?

Seems to me we should be looking at the source issue of supporting "env-vars and system properties (in that order) are overrides" first, and then seeing where that gets us. We might still want this pluggable ConfigurationFallback but we should probably solve the other issue first to make sure this still makes sense? Or said differently, I'd feel more comfortable leaving this PR un-merged until we've sorted the "overrides" issue just to make sure it still makes sense?

SentryMan · 2025-12-08T05:31:39Z

To implement the override behaviour, I am currently implementing ConfigurationPlugin and then iterating the keys and replacing the values but this is not working if there is a configuration key for which there is no values configured in a file.

Another approach might be to wrap the Config class in a wrapper that checks the env variable first before calling Config.get

public class ConfigWrap {
  public static String get(String key) {
    String override = deriveOverride(key);

    return override != null ? override : Config.get(key);
  }

  private static String deriveOverride(String key) {
    String systemPropertiesValue = System.getProperty(key);

    if (null != systemPropertiesValue) {
      return systemPropertiesValue;
    }

    return System.getenv(mapKeyToEnvVar(key));
  }

  private static String mapKeyToEnvVar(String key) {
    key = key.replace(".", "_");
    key = key.replace("-", "");
    return key.toUpperCase();
  }
}

rbygrave · 2025-12-08T08:04:43Z

A potential alternative is to modify the interface Configuration.ExpressionEval. Currently this only takes the value and doesn't also yet take the key ... but IF it also took the key and became String eval(String key, String value) ... and that was an implementation that was a plugin ...

... and a variation is to put a change into ConfigurationCore before (and after?) the call to eval(String) [and I'll have a play there ...]

andponlin · 2025-12-08T09:07:50Z

Is it possible to provide a more concrete example?

Consider a config key hds.tool.convert.limit-memory
optionally override hds.tool.convert.limit-memory-s configured value with
1. an env-var HDS_TOOL_CONVERT_LIMITMEMORY
2. a system property hds.tool.convert.limit-memory
optionally supply the key's value with the regular configuration file(s) (as-built)
no fallback

My gut is suggesting that this PR doesn't actually resolve the issue for your use case?

Yes you're right Rob, this PR doesn't achieve the whole desired outcome. It enables only the last bit. The rest would be a follow-on.

Another approach might be to wrap the Config class in a wrapper that checks the env variable first before calling Config.get
A potential alternative is to modify the interface Configuration.ExpressionEval.

OK, I think that would be a good extension point that sounds as if it would allow for bespoke overrides.

- add overrideValue() method - Invoke overrideValue() on initial load and builder load

rbygrave · 2025-12-08T09:58:59Z

I've pushed a change that adds a overrideValue() method to ConfigurationFallback, and invokes it on initial load and build.

Note that ConfigurationSource plugins like the AWS AppConfig one actually bypass the overrideValue() method. That is, the ConfigrationCode.applyChanges() method it doesn't use the fallback instance (but it could, hmm, pondering this).

In theory this gives us the options we need for this use case (A) control overriding the values (B) control fallback value

andponlin · 2025-12-08T10:06:32Z

avaje-config/src/main/java/io/avaje/config/ConfigurationFallback.java

+   * @param source The source of the key value pair
+   */
+  default String overrideValue(String key, String value, String source) {
+    return value;


nit: I guess the implementation of ConfigurationFallback is not able to manipulate the source as well here? That would be a nice addition if it could but likely involves returning a record, CoreEntry or similar. Maybe not worth it.

Pushed a commit that returns Entry, allows us to override the source along with the value.

rbygrave · 2025-12-08T10:28:02Z

limit-memory ... hypens

According to my research, hypens are not allowed in POSIX Standard environment variable names. So we could provide a helper method that does the env var name conversion properly including the trimming out of - chars (and treat the lack of that in the current implementation as a bug).

andponlin · 2025-12-08T10:33:28Z

According to my research, hypens are not allowed in POSIX Standard environment variable names.

In the example above, I'm mapping hds.tool.convert.limit-memory (config key / system property) to HDS_TOOL_CONVERT_LIMITMEMORY (env-var) with the hyphen removed. See what SpringBoot is doing in this regard.

This allows us to override the source along with the value. A downside is that we need to expose more methods on the Entry interface that were otherwise internal to CoreEntry.

rbygrave · 2025-12-08T11:08:41Z

(env-var) with the hyphen removed. See what SpringBoot is doing in this regard.

Yes. I think we should also trim the hypens and make that change as a [separate] bug fix.

Rather than nullable, and thus allow the source to also be provided by the fallback method.

andponlin · 2025-12-08T19:06:22Z

avaje-config/src/main/java/io/avaje/config/ConfigurationFallback.java

+import java.util.Optional;
+
+/**
+ * Implementations of this class are able to define a fallback value when there is no


I am wondering if the name ConfigurationFallback is the best since it has a dual purpose of providing a fallback and an overrride. Maybe ConfigurationValueInterceptor or ConfigurationOverrideAndFallback might be clearer?

Even if this name is retained as-is, the class comments would be good to update so that they describe the role of the SPI interface.

SentryMan · 2025-12-08T19:11:22Z

perhaps instead of a new fallback interface, we should change the behavior in general to prefer system/env variables over configuration files. That is, if a system property/env is set, prefer that over all else

rbygrave · 2025-12-08T19:11:36Z

Ok, I think I'm happy with this change.

In doing this, I have the thought that the behaviour being asked for is actually consistent with how Spring does it [I will see if I can confirm that today]. In that case, I'm wondering if we should actually change the default behaviour here (so as a 5.0 release actually change the default behaviour).

What are you guys thinking?

andponlin · 2025-12-08T19:14:53Z

avaje-config/src/test/java/io/avaje/config/CoreConfigurationTest.java

+    @Override
+    @Nullable
+    public Optional<Entry> fallbackValue(String key) {
+      String val = System.getProperty(key);


For testing purposes, it is better not to just simulate/mock the System.getProperty behaviour? I fear that if the tests are run concurrently that there may be clash in the runtime state of the properties as they are essentially static.

andponlin · 2025-12-08T19:23:40Z

perhaps instead of a new fallback interface, we should change the behavior in general to prefer system/env variables over configuration files.
I'm wondering if we should actually change the default behaviour here (so as a 5.0 release actually change the default behaviour).

I agree that getting alignment with Spring on this would be really good as "out of the box" behaviour on the next major version.
I think the interfaces Rob has added appear to make sense to me and if somebody wanted to do something more bespoke they would be enabled to do so by using these interfaces.

BTW; will need to drop off now -- will reply to anything this evening (NZDT).

SentryMan · 2025-12-08T19:35:45Z

I think the interfaces Rob has added appear to make sense to me

they make sense to me as well, but if the only usecase at the moment is solved by spring env behavior, I'm not really seeing the point in adding it

rob-bygrave · 2025-12-08T19:56:41Z

but if the only usecase at the moment is solved by ...

Yeah, I'm feeling that too. Leaning to making this a behaviour change without introducing the interface/plugin at least initially and seeing how that goes.

rbygrave · 2025-12-08T22:06:44Z

FYI: Spring Behaviour: System Properties overrides Environment Variables overrides ... property source values

[So Andrew, this was a touch different in that System Properties trumps Environment Variables ... is that an issue?]

I think we adjust this PR such that it matches Spring behaviour, and remove the interface/pluggable part at least initially and look at that as a 5.0 version with that behaviour change [and maybe release what's in master now as 4.3?]

rob-bygrave · 2025-12-08T22:52:35Z

I think we should use #239

rbygrave · 2025-12-09T07:33:05Z

Replaced by #239

rbygrave · 2025-12-09T07:35:38Z

@andponlin Try version 5.0-RC1 ... which matches Spring behaviour and I think is what you are looking for (except noting that System properties trump Environment vars).

andponlin · 2025-12-09T08:13:25Z

Hi @rbygrave ;

which matches Spring behaviour and I think is what you are looking for (except noting that System properties trump Environment vars).

It seems more logical that it would be the other way around with env-vars being precedence but for my situation specifically where I a using this it won't be a problem. Thanks both; I will try this out.

andponlin · 2025-12-09T08:26:23Z

Tried it out; seems to be working well 👍

andponlin and others added 2 commits December 5, 2025 21:14

implement the ability to plugin fallback values

8fe989b

Merge branch 'master' into andponlin-fallback-value

58e028e

SentryMan requested review from rbygrave and rob-bygrave December 5, 2025 15:06

rbygrave assigned andponlin Dec 6, 2025

andponlin commented Dec 7, 2025

View reviewed changes

Change ConfigurationFallback to also have an overrideValue() method

31e6d3c

- add overrideValue() method - Invoke overrideValue() on initial load and builder load

andponlin commented Dec 8, 2025

View reviewed changes

Add test for ConfigurationFallback overrideValue() method

d828b9a

Modify ConfigurationFallback overrideValue() to return Entry

d00ca80

This allows us to override the source along with the value. A downside is that we need to expose more methods on the Entry interface that were otherwise internal to CoreEntry.

rbygrave added 2 commits December 9, 2025 00:58

Modify ConfigurationFallback fallbackValue() to return Optional<Entry>

84ceafa

Rather than nullable, and thus allow the source to also be provided by the fallback method.

Merge branch 'master' into andponlin-fallback-value

8da4c81

rbygrave requested a review from SentryMan December 8, 2025 18:59

andponlin commented Dec 8, 2025

View reviewed changes

Merge branch 'master' into andponlin-fallback-value

303bd7d

rbygrave closed this Dec 9, 2025

implement the ability to plugin fallback values #236

implement the ability to plugin fallback values #236

Uh oh!

Conversation

andponlin commented Dec 5, 2025

Uh oh!

rbygrave commented Dec 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andponlin commented Dec 7, 2025

Uh oh!

rbygrave commented Dec 7, 2025

Uh oh!

andponlin commented Dec 7, 2025

Uh oh!

SentryMan commented Dec 7, 2025

Uh oh!

rbygrave commented Dec 7, 2025

Uh oh!

SentryMan commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

andponlin commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

andponlin commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SentryMan commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andponlin commented Dec 8, 2025

Uh oh!

SentryMan commented Dec 8, 2025

Uh oh!

rob-bygrave commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 8, 2025

Uh oh!

rob-bygrave commented Dec 8, 2025

Uh oh!

rbygrave commented Dec 9, 2025

Uh oh!

rbygrave commented Dec 9, 2025

Uh oh!

andponlin commented Dec 9, 2025

Uh oh!

andponlin commented Dec 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants