ecmwf / conflator Goto Github PK

Conflator is a configuration-handling library for Python. It is designed to simplify the handling of configuration from multiple sources, such as environment variables, command line arguments, and configuration files. As an application or library developer, you specify your configuration schema with a Pydantic model, and conflator will handle the rest.

Conflator loads configuration in the following order:

1. Default values specified in the Pydantic model
2. System-wide configuration in /etc/appname/config.json (and yaml)
3. User configuration in ~/.appname.json (and yaml)
4. Additional configuration files and values provided as command line args (e.g. -f filename or --set value.deeper=foo)
5. Environment variables
6. Command-line arguments
7. Dictionaries passed to the load method

...and then validates the merged configuration against the Pydantic model.

Conflator is available on PyPI:

pip install conflator

Or using poetry:

poetry add conflator

1. Define Your Configuration Model: Begin by defining your configuration schema using Pydantic models. Annotate your model fields with EnvVar and CLIArg for environment variable and command-line argument support, respectively.

from pydantic import Field
from conflator import EnvVar, CLIArg, ConfigModel
from annotated_types import Annotated

class AppConfig(ConfigModel):

    host: str = "localhost"
    port: Annotated[int, EnvVar("PORT"), CLIArg("--port")] = 5432
    user: Annotated[str, EnvVar("USER"), CLIArg("--user"), Field(description="Your username")] = "foo"

2. Initialize Conflator: Create an instance of the Conflator class, passing your application's name and the configuration model.

from conflator import Conflator

config = Conflator(app_name="my_app", model=AppConfig).load()

3. Access Configuration: Use the loaded configuration throughout your application, knowing that the configuration has been fully validated.

print(f"User: {config.user}")

Try setting MY_APP_USER in the environment to see the value change, or use the --user flag to override the value.

your-app -f ./config/base.yaml -f ./config/production.yaml

from annotated_types import Annotated
from conflator import EnvVar, CLIArg, ConfigModel

class DeeperConfig(ConfigModel):
    nested: Annotated[str, EnvVar("NESTED"), CLIArg("--nested")] = "default"

class Config(ConfigModel):
    host: str = "localhost"
    port: int = 543
    deeper: DeeperConfig = DeeperConfig()

config = Conflator(app_name="my_app", model=AppConfig).schema() # uses pydantic's schema method behind the scenes

• CLI arguments and environment variables are defined per model type, not per model instance. This means that you cannot have different CLI arguments or environment variables for different instances of the same model. Some support for this may be added in the future.

• Including ConfigModel's from other packages, to nest their configuration with yours, is possible. However, there is no way to resolve conflicts between CLI arguments and environment variable naming between two different config schemas. Some support for this may be added in the future.