[bug] Default value on empty string about dynaconf HOT 9 OPEN

Thank you @pedro-psb!

In the meantime, I've found a small hack that can be used as a workaround as well. You can use when + cast to set a default when the key is "":

I confirm that indeed the proposed hack works:

• No key: KML FILE: 'D:\Progetti\danea-automation\src\veryeasyfatt\output.kml'
• Empty key: KML FILE: 'D:\Progetti\danea-automation\src\veryeasyfatt\output.kml'
• Key with value (kml = "test"): KML FILE: 'test'

To be able to use cast as usual (in this case to cast the value to a Path object) i slightly changed your code:

Dynaconf(
        ...,
        Validator(
            "files.output.kml",
            when=Validator("files.output.kml", eq=""),
            cast=lambda value: Path("output.kml") if str(value).strip() == "" else Path(value),
            default="output.kml", # applies when the value was not set at all
        )
)

This way, even if the key is not defined, cast gets the default value as a parameter, in the end converting it to Path.

from dynaconf.

@LukeSavefrogs I'm not entirely sure if this is expected, because casting empty string to None is a specific behavior of the YAML parser.

But as you pointed out, this would probably be covered if the ideas in #973 are carried on, which would be great. There is an undergoing refactor on validation implementation of the default setting, so we should await for that, anyway.

In the meantime, I've found a small hack that can be used as a workaround as well. You can use when + cast to set a default when the key is "":

Dynaconf(
        ...,
        Validator(
            "files.output.kml",
            when=Validator("files.output.kml", eq=""),
            cast=lambda value: "output.kml", # hack to use default when str is empty
            default="output.kml", # applies when the value was not set at all
        )
)

from dynaconf.

We must add the workaround example as a test_functional so we ensure it keeps working.

from dynaconf.

Sorry I'm a bit confused, shouldn't the actual solution to use the apply_default_on_none as told in the documentation?

If so, this is a bug right?

from dynaconf.

YAML reads empty values as None in this case if you want to have Validator defaults applied to the None values you must set apply_default_on_none to True on Dynaconf class or specific on Validator instance.

@LukeSavefrogs Empty values in this case are key-value pairs with no pairs in yaml: key: (no value after), and it is not the same as empty strings. (e.g. TOML parser doesnt even allow empty values, in this sense).

When this happens, YAML parser assign a None value to key. The way I interpret this is that it should apply default when a strict None value is used, and "" is not None.

from dynaconf.

I've removed this as a bug because of what I've explained in the previous post about apply_default_on_none behavior.
So we should:

• add a functional test to ensure the workaround keeps working
• improve the meaning of apply_default_on_none in the documentation.

Also, one possible way to improve user experience in this use case is to add an "interpret-empty-str-as-none" option. I personally don't like it, as we already have the @empty token which translates to None, but it's a possibility.

from dynaconf.

Also, one possible way to improve user experience in this use case is to add an "interpret-empty-str-as-none" option. I personally don't like it, as we already have the @empty token which translates to None, but it's a possibility.

Instead of a new flag which could be very confusing how about a transform function (or list) which runs before any validation occurs and changes the value as per user?
This way one could write a simple lambda lambda s: s if s.strip() != "" else None and pass it to the transform parameter so that every key is converted

What do you think?

from dynaconf.

It's an interesting idea to have a more broad custom parsing step. I see the general use cases where you can't directly instruct the person writing the conf files about all dynaconf-specific idiosyncrasies and want to use a custom interpretation layer in between to match their specific needs.

which runs before any validation occurs and changes the value as per user

But "before" should really be completely out of the validation (just enforcing, not sure if you didn't already mean that). We are currently trying to remove any side-effect from the validation process in order to make the codebase more decoupled and maintainable, as validation side-effects have been a particularly noticeable source of bugs. Conceptually it's purpose is to validate a fixed state, so calls to it shouldn't change state.

I believe this should be placed the nearest possible from parsers, like an adapter layer for the user, which would be mostly his responsibility. And it should make it easier for him to debug and reason about later. We already have a basic hooking system, so this could be like a pos_parsing_hook or similar.

from dynaconf.

Another possible solution would be to add an access hook to the other end of the flow, on the access layer. When accessing any given key, we would run registered hooks that can implement this fallback logic on empty string values.

This fallback logic on access is related to what we are trying to do on #988. It's easier to implement when using settings.get("a.b.c"), but more tricky with settings.a.b.c.

Also, it may be helpful to have a look at #975 for hooking. Although it was implemented for a specific django use-case, we could try generalizing it.

from dynaconf.