Why do some functions return this undocumented pseudo-Result type? It makes normal Rust-like error handling a pain. I expected ConfigResult to be an alias for Result<T, ConfigError>, but it's not. I had to dig through the source code to find out what it actually is. It does not work with try! or the ? operator and does not expose any methods to turn it into an ordinary Result. It makes working with error handling crates such as error-chain extremely unwieldy. I assume it's supposed to be used to chain method calls, but even then there is no "terminal" method (deserialize?) other than unwrap/expect, which is just bad practice. Why not used an ordinary Result type and use ? to chain calls together instead of having a specialized, undocumented and panic-based ConfigResult type?

Right now, I have to do something silly like this:

if let Some(err) = config.foo().bar().err() {
    return Err(err);
}

Instead of just:

config.foo()?.bar()?;

I tried parsing some YAML files (using this example). However, if YAML happens to contain sub heading/parameter like:

sub_heading:
  key: value

It errors out with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: invalid type: map, expected a string', /checkout/src/libcore/result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.

It would be nice to have support for such YAML files.

Hi I use set_default() to set default values. How can I write these back to the config file?

Would you consider adding CLI support? A scenario I'm thinking of would work very nicely if you could override most/all of the config from CLI input, and it seems redundant to use clap to find the config file, get the contents of the config file, and then clobber those values with CLI inputs where they exist.

In the C# world, I'd add the https://www.nuget.org/packages/Microsoft.Extensions.Configuration.CommandLine/ package, which would then allow inputs in a number of forms, which would then override the config file settings depending on precedence.

Hi,

I am trying to use the environment only, to grep the configuration. Because that did not work, I also tried with an additional Settings.toml. Then the value from file is picked up, but not the environment.

Code:

let mut settings = config::Config::default();
settings
   .merge(config::File::with_name("Settings"))
    .unwrap()
    .merge(config::Environment::with_prefix("APP"))
    .unwrap();

let x = settings.get::<String>("foo").expect("ok");
println!("CFG: {:?}", x);

Settings.toml:
foo = "20"

Calling the program with APP_FOO=10 target/debug/prog. Only the value from toml is picked up.

I must be doing something very stupid, but I cannot get this to work. Can you spot my error?

UPDATE Also tried the simple example, with APP_DEBUG=FOO without success. The toml file wins.

The crazy thing is that to me the code in Environment looks fine, I am just not getting what is going on.

UPDATE
See new comment #47 (comment) below.

Global mutable state is hard (see #9) and doesn't seem worthwhile to me in this case. Have you considered exposing only the config::Config::get/set API rather than global config::get/set? Are there any use cases that you think would be difficult to express using config::Config rather than global?

Currently we support a basic x.y.z traversal (not on master currently but on v0.2 and (when released) v0.3) that traverse tables (but not arrays).

This issue is to discuss a more powerful path traversal.

I'm thinking of a subset of JSONPath:

{
  "a": [10, 20],
  "x": {
    "w": 30,
    "y": { "z": 32 }
  }
}

• . - Child - x.y.z = 32
• .. - Descent - ..z = 32
• * - Wildcard - x.* = `{ 30, { "z": 32 } }``
• [] - Subscript - a[0] = 10

http://jsonpath.com/?

How much of JSON Path should we include? I don't see any reason to support filters / scripting for instance.

Loading configuration directly using Serde produces way better error messages than this crate. By converting everything to config::Value you lose all the valuable information about where everything came from.

As an example of good error messages, the config in Alacritty contains:

#[derive(Deserialize)]
struct Config {
    font: Font,

    /* other fields */
}

#[derive(Deserialize)]
struct Font {
    use_thin_strokes: bool,

    /* other fields */
}

They load it like this:

let config: Config = serde_yaml::from_str(&config_yml_path)?;

Suppose somebody typos a boolean:

font:
  use_thin_strokes: tru

The error would look like:

font.use_thin_strokes: invalid type: string "tru", expected a boolean at line 51 column 21

The error message gives the exact line and column to fix the error.

I think it should be possible to improve by implementing config::get as a DeserializeSeed that runs against the original config source (the yaml file or whatever). Basically the seed is the key that you are looking up and the DeserializeSeed implementation will deserialize that key from the config file, keeping track of whether that key was found. If not found, it can try again on the next source.

let redis_client = redis::Client::open(settings.get("redis_url").unwrap()).unwrap();

invalid type: string "...", expected a borrowed string

This is because of type conversions. We must return a String because there may have been a type conversion.
I can see this fixed in two ways:

• Fix this by allowing to get out a &str if the underlying value is stored as a String. I don't like this because it'd only work sometimes.
• Fix this by storing the type conversion in the cache when the value is requested. This makes a kind of sense because I believe we can optimize for users requesting the same type of value over-and-over-again. I was already going to add an initial "guess" that losslessly converts your incoming config value on the assumption that if you give us a "1" through an environment variable you probably meant 1.

As a side note I'm starting a medium-large project in Rust (finally) and am using this crate. Hopefully will find a ton of usage issues and annoyances (like this).

I am using a json file as configuration file, however I can't get correct value with c.get("Server") when there are capital letters in the key field, then I found this:

    pub fn get(&self, key_path: &str) -> Option<Value> {
        let key_expr: path::Expression = match key_path.to_lowercase().parse() {
            Ok(expr) => expr,
            Err(_) => {
                // TODO: Log warning here
                return None;
            }
        };

        self.path_get(key_expr).cloned()
    }

I am curious why it design like this, am I missing something.

That way, we can merge configs after hand building them, or having already generated them from other sources

I just hit that.

I'm using config-rs in a piecemeal way, and I would like to be able to compare the equality of two tables. This should be as simple as adding the correct #[derive()] annotations.

If this makes sense, I would be happy to provide a PR.

I find the impl Source for Vec in the current version of the library ugly. It requires the list of config sources to always be a Vec, which is ugly and constraining.

I think it would be a much better idea to instead impl Source for slices and for IntoIterator.

impling it for slices would allow using static arrays as well as Vecs.

impling it for IntoIterator would make it possible to use the merge method on any Iterator of sources, without collecting into a Vec. This makes things more elegant and removes unnecessary performance overhead when working with things like glob (such as the glob example in the repo).

I think this change would make the API more elegant and would be desirable for version 1.0.

• API Documentation

• Clean up implementation and ensure all public methods have several unit tests (where possible). Sources should have integration tests. Look into code coverage.

• Clean up implementation warts. There is a few areas of duplicated impls. Mostly in the config deserializer.

• Change config::Environment to only collect env vars for keys that are already present in the config instance.

• Allow selecting which environment variables to consume

  Environment::new().keys(&["debug"])

• Consider deferring configuration loading until the first access. Would require interior mutability, however, the docs indicate that "caching" is an accepted usage of interior mutability and that's essentially what the configuration loading is doing.

• Make Config#merge API more ergonomic for simple use cases.

  config.merge("Settings.toml");  // &str impls Into<Path> which impls Into<config::File>

• Round out format support: xml, hjson, libconfig, .env

• Remote support: etcd and consul

• Rename Config#refresh to Config#reload

• #19

If you have any pain points when using Config or you'd like to share how you use Config, today, please comment below.

Currently this library can't be used in the stable channel, due to #[feature].

Is it possible to have a version of config-rs compatible with stable rust?

This is particularly nice for production systems that have potentially large number of config files (e.g. nginx, apache). I think that the sensu docs in particular describe the behavior and motivation pretty well.

In an application, sometimes there is no config present and you would just like to use the default values for your struct. If this is the case currently, you get an obtuse error like: invalid type: unit value, expected struct Settings. As an example usage:

#[derive(Debug, Deserialize)]
pub struct Settings {
    db_host: string,
}

impl Default for Settings {
    fn default() -> Self {
        Settings {
            db_host: IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)),
        }
    }
}

impl Settings {
    pub fn new() -> Result<Self, ConfigError> {
        let mut s = Config::new();
       // hand waiving pseudo code
        if config_file.present() {
           s.merge(File::with_name(config_file))?;
       }
       s.try_into()
}

Consider a configuration with a default of "a.a", one source that provides "a.b" and "a.c", a second source that provides "a.c" and "a.d" and an override for "a.e". If you get any of these specific keys from the config, it works fine. However, if you just get "a", then you will get a table that only has the "c" and "d" keys from the second source. It seems like there should be a way to get a merged object with all five keys.

One way this could be done would be to have a SubConfig object with a reference to the root config and a prefix that it uses to do lookups.

File::with_name("name");

^ Should attempt the basename and all registered extensions when looking for a file. Fallback to an error message if the file is found and not of an expected type.

Hi there,
I'm new to rust and config-rs, and I'm facing an annoying problem: I can't get structs with #[serde(rename_all = "camelCase")] to work.
I've setup a simple demo repository here to illustrate my problem.
My Toml:

snakeTest = "foo"

My struct:

#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct Settings {
    snake_test: String,
}

My main:

fn main() {
    let mut c = config::Config::default();
    c.merge(config::File::new("Settings", config::FileFormat::Toml))
        .unwrap();

    // Deserialize the entire file as single struct
    let s: Settings = c.try_into::<Settings>().unwrap();

    println!("{:?}", s);
}

The error:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: missing field `snakeTest`', libcore/result.rs:916:5

Am I missing something? Thanks!

Hello. I'm quiet new to rust, so it will be good for me if you add some advanced examples of usage, especially to hierarchical-env example.

What's the best or correct way of accessing nested config values in main.rs, like it's done here:

In the spirit of making this library easier to get into and use, I'm thinking we should recommend a convention for configuration (that could, of course, be disabled with feature flags).

1 - ./config folder with ./config/default.toml and ./config/{env}.toml

config::merge(config::File::with_name("default").path("config"));
config::merge(config::File::with_name(env!("RUST_ENV")).path("config").required(false));
config::merge(config::Environment::new(""));

2 - ./Settings.toml / ./Settings.{env}.toml

config::merge(config::File::with_name("Settings"));
config::merge(config::File::with_name(&format!("Settings.{}", env!("RUST_ENV"))).required(false));
config::merge(config::Environment::new(""));

3 - ./Settings.toml with namespaces

config::merge(config::File::with_name("Settings").namespace(env!("RUST_ENV")).required(false));
config::merge(config::Environment::new(""));

An example of 3:

[development]
debug = true

[production]
debug = false

• This is using a unwritten feature of config::File to auto-detect the file type

The idea is that the equivalent of the above is done automatically. I'm partial to 3 but I'm open to other suggestions.

Hi,

I'm interested in getting the underlying error out of ConfigResult if it happens to be an error. In particular, if the error is io::ErrorKind::NotFound, I'd like to just return the default value for config. Unfortunately, I can't seem to get anything I've tried to compile (I'm new at rust, so forgive me :):

fn get_config(file: &str, domain: String) -> Result<Domain, config::ConfigError> {         
    let mut c = config::Config::new();                                                     
    let err = c.merge(config::File::with_name(file)).err();                                
    match err.and_then(|e| e.cause()) {                                                    
        Some(std::io::Error{kind: ErrorKind::NotFound}) => return Ok(Default::default()),  
        Some(e) => return Err(e),                                                          
        None => (),                                                                        
    }                                                                                      
    let map = c.deserialize::<HashMap<String, Domain>>();                                  
    eprintln!("map: {:?}", map);                                                           
    map.map(|m| m.get(&domain).map(|d| d.clone()).unwrap_or(Default::default()))           
}

is my code, which unfortunately fails with:

error[E0308]: mismatched types
   --> src/main.rs:119:14
    |
119 |         Some(std::io::Error{kind: ErrorKind::NotFound}) => return Ok(Default::default()),
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found struct `std::io::Error`
    |
    = note: expected type `&std::error::Error`
               found type `std::io::Error`

error[E0026]: struct `std::io::Error` does not have a field named `kind`
   --> src/main.rs:119:29
    |
119 |         Some(std::io::Error{kind: ErrorKind::NotFound}) => return Ok(Default::default()),
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^ struct `std::io::Error` does not have field `kind`

error[E0027]: pattern does not mention field `repr`
   --> src/main.rs:119:14
    |
119 |         Some(std::io::Error{kind: ErrorKind::NotFound}) => return Ok(Default::default()),
    |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ missing field `repr`

error[E0308]: mismatched types
   --> src/main.rs:120:31
    |
120 |         Some(e) => return Err(e),
    |                               ^ expected enum `config::ConfigError`, found reference
    |
    = note: expected type `config::ConfigError`
               found type `&std::error::Error`

I guess I need to do something more clever here, but I'm at a loss for what. This also seems very convoluted, there must be an easier way. Thoughts?

Hello,
When merging a non-required file, in my opinion an error should be triggered if the file exists but cannot be parsed (invalid syntax in yaml, for example).
Currently, the result is Ok(), even if the parsing has failed.

What is your opinion about this behavior?
Thanks!

Currently it doesn't do a full rebuild for File sources. It should re-read the file contents.

This bears discussion though.

c.set("key", 5);
// c.refresh();

c.merge(File::with_name("Settings"));
// c.refresh();

c.set("key", 5);
// c.refresh();

See above. Currently a .refresh call is done automatically after any stack mutation.

• If we switch this to re-reading files it will definitely increase execution time of any .set's
• The Envrionment source should probably have a delete option (that deletes keys from the environment after reading). It wouldn't make any sense to re-load then.

It sounds like we a source needs to have its own frozen state.

Any thoughts on API here?

• Should sources be frozen by default?
• Should sources have a .freeze() / .live(true) method when building them?

maybe instead of settings programmatic defaults one by one, I can provide the whole config string as is to be loaded?
this way I can have a file structure as string embedded in the app/library and loaded and on top load an external file which is already supported.
loading a string structure instead of doing it one by one can really help readability of the config and its must easier to code.

Forward to the contained Value

• impl Deserialize for Config
• Add Config#try_into to mirror Value#try_into

extern crate config;

use std::collections::HashMap;

fn main() {

	let mut settings = config::Config::default();
	let file = config::File::with_name("bla");
	{
		let result = settings.merge(file);

		match result {
			Ok(_) => (),
			Err(e) => {
					println!("{:?}",e);
					println!("Failed to read the settings file, skipping...");
			}
		}
	}
	settings.merge(config::Environment::with_prefix("bla")).unwrap();
	println!("{:?}", settings.try_into::<HashMap<String, String>>().unwrap());
}

The Environment::with_prefix unwrap panics. Why does it? Output:

configuration file "bla" not found
Failed to read the settings file, skipping...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: configuration file "bla" not found', src\libcore\result.rs:906:4
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: process didn't exit successfully: `target\debug\vault-unseal.exe` (exit code: 101)

It would be useful to have the possibility to specify data ranges or callbacks to validate data read from a configuration file and returning a dedicated error if no match is found.

Is this something you'd be willing to incorporate?

Hello

Would it be possible to add a support for ini files? I know they are not as expressive as the other formats, but there may be reasons to want to work with them (like some other tools that produce/manage them and don't know anything better).

I was able to create a hacky solution for myself ‒ using the rust-ini crate, fill another Config instance and merge that as a source (and using keys with dots in them to refer to deeper levels), but a built-in would be nice. Though I don't know what to do with arrays properly.

Is there a way to merge two arrays by appending them instead of overwriting one with the other? Or how are sets that should be merged usually represented?

By the way, I had to try it out, this behaviour wasn't clear from the documentation.

@nugend definitely possible and something I'd probably use.

[dependencies]
config = { version = "0.3.1" features = ["panic"] } 

Would it be possible to expose ValueKind from value.rs to allow some introspection?

ie, change: pub use value::Value; in lib.rs to
pub use value::{Value, ValueKind};

Config::sources could be an ordered collection and the config sources themselves could have a weighting:

config::merge(config::Environment::with_prefix("rust")::with_weight(10));

The weight itself would be an arbitrary value, of course (and presumably the ordered collection would preserve insertion order within weighting).

If I source a config file where the user has a typo in a key, I want to be able to warn or error rather than silently ignore that entry.

Possibly relevant - this is the approach that Cargo will be using to warn about unused entries in Cargo.toml https://github.com/dtolnay/serde-ignored.

Hi, i have issue with using together serde_yaml and config. When I compile an application, I get an error

 0 => yaml::Yaml::Hash(BTreeMap::new()),
   |                               ^^^^^^^^^^^^^^^ expected struct `linked_hash_map::LinkedHashMap`, found struct `std::collections::BTreeMap`

The reason is that the serde_yaml uses the feature "linked-hash-map" of yaml-rust. This feature indicates that type yaml::Hash is equivalent to ::linked_hash_map::LinkedHashMap<Yaml, Yaml>;
https://github.com/chyh1990/yaml-rust/blob/0.3.5/src/yaml.rs#L65

You can reproduce this issue. Just add to Cargo.toml in another project this two crates. And try to compile.

Something is going wrong. I shouldn't be able to crash my program like this.

extern crate config;

fn main() {
    config::set_default("k", "----").unwrap();
    let v = config::get("k");
    config::set_default("k", "").unwrap();
    println!("{:?}", v);
}

Sometimes it reads corrupted memory:

Some(String("в©è"))

Sometimes it crashes:

thread 'main' panicked at 'byte index 5 is out of bounds of `вðO`'

thread 'main' panicked at 'byte index 3 is not a char boundary; it is inside '5' (bytes 2..3) of `Т5®`'

Allow to control upwards recursion when searching for configuration files. The only real question is if the upwards recursion should be default true or false.

// ./config/default.toml OR ../config/default.toml 
// OR ../../config/default.toml (etc., whatever ends up matching)
File::with_name("config/default.toml").recursive(true)

A Config instance should allow locking to prevent all future mutation. Sources would need to support some kind of roll-up.

config::freeze();
config::set("key", "value"); // does nothing and returns an error

I want to handle multiple configurations with the same fields.
I have the default settings and N configuration files. I want to merge the default settings with each file separately and as a result, get N structures with the settings.
How to reset values or parse them using only one source?

let mut config = Config::new();
config
    .set_default("host", "localhost")?
    .set_default("port", 21)?
    .set_default("user_id", 0)?;
config.merge(File::with_name("foo"))?;
let host_foo: Host = config.try_into()?;
config.reset(); // <-- reset to default
config.merge(File::with_name("bar"))?; // merge only with default values
let host_bar: Host = config.try_into()?;

New to Rust, so hopefully I'm not just making a fool out of myself.

Signature of get_array uses self rather than &self, causing it to get consumed when getting an array this way:

Minimal example, with Settings.toml:

arr = [12, 2, 5, 6]

extern crate config;

fn main() {
    let mut c = config::Config::new();

    // Read configuration from "Settings.toml"
    c.merge(config::File::new("Settings", config::FileFormat::Toml)).unwrap();

    println!("debug  = {:?}", c.get_array("arr"));
    println!("debug  = {:?}", c.get_array("arr"));
}

Upon building, results in:

λ cargo build
   Compiling config-bug v0.1.0 (file:///G:/Programmering/testing/config-bug)
error[E0382]: use of moved value: `c`
  --> src\main.rs:10:31
   |
9  |     println!("debug  = {:?}", c.get_array("arr"));
   |                               - value moved here
10 |     println!("debug  = {:?}", c.get_array("arr"));
   |                               ^ value used here after move
   |
   = note: move occurs because `c` has type `config::Config`, which does not implement the `Copy` trait

When using File, config-rs traverse the directory hierarchy looking for a match. This is potentially dangerous and could allow third parties to exploit this behavior to load settings for privileged processes using config-rs to load config.

Lets say we have a program, exploitable, that normally searches for a file conf.json in /a/b/c/d

Now lets say the admin of the box is running exploitable with config specified by env vars. So internally exploitable first tries to pull config from a file, then tries to pull it from env vars.

Therfor if any directory in the chain a/b/c/d was writable by an attacker, they could drop their own config file in there.

When exploitable is restarted, then some of its config will come from that file.

I know that good practice is to put config files in directories that are locked down, but the behavior of File can get around it.

File needs a parameter or setting that prevents directory crawling.

1. c.merge(/* config.yml */)
2. c.get("k")
3. modify config.yml -> k
4. c.get("k")

The second get should reflect the changes to the config file.

I think we get this for free with the DeserializeSeed approach proposed in #13.