grammarly / omniconf Goto Github PK

Hi, great configuration library. I want to propose a new variant of the population: properties population (e. g. java -Dsome.propertie="some value").
Example:
(populate-from-properties quit-on-error)

I have a config paramter that stores the HDFS location of a file. The parameter has HDFS IP, port and the file location, something like hdfs://172.21.1.11:54310/schema/schema.txt.
Since IPs and ports are always changing, I do not want to hard code their values. I tried setting the HDFS IP and port as environment variables and passing them to the value of config param in the configuration file, ex: hdfs://$HDFS_IP:$HDFS_PORT/schema/schema.txt, but it didn't work.

I know this can be achieved at the code level by reading the IP and port from the environment, and concatenating them with the file location, but is there a way to do it at the configuration level?

One thing that is not entirely clear in the documentation on SSM in the examples is the following:

Let's say I have an explicit mapping for an ssm-parameter, but I also want to allow that config value to be set in another manner. For example, by way of reading an edn file. What is the expected method of managing a local vs deploy config for a parameter like a db's password which will be fetched in production by way of SSM, but will be fetched via another method locally?

example:

(cfg/define
  {:database {:nested {:password {:type :string :required true :ssm-name "/foo/bar"}}}})

(cfg/populate-from-file "config.edn")
(cfg.ssm/populate-from-ssm "/")

Is the expected behavior here that populate-from-ssm will clobber the config pulled from the config.edn?

Hi,

The example in the README for set-logging-fn doesn't seem to work with the current version of timbre. As best as I can tell the closest match in the timbre README is https://github.com/ptaoussanis/timbre#log-levels-and-ns-filters

I have my configuration defined as

(cfg/define
  {:a {:nested {:b {:nested {:c {:type :string
                                 :required true}
                             :d {:type :string
                                 :required true
                                 :default "World"}}}}}
   :conf {:type :file
          :required true
          :verifier omniconf.core/verify-file-exists
          :description "Configuration file"}})

And I have a configuration file conf.edn with the content

{:a {:b {:c "Hello"}}}

Main function is

(defn -main
  "Omniconf Demo"
  [& args]
  (cfg/populate-from-cmd args)
  (when-let [conf (cfg/get :conf)]
    (cfg/populate-from-file conf false))
  (cfg/verify :quit-on-error true))

When I run lein run --conf conf.edn, I get the message ERROR: [:a :b :d] : Value must be provided. even though I have provided the default value in the configuration definition.

After some experimenting, I found that this error occurs for 2 level nesting but not for single level nesting, i.e. if my configuration definition is

(cfg/define
  {:a {:nested {:c {:type :string
                    :required true}
                :d {:type :string
                    :required true
                    :default "World"}}}
   :conf {:type :file
          :required true
          :verifier omniconf.core/verify-file-exists
          :description "MECBOT Auth configuration file"}})

And my file content is

{:a {:c "Hello"}}

And if I run lein run --conf conf.edn, I get the correct output

Omniconf configuration:
 {:a {:d "World", :c "Hello"},
 :conf #object[java.io.File 0x5d12a356 "conf.edn"]}

Could you please confirm whether the above error is on expected lines or not?

Hello,
I have a perhaps rare but not unheard of usecase where I need to read a configuration file with custom reader macros (like integrant's ig/ref). Therefore I've provided a simple patch which breaks the api somewhat (the depracted quit-on-error? arity) @ here.

Can you tell me if this goes against the philosophy of omniconf as I use the latter as a centralized configuration system-of-record which contains many other types of configurations, some of which make use of reader macros.

Thanks for any feedback,

Nicolas

I really like the idea behind omniconf, and have been wanting something like this that would make it possible to unify configuration from files, env variables, and command line arguments. The one catch is that for my application, I have subcommands/subsystems which run under a shared code base, and need to differentiate how options are processed depending on which subsystem is being activated, but still share most of the configuration. I'm a bit inspired here on the cli args side by the python argparse library, which is splendid. But I realize that this overall idea might be difficult to make compatible with the full functionality of omniconf. Do you see a way this could be made possible?

Thanks in advance

• Version: 0.2.5
• Type: Handling config params with default value as false
• Summary: Here are the steps to observe the behavior on REPL-

;; Define a single param conf1 of type boolean with a default value of false
user> (omniconf.core/define {:conf1 {:type :boolean :default false}})
user> (omniconf.core/verify)
Omniconf configuration:
 {}

nil

;; Now, set the default value as true
user> (omniconf.core/define {:conf1 {:type :boolean :default true}})
user> (omniconf.core/verify)
Omniconf configuration:
 {:conf1 true}

nil

;; Next, use a nested structure and see the behavior
user> (omniconf.core/define {:conf1 {:nested {:c1 {:type :number :default 0} :c2 {:type :string :default "c2"} :c3 {:type :boolean :default false}}}})
nil
user> (omniconf.core/verify)
Omniconf configuration:
 {:conf1 {:c1 0, :c2 "c2"}}

nil

;; Set default value to true
user> (omniconf.core/define {:conf1 {:nested {:c1 {:type :number :default 0} :c2 {:type :string :default "c2"} :c3 {:type :boolean :default true}}}})
nil
user> (omniconf.core/verify)
Omniconf configuration:
 {:conf1 {:c1 0, :c2 "c2", :c3 true}}

nil

• Expected: Default value of false must be returned if it is not set for the configuration param explicitly.

Clarification

On the similar lines, why is the check on this line- https://github.com/grammarly/omniconf/blob/master/test/omniconf/t_core.clj#L81 not listing the default value of :one?

;; Current
(is (= {:first "alpha", :more {:two "two"}, :second 70} (cfg/get :nested-option)))

;; Why not?
(is (= {:first "alpha", :more {:one "one" :two "two"}, :second 70} (cfg/get :nested-option)))

Question marks are not allowed in environment variables but are valid keywords in clojure. We should escape question mark to something (maybe _Q_) and use that as the variable name.

It is somewhat tricky to tell Omniconf where to look for a configuration file.

could you please elaborate on that? I have a .edn file in root directory of the project and it doesn't seem to be parsed.

Relevant code:

(defn ring-init []
  (let [local-config "dev-config.edn"]
    (if (.exists (io/as-file local-config))
      (cfg/populate-from-file local-config)
      (log/warn "Can't find local dev configuration file" local-config))
    (init)))

• Version: 0.2.5
• Type: Query/Clarification to understand if there is a better way to define multi-valued configs
• Summary: Need to define a validated configuration for parameters that may take nested params, like-
  • [{:host "192.168.4.40" :port 9092}]
  • [6379 6380]

Currently, I am defining a type as :edn. Is there a better way to define such multi-valued parameters where constraint can be applied on each value, like- map keys in the first case and integers in the second case.