The following two patterns keep occurring in my design.

First is the monitoring or reporter functionality which is not strictly part of the system but needs access to the whole system in order to report its state. The way I imagine it currently is to have a nested system, the outer layer contains two components - monitoring and original system as a component. Or, alternatively a special keyword :integrant/system which would indicate that a components accepts the whole system as input (recursive dependency in a sense).

Second pattern is when an application consists of almost identical sub-systems which are loosely connected through some statistics channels. This again calls for some notion of nested systems.

I wonder what are your thoughts on this and if you intend to add some explicit provisions for nested systems in the future. Thanks!

Would you consider an optional initialisation order, or expose the comparator that sorts system initialisation?

I find myself occasionally enforcing an order between unrelated subsystems using a key that is clearly not intended as a functional dependency, e.g.

:license   {}
:server    {:db/connection #ig/ref :jdbc/connection
            :_ #ig/ref :license}

In the example above :license and :server are unrelated, but I require :license to evaluate first.

If there are composite keys among the matching keys, the sort call raises an exception.
Maybe better not to sort the matching keys.

I couldn't for the life of me figure out what I'd broken, but eventually as I reduced my config down to almost nothing, I was able to find out where it's breaking. This is kind of a funny one :)

The following code will fail due to NPE.

(do
    (defmethod ig/init-key :a/a1 [_ _])
    (defmethod ig/init-key :a/a2 [_ _])
    (defmethod ig/init-key :a/a3 [_ -])
    (defmethod ig/init-key :a/a4 [_ -])
    (defmethod ig/init-key :a/a5 [_ -])
    (defmethod ig/init-key :a/a6 [_ -])
    (defmethod ig/init-key :a/a7 [_ -])
    (defmethod ig/init-key :a/a8 [_ -])
    (defmethod ig/init-key :a/a9 [_ -])
    (defmethod ig/init-key :a/a10 [_ _])

    (ig/init {:a/a1  {}
              :a/a2  {:_ (ig/ref :a/a1)}
              :a/a3  {}
              :a/a4  {}
              :a/a5  {}
              :a/a6  {}
              :a/a7  {}
              :a/a8  {}
              :a/a9  {}
              :a/a10 {}
              }
             ))

Comment out a single key in the map, and it will work. Or, remove the ig/ref and it will work. It's something to do with resolving references.

Currently, the configuration name and type are coupled together. I believe it would be much simpler to dispatch the configuration based on specific key in the configuration map itself. Both defining and referencing configuration would be much simpler.

Currently:

{[:adapter.sql/hikari-cp :db/app] {:adapter "postgresql"
                                   :username "app"
                                   :password "app"
                                   :server-name "localhost"
                                   :port-number 5432
                                   :database-name "app"}

 [:adapter.sql/hikari-cp :db/mono] {:adapter "postgresql"
                                    :username "mono"
                                    :password "mono"
                                    :server-name "localhost"
                                    :port-number 5432
                                    :database-name "mono"}

 [:adapter.migrations.sql/flyway :flyway/app] {:schemas ["app"]
                                               :locations ["/db/migration/app"]
                                               :migrate? true
                                               :db #ig/ref [:adapter.sql/hikari-cp :db/app]}

 [:adapter.migrations.sql/flyway :flyway/mono] {:schemas ["mono"]
                                                :locations ["/db/migration/mono"]
                                                :migrate? true
                                                :db #ig/ref [:adapter.sql/hikari-cp :db/mono]}}

, this could be rewritten as:

{:db/app {:ig/type :adapter.sql/hikari-cp
          :adapter "postgresql"
          :username "app"
          :password "app"
          :server-name "localhost"
          :port-number 5432
          :database-name "app"}

 :db/mono {:ig/type :adapter.sql/hikari-cp
           :adapter "postgresql"
           :username "mono"
           :password "mono"
           :server-name "localhost"
           :port-number 5432
           :database-name "mono"}

 :flyway/app {:ig/type :adapter.migrations.sql/flyway
              :schemas ["app"]
              :locations ["/db/migration/app"]
              :migrate? true
              :db #ig/ref :db/app}

 :flyway/mono {:ig/type :adapter.migrations.sql/flyway
               :schemas ["mono"]
               :locations ["/db/migration/mono"]
               :migrate? true
               :db #ig/ref :db/mono}}

All ig namespaced keys would be reserved for integrant internals and stripped away from the calls to init-key. There could also be :ig/init, :ig/halt, :ig/schema etc keys with symbols to functions or inlined sci code.

With this change, a top-level key could also have just data as values, without any need to be started or stopped. Would make handling options, feature flags etc. simpler (currently, one needs to create an custom init for all keys).

This would be ok, just data to be referenced with aero ref for example:

{:app/config {:http-port 3000
              :use-dummy-login false}}

Integrant2?

I've just tried to put a spec on a datastructure that contains a system map, and thought I'd use the function ig/valid-config-key?, however it appears it doesn't test for the keyword also being a namespaced keyword.

Might it be worth tightening the test here?

The choice to use multi-methods has an annoying side-effect: there's no way to inform editors/codox "this keyword corresponds (among other things) to an extension of integrant.core/init-key". I'm opening this issue to discuss ways in which integrant could (or shouldn't) integrate into tools like codox (and cljdoc) and even orchard/cider & family. I do this with the knowledge that you're also the maintainer of codox, so overlapping thoughts can be brought in.

I experimented with some ad-hoc methods of attaching documentation to multi-method implementations, but they unfortunately failed. My approach was attempting to attach metadata to the functions that go into the multimethod registry itself, and then reading the metadata back out later.

The major feature I'd want is for Codox to be able to detect which libraries are adding the multi-method extensions and show a line-item in the HTML. I think it's worth having an eye to making it reasonably easy for editor tooling to extend integration to whatever we come up with also, but I think the use-case is more constrained there.

This may be by design, or I may be misunderstanding something, but this behavior seemed pretty counterintuitive, so I figured I'd bring it up.

We have a single "root" key that expands to cover all needed keys, so we start the system like this

(ig/init config [:com.nextjournal/nextjournal])

When doing suspend/resume we do this

(ig/suspend! system)
(ig/resume new-config system [:com.nextjournal/nextjournal])

This however calls halt-key! for (remove #{:com.nextjournal/nextjournal} (keys system)), in other words it halts everything but the root, before resuming, so suspend/resume effectively becomes halt!/init, except for this one key.

We can't do this

(ig/resume new-config system)

because there are keys in there that should not be started.

This could be solved by having suspend! call dependent-keys before computing missing-keys. We will probably do this ourselves as a workaround, even though dependent-keys is private.

(ig/resume new-config system (#'ig/dependent-keys new-config [:com.nextjournal/nextjournal]))

Do you agree that this is an issue? Would you accept a patch for this?

Naming of this config option is a bit misleading, it's called :include-refsets? but what it actually does is :include-reflikes?, and it's misleading particularly post #68

Hi,

What is the idiomatic way to manage faults and restarts from within the system? Let's say I have a web socket and a couple of components that depend on that socket:

(def config
  {:feed/db {:name :blabla
             :foo (atom nil)
             :bar (atom nil)}}
  :feed/ws {:url "wss:/ws-feed.example.com"}
  :periodic/ping {:feed (ig/ref :feed/ws)
                  :period 1000
                  :db (ig/ref :feed/db)}
  :periodic/balances {:feed (ig/ref :feed/ws)
                      :period 60000
                      :db (ig/ref :feed/db)})

When web socket :feed/ws breaks for external reasons I would like to automatically restart it and all its dependencies. Thanks.

Hi, are there API docs hosted anywhere? I couldn't see anything in the README.

Is there a reason not to do so ? It would be useful for example for pre-processing dates or durations from config files.

How can I make Integrant auto (reset) or "hot reload" server-side pages when I save my *.clj files? Currently using this with Duct.

Very similar to #25. Perhaps also touching on #21.

I'm trying to print out some metadata about the running system in dev mode, namely running http server(s). My approach to this has been to query the integrant-repl.state variables system and config. However, I realized that (obviously) the config hasn't resolved ig/ref at all.

#21 would suggest that one solution is to have a printing-component, which has an #ig/ref to all components in the system (perhaps filled by a function which performs a transformation).

I've found the strategy employed by your load-namespaces function to be generically useful for similar architectures that rely on multimethods and namespaced keywords.

I'm currently pulling integrant into a project that doesn't do any state management, just to get the functions called by load-namespaces. I had to make a more generic copypasta version for my own use:

(defn load-namespaces [kws]
  (doall (->> kws
              (filter keyword?)
              (mapcat #(conj (ancestors %) %))
              (mapcat #'ig/key->namespaces)
              (distinct)
              (keep #'ig/try-require))))

This works perfectly for me. What do you think about spinning this stuff off into a separate lib (or maybe adding it to medley?)

I had some troubles using it with Clojure 1.8.0, is it right that it's not compatible?

It might be useful to have an official way for sharing init and halt! behavior among keys.
One option is to use derive and the hierarchy system for multimethods.
However my impression is that this feature is only rarely used in general and that it might not be the right way to solve this particular issue.
In general it seems useful to be able to specify the init and halt! behavior for a key as part of the config as opposed to somewhere in the code. (I think Duct is doing something similar.)
For example, if the value for a key is

{:methods 
 {:init foo.component/init
  :halt! foo.component/halt!}
 :options
 {,,,}}

the init behavior could be to apply foo.component/init to the (expanded) options, and add foo.component/halt! as meta-data (similar to the current ::build meta-data).
The halt! behavior would be to apply foo.component/halt!.
With this implementation, it would be easy to share init and halt! behavior (one could even use Refs).
This implementation would also support a more concise form

{:methods foo.component/methods
 :options {,,,}}

where the value of foo.component/methods has the appropriate shape.

Could it make sense to integrate this kind of implementation with the current code base?
One option is to change the :default method for init-key such that it checks if the value for the key has this form (and maybe also if it has some metadata tag like ^:custom-methods or so).
A much more drastic change would be to just replace the current multimethods with this kind of implementation. The main downside I can see here is that it is not clear how to achieve the same level of conciseness as with multimethods. However, a potential advantage is that it seems to solve #8 and maybe also #6.

Not all the lifecycle functions end with an exclamation mark:

init
halt!
resume
suspend!
init-key
halt-key!
etc.

I'd expect all the lifecycle functions to have ! (because they are all equally side-effectful/unsafe) or none at all (because I know I deal with stateful stuff when ig/ functions are invoked).

In the examples ring.jetty.adapter is required instead of ring.adapter.jetty

Using 0.7.0 in the REPL,

test.core> (ig/ref :help)
#integrant.core.Ref{:key :help}
test.core> #ig/ref :help
Syntax error reading source at (REPL:116:1).
No reader function for tag ig/ref

This is my first attempt to use refs so I don't have too much more to offer. The readme examples didn't work so I poked at it this far and filed.

Might it be worth adding support for refsets to composite keys?

RefMap would be like RefSet but would resolve to a map {k -> ref} instead a refs set. Knowing the key can be useful (think handler registration).
Alternatively, extend the RefLike protocol with a resolve method and in expand-key postwalk the config via it, this would allow custom RefThings. Sample:

(defprotocol RefLike
  (ref-key [r] "Return the key of the reference.")
  (resolve [r config]))
;; ...
(defn- expand-key [config value]
  (walk/postwalk
   #(cond
      (reflike? %) (resolve % config)
      :else %)
   value))

In the current workflow, integrant initializes keys one by one, checking each time if the provided data is valid according to pre-init-spec. When the data is not valid, integrant throws and it actually gets messy. For instance, the resources allocated by the previous keys are not released. Sometimes, this means we have to restart the repl.

What I suggest is that integrant validates all data before initializing anything. The specs we write for our system means that if the data is not valid, something will probably fail. It is best to fail fast instead of starting the system, opening ports and stuff', and eventually failing anyway and leaving a big mess.

;; test case

(defmethod ig/init-key :test [_ {:keys [seq]}]
  seq)

(def system
  (ig/init {:test {:seq (range)}}))

I'm not sure if this is to be expected or not but even if it is intended there should be some sort of guard IMHO.

My assumption was that load-namespaces should be passed the same configuration map that gets handed to init, but I noticed that it doesn't handle the case when composite keys are used.

I can submit a PR to handle this case, just thought I'd check the expected behavior before doing so.

Hi there,

I am trying to rely on derived keywords to be able to start the system with or without db connection pooling. The configuration I have implemented is the following:

(def system
{[:system.db/db :system.db/pool] {:jdbc-url (:jdbc-url config)}
 [:system.db/db :system.db/conn] {:jdbc-url (:jdbc-url config)}
 :system.service/service {::http/port (:http-port config) :db (integrant/ref :service.db/db)}})

I also have the respective multimethod implementations for :system.db/pool and :system.db/conn.

However, when I try to start the system with just a set of keys I get an Ambigous key exception:

(integrant/init system [service.db/conn system.service/service])

Shouldn't that work? It seems to me that should work as I am actually specifying what implementation to use. Just a thought.

Thank you

If a logo for the Integrant project would be of interest, are any of the designs here meaningful to the maintainers of this project.

https://github.com/practicalli/graphic-design/blob/master/integrant-logo-designs.svg

Adjustments to the designs can be made if any of them are of interest.

Logos are created with the Open Source Inkscape software. If any of these designs are interested they would be donated to the project under a creative commons license, along with original SVG assets.

I was surprised to not get an error on undefined init-key/halt-key! methods for the particular key. Looking at code, this seems to be an intended behaviour: to silently treat keys without methods as plain maps which don't need initialisation/halting logic. What could be the use-case for such feature? I expect most, if not all, keys in the config/system correspond to stateful objects which implement start/stop behaviour.

1. I'd like to suggest removing :default cases from multimethods to force the user to explicitly specify all the needed methods. This is a less surprising behaviour and helps to early on detect cases when namespace with defined methods for the key was mistakenly not required.

2. In case the "plain map" pattern must also be supported library could implement the ig/consthelper, e.g.:

(def config
  {:adapter/jetty {:port 8080, :handler (ig/ref :handler/greet)}
   :handler/greet {:name "Alice"}}

   ; just a bag of stuff that will be added into system as is
   :bag-of-dependencies (ig/const {:foo 123 :server (ig/ref :adapter/jetty) :handler (ig/ref :handler/greet)}))

Hi @weavejester,

As discussed on Slack, we talked about how integrant's halt-key! method could have access to the original init-key arguments. It was the following example that spurred the discussion:

(defmethod ig/init-key ::datasource
  [_ {:keys [logger]}]
  (log logger :info ::starting.datasource)
  (hikari/make-datasource ...))

(defmethod ig/halt-key! ::datasource
  [_ datasource]
  (log ??? :info ::closing.datasource)
  (.close datasource))

How to access the logger at the ??? in halt-key? A solution would be to have the following:

(defmethod ig/init-key ::datasource
  [_ {:keys [logger]}]
  (log logger :info ::starting.datasource)
  {:datasource (hikari/make-datasource ...)
   :logger logger})

(defmethod ig/halt-key! ::datasource
  [_ {:keys [datasource logger]}]
  (log logger :info ::closing.datasource)
  (.close datasource))

However, in my opinion this exposes an implementation detail (::datasource wanting to access the logger, so the actual datasource is inside a map) to the "consumers" of this component.

We discussed three possible solutions.

1. Adapting halt-key's arguments to something like [_ pre-init-value post-init-value]. Upside is, that this is fairly straightforward. Downside is, it's not clear how to do this without breaking backward compatibility. The example however would be changed to the following:

(defmethod ig/halt-key! ::datasource
  [_ {:keys [logger]} datasource]
  (log logger :info ::closing.datasource)
  (.close datasource))

2. Adding an Unref protocol having an unref function, together with some helper functions. Integrant will call unref if init-key's return value satisfies the Unref protocol, before passing it to other components. For example:

(defrecord Datasource [datasource logger]
  Unref
  (unref [_] datasource))

(defmethod ig/init-key ::datasource
  [_ {:keys [logger]}]
  (log logger :info ::starting.datasource)
  (map->Datasource
    {:datasource (hikari/make-datasource ...)
     :logger logger}))

Or, with some helper function:

(defmethod ig/init-key ::datasource
  [_ {:keys [logger]}]
  (log logger :info ::starting.datasource)
  (ig/expose :datasource
    {:datasource (hikari/make-datasource ...)
     :logger logger}))

Benefits of this approach are backward compatibility and the developer can keep even more values than just the pre-init value.

3. A workaround for now without changing integrant, by using the ::build metadata on a system.

As I also stated on Slack, I am just getting started with Integrant, so we agreed we would both ponder if this is an actual problem, and if so, what the best approach would be to solve it.

Hi @weavejester
Looking forward to learning this microframework and duct but I need (and hopefully others too) as much documentation as possible on understanding it.

Do you have a new location to the Enter Integrant presentation ? or can you make it part of this repo ?

thanks
James (yes.. another one)

I am using integrant with eftest, and starting systems in my fixtures. Because load-namespaces is not thread safe, this causes sporadic namespace errors with eftest.

I can sync on my side, but I thought it might be worth doing some kind of sync in integrant.

When integrant fails to initialise, it throws an exception with the whole config as part of the ExceptionInfo. So when I run an app that looks like this:

(defmethod init-key :failing-thing
  [_ config]
  (throw (Exception. "Kerpow!")))

(defn -main
  []
  (init {:failing-thing {:secret "foo"}}))

I get output that includes this line:

Exception in thread "main" clojure.lang.ExceptionInfo: Error on key :failing-thing when building system {:reason :integrant.core/build-threw-exception, :system {}, :function #object[clojure.lang.MultiFn 0x1c33c17b "clojure.lang.MultiFn@1c33c17b"], :key :failing-thing, :value {:secret "foo"}}, compiling:(C:\Users\david.conlin\AppData\Local\Temp\form-init3479267167971658745.clj:1:115)

This isn't ideal, since we'd rather not be logging secret config values in plaintext. Is there some way we can exclude or obfuscate certain values in any logged/exception values of the config? Alternatively, is there a "production" setting that prevents the config being returned on an exception? If not, would these features be useful (happy to help out with a PR), or have we just gone about this completely the wrong way?

It appears that #ig/refset references are not initialized when this is the only place the keys are referenced, and you're loading only a subset of the system.

This makes sense: integrant lazily initialized derived keywords, and when the refset is first encountered, there are no derived keywords yet.

Are there any known workarounds for this?

For example I have following system:

(def config
  {::logger         {}
   ::sms-sender     {}
   ::component-1    {:sms-sender (ig/ref ::sms-sender)}
   ;; ...
   ::component-1000 {:sms-sender (ig/ref ::sms-sender)}})

(defmethod ig/init-key ::logger [_ _]
  #(prn :log %))

(defmethod ig/init-key ::sms-sender [_ _]
  #(prn :sms %))

(defmethod ig/init-key ::component-1 [_ {:keys [sms-sender]}]
  (fn []
    (sms-sender "Hi! I'm 1")))

;; ...

(defmethod ig/init-key ::component-1000 [_ {:keys [sms-sender]}]
  (fn []
    (sms-sender "Hi! I'm 1000")))

And I want to add logged-sms-sendercomponent that logs every sms. I can replace (ig/ref ::sms-sender) with (ig/ref ::logged-sms-sender) 1000 times but I don't want to do this.

Can I decorate a component without changing reference to it?

(First of all, I'm loving Integrant! It's changed the way I look at application architecture fundamentally. Thank you for all of your hard work!)

My use case is that I test the initialization and halting of my Integrant keys using config containing Midje metaconstants.

The below:

(require '[integrant.core :as ig]
         '[midje.sweet :refer :all])

(defmethod ig/init-key :some/component
  [_ config]
  (println config))

(fact
  (ig/init {:some/component {:config-key ..value..}})

  => {:some/component {:config-key ..value..}})

Produces:

{:config-key {}}

FAIL at (*cider-repl some-namespace*:479)
    Expected:
{:some/component {:config-key ..value..}}
      Actual:
{:some/component nil}
       Diffs: in [:some/component] expected {:config-key ..value..}, was nil

Consider this scenario:

There is a system described as

(def system-map
 {:adapter/jetty {:port 8080, :handler (ig/ref :handler/greet)}
   :data/store {:jdbc-url "some-yrl}
   :handler/greet {:name "Alice",  :data-store (ig/ref :data/store)}})

For development and production env I want :data/store to be some sql db - with init-key :sql/store.
But for test env I want it to be some mock (which just stores data in atom, for instance) -
with init-key :inmemory/store
(And obviously I don't want to use different system-maps).

It seems natural that I can specify that both keys (:sql/store and :inmemory/store) derived from the :data/store - but this leads to the exception because clojure cant decide which key to use for init-key multimethod.

As a solution, I propose to extend init fn so it accepts local hierarchy, where one can specify how the particular key should be initialised.

For instance, with mentioned system-map it may look like this:

(ig/init system-map (keys system-map) (-> (make-hierarchy) (derive :inmemory/store :data/store))

And during key building integrant just get all descendants of the :data/store and if there is no ambiguity - the descendant will be used.

what do you think?

UPD:

I've started the PR #80.

But I see conflicts with how composite keys now defined (they are also build with derive). Please advise how can I proceed.

Your feedback is much appreciated!

Thanks!

Sometimes some state is needed in components for efficiency, being able to react to external events, or for being able to properly shut down later. But even though the state is needed, it doesn't necessarily make sense to expose this state to other components. Exposing the additional state would only open opportunities for mistakes, and/or make the component clunkier to use.

Example

Let's say I create a component that on start, creates a db object, and and some object that will periodically mutate it. For the private state, I would then hold on to a mutable db object, and a timer-type object (in order to be able to use it in ":halt").

But it would 0 sense to expose these things to other components. The only thing they should care about is an immutable view of the current db.

I talked to a few people in Clojurians, and they pointed out that one can solve the problem by convention or other tricks, and that makes sense. In some cases, it can also be solved by using "higher-order-components" , e.g., duct.scheduler.simple. Thought I'd share this idea anyway, in case, in case you think it makes sense:

;; In integrant, new record
(defrecord State [private-state public-state-fn])

;; Implementation changes...

In client code:

(defmethod ig/init-key ::service [_ _]
  (ig/map->State
    :private-state {:exec-service  (create-exec-service) :datascript-conn (create-conn) } ;; Constructors omitted
    :public-state-fn
    (fn [private-state]
      ;; Returns an immutable view in this case. 
      ;; A function that returns this (requiring 0 arguments) is what would get exposed to other components.
      @(:datascript-conn private-state)))

  (defmethod ig/halt-key! ::service [_ private-state]
    (.stop (:exec-service private-state))
    private-state)

In the init-key, if people don't return this new State record, it would just work like it does now. (That is, private state is always the same as public state, and doesn't require an invocation to access).

I believe we can introduce cljs feature parity with some of the fns that are currently clj-only (basically read-string & load-namespaces) by means of macros, at least to some extent. Example:

foo (a clojurescript project)
├── resources
│   └── foo
│       └── config.edn
└── src
    └── foo
        ├── core.clj
        └── core.cljs

;;; resources/foo/config.edn
{:foo/bar "42"
 :foo/baz #ig/ref :foo/bar}

;;; src/foo/core.clj
(defmacro read-config [path]
  (-> path io/resource slurp ig/read-string))

;;; src/foo/core.cljs
(def config (read-config "foo/config.edn"))
;; {:foo/bar "42"
;;  :foo/baz #integrant.core.Ref{:key :foo/bar}}

If this seems desirable I can make a more concrete proposal and PR.

Hello James and dear contributors,

I'd like to show my support to this and other important work that you did/do in a form of monthly donations. I'm naively hoping that this would allow you to devote more time to them. If you accept such things – maybe add a Patreon link or do a github own thing?

And thanks for some of the awesomest stuff out there!

Kind regards,
Ivan

Currently, when an exception is thrown during build, there is no way to access the partially built system.
It would be useful to have access to it for example in order to halt the keys that have already been built before the exception was thrown.
Could we add an exception handler argument to the build function or have it rethrow the exception with the partially built system attached. (I think component is doing the latter.)

For example, component A depends on B and C but both B and C are independent and can take quite some time to initialize. Could Integrant start B and C concurrently instead of serially?

I appreciate the succinct nature of the namespace free reader tags e.g. #ref but ideally these tags would be namespaced #juxt.aero/ref or perhaps just #aero/ref. clojure.org has this to say:

Reader tags without namespace qualifiers are reserved for Clojure.

And the edn-format spec re-echoes the same sentiment:

Tag symbols without a prefix are reserved by edn for built-ins defined using the tag system.

There may already be a collision here too, as aero also defines #ref. How would one use both aero and integrant together?

It would be great to have a section about the reloaded REPL workflow in the README because it's one of the main reasons one would want to use Component-like pattern.

In particular, it would be interesting to see if there's a reliable way of reloading multimethods. I've stumbled upon a problem of old definition of method still working after it was removed from ns and reset-ing. To solve this particular problem I've ended with this in a sample project:

(ns user
  "Namespace to support hacking at the REPL.

  Usage:
  o (reset) - [re]run the app.
  o In case REPL fails after syntax error, call (refresh) and try again.
  "
  (:require [my-app.core :as core]
            [integrant.core :as ig]
            [clojure.tools.namespace.repl :refer [refresh refresh-all]]
            [clojure.repl :refer :all]
            [clojure.pprint :refer :all]))

(println "Hi dev!")

(def system (atom nil))

(defn -start
  []
  (reset! system (ig/init core/config)))

(defn -stop
  []
  (swap! system (fn [s] (when s (ig/halt! s)))))

(defn reset
  []
  (-stop)

  ; this will force reloading of the methods
  (remove-all-methods ig/init-key)
  (remove-all-methods ig/halt-key!)
  ; etc.

  ; refresh-all is used instead of refresh because we've just removed all the methods and want to find all the definitions
  (refresh-all :after 'user/-start))

But I'm not sure if refresh-all is going to be fast enough in bigger projects.

Problem

Currently I have a setup with an HTTP server that returns EDN. I need to be able to read this from my ClojureScript application. However ig/read-string is currently only available for Clojure. Possibly for the lack of a built-in EDN library?

Solution

We could add tools.reader as a dependency to integrant. That way we can import cljs.tools.reader.edn specifically for ClojureScript. This would really help me since I'm now copy pasting integrant's read-string function in my ClojureScript application.

What do you think of this idea? Do you possibly have a reason to not support ClojureScript?
If you agree to this change then I'd be willing to create a pull request, to save you the hassle.

Some build tools (tools.deps) need a pom.xml manifest to work from a Git URL. By ignoring it, people using tools.deps are unable to run from an unpublished branch.

I'm refactoring my app to use integrant instead of mount. So far it's awesome, exactly what I needed. My app starts an Apache Ignite cluster and associated services, so there is a lot of state flying around, and this is by far the most elegant way of handling it that I've discovered.

One problem I have is that if any init-key methods throw an exception, it causes the entire value returned by init to be nil. This makes it hard to gracefully shut down. I actually have two integrant systems starting at different times in my app. If something fails, I need to get a handle on the existing services to tear them down, otherwise my app can hang on shutdown and cause problems in the cluster.

I propose integrant capture the exception thrown by init-key (which it must already be doing) and substitute that for the value that would have otherwise been set for the key. halt! would also need to be modified to skip such keys.

I realize this muddles the ability to detect a failed init by testing for nil, so perhaps this could be a third arity for init that lets the user explicitly opt in for this behavior if they require it. To simplify the detection of startup failures, perhaps the map init returns in this mode could include a key like {:integrant/status :integrant/failed}. That would be your clue to find and log the exceptions, start a shutdown, or even change something and retry.

Happy to hack on this for you, but wanted to get your thoughts before cutting a PR.

My expectation is that a value under a composite key should somehow conform to the specs returned by pre-init-spec for its component(s).

Using your example:

{[:group/a :adapter/jetty] {:port 8080, :handler #ig/ref [:group/a :handler/greet]}
 [:group/a :handler/greet] {:name "Alice"}
 [:group/b :adapter/jetty] {:port 8081, :handler #ig/ref [:group/b :handler/greet]}
 [:group/b :handler/greet] {:name "Bob"}}

I'd want to be able to define a pre-init-spec method for :handler/greet and :adapter/jetty:

(defmethod ig/pre-init-spec :adapter/jetty [_]
  (s/keys :req-un [::port ::handler]))

(defmethod ig/pre-init-spec :handler/greet [_]
  (s/keys :req-un [::name]))

But right now this will only work if I define four other specs, one per composite all returning spec(s) defined for the components.

(defmethod ig/pre-init-spec :adapter/jetty [_]
  (s/keys :req-un [::port ::handler]))

(defmethod ig/pre-init-spec :handler/greet [_]
  (s/keys :req-un [::name]))

(defmethod ig/pre-init-spec [:group/a :adapter/jetty] [_]
  (ig/pre-init-spec :adapter/jetty))

(defmethod ig/pre-init-spec [:group/b :adapter/jetty] [_]
  (ig/pre-init-spec :adapter/jetty))

(defmethod ig/pre-init-spec [:group/a :handler/greet] [_]
  (ig/pre-init-spec :handler/greet))

(defmethod ig/pre-init-spec [:group/b :handler/greet] [_]
  (ig/pre-init-spec :handler/greet))

That's a lot of code for what seems like an 80% use-case?

What this allows however is to fully customize the spec for any given composite which might be desirable sometimes I guess.

Maybe a middleground would be to use the spec returned by the composite key if there is one, and if there is none fallback to building an s/and of (map ig/pre-init-spec composite-key) or alternatively use the first spec found, but that leaves the question of which one should be used?

The jar submitted seems not align to the code tagged on github?

;; git clone and lein install locally
user=> (require '[integrant.core :as ig])
nil
user=> (doc integrant.core/load-namespaces)
-------------------------
integrant.core/load-namespaces
([config] [config keys]) ;; <====== correct signature
  Attempt to load the namespaces referenced by the keys in a configuration.
     If a key is namespaced, both the namespace and the namespace concatenated
     with the name will be tried. For example, if a key is :foo.bar/baz, then the
     function will attempt to load the namespaces foo.bar and foo.bar.baz. Upon
     completion, a list of all loaded namespaces will be returned.
nil
user=> exit
Bye for now!

from Clojars

Retrieving integrant/integrant/0.6.1/integrant-0.6.1.pom from clojars
Retrieving integrant/integrant/0.6.1/integrant-0.6.1.jar from clojars
user=> (require '[integrant.core :as ig])
nil
user=> (doc integrant.core/load-namespaces)
-------------------------
integrant.core/load-namespaces
([config])  ;; <====== incorrect signature
  Attempt to load the namespaces referenced by the keys in a configuration.
     If a key is namespaced, both the namespace and the namespace concatenated
     with the name will be tried. For example, if a key is :foo.bar/baz, then the
     function will attempt to load the namespaces foo.bar and foo.bar.baz. Upon
     completion, a list of all loaded namespaces will be returned.
nil

I like very much the fact we can provide keys to ig/init. When in dev, it is useful to be able to work only with a few keys at a time. Nonetheless, it looks like dependent keys (children) of initialized dependencies (parents) are always halted when calling ig/halt!, even if those dependent keys have not been initialized in the first place.

If ::b needs ::a and I only init ::a, ::b will be later halted as well.

Here is a simple gist showing that behavior.

Is it on purpose ? If so, I can't understand why as I find it very inconvenient for my use cases.

So this looks great, but I have a few clarity questions:

pulling in external environment vars

Is prep-key the right place to do this? Specifically for things like JDBC usernames/passwords etc.

separating prod and dev

In production Jetty isn't used as we deliver a WAR file, so at the moment we have a common.clj which bootstraps the "main" system and is called from either a prod.clj or a dev.clj, both of which add environment specific things (e.g. jetty in dev).

Is this pattern still relevant, and I can see how dev and prod namespaces would include the relevant multimethod implementations, but how do I implement "dev has jetty but prod doesn't"?

That's it for now - thanks!