datomic / simulant Goto Github PK

I can use h2 with jdbc(create database, insert ...)

But when I run test (et/-main) , the program hung up at (d/delete-database sim-uri)
I'm tested (d/create-database sim-uri), also, hung up.

The trading example is great for running from the REPL, but it has two problems as a didactic tool:

1. To a newcomer, the boundary between the sim and the system under test (SUT) is hard to discern.
2. It doesn't clearly communicate which steps of the lifecycle are repeatable.

This issue is a request for an example sim with the following characteristics:

1. It can be run from REPL or command line.
2. Each step can be executed individually.
3. The system under test runs in a different process than the simulation.

It would be helpful to have a lein template that creates a simulant project, including placeholders for the parts the project must contribute.

Why schema.edn from jar file is different than from git repository. For instance I don't have fields like :actionLog/failure-type and :actionLog/failure-trace.
I see those fields in repo but schema.edn from jar file doesn't have those fields. Do I missing something ?

simulant-0.1.6.jar as sent to clojars does not have the resources/ directory. It is therefore missing the Datomic schema needed to run.

It would be nice if there were an option for processes to wait for each other before starting the simulation. It's not a huge deal for long-running sims, but for shorter ones the coordination would be helpful.

The current log service writes to local files, which are loaded into Datomic at the end of the simulation run. This is one of the things that requires every simulation runner to be a Datomic peer.

It would be helpful to allow a substitute log service that writes through to S3. This requires the following changes:

1. Create the service and attach it to the sim.
2. Assign S3 object IDs to each runner at sim startup time.
3. Construct the service at sim-execution time. It must inject a function that accepts a data structure and serializes it to the S3 file for that runner.
4. Load S3 EDN files into Datomic at sim completion.

The recording function must be a drop-in replacement for the current recorder. It must be possible to switch this out without changing the action functions.

The call to await-all at
https://github.com/Datomic/simulant/blob/master/src/simulant/sim.clj#L378
waits for all the actions dispatched at the time of calling to have occurred. In my simulation there is some nondeterministic behavior in the environment (in reaction to nondeterministic behavior of the software under test). This I have modeled by sending actions back and forth using a send-action service (see below). This means however that new actions will be dispatched by the current actions being performed. As a consequence the services are finalized too early (e.g. the action log temp file gets closed).

I don't know enough about java concurrency code yet to have a fix, but other than the obvious race condition something like:

(defn await-all
"Given a collection of objects, calls await on the agent for each one"
[coll](let [agents %28map via-agent-for coll%29]
%28apply await agents%29
%28when-not %28every? %28fn [^clojure.lang.Agent a] %28zero? %28.getQueueCount a%29%29%29 agents%29
%28recur coll%29%29))

could work. If I find a real solution (other than the (Thread/sleep 6000) that keeps things from crashing now) I'll submit it. Very curious about other solutions for this.

(defmethod sim/start-service :service.type/send-action
[conn process service](fn [action process]
;; test action has required keys
%28assert %28every? %28partial contains? action%29 [:action/type :agent/_actions]%29%29
;; send the action
%28sim/feed-action action process%29))

;; This service has nothing to finalize
(defmethod sim/finalize-service :service.type/send-action
[conn process service services-map]
true)

(defn create-send-action-service
"Create a send-action service for the sim."
[conn sim](let [id %28d/tempid :sim%29]
%28-> @%28d/transact conn [{:db/id id
:sim/_services %28e sim%29
:service/type :service.type/send-action
:service/key :simulant.sim/send-action}]%29
%28tx-ent id%29%29))

Long action log data can be elided and replaced with an ellipsis. This fails to load into the database when the action log gets finalized.

For sims that run from the command-line, it would be helpful to have a starting point.

The command line interface should

• Execute each part of the simulation lifecycle independently.
• Allow customization or extension (i.e., with arguments for model parameters).
• Allow parts of the lifecycle to be rexecuted.

If you have multiple agents running in a sim, the temp file that ActionLog writes to can wind up with interleaved forms printed in it.

In line
https://github.com/Datomic/simulant/blob/master/src/simulant/sim.clj#L85
(doto (create-service ...) (start-service))

Since doto is used, the result of (create-service) is returned after (start-service) is called on it. This is not what is documented at
https://github.com/Datomic/simulant/blob/master/src/simulant/sim.clj#L69

where is states that the return value of start-service is assigned to the services map. I prefer the documented version over the implemented one, but clearly this could not be fixed while maintaining backward compatibility. So maybe just update the doc?

We have a project where we need to populate a Datomic database with a bunch of information before we start the simulation running. We're currently modeling this as a service that does the population during its start-service phase. However, because we only want the population to happen once, and not once for every process, we want to associate the service with the process entity, not with the simulation entity. While associating services with particular processes is supported by the code, the process entities are not created until the simulation starts to run. Which is inconvenient, since we'd like to associate the service with the process when we're creating the simulation.

When running from a REPL, it is common to run multiple sims in the same JVM.

If an exception reaches the agent, it is marked as failed. Once failed, there is no built-in procedure to reset it to a usable state or to create replacement agents.

Expected Result

Transacting schema.edn should prepare a database for models, agents, and sims.

Actual Result

Datomic rejects schema.edn due to the :http/method attribute. It is marked as value type :db.type/keyword and also marked as :db/isComponent true.

Older versions of Datomic would accept this even though there's no meaning to making a value attribute a component.

Newer versions of Datomic will reject this with this exception:

IllegalArgumentExceptionInfo :db.error/invalid-install-attribute Error: {:db/error :db.error/components-must-be-refs, :entity {:db/id 91, :db/ident :http/method, :db/valueType 21, :db/cardinality 35, :db/isComponent true, :db/doc "The method of the HTTP request."}}  datomic.error/deserialize-exception (error.clj:124)

Solution

:http/method should not be a component.

The default batch size of 1000 log entries in a transaction can result in too-large transactions, causing transactor failures. The action log definition should include an optional attribute that allows overriding this default.