roboy / ravestate Goto Github PK

Currently cleanup of conversations is only done in multiprocessing mode.
To avoid large dictionaries of active users and chats, this has to be done in the single process case as well.

See https://github.com/huggingface/pytorch-pretrained-BERT

Currently, parallel telegram conversations always share a context (mutiparty use-case is assumed) Alternately, TelegramIO should offer the possibility to fork ravestate for multiple convos, such that the conversations are separated.

This is going to require the introduction of an additional head-less IO module, e.g. process bridge.

Everybody puts 1+ example per (Predicate, Sentence Type, Subject Type) combination here please 😊 : https://docs.google.com/spreadsheets/d/18i_0Izgyz2pwxN_d-GDHrE0pN_pdpYvRVCAGgPEKn7o/edit?usp=sharing

• @josephbirkner
• @nbasargin
• @Toseban
• @NeginsCode
• @l-laura
• @emlozin
• @Aevyz
• @Waguramu
• @missxa
• @xtin

Implement client for server implemented in #92.

They changed the API, so we need to change accordingly.

Currently, RoboyQA errors when it's default nodes for Roboy (the dialog agent) do not exist. A more user-friendly approach would be to just create these nodes instead.

While ravestate development is turning from developing base functionality to dialog skills, it is becoming apparent that most skill states require emit_detached=True. This is, because for base functionality, most states are "refining" or "filtering" states that do not participate in a state machine across multiple dialog turns (e.g. nlp states).

Such "filter" states usually just read properties and write a processed result to another property. In this case, it is necessary, that the causing changed signals are parents of the resulting offspring changed signals (emit_detached=False).

The proposed change is to introduce a @filter(...) decorator, which retains emit_detached=False. However, the default value for normal skill states should be emit_detached=True.

Roboy has three major show moments which concern roboy_dialog/ravestate!

1. Roboy has to react to a Telegram message which summons his rickshaw to a predetermined pickup point.
2. Roboy has to react to a verbal "start driving" command.
3. Roboy has to react to a signal from autonomous driving upon which he says we have arrived.
4. Roboy has to recognize Lennart; offer him a handshake and conduct small talk.

Implementation:
We create a new ravestate module called ravestate_roboy_ad_demo. The module contains the following states:

• pickup_requested → reads nlp:triples, recognises trigger phrase, signals "go!" to AD
• arrived_at_pickup_point → processes respective signal from AD, voices ~"Hop on!"
• start_driving → reads nlp:triples, recognises trigger phrase, signals "go!" to AD
• arrived_at_drop_off_point → processes respective signal from AD, voices ~"We have arrived!"
• lennart_recognized → reacts to stalker:recognized_faces:changed` with high weight (superseding generic reaction. If lennart is the recognized person, voices custom greeting and triggers handshake to ReplayTrajectory which takes a string as a name.)

For the signals from AD, we have to create respective Ros2SubProperties or Pipes. Note: Bidirectional comms. needed (AD → Dialog | Dialog → AD)! To trigger the handshake, we have to create a respective Ros2CallProperty to trigger the pre-recorded motion.

Implement data model defined in #91 in Python as a Context subclass (Use current code as reference).

Currently, there's a sizable if-else negotiation going on in the RoboyQA state. This could be split up into multiple states, where each state reacts to a certain question word. Consequently, this would imply the introduction of fine-grained NLP signals for different question words.

After everybody has filled the examples sheet, we check the examples against the amazing new rule-based inference state.

The exception occurred while starting a new conversation with "Hey" with the telegram bot.
The bot did not answer anything, but after sending another message, everything worked fine.
The bug was not reproducible.

Create ravestate_roboy and ravestate_generic to wrap the respective config files.

Currently, the roboy_parlai generative model adds spaces before every punctuation character. This looks weird. It should be fixed.

Currently, a Signal will not complain if it's minimum age is larger than it's maximum age, even though this makes it unfulfillable. It should be able to detect this condition and log a warning in response.

EmitStep() is different from Emit(), in that the emitted signal will never be finally consumed. On the contrary, the only way to "turn off" a signal that was produced by EmitStep() is to return Wipe() in a subsequent state run. Logically, EmitStep() is always automatically non-agglomerating.

The semantics behind EmitStep() are, that some signals indicate a mode rather than an event: This can be evident from the verbs that are used to describe the signal: "Changed", "Pushed" etc. are momentous observations, while "interlocutor-is-x" describes a modal, lasting observation.

For active engagement, the engagement state is currently up to a race condition. This should be changed, such that two new parameters may be used to control activity of a state:

• weight: Will be applied to a state's specificity permanently.
• cooldown: Refractory time period after activation, during which a state's specificity rebuilds (from zero)

Currently, fillers are rather boring two-letter words like Uh and Oh. They should be changed to phrases like Let me think... or Give me a moment... or About that... etc.

Also, they are only executed once currently.

Currently, questions are only detected for W-questions. It should also detect other questions, such as:

• Do you know unicorns?
• Are you ..?
• Can you... ?
• May I...?

Currently, it is not possible to detach emitted signals in a state from the state's constraint's spikes causal groups, without simultaneously relaxing the need for the state to get consent from the detached constraint for consuming it's resources.

Therefore, in addition to s(..., detached=True), the state API should provide an additional detached_emit parameter for the @state decorator

Try to come up with an rule-based inference state implementation, that does...

• ... recognize predicates
• ... recognize subject (interlocutor vs roboy vs 3rdparty)
• ... recognize object answer and
  • ... write it to memory if it's a statement!
  • ... determine truth if it's a question!

Please put this state into a new ravestate_inference module. ravestate_inference could offer prop_predicate and prop_subject which would be written by the ravestate_persqa smalltalk state.

About

Gaybar is an algorithm for matching aging picky signal instances (Spikes) with picky State activations, modeled as a bipartite bidirectional graph G_Bar = {I, A, E}:

• I is a set of signal instances (spikes)
• A is a set of unfulfilled state activations
• E are edges, where E ⊑ {(i, a) | i in I, a in A}.

Algorithm

Eager state activations are sitting at the bar, waiting for matching Spikes. They have criteria wrt/ the Spikes they wish to run their state activation function with, including...

• ... the spike's signal's name,
• ... the amount of time the spike has spent in the bar (it's age),
• the quantity of instances.

Spikes entering the bar wish to activate as many attractive states as possible, for as many different context properties as possible. A state is considered more attractive, the more specific it's criteria are, promising a higher reduction of local entropy in the bar. The limiting constraint is given by the context properties that are written to by a state: A Spike may only activate one writing state for every context property. Here is the formula for calculating the exact attractiveness of a state, wrt/ each of it's possible disjunct constraint conjunctions:

As the spikes enter the bar, they approach interested activations. The activations take note of Spikes which match their criteria, and notify the Spikes about their desire. As soon as a state activation X has gathered a sufficient number of spikes, it proceeds to tell the lucky candidate spikes, which in turn tell competing activations about their impending non-availability. If no activation more attractive than X is interested in the spike, it will consent to the activation. Otherwise, it won't: X will need to wait until it is the most attractive remaining suitor. As such, responsibility to determine whether the future fulfillment of an activation's constraints is likely lies entirely on the activation, and it is necessary for an activation to reject a spike when it sees it's hopes for activation deminishing. This process is called auto-elimination.

Once all spikes have consented, X runs it's respective state's activation function. Henceforward, X's partaking Spikes will no longer associate themselves with state activations that would write to context properties such as the those they just saw (let's just say: Spikes are always looking for new experiences).

In the bar, time is of the essence though; asynchronously and impartially progressing at a certain number of ticks per second. With each tick, Spikes gain in age. As ticks pass for a Spike, new suitable candidate activations may become available for it, while previous activations may lose interest. A Spike finally leaves the bar, when no activations are interested in it anymore.

Spike Offspring and the Causal Group

As a state's activation function runs, it may trigger a number of offspring spikes (property:changed, state:fancy-signal, etc.). It is grave here, that whichever Spikes caused this activation be notified about these offspring spikes, such that the offspring and parents synchronize wrt/ to their written properties: Offspring will not write to properties that were written by any parent, and parents will not write to properties that were written by any child. Such a set of causally linked spikes is called a causal group.

E.g. as a consequence, a verbalizer:intent:changed spike indirectly caused by some rawio:in:changed is not going trigger a write to rawio:out, if the same rawio:in:changed already triggered a write to rawio:out through a different activation path:

TLDR: Causally connected spikes share a causal group, and as such have identical unwritten properties.

Currently, it is up to an activation (and the completed constraint's default max_age) to figure out, that a completed constraint's spike does not cause the activation of the desired follow-up spike.

This should be registered by CausalGroup: if an activation fails to produce the spikes it promised, and there are no other activations in the CausalGroup that could produce the same spike for the same cause, Activations which depend on the forgone spikes will be notified via Activation.effect_not_caused(effect, causal_group) where effect is a Signal.

Successively, the activation searches it's conjuncts for signals of type effect. If a match was found, the activation will reject all spikes from causal_group for the affected conjunct.

This also plays into the behavior of a pressured activation:

• It only has to use it's death_clock to reject fully-fulfilled causal signal subsets.
• If it is only holding on to completion signals, then it can rely on effect_not_caused to trigger it's auto-elimination.

Therefore, it is concluded:

• §1) Auto-elimination can be partially determined by effect_not_caused. Activations will release spikes naturally, if their cause-effect assumptions are not being fulfilled.
• §2) The activation's pressure 💥 ↔ 🕙 death_clock mechanism is only needed to end acquisitions of Fully-fulfilled Conjunct Causal Signal Subsets (FFCCSS). If the activation is pressured because it is waiting for completions, Pressure becomes an observation rather than a cause for action. Activations must reject FFCCSS for pressured causal groups after a given amount of time. Therefore, the death_clock mechanism requires the following new/amended APIs:
  • Constraint.fullfilled_causal_groups(): Retrieve the causal groups, for which there are FFCCSS. If the return value intersects with the activation's pressuring_causal_groups, the activation's death_clock will be started. This is checked both in Activation.pressure() and in Activation.acquire().
  • Constraint.dereference(spike, causal_groups): This API will be changed, such that it will only reject spikes from causal_groups if they are part of a FFCCSS. Note: It should be ok to leave pressuring_causal_groups unchanged, even if the causal group may not be referenced anymore after dereference.
• §3) The maximum age of a completion signal may safely bet set to ∞
• §4) Context.lowest_upper_bound_eta() may be removed.
• §5) It should be considered, whether adding a wait=<t> parameter to Signal constraints may be worth the effort. This parameter would replace a hard-coded death clock value for CCSS.
• §6) [known] The acquisition of a spike by an activation always resets the death_clock
• §7) [known] FFCCSS must not be rejected, if they are part of a fully-fulfilled conjunction.

Currently, RoboyQA crashes when the configured ID is bad. This should be checked, and the state should return Resign().

There is currently no active engagement when ravestate_telegramio is active because the telegram_run state is always running

Currently, a state 🅰️ which writes to property 💙 in reaction to signal ❗️may be completed with a state 🅱️ that also writes to 💙 while producing signal ❗️.

If 🅱️ has lower specificity than 🅰️, this means that 🅰️ will not get an effect_not_caused call for ❗️, even though ❗️ can not be produced by 🅱️.

Difference in parsed triples:

”Who...”: [NLP:triples]:
[<Triple object person:be:moon>]
vs
“who...”: [NLP:triples]:
[<Triple object PERSON:be:moon>]

Dims:

• X: Sentence Type (Question|Statement) per Subject type (Roboy|Interloc|3rd party)
• Y: Predicate

After creating this sheet, create the Sheetfillingtask.

For many states, it is logically necesssary to know a property value from the exact moment when a changed-signal was emitted; not from the rather arbitrary point in time when the state is actually activated. For example, a state might handle the addition of an interlocutor by reacting to interloc:all:pushed. However, multiple interlocutors might enter Roboy's vision simultaneously, at which point it becomes intractable not know which of the interlocutors is new and which is not.

For this purpose, signal payloads (or "values") are introduced through the following APIs:

• A payload (e.g. an interlocutor id) may be submitted through the state return value state.emit(payload)
• A payload may be retrieved through special context wrapper entries for each trigger signal: An entry always exists, even for triggers which did not contribute to the activation. An error is only raised for signals that are not mentioned as triggers at all.
• The payload for the :changed signal is always the new property value. The value for the :pushed and :popped signal is always the full property path of the created/deleted property.

Maybe move creation of telegram output state into the startup state and refer to the token via closure.

• Define object types that are streamed to Angular
  • IDs
  • Properties

Figure it out!

Ravestate context should recognize whenever it encounters indecision/uncertainty/inactivity (:thinking: / :sleepy:). This is useful for the following use-cases:

1. Indecision when waiting for a missing signal to activate a high-specificity state ↪️ Fillers
2. Indecision when no activation pressure is present at all ↪️ Active engagement (Fillers, Personal QA, Games) -> Roboy is not waiting for user input.

Both situations may be modeled as functions of two new core variables:

1. :pressure: A flag which indicates, whether or not activation pressure is present at all; i.e. whether a partially fulfilled high-specificity state is blocking a fulfilled low-specificity one.
2. :activity: A counter for the number of states that are currently partially fulfilled.

For example:

• Fillers may be adequate, when :pressure has been true for some minimum time (Filler to bridge "awkward" waiting)
• Active engagement may be adequate, when :activity has dropped to zero.

Note, that :pressure is true -> :activity > 0, but not :activity > 0 -> :pressure is true: Multiple partially fulfilled states may be present, without a fulfilled one generating activation pressure.

Therefore, two idle signals will be realized in an idle module:

1. idle:impatient ↪️ Fillers
2. idle:bored ↪️ Active engagement (Fillers, Personal QA, Games)

Especially for @rs.state and rs.Module.

This allows a state to "undo" a previous emission. For example, a state active-interloc-is-x may fire when the active interlocutor changes to "X". Now, a certain other state "Q" may depend on active-interloc-is-x AND rawio:in:changed. However, as soon as the active interlocutor changes from "X" to "Y", a previous emission of active-interloc-is-x should not be valid anymore to activate "Q".

For this purpose, the state of active-interloc-is-x may return state.Wipe() to indicate that it's signal should not be considered active anymore. It will then call context.wipe(signalname: str), which removes signalname from all current activation candidates.

Another use-case for this function lies in wiping outdated :changed signals which should not "agglomerate": I.e., An old nlp:tokens:changed signal from 10 minutes ago should not be considered "valid" anymore if there is a new nlp:tokens:changed signal, even if the old signal was not finally consumed by a state activation. However, wiping nlp:tokens:changed only when the respective state is running may not be timely enough: Consider that a state might depend on nlp:tokens:changed and nlp:is-question, but nlp:is-question may be emitted before nlp:tokens:changed, and thus cause consumption of an "almost outdated" signal.

For this reason, another rule must be introduced:

When a signal is wiped, all signals that were caused by it must be wiped too.

This will be implemented by annotating all current signal instances with a set of references to the signal instances that were involved in causing them. This will solve the above stated problem, because

1. rawio:in:changed will be wiped before a new instance is produced
2. Since nlp:*:changed is based on rawio:in:changed, all nlp:*:changed signals will be wiped too.

It also becomes increasingly obvious, that the concept of a SignalInstance is unavoidable.

Note, that context.wipe(signal) will also need to make sure, that states that are still running as a (direct or indirect) consequence of signal must not be able to fire new signals anymore!

Currently, the nlp:yesno property is a string, with magic numbers representing certain answers, which must be matched exactly. This should be replaced with a class that has explicit operators for specific answers:

You could then access these operators in a state with read-permissions for nlp:yesno like

if ctx[nlp.prop_yesno].yes():
    pass

Hint: Do not emit changed() for nlp:yesno if no yes/no value can be inferred from the input. instead, silently set the property to unk.

Currently, reggol log files have two shortcoming: They do ...
... not contain timestamps per log output
... retain color literals which clutter the logfile.

This should be fixed.

Currently, there is a hard-coded wait time for additional primary signals of 2 seconds.

This should be settable by the user on the signal.

The activation's death clock should always be set to the smallest configured wait time among the missing primary signals in the constraint.

Ravestate Akinator already detects unexpected inputs and asks for suitable input.
This state (wrong_input) is a good spot to give the user the opportunity to exit the game if he/she wants to.

Additionally get rid of infinite constraints in Akinator.

Since properties may be used to represent instances of particular objects (interlocutors, visual objects, etc.), it makes sense to provide the ability to bundle related properties under a parent property. Through this mechanism, properties can hold collections of child "nodes". Access to these child properties is modeled in API as follows:

• Children are addressed with property paths in the form of parent-path:child-name
• A child property may be created through ContextWrapper::push(childpath)
• A child property may be deleted through ContextWrapper::pop(childpath)
• Children do not know their parents.
• Read/Write access to a parent will implicitly grant the same access level to all children recursively.

Currently, the lack of symbols for property/signal names has 3 drawbacks:

1. unused import warnings for dependencies
2. typos in property paths lead to errors
3. forgetting dependencies

Therefore, the following set of ravestate API changes should be introduced:

• §0 Properties may be constructed like my_prop = PropertyBase(...) in a with Module(...) clause, module scope is assigned automatically. Dependencies may then use my_prop.
• §1 Signal can be constructed in with Module ... clause, module scope will be set automatically. Signal may be used as parameter signal=my_signal in state declaration
• §2 Referring to signals by name should become illegal. The s(...) function will be removed. Refer either by exported module variable, or by the respective Property/State APIs.
• §3 Setting Signal constraint properties like min_age, max_age, detached, will be moved from constructor parameters to chainable function calls: my_signal.min_age(30).detached(). Each function call returns a copy of the signal with the respective member value change.
• §4 Properties will be referrable by symbols. The context's push/pop/setitem/getitem functions will be changed accordingly.
• §5 It should be prevented, that developers call a property's read/write function directly. For this purpose, context should always adopt copies of the respective property objects. The originals will be untouched. An exception to this rule is generally made for child properties, since they mark an advanced use-case where properties are generally created dynamically, and usually accessed through their path anyway.
• §6 The PropertyBase class will be renamed to Property.

Set up an angular project with TypeScript, Angular, ...

Location: modules/ravestate_uix (name it!)

Handling HTTP get request errors in ravestate_genqa.
The drqa_module() handles the request.

Define this once done with #93.