type User struct { name string, age int }

func someFunc(u User) {
  u1 := User{u.name, u,age+1} // <- dynamic struct building
  someOtherFunc(u2)
}

How to do this in FBP?

TODO: think about map and list (are they different?)

UPD: maps and list are different because they are dynamic data structures - impossible to now keys/elements count at compile time (so component approach will be ok)

Might be related to http://www.jpaulmorrison.com/fbp/simpapps.shtml (assign)

FBP has lots of generic components like Select or Count. Current implementation assumes compile-time type-checking so there must be a concept of generic types.

Goals:

• No any
• Nested expressions X<Y<Z>>

How to resolve generics between two components in a scope?

deps:
    me:
        io:
            arg:
                T
            in:
                l:
                    in:
                        type: list
                        args: [T]
            
io:
    arg:
        T
    in:
        l:
            in:
                type: list
                args: [T]

Related to #62 and #43

1. Create program from scratch
2. Modify existing program
3. Debug program

Problem: in Go it's convenient to describe some type and use it everywhere in the package (type could also be exported like function or variable) while current implementation will force user to duplicate same data-structures for every component.

Maybe there should be a way to declare types at a package level?

Relates to #43

in:
a:
b:

out:
a:
b:

Case: suppose some subtractLeftToRight component with numbers array-inport - it must process messages from ports in sorted order.

Problem: as it turns out there's no ordering guarantee in the current mplementation due to map[PortAddr]chan Msg usage in core/runtime

In Go it's possible to pass pointer to function that takes interface{} like json.Unmarshal:

var v struct{x int}

if err := json.Unmarshal(&v, []bytes{" {x:42} "}); err != nil {
  panic(err)
}

How to do this in FBP where we don't have pointers?

NOTE: this is fundamenal since almost any program gets its input from outside as bytes

• Problem at API level for user of the compiler
• Problem at implementation (runtime) level

Context

In case if #26 will be supported

Problem

Figure out if its needed to support taking element from list by index and, if so, what to do with "out of borders" issue?

Case

For tasks like "subtract numbers from first to last" where:

1. Arguments count unknown
2. Order matters

We need a way to represent ordered variadic arguments

Problem

Imagine some component with an array-outport in a module's network.
It's dangerous to refer specific indexes of that outport because we don't know how many indexes will be used inside that component.

Another case is when some module has an array-inport.
It's dangerous to refer some specific indexes of that inport because we don't know how many indexes will be used by parent network.

But port reference assumes some index!

Actually these two cases are the same though we talking about outport in the first one and about inport in the second.
This happens because for module's network - inport is actually an outport, the port to read from!

UPD: array ports are like variadic arguments / rest in Go/TS. We shouldn't use specific positions args[2] from ...args as well as we should't use results = returnList(); results[2]. By using only args/results as is we are safe. This is what bypass is!

Introduce some testing facility and figure out how to make it as easy to use as possible

• https://swr.vercel.app/
• https://react-query.tanstack.com/
• https://redux-toolkit.js.org/
• https://recoiljs.org/
• https://effector.dev/

Problem: Suppose we have userGenerator with user outport of type User {name, age, friend: User } and some friendAgePrinter component. How do we pass user.friend.age to its friendAge inport?

It's easy to x.y.z in conventional languages but FPB has no concept of expression. How to address specific place in memory?

Context

In classical FBP we have ASSIGN component that takes an IP and somehow modifies it taking value and field from OPT port. How do we solve this "data modification" task here?

Expected problems

• Race conditions

Might be related/solution for #53

k9s is a good example

What CLI used for?

Looks like every inport must be used to avoid blocking of the program but what to do if things gets wrong?

Might be related to #54

There are already const nodes with predefined static values but what to do if what we want is to have some default value and update it from time to time?

Might be related to http://www.jpaulmorrison.com/fbp/simpapps.shtml (assign)

• create programs
• edit programs
• debug programs

Must be a way to represent recoursive data structures like this:

{
  id: 0
  children: [
    {id: 1, children: []}
  ]
}

Related to #33 because without type references its impossible to describe recursive data structures

Also related to #51

Context

Current implementation of most operators like Mul that operates on a list of values uses array ports as its API

Problem

As it turns it's impossible to solve such simple task as "read numbers from stdin and multiply them" with such mechanics.

The reason for that - array ports has static nature, more static than e.g. variadic arguments (because dynamic list could be "unpacked" as arguments). I.e. all ports/channels must be defined at compile-time. It seems undesirable to modify program's network at a runtime (it is an interesting topic but this is not how and why this should be implemented) which is needed to dynamically create channels and connect them together.

Shortly speaking - if input comes from "external" place like stdin or network (which is usually the case), it's impossible to predict how many channels there will be at compile-time. So all operators that work with array-ports at this time must work with normal ports instead. And they must use some different approach that will be compatible with dynamic nature of lists.

• Package - consists of modules
• Public/private modules?
• Where to store?
• CLI interface?
• How to create reused packages with only abstract dependencies?

Sub streams with support of unlimited nesting allows asynchronously process nested data but they are not implemented and it seems to be very hard to achieve.

Example from FBP book:

 < city1 
     < cust11 detl111 detl112...> 
     < cust12 detl111 detl112...>
  >
  < city2 
     < cust21 detl211 detl122...> 
     < cust22 detl221 detl222...>
  >
  ...

Problems

• Type-Checking: sub-streams represent not just (nested) lists but (also) hierarchic structures. It's not clear how to type-check such mechanics.
• Invalid bracketing - what to do (and how to implement such handling) if structure of sub-streams is invalid? e.g. not enought closing brackets or ordering of items is messed

Related to https://github.com/emil14/fbp-book-ru/blob/main/7_compos.md (sub-streams) and #58

Related to #58

Are staticly typed sub-streams even possible? Imagine a component with sub-stream sensitive inport - where does it takes input? From several different components that we hope sends their data in the right order?

• gogoproto
• original proto
• easyjson

How to implement this if we don't control flow?

fn() {
  v1, err = f1()
  if err != nil {
    return err + "f1"
  }

  v2, err = f2()
  if err != nil {
    return err + "f1"
  }

  v3, err = f3()
  if err != nil {
    return err + "f1"
  }

  return {v1, v2, v3}, nil
}

It's pretty clear that every inport must be filled not to block program (maybe with some default value but that's different story) but what to do if some outport isn't used by network?

Context

Creating module with array-inport suggests that we don't know how many elements will be used by parent network. We want to have an ability to modify parent network without modifying the module (especially because it could be third-party module). This means we need a way to "bypass" array-port elements (in and out).

Problem

How to determinate elements count for array-ports that connected to another array-port with bypass connection? It's clear how to compute channels for:

A -> X[0]
B -> X[1]
C -> X[2]

But not for D[:] -> X[:]

In classical FBP process can be 1) initiated 2) terminated 3) resumed. Process is initiated when it has data on the inport (any of them) or (if it doesn't have inports) at startup. It's terminated when all the upstream processes are terminated (their outports are closed).

There's at leas one example of how classical FBP uses this technique described in chapter 7:

If D was a Writer, it would then close the file it was writing to.

Another example is a Counter

Because of lack of this logic several questions must be answered:

1. What we loose by not implementing this?
2. How hard would it be to implement this?

Context

Let there be some x stream of User messages and we want to split those messages into two separated flows - one for women and one for men. How to do it?

In FBP there's a special component for that which is called Splitter. It's a component that has inport and array outport, goal of that component is to split messages from that inport to different outport slots. This could be viewed as inverted merging. Splitter must be some how set (via OPT port in classical FBP) so it known what to look at (e.g. look at the x field of message and if its

See example http://www.jpaulmorrison.com/fbp/Fig6.7.gif

How to implement?

Might be a solution for #38

For recursive data-structures with direct or indirect struct-based cycles there must be a way to tell "this time recursion stops". In other words it's not a problem for type like this:

node:
  id: int
  children: [node] # <- indirect list based self reference

node:
  id: int
  children: map[string]node # <- indirect map based self reference

But it is for this:

node:
  id: int
  left: node # <- direct self reference
  right: node

Or

node:
  id: int
  child:
    meta: string
    value: node # <- indirect struct-based reference
  

Because we need something like left: node? or left: node | nil.

Also questions that must be answered:

• should we have special null type or null is a valid value for some (all?) existing types?

TODO: think if this problem exist for substreams

Problem:

Current implementation has form of

x: int // norm port
y[]: int // arr port

Map-port (if it'll be added) could have form of z{} but:

1. What to do with struct ports?
2. What if there will be more of different ports?

related to #43

Define solid program structure at compiler and runtime levels

Context

Outer world represents list as a fixed (not flowing) data. E.g. suppose we need to parse this JSON into some struct:

{
 "vv": [1,2,3]
}

Problem

Sub-streams (if they will be implemented) hardly fits this need. Array-ports also can't solve this due their static nature.
List data-type must be implemented.

It's not clear what data must be present at runtime level due to issues with maps and structs. Current implementation assumes names and indexes but it may be wrong

TODO figure out real cases that must be solved

Now it is possible to put outport of a const node and outport of a regular node to the same inport which lead to merging of streams. Looks like it doesn't make sense because of how const works - they are always ready to send a value

There should be bytes type to work with stuff like JSON

Be careful because types are may not be exportable

Context

There must be ability to:

1. Set breakpoints
2. Pause and resume execution
3. Move backward and forward

Related to #34

1. Add strict typescript config
2. Add ESLint with TS support and strict config

Since operator defined as compiled go code there is no way to choose - include some operator to runtime or not!

Any outside world connectivity should be done by operators (elementary components)

Because of this it's not clear how to implement low level interactions with network e.g.

Or maybe we should move this to some outside system...?

Direct dependency

# root
deps;
  root: ...
workers:
  self: root

# pkg
scope:
  root: root

Current implementation of generator will stuch in an endless loop:

1. create self worker for root
2. since it's not an operator put its dep (root) in the queue
3. take the element from the queue
4. ...
5. it's root again, go to step 1

Indirect cycle dependency

It's possible to create package where a depends on b while b depends on a:

# a

deps:
 x: ...
workers:
  xw: x

# b

deps:
  y: ...
workers:
  yw: y

# pkg.yml

scope:
  x: b
  y: a

Which will lead to endless loop in a runtime-program generation step:

1. To create xw worker for any instance of a component we need to create an instance of b
2. To create yw worker for any instance of b component we need to create an instance of a
3. Go to first step

Related to #46 (recursion of components)

• runtime is fast, compiler is reliable
• code should not be written
• if it compiles and doesn't crush on startup - it works without exceptions

https://github.com/emil14/refactored-garbanzo/blob/5499e4a975baff290212a4064081df87e58f6ef3/internal/runtime/custom.go#L72

For tasks where:

1. fast reading by key is preferred
2. duplicates should not be allowed

There must be map/dict/hashTable representation

Might be related to https://camo.githubusercontent.com/aa0cb1abf777d6b79436c102edea6902888c5da9c75c74791e2e5645aeddaa18/687474703a2f2f7777772e6a7061756c6d6f727269736f6e2e636f6d2f6662702f466967362e31342e676966

for json and yaml schemas

1. Use two different types to represent regular and array ports

type Ports map[string]Chan
type Chan interface{}
type RegularChan chan Msg
type ChanArr []chan Msg

Context

Original goal was to implement "if it compiles then it runs" strategy but the more stuff done the less real this claim seems to be

Problems

Recursion

If it will be available it's hard to see how to catch at compile time endless recursion algorithms

Array-outports indexing

Described in #29

Array port defenition

portName[]: int

portName:
  type: int
  arr: true # optional

portName: int[]

Array port reference:

node:
  outport[0]:
    node:
      - inport[0]
      - inport[1]

There must be a way to map n things to n directions. E.g. implement http router for a set of routes so the request will be redirected to specific handlers

x ->
y ->
z ->
  router
    -> handleX
    -> handleY
    -> handleZ

Relates to #29

Maybe relates to #37

Might be related to http://www.jpaulmorrison.com/fbp/simpapps.shtml (see assign component) (TODO: create solution-comment)

Right now operators are implemented as go plugins. Problem is - every <filename>.so about 2mb big.
Issues related - slow deploy, slow start (time to load all plugins).
For example - all operators could be loaded as a single package.
Alternative is to create math, logic packages and