Integrate Spores3 into the Portals Actor library about portals HOT 3 OPEN

If everything goes smooth, then we can also consider adding Spores into the core library.

from portals.

I did some quick testing on Spores this morning. The following shows a first approach to integrating Spores into Portals.

import com.phaller.spores.Spore

object Log:
  def log(msg: Any): Unit = println(msg.toString())

class TaskContext[T, U]:
  def log(msg: Any): Unit = Log.log(msg)
  def emit(event: U): Unit = Log.log(event)

trait Task[T, U]:
  def onNext(t: T)(using ctx: TaskContext[T, U]): U

object Tasks:
  inline def map[T, U](inline f: T =>  TaskContext[T, U] ?=> U): Task[T, U] =
    val sp = Spore.apply((t: T) => (c: TaskContext[T, U]) ?=> f.apply(t)(using c))
    MapTask(f)
  
  case class MapTask[T, U](f: T => TaskContext[T, U] ?=> U) extends Task[T, U]:
    def onNext(t: T)(using ctx: TaskContext[T, U]): U = f(t)

object DSL:
  def ctx[T, U](using TaskContext[T, U]): TaskContext[T, U] = summon[TaskContext[T, U]]

@main def example(): Unit =
  import DSL.*
  given TaskContext[Int, String] = TaskContext[Int, String]()

  val task = Tasks.map[Int, String](x =>
    ctx.log(x)
    x.toString()
  )
  task.onNext(3)

  // def badFactory(c: AnyRef): Task[Int, String] =
  //   Tasks.map[Int, String](x =>
  //     // Error: Invalid capture of variable `c`. Use first parameter of spore's body to refer to the spore's environment.bloop
  //     c
  //     x.toString()
  //   )

In this first presented example, we use the Spore factory for checking that the functions do not capture any variables that are unsafe to capture. We could also use the Spore type directly as the type of the function in the provided case class. A similar approach should be feasible for the Actor Runtime, where we can try it out first. We should try it out to see: how the Spore factory can be used for verifying that the functions are safe, and we could also try to use the Spore type as the type of the function in the ActorBehavior case class.

Some issues which were encountered, for which we would like to have a more elegant solution, is the fact that the Spore factory requires a function literal. This is the reason why we have Spore.apply((t: T) => (c: TaskContext[T, U]) ?=> f.apply(t)(using c)) and not just Spore.apply(f). It should be possible to preserve the function literal used as the parameter f, although I didn't manage to do so this time. It probably requires some use of macros. This should be solved, such that we can pass the literal directly to Spore.apply. Perhaps, an option would also be to directly call the checkBody method (which is private in Spores3) to check these properties.

Note that this does not use the Builder, which, to my understanding, enables serialization/deserialization using the uPickle framework. This would be something for future considerations, however, to my understanding, this would require that every Builder object is a top-level object, which will increase the burden on the application-side (we should verify this).

from portals.

After some digging the other day, it seems as though it would be feasible to make the Builder accept nested objects. It would be feasible to submit this as a PR to the Spores3 repo for enabling nesting objects. The corresponding macro code can easily be generated by chat-gpt. It can be checked at compile time that the passed reference is 1) an object; and 2) extract its Java Class path. This class path can then be used as the function identifier (it will be slightly longer, one extra dollar per nesting AFAIK, but this will be fine.
Another interesting direction to take could be to encode the rather long function identifiers as entries in a Map or similar. This could probably be done at compile-time also, and instead of serialiazing the entire function identifier, it can be compressed to a shorter identifier (one byte, or so). This would be one potential for optimization, in case this is a bottleneck. This is loosely inspired by some work I read a longer time ago on efficient offloading in HPC by Matthias Noak (at ZIB), one example is: "Heterogeneous Active Messages for Offloading on the NEC SX-Aurora TSUBASA"

from portals.