asynkron/protoactor-go#56

The C# implementation lacks the supervision support. this needs to be copied from Go

There is some missing bits of code for the Watching mechanics in the actors, the _watching field is always null.
This should be ported from Go

When an actor is stopped we don't notify the watchers.

Go implements this: https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/local_context.go#L361

I generally try to avoid discussions about code style, I think that the default Resharper settings are almost always the way to go.

But, being a Go developer also, I have started to like forced braces on conditionals.
e.g.

if (x) 
{
     Foo();
}

//vs

if (x)
   Foo();

My personal opinion is that it adds a nice symmetri to the code.

Another thing I personally like to see, is to use C# 7 features when possible.
e.g. "throws expressions" for null validation. named tuples etc.

Thoughts?
Is this something we can do or am I too damaged by Go? :)

In Go you can replace the default subscriber for deadletter and supervision events.

See e.g.:
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/options.go
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/deadletter.go

I want to base the initial cluster support on Consul.IO.
Making each Proto Actor process register themselves into Consul as services.
We can then simply use the Consul REST API to query for cluster changes.

Consul supports "blocking queries" where you get a token each time, and if you provide this token for the next call, the call will block until any changes occur.
This prevents polling, only returning once status changes.

This has some nice benefits:

We don't need to build any custom cluster code ourselves. (e.g. Paxos, Raft, or implement a gossip protocol).
All of the complexity is simply offloaded to a battle proven tested cluster software, and we piggy back on that.

It will work the same way for any language implementation.
(currently the Go implementation uses MemberList from Hashicorp, the lib that Consul is built ontop of.. )

Consul supports "Tagging" of services. this means that each Proto Actor process could provide tags when it boots up.
These tags could be used to describe "Roles". e..g. "Worker", "Frontend" etc.
This means we can have automatic placement of actors/grains across different roles.
Some actors/grains might only be available on a specific role.

Akka has similar support, but I don't think there is anything like this in Orleans(?)

Currently, the user message pipeline is fully async using Tasks.
The SystemMessage pipeline is not.
It would be nice to be able to do non blocking operations in the system message pipeline too, e.g. tracing, or other IO.

The Go plugin pipeline has changed a bit since it was initially ported. Port these changes.

To have some complier enforcement, messages should have IMessage or IMessage

This way, you can use the Tell or Request methods and have strongly typed enforcement of messages akin to how MediatR works with its messaging: https://github.com/jbogard/MediatR/wiki

This should have no reflection in the serialized messages, this is purely a C# nice to have.

As of yesterday, Remote performance in Proto.Actor .NET is now slightly better than the Go counterpart.

2.5+ million messages per sec in .NET
vs
2.4+ million messages per sec in Go

Which puts us on par with ZeroMQ.

This can be seen in contrast to the performance of Akka NET
akkadotnet/akka.net#2378 (comment)

We are now more than 50 times faster over the network, with version tolerant serialization and cross platform communication.

Currently, when using port 0, gRPC will use a free port as expected. but, the system does not know what port this is and thus PID lookups fail.
We need to either extract the port from gRPC, or find a free port if port 0 is used.

We need proper error handling in the event stream handlers.
If the code below fails, that might kill the application

        public void Publish(T msg)
        {
            foreach (var sub in _subscriptions)
            {
                sub.Value(msg);
            }
        }

The RestartStatistics in Go have changed slightly to carry intent rather than just expose state.
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/child_restart_stats.go
It now has a few methods to show what you can do with the stats.

(It can still be improved, e.g. not exposing mutable fields in the way it does now)

The current implementation of assigning the cached Process in the PID, is in the ProcessRegistry.
https://github.com/AsynkronIT/protoactor-dotnet/blob/master/src/Proto.Actor/ProcessRegistry.cs#L43

This is not how it works in Go.
The Go impl, will check if the cached process is nil, and if so, it will resolve the process and assign to the cache field.
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/pid.go#L11

The benefit of doing this is that when a process dies, the cache can be invalidated for each PID instance pointing to that process.

Why can there be multiple PID instances pointing to the same process?

The PIDs might come from deserialized messages, and thus they will all contain the same values, but be different instances.

The bounded mailbox has some nice properties in terms of low GC overhead.
It would be good to have the same for C# too.

There is a new interface in Go:
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/supervision.go#L14

The implementation is here:
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/local_context.go#L395
and here for the root supervisor
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/root_supervisor.go

This interface is now used in the SupervisionStrategy, like so:
https://github.com/AsynkronIT/protoactor-go/blob/dev/actor/strategy_one_for_one.go#L23

This means that the "Supervisor" is responsible for how to Stop, Restart and Escalate failures.
It's mostly an abstraction to not have to deal with system messages inside the supervisor strategy.
But it can also make testing SupervisorStrategies easier.

When creating a child actor and in the child actors context calling the parent by typing "context.Parent" it is always null.

PersistEvent and PersistSnapshot are blocking.
We should change these to be async instead.

In Go we now have a Tell method on context, and a middleware chain for outgoing messages: asynkron/protoactor-go#100

EndpointWriterMailbox should use a big bounded mailbox for user messages.
This is what we do in Go, in C# we still use a concurrent queue here.

As there will not be a lot of endpoint writers, it's ok to have a bigger queue.

Currently we use a FutureActor for Futures.
This results in some extra weight for Future resolution.
Follow the go implementation instead

In Go we have the following config object that can be passed when starting the remote system:

type remoteConfig struct {
	serverOptions            []grpc.ServerOption
	callOptions              []grpc.CallOption
	dialOptions              []grpc.DialOption
	endpointWriterBatchSize  int
	endpointWriterQueueSize  int
	endpointManagerBatchSize int
	endpointManagerQueueSize int
}

@cpx86 gave a very valid reason in #32 about having the mailboxes as a separate project.
One might want to use the mailbox in isolation just to serialize async work, a DIY lightweight actors alternative.

We need to get the Inprocess ping pong benchmark from Go ported so we can measure the infrastructure overhead in C#.

The routing package in Go has been refactored since the .NET implementation was done and is now standalone. The same should be done here.

Also a bit related, maybe the Proto namespace should be moved to Proto.Actor, so we align the Go packages/.NET assemblies.

Add .NET build pipeline in Travis. possibly using Cake build

The persistence module needs to be ported over to C# too.

We need to come up with a plan how to use plugins in C#.
In Go, we use embedded fields, this is not available in C#.

I would like to avoid basing this kind of functionality on inheritance at all cost.
We did this in Akka.NET resulting in many many permutations of ReceiveActor, PersistentReceiveActor etc.

Suggestions?

The remote activation features is missing from the C# implementation.
I've added the Register and GetKnownKinds methods.
We need the Activator actor to be implemented too.

The batching EndpointWriterMailbox used by Proto.Remote could be useful to have for other things as well, e.g. batching Zipkin spans.

We now have two different PR's for scheduling messages.
Before picking an implementation, we should think about middleware semantics.

e.g.
Let's say we have a tracing middleware.
And now someone tries to schedule a message in 10 seconds.
Should that message go through the middleware?
Should the trace be continued from the original point forward to the receiver of the scheduled message?

This relates to #95 #69

In Go, we have this Protoc.exe generator:
https://github.com/AsynkronIT/protoactor-go/blob/dev/protobuf/protoc-gen-typedgam/template.go

We need to have a similar for C#.
It should be possible to re-use the same Go generator and just switch out the template, no need to rebuild the actual generator in C#.

In Go we have switched from catching exceptions per message, to catching per mailbox run and invoke the MessageInvoker.EscalateFailure from there.

This means that the mailbox must keep track of the current message, be it system or user msg.

Test coverage is quite so-so at the moment, and should be improved.

I don't think 100% coverage is a target worth striving for, but there should be at least a few tests for each feature.

Hi,

when I try to add the package proto.actor via the myget feed, I get the following error:

Resolving packages for group Main:
 - proto.actor 0.1.0.56
 - System.ValueTuple 4.3.0
 - Google.Protobuf 3.2.0
 - NETStandard.Library 1.6.1
 - NETStandard.Library 1.6.0
 - proto.actor 0.1.0.51
There was a version conflict during package resolution.
  Resolved packages:
   - Proto.Actor 0.1.0.51
  Could not resolve package Proto.Mailbox:
   - proto.actor 0.1.0.51 requested package Proto.Mailbox: >= 1.0
   - Available versions:
     - (0.1.0.56, [https://www.myget.org/F/protoactor/api/v2])
     - (0.1.0.51, [https://www.myget.org/F/protoactor/api/v2])

  Please try to relax some conditions.

I was able to install it by just pinning the package versions. =)

(https://twitter.com/RogerAlsing/status/833675614360055808)

Currently the outgoing middleware has all the parts it needs internally.
but, the Process.SendUserMessage has the signature: (PID pid, object message, PID sender)
which does not allow us for sending outgoing headers.
And if we try to send a MessageEnvelope, things get double wrapped if there is a sender as the LocalProcess checks for sender and wraps the current message in another MessageEnvelope.

TLDR;

We should change the signature of SendUserMessage to just (PID pid, object message) and do the envelope wrapping already in Request

There has been quite an influx lately of contributions (or ideas for) extending Proto.Actor in various ways. E.g. new persistence providers, scheduling/delaying of message sending, DI container integrations, logging, et cetera. I very much welcome these contributions, however, there's always a risk of bloating a library with non-essential features that complicate the code base or makes maintenance of it harder. On the other hand, these contributions are valuable enhancements to the library because they make life easier for users. Another thing is that the core library should now be very stable and between releases very little if anything should change in the API, whereas for example extensions for third-party integrations might require more frequent changes.

What I would like to discuss in this issue is how we can setup a governance/maintenance model for these extensions that achieves the following:

1. Protects the library from feature bloat.
2. An easy and contributor-friendly way of extending Proto.Actor with new functionality.
3. Allows for the core library and the extensions to move and evolve separately at their own pace.
4. Doesn't leave the maintainers with a slew of extensions that require support and maintenance.

I have two suggestions for how we could set this up, but please pitch in :)

1. Move everything except the core libraries (Actor, Mailbox, Cluster, Persistence, Remote and Router) into a separate repository, e.g. protoactor-dotnet-extensions, that is versioned and built separately.
2. Move each extension into a separate repo outside of the organization, so each of those extensions are maintained completely by their respective authors. The main repo would maintain a curated list of available extensions.
3. ???

Since we're starting to get the ball rolling on porting all the major features from Go, I think it's time to start discussing how we should structure the solution (files, directories, projects, etc) and also the API surface (packages, naming conventions, namespacing, etc).

To me, the main challenge is that there are two desirable goals that can conflict:

1. The API surfaces in .NET and Go should be as similar as possible
2. .NET and Go have different conventions and capabilities

Some concrete questions to get the discussion started:

• Should the structure of the NuGet packages map 1-to-1 to the structure of the Go packages? Importing Go packages is a lot more lightweight than importing NuGet packages, both due to different versioning strategy and due to the standard tooling. Having a granular separation of packages has a lot of value, but maintaining installs/upgrades of all packages in a .NET solution can have a tendency to be a bit painful.
• Go lacks namespaces - in C# code they are often abundant. Should we keep the flat structure of the Go API or should certain things be moved into sub-namespaces?
• In Go, non-receiver functions are accessed through <package>.<function>, e.g. actor.Spawn(). The closest emulation of this in C# is static methods. This can cause internal conflicts with package/namespace names. E.g. if you have a namespace Proto.Actor containing a static class Actor, it will have to be referenced as Actor.Actor. Shouldn't cause issues for user-code, but might be a bit messy internally.
• Go doesn't have classes or constructors - structs are created through "factory" functions, e.g. mailbox.NewBoundedProducer(). Should we keep this pattern/naming convention to make the APIs identical, or should we instead use class constructors, e.g. new BoundedMailboxProducer()?
• In Go, there is only one way to access a field on the underlying struct of an interface - through receiver functions defined on the interface, e.g. context.SetReceiveTimeout() or context.ReceiveTimeout(). In C# this could be implemented with a non-automatic property context.ReceiveTimeout for both get/set; or with an automatic property with a private setter and a public context.SetReceiveTimeout() (if you don't want complex logic inside the setter); or with a private field and both a context.GetReceiveTimeout() and a context.SetReceiveTimeout() method. How should we value idiomaticity vs. consistency?

We need more documentation for C#. I would be willing to contribute, if there is a way? A Contributing.md would be awesome, too.

Routers are currently missing from the C# implementation

My impression is that the https://www.nuget.org/packages/microsoft.extensions.logging.abstractions/ is what will become the de-facto standard for logging in .NET.
After all, this is what is baked into Asp.NET Core.

Thoughts, should we aim for this?

I modeled this after MediatR. I basically want the same behavior. Invoke a message and have it routed to an Actor

Declaration

    public class TestActor : IActor,
        IActorRequestHandler<TestRequest1>,
        IActorRequestHandler<TestRequest2, string>
    {
        public Task Handle(TestRequest1 request, IContext context)
        {
            Console.WriteLine("Got test 1");
            return Task.CompletedTask;
        }

        public Task<string> Handle(TestRequest2 request, IContext context)
        {
            Console.WriteLine("Got test 2");
            return Task.FromResult("test");
        }
        //default
        public Task ReceiveAsync(IContext context)
        {
            return Task.CompletedTask;
        }
    }

Test usage

  static void Main(string[] args)
        {
            ActorRegistry.Scan(Assembly.GetEntryAssembly());

            var serviceCollection = new ServiceCollection();
            serviceCollection.AddProtoActor();
            var sp = serviceCollection.BuildServiceProvider();

            ActorHandle handle = sp.GetRequiredService<IActorFactory>()
                .GetActor<TestActor>();
            
            handle.Publish(new TestRequest1());
            handle.Publish(new TestRequest1());
            handle.Publish(new TestRequest1());

            var response = handle.Send(new TestRequest2()).Result;
            Console.WriteLine(response);

            response = handle.Send(new TestRequest2()).Result;
            Console.WriteLine(response);

            Console.ReadLine();
        }

Currently there are two processes involved when using Proto.Router - the RouterActor and it's LocalProcess, and the RouterProcess which proxies system and management messages to the RouterActor, and sends other messages to the routees.

A problem with this is that we get two PIDs for the router, one for each process. This messes up the watch/unwatch functionality since any watch message would be for the RouterActor's LocalProcess, but the PID you get when you create the router is actually the RouterProcess

This should be merged so that we only have one Process involved. A way forward is to change the spawn logic so that instead of having a RouterProcess referencing a LocalProcess referencing a RouterActor, you would just have a RouterProcess referencing a RouterActor. The RouterProcess would need to handle the mailbox in the same way as the LocalProcess does though.

I cannot open the solution in VS 2015. I get the following error message in the output:

C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Messages\Messages.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Messages\Messages.csproj

C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Node2\Node2.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Node2\Node2.csproj

C:\Projects\other\protoactor-dotnet\examples\SpawnBenchmark\SpawnBenchmark.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\examples\SpawnBenchmark\SpawnBenchmark.csproj

C:\Projects\other\protoactor-dotnet\src\Proto.Actor\Proto.Actor.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\src\Proto.Actor\Proto.Actor.csproj

C:\Projects\other\protoactor-dotnet\src\Proto.Remoting\Proto.Remoting.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\src\Proto.Remoting\Proto.Remoting.csproj

C:\Projects\other\protoactor-dotnet\tests\Proto.Actor.Tests\Proto.Actor.Tests.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\tests\Proto.Actor.Tests\Proto.Actor.Tests.csproj

C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Messages\Messages.csproj : error  : The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.  C:\Projects\other\protoactor-dotnet\examples\RemoteBenchmark\Messages\Messages.csproj

I have the following VS:

Microsoft Visual Studio Professional 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01586

Installed Version: Professional

Visual Basic 2015   00322-50050-22180-AA047
Microsoft Visual Basic 2015

Visual C# 2015   00322-50050-22180-AA047
Microsoft Visual C# 2015

Visual C++ 2015   00322-50050-22180-AA047
Microsoft Visual C++ 2015

Visual F# 2015   00322-50050-22180-AA047
Microsoft Visual F# 2015

Application Insights Tools for Visual Studio Package   7.15.01215.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2015.1   14.1.21111.0
ASP.NET and Web Tools 2015.1

ASP.NET Web Frameworks and Tools 2012.2   4.1.41102.0
For additional information, visit http://go.microsoft.com/fwlink/?LinkID=309563

ASP.NET Web Frameworks and Tools 2013   5.2.40314.0
For additional information, visit http://www.asp.net/

Azure App Service Tools v2.9   14.0.20316.0
Azure App Service Tools v2.9

Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.

Azure Data Lake Tools for Visual Studio   2.0.6000.0
Microsoft Azure Data Lake Tools for Visual Studio

Common Azure Tools   1.8
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Intel® VTune™ Amplifier XE 2017   
Intel® VTune™ Amplifier XE 2017, (build 478468), Copyright © 2009-2016 Intel Corporation. All rights reserved.

JavaScript Language Service   2.0
JavaScript Language Service

JavaScript Project System   2.0
JavaScript Project System

JetBrains ReSharper Ultimate 2016.2.2    Build 106.0.20160913.91321
JetBrains ReSharper Ultimate package for Microsoft Visual Studio. For more information about ReSharper Ultimate, visit http://www.jetbrains.com/resharper. Copyright © 2017 JetBrains, Inc.

Microsoft .NET Core Tools (Preview 2)   14.1.21111.0
Microsoft .NET Core Tools (Preview 2)

Microsoft Azure Mobile Services Tools   1.4
Microsoft Azure Mobile Services Tools

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2015 - v2.9.40323.3

NuGet Package Manager   3.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

PreEmptive Analytics Visualizer   1.2
Microsoft Visual Studio extension to visualize aggregated summaries from the PreEmptive Analytics product.

SQL Server Data Tools   14.0.60519.0
Microsoft SQL Server Data Tools

ToolWindowHostedEditor   1.0
Hosting json editor into a tool window

TypeScript   1.8.36.0
TypeScript tools for Visual Studio

ReceiveTimeout is implemented in Go, it needs to be ported over to C# too

Previously the PID has a Host property in the protofiles. we have now changed this in the Go version.
The property is now called Address as it can be both host and port together.

We need to reflect this change in C# too.
There is also a change in the remoting proto to make the grpc service return a stream of Unit. (we need this to detect disconnects in on the caller side)

The current implementation of the Started message is handled synchronous from the creators perspective.
This has been changed in Go, we should follow here.

...but looking at examples I just spotted my brand new VS2017 shows red swiggles for any C#7 language statement (e.g. switch on non integral values or switch case patterns).

Any suggestion? Maybe it depends on Resharper?

Thanks 😄

Hello,
I cloned and compiled your stack in VS2017 RC1 + Net.Core 1.1. Everything is running smoothly.
What I want to succeed is to use it as a simple but powerful managed actor system in-process, which will use actors internally and also to communicate with other distributed actor based systems. So to have an unified software design based on actor model (maybe partially, e.g. services and actors when I need such behavior for specific entities ) .
As I have understood from your examples, I can use protoactors in process as C# objects and only if I need remoting (between different processes or other distributed systems), then to create an instance of the gam server and to use the Protoc.exe generator ( or in the future, the C# equivalent).
There are a lot of modules that you wanted help, I don't know if I am capable to help, as my mind is in the application design with the actor model, but if I find something as I will use it, I will try to help.
thanks

I used Props.Spawn(string, PID) to try to assign an actor an parent. However, it looks like only Context.SpawnNamed(Props, string) will do what I want.

Am I misunderstanding something or is there a disconnect in the code?

Hello, i have next problem.
I have .net core web application created from template (.net core 1.1)
In Configure method of Startup class i spawn actor and store PID in static field.
Then i send message to Actor in Controller Get Method.
Actor receive only first (firstly open page) and second (after refresh page) messages,
but next messages doesn't receive.
If i delete async, it works correctrly.
Is it bug or ProtoActor doesn't support async calling?

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
    {
        loggerFactory.AddConsole(Configuration.GetSection("Logging"));
        loggerFactory.AddDebug();
        var props = Actor.FromProducer(() => new HelloActor());
        pid = Actor.Spawn(props);
        app.UseMvc();
    }  
  // actor
   internal class HelloActor : IActor
    {
        public async Task ReceiveAsync(IContext context)
        {
            var msg = context.Message;
            if (msg is Hello)
            {
                await Task.Delay(5000);
                Console.WriteLine($"Hello {((Hello)msg).Who}");
                context.Sender.Tell(999);
            }
        }
    }
   // controller
    [HttpGet]
    public IEnumerable<string> Get()
    {
        Startup.pid.Tell(new Hello
        {
            Who = "Alex"
        });
        return new string[] { "value1", "value2" };
    }