jasperfx / jasper Goto Github PK

• Use the old [assembly: FubuModule] trick to know what to grab
• Use the old IFubuRegistryExtension trick too
• The FubuPackageRegistry for easy functional extensions

The overwhelming feedback on twitter was to avoid taking any kind of hard reference on a logging library, so back to the drawing board. I'm not enthusiastic about LibLog and even less so about Common.Logging or the new, massive logging abstraction in ASP.Net Core.

Exact same issue we hit w/ Marten when we were using the Roslyn codegen.

When we do the dynamic compilation, you need to explicitly tell Roslyn about every single assembly that needs to be referenced and used by the code being generated. That includes any assemblies that the application references. I think we need to either:

1. Use the DependencyContext to just add every loaded assembly. Not sure if that would hurt anything, and it might be easier mechanically
2. When adding an assembly to AssemblyGenerator, have it quietly walk up the dependency tree of that assembly and add its dependencies.

What if you get a message that the bus doesn't recognize? What do you do with it? Ignore it, log an error, send an "I don't understand this" message?

All TBD. We didn't do much with this in fubu, so there isn't a huge amount of prior art here.

A separate package to provide a diagnostic view into the configuration of the running application. Should visualize the messages that are handled and how, the routing rules, the subscriptions, and provide some information about the messages that have been handled. I see this as mostly a development tool, but it could be more than that.

I've been thinking that it'd be a small web app with a React/Redux frontend backed by websockets to a small embedded Kestrel application.

There's a few questions. Should this be specific to the service bus? Do we have a pluggable model for extensions to be able to add their own diagnostic views like we did w/ fubumvc?

I'm also thinking that if we go down the websockets path that I could extract the WebSockets/Kestrel handling mechanism in the Storyteller UI as a separate Nuget and use it here.

Right now, we're completely dependent upon shared DTO classes being sent from place to place. Longer term we'll almost certainly need a mechanism to translate their message type in the Envelope headers to what type the current app supports. It could be just a mapping of their Type to our Type, or it could be a custom deserializer for just that and some new pluggable interface.

There's logic in Fubu to always convert dates to their UTC representation, but you know that can get missed. Let's use the Offset time instead.

Right now I think it's going to look like fubu's. Probably just bring that over as is.

FubuMVC had a pluggable interface that could be used to apply customizations to each and every Envelope sent to a particular channel. We used this in fubu for backward compatibility with RhinoServiceBus. Not sure this is really useful today, but I'm keeping the issue here until we know that for sure.

    public interface IEnvelopeModifier
    {
        void Modify(Envelope envelope);
    }

The key configuration class in Jasper right now is the JasperRegistry -- which is basically the same thing as FubuRegistry for fubumvc veterans. The basic idea is that you completely define what an application is and all of its configuration in the registry class with a (light) fluent interface that hangs off of the base JasperRegistry. For some background, it's the Object Scoping pattern for shortening a DSL by hanging it off of a base class. Conceptually, this is very similar to the role that the Startup class serves in ASP.Net Core applications.

For the bus applications, it might be a little easier to have a JasperBusRegistry that
adds the channel configuration.

I think the consensus of fubumvc service bus users was that the configuration API was confusing and unclear, so let's start from scratch and reconsider our approach.

Here's my trial balloon for the syntax:

public class MyServiceBusApp : JasperRegistry
{
    public MyServiceBusApp()
    {
        // To set up a queue listener at a named queue.
        // Just like fubu, queues are identified by a Uri
        ListenForMessagesFrom(new Uri("lq.tcp://localhost:2000/incoming"));

        // It would be legal to chain multiple "To()" methods 
        SendMessage<Message1>()
             .To(new Uri("lq.tcp://server1:2100/app1"));

        // we'd have an overload for routing by Func<Type, bool>, and overloads
        // for the common usages like "in this namespace" or "from this assembly"
        SendMessages(type => type.IsInNamespace("MyApp.Messages")
             .To(new Uri("lq.tcp://server1:2100/app1"));
    }


}

There'll have to be additions later for dynamic subscriptions, but one thing at a time.

To the obvious question, how do the Uri's get in there in the first place? For the first go, I say we have users just crudely load a Uri from an expected slot in configuration inside of their JasperRegistry code. This does of course assume that the new configuration model will be built up before the JasperRegistry is bootstrapped.

Another approach down the line after #9, would be:

public class MyChannelSettings
{
    
    public Uri Incoming = new Uri("lq.tcp://localhost:2000/incoming");
    public Uri App1 = new Uri("lq.tcp:/server1:2200/app1");
}

public class MyServiceBusApp : JasperRegistry
{
    public MyServiceBusApp()
    {
        // This would be executed after MyChannelSettings has been
        // read from configuration and had any alterations applied
        Settings.Using<MyChannelSettings>(_ => {
            ListenForMessagesFrom(_.Incoming);

            SendMessage<Message1>()
                .To(_.App1);

        });

    }


}

We'll definitely need some kind of mechanism to extend and expand the code generation. There's an existing class in nascent form called GenerationConfig. Right now I'm thinking that we allow additions to that either globally or by specific feature.

Different "features" will probably need to be able to use a conglomerate GenerationConfig with the general things and the ones specific to that feature.

As part of constructing itself, HandlerSet should tag each chain w/ the generated code

Low allocations FTW!

Same trick as FubuMVC. If any attribute inheriting from [ModifyHandlersAttribute] is on either hte handler type or the method, call it against the chain as it calls the ToHandlerCode()

Might depend on #21 a little bit. I pulled in the HandlerSource almost directly from FubuMVC, but the logic for determining candidate handler call methods is slightly different. At a bare minimum, we need some tests around this functionality just to lock it down.

Just sticking in the TODO so I can ignore this right now

Investigate this later as a minor optimization.

For the service bus, maybe the nested container is still created with the Envelope injected in. Maybe.

For the HTTP side of things, add the HttpContext in. Might enable folks to override the nested container creation so that you can utilize SM for multi-tenancy

Not sure how different this will be than fubu3

Much more detail needed...

It's partially implemented, but the one test fails intermittently. Just say we're starting from scratch on this one.

Down the road, we never used what was in fubu, and I didn't like the way we did it anyway (even though I coded it)

2 scenarios:

1. If a handler chain is completely synchronous, have it return a Task.Completed and bypass the async
2. If a handler chain has only one async frame, it's the last one, and that Frame.CanReturnTask(), return that task.

This time, I want this to be production quality. Use the TPL dataflow instead of homegrown code. Probably still make it a static so it's easily shared between running nodes in testing scenarios.

If the bus is requested to send a message to a previously unknown channel, build a channel on the fly for just that Uri, but do not keep it in memory so it doesn't blow up the memory.

It's pulled over from FubuMVC almost as is, and it's unit tested, but it needs some end to end tests to finish it off.

I think this is going to be much simpler in Core's much better configuration model, but still.

Personally, I think that the way ASP.Net Core uses its IOptions mechanism is bonkers and unnecessarily clumsy. Admittedly, their configuration design is based on the idea that configuration is variable during the lifetime of the application. If we forgo that capability, I think our usage could be much simpler.

The goal of this story is to provide strong typed configuration much the same way we did in FubuMVC. We'd surely want to also support the IOptions<T> pattern that ASP.Net Core supports. The basic idea is that at application bootstrapping time you hydrate all the Settings Type's known upfront and each is registered in the StructureMap container as a singleton. In addition, if a previously unknown Settings type is requested at runtime, we should have a StructureMap policy (like this one from FubuMVC)

Using fubu as a guide, we should support a combination of loading configuration objects from:

• The ASP.Net Core Configuration infrastructure -- but there is an open question about whether that will all be from trying to deserialize a matching configuration section or if we bind key/value
  pairs like we did in FubuMVC. For the sake of environment variables, I'd guess the answer is "both".
• If there is no matching configuration found, just new up requested Settings
• Directly injected or "replaced" objects built into the IoC container
• Applying user specified alteration lambda's registered in the JasperRegistry

This syntax from FubuMVC:

public Task SendPingWithAck(IServiceBus bus)
{
    return bus.SendAndWait(new PingMessage());
}

All you're wanting to do here is to send a message and wait to get a response back that the message was received and processed by the downstream node.

This is going to have some prerequisite work first, but this is the very first big thing we need to be able to support for the MVP.

At a minimum, messages being published through the service bus should be routable via:

• Static channel rules established in a JasperRegistry
• Dynamic subscriptions from #18

We'll need some kind of configuration syntax for the static rules.

The way it worked in FubuMVC was that each channel node had one or more IRoutingRule objects with this little signature:

    public interface IRoutingRule
    {
        bool Matches(Type type);
        string Describe(); // we can just use ToString() here, this was for diagnostics
    }

Any time there was a message to send, fubu runs over each channel that has a rule that matched the message type and sent a copy to that channel. The publishing can be one message to multiple queues.

TBD.

Support loosely coupled request/reply through the service bus.

Here's what it looked like in FubuMVC, and I don't have any different ideas for the syntax:

public async Task RequestReply(IServiceBus bus)
{
    var pong = await bus.Request<PongMessage>(new PingMessage()).ConfigureAwait(false);
}

It's really just Jasper sending a message with some extra header values requesting a reply. The service bus has to track the correlation id's between the messages and route replies to the Task<TResponse> which in fubu, was a TaskCompletionSource<TResponse> that was cached in a singleton somewhere in the app.

There's quite a bit to flesh out here, but my thought is to avoid the polling job thing we used in fubu to clean up hanging/expired listeners and just do some Task.Delay() trickery to do the timeouts and clean up's.

If a method returns a value, like say a handler call uses a cascading message, the method call should create the variable according to the Frame rules.

Use BenchmarkDotNet. At a minimum, let's hit:

• Sending simple messages to one location
• Sending bigger, more complex messages to one location
• Sending simple messages to multiple locations
• Sending bigger, more complex messages to multiple locations
• Request/Reply
• Sending with dynamic subscriptions

We need some way to explicitly set the control channel that is used to send and receive subscription information.

Duh, gotta take .Net objects and ultimately serialize to a byte[]. I've already adapted the work that @JordanZaerr did in fubumvc for some intelligent "content negotiation" logic in the serialization. The test harness isn't working yet, so here's an issue to track that work.

Allows users to:

• Register new types of serializers
• State a preference for which serializers to choose at the app level
• State a preference for which serializers to choose per channel

I want this completely decoupled from the transport layer. Might continue to use LightningDB to store locally, but I'd like an option that sticks them somewhere else.

Support the same functionality as fubu in this regard: https://fubumvc.github.io/documentation/servicebus/cascading/

A lot of configuration and probably even functionality is going to happen on the chain objects themselves, so let's have a mechanism where the Chain is attached to the generated IHandler objects like shown below.

Example needs:

1. Authorization rules
2. Content negotiation rules
3. Exception handling

Now:

    public interface IHandler<TInput>
    {
        Task Handle(TInput input);
    }

Later:

    public interface IHandler<TInput, TChain>
    {
        Task Handle(TInput input);
        TChain Chain {get; }
    }

Check:

• If there is an unknown transport type listed
• If LQ doesn't know about a reply uri

What else?

Not 100% sure that that's possible as I write this, but my reading of the new ValueTask<T> feature makes me think it'd make some elements of Jasper run faster.

Just to make exception messages easy to craft. Show the correlation id? Received from, sent to, what else?

This isn't an immediate need, but we'll eventually want the findThisAssembly() functionality to walk up a stack trace to find the main assembly. Steal the functionality from StructureMap's Netstandard 1.5 implementation?

@JordanZaerr 's favorite issue within FubuMVC today. The configuration for an Uri endpoint will probably be something like: "lq.tcp://localhost:2000/incoming", the underneath the covers it's "corrected" to "lq.tcp://[machine name]:2000/incoming"

Some how or another, we need to make the usage of these two Uri's be effectively interchangeable to make the services easier to use.

I think this is a low priority, but we had it in fubumvc. Not sure it ever got used much. Basically, this is the ability to send a batch of messages as one aggregate message over the wire, and have all of them handled independently when they're received.

Today it's writing everything in one source writer. Later, we'll need to scrape that out

Not sure about this one.

[MaximumAttempts(3)]
[RetryOn(typeof(SqlException)]
[RequeueOn(typeof(SqlException)]
[MoveToErrorQueue(typeof(SqlException))]
[RetryLaterOn(Type, seconds)]

or

[OnError(Type, mode, max attempts, delay)]

// where

* Type is the exception type
* mode = Retry/Requeue/MoveToErrorQueue/RetryLater
* attempts is optional and an int
* delay is optional, and would refer to the number of seconds to wait to retry it

Just a TODO so I can ignore it for the moment

We kind of did this in a hokey, not super effective way with early FubuTransportation. This time, we're pushing much more logic to the individual transports. So, let's:

• Add some kind of property to ChannelNode for "maximum parallel requests". Just steal whatever the variable name from the TPL Dataflow equivalent is here
• The ControlChannel always has to be parallelism == 1, or strictly ordered
• Transports need to use the parallelism numbers

What do we do with failed messages? We just pushed them to a special queue in fubu and...

...didn't really do anything else with it.

Since this was absolutely successful (in moderation at least), we should have the ability for users to plug in policies that apply middleware to matching chains.

For the service bus anyway, I think the policy signature looks like:

public interface IHandlerPolicy
{
     void Apply(HandlerGraph graph); // the collection of HandlerChain's
}

These will be added in JasperRegistry classes something like:

Policies.Add<MyHandlerPolicy>();

Okay, so FubuMVC was awesome at conventions and policies to edit/alter/customize how any given chain worked, but it was much less so about per-chain customization. To alleviate that, how about this convention. If the concrete type for any HandlerCall has this method:

public static void Configure(HandlerChain)
{
     // apply middleware?
    // set up exception handling per chain?
}