Downstream requests aggregation about ocelot HOT 31 CLOSED

@daniellaoding @MykhailoKyrychenko thanks for your interest in the project.

I personally have not worked on this but I am coming to the end of a feature so might be able to take a look.

what should we do when one of downstreams fails?
Allow the user to decide if they want to try and continue or fail immediately? We might be able to write something that partially composes the return object.

how should we merge headers from all the downstreams to single upstream?
That is an interesting problem :) I guess you could again have a config that lets the user decide what takes precedence.

This is a very tricky feature to get right :(

from ocelot.

Im going to pick this up again!

from ocelot.

@MykhailoKyrychenko at the moment I don't have any specific plans to do this. I talked to a friend at work and he suggested this would be a good feature!

You can inject custom functions as middleware at various stages in the pipeline that allow you to do anything but this is probably not ideal!

from ocelot.

@TomPallister thank you for your answer! It looks like the only way to have some complex logic inside of injected middleware is to use Service Locator pattern. The other option is to use MVC, but it is not that good at all, because it will support routing only via attributes, so there is no chance to have everything in configuration file. I am going to check possibility of using middleware injection one more time. Thank you for your answer once again!

from ocelot.

@MykhailoKyrychenko Yeah service locator sucks :( this annoys me that it is the only way you can inject middleware into Ocelot.

Can you explain how you would see this feature working?

From my point of view you would need to describe in the ocelot configuration that you want to call multiple endpoints and map the results from each into some kind of object to return?

e.g.

I have a mobile app client and I want to get a product search results page. In order to get all of this data I need to call the search service to get some information and then we also want something that isn't in the search service say, offers/promotions so we call into the offers service to see if we can show any discounts. This means the mobile client would have to make two calls but we want to only make one.

In this case we want to aggregate the calls and return one object to the mobile client.

I think something like this would do the job http://blog.tamizhvendan.in/blog/2015/12/29/implementing-api-gateway-in-f-number-using-rx-and-suave/

I think the hardest part is making this configurable.

from ocelot.

@TomPallister yes, I have seen that article and I liked it very much. The problem is that F# in our current solution would be yet one more new thing for half of team, so we have to skip it. At least for now.

Regarding of how I see the feature itself:

As for me, aggregation of downstream requests can be freely applied only to GET requests, since in PUT and POST you need to provide some additional logic for how to decouple different parts of POST body between different downstreams. And aggregation of DELETE requests does not sound safe for me.

As for the implementation, you can map upstream as done now in Ocelot, and then just run in parallel downstream requests. Then they can be combined into one single response object under defined in configuration property names. Configuration file would look like example below.

"ReRoutes": [
    {
      "Downstreams": [
        {
          "DownstreamPathTemplate": "/posts/{postId}",
          "DownstreamScheme": "http",
          "DownstreamHost": "jsonplaceholder.typicode.com",
          "DownstreamPort": 80,
          "UpstreamKey": "post"
        },
        {
          "DownstreamPathTemplate": "/posts/{postId}/comments",
          "DownstreamScheme": "http",
          "DownstreamHost": "jsonplaceholder.typicode.com",
          "DownstreamPort": 80,
          "UpstreamKey": "comments"
        }
      ],
      "UpstreamPathTemplate": "api/post/{postId}",
      "UpstreamHttpMethod": "Get",
      "QoSOptions": {
        "ExceptionsAllowedBeforeBreaking": 3,
        "DurationOfBreak": 10,
        "TimeoutValue": 5000
      }
    }
  ] 

from ocelot.

@MykhailoKyrychenko Yep makes sense, I guess we will make this next!

from ocelot.

@TomPallister great news! Do you need any help with that?

from ocelot.

@MykhailoKyrychenko yes, at the moment I'm working on another feature which is quite complex. If you have the time it would be great if you could look at implementing this feature. Dont worry if you dont have time!

from ocelot.

@TomPallister I have been trying to implement this particular feature inside Ocelot for a while now. But I am still not sure about architecture: should it be a separate execution flow starting from point when upstream path was matched or should it be just a unified flow. In case of unified flow all current configurations have to be updated to support multiple downstreams with restriction on config validation level, that only GET really supports multiple downstreams. That might be confusing for the end user.

Also, some features like Load balancing and QoS are pretty complex for this use case as well.

from ocelot.

@MykhailoKyrychenko Yeah its pretty complicated.

I would add a new piece of middleware somewhere that says...OK this person wants to call two different downstream re routes to compose their return object. That would then call into the ocelot stack for each downstream re route in parallel....wait and then compose the return object.

Add the new middleware after DownstreamRouteFinderMiddleware

In the middleware have a branch that says if you just have one re route call the next middleware as normal and return.

If you have more than one re route call each and then compose the object and return it.

I think that might work, though could not say for sure without actually writing the code!

from ocelot.

@MykhailoKyrychenko @TomPallister this feature has any progress ?

from ocelot.

Had draft version implemented as custom fork for internal project. Now, as Ocelot evolved, I'm not even sure if it's possible to simply solve merge conflicts 😞 Even if that won't consume a lot of time, unfortunately, I won't be able to continue on that feature before November.

Also, there are a few moments I'm not sure about:

• what should we do when one of downstreams fails?
• how should we merge headers from all the downstreams to single upstream?

from ocelot.

good idea @TomPallister @MykhailoKyrychenko 。
it will be perfect if it support wrap up 、transfer or format the data of downstream service by a callback function or custom function before return the data to upstream service request ；
we are interesting in this feature ****

from ocelot.

Middleware Analysis

UseDownstreamRouteFinderMiddleware - finds the ReRoute specific to the upstream request...maybe doesnt need to change...

** Could you just add a branching middleware here that calls next for each downstream and then collates everything on the way back up??? **

UseHttpHeadersTransformationMiddleware - does find and replace on headers...would need applying to all downstreams..

UseDownstreamRequestInitialiser - creates initial httprequestmessage object - would need applying to all downstreams..

UseRateLimiting - rate limits downstream requests - would need applying to all downstreams..

UseRequestIdMiddleware - sets request id would need applying to all downstreams..

UseAuthenticationMiddleware - would need applying to all downstreams..

UseClaimsBuilderMiddleware - would need applying to all downstreams..

UseAuthorisationMiddleware - would need applying to all downstreams..

UseHttpRequestHeadersBuilderMiddleware - runs claims to headers logic - would need applying to all downstreams..

UseQueryStringBuilderMiddleware - would need applying to all downstreams..though this might not but could be added later

UseLoadBalancingMiddleware - would need applying to all downstreams..

UseDownstreamUrlCreatorMiddleware - this middleware probably doesnt need to be on its own, its a bit shit could maybe be in UseHttpRequestBuilderMiddleware or UseDownstreamRequestInitialiser? - would need applying to all downstreams..

UseOutputCacheMiddleware - would need applying to all downstreams..

UseHttpRequestBuilderMiddleware - would need applying to all downstreams..

UseHttpRequesterMiddleware - would need applying to all downstreams..

Configuration Options

First just make user set a key on ReRoutes then call them all and aggregate

Tradeoffs - makes the user add a key to ReRoutes they want to aggregate

"AggregateReRoutes" : [
{
"ReRouteKeys": "route1, route2, route3"
},
{
"ReRouteKeys": "route1, route2"
}
]

Or

have a new section for aggregates called Downstreams (dont pay too much attention to data in the json below or names etc)

Tradeoffs - doesnt make the user add a key to ReRoutes they want to aggregate

"ReRoutes": [
{
"Downstreams": [
{
"DownstreamPathTemplate": "/posts/{postId}",
"DownstreamScheme": "http",
"DownstreamHost": "jsonplaceholder.typicode.com",
"DownstreamPort": 80,
"UpstreamKey": "post"
},
{
"DownstreamPathTemplate": "/posts/{postId}/comments",
"DownstreamScheme": "http",
"DownstreamHost": "jsonplaceholder.typicode.com",
"DownstreamPort": 80,
"UpstreamKey": "comments"
}
],
"UpstreamPathTemplate": "api/post/{postId}",
"UpstreamHttpMethod": "Get",
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 10,
"TimeoutValue": 5000
}
}
]

Implementation Ideas

foreach downstream route call a pipeline of some kind async..
the pipeline will need to encapsulate all the middlewares noted above..
could use mediator and send commands to each piece of middleware with the data it needs
could build own pipeline code
prefer something like mediator, is mediator license ok?
could try and hack it into asp.net middleware
foreach downstream call next middleware and somehow set a key for that call only? then store response in httpcontext and pull out with key then aggregate...is this possible
get http responses from pipelines
when all or timed out or errors
aggregate http responses
need to decide what to do with time outs, errors
need to decide what to do with mix of success and either of the above
return one http response

from ocelot.

For anyone interested I think I’m going to leave the asp.net middleware after working out the downstream route. Then multiplex further requests into an ocelot middleware pipeline which will be the same as the asp.net one apart from it will take a different object. Not httpcontext.

For multiplex I will either just use tasks or something like rx.net. I probably will use tasks so I don’t have to take the dependency.

Anyway let’s see!

from ocelot.

@TomPallister This is a good way to support multiple protocols for downstream services

from ocelot.

• Update Middleware so it can multiplex
• Update configuration.json so that users can specify what to aggregate
• Validate configuration.json
• Update internal config
• Aggregate responses
• Tidy up request id stuff, its a mess at the moment on context and repo
• Ensure pipeline is threadsafe
• Ensure pipeline is same performance and old pipeline
• Ensure pipeline is threadsafe for each downstream request in the aggregate including things that happen before the multiplexing

from ocelot.

previous pipeline apache bench
C:\Program Files (x86)\PostgreSQL\EnterpriseDB-ApachePHP\apache\bin> ./ab.exe -n 1000 -c 100 http://localhost:5000/post
s
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software: cloudflare
Server Hostname: localhost
Server Port: 5000

Document Path: /posts
Document Length: 27520 bytes

Concurrency Level: 100
Time taken for tests: 4.343 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 28145872 bytes
HTML transferred: 27520000 bytes
Requests per second: 230.26 [#/sec] (mean)
Time per request: 434.301 [ms] (mean)
Time per request: 4.343 [ms] (mean, across all concurrent requests)
Transfer rate: 6328.84 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.6 1 46
Processing: 74 416 259.5 353 1923
Waiting: 73 412 257.7 350 1922
Total: 75 416 259.5 354 1924

Percentage of the requests served within a certain time (ms)
50% 354
66% 427
75% 477
80% 520
90% 897
95% 971
98% 1188
99% 1230
100% 1924 (longest request)

from ocelot.

new pipeline apache bench
C:\Program Files (x86)\PostgreSQL\EnterpriseDB-ApachePHP\apache\bin> ./ab.exe -n 1000 -c 100 http://lo
s
This is ApacheBench, Version 2.3 <$Revision: 1604373 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests

Server Software: cloudflare
Server Hostname: localhost
Server Port: 5000

Document Path: /posts
Document Length: 27520 bytes

Concurrency Level: 100
Time taken for tests: 3.647 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 28145157 bytes
HTML transferred: 27520000 bytes
Requests per second: 274.19 [#/sec] (mean)
Time per request: 364.706 [ms] (mean)
Time per request: 3.647 [ms] (mean, across all concurrent requests)
Transfer rate: 7536.34 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.6 1 12
Processing: 61 339 195.2 293 1088
Waiting: 58 336 195.1 291 1086
Total: 61 340 195.2 293 1088
WARNING: The median and mean for the initial connection time are not within a normal deviation
These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
50% 293
66% 343
75% 387
80% 431
90% 576
95% 792
98% 996
99% 1063
100% 1088 (longest request)

from ocelot.

Hi - I am trying to use the "Aggregation feature" of Ocelot but its throwing an error -

"ERROR|Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware|DownstreamRouteFinderMiddleware setting pipeline errors. IDownstreamRouteFinder returned Error Code: UnableToFindDownstreamRouteError Message: UnableToFindDownstreamRouteError : OcelotRequestId - not set
2018-03-29 14:04:49.3305||ERROR|Ocelot.Responder.Middleware.ResponderMiddleware|1 pipeline errors found in ResponderMiddleware. Setting error response status code : OcelotRequestId - not set "

Here is the configuration.json

{
"ReRoutes": [
{
"DownstreamPathTemplate": "/api/customers",
"DownstreamScheme": "http",
"DownstreamHost": "localhost",
"DownstreamPort": 9001,
"UpstreamPathTemplate": "/customers",
"UpstreamHttpMethod": [ "Get" ],
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 10,
"TimeoutValue": 5000
},
"Key": "Item1"

},
{
  "DownstreamPathTemplate": "/api/customers/{id}",
  "DownstreamScheme": "http",
  "DownstreamHost": "localhost",
  "DownstreamPort": 9001,
  "UpstreamPathTemplate": "/customers/{id}",
  "UpstreamHttpMethod": [ "Get" ],
  "QoSOptions": {
    "ExceptionsAllowedBeforeBreaking": 3,
    "DurationOfBreak": 10,
    "TimeoutValue": 5000
  },
  "Key": "Item3"
},
{
  "DownstreamPathTemplate": "/api/products",
  "DownstreamScheme": "http",
  "DownstreamPort": 9002,
  "DownstreamHost": "localhost",
  "UpstreamPathTemplate": "/products",
  "UpstreamHttpMethod": [ "Get" ],
  "QoSOptions": {
    "ExceptionsAllowedBeforeBreaking": 3,
    "DurationOfBreak": 10,
    "TimeoutValue": 5000
  },
  "Key": "Item2"
}

],
"Aggregates": [
{
"ReRouteKeys": [
"Item1",
"Item2"
],
"UpstreamHost": "localhost",
"UpstreamPathTemplate": "/aggregated",
"UpstreamHttpMethod": [ "Get" ]
}
],

"GlobalConfiguration": {
"RequestIdKey": "OcRequestId",
"AdministrationPath": "/administration"
}
}

I am trying to aggregate the output of 2 routes identified by keys "Item1" &"Item2".Am I missing something ??I get the above error when access
"http://localhost:9000/aggregated"

However when I use "http://localhost:9000/products" or "http://localhost:9000/customers" I get the desired output.Can somebody help me out on the same.

Awaiting reply.

from ocelot.

@skg170383 thanks for your interest in the project! Can you try removing UpstreamHost from your aggregate config and see if that makes a difference?

from ocelot.

I tried removing UpstreamHost but got the same error.

2018-03-30 12:46:33.6518||ERROR|Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware|DownstreamRouteFinderMiddleware setting pipeline errors. IDownstreamRouteFinder returned Error Code: UnableToFindDownstreamRouteError Message: UnableToFindDownstreamRouteError : OcelotRequestId - not set
2018-03-30 12:46:33.6518||ERROR|Ocelot.Responder.Middleware.ResponderMiddleware|1 pipeline errors found in ResponderMiddleware. Setting error response status code : OcelotRequestId - not set

Can you kindly help as we are evaluating "Ocelot" for our solution architecture

from ocelot.

@skg170383 I think I can see the problem now.

Please try this configuration.json

{
    "ReRoutes": [
        {
            "DownstreamPathTemplate": "/api/customers",
            "UpstreamPathTemplate": "/customers",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 9001
                }
            ],
            "Key": "Item2"
        },
        {
            "DownstreamPathTemplate": "/api/customers/{id}",
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 9001
                }
            ],
            "UpstreamPathTemplate": "/customers/{id}",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "QoSOptions": {
                "ExceptionsAllowedBeforeBreaking": 3,
                "DurationOfBreak": 10,
                "TimeoutValue": 5000
            },
            "Key": "Item3"
        },
        {
            "DownstreamPathTemplate": "/api/products",
            "UpstreamPathTemplate": "/products",
            "UpstreamHttpMethod": [
                "Get"
            ],
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 9002
                }
            ],
            "Key": "Item1"
        }
    ],
    "Aggregates": [
        {
            "ReRouteKeys": [
                "Item1",
                "Item2"
            ],
            "UpstreamPathTemplate": "/aggregated"
        }
    ],
    "GlobalConfiguration": {
        "RequestIdKey": "OcRequestId",
        "AdministrationPath": "/administration"
    }
}

I think it's because you are not using

            "DownstreamHostAndPorts": [
                {
                    "Host": "localhost",
                    "Port": 9002
                }
            ],

This is the way of setting config in Ocelot now! Can I ask where did you find your example?

from ocelot.

Hi Tom-
I tried the exact configuration.json what you provided, but I still get the error. In fact the earlier .json was at least giving individual responses from the routes

For example - http://localhost:9000/customers
http://localhost:9000/products

But now this individual urls do not respond. Have attached log file for your reference.
Kindly help !!Awaiting reply. I have taken the sample app from C# corner

debug-2018-03-31.log
pond-

from ocelot.

@skg170383 mmmm this is very strange that config worked fine for me :( what version of Ocelot are you using?

from ocelot.

I am using 2.0.1.Do you have any suggestion... Kindly help

from ocelot.

@skg170383 ok so that’s probably why the config I suggested didn’t work. Also version 2.0.1 did not support aggregates. Please try the latest version 5.3.0!

from ocelot.

Thanks a lot for all the help !! It works with 5.3.0.
Looking forward to try out other features available.

from ocelot.

@skg170383 no problem, glad it worked! :)

from ocelot.

hi, I have a question about the request to aggregates from Angular CLI.
I set the configuration that I needed for "Aggregates". Normally this Route work on the requested URL but when I try to this on Angular CLI it was returned response "Cors Origin Error". I set the configuration for "Cors Origin Error" and those are work one by one without returned response "Cors Origin Error".

from ocelot.