Code Monkey home page Code Monkey logo

grpcjsontranscoder's Introduction

Grpc-Json Transcoder project

Price version

This is a filter that allows a RESTful JSON API client (Ocelot Gateway) to send requests to .NET Web API (Aggregator Service) over HTTP and get proxied to a gRPC service (on the downstream).

This project is inspired by grpc-gateway which is totally for golang, grpc-dynamic-gateway is for nodejs. And especially, Envoy gRPC-JSON transcoder is the best of transcoding in this area, but it is only on the infrastructure level. You also can use it just like my project used at coolstore-microservices. We might use the approach from Microsoft at GrpcHttpApi, but it is not publish in the official release by Microsoft in the meantime.

gRPC parser borrows the idea from Ocelot.GrpcHttpGateway code-based.

Give a Star!

If you liked GrpcJsonTranscoder project or if it helped you, please give a star ⭐ for this repository. That will not only help strengthen our .NET community but also improve cloud-native apps development skills for .NET developers in around the world. Thank you very much 👍

Check out my blog or say hi on Twitter!

How to run it!

$ docker-compose up

or

$ bash
$ start.sh # I haven't done it yet :p

In the mean time, we open up the visual studio, run multiple projects included OcelotGateway, AggregationRestApi, ProductCatalogGrpcServer and GreatGrpcServer

Test it as below:

# gRPC
$ curl -X GET -H 'content-type: application/grpc' -k http://localhost:5000/say/Bob
$ {"Message":"Hello Bob"}
# gRPC
$ curl -X GET -H 'content-type: application/grpc' -k http://localhost:5000/products
$ {"Products":[{"Id":1,"Name":"product 1","Quantity":52,"Description":"description of product 1"},...]}
# gRPC
$ curl -X POST -H 'content-type: application/grpc' -d '{ "name": "product 1", "quantity": 1, "description": "this is product 1" }' -k http://localhost:5000/products
$ {"Product":{"Id":915,"Name":"product 1 created","Quantity":1,"Description":"this is product 1 created"}}
# REST Api
$ curl -X GET -H 'content-type: application/json' -k http://localhost:5000/weather 
$ [{"date":"2019-08-17T18:34:41.1090164+07:00","temperatureC":-6,"temperatureF":22,"summary":"Sweltering"},{"date":"2019-08-18T18:34:41.1090371+07:00","temperatureC":27,"temperatureF":80,"summary":"Hot"},{"date":"2019-08-19T18:34:41.1090499+07:00","temperatureC":33,"temperatureF":91,"summary":"Balmy"},{"date":"2019-08-20T18:34:41.1090617+07:00","temperatureC":-14,"temperatureF":7,"summary":"Chilly"},{"date":"2019-08-21T18:34:41.1090743+07:00","temperatureC":22,"temperatureF":71,"summary":"Hot"}]

Notes:

  • REST method: content-type: application/json is for REST method on the downstream services.
  • gRPC method: content-type: application/grpc is really important if you call to gRPC endpoint on the downstream services.

How to understand it!

The project aims to .NET community and its ecosystem which leverage the power of Ocelot Gateway which is very powerful in the gateway components were used by various of companies and sample source code when we try to adopt the microservices architecture project.

That's quite simple with only a few steps to make it work :) Create the .NET Core project with Ocelot in place, then put the configuration as below

{
  "ReRoutes": [
    {
      "UpstreamPathTemplate": "/say/{name}",
      "UpstreamHttpMethod": [ "Get" ],
      "DownstreamPathTemplate": "/Greet.Greeter/SayHello",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "127.0.0.1",
          "Port": 5003
        }
      ]
    },
    {
      "UpstreamPathTemplate": "/products",
      "UpstreamHttpMethod": [ "Get" ],
      "DownstreamPathTemplate": "/ProductCatalog.Product/GetProducts",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "127.0.0.1",
          "Port": 5002
        }
      ]
    },
    {
      "UpstreamPathTemplate": "/products",
      "UpstreamHttpMethod": [ "Post" ],
      "DownstreamPathTemplate": "/ProductCatalog.Product/CreateProduct",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "127.0.0.1",
          "Port": 5002
        }
      ]
    },
    {
      "UpstreamPathTemplate": "/weather",
      "UpstreamHttpMethod": [ "Get" ],
      "DownstreamPathTemplate": "/weatherforecast",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5001
        }
      ]
    }
  ],
  "GlobalConfiguration": {
  }
}

Then in code Program.cs, you only put a few line

var configuration = new OcelotPipelineConfiguration
{
    PreQueryStringBuilderMiddleware = async (ctx, next) =>
    {
        await ctx.HandleGrpcRequestAsync(next);
    }
};

app.UseOcelot(configuration).Wait();

Don't believe what I said. Try it!

We haven't tested it with stream and full-duplex transport protocols yet. So we feel free to contribute by the .NET community.

Contributing

  1. Fork it!
  2. Create your feature branch: git checkout -b my-new-feature
  3. Commit your changes: git commit -am 'Add some feature'
  4. Push to the branch: git push origin my-new-feature
  5. Submit a pull request :p

grpcjsontranscoder's People

Contributors

butterngo avatar thangchung avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

grpcjsontranscoder's Issues

Ocelot grpc without SSL

Hi,

I am trying to user GrpcJsonTranscoder to connect an Ocelot Api Gateway to a GRPC server hosted on .net-core but I am unable to do it with or without TLS.

With TLS activated on grpc-server I get:

 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
         at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
         at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)

without TLS enabled:

       ---> System.IO.IOException: The response ended prematurely.
         at System.Net.Http.HttpConnection.FillAsync()
         at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
         at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
         at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
         at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)

How can I set GrpcJsonTranscoder to work without TLS?

I know about this https://docs.microsoft.com/en-us/aspnet/core/grpc/troubleshoot?view=aspnetcore-3.0#call-insecure-grpc-services-with-net-core-client; is there a way to use it?

Example failed to run

Hi anh Thang Chung,
Cảm ơn anh đã phát triển bộ Transcoder này.
Hiện tại, e lấy code về và chạy thử thì gặp lỗi như này:
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 GET http://localhost:5000/say/Bob Ocelot.Errors.Middleware.ExceptionHandlerMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: ocelot pipeline started Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: Upstream url path is /say/Bob Ocelot.DownstreamRouteFinder.Middleware.DownstreamRouteFinderMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: downstream templates are /Greet.Greeter/SayHello Ocelot.RateLimit.Middleware.ClientRateLimitMiddleware:Information: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /Greet.Greeter/SayHello Ocelot.Authentication.Middleware.AuthenticationMiddleware:Information: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: No authentication needed for /say/Bob Ocelot.Authorisation.Middleware.AuthorisationMiddleware:Information: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: /Greet.Greeter/SayHello route does not require user to be authorised Ocelot.DownstreamUrlCreator.Middleware.DownstreamUrlCreatorMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: Downstream url is http://localhost:5003/Greet.Greeter/SayHello Ocelot.Requester.Middleware.HttpRequesterMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: IHttpRequester returned an error, setting pipeline error Ocelot.Requester.Middleware.HttpRequesterMiddleware:Warning: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: Error making http request, exception: **System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: The server returned an invalid or unrecognized response.** at System.Net.Http.HttpConnection.FillAsync() at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Ocelot.Requester.HttpClientHttpRequester.GetResponse(DownstreamContext context)
Ocelot.Responder.Middleware.ResponderMiddleware:Warning: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: Error Code: UnableToCompleteRequestError Message: Error making http request, exception: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.IO.IOException: The server returned an invalid or unrecognized response.
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Ocelot.Requester.HttpClientHttpRequester.GetResponse(DownstreamContext context) errors found in ResponderMiddleware. Setting error response for request path:/say/Bob, request method: GET Ocelot.Errors.Middleware.ExceptionHandlerMiddleware:Debug: requestId: 0HLPSQDD4RDKQ:00000001, previousRequestId: no previous request id, message: ocelot pipeline finished
Anh có thể kiểm tra giúp e xem tại sao không.
Cảm ơn anh

Could not merge query string and form data in PUT/PATCH request

If we have the PUT request just like

http://localhost:5000/api/products/5D18B427-FF94-4D34-9C5A-D5312E407EB5

and form-data

{
	"name": "sample product 123",
	"desc": "This is sample project 12345",
	"price": "1200",
	"imageUrl": "http://samplepicture.img"
}

We would like to have a JSON object in the downstream service like

{
	"id": "5D18B427-FF94-4D34-9C5A-D5312E407EB5",
	"name": "sample product 123",
	"desc": "This is sample project 12345",
	"price": "1200",
	"imageUrl": "http://samplepicture.img"
}

Then, we always get the object with id is null

Solution: we would like to merge the query string and form data in PUT/PATCH requests

And, also Delete cannot work well becasue it always wants us to put into form-data. We would like to pass like DELETE http://localhost:5000/api/products/5D18B427-FF94-4D34-9C5A-D5312E407EB5

#ref vietnam-devs/coolstore-microservices#48

Usage of https protocol

Really appreciate for your efforts but i have few clarification to confirm they are as below.

  1. While communicating from ocelot gateway to grpc service (microservice) the protocol should be used is https (grpc is designed to use for https) but the protocol that you have mentioned is http in configuration file of ocelot gateway. However we can use http with grpc but the features like encryption, compression will not work.
  2. The client side auto generated code for the grpc which is based on proto file is the one which does the communication by using the features of grpc how are you handling all those features.

If possible can you please put a video of the code functionality in detail so that it will be used for everyone.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.