Code Monkey home page Code Monkey logo

grpc / grpc Goto Github PK

View Code? Open in Web Editor NEW
40.8K 40.8K 10.4K 501.66 MB

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

Home Page: https://grpc.io

License: Apache License 2.0

C++ 71.83% C 4.35% C# 0.51% JavaScript 0.02% Objective-C 1.79% PHP 1.63% Python 12.76% Shell 2.76% Ruby 2.21% Batchfile 0.16% Swift 0.06% CMake 0.11% Objective-C++ 0.89% Go 0.12% Java 0.05% Dockerfile 0.69% HTML 0.01% XSLT 0.03% CSS 0.01% PowerShell 0.02%

grpc's Introduction

gRPC โ€“ An RPC library and framework

gRPC is a modern, open source, high-performance remote procedure call (RPC) framework that can run anywhere. gRPC enables client and server applications to communicate transparently, and simplifies the building of connected systems.

Homepage: grpc.io
Mailing List: [email protected]

Join the chat at https://gitter.im/grpc/grpc

To start using gRPC

To maximize usability, gRPC supports the standard method for adding dependencies to a user's chosen language (if there is one). In most languages, the gRPC runtime comes as a package available in a user's language package manager.

For instructions on how to use the language-specific gRPC runtime for a project, please refer to these documents

  • C++: follow the instructions under the src/cpp directory
  • C#/.NET: NuGet packages Grpc.Net.Client, Grpc.AspNetCore.Server
  • Dart: pub package grpc
  • Go: go get google.golang.org/grpc
  • Java: Use JARs from Maven Central Repository
  • Kotlin: Use JARs from Maven Central Repository
  • Node: npm install @grpc/grpc-js
  • Objective-C: Add gRPC-ProtoRPC dependency to podspec
  • PHP: pecl install grpc
  • Python: pip install grpcio
  • Ruby: gem install grpc
  • WebJS: follow the grpc-web instructions

Per-language quickstart guides and tutorials can be found in the documentation section on the grpc.io website. Code examples are available in the examples directory.

Precompiled bleeding-edge package builds of gRPC master branch's HEAD are uploaded daily to packages.grpc.io.

To start developing gRPC

Contributions are welcome!

Please read How to contribute which will guide you through the entire workflow of how to build the source code, how to run the tests, and how to contribute changes to the gRPC codebase. The "How to contribute" document also contains info on how the contribution process works and contains best practices for creating contributions.

Troubleshooting

Sometimes things go wrong. Please check out the Troubleshooting guide if you are experiencing issues with gRPC.

Performance

See the Performance dashboard for performance numbers of master branch daily builds.

Concepts

See gRPC Concepts

About This Repository

This repository contains source code for gRPC libraries implemented in multiple languages written on top of a shared C++ core library src/core.

Libraries in different languages may be in various states of development. We are seeking contributions for all of these libraries:

Language Source
Shared C++ [core library] src/core
C++ src/cpp
Ruby src/ruby
Python src/python
PHP src/php
C# (core library based) src/csharp
Objective-C src/objective-c
Language Source repo
Java grpc-java
Kotlin grpc-kotlin
Go grpc-go
NodeJS grpc-node
WebJS grpc-web
Dart grpc-dart
.NET (pure C# impl.) grpc-dotnet
Swift grpc-swift

grpc's People

Contributors

apolcyn avatar aspirinsjl avatar ctiller avatar dgquintas avatar donnadionne avatar drfloob avatar ericgribkoff avatar gnossen avatar jboeuf avatar jcanizales avatar jtattermusch avatar karthikravis avatar kpayson64 avatar lidizheng avatar markdroth avatar matt-kwong avatar mehrdada avatar murgatroid99 avatar muxi avatar nathanielmanistaatgoogle avatar ncteisen avatar nicolasnoble avatar sreecha avatar stanley-cheung avatar tbetbetbe avatar veblush avatar vjpai avatar y-zeng avatar yang-g avatar yashykt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

grpc's Issues

win32 file suffix is confusing

Windows specific files (for both 64bit and 32bit) have _win32 suffix and are using GPR_WIN32 header guard.
Change this to _windows file suffix and GPR_WINDOWS header.

Move to grpc_start_invoke

On the new_invoke_api branch, I've pushed the latest C API changes. Please update the wrapping there, and we'll push it in one batch once all the wrappings are fixed.

Libraries are stripped in place before being installed.

If you run make && make install, the libraries will be stripped of their debugging symbols in the libs directory. This makes in-place debugging hard.

The "install" target should depend on strip to copy the files to a strip directory first.

Unit testing framework for channel filters

  1. have the fixture supply the client/server cq_verifier
  2. add something to the cq verifier to add metadata checks to particular events when they actually come in

We want to be able to configure a cq verifier to be able to say:
'whenever I get one of client_metadata_read, server_rpc_new, or finished, I expect these pieces of metadata to be present'

And we want the fixture to be able to configure this - for verifying oauth code.

Bind server to port 0 in tests

This depends on a change in the core such that on binding 0, the server returns the actual port address that it bound

Optimize Completion Queue

This is a catch all to gather thoughts on a more optimal completion queue implementation (the current implementation is very placeholder).

For each reported event we (minimally):

  1. allocate an event structure
  2. (atomically) increment and decrement a reference count
  3. fill in a data structure
  4. run an indirect call
  5. lock the completion queue mutex
  6. broadcast on the completion queue condition variable

A simple first pass on this would be to change 6. to a signal to avoid a thundering herd, and find opportunities to inline the allocations in 1. to within the call object (i.e. we know that in the common case we'll have at most one read and one write event in flight, and we know we'll need one finish event).

It may also be worth merging locks between the completion queue and the read lock on the call object. This would introduce the possibility of slightly higher contention, but my gut feel (needs data) is that the work performed under the read lock is minimal enough that merging it with the cq lock would result in a net gain.

A larger problem is the thread hops that can be incurred getting work from the network threads to the application. This will be especially important to solve for our lower latency transports. Is there a design that can merge polling and completion queue waiting in an elegant way? My first thought:

  • create completion queues from channels
  • restrict channels to communicate over only one transport type (or transports with equivalent polling mechanisms at best)
  • have cq_next() on an empty queue start polling - when we finish the poll we run callbacks and then re-examine the queue
  • this may need a mechanism to kick other pollers out if we end up with multiple pollers

nptl failure while running ruby tests on a docker image

The ruby unit tests fail when run on a docker image with an error 'ruby: __pthread_mutex_cond_lock_adjust...'
This is happening at the c-layer, and does not appear to be in any code directly wrapped by the ruby.

The failure can be repeated outside of test code by attempting to connect to non-existent addresses.
The ruby tests do this is some places, which is probably why they fail in this way.

Kick sharding is unsafe

iomgr currently uses a sharding mechanism where multiple pollsets may share kick-pipes.

This is unsafe: consider two threads with two separate pollsets sharing a kick pipe. The first thread is polling on pollset A. The second thread is about to poll on pollset B. A third thread tries to kick pollset B. That wakes up the first thread, which drains the pipe. Now thread 2 enters poll and never sees the wakeup.

It is possible to avoid this in principle by ensuring only the target of the wakeup does the drain, but we're probably better off just freelisting and dynamically assigning kick pipes to begin with.

Move to grpc_start_invoke

On the new_invoke_api branch, I've pushed the latest C API changes. Please update the wrapping there, and we'll push it in one batch once all the wrappings are fixed.

Make gRPC C Core work on Windows

High level task to compile and test grpc library on Windows.

Currently the biggest blocker is the layer that works with sockets (needs to be rewritten to work on Windows).

consolidate the base64 implementation

We need base64 for both auth/security and chttp2 with different options. However the chttp2 code base currently does not support decoding (but this is coming soon) and not the options needed for encoding.

The plan is as follows.
1 ctiller@ submits the base64 decode in chttp2
2 jboeuf@ moves it to support/base64 and adds the options needed for security/auth and adapts both the chttp2 and auth code.

Craig, your turn!

Prototype technical solution wrapping gRPC C Core library in C#.

Create some very simple prototypes that demonstrate feasibility of using P/Invoke (or other applicable technology).
The windows grpc library is not ready yet, but I can start experiments with Mono (which supports .NET 4.5 which we are likely to target and also supports P/Invoke).

Move to grpc_start_invoke

On the new_invoke_api branch, I've pushed the latest C API changes. Please update the wrapping there, and we'll push it in one batch once all the wrappings are fixed.

Afford insight into how many RPCs are still being processed after server_shutdown has been called

Nathaniel:
After calling server_shutdown which stops a server from accepting new RPCs and connections, wanting to know how many RPCs the server is still processing and wanting to be alerted when all RPCs are completed are reasonable requests for code using the GRPC C core to want to make.

If we decide not to do this, they could count how many GRPC_SERVER_RPC_NEW events they've accepted against how many GRPC_FINISHED events they've processed.

Craig:
The count of inflight requests is something I'd rather not expose: anything relying on it is inherently racy.

I propose changing grpc_shutdown to:
/* Begin shutting down a server.
After completion of this call, no new calls or connections will be admitted.
Existing calls will be allowed to complete.
When all calls have completed, emit a GRPC_SHUTDOWN_COMPLETE event with tag 'tag'. */
void grpc_server_shutdown(grpc_server *server, void *tag);

Improving server signal handling

  • ruby can only receive signals on the main interpreter thread, and can't receive signals when the ruby GIL is dropped
  • by default rpc_server currently runs on the current thread, which will sometimes be the main thread
  • update rpc_server so that it runs on a separate thread by default, and has a class method that should be called by the main thread that installs signal handler and waits for interrupts

run_tests.py runs too many tests

The run_tests.py script's CLanguage should load build.json and parse the list of the tests to run.

In build.json, the array targets can have objects with the following properties:

"c++" set to true or false; if not set, default to false.
"run" set to true or false; if not set, default to true.

So, the code should select the appropriate tests to run, based on the language tag, and based on the fact it's a test that needs to be run.

Note that we will probably need to extend the build.json file to include the notion of complex tests - aka, tests that run off a shell script or something similar, and that execute several binaries with arguments.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.