yarpc / yarpc-go Goto Github PK
View Code? Open in Web Editor NEWA message passing platform for Go
License: MIT License
A message passing platform for Go
License: MIT License
We don't want some of the primary use-cases to cause users to reach down into the transport
package, lets move headers up as a top-level shape instead.
Instead of by hand as in #30
Blocked on thriftrw/thriftrw-go#61
Right now we have something like:
import "github.com/yarpc/yarpc-go"
rpc := yarpc.New()
Which feels wrong because yarpc
doesnt match the package name, yarpc-go
.
I suggest moving all go code into a package yarpc
like so:
import "github.com/yarpc/yarpc-go/yarpc"
rpc := yarpc.New()
The import paths aren't the prettiest, but I think it's a good tradeoff.
@yarpc/golang
You'll also need to switch XLANG_DIMENSION_*
to CROSSDOCK_AXIS_*
.
I plan on deleting yarpc/xlang
out of Docker Hub tomorrow.
Currently the request meta struct is just named Request
- this is bound to cause confusion with our users. Instead of:
res, rbody, err := service.Call(req, body)
Let's do something like:
respMeta, respBody, err := service.Call(reqMeta, reqBody)
It's a bit more verbose, but really nails home that the request is "exploded" because of Go's type system.
cc @yarpc/golang
I've struggled with the sink
terminology in various Crossdock behaviors - for examlpe, https://github.com/yarpc/yarpc-go/pull/147/files#diff-b84f2b2ac8e475a9c0fb53615befc070R47
@kriskowal mentioned the other day that we should piggyback off of Go's testing.T
naming - that's probably a wise idea.
func Run(t crossdock.T) {
behavior := t.Behavior
transport := t.Param("transport")
encoding := t.Param("encoding")
t.NoError(err, "something went wrong")
t.AssertEqual("bob", "tod")
if !t.ChecksTrue(...) {
...
}
}
cc @yarpc/golang
Right now, we're letting panics bubble up to the HTTP/TChannel server. We should catch them and translate them to unexpected errors.
We'll want to log the stack trace in case of panics.
parent: https://github.com/yarpc/yarpc/issues/17
TODO:
Currently, we have 3 different ReqMetas:
raw.ReqMeta
json.ReqMeta
thrift.ReqMeta
The only difference between the 3 is that thrift.ReqMeta
does not have a Procedure
field, since we determine the procedure name based on the idl method they are calling. We chose this path because, in theory, each encoding might have different request metadata. This also makes it really clear that you don't need to set Procedure
when making Thrift calls.
I'd like to reconsider that position. Maintaining a separate ReqMeta
object per encoding adds mental overhead, makes it a bit awkward to add additional encodings, and is not the most obvious experience. We've received feedback that this is a bit surprising.
Instead, I'd like to see a single top-level yarpc.ReqMeta
and yarpc.ResMeta
structs:
type ReqMeta struct {
Context context.Context
Procedure string
Headers Headers
TTL time.Duration
}
type ResMeta struct {
Headers Headers
}
This produces an experience like so:
// raw
resBody, resMeta, err := rawClient.Call(
yarpc.ReqMeta{
Procedure: "yolo-raw",
Context: context.Background(),
Headers: yarpc.Headers{"bob": "ishome"},
TTL: time.Second,
},
[]bytes{"raw bytes yall"},
)
// thrift
// note we don't provide a Procedure
// that will get written (or overwritten) by the encoding,
// you could even imagine allowing someone to override at the call-site if provided
resBody, resMeta, err := userServiceClient.Search(
yarpc.ReqMeta{
Context: context.Background(),
Headers: yarpc.Headers{"bob": "ishome"},
TTL: time.Second,
},
userservice.Filter{FirstName: "bob"},
userservice.LastUUID("..."),
)
// protobufs (feels same as thrift, except only 1 respBody struct)
resBody, resMeta, err := emailServiceClient.Send(
yarpc.ReqMeta{
Context: context.Background(),
Headers: yarpc.Headers{"bob": "ishome"},
TTL: time.Second,
},
emailservice.To{Email: "[email protected]", From: "Your Friends"},
)
ReqMeta
, but has varying request bodiesResMeta
, but has varying response bodiesI think that would be the least confusing considering this is a lib and not an app - @abhinav, whatchu think?
Instead of provided by hand as in #30
Creating this issue to discuss how we want application middleware to look.
Something that came up during the in-person discussion: Thrift middleware could have a generic request body shape with an interface similar to the following:
type ReqBody interface {
GetParam(name string) (interface{}, ok bool)
SetParam(name string, value interface{}) error
Params() []string
}
CC @yarpc/yarpc @shawnburke
We should add a function WithoutBaggage
that returns a copy of the context with the given baggage removed (if present).
ctx = context.WithoutBaggage(ctx, "foo")
This may be difficult/impossible with opentracing integration, though.
Go has a really different register story - work with the other langs to determine if we want this across the board.
We should figure out if we need a function that allows users access to all baggage added to the context.
Couple things to keep in mind:
map[string]string
because the keys are case insensitive (canonicalized to lower case), which is a detail the user is not aware of. We would need either a custom type with accessor functions (Get(key)
), or a CanonicalizeBaggageKey(key)
function.Deepends on #172
We could have the low-level handler provide a ResponseWriter (similar to http's ResponseWriter) which will let us skip the io.Copy
we have to do from the response Reader
to the http.ResponseWriter
.
We should blowup if a timeout hasn't been set on a context, like tchannel currently does:
tchannel.SystemError{code:0x6, msg:"timeout required", wrapped:error(nil)}
Instead of leaking the TChannel error, we should detect this in a single place for all transports and emit a specific YARPC error.
Instead of sleeping as we did in #30
Users shouldn't have to use transport.Headers
.
Instead of taking a map[string]Outbound
, lets have the rpc object take only 1 outbound (which can be used to make custom outbound behavior).
This way we can create ship a ServiceMapOutbound(map[string]Outbound)
which has the same behavior.
Per the Thrift over TChannel Protocol doc -
In case of failure, the Response Code (code:1) must be set to 0x01.
Instead, we're setting 0x00
-
$ sudo tcap -i lo0 -i en0 -p 8080 -p 8082
...
ts=1464736322.907 session=4 127.0.0.1:64892 <-- 127.0.0.1:8082 frame=2 type=0x04 Ok
CALL RESPONSE id=0x0002 (2) flags=0x00
headers
as: thrift
tracing: spanid=0,0 parentid=0,0 traceid=0,0 flags=0x00
args[0]
00: empty
args[1]
00: 0000 ..
args[2]
00: 0c00 0108 0001 0000 03e9 0b00 0200 0000 ................
10: 0858 6365 7074 696f 6e00 00 .Xception..
arg3 as thrift
{ '1':
{ '1': 1001,
'2': 'Xception' } }
This is blocking #162's TestException
test since the error is being set to nil
instead of gauntlet_apache.Xception
.
cc @yarpc/golang
I was able to get a panic: runtime error: invalid memory address or nil pointer dereference [recovered]
by making a raw call without assigning a Context:
to reqMeta
Use content-types as specified in https://github.com/yarpc/yarpc/blob/master/http.md
request.TTL
is set
It's possible that we'll need to be able to apply transport interceptors on a per-handler basis - for example, protecting a subset of handlers with a certain auth interceptor:
rpc.AddInterceptor([myService.ThisProcedure, myService.ThatProcedure], SuperStrongAuthInterceptor{})
Brought up by @shawnburke
@yarpc/golang thoughts? Is this the right path or is there a better way to think about this?
The TChannel inbound won't write the application headers to arg2 until Write()
is called. So if you have an empty body and don't call Write()
the headers won't get written either.
https://github.com/yarpc/yarpc-go/blob/master/transport/tchannel/handler.go#L185
Right now, users could use these from the transport
package if they wanted. Do we want to disallow that?
Worth noting that moving to an internal package isn't as straightforward because the corresponding types for these functions implement package-local functions of the BadRequest
and UnexpectedError
interfaces.
Let's try this out:
func (h Handler) Get(context Context, request Request) response, context, err
Will look more like node, python, & java.
For raw, the request and response body is always []byte
so we don't need to explode them into {Req,Res}{Meta,Body}
like the other encodings.
Having a separate TTL concept is a bit surprising:
yarpc.ReqMeta(
Context: context.WithTimeout(time.Second),
TTL: time.Second,
...
)
Instead, lets start with just the context timeout.
cc @yarpc/golang
Instead of what we're doing in #30
We should add the ability to log to crossdock.T
similar to testing.T
. For the normal case, that would just log to stdout, but for the crossdock test, we can wrap the testing.T
into a crossdock.T
, and log to the test object. This will make the tests silent except in case of failure or when we use -v
.
Right now, the user has to construct the tchannel.Channel
manually and thread it down to the inbound and outbound transports. There's some duplication of information there too (the caller name). It would be nice if we could provide an API that would take care of constructing the Channel for the user and making the TChannel inbound and outbounds share it. It would need to pick up the service name from the channel.
All outgoing requests should be validated by the system.
Let's change the terminology of Scheme
to Encoding
- @kriskowal used this terminology in the Open Tracing meeting the other day, and everybody knew exactly what he meant. That wouldn't have been the case for Scheme
.
Change:
The transport level APIs need to be wrapped into a nicer API similar to JSON/Thrift encodings for Raw.
I would expect to, from a pure-tch client, be able to make a raw request with no headers supplied, []byte{}
, like so:
arg2, arg3, _, err := raw.Call(
ctx,
ch,
ch.PeerInfo().HostPort,
ch.ServiceName(),
"echo",
[]byte{}, // headers here
[]byte("hello"),
)
Instead, I get the following error:
tchannel.SystemError{code:0x6, msg:"BadRequest: failed to decode \"raw\" request headers for procedure \"echo\" of service \"holler\" from caller \"holler\": EOF", wrapped:error(nil)}
Not a list of outbounds
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.