Comments (9)
Bandit doesn't support sending headers today (though it supports receiving them, and just throws them on the floor). However, I've always planned on revisiting the case in #535 in the future, and so Bandit has been designed to ensure that adding trailer support in the future is near trivial. Cowboy also supports trailers today, so it would be a straightforward addition to plug.
I agree that complete_chunked/2
is probably the best home for this; it also ties off a loose end on the chunked encoding API. I'm not fussed about the name, but I do think it should semantically the same API call that closes off a chunked response.
TBH I'm surprised that a need for this has never come up before, given Elixir's applicability in streaming scenarios where trailers are useful (one of the few places they are!). I think this is probably something worth pursuing, though I wasn't personally planning on tackling it until after Bandit 0.8 (likely early 2023).
A rough approach would be:
- Define
Plug.Conn.Adapter.complete_chunked/?
callback (I'm unsure of arity) - Define
Plug.Conn.complete_chunked/2
function in terms of the adapter - Implement this on top of Cowboy (should be straightforward)
- Implement trailer sending in Bandit for HTTP/1 & HTTP/2. This should also be straightforward, though it will be more involved than Cowboy since the support isn't there already
from plug.
This was first mentioned in #535
I believe trailers can be sent in chunked responses for HTTP/1.1 (https://www.rfc-editor.org/rfc/rfc7230#section-4.1.2)
These are also supported in HTTP/2 https://www.rfc-editor.org/rfc/rfc7540#section-8.1
I'm not sure what the API should look like if we support this. Given that it is only applicable to streaming responses, perhaps something like the proposed complete_chunked/2
from the issue would work. Maybe a more explicit function name like chunk_trailers/2
?
@mtrudel how does this apply to Bandit?
from plug.
Do you have examples of where trailers are useful? Last time I checked they were not really recommended for HTTP clients because not all proxies implemented them properly which led to them being dropped.
from plug.
@josevalim An example for trailers would be gRPC. In the official HTTP2 specification for gRPC you can see that response trailers are used to signal the response status code and error reason when applicable — see the Responses section.
However, in that same specification in the Requests section they mention a special request header:
- TE → "te" "trailers" # Used to detect incompatible proxies
Which, according to the following message from the gRPC mailing list:
Eric Anderson Jul 13, 2019, 12:56:34 AM
See https://tools.ietf.org/html/rfc7230#section-4.3
Proxy would mainly be an L7 reverse proxy, like, say Nginx. When HTTP reverse proxies don't support trailers, they are supposed to remove "TE: trailers" from the request. If "TE: trailers" makes it from the client to the server it provides stronger confidence that trailers sent from the server to the client will actually arrive. gRPC depends on the trailers, and doesn't operate when they are dropped. See also the last paragraph of section 4.1.2: https://tools.ietf.org/html/rfc7230#section-4.1.2
means that such a header is used to indicate whether all downstream clients accept trailers. So then an incompatible proxy would remove the header to indicate it does not support trailers. And whenever such an "incompatible" request is received by the gRPC server, it must be rejected because the protocol relies on HTTP 2.0 and requires response trailers.
(Side note: the elixir-grpc library calls :cowby_req.stream_trailers/2
the same way I explained above — see here.)
I should be able to get back to this and implement the suggested functionality in the upcoming weeks. Thank you all for taking the time to look into this. 🙂
from plug.
This work never came up from any personal need (though I've used them once in the past and they've always been a nice little bit of obscure trivia I've held on to). IIRC I'd stumbled over it when implementing Bandit's HTTP/2 support, and in furtherance of Bandit's goal to codify as little policy as possible I'd wanted to see if there was an appetite to bring them into the Plug API.
Honestly though, it does seem like a curious omission, given that the places where trailers are useful (providing trailing checksums and other such 'after the fact' data to streaming (usually chunked) responses) are exactly the sorts of places where Elixir's strengths are most apparent.
I still think the approach I suggest above stands, and I'm happy to work it up 'soon' (sometime after April, as I've got my hands full with Bandit work until ElixirConf). It would both tie off @satom99's ask, and also the naggling inconsistency in the chunked API I'd noted in #535.
If there's consensus for this (and nobody else wants to pick it up) I'm happy to schedule the work right after ElixirConf EU.
from plug.
Could Plug realistically be used for gRPC if we add trailing commas?
from plug.
Could Plug realistically be used for gRPC if we add trailing commas?
Not in a very comprehensive or ergonomic sense, no. There has been some previous discussion of this here and here. The short version of it is that a comprehensive implementation requires more of a rich lifecycle than an in-process c:Plug.call/2
call. It's come up quite a bit over the past year though; may be worth exploring how to extend Plug (or maybe use our new upgrade support to punt to a new library).
from plug.
With trailer support you could kinda-sorta do part of the spec with Plug as it exists today, but it's pretty awkward.
from plug.
Alright, so I think I would wait for a use case outside of gRPC, given that is not suitable for Plug anyway. Again, I am fine with having the feature, but I haven't heard about use cases for Plug yet. :) Thank you!
from plug.
Related Issues (20)
- Interest in an extension for Plug.Parsers.MULTIPART for FormData? HOT 2
- Mismatch spec for `Plug.Conn.Query.decode` HOT 1
- Plug.RequestId: put the `request_id` in `:private` or `:assigns`? HOT 1
- 1.15 changes decoding behaviour of `[]` HOT 5
- Request: Support MFA tuple for Plug.Session opts HOT 4
- Plug.Upload's random-tempdir generation is not very random HOT 1
- `strip_spaces` fails on `nil` since 1.15.0 HOT 1
- Plug.Conn.Query has changed the way it decodes parameters from 1.14 to 1.15 without warning HOT 1
- More flexible dynamic routes with `:` parameters HOT 2
- Compatibility issue with plug_crypto 2.0.0 HOT 3
- Error: There isn't enough space to open the popup HOT 1
- Docs inconsistency between Plug moduledoc and Plug.run HOT 1
- Plug.Upload: demonitor previous owner when giving away HOT 1
- Compilation issues on Erlang 26.x HOT 3
- html array items are dropped. HOT 1
- Plug.Conn is not closed when user closes the browser's tab (SSE) HOT 2
- setting max_age to nil in put_resp_cookie/4 causes crash HOT 1
- Sending a zip created on-the-fly via Plug.Conn.chunk fails after around 200Mb HOT 8
- Performance Regression in Plug.Conn.Cookies.decode/1 after Upgrading Elixir and Erlang HOT 3
- Plug.Router macros and functional plugs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from plug.