nxtedition / node-http2-proxy Goto Github PK
View Code? Open in Web Editor NEWA simple http/2 & http/1.1 to http/1.1 spec compliant proxy helper for Node.
License: MIT License
A simple http/2 & http/1.1 to http/1.1 spec compliant proxy helper for Node.
License: MIT License
I'm trying to forward a request and inject an API key into the request, so that the client doesn't have to know / expose the key.
I was looking at the example where the request is modified in onReq, however I am running into a few issues.
When calling the proxy code below on node (v8.7) it crashes with the message:
_stream_readable.js:583
dest.on('unpipe', onunpipe);
^
TypeError: dest.on is not a function
at IncomingMessage.Readable.pipe (_stream_readable.js:583:8)
at proxy (/Users/me/myproject/node_modules/http2-proxy/index.js:136:6)
Also, according to Typescript res.headers
is not available...
import * as http from "http";
import * as proxy from "http2-proxy";
const server = http.createServer();
server.listen(8008);
server.on("request", async (req, res) => {
req.params.appId = "234253";
req.params.appKey = "asdfsadlkajfglkdjgslkfdjgfd";
try {
await proxy.web(req, res, {
hostname: "https://api.flightstats.com",
onReq: (req, res) => {
// @ts-ignore
res.headers["x-forwarded-host"] = req.headers.host!;
return true;
}
});
} catch (err) {
console.error("Failed to proxy:", err.stack);
}
});
Also I would like to do this in an existing express app. Is there anything keeping me from combining http2-proxy with express? Are there any examples available maybe?
It seems like this sentence from readme is not correct
path: Target pathname. Defaults to req.originalUrl || req.url.
as you always override path with ureq.path = path
.
It should probably be something like if (path !== undefined) ureq.path = path
add typescript support
Is there a way to pass secrure: false or something equivalent like on webpack proxing?
or
disable ssl verification?
I am connecting to https://website.se but I need to pass secrure false since our certificates are not valid and all is in our local network.
Config:
{
src: '/api/.*',
dest: (req, res) => {
console.log('URL? ', req)
req.url = req.url.replace(/^\/api/, '');
console.log('URL? ', req.url)
return proxy.web(req, res, {
hostname: 'some.url.com',
protocol: 'https'
})
},
},
],
I get this error with the following
(node:90948) UnhandledPromiseRejectionWarning: Error: unable to verify the first certificate
at TLSSocket.onConnectSecure (_tls_wrap.js:1497:34)
at TLSSocket.emit (events.js:315:20)
at TLSSocket._finishInit (_tls_wrap.js:932:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:706:12)
(Use `node --trace-warnings ...` to show where the warning was created)
Re-implement closure pooling to reduce memory pressure and help the optimizer.
I'm using your proxy for a Gitlab-Server.
If i try: git clone ......
Iโm getting this:
remote: Counting objects: 3857, done.
remote: Compressing objects: 100% (99/99), done.
error: RPC failed; curl 56 GnuTLS recv error (-110): The TLS connection was non-properly terminated.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
My Git version is: 2.15.1
On node-http-proxy
there is no issue.
I tried to add helmet to my onRes()
handler as described in the docs, but every request results in the following error:
Error: It appears you have done something like `app.use(helmet)`, but it should be `app.use(helmet())`.
at helmet (/usr/src/app/node_modules/helmet/index.js:16:11)
at onRes (/usr/src/app/src/reverse-proxy.js:108:28)
at ClientRequest.onProxyResponse (/usr/src/app/node_modules/http2-proxy/index.js:239:25)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at ClientRequest.emit (/usr/src/app/node_modules/raven/lib/instrumentation/http.js:46:23)
at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:551:21)
at HTTPParser.parserOnHeadersComplete (_http_common.js:117:23)
at Socket.socketOnData (_http_client.js:440:20)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at Socket.Readable.push (_stream_readable.js:208:10)
at TCP.onread (net.js:594:20)
Adding "http2 to http2" proxy functions for proxy servers is now much easier.
The following modules that support http2 can be used.
https://github.com/hisco/http2-client
https://github.com/spdy-http2/node-spdy
Add support for native HTTP2 (i.e. no compat) API for lower overhead.
Using [email protected] to test proxies.
My sample code:
const http2 = require('http2')
const proxy = require('http2-proxy')
const fs = require('fs')
const server = http2.createSecureServer ({ allowHTTP1: true, cert: fs.readFileSync('self.pem'), key: fs.readFileSync('self.pem') })
server.listen(8083)
server.on('request', (req, res) => {
console.error(`request here ${req.url}`)
proxy.web(req, res, {
hostname: 'localhost',
port: 8000
}, null)
})
server.on('upgrade', (req, socket, head) => {
console.error(`upgrade here ${req.url}`)
proxy.ws(req, socket, head, {
hostname: 'localhost',
port: 5900,
}, (err, req, socket, head) => {
console.error('handler here');
if (err) {
console.error('proxy error', err)
socket.destroy()
}
})
})
proxy.web works. But proxy.ws does not work properly. I see connection getting relayed to server but the proxy quickly closed the connection to the actual websocket server.
Comparing using the above nodeJS vs using simple TCP proxy, I see that when using node, the server is returning encoded data (WebSocket-Protocol: base64) instead of simple text "RFB 003.008" for VNC. I also notice that my client sends "Accept-Encoding: gzip, deflate" but node proxy forwards with "accept-encoding: gzip, deflate, br".
im using the following code:
const proxy = require('http2-proxy');
const finalhandler = require('finalhandler');
const defaultWebHandler = (err, req, res) => {
if (err) {
console.error('proxy error', err)
finalhandler(req, res)(err)
}
}
const server = require('http2').createServer((req, res) => {
console.log(req.headers)
console.log(req.url)
}).listen(8080)
server.on('request', (req, res) => {
proxy.web(req, res, {
hostname: 'localhost',
port: 8081
}, defaultWebHandler)
})
and getting the error:
proxy error Error: Parse Error: Expected HTTP/
at Socket.socketOnData (_http_client.js:509:22)
at Socket.emit (events.js:315:20)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
bytesParsed: 0,
code: 'HPE_INVALID_CONSTANT',
reason: 'Expected HTTP/',
rawPacket: <Buffer 00 00 18 04 00 00 00 00 00 00 04 00 40 00 00 00 05 00 40 00 00 00 06 00 00 20 00 fe 03 00 00 00 01 00 00 04 08 00 00 00 00 00 00 3f 00 01>,
statusCode: 502,
connectedSocket: true,
reusedSocket: false
}
Great lib!!
I'm using it now instead of node-http-proxy.
But I love TypeScript so here are the definition file, maybe you can at it inside your lib.
THX
declare module "http2-proxy" {
import * as Http from "http";
import * as Http2 from "http2";
import * as Net from "net";
function web(req: Http.IncomingMessage | Http2.Http2ServerRequest, res: Http.ServerResponse | Http2.Http2ServerResponse, options: webOptions, callback?: (err: Error) => Function): Promise<Error>
function ws(req: Http.IncomingMessage | Http2.Http2ServerRequest, socket: Net.Socket, head: Buffer, options: wsOptions, callback?: (err: Error) => Function): Promise<Error>
interface options {
hostname: string;
port: number;
proxyTimeout?: number;
proxyName?: string;
timeout?: Http.IncomingMessage | Http2.Http2ServerRequest;
onReq?(req: Http.IncomingMessage, options: Http.RequestOptions): void;
}
interface webOptions extends options {
onRes?(req: Http.IncomingMessage | Http2.Http2ServerRequest, res: Http.ServerResponse | Http2.Http2ServerResponse): void
}
interface wsOptions extends options{
onRes?(req: Http.IncomingMessage | Http2.Http2ServerRequest, socket: Net.Socket): void
}
}
Add "ca", "key", "cert" options and "rejectUnauthorized" options for TLS-based HTTP servers using self-signed certificates.
https://nodejs.org/dist/latest-v14.x/docs/api/tls.html#tls_tls_connect_options_callback
Proxying to a https target are not working. Some option for this case would-be cool and an option to allow self-signed certificate from target (rejectUnauthorized: false).
A simple high performance http/2 & http/1 to http/1 spec compliant proxy helper for Node.
Do you have any benchmarks?
My reverse proxy works a charm with 4.2.15, but not at all with 5.0.31. After staring deeply at it, I'm stumped. The code in question (from https://github.com/risacher/yxorp-edon/blob/master/src/yxorp-http2.js) looks like this:
const listener = function (req, res) {
var target = route(req);
if (null == target) {
res.writeHead(502);
res.end("502 Bad Gateway\n\n" + "MATCHLESS request: "+ req.headers.host+req.url);
} else {
proxy.web(req, res,
{ onReq: (req, options) => {
options.headers['x-forwarded-for'] = req.socket.remoteAddress;
options.headers['x-forwarded-port'] = req.socket.localPort;
options.headers['x-forwarded-proto'] = req.socket.encrypted ? 'https' : 'http';
options.headers['x-forwarded-host'] = req.headers['host'];
options.headers['host'] = req.headers['host'];
options.rejectUnauthorized = false;
options.trackRedirects = true;
options.host = target.host;
options.hostname = target.hostname;
options.port = target.port;
options.path = target.path;
options.protocol = target.protocol+':';
var r = (target.protocol === 'http')?
http.request(options)
: https.request(options);
return r;
},
}, defaultWebHandler );
}
};
After instrumenting http2-proxy to death, I can see that deferToConnect() is called, but the 'socket' event on the proxyReq never fires - I don't understand why. As a result, the proxyReq is never issued, and the clientReq eventually times out. Can anyone help?
Reduce allocations
I have a reverse-proxy server that works fine with 4.2.15, and proxies both web and ws traffic. WIth 5.0.34 it does not proxy ws connections correctly if the path is mapped to a different route.
// works 4.2.15, not 5.0.34:
const upgrade = function (req, socket, head) {
var target = route(req);
if (null != target) {
proxy.ws(req, socket, head, target, defaultWSHandler);
} else {
socket.close()
}
};
The above code doesn't work for websockets with 5.0.34 if the proxy target has a different path than the request. (i.e. ws://fqdn/app/socketEndpoint is proxied to ws://localhost:4000/socketEndpoint) Instead, I get errors on both server and client: "ws proxy error Error: socket hang up" and 'WebSocket connection to 'wss://servername.org:3003/alt/socketEndpoint' failed: Connection closed before receiving a handshake response"
The following code, that uses an onReq handler to create the proxy request does not have this issue:
// works in 5.0.34 (and 4.2.15)
const upgrade2 = function (req, socket, head) {
var target = route(req);
if (null != target) {
proxy.ws(req, socket, head, {
onReq: (req, options) => {
options.host = target.host;
options.hostname = target.hostname;
options.port = target.port;
options.path = target.path;
var r = (target.protocol.match(/^(http|ws):?$/))?
http.request(options)
: https.request(options);
return r;
}
}, defaultWSHandler);
} else {
socket.close()
}
};
This issue has become low-priority since I found this workaround, but it still seems like there's a bug in there somewhere. My original proxy used onReq for web traffic to add X-proxy-for headers, but didn't use an onReq handler for ws traffic, (since the websocket apps don't usually pay attention to the proxy headers anyway.)
I just come across a strange bug here:
I wanted to test the speed of the proxys and use the cluster module. Very often I get this error:
connect EADDRINUSE 192.168.1.4:8090
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) {
errno: 'EADDRINUSE',
code: 'EADDRINUSE',
syscall: 'connect',
address: '192.168.1.4',
port: 8090,
statusCode: 503,
connectedSocket: false,
reusedSocket: false
}
But if I only use one worker, I get no error ???
Is http.request compatible with cluster?
Why this code is not working?
HttpProxy2.web(req, source.http.res, {
hostname: domainTarget.target.split("/")[0],
port: domainTarget.port,
onRes: function (req: Http2.Http2ServerRequest, res: Http2.Http2ServerResponse, proxyRes: Http.ServerResponse, callback: ()=>any) {
callback();
}
}, function (err: Error, req: Http.IncomingMessage | Http2.Http2ServerRequest, res: Http.ServerResponse | Http2.Http2ServerResponse){
if(err){
FinalHandler(req, res)(err);
}
});
Exception:
TypeError: Cannot read property 'Symbol(req)' of undefined
at onComplete (E:\***\***\node_modules\http2-proxy\index.js:165:18)
at Object.onRes (E:\***\***\lib\Worker.ts:332:29)
at ClientRequest.onProxyResponse (E:\***\***\node_modules\http2-proxy\index.js:269:24)
at ClientRequest.emit (events.js:182:13)
at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:546:21)
at HTTPParser.parserOnHeadersComplete (_http_common.js:109:17)
at Socket.socketOnData (_http_client.js:432:20)
at Socket.emit (events.js:182:13)
at addChunk (_stream_readable.js:283:12)
at readableAddChunk (_stream_readable.js:264:11)
at Socket.Readable.push (_stream_readable.js:219:10)
at TCP.onread (net.js:639:20)
Up front I apologize for posting this question here, but I couldn't find anywhere else to do so. But,
I'm attempting to use this to proxy ajax requests to/from another host. However, I'm getting this error when I run index.js:
node dist/index.js
/Users/johndoe/rpd/dig-http2-proxy/node_modules/http2-proxy/index.js:22
} = http2.constants
^
TypeError: Cannot match against 'undefined' or 'null'.
at Object.<anonymous> (/Users/johndoe/rpd/dig-http2-proxy/node_modules/http2-proxy/index.js:22:10)
at Module._compile (module.js:624:30)
at Object.Module._extensions..js (module.js:635:10)
at Module.load (module.js:545:32)
at tryModuleLoad (module.js:508:12)
at Function.Module._load (module.js:500:3)
at Module.require (module.js:568:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/Users/johndoe/rpd/dig-http2-proxy/dist/index.js:4:13)
at Module._compile (module.js:624:30)
Here is the offending code (index.js):
"use strict";
var http2 = require('http2');
var proxy require('http2-proxy');
var fs = require('fs');
var port: 3000;
// Files hard coded for troubleshooting purposes
var serverOptions = {
target: 'http://shiva.rpd.com',
allowHTTP1: true,
key: fs.readFileSync('dist/security/server.key'),
cert: fs.readFileSync('dist/security/server.crt')
};
var server = http2.createServer(serverOptions);
server.on('request', onRequest);
server.listen(port);
function onRequest(req, res) {
var proxyOpts = {
hostname: 'shiva.rpd.com',
port: 80
}
proxy.web(req, res, proxyOpts, (err) => { console.log('Error', err); });
// res.end('Add a response');
}
Now, if I comment out the require('http2-proxy'), proxy.web(req, res....) line and uncomment the res.end() call, this all works (of course with the exception of proxying requests). Also, I tried "http://shiva.rpd.com" and "shiva.rpd.com" as the hostname value in proxyOpts and also no port property with the same result.
Environment:
macOS: 10.12.6
nodejs: 8.5.0
I assume I'm missing something here as I'm a total newb when it comes to nodejs. So, am I missing something or is this a bug?
let configDir = path.join(rootPath, 'config');
let serverOptions = {
key: fs.readFileSync(path.join(configDir, 'ker.pem')),
cert: fs.readFileSync(path.join(configDir, 'cert.pem')),
allowHTTP1: true
};
this.httpProxyServer = http2.createSecureServer(serverOptions
);
this.httpProxyServer.on('request', (req, res) => {
http2Proxy.web(req, res, {
hostname: localAddress.host,
port: localAddress.port,
onReq: (req, options) => {
let headers=options.headers;
headers['X-Forwarded-For'] = req.socket.remoteAddress,
headers['X-Real-IP'] = req.socket.remoteAddress
headers['X-Forwarded-Proto'] = req.socket.encrypted ? 'https' : 'http';
headers['host']=${localAddress.host}:${localAddress.port}
;
// redirectHttp.request(options);
},
onRes: (req, res, proxyRes) => {
res.setHeader('x-powered-by', 'fastnat');
proxyRes.pipe(res)
}
}, defaultWebHandler);
})
Following code accept only upgrade header equal to websocket
, thats not correct as stated in link
Lines 30 to 58 in 999b05f
For example, the client might send a GET request as shown, listing the preferred protocols to switch to (in this case "example/1" and "foo/2"):
GET /index.html HTTP/1.1
Host: www.example.com
Connection: upgrade
Upgrade: example/1, foo/2
Send back a 101 Switching Protocols response status with an Upgrade header that specifies the protocol(s) being switched to. For example:
HTTP/1.1 101 Switching Protocols
Upgrade: foo/2
Connection: Upgrade
Simple fix example
async function proxy (
{ req, socket, res = socket, head, proxyName },
onReq,
onRes
) {
if (req.aborted) {
return
}
const headers = getRequestHeaders(req, proxyName)
if (head !== undefined) {
if (req.method !== 'GET') {
throw new HttpError('only GET request allowed', null, 405)
}
if (req.headers[UPGRADE] === undefined) {
throw new HttpError('missing upgrade header', null, 400)
}
if (head && head.length) {
res.unshift(head)
}
setupSocket(res)
headers[CONNECTION] = 'upgrade'
headers[UPGRADE] = req.headers[UPGRADE]
}
proxy()
returns a promise which neither ws()
nor web()
returns.
This causes Unhandled Promise Rejections
I'm using http2-proxy with a custom onRes
handler to do passthrough compression on the response stream.
And the results are really weird - it works perfectly with Chrome as client, but with Firefox the data received by client seems truncated by one byte (which of course results in error on the client)!
E.g., if the target server sends a JSON, it will be fine received by Firefox except closing '}' is missing!
Same on binary files.
And I've tried with gzip and brotli.
const zlib = require('zlib');
app.use((req, res) => proxy.web(req, res, findTarget(req), errorHandler));
function findTarget(req) {
// ...
const res = {
// ...
onRes: (_req, output, input) => {
const accept = _req.headers['accept-encoding'] ? _req.headers['accept-encoding'].includes('br') : false;
if (accept) {
// trid with zlib.createGzip() and content-encoding 'gzip' as well
const compress = zlib.createBrotliCompress();
output.setHeader('content-encoding', 'br');
input.pipe(compress).pipe(output);
} else {
input.pipe(output);
}
},
};
return res;
}
According to https://tools.ietf.org/html/rfc7230#section-5.7.1 the Via
header must contain the protocol version.
See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Via
In http-proxy
you can forward requests to a unix socket, using: proxy.web(req, res, {target: {socketPath: '/some/path'}})
I can't seem to find a way to do the same with this proxy though, but I need it to improve throughput.
[async] onReq(req, options[, callback]): Called before proxy request. If returning a truthy value it will be used as the request.
If I do something like:
onReq: (req, ureq) => {
ureq.headers['x-user-id'] = 'TEST'
return Http.request(ureq)
}
it works. If I do:
onReq: (req, ureq) => {
ureq.headers['x-user-id'] = 'TEST'
}
It does not work ;).
EDIT:
I guess it should be changed to something like:
const callOnReq = async () => {
if (!onReq) return
if (onReq.length <= 2) {
return onReq(req, ureq)
} else {
// Legacy compat...
return new Promise((resolve, reject) => {
const promiseOrReq = onReq(req, ureq, (err, val) =>
err ? reject(err) : resolve(val)
)
if (promiseOrReq) {
if (promiseOrReq.then) {
promiseOrReq.then(resolve).catch(reject)
} else if (promiseOrReq.abort) {
resolve(promiseOrReq)
} else {
throw new Error(
'onReq must return a promise or a request object'
)
}
}
})
}
}
const request = await callOnReq()
if (request) {
return request
} else {
let agent
if (protocol == null || /^(http|ws):?$/.test(protocol)) {
agent = http
} else if (/^(http|ws)s:?$/.test(protocol)) {
agent = https
} else {
throw new Error('invalid protocol')
}
return agent.request(ureq)
}
Line 107 in db65511
Based on my experiments, I never seem to get a ServerResponse
for proxyRes
. Both the node documentation and inspecting runtime values suggest that proxyRes
should actually be an IncomingMessage
The following code works for me with http2-proxy 4.2.15, but not with 5.0.30:
proxy.web(req, res,
{ onReq: (req, options, cb) => {
options.headers['x-forwarded-for'] = req.socket.remoteAddress;
options.headers['x-forwarded-port'] = req.socket.localPort;
options.headers['x-forwarded-proto'] = req.socket.encrypted ? 'https' : 'http';
options.headers['x-forwarded-host'] = req.headers['host'];
options.headers['host'] = req.headers['host'];
options.rejectUnauthorized = false;
options.trackRedirects = true;
options.host = target.host;
options.hostname = target.hostname;
options.port = target.port;
options.path = target.path;
options.protocol = target.protocol+':';
var r = (target.protocol === 'http')?
http.request(options)
: https.request(options);
return r;
},
}, defaultWebHandler );
In 5.0.30, the options parameter to onReq does not have a 'headers' entry, and in fact, equals{ onReq: [Function: onReq] }
, but this does not seem to agree with the documentation in README.md.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.