http-party / node-http-proxy Goto Github PK
View Code? Open in Web Editor NEWA full-featured http proxy for node.js
Home Page: https://github.com/http-party/node-http-proxy
License: Other
A full-featured http proxy for node.js
Home Page: https://github.com/http-party/node-http-proxy
License: Other
But can you load balance the same hostheader name to different ports on different machines.
The Tables feature does not support this i think from reading of example.
Also would be create to support sticky and round robin patterns as default ones.
Round robin can use pinging to the web servers to determine the load and they can return a number between 1 and 100 to tell the load balancer their load indication.
I created a very simple http-proxy instance for some testing: https://gist.github.com/790342
Running on node v0.3.5, this immediately returns the warning:
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
at Pool.<anonymous> (events.js:101:17)
at Object.proxyRequest (/usr/local/lib/node/.npm/http-proxy/0.3.1/package/lib/node-http-proxy.js:185:7)
at Server.<anonymous> (/usr/local/lib/node/.npm/http-proxy/0.3.1/package/lib/node-http-proxy.js:91:13)
at Server.emit (events.js:45:17)
at HTTPParser.onIncoming (http.js:862:12)
at HTTPParser.onHeadersComplete (http.js:85:31)
at Socket.ondata (http.js:787:22)
at Socket._onReadable (net.js:623:27)
at IOWatcher.onReadable [as callback] (net.js:156:10)
when an http request is made. Under load, it appears this is indeed leaking memory. After about 5 minutes with Siege hitting the proxy with 400 concurrent requests, it dies with:
FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory
However, removing the (empty?) pool error handler on line 184 of node-http-proxy.js appears to fix this. Should this be set once upon pool creation, rather than when the pool is retrieved for each request?
calling proxy.close() does not work like it node's httpServer.close() does.
here is a gist to reproduce: https://gist.github.com/857228
When I install http-proxy with npm, with node 0.4.2, and I run the example in the README doing a routing using server.on('upgrade', I get:
node.js:116
throw e; // process.nextTick error, or 'error' event on first tick
^
TypeError: Cannot read property 'headers' of undefined
at Object. (/usr/local/lib/node/.npm/http-proxy/0.3.1/package/lib/node-http-proxy.js:133:11)
at Object. (/Users/elise/Rails/MediaSquare/proxy.js:7:15)
at Module._compile (module.js:383:26)
at Object..js (module.js:389:10)
at Module.load (module.js:315:31)
at Function._load (module.js:276:12)
at Array. (module.js:402:10)
at EventEmitter._tickCallback (node.js:108:26)
It looks like using new httpProxy.HttpProxy() without parameters fails ...
Say I have a server at example.com and each time a user signs up I want to spin up a new node process that maps to username.example.com, is there an easy way to add this to the route list if I'm using hostname routing, or will I need to just use the custom logic mode?
I'm thinking it'd be handy to be able to do something like this;
var proxy = httpProxy.createServer(options);
proxy.addHost('username.example.com', '127.0.0.1:9000');
as discussed on twitter, it would be very interesting to be able to hack proxied ressources before sending them back to the client.
something like ressource filtering where we'll be able to change the response body/headers
wonder how to integrate this into node http proxy
I'm getting this error when using https:
An error has occurred: {"stack":"Error: socket hang up\n at CleartextStream. (http.js:1272:45)\n at CleartextStream.emit (events.js:61:17)\n at Array.0 (tls.js:617:22)\n at EventEmitter._tickCallback (node.js:126:26)","message":"socket hang up"}
When running http it all works just fine.
Someone else has recently had a similar problem and was able to solve it (not sure if this would apply here though):
Forgive me if this post is slightly uninformed - I am new to Proxying in general. Today I tried to start a proxy to get around some cross domain issues I was having. Apparently the urls I need to proxy to depend on the Host header being the same as the actual domain. When I was hitting it with node-http-proxy, I got some 403s, since it was sending Host : localhost (or whatever). It turns out my service doesn't honor the x-forwarded-for header and I have to fake it, but I'm surprised that this library doesn't automatically append the x-forwarded-for header to the request. From my general understanding it seems like most proxy systems do inject this header, so why doesn't node-http-proxy?
It would be awesome if node-http-proxy could gzip the responses so I could use it to add gzip capability do an "not-so-modern" backend server.
I'm not really sure how to explain so
I whipped up some code to demonstrate.
https://gist.github.com/967964
please run that, then open 3 (console-supporting) browsers at http://localhost:8000
(i hope to find something to simulate in the future)
hit send in each
(you may need to modify node-http-proxy to not change headers)
console will log msgs rcvd from server.
then open 3 more browsers at http://localhost:9000 (no proxy)
hit send in each
notice the output is different.
also, the server output is different.
when proxied, messages are duplicated for client X depending on the number of clients connected after X.
also, connections disconnect after some time when proxied while they remain open indefinitely when not proxied.
node 0.4.7
npm 1.0.6
node-http-proxy 0.5.1
socket.io 0.6.17
Is node-http-proxy supposed to support websockets? It seems that websocket are not passed through (according to my testing), is that correct or do I have to configure something differently?
I'm running into issues that seem like a problem with just my system but I was hoping to find some assistance.
node 0.4.7
http-proxy 0.5.0
vows 0.5.8
colors 0.5.0
optimist 0.2.0
request 1.9.5
socket.io 0.6.17
on mac os x 10.6.7
When I run
vows test/*-test.js --spec
all tests fail that are not expecting a 404 or 500.
The errors look like this:
When using server created by httpProxy.createServer() with no latency and a valid target server
✗ should receive 'hello localhost'
» expected 'hello localhost',
got 'An error has occurred: {"stack":"Error: ECONNREFUSED, Connection refused\\n
at Socket._onConnect (net.js:599:18)\\n at IOWatcher.onWritable [as callback
(net.js:186:12)","message":"ECONNREFUSED, Connection
refused","errno":61,"code":"ECONNREFUSED","syscall":"connect"}' (==) // helpers.js:77
Anyone seen this before? Seems like I'm missing something important. There is very little information on ECONNREFUSED. I do know that the error happens on making the request from proxy to target using the 'request' module (which is expected given the error) but I can't determine why. I'm hoping there's something simple I missed.
Sample code:
require('http-proxy').createServer(80, 'localhost').listen(8001);
Sample request:
GET /phpinfo.php HTTP/1.0
Host: localhost:8001
Request that goes to the server:
GET /phpinfo.php HTTP/1.1
host: localhost:8001
x-forwarded-for: 127.0.0.1
x-forwarded-port: 38349
x-forwarded-proto: http
Connection: close
The server's response for this page looks like this:
HTTP/1.1 200 OK
Date: Thu, 09 Jun 2011 14:53:41 GMT
Server: Apache/2.2.17 (Ubuntu)
X-Powered-By: PHP/5.3.5-1ubuntu7.2
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
259f
<html><head>...lots of HTML
The response to the client looks identical. This, unfortunately, breaks the HTTP/1.0 response since now we are both sending an HTTP/1.1 status and we are doing chunked encoding. The HTTP/1.0 client can't understand this.
The fix: Make the request to the server an HTTP/1.0 request to mirror the client's desires perfectly.
var httpProxy = require('http-proxy');
var options = {
router: {
'domain.com/path: '127.0.0.1:3000'
}
var proxyServer = httpProxy.createServer(options);
proxyServer.listen(80);
When now requesting domain.com/path in the browser, the answer is 127.0.0.1:3000/path but should be 127.0.0.1:3000/ instead
I have a problem with proxying messages from the client to the websocket server.
The messages do not reach the websocket server at all. When I connect to the server directly, it works.
However, all other things work. I can conenct & send messages to the client.
I´m using the following websocket server: https://github.com/miksago/node-websocket-server
Has anyone the same problem?
One of the common issues these days is making requests to third part domains that don't support json and/or jsonp. Please provide an example of this in your demo.
I wrote one that kind of sucks here: https://gist.github.com/864549. 75% of the code is to prevent errors that would crash NodeJS.
For example,
/?url=http://www.google.com/calendar/feeds/[email protected]/public/full?alt=json
Or for XML, User needs to convert this to JSON
/?url=http://www.google.com/calendar/feeds/[email protected]/public/full
This sort of works, obviously its broken b/c I don't rebuild absolute URLs:
?url=http://google.com/
However, this one seriously breaks and I don't know why
?url=http://woot.com/
Remark: Extrapolated from the nodejs-dev mailing list (thanks fidian)
I have updated the proxy code. Here's the newer invocation:
require('http-proxy').createServer(8002, 'localhost').listen(8001);
I have tried the following scenarios:
I have a logging TCP proxy both on the incoming and outgoing sides of
this proxy to see what's sent and received. It is identical to what I
saw when my older code was using sys.puts(). I see that the server is
indeed sending a response and that the JS proxy is not relaying that
information to the client. The time stamps indicate that the
connection is actually closed before the connection to the destination
server is made. Incoming:
-------- CLIENT TO PROXY START --------
[00:00.000 - client 127.0.0.1:35062 forwarded to :8001]
POST /test.php?postclosed HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 47
Connection: close
Host: censored.com
AUTHTOKEN=FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
[00:00.001 - server connected]
[00:00.002 - server closed]
-------- CLIENT TO PROXY END --------
And here's what's logged on the other side:
-------- PROXY TO SERVER START --------
[00:00.000 - client 127.0.0.1:33945 forwarded to devweb1:81]
POST /test.php?postclosed HTTP/1.1
content-type: application/x-www-form-urlencoded; charset=utf-8
content-length: 47
connection: close
host: censored.com
x-forwarded-for: 127.0.0.1
AUTHTOKEN=FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
[00:00.036 - server connected]
HTTP/1.1 200 OK
Date: Tue, 26 Apr 2011 16:35:00 GMT
Server: Apache/2.2.14 (Fedora)
X-Powered-By: PHP/5.3.3
Content-Length: 503
Connection: close
Content-Type: text/html; charset=UTF-8
{ ... hundreds of bytes of JSON data removed ... }
[00:00.079 - server closed]
-------- PROXY TO SERVER END --------
(Reported by @bluescreen303)
I noticed x-forwarded-for was set to 'undefined' when proxying https.
It seems someone else was hit by this too.
http://stackoverflow.com/questions/5999379/how-to-find-out-the-remote-address-in-node-js-if-it-is-https-request
I'm not sure if this is a node bug, but as a workaround, would you please change
req.headers['x-forwarded-for'] = req.connection.remoteAddress;
into
req.headers['x-forwarded-for'] = req.connection.remoteAddress || req.connection.socket.remoteAddress;
Or maybe there's some other way around this problem.
Hi, I download node-http-proxy source package and put the node-http-proxy.js under $Node_Lib path. But I can't find the pool that required in node-http-proxy. So it throw the error. I also can't find this module in github.
Could u help me? Thanks in advanced.
http://server.com/foo reversed to http://internal-server1/foo
http://server.com/bar reversed to http://internal-server2/bar
I would like the proxy headers that are set for normal proxyd requests to appear on websocket requests as well.
Thanks a lot,
Mathijs
Hi, I'm using faye to broadcast realtime messages, but using node-http-proxy I receive this error:
/usr/local/lib/node/.npm/http-proxy/0.4.2/package/lib/node-http-proxy.js:439
headers = new _headers(req.headers), CRLF = '\r\n';
^
TypeError: Cannot read property 'headers' of undefined
at [object Object].proxyWebSocketRequest (/usr/local/lib/node/.npm/http-proxy/0.4.2/package/lib/node-http-proxy.js:439:33)
at Server.<anonymous> (/usr/local/lib/node/.npm/http-proxy/0.4.2/package/lib/node-http-proxy.js:142:13)
at Server.emit (events.js:81:20)
at Socket.<anonymous> (http.js:1023:14)
at Socket._onReadable (net.js:677:27)
at IOWatcher.onReadable [as callback] (net.js:177:10)
And in chrome console I see POST http://nodelytics.strx.int/faye undefined
My proxy configuration in simple
var httpProxy = require('http-proxy');
var options = {
hostnameOnly:true,
router: {
'nodelytics.strx.int': '127.0.0.1:8001',
'pagetitle.strx.int': '127.0.0.1:8002',
}
};
var proxyServer = httpProxy.createServer(options), port=80;
proxyServer.listen(port);
log('HTTP Proxy Ready on port ', port, ', options: ', options);
I'm not sure, but seams to happen when client side code tries to connect to websocket
var client = new Faye.Client('/faye', {
timeout: 120
});
Can you help me? Thanks
Node 0.4.3
npm 1.0rc8
Ubuntu Server 10.04
When I try to run the binary, I get the following error:
$ sudo ./node_modules/http-proxy/bin/node-http-proxy --config=/node-proxy-conf.json --port=80
node.js:134
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Cannot find module './lib/index'
at Function._resolveFilename (module.js:320:11)
at Function._load (module.js:266:25)
at require (module.js:348:19)
at Object. (/home/mcurtis/node_modules/http-proxy/node_modules/winston/node_modules/riak-js/index.js:1:80)
at Module._compile (module.js:404:26)
at Object..js (module.js:410:10)
at Module.load (module.js:336:31)
at Function._load (module.js:297:12)
at require (module.js:348:19)
at Object. (/home/mcurtis/node_modules/http-proxy/node_modules/winston/lib/winston/transports/riak.js:10:14)
Any ideas? according to the riakjs issues list, there isn't an officially-compatible node 0.4 version of riakjs. I'm guessing that plus the new architecture of npm has something to do with this.
(Reported by @bluescreen303)
While you are at it, can you please add the 'x-forwarded-proto' header as well (http, https).
In my case, I let the proxy do ssl, and use plain http backend servers.
This works very well, but my backend servers need to know what urls to generate for internal links.
This header does the trick. I'm not sure if it's standard in any way, but at least rails uses it.
Also, see https://forums.aws.amazon.com/ann.jspa?annID=805 for more information about these headers.
changing the headers "location" & "origin" when proxying websocket request should be an option, because this is important to make the proxying behave like "not happened".
I'm new to http-proxy but after running the example here:
https://github.com/nodejitsu/node-http-proxy/blob/v0.5.0/examples/basic-proxy.js
With some modifications (changed path of http-proxy to work in my environment) it starts but immediately throws an exception when the first client tries to connect. Is there additional setup I need to do to get it running? I am running node v0.5.0-pre according to "node --version" and just pulled http-agent from npm today (2011, April 26th).
Here is the full exception:
http.js:1446 throw new TypeError('Invalid argument to getAgent'); ^ TypeError: Invalid argument to getAgent at Object.getAgent (http.js:1446:11) at _getAgent (/home/tim/local/node/lib/node/.npm/http-proxy/0.5.0/package/lib/node-http-proxy.js:48:30) at [object Object].proxyRequest (/home/tim/local/node/lib/node/.npm/http-proxy/0.5.0/package/lib/node-http-proxy.js:377:12) at Server. (/home/tim/local/node/lib/node/.npm/http-proxy/0.5.0/package/lib/node-http-proxy.js:146:13) at Server.emit (events.js:67:17) at HTTPParser.onIncoming (http.js:1109:12) at HTTPParser.onHeadersComplete (http.js:108:31) at Socket.ondata (http.js:1008:22) at Socket._onReadable (net.js:682:27) at IOWatcher.onReadable [as callback] (net.js:177:10)
There is a max redirects bug in the request module. We should pull in this fix as soon as the package is available on npm.
https://github.com/mikeal/node-utils/issues/issue/9/#comment_697521
Hi there,
I've come across what I think is a bug (or at least an incompatibility) within http-proxy.
What I expect to happen: when I make a proxied POST request to the backend, it should process the request normally.
What happens instead: when I make a proxied POST request with the Connect.bodyDecoder() middleware installed, the request hangs forever in the proxy.
I created a very simple test case here: http://vivoh.com/node-proxy.zip
If you unzip this (and have ruby and the sinatra gem installed), you can see what I mean.
Run as is using the run.sh script, and then hit http://localhost:4568. Then, hit the "Submit Query" button. You'll see the http-proxy module successfully proxies the POST request request to the backend sinatra ruby server.
If you then edit "app.js" and uncomment the Connect.bodyDecoder() line, and then kill the node and ruby servers, and then restart using run.sh, if you try the POST request again, you'll see it hangs forever. I speculate there is something happening inside the bodyDecoder() middleware which is incompatible with the http-proxy module. I've looked around, but have not figured it out myself.
Chris
I am working with a system that is very sensitive, against the standards, to the case of headers that are passed back to it. The current code in node-http-proxy takes all of the headers (ie. "Content-Type") and converts them to lower case. I know that the client is behaving badly here but it would be nice to have this configurable. I will see if I can code this up and send a pull request.
In the meantime, is there a way to disable this behavior?
I just had this happen, dunno what to do. This could be a problem with optimist?
$ node-http-proxy --port 8000 --target localhost:8000 --target test.localhost:1234 The 'sys' module is now called 'util'. It should have a similar interface. node.js:50 throw e; // process.nextTick error, or 'error' event on first tick ^ TypeError: Object true,true has no method 'split' at Object. (/Users/david/local/lib/node/.npm/http-proxy/0.3.1/package/bin/node-http-proxy:52:31) at Module._compile (node.js:348:23) at Object..js (node.js:356:12) at Module.load (node.js:279:25) at loadModule (node.js:251:12) at require (node.js:291:14) at Object. (/Users/david/local/bin/node-http-proxy:11:18) at Module._compile (node.js:348:23) at Object..js (node.js:356:12) at Module.load (node.js:279:25)
was getting a couple of people in the IRC having issues with v0.2.0 of the proxy.
it seems when using the proxy inside of a Connect app, you really need the third API usage.
here is a gist with one of the example usages:
http://gist.github.com/571216
current workaround is to run:
sudo npm install [email protected]
this will install http-proxy version 0.1.5, which has support proxying requests inside of an http.Server
I want to proxy out multiple web servers running on a single ip address with different host names. I need to be able to listen to port 80 and, depending on the domain name, serve up the respective website.
I'm not sure if this is the best place, but I wanted to get a dialog going regarding the new branch of http-proxy which is intended to use the net module.
I know that @ry and @mikeal both have a lot of thoughts about this, so if either of you guys feel like bike shedding, please post whatever information you can that could help direct @indexzero and @olauzon
:-)
Just a minor nitpick
It seems xhr-multipart (keep-alive chunked) socket.io requests get dropped/killed by node-http-proxy.
Since there are enough other ways to connect, this is not a big issue, but I'm interested into why this happens.
I assume this piece of code is to "blame", but I couldn't find much info about keep-alive not being supported by node's http client.
// Force the connection
header to be 'close' until
// node.js core re-implements 'keep-alive'.
outgoing.headers['connection'] = 'close';
So as stated, not a big issue, just would like to track this so I know when I can enable this transport again.
Hmm,
This is a bit of an lousy bug report, more of an notion. I had the http-proxy module fail with the following error message:
/root/local/node/lib/node/.npm/http-proxy/0.3.1/package/lib/proxy-table.js:80
var target = proxy.req.headers.host.split(':')[0] + proxy.req.url;
^
TypeError: Cannot call method 'split' of undefined
at [object Object].proxyRequest (/root/local/node/lib/node/.npm/http-proxy/0.3.1/package/lib/proxy-table.js:80:39)
at Server.<anonymous> (/root/local/node/lib/node/.npm/http-proxy/0.3.1/package/lib/node-http-proxy.js:93:18)
at Server.emit (events.js:45:17)
at HTTPParser.onIncoming (http.js:1078:12)
at HTTPParser.onHeadersComplete (http.js:87:31)
at Socket.ondata (http.js:977:22)
at Socket._onReadable (net.js:654:27)
at IOWatcher.onReadable [as callback] (net.js:156:10)
Was not able to identify what caused it.... trying reproduce. Any idea?
I got the proxy working with high numbered ports. When using port 80 I (think I) have to use sudo and I then get the error mentioned in the title.
Hi, I come across this error when stress test node-http-proxy, using basic-proxy-https script.js
The test is based on a simple python script that use wget:
import os,signal,sys
count = 0
while(count < 1000):
os.system("wget --no-check-certificate --secure-protocol=SSLv3 --spider 'https://172.168.1\
.19:8081/'");
node proxy will run for a while then will shutdown with error:
net.js:825
close(this.fd);
^
Error: ECONNRESET, Connection reset by peer
at Socket.destroy (net.js:825:5)
at Socket._onReadable (net.js:652:30)
at IOWatcher.onReadable as callback
I am trying to install node http-proxy with the following:
npm install http-proxy
But I get the following error:
npm ERR! Unsupported
npm ERR! Not compatible with your version of node/npm: [email protected]
npm ERR! Required: {"node":"= 0.4.7"}
npm ERR! Actual: {"npm":"1.0.6","node":"v0.5.0-pre"}
npm ERR!
npm ERR! System Linux 2.6.18-028stab070.14
npm ERR! command "node" "/usr/local/bin/npm" "install" "http-proxy"
It says that it is unsupported with my version of Node. But I have v0.5.0 installed which is above the necessary v0.4.7.
Please advise.
Hi Charlie,
I'm new to node.js (I learned of it... today), and your http-proxy is one of the first apps I found. The approach is new and very interesting.
However, from your explanation, it seems it can also be used as a kind of load balancer (I have 2 web servers, ws1 and ws2, and the http-proxy at hp1 could connect to both).
Is there an example about that? Maybe it's trivial and a I'm missing something. Or this use-case has to be developed from scratch?
Thank you
Paolo
error in chrome:
Error during WebSocket handshake: origin mismatch : http://outside.host != http://localhost
the websocket reverse proxy doesn't seem to do its job completely.
It does proxy the handshake to the desired port and host (localhost), but the answer comes back with the back-end host instead of the origin.
In case of an extra proxy outside.host in front of the http-proxy, this makes the websocket connections fail
(unless a trick is applied with /etc/hosts to make to map outside.host to localhost).
http-proxy version 0.5.0, node version 0.4.6
Tested in chrome.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.