Meanwhile, thank you very much, the tool is perfect and simple.
My situation is this:

I have the first server (http1 - 192.168.2.7:4141) where there apache listening on port 4141, listening teeproxy on 80 and sends the first (192.168.2.7 and second (HTTP2). In both apache log (access_log) appear the same demands, so all ok, perfect.

But almost every request, stdout of the first server, the message
Failed to receive from 192.168.2.7:4141: persistent connection closed
What does it mean?

Thank you very much

Hi,

we are currently using teeproxy in a migration use case. Our complete tracking infrastructure is about to be migrated to an alternative hosting facility. Therefore we have to duplicate HTTP traffic for a limited time period.

We installed and activated teeproxy and it is working like a charm ... but ... our syslog is filled with warnings like this:
2018/08/28 08:58:25 http: multiple response.WriteHeader calls

Looking into the source, we were able to identify that the WriteHeader() method is indeed executed twice:

...
        resp := handleRequest(productionRequest, timeout)

        if resp != nil {
                defer resp.Body.Close()

                // Forward response headers.
                for k, v := range resp.Header {
                        w.Header()[k] = v
                }
                w.WriteHeader(resp.StatusCode)

                // Forward response body.
                body, _ := ioutil.ReadAll(resp.Body)
                w.Write(body)
        }
        w.WriteHeader(resp.StatusCode)
        io.Copy(w, resp.Body)
...

IMHO the second call is obsolete. The error messages do not inflict the functionality of teeproxy but they are definitely filling our logfiles. So we removed the line in question and rebuild teeproxy. It is still working without problems. The error messages have been gone.

Hello,

On a production server we have several web servers running on the same host, under distinct ports (8080, 8081, 8082). We want to have teeproxy forwarding to each of these ports, but we have no luck on doing this.

If running a single teeproxy process (eg.: using 8080 as A), everything works as expected.

Although, if launching a second or more teeproxy processes, nothing else is forwarded to the A server.

For example:

teeproxy -l :9000 -a localhost:8080 -b another:7777
teeproxy -l :9001 -a localhost:8081 -b another:7777

If kept the first line, A and B servers receive the requests. When launching the second process, A:8080 stops receiving requests, but B:7777 keeps receiving them (when posting to :9000).

Is this the expected behavior?

@chrislusf

hi, i have the following architecture with have http trafffic (POST request/response):

master_server ---- process_server 
                  (service: calcData)

here is how i would like to use teeproxy. would run on process_server read the incomming traffic from master_server and forward the traffic to:

• service: calcData running on process_server
• service: calcData running on qa_server

master_server ----- process_server ----- qa_server
                       teeproxy <----->  service: calcData
                          ^
                          |
                          v
                    service: calcData

i have a few questions:

• how much traffic can teeproxy handle? is it able to handle 60k QPS?
• i have traffic with low latency:
  the latency between master_server and process server is 1-20ms
  teeproxy would run on process_server and forward the traffic to qa_server
  is teeproxy able to handle such low latency?
• whats the overhead to include teeproxy for forwarding http traffic to qa_server?

Hello,

We have a use case where duplication of traffic is needed, but we only want to duplicate specific HTTP methods, ex. GETs.
One reason to do this is if both A and B use the same underlying datastores in which case multiple POSTs, etc are not idempotent, but GETs probably are.

I'm happy to volunteer a patch for this if there's interest.

I noticed that I was never getting back the data or headers I was expecting from my "targetProduction" (aka the "-a" server).

Digging in I found that while the -a machine was returning a 200 (and all of the expected data) teeproxy was also see an error message. Because of that error message it would discard the data and just return the default 200 OK message.

(For the curious, the error message was: "Failed to receive from localhost:9090: persistent connection closed")

In Pull Request #10 I added in a check for the targetProduction side.

I would like to hard code 1 IP to drop the connection and not make it return any values. I realised you have
defer resp.Body.Close()
in the code. Can you teach me where to slot in the code for the 1 IP to drop the connection?

more like for a firewall application, i will make it detect an IP and drop the connection instead of returning an empty 0 byte "html" request. pls help on this. i'm new to golang.

Please show me the lines inside this function
func (h handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {

//e.g. ip = '123.123.123.123'
if ( ip == "123.123.123.123" ) {
drop connection (without opening more files of affecting connections etc)
}

It appears the response headers are being a bit mangled. For example, the content-type changes to text/plain instead of application/json, and the HTTP/1.1 line appears in the http response body.
I am using teeproxy in front of a json rest web service hitting with chrome using Advanced REST Client chrome plugin. Hitting the web service directly works.

i tried to teeproxy some websites with .my extension and it just doesnt work. not sure if it's the extension problem or otherwise. how do I check?

tee proxy also crashed with high load without any apparent warnings.
where is the log output? i've tried -debug but there's nothing shown. appreciate this help.

update : for the .my extension, i think was backend server side issue

I'm trying to put teeproxy between two nginx servers:

                             /- nginx prod:2030
nginx:80 ---> teeproxy:8080 <
                             \- nginx dev:2031

but it's failing miserably:

Failed to receive from 127.0.0.1:2031: persistent connection closed
Failed to receive from 127.0.0.1:2030: persistent connection closed

Using directly curl teeproxy:8080 works fine.

Hello,

I would like to suggest an addition to the feature set provided by teeproxy. We have a use case where duplication of traffic isn't enough. We'd like to be able to gain insights about failing requests. We define a failing request as a request, the responses to which aren't an exact match between A and B.

I understand that not everyone might care about this, and most users probably don't, so it's probably a good idea to put this behind a feature flag.

I'm happy to volunteer a patch for this if there's interest. Also happy to expand on the approach we're taking to report failures.

http: multiple response.WriteHeader calls

The above "error", not sure if it's critical but getting a lot of that.

Is there a fix for this?

2017/10/15 01:00:55 http: Accept error: accept tcp [::]:80: accept4: too many open files; retrying in 1s

How can I resolve this? How do I get too many open files? i've set ulimit -n 524288

Handling > 1 mil test traffic

I found this via a StackOverflow post, and it looks like it is exactly what I need. I was able to get it working on Windows with no issues (go FTW).

I was planning to put a server running this into my haproxy rotation, and I can limit the traffic, but I'm not sure what it can handle. My question is, has this been used under load? If so, how much?

Thanks!
Erick

Is teeproxy keep-alive enabled? How do I make it keep-alive?

I've been testing it for quite some time, though I havent run it for a week, it seems to hold up ok for 3 days. I needed to reset from time to time (because of some other additional features I've added to it)

How do I do keep-alive again?

Connection close seems to prolonged the crash for 3 days... it "crashed" after 3 days. I'm still not very sure if it's teeproxy problem. will try and see. any suggestions do mention. thanks.

I keep getting following output from my "b" site.

Failed to receive from aws-fcd-dev.datasvc.abc.com:443: unexpected EOF

Docker Hub allows you to create Automated Builds from source: https://docs.docker.com/docker-hub/builds/
It would add a packaging/distribution/installation method, whose buildings would be triggered automatically on each commit. It also allows to create different image tags automatically from git tags & branches.

By making the image build via an AB, you give the resulting image verifiability and auditability. Also, the build is fully automatic. You can have the latest image tag build from HEAD and individual image tags from git's release tags. Some people avoid non-verifiable (manually uploaded) images due to security & traceability reasons.
Added to that, there are a handful of teeproxy public images in Docker Hub, some of them unmaintained for 4 years. Linking the 'official', straight from source build image on the readme would avoid people using an outdated version of the software by mistake,

Since there is already a Dockerfile in the repo, just a free Docker Hub account and a quick setup will do. Ping me if I can be of any help.

It looks like if you specify "-b.rewrite true" as stated in the docs, it does not get picked up correctly. However if you use "-b.rewrite=true" it works fine. Bizarrely, it does seem to work for "-a.rewrite true".

Hi,
I need to duplicate traffic and also I need to add some HTTP header before sending to server.
Is this function is available?

from the log... i removed additional line information etc. but it seems if the speed is too high on the requesting side, sometimes i get "800http:" stuck together for ":800" as the forwardeda, also if the return timed out or error returned, teeproxy will crash at the line of the Logger.Print below. I think "r" value is wrong or something. Can you check it pls? Thanks. I have no way to simulate this but I think it's the returned value from the proxied A that's giving the problem.

Request failed:context canceled
Request failed:context canceled
Request failed:context canceled
Request failed:invalid URL port "800http:"
Request failed:context canceled
Request failed:context canceled

    defer func() {
            if r := recover(); r != nil && *debug {
                    Logger.Print("Recovered in ServeHTTP(production request) from:", r)
            }
    }()

I ran teeproxy in front of varnish etc. but I can't seem to get the actual IP or the x-forwarded-for IPs.
can you please guide me on how I can forward the actual IPs to varnish / haproxy / nginx? I'm forwarding the request but the IP will show it comes from the server instead of from the client.

I replaced some meanwhile deprecated library methods with more recent code. Before creating a merge request I'd like to know which Go version I should aim for. Go 1.8 is out since February, but probably not many environments have it yet. Do you have any preferences?

Also I had to implement some new features to make it applicable for my scenario. For instance, optional setting of XFF headers, closing of connections or response error handling. I'd like to make these available, too.

I tried to use this to split a video encoder output to dual Unified Sreaming ingest servers. It fails, and I think I know why. The protocol uses one long POST request and keeps that running, sending the data as chunked transfers. This causes the tee proxy to wait for the end of the request which will never happen, and fill up memory until it crashes. Does anyone have an idea on if it would be possible to modify this to support that kind of traffic?

This is a shameless plug so please remove if you wish.

A while back I was using multiple teeproxy instances to chain requests to multiple endpoints, I ended up rewriting teeproxy to allow for any number of endpoints to be specified, it also uses a JSON configuration file that can be hot-reloaded.

If any one else is interested here is the link:

hans-strudle/forw

As far i teeproxy can be used for HTTP requests. Is it applicable for TCP connections also?

Hi Chris,

I came across your project and wondered how to do this with https enabled. I thought about putting an nginx for ssl offloading in front of teeproxy. But then I found the @post_action directive of nginx and teeproxy became obsolete for me right now: https://gist.github.com/bsingr/7107a5074c6390d5ef70

I just wanted to share this with you, Chris. Maybe you'll find it helpful or even find some drawbacks in my solution... Anyhow, thanks for sharing teeproxy with the rest of us ;-)

jens

1. If fronted by Cloudflare, XFF shows CloudFlare IP

I'm doing load testing currently but I really need the IP to reflect real IPs.
e.g. XFF = Client IP, Cloudflare IP etc

2. I also realised something with teeproxy, I'm not sure how to "DROP" the connection when an IP is detected. How do I drop the connection instead of the connection always returning some value? (even if blank html)

Could you compile and release a binary version?

I'm using it currently without any issues except I think it's not http2 compatible. How can I make that happen?

I got a segmentation fault when trying to build on MacOS. Could you please help?