mholt / caddy-l4 Goto Github PK

View Code? Open in Web Editor NEW

914.0 914.0 69.0 407 KB

Layer 4 (TCP/UDP) app for Caddy

License: Apache License 2.0

Go 100.00%

caddy-l4's People

Contributors

Stargazers

Watchers

Forkers

isgasho moneytech freemind-lj pwfoo hslatman mildred scorpion miloradlalovic icodein zhui177 jk contrun renowncoder mr-jiggledaddy madhuvanthg crt-fork rijul-a francislavoie xuzhenglun sw8fbar fundeckhermit unwebio xerkus robert-hedler iamroryb ffengill suryatmodulus dekimsey jamison-phillips aksial dinglinxiao honeytreelabs weidideng dumpmemory nodece rios0rios0 superuser5 kkroo regbo xray-manager since1986 dyhkwong gr33nbl00d peace4j jtackaberry l1labs quangtuyen88 1121170088 qs-wang sybenx agittins wuxiaoxrj andrewpalumbo dezren39 francoismichel devsnek metafeather hsz0707 vgerris diyism manassehzhou legobeat thebarton vnxme minecodes opvexe louis-lau

caddy-l4's Issues

Build fails with malformed module path

I cant install anymore using the xcaddy build. Last year it works fine.
Go version go version go1.13.8
xcaddy version v0.2.0
Here are the logs

2022/02/09 08:58:08 [INFO] Cleaning up temporary folder: /tmp/buildenv_2022-02-09-0858.346512074
2022/02/09 08:58:08 [FATAL] exit status 1
azureuser@pwf:~/caddy-l4$ sudo xcaddy build --with github.com/mholt/caddy-l4
2022/02/09 08:58:12 [INFO] Temporary folder: /tmp/buildenv_2022-02-09-0858.766168378
2022/02/09 08:58:12 [INFO] Writing main module: /tmp/buildenv_2022-02-09-0858.766168378/main.go
2022/02/09 08:58:12 [INFO] Initializing Go module
2022/02/09 08:58:12 [INFO] exec (timeout=10s): /usr/bin/go mod init caddy
go: creating new go.mod: module caddy
2022/02/09 08:58:12 [INFO] Pinning versions
2022/02/09 08:58:12 [INFO] exec (timeout=0s): /usr/bin/go get -d -v github.com/caddyserver/caddy/v2
2022/02/09 08:58:18 [INFO] exec (timeout=0s): /usr/bin/go get -d -v github.com/mholt/caddy-l4
go: finding github.com/mholt/caddy-l4 latest
2022/02/09 08:58:21 [INFO] Build environment ready
2022/02/09 08:58:21 [INFO] Building Caddy
2022/02/09 08:58:21 [INFO] exec (timeout=0s): /usr/bin/go mod tidy
caddy imports
        github.com/caddyserver/caddy/v2/modules/standard imports
        github.com/caddyserver/caddy/v2/modules/caddyhttp/standard imports
        github.com/caddyserver/caddy/v2/modules/caddyhttp/encode/zstd imports
        github.com/klauspost/compress/zstd tested by
        github.com/klauspost/compress/zstd.test imports
        github.com/klauspost/compress/zip imports
        io/fs: malformed module path "io/fs": missing dot in first path element
caddy imports
        github.com/caddyserver/caddy/v2/modules/standard imports
        github.com/caddyserver/caddy/v2/modules/caddyhttp/standard imports
        github.com/caddyserver/caddy/v2/modules/caddyhttp/encode/zstd imports
        github.com/klauspost/compress/zstd tested by
        github.com/klauspost/compress/zstd.test imports
        github.com/klauspost/compress/zip tested by
        github.com/klauspost/compress/zip.test imports
        testing/fstest: malformed module path "testing/fstest": missing dot in first path element
2022/02/09 08:58:23 [INFO] Cleaning up temporary folder: /tmp/buildenv_2022-02-09-0858.766168378
2022/02/09 08:58:23 [FATAL] exit status 1

Configuration Help

Hi, I really don't understand how to use json format so please bear with me for this simple question.
I want to route ssh to say 192.168.0.11:22, from ssh.mydomain.com.
I added this at the end of my json after the cloudlfare dns stuff. It doesnt seem to work, I get a kex_exchange_identification: Connection closed by remote host error.

}, # end of cloudflare plugin block
		"layer4": {
			"servers": {
				"srv0": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"match": [
								{
									"ssh": {}
								}
							],
							"handle": [
								{
									"handler": "proxy",
									"upstreams": [
										{
											"dial": [
												"192.168.0.11:22"
											]
										}
									]
								}
							]
						}
					]
				}
			}
		}
	}

Any help would be greatly appreciated. Thanks

How to load caddy.json ?

How to I make Caddy load the json file?

I am trying to run the simple echo example.

xcaddy build --with github.com/mholt/caddy-l4
created caddy.json
./caddy run

$ ./caddy run
2022/09/05 18:00:27.101 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["//localhost:2019", "//[::1]:2019", "//127.0.0.1:2019"]}
2022/09/05 18:00:27.101 INFO    serving initial configuration

Sorry for the stupid question, but I can't seem to find docs about how to start a Caddy server that reads the caddy.json file. I'm new to Caddy.

Nested matching and handling

It is a common case that the client sends the HTTP traffic in a TLS session and the proxy forwards it upstream in PROXY protocol. I want to match PROXY protocol and then match TLS session, if the match fails then we should fallback to next route. To the best of my knowledge, there is currently no way to achieve such effect. I propose we implement a special handler called nested to handle such case.

Below is an example configuration. Assume that we have already implemented a matcher called proxyprotocol which matches all traffic in PROXY protocol and a handler called proxyprotocol which turns PROXY protocol traffic into normal TCP traffic. The below configuration instructs caddly-l4 to first matches proxy protocol, then unwraps PROXY protocol traffice and handles the unwrapped traffic with the specified routes, which in turn try to match it with tls and then terminate tls, if that fails, match it with the http matcher.

If the entire nested matcher failed to match the connection, we should fallback to the next route. My configuration below makes caddy-l4 accept both traffic in PROXY protocol and in normal TCP connection.

{
  "apps": {
    "layer4": {
      "servers": {
        "example": {
          "listen": ["0.0.0.0:5000"],
          "routes": [
            {
              "match": [
                {
                  "proxyprotocol": {}
                }
              ],
              "handle": [
                {
                  "handler": "proxyprotocol"
                },
                {
                  "handler": "nested",
                  "routes": [
                    {
                      "match": [
                        {
                          "tls": {}
                        }
                      ],
                      "handle": [
                        {
                          "handler": "tls"
                        },
                        {
                          "handler": "proxy",
                          "upstreams": [{ "dial": ["localhost:8080"] }]
                        }
                      ]
                    },
                    {
                      "match": [
                        {
                          "http": [{ "host": ["example.com"] }]
                        }
                      ],
                      "handle": [
                        {
                          "handler": "proxy",
                          "upstreams": [{ "dial": ["localhost:8080"] }]
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "match": [
                {
                  "tls": {}
                }
              ],
              "handle": [
                {
                  "handler": "tls"
                },
                {
                  "handler": "proxy",
                  "upstreams": [{ "dial": ["localhost:8080"] }]
                }
              ]
            },
            {
              "match": [
                {
                  "http": [{ "host": ["example.com"] }]
                }
              ],
              "handle": [
                {
                  "handler": "proxy",
                  "upstreams": [{ "dial": ["localhost:8080"] }]
                }
              ]
            }
          ]
        }
      }
    }
  }
}

L4 proxy duplicates messages

I have a docker swarm of caddy servers running the following config.json:

v1.0.1

{
"admin": {"disabled": true},
"logging": {
"logs": {
"default": {
"level": "ERROR",
"writer": {"output": "stderr"}
}
}
},
"apps": {
"layer4": {
"servers": {
"logstash_proxy": {
"listen": ["0.0.0.0:5145"],
"routes": [
{
"handle": [
{
"handler": "proxy",
"upstreams": [
{
"dial": ["10.222.107.135:5146","10.222.95.131:5146","10.222.107.143:5146","10.222.94.121:5146","10.222.95.144:5146","10.222.106.119:5146","10.222.93.125:5146","10.222.96.142:5146","10.222.107.147:5146","10.222.107.135:5147","10.222.95.131:5147","10.222.107.143:5147","10.222.94.121:5147","10.222.95.144:5147","10.222.106.119:5147","10.222.93.125:5147","10.222.96.142:5147","10.222.107.147:5147"]
}],
"load_balancing": {"selection": {"policy": "random"}}
}
]
}
]
}
}
}
}
}

And it is sending the same incoming message to each of the addresses in the dial stanza. I clearly don't understand how this is supposed to work. Can you tell me how to get it to load balance between all those servers instead?

SSH Proxy - Question

Sorry if this is a noob question.
This plugin will allow me to proxy ssh to a local ip? For example ssh.myserver to localhost:22?
The reason I want this is that I’m behind a corporate firewall that blocks outgoing ssh connections.
However if I could cloak it in https traffic it would work. As long I configure sshd to respond to http requests then this plugin should is what I’m looking for correct ? I believe you can do this with apache using http connect module.

[Question] Versioning

I want to use caddy-l4 in docker.

The latest caddy-builder image on Docker Hub is 2.6.1.

The Dockerfile

FROM caddy:2.6.1-builder-alpine AS builder

RUN xcaddy build \
    --with github.com/mholt/caddy-l4/layer4

FROM caddy:2.6.1-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

fails with

#0 108.7 go: github.com/mholt/[email protected] (matching github.com/mholt/caddy-l4/layer4@upgrade) requires github.com/caddyserver/caddy/[email protected], not github.com/caddyserver/caddy/[email protected]

So the obvious solution is to build caddy manually.

I am following the steps on the caddy readme.

My main.go looks like this:

import (
	// ...
	_ "github.com/caddyserver/caddy/v2/modules/standard"
	_ "github.com/mholt/caddy-l4"
)

// ...

On master, caddy-l4 depends on caddy 2.6.2

go get github.com/mholt/caddy-l4 yields

go: github.com/caddyserver/caddy/[email protected]: reading github.com/caddyserver/caddy/go.mod at revision v2.6.2: unknown revision v2.6.2

The latest release on caddy appears to be 2.6.1

There are no tags/versions on this repo either, and the Caddy download build with caddy-l4 fails.

I must admit, I don't have a clear picture of building go with specific commits/branches/tags of a repo, so I might need some pointers there.

Would you consider versioning this project?

~~How can I build master (or a recent commit)?~~

Environment

$ go version
go version go1.19 linux/amd64

$ uname -r
5.4.0-126-generic

ETA:

Deleting github.com/caddyserver/caddy/v2 and github.com/mholt/caddy-l4 from go.mod and then running go get github.com/mholt/caddy-l4 followed by go mod tidy worked. I was still drafting this issue.

I will still leave this open in case someone is able to shed light on releases.

TLS termination hangs in latest commit

I am not sure why, but the latest commit of this plugin hangs when terminating TLS connections. When using the below config, this build will hang indefinitely:

xcaddy build --with github.com/mholt/caddy-l4

This build however, will not:

xcaddy build --with github.com/mholt/caddy-l4@6587f40d4eb632d15f295cf9791d7d145434d0da

No errors are reported in logs. For what it's worth I'm running this on Windows/

{
	"logging": {
		"logs": {
			"default": {
				"level": "DEBUG"
			}
		}
	},
	"apps": {
		"tls": {
			"certificates": {
				"automate": ["mongodb.127.0.0.1.nip.io", "tendis.127.0.0.1.nip.io", "webserver.127.0.0.1.nip.io"]
			},
			"automation": {
				"policies": [{
						"issuer": {
							"module": "internal"
						},
						"key_type": "rsa4096"
					}
				]
			}
		},
		"layer4": {
			"servers": {
				"server_0": {
					"listen": ["0.0.0.0:443"],
					"routes": [{
							"match": [{
									"tls": {
										"sni": ["mongodb.127.0.0.1.nip.io"]
									}
								}
							],
							"handle": [{
									"handler": "tls",
									"connection_policies": []
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["localhost:27017"]
										}
									]
								}
							]
						}, {
							"match": [{
									"tls": {
										"sni": ["tendis.127.0.0.1.nip.io"]
									}
								}
							],
							"handle": [{
									"handler": "tls",
									"connection_policies": []
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["localhost:51002"]
										}
									]
								}
							]
						}, {
							"match": [{
									"tls": {
										"sni": ["webserver.127.0.0.1.nip.io"]
									}
								}
							],
							"handle": [{
									"handler": "tls",
									"connection_policies": [{
											"alpn": ["http/1.1"]
										}
									]
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["localhost:8282"]
										}
									]
								}
							]
						}
					]
				}
			}
		}
	}
}

No support for UDP

As I'm giving caddy-l4 a try to a tentative hacky ingress gateway in my homelab, I tried to have it proxying DNS UDP but as soon as I give the listen address "udp/127.0.0.1:5000", caddy fails to start with the following error:

run: loading initial config: loading new config: layer4 app module: start: listen udp 127.0.0.1:5000: address 127.0.0.1:5000: unexpected address type

Supports HAProxy PROXY protocol in handler

It allows to keep the source address over any kind of protocol:

https://www.haproxy.com/blog/haproxy/proxy-protocol/

It is supported for example by Exim:

https://exim.org/exim-html-4.93/doc/html/spec_html/ch-proxies.html

Whether this plug-in can implement SNI shunt？

Whether this plug-in can implement SNI shunt, similar to nginx or haproxy SNI shunt.

Is it possible to log the request url?

Hello,
I'm using l4 and I would like to be able to log the requested url and haven't been able to do it.
Is is possible?

PacketConn (udp) listener being closed by server handler

Once a handler chain completes (echo for instance), the server's PacketConn is closed when handle() returns. Xref: https://github.com/mholt/caddy-l4/blob/master/layer4/server.go#L97 ... This means the server will no longer accept incoming requests for this listener. I see the server's PacketConn is wrapped in a fakeClosePacketConn but is destructed in this case since it's the last usage.

I'm pretty sure this a logic error for datagram protocols at least? And apologies, but I'm too inexperienced with this framework to suggest a fix at this point.

This can be reproduced using just the echo handler:

{
  "apps": {
    "layer4": {
      "servers": {
        "test": {
          "listen": ["udp/0.0.0.0:12345"],
          "routes": [
            {
              "handle": [{"handler": "echo"}]
            }
          ]
        }
      }
    }
  }
}

Then send any UDP traffic to port 12345. I use dig @localhost -p 12345 example.com. The first request will be echoed back (with warnings but that's beside the point) but subsequent requests will fail.

I'm able to reproduce this with caddy-l4@master using Caddy v2.4.6 and v2.5.0-beta.1.

Does the module support RTMP?

I am using a docker with an RTMP server and I want to route requests on rtmp://example.com:1935/live to docker 172.15.0.1:1935.

Can I do this with this module and if so, what should be the configuration?

[Question]: TLS termination for PostgreSQL

I have two local containers running in docker:

caddy - bridged network mode, local ip 172.17.0.5, also just reachable via 127.0.0.1 and all open port (80, 443, 11443)
postgres - also bridged network mode, basically same as above but port 65432 open on host and the usual 5432 internally

Now I'm trying to terminate the tls connection via caddy and connect to the postgres instance.

I spent hours trying to make this work with the help of the l4 module and I think my config should work but somehow it doesn't.

apps:
  tls:
    certificates:
      automate: ["pg.test"]
    automation:
      policies:
        - issuers:
            - module: "internal"
  layer4:
    servers:
      ts14:
        listen: [":11443"]
        routes:
          - handle:
              - handler: tls
                connection_policies: []
              - handler: proxy
                upstreams:
                  - dial: ["172.17.0.2:5432"]
                    tls:
            match:
              - tls:
                  sni: ["pg.test", "pg.test:11443"]
logging:
  logs:
    default: { level: DEBUG }

So from my host machine, I can connect to postgres via 127.0.0.1:65432 as 65432 ist mapped from my host to 5432 in the postgres container. As caddy and postgres are on the same host bridge, they can reach each other no problem, so caddy can reach postgres on 172.17.0.2:5432. So far so good and everything works just fine without tls.

But as soon as I try to make the tls config work, it simply won't connect. I've gone through all issues in this repo and it seems as if some have already got it to work?

Also, I just found this issue what made me wonder if it's actually possible at all? I'm really not good with the tls termination stuff so I have no idea of the limitations. I would greatly appreciate any help or a point into the right direction.

PS: *.test is configured on my mac via dnsmasq and simply resolves to 127.0.0.1.

[FEATURE]: Add support for logs and subroute modules

Should this plugin support logs and subroute modules ?
My configuration like this:

  "apps":{
    "layer4":{"servers":{"stream":{
      "listen":[":443"],
      "logs":{"default_logger_name":"log0"},
      "routes":[
        { "terminal":true,
          "match":[{"tls":{"sni":["MyDomain.com"]}}],
          "handle":[{"handler":"subroute","routes":[
          { "handle":[{"handler":"tls"}]},
          { "handle":[{"handler":"proxy","upstreams":[{"dial":["unix//dev/shm/websocket"]}],"proxy_protocol":"v2"}],
            "match":[{"http":[{"header":{"Upgrade":["websocket"]},"path":["/websocket"]}]}]},
          { "handle":[{"handler":"proxy","upstreams":[{"dial":["local:80"]}]}]}]}]}]
    }}},
    "http":{"servers":{
      "srv1":{
        "allow_h2c":true,
        "listen":[":80"],
        "logs":{"default_logger_name":"log1"},

error log:

Mar 27 04:55:27  caddy[15504]: 2022/03/26 20:55:27.015        ERROR        watcher        applying latest config        {"config_file": "/etc/caddy/Caddyfile.json", "error": "loading new config: loading layer4 app module: decoding module config: layer4: json: unknown field \"logs\""}

Mar 27 04:57:50  caddy[15504]: 2022/03/26 20:57:50.015        ERROR        watcher        applying latest config        {"config_file": "/etc/caddy/Caddyfile.json", "error": "loading new config: loading layer4 app module: decoding module config: layer4: json: unknown field \"terminal\""}

Mar 27 04:58:33  caddy[15504]: 2022/03/26 20:58:33.015        ERROR        watcher        applying latest config        {"config_file": "/etc/caddy/Caddyfile.json", "error": "loading new config: loading layer4 app module: provision layer4: server 'srv0': route 0: position 0: loading module 'subroute': unknown module: layer4.handlers.subroute"}

Logs module makes the layer4 plugin output pretty logs by apart logs from program's kernel log .

And will subroute make caddy more secure by filter sni before actions ?

Specify IP addresses for the IP matcher, send all other packets to a sort-of sinkhole port, and set a specific address or subdomain for the ssh matcher

Hello!

In a similar vein to this Configuration Help issue by @gt2416, several questions:

How do I specify IP addresses for the IP matcher? Is it a simply array, such as the HTTP matcher, or a hash table / dictionary, such as the TLS matcher, or something else entirely?
How can I send all other packets to a sort-of sinkhole port or domain, such as the anyprot option of sslh?
And as in the aforementioned issue, how do I set a specific address or subdomain for the ssh matcher?

Thank you kindly for the help!

Ratelimit handler support

Hello everyone,

Are we able to leverage the ratelimit handler within a layer4 listener using proxy protocol?

Thanks!

Proxy Protocol Support

Hello @mholt , I'm working on a project that requires both the L4 and Proxy Protocol plugins and I'm running into an error when starting the server:

run: loading initial config: loading new config: loading layer4 app module: decoding module config: layer4: json: unknown field "listener_wrappers"

I need to accept TCP traffic from an NLB, using Proxy Protocol in order to preserve the source IP and then proxy traffic upstream over HTTP. Is this currently supported?

Version: 2.4.3

Plugins: mastercactapus/caddy2-proxyprotocol, mholt/caddy-l4, caddy-dns/route53, silinternational/certmagic-storage-dynamodb

Caddyfile.json:

{
    "admin": {
        "disabled": true
    },
    "logging": {
        "logs": {
            "default": {
                "writer": {
                    "output": "stdout"
                },
                "encoder": {
                    "format": "json"
                }
            }
        }
    },
    "storage": {
        "aws_region": "us-west-2",
        "module": "dynamodb",
        "table": "ACME"
    },
    "apps": {
        "tls": {
            "certificates": {
                "automate": [
                    "app.my.domain.com"
                ]
            },
            "automation": {
                "policies": [
                    {
                        "subjects": [
                            "app.my.domain.com"
                        ],
                        "issuers": [
                            {
                                "ca": "https://acme-staging-v02.api.letsencrypt.org/directory",
                                "challenges": {
                                    "dns": {
                                        "provider": {
                                            "name": "route53"
                                        }
                                    }
                                },
                                "email": "[email protected]",
                                "module": "acme"
                            }
                        ]
                    }
                ]
            }
        },
        "layer4": {
            "servers": {
                "srv0": {
                    "listen": [
                        ":6312"
                    ],
                    "listener_wrappers": [
                        {
                            "allow": [
                                "0.0.0.0/0"
                            ],
                            "timeout": 900,
                            "wrapper": "proxy_protocol"
                        }
                    ],
                    "routes": [
                        {
                            "match": [
                                {
                                    "tls": {
                                        "sni": [
                                            "app.my.domain.com"
                                        ]
                                    }
                                }
                            ],
                            "handle": [
                                {
                                    "handler": "tls",
                                    "connection_policies": [
                                        {
                                            "cipher_suites": [
                                                "TLS_AES_128_GCM_SHA256",
                                                "TLS_AES_256_GCM_SHA384",
                                                "TLS_CHACHA20_POLY1305_SHA256",
                                                "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
                                                "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
                                                "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
                                                "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
                                                "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
                                            ],
                                            "curves": [
                                                "x25519",
                                                "secp256r1",
                                                "secp384r1",
                                                "secp521r1"
                                            ],
                                            "protocol_min": "tls1.2",
                                            "protocol_max": "tls1.3"
                                        }
                                    ]
                                },
                                {
                                    "handler": "proxy",
                                    "upstreams": [
                                        {
                                            "dial": [
                                                "localhost:6311"
                                            ]
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            }
        }
    }
}

Is there any way to match the destination host for SSH matcher ?

First, thank you very much for this amazing piece of code.

I have multiple vhosts running on the proxy and one of them offers ssh access. For extra security I want to proxy ssh connection only when they match a specific host. How could I do that ?

Stuck trying to proxy DNS through a Wireguard tunnel

My setup has a VPS connected to my Homelab using Wireguard (10.14.0.1 <->10.14.0.6). In my homelab I have a DNS server on 192.168.1.59. I want to forward/proxy all DNS requests from 10.14.0.6 to the 192.168.1.59 device. After some fiddling around I got Caddy l4 working in Docker/Docker Compose and the example http json file works as intended.

Calling dig @10.14.0.6 google.com from the Docker host or from the otherside of the tunnel results in a timeout:

Docker seems to be exposing the port to the correct interface:

What have I done wrong? Why is Caddy not proxy-ing the DNS request upstream to 192.168.1.59 ?

I have made the following Dockerfile:

FROM caddy:builder AS builder

RUN xcaddy build \
    --with github.com/mholt/caddy-l4

FROM caddy:latest

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

CMD ["caddy", "run", "--config", "/etc/caddy/Caddyfile.json"]

And the following Docker Compose file:

version: "3.7"

services:
  caddy:
    build: .
    restart: unless-stopped
    ports:
      - "10.14.0.6:53:53"
    volumes:
      - ./caddy:/etc/caddy

And the folowing config file: ./caddy/Caddyfile.json

{
        "apps": {
                "layer4": {
                        "servers": {
                                "dns-example": {
                                        "listen": ["0.0.0.0:53"],
                                        "routes": [
                                                {
                                                        "handle": [
                                                                {
                                                                        "handler": "proxy",
                                                                        "proxy_protocol": "v2",
                                                                        "upstreams": [
                                                                                {"dial": ["192.168.1.59:53"]}
                                                                        ]
                                                                }
                                                        ]
                                                }
                                        ]
                                }
                        }
                }
        }
}

The container is able to reach 192.168.1.59 as I can ping the host from inside the container (exec).

[FEATURE]: Add support for lookup_srv for upstreams discovery

The same way as the caddy http app, it would be nice to have not only the dial option but also the lookup_srv for upstream service discovery.

Feature Request: Support Caddyfile

I'm creating this issue just to be able to gauge support for Caddyfile being supported in addition to JSON. I'm sure this is already a niche plugin/module and the desire for Caddyfile support is even more niche within that niche...but nonetheless.

Does caddy-l4 proxy support UDP?

nginx configuration example：

.......
stream {
    map $ssl_preread_server_name $backend_name {
        vw.xx.yy  vless;
        tw.xx.yy trojan;
        default   https;
    }
    upstream vless {
        server unix:/dev/shm/uds5443.sock;
    }
    upstream trojan {
        server unix:/dev/shm/uds6443.sock;
    }
    upstream https {
        server 127.0.0.1:7443;
    }
    server {
        listen 443;
        ssl_preread on;
        proxy_pass $backend_name;
        proxy_protocol on;
    }
    server {
        listen 443 udp;
        proxy_pass 127.0.0.1:7443;
        proxy_timeout 20s;
    }
}

Change to caddy configuration example：

.......
    "layer4": {
      "servers": {
        "sni": {
          "listen": [":443"],
          "routes": [{
            "match": [{
              "tls": {
                "sni": ["vw.xx.yy"]
              }
            }],
            "handle": [{
              "handler": "proxy",
              "upstreams": [{
                "dial": ["unix//dev/shm/uds5443.sock"]
              }],
              "proxy_protocol": "v1"
            }]
          },
          {
            "match": [{
              "tls": {
                "sni": ["tw.xx.yy"]
              }
            }],
            "handle": [{
              "handler": "proxy",
              "upstreams": [{
                "dial": ["unix//dev/shm/uds6443.sock"]
              }],
              "proxy_protocol": "v1"
            }]
          },
          {
            "match": [{
              "tls": {}
            }],
            "handle": [{
              "handler": "proxy",
              "upstreams": [{
                "dial": ["127.0.0.1:7443"]
              }],
              "proxy_protocol": "v1"
            }]
          }]
        },
        "udp": {
          "listen": ["udp/:443"],
          "routes": [{
            "handle": [{
              "handler": "proxy",
              "upstreams": [{
                "dial": ["udp/127.0.0.1:7443"]
              }]
            }]
          }]
        }
      }
    },
.......

The caddy SNI configuration works normally, but the UDP proxy is invalid. Does caddy-l4 proxy support UDP? Is this part of the configuration correct?

Compare to socat

Hi there,

I have a question about this module and the final goal. From my interpretation this is a little similar to some features of socat (proxy for TCP and UDP) but excluding other socket types (Unix, files, etc).

Is this true or am I completely mistaken?

Cannot proxy postgres database connections through caddy l4

Hello @mholt
I am trying to proxy the postgres database connection via caddy-l4. But something is not working out

I have tried the same thing with Nginx and it's working, This is my Nginx config. I have got this nginx config from here https://stackoverflow.com/questions/35545648/tcp-proxy-to-postgres-database-as-an-upstream-server-in-nginx/62786911#62786911

stream {
    upstream postgres {
        server localhost:5432;
    }

    server {
        listen 5000 so_keepalive=on;
        proxy_pass postgres;
    }
}

This is my caddy-l4 config

{
        "apps": {
                "layer4": {
                        "servers": {
                                "imap-example": {
                                        "listen": ["localhost:5000"],
                                        "routes": [
                                                {
                                                        "handle": [
                                                                {
                                                                        "handler": "proxy",
                                                                        "proxy_protocol": "v2",
                                                                        "upstreams": [
                                                                                {"dial": ["localhost:5432"]}
                                                                        ]
                                                                }
                                                        ]
                                                }
                                        ]
                                }
                        }
                }
        }
}

I am using the latest version of caddy-l4, when I don't see any logs also

sharad@DESKTOP-OTLHI96:~$ ./caddy-l4 run --config Caddyfile.json
2021/09/03 14:25:50.337 INFO    using provided configuration    {"config_file": "Caddyfile.json", "config_adapter": ""}
2021/09/03 14:25:50.338 INFO    admin   admin endpoint started  {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2021/09/03 14:25:50.340 INFO    autosaved config (load with --resume flag)      {"file": "/home/sharad/.config/caddy/autosave.json"}
2021/09/03 14:25:50.340 INFO    serving initial configuration

Add mTLS client attributes to matching/routing

The http app exposes a number of placeholders that'd be equally valuable for the L4 side (http.request.tls.*). Right now matching only works for sni, alpn, and remote_ip.

Use-case:
I'd like to be able to have an mTLS stripping proxy that can also validate the client's certificate has a given SAN in addition to being validly signed by the CA. Right now, that doesn't appear possible.

(Apologies if I have screwed up the nomenclature here, I'm a bit new to Caddy)

l4proxy: Upstream connections may be closed before reading from them

Problem

The close of upstream connections:

caddy-l4/modules/l4proxy/proxy.go

Lines 173 to 178 in 31d74d3

    
           // make sure upstream connections all get closed 
        
           defer func() { 
        
           	for _, conn := range upConns { 
        
           		conn.Close() 
        
           	} 
        
           }()

may happen before the reading of replies from the upstream connections:

caddy-l4/modules/l4proxy/proxy.go

Lines 256 to 263 in 31d74d3

    
           _, err := io.Copy(down, up) 
        
           if err != nil && atomic.LoadInt32(&done) == 0 { 
        
           	h.logger.Error("upstream connection", 
        
           		zap.String("local_address", up.LocalAddr().String()), 
        
           		zap.String("remote_address", up.RemoteAddr().String()), 
        
           		zap.Error(err), 
        
           	) 
        
           }

As a result, the replies from upstream servers will never be forwarded to the downstream clients.

Reproducer

Given a slow Echo server:

package main

import (
	"io"
	"log"
	"net"
	"time"
)

type SlowConn struct {
	net.Conn
}

func (c SlowConn) Write(p []byte) (n int, err error) {
	time.Sleep(5 * time.Millisecond) // Write slowly
	n, err = c.Conn.Write(p)
	return
}

func main() {
	ln, err := net.Listen("tcp", ":7070")
	if err != nil {
		log.Panicln(err)
	}
	defer ln.Close()

	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Panicln(err)
		}

		go handle(conn)
	}
}

func handle(conn net.Conn) {
	c := SlowConn{Conn: conn}
	if _, err := io.Copy(c, c); err != nil {
		log.Printf("err: %v\n", err)
	}
	conn.Close()
}

Everything is OK:

$ echo "hi" | nc localhost 7070
hi

Run Caddy with the following JSON config:

{                                       
        "apps": {                       
                "layer4": {                     
                        "servers": {                    
                                "srv0": {
                                        "listen": [":8080"],
                                        "routes": [                             
                                                {                       
                                                        "handle": [     
                                                                {
                                                                        "handler": "proxy",
                                                                        "upstreams": [
                                                                                {"dial": ["localhost:7070"]}
                                                                        ]
                                                                }
                                                        ]
                                                }
                                        ]
                                }
                        }
                }
        }
}

Nothing will be displayed:

$ echo "hi" | nc localhost 8080

Cannot create TLS connection over layer 4 connection between Apache/PHP & MariaDB.

I am currently using caddy-l4 as a proxy between my apache web server and MariaDB. My problem is that the connection isn't enforcing a TLS connection and I have tried everything I can think of to get it to work. This is my caddy-l4 configuration for my Maria DB database / Apache server:

		"srv1": {
				"listen": ["tcp/caddy:50000"],
				"routes": [
					{
						"handle": [
							{
								"handler": "proxy",
								"upstreams": [
									{"dial": ["tcp/database:3306"]}
								]
							}
						]	
					}
				]
			}

This is where I am loading the certifications:

	  "tls": {
		"certificates": {
		  "load_files": [
			{
			  "certificate": "/certs/localhost-cert.pem",
			  "key": "/certs/localhost-key.pem",
			  "tags": [
				"cert0"
			  ]
			}
		  ]
		}
	  }

And this is where I am routing HTTP traffic through caddy-l4, which is enforcing the TLS connection correctly. When I access the website via localhost I see the lock indicating that the certificate is valid. I am terminating the TLS connection and proxying the connection to webserver:80. The 'alpn' connection_policy was necessary for this to work correctly:

		"srv0": {
				"listen": ["0.0.0.0:443"],
				"routes": [
					{
						"handle": [
							{
								"handler": "tls",
								"connection_policies": [
									{"alpn": ["http/1.1"]}
								]
							},
							{
								"handler": "proxy",
								"upstreams": [
									{"dial": ["webserver:80"]}
								]
							}
						]	
					}
				]
			}

As you can see, the 0.0.0.0:443 route is enforcing TLS correctly because the traffic is HTTP. However, caddy:50000 is proxying the information correctly but without the TLS connection. The reason I believe this is because when I use the TLS termination or the TLS match handlers, the connection just hangs and doesn't work.

What else can I try here to continue troubleshooting this problem? Thank you.

UDP example

Hi Matt,

Do you have an example of a json config with UDP proxy ?

Thanks
Vincent

SNI matching after proxy_protocol handler does not work

Hi,
thanks for this awesome project, it makes very cool configurations possible.
But I think I found a bug in the way the proxy_protocol handler works. It seems not to be possible to do sni matching after the proxy_protocol handler was executed.
I would expect the following config to work, but instead I see this in the logs:

debug layer4.handlers.proxy_protocol  received the PROXY header       {"remote": "127.0.0.1:37055", "local": "127.0.0.1:10443"}
debug layer4.matchers.tls     matched {"remote": "127.0.0.1:37055", "server_name": "localhost"}
error    layer4  handling connection     {"error": "tls: first record does not look like a TLS handshake"}
debug layer4  connection stats        {"remote": "127.0.0.1:37055", "read": 230, "written": 0, "duration": 0.0081304}

caddy.json

{
  "admin": {
    "disabled": true
  },
  "logging": {
    "logs": {
      "default": {"level":"DEBUG", "encoder": {"format":"console"}}
    }
  },
  "apps": {
    "tls": {
      "certificates": {
        "automate": ["localhost"]
      },
      "automation": {
        "policies": [{
          "subjects": ["localhost"],
          "issuers": [{
            "module": "internal"
          }]
        }]
      }
    },
    "layer4": {
      "servers": {
        "https": {
          "listen": ["0.0.0.0:10443"],
          "routes": [
            {
              "match": [
                {"proxy_protocol": {}}
              ],
              "handle": [
                {
                  "handler": "proxy_protocol",
                  "timeout": "2s",
                  "allow": ["127.0.0.1/32"]
                }
              ]
            },
            {
              "match": [
                {"tls": {"sni": ["localhost"]}}
              ],
              "handle": [
                {"handler": "tls"},
                {
                  "handler": "proxy",
                  "upstreams": [{"dial": ["127.0.0.1:10080"]}]
                }
              ]
            }
          ]
        }
      }
    },
    "http": {
      "servers": {
        "backend": {
          "allow_h2c": true,
          "listen": ["127.0.0.1:10080"],
          "routes": [
            {
              "handle": [{
                "handler": "static_response",
                "status_code": "200",
                "body": "Hello World\n",
                "headers": {
                  "Content-Type": ["text/plain"]
                }
              }]
            }
          ]
        }
      }
    }
  }
}

You should be able to run this config yourself since it only uses localhost.
For testing I used curl -v --insecure --haproxy-protocol https://localhost:10443

When the match key for the second route is completely removed, so it matches all requests the above curl works and prints Hello World. But in my real config I can not use a match all route since I have to distribute requests to different servers.

Loading a config file twice causes admin app to crash

If I post a config file to the admin endpoint: localhost:2019/load it works fine.

If I post a second time, with a slightly altered file, I get this:

2020/09/30 15:06:04 http: panic serving 172.17.0.1:51590: runtime error: hash of unhashable type []string
goroutine 37 [running]:
net/http.(*conn).serve.func1(0xc0001da1e0)
        net/http/server.go:1801 +0x147
panic(0x15858c0, 0xc0002ff0b0)
        runtime/panic.go:975 +0x3e9
github.com/caddyserver/caddy/v2.(*UsagePool).Delete(0xc0006e00a0, 0x14f95c0, 0xc000814cb0, 0x70000c0000ac960, 0xffffffffffffffff, 0x7fc53f789258)
        github.com/caddyserver/caddy/[email protected]/usagepool.go:164 +0x8a
github.com/mholt/caddy-l4/modules/l4proxy.(*Handler).Cleanup(0xc0002dc310, 0x168a560, 0xc0002dc310)
        github.com/mholt/[email protected]/modules/l4proxy/proxy.go:293 +0x85
github.com/caddyserver/caddy/v2.NewContext.func1()
        github.com/caddyserver/caddy/[email protected]/context.go:66 +0x1c5
github.com/caddyserver/caddy/v2.unsyncedStop(0xc00068ea00)
        github.com/caddyserver/caddy/[email protected]/caddy.go:462 +0x19b
github.com/caddyserver/caddy/v2.unsyncedDecodeAndRun(0xc0006a0000, 0x34b, 0x380, 0x7, 0xc0005e3cb0)
        github.com/caddyserver/caddy/[email protected]/caddy.go:259 +0x139
github.com/caddyserver/caddy/v2.changeConfig(0x17587ae, 0x4, 0x1762a2f, 0x7, 0xc000800000, 0x63a, 0xe00, 0x600, 0x0, 0x0)
        github.com/caddyserver/caddy/[email protected]/caddy.go:156 +0x4da
github.com/caddyserver/caddy/v2.Load(...)
        github.com/caddyserver/caddy/[email protected]/caddy.go:102
github.com/caddyserver/caddy/v2/caddyconfig.adminLoad.handleLoad(0x1a0b580, 0xc0007e2240, 0xc000682600, 0x0, 0x0)
        github.com/caddyserver/caddy/[email protected]/caddyconfig/load.go:136 +0x508
github.com/caddyserver/caddy/v2.AdminHandlerFunc.ServeHTTP(0xc0004abb50, 0x1a0b580, 0xc0007e2240, 0xc000682600, 0xc0007376a8, 0x40d9d0)
        github.com/caddyserver/caddy/[email protected]/admin.go:793 +0x44
github.com/caddyserver/caddy/v2.AdminConfig.newAdminHandler.func2.1(0x1a0b580, 0xc0007e2240, 0xc000682600)
        github.com/caddyserver/caddy/[email protected]/admin.go:123 +0x97
net/http.HandlerFunc.ServeHTTP(0xc000095080, 0x1a0b580, 0xc0007e2240, 0xc000682600)
        net/http/server.go:2042 +0x44
github.com/caddyserver/caddy/v2.instrumentHandlerCounter.func1(0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        github.com/caddyserver/caddy/[email protected]/metrics.go:46 +0xad
net/http.HandlerFunc.ServeHTTP(0xc0006e1460, 0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        net/http/server.go:2042 +0x44
net/http.(*ServeMux).ServeHTTP(0xc0006e8b80, 0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        net/http/server.go:2417 +0x1ad
github.com/caddyserver/caddy/v2.adminHandler.serveHTTP(0x0, 0xc0006deca0, 0x1, 0x1, 0xc0006e8b80, 0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        github.com/caddyserver/caddy/[email protected]/admin.go:363 +0xe8
github.com/caddyserver/caddy/v2.adminHandler.ServeHTTP(0x0, 0xc0006deca0, 0x1, 0x1, 0xc0006e8b80, 0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        github.com/caddyserver/caddy/[email protected]/admin.go:321 +0x605
net/http.serverHandler.ServeHTTP(0xc0002fc8c0, 0x1a0d580, 0xc0002fc0e0, 0xc000682600)
        net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc0001da1e0, 0x1a12380, 0xc0005f0080)
        net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
        net/http/server.go:2969 +0x36c

Here's two config files that will cause the crash. All that's changed in the second is a subdomain:

{
	"admin": {
		"listen": "0.0.0.0:2019"
	},
	"logging": {
		"logs": {
			"default": {
				"level": "DEBUG"
			}
		}
	},
	"apps": {
		"tls": {
			"certificates": {
				"automate": ["mongo.127.0.0.1.nip.io"]
			},
			"automation": {
				"policies": [{
						"issuer": {
							"module": "internal"
						},
						"key_type": "rsa4096"
					}
				]
			}
		},
		"http": {
			"servers": {
				"http-srv0": {
					"listen": ["127.0.0.1:444", "127.0.0.1:80"],
					"routes": [{
							"match": [{
									"host": ["webserver.127.0.0.1.nip.io"]
								}
							],
							"handle": [{
									"handler": "subroute",
									"routes": [{
											"handle": [{
													"handler": "reverse_proxy",
													"upstreams": [{
															"dial": "192.168.1.38:8080"
														}
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		},
		"layer4": {
			"servers": {
				"layer4-srv0": {
					"listen": ["0.0.0.0:443"],
					"routes": [{
							"match": [{
									"tls": {
										"sni": ["mongo.127.0.0.1.nip.io"]
									}
								}
							],
							"handle": [{
									"handler": "tls"
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["192.168.1.38:27017"]
										}
									]
								}
							]
						}, {
							"match": [{
									"tls": {}
								}
							],
							"handle": [{
									"handler": "proxy",
									"upstreams": [{
											"dial": ["127.0.0.1:444"]
										}
									]
								}
							]
						}
					]
				}
			}
		}

	}
}

"webserver" changed to "tweak"

{
	"admin": {
		"listen": "0.0.0.0:2019"
	},
	"logging": {
		"logs": {
			"default": {
				"level": "DEBUG"
			}
		}
	},
	"apps": {
		"tls": {
			"certificates": {
				"automate": ["mongo.127.0.0.1.nip.io"]
			},
			"automation": {
				"policies": [{
						"issuer": {
							"module": "internal"
						},
						"key_type": "rsa4096"
					}
				]
			}
		},
		"http": {
			"servers": {
				"http-srv0": {
					"listen": ["127.0.0.1:444", "127.0.0.1:80"],
					"routes": [{
							"match": [{
									"host": ["tweak.127.0.0.1.nip.io"]
								}
							],
							"handle": [{
									"handler": "subroute",
									"routes": [{
											"handle": [{
													"handler": "reverse_proxy",
													"upstreams": [{
															"dial": "192.168.1.38:8080"
														}
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		},
		"layer4": {
			"servers": {
				"layer4-srv0": {
					"listen": ["0.0.0.0:443"],
					"routes": [{
							"match": [{
									"tls": {
										"sni": ["mongo.127.0.0.1.nip.io"]
									}
								}
							],
							"handle": [{
									"handler": "tls"
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["192.168.1.38:27017"]
										}
									]
								}
							]
						}, {
							"match": [{
									"tls": {}
								}
							],
							"handle": [{
									"handler": "proxy",
									"upstreams": [{
											"dial": ["127.0.0.1:444"]
										}
									]
								}
							]
						}
					]
				}
			}
		}

	}
}

hello, does ii support quic protocol?

Listen on domain

Hi,
I'd like to use Caddy as proxy for multiple MySQL server and others services in one server. I want to use same standard port 3306. The idea is use domains to decide which port should be used (1st MySQL 9006, 2nd MySQL 9007 etc.). I tried this config.json, but I've got an error run: loading initial config: loading new config: layer4 app module: start: listen tcp 10.255.254.17:13222: bind: cannot assign requested address

Do you have an idea how to fix it? Thanks.

{
  "apps": {
    "layer4": {
      "servers": {
        "mysql-dev": {
          "listen": [
            "mysql-dev.mydomain.com:3306"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "proxy",
                  "upstreams": [
                    {
                      "dial": [
                        "10.0.0.40:9006"
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        },
        "mysql-test": {
          "listen": [
            "mysql-test.mydomain.com:3306"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "proxy",
                  "upstreams": [
                    {
                      "dial": [
                        "10.0.0.40:9007"
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      }
    }
  }
}

[Question] Automatic HTTP->S redirects with the `http` and `layer4` apps

Hi Matt, Francis,

I'm trying to use the Layer 4 plugin as a multiplexer by forwarding requests over port 443 to their relevant destinations while continuing to leverage Caddy's more traditional reverse proxy functionality. However, I am unable to get the HTTP -> HTTPS redirects to work with the standard http app. I used this post in the Caddy community as a guide.

I have boiled down the troubling configuration to an extremely basic form.

logging:
  logs:
    default:
      level: DEBUG
apps:
  layer4:
    servers:
      srv0:
        listen:
        - ":443"
        routes:
        - handle:
          - handler: proxy
            upstreams:
            - dial:
              - 127.0.0.1:1337 # simply forward everything to Layer 7
  http:
    https_port: 1337 # since port 443 is occupied, use 1337 here
    servers:
      srv1:
        listen:
        - ":80"
        - "127.0.0.1:1337"
        logs: {}
        routes:
        - match:
          - host:
            - subdomain.example.tld
          handle:
          - handler: subroute
            routes:
            - handle:
              - handler: reverse_proxy
                upstreams:
                - dial: localhost:1234
          terminal: true

I run this configuration by using

$ xcaddy build \
  --with github.com/mholt/caddy-l4 \
  --with github.com/abiosoft/caddy-yaml
$ caddy run --config config.yaml --adapter yaml

My test cases include an HTTP curl, and an HTTPS curl. I would expect the HTTP curl to produce a 308 Permanent Redirect, but it does not. Note that Caddy's logs do contain a "msg":"enabling automatic HTTP->HTTPS redirects".

$ curl -vvI http://subdomain.example.tld
*   Trying XX.XX.XXX.XXX:80...
* Connected to subdomain.example.tld (XX.XX.XXX.XXX) port 80 (#0)
> HEAD / HTTP/1.1
> Host: subdomain.example.tld
> User-Agent: curl/7.80.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
< Content-Type: text/html
Content-Type: text/html
< Date: Mon, 20 Dec 2021 02:53:58 GMT
Date: Mon, 20 Dec 2021 02:53:58 GMT
< location: /loginRedir
location: /loginRedir
< server: Caddy
server: Caddy
< server: SomeOtherServer
server: SomeOtherServer

< 
* Connection #0 to host subdomain.example.tld left intact

$ curl -vvI https://subdomain.example.tld
*   Trying XX.XX.XXX.XXX:443...
* Connected to subdomain.example.tld (XX.XX.XXX.XXX) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=subdomain.example.tld
*  start date: Dec 20 01:39:41 2021 GMT
*  expire date: Mar 20 01:39:40 2022 GMT
*  subjectAltName: host "subdomain.example.tld" matched cert's "subdomain.example.tld"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e7f03029d0)
> HEAD / HTTP/2
> Host: subdomain.example.tld
> user-agent: curl/7.80.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 303 
HTTP/2 303 
< content-type: text/html
content-type: text/html
< date: Mon, 20 Dec 2021 02:56:08 GMT
date: Mon, 20 Dec 2021 02:56:08 GMT
< location: /loginRedir
location: /loginRedir
< server: Caddy
server: Caddy
< server: SomeOtherServer
server: SomeOtherServer

< 
* Connection #0 to host subdomain.example.tld left intact

Let me know if you have any ideas about this. Thanks for your help!

[Question]: layer4 module seems cannot match http/2 or gRPC's header while caddy server does ?

The layer4 module seems cannot reconizge grpc/http2 connections with configuration below.
There's no upstream record in logs :

        { "handle":[{"handler":"tls"}]},
        { "handle":[
            {"handler":"proxy","upstreams":[{"dial":["127.0.0.1:8003"]}]}
          ],
          "match":[{"http":[{"header":{"Content-Type":["application/grpc*"]}}]}
          ]
        { "handle":[
            {"handler":"proxy","upstreams":[{"dial":["127.0.0.1:8003"]}]}
          ],
          "match":[{"http":[{"path":["/grpc"]}]}
          ]
        },

Both matchers does not work.

I only find a way to upstream gRPC now is using layer4 to proxy all connections (incl gRPC) into caddy server and then use reverse_proxy by match Content-Type or path.
example:

          { "handle":[{
              "handler":"reverse_proxy",
              "flush_interval":-1,
              "upstreams":[{"dial":gRPC Server"}],
              "transport":{"protocol":"http","versions":["h2c"]}}],
            "match":[{"header":{"Content-Type":["application/grpc*"]},"path":["/grpc"]}]},

So what's the difference between caddy server and layer4 in gRPC's header matcher ?
May I upstream gRPC connections by using layer4 module ?

Caddy-l4 proxy support UDP problems

This question comes from #23
bug report：
root@VM-4-15-debian:~# journalctl -u caddy --no-pager
-- Logs begin at Thu 2021-06-17 04:32:40 CST, end at Thu 2021-06-17 04:46:38 CST. --
Jun 17 04:32:46 VM-4-15-debian systemd[1]: Starting Caddy...
Jun 17 04:32:47 VM-4-15-debian caddy[957]: {"level":"warn","ts":1623875567.0497217,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"neither $XDG_CONFIG_HOME nor $HOME are defined"}
Jun 17 04:32:47 VM-4-15-debian caddy[957]: caddy.HomeDir=.
Jun 17 04:32:47 VM-4-15-debian caddy[957]: caddy.AppDataDir=./caddy
Jun 17 04:32:47 VM-4-15-debian caddy[957]: {"level":"warn","ts":1623875567.372122,"msg":"unable to determine directory for user configuration; falling back to current directory","error":"neither $XDG_CONFIG_HOME nor $HOME are defined"}
Jun 17 04:32:47 VM-4-15-debian caddy[957]: caddy.AppConfigDir=./caddy
Jun 17 04:32:47 VM-4-15-debian caddy[957]: caddy.ConfigAutosavePath=caddy/autosave.json
Jun 17 04:32:47 VM-4-15-debian caddy[957]: caddy.Version=v2.4.2 h1:chB106RlsIaY4mVEyq9OQM5g/9lHYVputo/LAX2ndFg=
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.GOOS=linux
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.GOARCH=amd64
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.Compiler=gc
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.NumCPU=1
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.GOMAXPROCS=1
Jun 17 04:32:47 VM-4-15-debian caddy[957]: runtime.Version=go1.16.5
Jun 17 04:32:47 VM-4-15-debian caddy[957]: os.Getwd=/
Jun 17 04:32:47 VM-4-15-debian caddy[957]: LANG=en_US.utf8
Jun 17 04:32:47 VM-4-15-debian caddy[957]: PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
Jun 17 04:32:47 VM-4-15-debian caddy[957]: NOTIFY_SOCKET=/run/systemd/notify
Jun 17 04:32:47 VM-4-15-debian caddy[957]: LOGNAME=nobody
Jun 17 04:32:47 VM-4-15-debian caddy[957]: USER=nobody
Jun 17 04:32:47 VM-4-15-debian caddy[957]: INVOCATION_ID=56f8500c844c4f1c83185325bdb808c4
Jun 17 04:32:47 VM-4-15-debian caddy[957]: JOURNAL_STREAM=9:15996
Jun 17 04:32:47 VM-4-15-debian caddy[957]: {"level":"info","ts":1623875567.3747764,"msg":"using provided configuration","config_file":"/usr/local/etc/caddy/caddy.json","config_adapter":""}
Jun 17 04:32:47 VM-4-15-debian caddy[957]: {"level":"info","ts":1623875567.377124,"msg":"redirected default logger","from":"stderr","to":"/var/log/caddy/access.log"}
Jun 17 04:32:50 VM-4-15-debian systemd[1]: Started Caddy.
Jun 17 04:32:50 VM-4-15-debian caddy[957]: 2021/06/17 04:32:50 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
Jun 17 04:32:50 VM-4-15-debian caddy[957]: 2021/06/17 04:32:50 [DEBUG] Fake-closing underlying packet conn

feature request: reverse proxy ssh via username

It would be cool if the ssh matcher allowed matching against a user, and if the handler allowed you to pass the request on to a different user. ie,

[email protected]   -> caddy-l4 -> [email protected]
[email protected] -> caddy-l4 -> [email protected]

basically exactly what tg123/sshpiper does.
i think itd be cool if caddy-l4 could support reverse proxying ssh in this manner by default.

apologies in advance if this doesnt seem feasible or its outside of the scope of this plugin !

AMQP Connection not possible

While connecting through the proxy a connection to a RabbitMQ Server via AMQP Protocol was not possible.
Error Message given is:
com.rabbitmq.client.MalformedFrameException: AMQP protocol version mismatch; we are version 0-9-1, server sent signature 0,0,9,1

It seems like a problem in initial handshake. Version is correct on both sides and connection can be established without problems not using caddy-l4 proxy

Log remote addrs for layer4 servers

I'm currently trying out caddy-l4 for SSL passthrough to backend service and trying to debug remote address problems throughout the stack.

In this process, I noticed that even with top level logging config set to DEBUG my logs I see logs from my routes but they don't contain any information about the remote client which would be useful for ascertaining what IP caddy thinks is hitting me and implementing remote IP based routing.

logging:
  logs:
    default:
      level: "DEBUG"

problems with smallstep

Hi, I am not sure if this is a Problem with caddy or caddy-l4.

I pulled this repo, entered project directory and ran: xcaddy list-modules --versions

the output was pretty normal (downloading all the modules), thill this happened:

# github.com/smallstep/certificates/authority/provisioner
/home/XXX/go/pkg/mod/github.com/smallstep/[email protected]/authority/provisioner/jwk.go:155:24: assignment mismatch: 3 variables but x509util.SplitSANs returns 4 values
/home/XXX/go/pkg/mod/github.com/smallstep/[email protected]/authority/provisioner/x5c.go:197:24: assignment mismatch: 3 variables but x509util.SplitSANs returns 4 values
2020/08/25 16:26:26 [INFO] Cleaning up temporary folder: /tmp/buildenv_2020-08-25-1625.642148159
2020/08/25 16:26:26 [ERROR] exit status 2

TCP proxying not working as expected

Hi Matt!

First off - love Caddy and its premise!

I'm running Caddy v2.4.3 (h1:Y1FaV2N4WO3rBqxSYA8UZsZTQdN+PwcoOcAiZTM8C0I=) on Ubuntu 18.04 after doing a custom build on the website, making sure I added caddy2-proxyprotocol and caddy-l4 as addon features. Here the basic idea is to have a simply proxy listening on port 3306, for MySQL on another server. Here's my JSON config:

{
   "apps":{
      "layer4":{
         "servers":{
            "mysql":{
               "listen":[
                  ":3306"
               ],
               "routes":[
                  {
                     "handle":[
                        {
                           "handler":"proxy",
                           "proxy_protocol":"v2",
                           "upstreams":[
                              {
                                 "dial":[
                                    "tcp/10.0.0.3:3306"
                                 ]
                              }
                           ]
                        }
                     ]
                  }
               ]
            }
         }
      }
   }
}

The MySQL server is reachable from the server running caddy, so the network isn't an issue. But running telnet produces an odd error:

$ telnet 127.0.0.1 3306
Connected to localhost.
Escape character is '^]'.
[
5.7.34-0ubuntu0.18.04.1E"n �!b,dMj	HD3=mysql_native_password!#08S01Got packets out of order^CConnection closed by foreign host.

So it's able to proxy requests to the MySQL server - but something's going awry with the packets. Trying to access this from a MySQL client throws the following error:

Lost connection to MySQL server at 'reading authorization packet', system error: 0

I went through MySQL's documentation, thinking maybe there's something up with MySQL - although this hasn't happened when I try accessing the MySQL server directly. Anyway, I tried tweaking connection timeouts, but to no avail. I then tried running the same configuration against a Redis instance, substituting the correct IPs and ports in the upstream - it didn't work, redis-cli refused to connect to Caddy.

At this point I'm fairly certain it's definitely something up with the configuration I've defined - I adapted it from the exampes in the README. Would love some tips :)

EDIT - I've also tried various combinations of using/not using the tcp/ prefixes in the listener and the handlers - I'm afraid it doesn't make a difference.

Pass non terminated (SNI matched) TLS connection to http app

I've recently been looking into improving a setup that uses NGINX streams to accept (but not terminate) incoming TLS connections and proxy them to two caddy instances depending on the SNI (one of which running on the same host).

For the remove instance its obviously required to use the PROXY protocol to retain remote IP metadata and the overhead of an outgoing TCP connection is also necessary, however for the local caddy both of those issues are just design issues with the setup.

In a perfect world I would have one caddy which handles incoming traffic and before terminating TLS can decide to proxy a connection instead of handling it itself based on the SNI. Keeping the connection in process obviously benefits performance but also removes the need to wrap everything in the PROXY protocol.

While exploring I came across various approaches which went into the right direction but didn't fully work:

A listener wrapper seems like an obvious candidate but after successfully implementing ClientHello reading my efforts came to a halt due to what I far is caddy simply not being designed to split off / proxy connections on that level.

To my understanding this amazing project basically can be used as a drop in replacement for the NGINX streams process, however the requirement to use the PROXY protocol for connection metadata still remains. The performance can probably be improved by using unix domain sockets for on device proxying but to me its still rather unsatisfying.

The missing link here - and what I think would be a great addition in general - is being able to pass a connection to a different caddy app in process (provided that the protocols are compatible). With such a feature caddy-l4 would simply be able to "dial" http and let it handle TLS termination and so on while retaining all connection metadata (maybe even expose L4 variables).

It could be that im missing something here entirely or that there's no measurable (performance) impact (besides convenience) to support something this. I'm grateful for any pointers or ideas where to start looking / implementing

Is this tool stable enough in the production environment?

I learned about this project from the comment here, and it seems that it's similar to but more powerful than the nginx's stream module. But I'm not sure whether this project is stable enough for the product use scenario. Any hints will be highly appreciated.

Regards,
HZ

[Feature] close downstream if upstream is closed

HI:

I'm using Caddy as a layer4 load balancer for Kubelet to Kube-APIServer. And I found Kubelet could be hang in some time.

After days of digging, I think I found the reason:
In the scenario of Kubernetes, the Client uses Watch&List to describe resources changing events. This means that kube-apiserver returns all resources and pushes events only if resources changed. however, after kube-apiserver restarted, the upstream of Caddy is closed, but the downstream is still connected. At the same time, the client is still waiting for events to come which is impossible. And the worst thing is the client will never find out this connection should be reset.

Kubernetes fix this issue in 1.20 in kubernetes/client-go#374. but in some case, we can't upgrade to the latest version so quickly.

Here is my proposal:
Maybe we can add an option to the LoadBalancer struct, such as DisconnectDownstreamPolicy. And the value could be Never、OnAnyUpsteamClosed or OnAllUpstreamClosed. the default value is Never which keeps the same behavior as current.

How does this sound to you? And If you are busy, I'm ok to submit a Pull&Request and wait for you to review it. the poc change is working for me. xuzhenglun@f013a43

Certificate obtaining does not seem to work for tcp

Hello, how am I supposed to obtain certificates over tcp ? If I bind to just ":443", it binds to tcp6, and that is unacceptable.
If I bind to tcp4 with "xxx.xx.xx.xxx:443", layer4 app still creates binding to tcp6 :::80, and certificate obtaining fails.

"issuer":"acme.zerossl.com-v2-DV90","error":"[mydomain] solving challenges: [mydomain] authorization took too long

$ sudo netstat -ntlp
tcp 0 0 172.16.42.228:2022 0.0.0.0:* LISTEN 12808/caddy
tcp 0 0 172.16.42.228:443 0.0.0.0:* LISTEN 12808/caddy
tcp6 0 0 :::80 :::* LISTEN 12808/caddy
tcp6 0 0 :::22 :::* LISTEN 914/sshd: /usr/sbin

Also, how much is this true ? hashicorp/nomad#10189

http.matchers does not work in caddy-l4

I'm going to reverse proxies HTTP to backend with proxy protocol for websocket with special path, but it seems http.matchers in caddy-l4 does not work even I just write one http match, here is my configuration:
Caddyfile.json

{
  "admin":{"disabled":true},
  "logging": {"logs": {
    "default":{
      "level":"DEBUG","encoder":{"format":"json"},
      "writer":{"filename":"/tmp/caddy.log","output":"file"},
      "exclude":["http.log.access.log0","http.log.access.log1"]},
    "log0":{
      "level":"DEBUG","encoder":{"format":"json"},
      "writer":{"filename":"/tmp/http.log","output":"file"},
      "include":["http.log.access.log0","http.log.access.log1"]}}},
  "apps": {
    "layer4":{"servers":{
      "stream0":{
        "listen":[":443"],
        "routes":[
          {
            "handle":[{"handler":"tls"},{"handler":"proxy","upstreams":[{"dial":["127.0.0.1:8001"]}],"proxy_protocol":"v2"}],
            "match":[
              { 
                "http":[
                  { 
                    "header":{
                      "Upgrade":["websocket"]
                    }
                  }
                ]
              }
            ]
          },
          { 
            "handle":[{"handler":"tls"},{"handler":"proxy","upstreams":[{"dial":["127.0.0.1:8002"]}],"proxy_protocol":"v2"}],
            "match":[
              { 
                "http":[
                  { 
                    "path":["/8002"]
                  }
                ]
              }
            ]
          },
          { "handle":[{"handler":"tls"},{"handler":"proxy","upstreams":[{"dial":["127.0.0.1:8000"]}],"proxy_protocol":"v2"}]}
        ]
      }
    }},
    "http": {"servers": {
      "http0":{
        "listen":[":80"],
        "logs":{"default_logger_name":"log0"},
        "routes":[{"handle":[{
          "handler":"static_response","headers":{"Location":["https://{http.request.host}{http.request.uri}"]},"status_code":301}]}]
      },
      "http1": {
        "allow_h2c":true,
        "listen":["127.0.0.1:8000"],
        "listener_wrappers":[{"wrapper":"proxy_protocol"}],
        "logs":{"default_logger_name":"log1"},
        "routes": [
          { "handle": [{
              "handler": "headers",
              "response": {"set": {
                "Strict-Transport-Security": ["max-age=31536000; includeSubDomains; preload"],
                "X-Content-Type-Options": ["nosniff"],
                "X-Frame-Options": ["SAMEORIGIN"],
                "X-Robots-Tag": ["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"]}}}]},
          { "handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"127.0.0.1:8003"}]}],
            "match":[{"header":{"Upgrade":["websocket"]},"path":["/8003"]}]},
          { "handle":[{"handler":"reverse_proxy","upstreams":[{"dial":"127.0.0.1:8004"}]}],
            "match":[{"header":{"Upgrade":["websocket"]},"path":["/8004"]}]},
          { "handle": [{"handler": "file_server","hide": ["/etc/caddy/Caddyfile.json"],"root": "/var/home"}]}
        ],
        "errors": {"routes": [
          { "handle": [{"handler": "rewrite","uri": "/index.html"}]},
          { "handle": [{"handler": "file_server","hide": ["/etc/caddy/Caddyfile.json"],"root": "/var/home"}]}
        ]}
      }
    }},
    "tls":{
      "certificates":{"automate":["domain.com"]},
      "automation":{"policies":[{"issuers":[{"module":"zerossl"}]}]}
    }
  }
}

caddy.log here:

{"level":"warn","ts":1631097143.2176166,"logger":"admin","msg":"admin endpoint disabled"}
{"level":"info","ts":1631097143.2184045,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"http0","http_port":80}
{"level":"info","ts":1631097143.2245347,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000393030"}
{"level":"debug","ts":1631097143.2974496,"logger":"layer4","msg":"listening","address":"tcp/[::]:443"}
{"level":"debug","ts":1631097143.3084638,"logger":"tls.cache","msg":"added certificate to cache","subjects":["mydomain.com"],"expiration":1638662399,"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097143.3085592,"logger":"http","msg":"starting server loop","address":"[::]:80","http3":false,"tls":false}
{"level":"debug","ts":1631097143.3086965,"logger":"http","msg":"starting server loop","address":"127.0.0.1:8000","http3":false,"tls":false}
{"level":"info","ts":1631097143.3086019,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy/.local/share/caddy"}
{"level":"info","ts":1631097143.3092394,"logger":"tls","msg":"finished cleaning storage units"}
{"level":"info","ts":1631097143.3093457,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
{"level":"info","ts":1631097143.3116603,"msg":"serving initial configuration"}
{"level":"info","ts":1631097143.3117192,"logger":"watcher","msg":"watching config file for changes","config_file":"/etc/caddy/Caddyfile.json"}
{"level":"debug","ts":1631097427.6923897,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097427.6924815,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097427.6924927,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097427.8528264,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:30464","server_name":"mydomain.com"}
{"level":"debug","ts":1631097427.8530192,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:30464","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097427.853516,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097427.8536227,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:30464","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Upgrade":["websocket"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Cf-Worker":["cf.workers.dev"],"Connection":["Upgrade"],"Accept-Encoding":["gzip"],"X-Forwarded-Proto":["https"],"Cf-Connecting-Ip":["IPIP"],"X-Forwarded-For":["IPIP"],"Cf-Ray":["68b78a29579eebd5-LAX"],"Sec-Websocket-Key":["go4E7uifiXyQGsqbXISc8w=="],"Sec-Websocket-Version":["13"],"Cf-Ew-Via":["15"],"Cdn-Loop":["cloudflare; subreqs=1"],"User-Agent":["Go-http-client/1.1"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097427.8536372,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097427.8536615,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097496.7244058,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097496.7245393,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097496.7245498,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097497.1746256,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:51993","server_name":"mydomain.com"}
{"level":"debug","ts":1631097497.1747851,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:51993","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097497.1765633,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8001","result":"/var/home/8001"}
{"level":"debug","ts":1631097497.176639,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:51993","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["websocket.key"],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097497.1766522,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097497.1766653,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097497.5578418,"logger":"layer4","msg":"connection stats","remote":"IPIP:51993","read":594,"written":10899,"duration":0.834689686}
{"level":"debug","ts":1631097498.0101454,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097498.0102103,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097498.0102196,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097498.4510102,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:51994","server_name":"mydomain.com"}
{"level":"debug","ts":1631097498.4511786,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:51994","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097498.4514956,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8001","result":"/var/home/8001"}
{"level":"debug","ts":1631097498.4515553,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:51994","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["websocket.key"],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097498.4515758,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097498.451593,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097498.8536725,"logger":"layer4","msg":"connection stats","remote":"IPIP:51994","read":594,"written":10898,"duration":0.845920969}
{"level":"debug","ts":1631097499.993334,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097499.99341,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097499.99342,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097500.4150765,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:51996","server_name":"mydomain.com"}
{"level":"debug","ts":1631097500.4152358,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:51996","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097500.42214,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8001","result":"/var/home/8001"}
{"level":"debug","ts":1631097500.422234,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:51996","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["kfmeD0dquEhvaEy1m7aemw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097500.4222484,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097500.4222617,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097500.8077915,"logger":"layer4","msg":"connection stats","remote":"IPIP:51996","read":594,"written":10899,"duration":0.816859118}
{"level":"debug","ts":1631097501.7454286,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097501.7455046,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097501.7455142,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097502.1724303,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:51998","server_name":"mydomain.com"}
{"level":"debug","ts":1631097502.1726205,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:51998","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097502.1735847,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8001","result":"/var/home/8001"}
{"level":"debug","ts":1631097502.1736438,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:51998","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["+c3aYixUa0vkLlk9uQjvpA=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097502.1736562,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097502.1736805,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097502.5579233,"logger":"layer4","msg":"connection stats","remote":"IPIP:51998","read":594,"written":10899,"duration":0.813359319}
{"level":"debug","ts":1631097503.5637326,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097503.5637982,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097503.5638072,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097503.9857993,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52000","server_name":"mydomain.com"}
{"level":"debug","ts":1631097503.985975,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52000","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097503.986999,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8001","result":"/var/home/8001"}
{"level":"debug","ts":1631097503.9870715,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52000","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["zG//cFwo7T/aWeonWGSunw=="]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097503.9870846,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097503.9870977,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097504.3665636,"logger":"layer4","msg":"connection stats","remote":"IPIP:52000","read":594,"written":10899,"duration":0.809525412}
{"level":"debug","ts":1631097509.3183365,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097509.3184264,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097509.3184366,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097509.7621365,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52006","server_name":"mydomain.com"}
{"level":"debug","ts":1631097509.7623522,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52006","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097509.7633018,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097509.7633681,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52006","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["NhGjlsHMA838N3IUhstn2A=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097509.7633922,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097509.7634058,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097510.1364229,"logger":"layer4","msg":"connection stats","remote":"IPIP:52006","read":594,"written":10900,"duration":0.825733884}
{"level":"debug","ts":1631097510.5461917,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097510.5462697,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097510.5462794,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097510.9690325,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52007","server_name":"mydomain.com"}
{"level":"debug","ts":1631097510.969252,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52007","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097510.9702334,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097510.970295,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52007","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["5j++1JTZbNd/nDnUR1Rh/w=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097510.9703076,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097510.9703207,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097511.2072554,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097511.2073295,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097511.2073395,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097511.353876,"logger":"layer4","msg":"connection stats","remote":"IPIP:52007","read":594,"written":10900,"duration":0.814580101}
{"level":"debug","ts":1631097511.3665795,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:37004","server_name":"mydomain.com"}
{"level":"debug","ts":1631097511.366737,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:37004","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097511.367178,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097511.3672657,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:37004","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Version":["13"],"Cf-Connecting-Ip":["IPIP"],"Cf-Worker":["cf.workers.dev"],"Accept-Encoding":["gzip"],"X-Forwarded-For":["IPIP"],"Cf-Ray":["68b78c3382f6eb41-LAX"],"X-Forwarded-Proto":["https"],"Connection":["Upgrade"],"Sec-Websocket-Key":["2soWKtUR1wZITQvbXmLfpQ=="],"Cf-Ew-Via":["15"],"Cdn-Loop":["cloudflare; subreqs=1"],"Upgrade":["websocket"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"User-Agent":["Go-http-client/1.1"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097511.3673146,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097511.3673325,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097512.3475347,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097512.347602,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097512.3476114,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097512.7838993,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52015","server_name":"mydomain.com"}
{"level":"debug","ts":1631097512.784145,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52015","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097512.7851133,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097512.7851973,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52015","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Key":["qpb+rkDSXTYfjAmisOUt6g=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097512.7852116,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097512.7852252,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097512.8735797,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097512.873662,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097512.8736732,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097513.0358064,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:39534","server_name":"mydomain.com"}
{"level":"debug","ts":1631097513.0360348,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:39534","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097513.0364077,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097513.0365279,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:39534","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"X-Forwarded-For":["IPIP"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Websocket-Key":["Bqxw7rNH9YZLO8Jis87Kdw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"Cf-Ew-Via":["15"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Cf-Ray":["68b78c3de60c368c-LAX"],"Cdn-Loop":["cloudflare; subreqs=1"],"Cf-Worker":["cf.workers.dev"],"Accept-Encoding":["gzip"],"X-Forwarded-Proto":["https"],"Cf-Connecting-Ip":["IPIP"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097513.036576,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097513.0365932,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097513.1738474,"logger":"layer4","msg":"connection stats","remote":"IPIP:52015","read":594,"written":10898,"duration":0.830936741}
{"level":"debug","ts":1631097514.001975,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097514.0021365,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097514.002149,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097514.4457097,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52019","server_name":"mydomain.com"}
{"level":"debug","ts":1631097514.445968,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52019","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097514.4477167,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097514.447783,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52019","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Connection":["Upgrade"],"Sec-Websocket-Key":["6bN+kohY7NXKXn7camr0UA=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097514.4477954,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097514.4478087,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097514.8454726,"logger":"layer4","msg":"connection stats","remote":"IPIP:52019","read":594,"written":10900,"duration":0.850479567}
{"level":"debug","ts":1631097515.4172804,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097515.4173877,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097515.4174001,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097515.5749407,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:42954","server_name":"mydomain.com"}
{"level":"debug","ts":1631097515.5751567,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:42954","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097515.5756023,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097515.5756893,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:42954","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Connection":["Upgrade"],"Cf-Ray":["68b78c4de7660cdf-LAX"],"Sec-Websocket-Version":["13"],"Cf-Ew-Via":["15"],"Accept-Encoding":["gzip"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Websocket-Key":["dNUNBjJFHEYkFHVTDYq/rQ=="],"Upgrade":["websocket"],"X-Forwarded-For":["IPIP"],"Cdn-Loop":["cloudflare; subreqs=1"],"X-Forwarded-Proto":["https"],"User-Agent":["Go-http-client/1.1"],"Cf-Connecting-Ip":["IPIP"],"Cf-Worker":["cf.workers.dev"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097515.5757039,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097515.5757434,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097515.8476615,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097515.8477476,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097515.8477583,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097516.2760677,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52025","server_name":"mydomain.com"}
{"level":"debug","ts":1631097516.27627,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52025","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097516.2771955,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/8002","result":"/var/home/8002"}
{"level":"debug","ts":1631097516.2773259,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"IPIP:52025","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["5r7xARBahO4LJCEuiC5hNw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"method":"GET","uri":"/index.html"}
{"level":"debug","ts":1631097516.2773411,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/home","request_path":"/index.html","result":"/var/home/index.html"}
{"level":"debug","ts":1631097516.277355,"logger":"http.handlers.file_server","msg":"opening file","filename":"/var/home/index.html"}
{"level":"debug","ts":1631097516.6638021,"logger":"layer4","msg":"connection stats","remote":"IPIP:52025","read":594,"written":10899,"duration":0.823057503}
{"level":"debug","ts":1631097520.512052,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097520.5121393,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097520.5121615,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097520.9416094,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52032","server_name":"mydomain.com"}
{"level":"debug","ts":1631097520.9418695,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52032","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097520.942889,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:8003","request":{"remote_addr":"IPIP:52032","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8003","headers":{"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"],"Sec-Websocket-Key":["websocket.key"]}},"headers":{"Upgrade":["websocket"],"Connection":["Upgrade"],"Sec-Websocket-Accept":["PIVfT3VFGVicvBMuIs9f/HSYILc="]},"status":101}
{"level":"debug","ts":1631097520.942973,"logger":"http.handlers.reverse_proxy","msg":"upgrading connection","upstream":"127.0.0.1:8003","request":{"remote_addr":"IPIP:52032","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8003","headers":{"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"],"Sec-Websocket-Key":["websocket.key"]}}}
{"level":"debug","ts":1631097525.420092,"logger":"http.handlers.reverse_proxy","msg":"connection closed","upstream":"127.0.0.1:8003","request":{"remote_addr":"IPIP:52032","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8003","headers":{"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"],"Sec-Websocket-Key":["websocket.key"]}},"duration":4.477046985}
{"level":"debug","ts":1631097526.612511,"logger":"layer4","msg":"connection stats","remote":"IPIP:52032","read":771,"written":10509906,"duration":6.101738768}
{"level":"debug","ts":1631097532.5297396,"logger":"tls.handshake","msg":"choosing certificate","identifier":"mydomain.com","num_choices":1}
{"level":"debug","ts":1631097532.529821,"logger":"tls.handshake","msg":"default certificate selection results","identifier":"mydomain.com","subjects":["mydomain.com"],"managed":true,"issuer_key":"acme.zerossl.com-v2-DV90","hash":"hashcode"}
{"level":"debug","ts":1631097532.5298455,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["mydomain.com"],"managed":true,"expiration":1638662399,"hash":"hashcode"}
{"level":"debug","ts":1631097532.9571102,"logger":"layer4.handlers.tls","msg":"terminated TLS","remote":"IPIP:52042","server_name":"mydomain.com"}
{"level":"debug","ts":1631097532.9572647,"logger":"layer4.handlers.proxy","msg":"dial upstream","remote":"IPIP:52042","upstream":"127.0.0.1:8000"}
{"level":"debug","ts":1631097532.9589086,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"127.0.0.1:8004","request":{"remote_addr":"IPIP:52042","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8004","headers":{"Sec-Websocket-Key":["websocket.key"],"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"]}},"headers":{"Connection":["Upgrade"],"Sec-Websocket-Accept":["u5cdlu5W2lnrWQLQtxD6Taq++O8="],"Upgrade":["websocket"]},"status":101}
{"level":"debug","ts":1631097532.9589906,"logger":"http.handlers.reverse_proxy","msg":"upgrading connection","upstream":"127.0.0.1:8004","request":{"remote_addr":"IPIP:52042","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8004","headers":{"Sec-Websocket-Key":["websocket.key"],"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"]}}}
{"level":"debug","ts":1631097536.927742,"logger":"http.handlers.reverse_proxy","msg":"connection closed","upstream":"127.0.0.1:8004","request":{"remote_addr":"IPIP:52042","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8004","headers":{"Sec-Websocket-Key":["websocket.key"],"Sec-Websocket-Version":["13"],"X-Forwarded-For":["IPIP"],"X-Forwarded-Proto":["http"],"Connection":["Upgrade"],"User-Agent":["Go-http-client/1.1"],"Upgrade":["websocket"]}},"duration":3.968730112}
{"level":"debug","ts":1631097538.2261362,"logger":"layer4","msg":"connection stats","remote":"IPIP:52042","read":1008,"written":10589204,"duration":5.702411312}

http.log here:

{"level":"error","ts":1631097497.1767857,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:51993","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"Connection":["Upgrade"],"Sec-Websocket-Key":["UhMK2oPY4HzA+jmdbCNIGw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:17 +0800] \"GET /8001 HTTP/1.1\" 404 6735","user_id":"","duration":0.000111421,"size":6735,"status":404,"resp_headers":{"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Accept-Ranges":["bytes"],"Content-Length":["6735"],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"Etag":["\"qb3qt3573\""]}}
{"level":"error","ts":1631097498.4517317,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:51994","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"Sec-Websocket-Key":["I/HKNIt1uaOBdUrHDFeZSQ=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:18 +0800] \"GET /8001 HTTP/1.1\" 404 6735","user_id":"","duration":0.000097694,"size":6735,"status":404,"resp_headers":{"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Server":["Caddy"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Content-Length":["6735"],"X-Content-Type-Options":["nosniff"]}}
{"level":"error","ts":1631097500.4224265,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:51996","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["kfmeD0dquEhvaEy1m7aemw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:20 +0800] \"GET /8001 HTTP/1.1\" 404 6735","user_id":"","duration":0.000204947,"size":6735,"status":404,"resp_headers":{"Accept-Ranges":["bytes"],"Content-Length":["6735"],"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"]}}
{"level":"error","ts":1631097502.1738088,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:51998","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["+c3aYixUa0vkLlk9uQjvpA=="]}},"common_log":"IPIP - - [08/Sep/2021:18:38:22 +0800] \"GET /8001 HTTP/1.1\" 404 6735","user_id":"","duration":0.000104078,"size":6735,"status":404,"resp_headers":{"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Content-Length":["6735"],"Server":["Caddy"],"X-Content-Type-Options":["nosniff"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Accept-Ranges":["bytes"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"]}}
{"level":"error","ts":1631097503.9872165,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52000","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8001","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["zG//cFwo7T/aWeonWGSunw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:23 +0800] \"GET /8001 HTTP/1.1\" 404 6735","user_id":"","duration":0.000102274,"size":6735,"status":404,"resp_headers":{"Content-Length":["6735"],"X-Content-Type-Options":["nosniff"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Server":["Caddy"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"]}}
{"level":"error","ts":1631097509.7635381,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52006","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["NhGjlsHMA838N3IUhstn2A=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:29 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000094108,"size":6735,"status":404,"resp_headers":{"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Server":["Caddy"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Content-Length":["6735"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"]}}
{"level":"error","ts":1631097510.9704752,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52007","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["5j++1JTZbNd/nDnUR1Rh/w=="]}},"common_log":"IPIP - - [08/Sep/2021:18:38:30 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000096753,"size":6735,"status":404,"resp_headers":{"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Content-Length":["6735"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Accept-Ranges":["bytes"],"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"]}}
{"level":"error","ts":1631097511.3675778,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:37004","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Version":["13"],"Cf-Connecting-Ip":["IPIP"],"Cf-Worker":["cf.workers.dev"],"Accept-Encoding":["gzip"],"X-Forwarded-For":["IPIP"],"Cf-Ray":["68b78c3382f6eb41-LAX"],"X-Forwarded-Proto":["https"],"Connection":["Upgrade"],"Sec-Websocket-Key":["2soWKtUR1wZITQvbXmLfpQ=="],"Cf-Ew-Via":["15"],"Cdn-Loop":["cloudflare; subreqs=1"],"Upgrade":["websocket"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"User-Agent":["Go-http-client/1.1"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:31 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000130647,"size":6735,"status":404,"resp_headers":{"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Accept-Ranges":["bytes"],"X-Frame-Options":["SAMEORIGIN"],"Content-Length":["6735"]}}
{"level":"error","ts":1631097512.7853777,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52015","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["qpb+rkDSXTYfjAmisOUt6g=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:32 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000187975,"size":6735,"status":404,"resp_headers":{"X-Content-Type-Options":["nosniff"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Server":["Caddy"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"Accept-Ranges":["bytes"],"Content-Length":["6735"],"X-Frame-Options":["SAMEORIGIN"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"]}}
{"level":"error","ts":1631097513.0367463,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:39534","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Upgrade":["websocket"],"Cf-Ew-Via":["15"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Cf-Ray":["68b78c3de60c368c-LAX"],"Cdn-Loop":["cloudflare; subreqs=1"],"Cf-Worker":["cf.workers.dev"],"Accept-Encoding":["gzip"],"X-Forwarded-Proto":["https"],"Cf-Connecting-Ip":["IPIP"],"X-Forwarded-For":["IPIP"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Websocket-Key":["Bqxw7rNH9YZLO8Jis87Kdw=="],"Sec-Websocket-Version":["13"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:33 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000157738,"size":6735,"status":404,"resp_headers":{"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Content-Length":["6735"],"Server":["Caddy"],"X-Frame-Options":["SAMEORIGIN"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Accept-Ranges":["bytes"]}}
{"level":"error","ts":1631097514.4479496,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52019","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Key":["6bN+kohY7NXKXn7camr0UA=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:34 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000094709,"size":6735,"status":404,"resp_headers":{"Server":["Caddy"],"Etag":["\"qb3qt3573\""],"Accept-Ranges":["bytes"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Content-Length":["6735"]}}
{"level":"error","ts":1631097515.5760002,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:42954","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Cf-Worker":["cf.workers.dev"],"X-Forwarded-Proto":["https"],"User-Agent":["Go-http-client/1.1"],"Cf-Connecting-Ip":["IPIP"],"Sec-Websocket-Version":["13"],"Cf-Ew-Via":["15"],"Connection":["Upgrade"],"Cf-Ray":["68b78c4de7660cdf-LAX"],"Accept-Encoding":["gzip"],"Cf-Visitor":["{\"scheme\":\"https\"}"],"Sec-Websocket-Key":["dNUNBjJFHEYkFHVTDYq/rQ=="],"Upgrade":["websocket"],"X-Forwarded-For":["IPIP"],"Cdn-Loop":["cloudflare; subreqs=1"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:35 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.00013804,"size":6735,"status":404,"resp_headers":{"X-Content-Type-Options":["nosniff"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Content-Length":["6735"],"Server":["Caddy"],"X-Frame-Options":["SAMEORIGIN"],"Content-Type":["text/html; charset=utf-8"],"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Accept-Ranges":["bytes"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"]}}
{"level":"error","ts":1631097516.2776096,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52025","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8002","headers":{"Sec-Websocket-Key":["5r7xARBahO4LJCEuiC5hNw=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:36 +0800] \"GET /8002 HTTP/1.1\" 404 6735","user_id":"","duration":0.000155954,"size":6735,"status":404,"resp_headers":{"Last-Modified":["Fri, 29 May 2020 16:51:03 GMT"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Etag":["\"qb3qt3573\""],"Content-Type":["text/html; charset=utf-8"],"Accept-Ranges":["bytes"],"Content-Length":["6735"],"Server":["Caddy"]}}
{"level":"info","ts":1631097525.420356,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52032","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8003","headers":{"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["ly9zzRiaY5iYa7Ec0bCnMA=="]}},"common_log":"IPIP - - [08/Sep/2021:18:38:45 +0800] \"GET /8003 HTTP/1.1\" 0 0","user_id":"","duration":4.478228997,"size":0,"status":0,"resp_headers":{"Sec-Websocket-Accept":["PIVfT3VFGVicvBMuIs9f/HSYILc="],"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"],"Upgrade":["websocket"],"Connection":["Upgrade"]}}
{"level":"info","ts":1631097536.927871,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"IPIP:52042","proto":"HTTP/1.1","method":"GET","host":"mydomain.com","uri":"/8004","headers":{"User-Agent":["Go-http-client/1.1"],"Connection":["Upgrade"],"Sec-Websocket-Key":["2mT4eE5OqTvCoXvaCari2Q=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"]}},"common_log":"IPIP - - [08/Sep/2021:18:38:56 +0800] \"GET /8004 HTTP/1.1\" 0 0","user_id":"","duration":3.969616145,"size":0,"status":0,"resp_headers":{"Upgrade":["websocket"],"Connection":["Upgrade"],"Sec-Websocket-Accept":["u5cdlu5W2lnrWQLQtxD6Taq++O8="],"Server":["Caddy"],"Strict-Transport-Security":["max-age=31536000; includeSubDomains; preload"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["SAMEORIGIN"],"X-Robots-Tag":["noindex, nofollow, noarchive, nosnippet, notranslate, noimageindex"]}}

It seems the http matchers does not work, all websocket connections and URI path goes 127.0.0.1:8000, so how can I forward websocket connections with special path to a certain upstream server ?

Default protocol type for listen addr `tcp/0.0.0.0:x` is `tcp6`

I've recently started using layer4 to perform some tcp proxying for ingress to some services. Currently our edge is accepting traffic with ipv4 addresses. My caddy file looks like the following.

{
  "apps": {
    "layer4": {
      "servers": {
        "tcp": {
          "listen": ["tcp/0.0.0.0:1010"],
          "routes": [{
            "handle": [{
              "handler": "proxy",
              "proxy_protocol": "",
              "upstreams": [
                {"dial": ["tcp/${cluster_internal_ip}:1010"]}
              ]
            }]
          }]
        }
      }
    }
  }
}

You'll notice that my listener network addr is default tcp and provides the ipv4 catchall addr 0.0.0.0. This configuration would lead me to believe that the socker should open on tcp 0.0.0.0:1010 but what I've found is that it's opening on tcp6 :::1010.

~$ sudo netstat -plunt | grep 1010
tcp6       0      0 :::1010                 :::*                    LISTEN      13739/caddy-l4

I'm noticing the same behavior on Ubuntu 16.04.4 LTS and Ubuntu 18.04.5 LTS. I have not checked on Ubuntu 20.*.

host1~$ uname -r -v && lsb_release -a 2>/dev/null | grep Desc
5.4.0-1038-aws #40~18.04.1-Ubuntu SMP Sat Feb 6 01:56:56 UTC 2021
Description:	Ubuntu 18.04.5 LTS
host2~$ uname -r -v && lsb_release -a 2>/dev/null | grep Desc
4.4.0-1128-aws #142-Ubuntu SMP Fri Apr 16 12:42:33 UTC 2021
Description:	Ubuntu 16.04.4 LTS

I'm running caddy with layer4 as a docker container using --network-mode=host from this docker container I've created

Https Termination

I'm coming from HAProxy where I can terminate TLS (TCP or HTTP traffic) and forward connections to backend servers depending on the traffic type.

For example, it would be great to run two services on the same port, one being TLS terminated and the other Https terminated. A sample config is below.

In the config I have a mongodb server running on port 27017, that uses an SNI matcher for mongo.mydomain.com. This works perfectly.

I also have an http server running on port 8181, and I also use an SNI matcher for webserver.mydomain.com. This does not work, where in HAproxy it would. A curl of the address results in "curl: (16) Error in the HTTP2 framing layer"

I have also tried adding an http matcher, and replacing the tls matcher with the http matcher. I am able to proxy http traffic on the same port, but ideally I'd like the https connection terminated.

Any thoughts on how to achieve this?

{
	"logging": {
		"logs": {
			"default": {
				"level": "DEBUG"
			}
		}
	},
	"apps": {
		"layer4": {
			"servers": {
				"example": {
					"listen": ["0.0.0.0:443"],
					"routes": [{
							"match": [{
									"tls": {
										"sni": ["webserver.mydomain.com"]
									}
								}
							],
							"handle": [{
									"handler": "tls"
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["localhost:8181"]
										}
									]
								}
							]
						}, {
							"match": [{
									"tls": {
										"sni": ["mongo.mydomain.com"]
									}
								}
							],
							"handle": [{
									"handler": "tls"
								}, {
									"handler": "proxy",
									"upstreams": [{
											"dial": ["localhost:27017"]
										}
									]
								}
							]
						}
					]
				}
			}
		},
		"tls": {
			"certificates": {
				"automate": ["mongo.mydomain.com", "webserver.mydomain.com"]
			}
		}
	}
}

"Throttle data flow to simulate slow connections."

Hey,

As per the subject, I am interested in adding some throttling to my routes to simulate slow connections

I am using this module to load balance connections for a game server

Is this currently possible? If not is it still a planned feature?

Thanks

	// make sure upstream connections all get closed
	defer func() {
	for _, conn := range upConns {
	conn.Close()
	}
	}()

	_, err := io.Copy(down, up)
	if err != nil && atomic.LoadInt32(&done) == 0 {
	h.logger.Error("upstream connection",
	zap.String("local_address", up.LocalAddr().String()),
	zap.String("remote_address", up.RemoteAddr().String()),
	zap.Error(err),
	)
	}