katelynhaworth / go-tproxy Goto Github PK

In tproxy_udp.go we have:

if err = binary.Read(bytes.NewReader(msg.Data), binary.BigEndian, originalDstRaw); err != nil {

This failed on a big endian box, but adjusting it to

if err = binary.Read(bytes.NewReader(msg.Data), nativeEndian, originalDstRaw); err != nil {

where nativeEndian was defined here: https://stackoverflow.com/questions/51332658/any-better-way-to-check-endianness-in-go worked fine.

Hi,

Thanks for this project! Just needed TPROXY in Go and thought I would need to implement it on my own, luckily you already tackled that :)

I just wanted to point out two minor "bugs": In the UDP and TCP implementations, you are using syscall.Close to close the cloned file-descriptor for the listeners. However, due to the defered Close() on the *os.File, this actually double-closes the FD. It doesn't do any harm but just wanted to point it out.

tproxy_tcp.go#L73
tproxy-tcp.go#L31 and #L36

Cheers,
Patrick

When set IP_PKTINFO, dst ip will in oob as struct in_pktinfo.
By using this, it would only need one listening socket.

copy oob from ReadMsgUDP to WriteMsgUDP in golang

func (c *UDPConn) ReadMsgUDP(b, oob []byte) (n, oobn, flags int, addr *UDPAddr, err error)
func (c *UDPConn) WriteMsgUDP(b, oob []byte, addr *UDPAddr) (n, oobn int, err error)

as in C

ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
ssize_t sendmsg(int sockfd, const struct msghdr *msg, int flags);

IP_PKTINFO (since Linux 2.2)
Pass an IP_PKTINFO ancillary message that contains a pktinfo structure that supplies some information about the incoming packet. This only works for datagram oriented sockets. The argument is a flag that tells the socket whether the IP_PKTINFO message should be passed or not. The message itself can only be sent/retrieved as control message with a packet using recvmsg(2) or sendmsg(2).

struct in_pktinfo {
    unsigned int   ipi_ifindex;  /* Interface index */
    struct in_addr ipi_spec_dst; /* Local address */
    struct in_addr ipi_addr;     /* Header Destination address */
};

ipi_ifindex is the unique index of the interface the packet was received on. ipi_spec_dst is the local address of the packet and ipi_addr is the destination address in the packet header. If IP_PKTINFO is passed to sendmsg(2) and ipi_spec_dst is not zero, then it is used as the local source address for the routing table lookup and for setting up IP source route options. When ipi_ifindex is not zero, the primary local address of the interface specified by the index overwrites ipi_spec_dst for the routing table lookup.

In golang, in_pktinfo is syscall.Inet4Pktinfo

type Inet4Pktinfo struct {
	Ifindex  uint32
	Spec_dst [4]byte /* in_addr */
	Addr     [4]byte /* in_addr */
}

Modify demo:

// tproxy_udp.go
func ListenUDP(network string, laddr *net.UDPAddr) (*net.UDPConn, error) {
...
	if err = syscall.SetsockoptInt(fileDescriptor, syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1); err != nil {
		syscall.Close(fileDescriptor)
		return nil, &net.OpError{Op: "listen", Err: fmt.Errorf("set socket option: SO_REUSEADDR: %s", err)}
	}

	if err = syscall.SetsockoptInt(fileDescriptor, syscall.IPPROTO_IP, syscall.IP_PKTINFO, 1); err != nil {
		syscall.Close(fileDescriptor)
		return nil, &net.OpError{Op: "listen", Err: fmt.Errorf("set socket option: IP_PKTINFO: %s", err)}
	}
...
}

func ReadFromUDP(conn *net.UDPConn, b []byte) (int, []byte, *net.UDPAddr, *net.UDPAddr, error) {
...
	for _, msg := range msgs {
		if msg.Header.Level == syscall.IPPROTO_IP && msg.Header.Type == syscall.IP_PKTINFO {
			originalDstRaw := &syscall.Inet4Pktinfo{}
			if err = binary.Read(bytes.NewReader(msg.Data), binary.LittleEndian, originalDstRaw); err != nil {
				return 0, nil, nil, nil, fmt.Errorf("reading original destination address: %s", err)
			}

			// switch originalDstRaw.Family {
			// case syscall.AF_INET:
			pp := (*syscall.Inet4Pktinfo)(unsafe.Pointer(originalDstRaw))
			// p := (*[2]byte)(unsafe.Pointer(&pp.Port))
			originalDst = &net.UDPAddr{
				IP:   net.IPv4(pp.Addr[0], pp.Addr[1], pp.Addr[2], pp.Addr[3]),
				Port: 53,
			}
...

// example/tproxy_example.go
func handleUDPConn(data []byte, oob []byte, srcAddr, dstAddr *net.UDPAddr) {
...
  // oob copy from ReadMsgUDP
  bytesWritten, _, err = udpListener.WriteMsgUDP(data, oob, srcAddr)
...

Hello @KatelynHaworth

Is this project still active, if not would it be possible to transfer it to someone else to continue maintanance?

Let me know, I'd be interested!
Thanks!

Hello!

I am trying to understand how TPROXY works and I keep getting infinite loops (or traffic stuck, in this case), no matter the application that I use TPROXY with.
I stumbled upon your repository and I was hoping that you could shed some light on what I'm doing wrong. I'm sorry in advance if my question is out of place.

This is my setup on CentOS 7 3.10.0-1160.83.1.el7.x86_64:

ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:50:56:8e:06:d8 brd ff:ff:ff:ff:ff:ff
    inet 10.17.133.214/21 brd 10.17.135.255 scope global noprefixroute dynamic ens192
       valid_lft 81182sec preferred_lft 81182sec
    inet6 fe80::8eda:d7f8:1055:4ab/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

lsmod | grep -i tproxy
xt_TPROXY              17327  1 
nf_defrag_ipv6         35104  2 xt_socket,xt_TPROXY
nf_defrag_ipv4         12729  3 xt_socket,xt_TPROXY,nf_conntrack_ipv4

Routing rules:

ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

ip route list table 100
local default dev lo scope host 

ip rule list
0:	from all lookup local
32762:	from all fwmark 0x1 lookup 100
32763:	from all fwmark 0x1 lookup 100
32764:	from all fwmark 0x1 lookup 100
32765:	from all fwmark 0x1 lookup 100
32766:	from all lookup main
32767:	from all lookup default

I have these iptables rules in place:

iptables -t mangle -L -nv
Chain PREROUTING (policy ACCEPT 2351 packets, 481K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 255K  140M DIVERT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            socket
    2   112 TPROXY     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 TPROXY redirect 0.0.0.0:8080 mark 0x1/0x1

Chain INPUT (policy ACCEPT 2915 packets, 524K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 369 packets, 44806 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 369 packets, 44806 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain DIVERT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 255K  140M MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK set 0x1
 255K  140M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

How I reproduce this (output provided after running all the steps):

1. I have an app that listens on port 80 on the machine:

python -m SimpleHTTPServer 80
Serving HTTP on 0.0.0.0 port 80 ...

Nothing gets here.

2. I start the go-tproxy example on the same machine:

./example 
2023/04/25 16:10:44 Starting GoLang TProxy example
2023/04/25 16:10:44 Binding TCP TProxy listener to 0.0.0.0:8080
2023/04/25 16:10:44 Binding UDP TProxy listener to 0.0.0.0:8080
2023/04/25 16:11:40 Accepting TCP connection from 10.17.132.130:40448 with destination of 10.17.133.214:80

Then it gets stuck.

3. From another machine, I run:

curl -v http://10.17.133.214:80/serverfile
*   Trying 10.17.133.214:80...
* Connected to 10.17.133.214 (10.17.133.214) port 80 (#0)
> GET /serverfile HTTP/1.1
> Host: 10.17.133.214
> User-Agent: curl/7.81.0
> Accept: */*
>

Then it gets stuck.

I was expecting that the traffic will reach the webserver and it will appear as it was coming from that other machine(the one I ran curl from), instead of the local machine, even though it was routed through the go-proxy application.

Please help me figure out is wrong in my usage or understanding of TPROXY, if you've got the time.

Thank you very much!

I was trying to integrate (didn't actually need to at then end of the day) and found that if I passed an laddr as nil, though the method doc string says that should work, I would get panics because it looksl ike the laddr can be referenced in some of the private methods without checking for being nil.

Sorry I don't have the dumps from it but I was definitely on latest and hitting panics (mostly this is an FYI)

There is no DialTCP function which can create arbitrary tproxy connections.
It can be used in HTTP proxies with a customized http.client .

I have followed all the instructions and run the example but I cannot get it to work.
I setup go-tproxy on a local server and have my computer setting default gateway to that server.
I see log printing "Accepting TCP connection from [myComputerIP:port] with the destination of [destinationIP:port]"
However the browser keeps spinning and waiting. I put some log printing in the example and it seems like it got stuck in the streamConn (copy 0 bytes and nil error).
Can you please help me fixing this issue
Thank you