andrsharaev / xt_nat Goto Github PK

I want to change openvpn NAT type,so can I set like this? eth0 is internet interface ,tun0 is openvpn virtual interface.openvpn net pool is 10.8.0.0 and my client IP is 10.8.0.10

iptables -A FORWARD -d 10.8.0.10 -i eth0 -o tun0 -j ACCEPT

iptables -t raw -A PREROUTING -d 10.8.0.0 -j NAT --dnat

# uname -a
Linux nat 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux


# make
make -C /lib/modules/4.19.0-8-amd64/build/ M=/usr/local/src/xt_NAT modules CONFIG_DEBUG_INFO=y                                                                                   
make[1]: Entering directory '/usr/src/linux-headers-4.19.0-8-amd64'
  CC [M]  /usr/local/src/xt_NAT/xt_NAT.o
/usr/local/src/xt_NAT/xt_NAT.c: In function ‘stat_seq_show’:
/usr/local/src/xt_NAT/xt_NAT.c:1547:43: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Active NAT sessions: %ld\n", atomic64_read(&sessions_active));
                                         ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                         %lld
/usr/local/src/xt_NAT/xt_NAT.c:1548:42: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Tried NAT sessions: %ld\n", atomic64_read(&sessions_tried));
                                        ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                        %lld
/usr/local/src/xt_NAT/xt_NAT.c:1549:44: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Created NAT sessions: %ld\n", atomic64_read(&sessions_created));
                                          ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                          %lld
/usr/local/src/xt_NAT/xt_NAT.c:1550:41: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "DNAT dropped pkts: %ld\n", atomic64_read(&dnat_dropped));
                                       ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                       %lld
/usr/local/src/xt_NAT/xt_NAT.c:1551:39: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Fragmented pkts: %ld\n", atomic64_read(&frags));
                                     ~~^     ~~~~~~~~~~~~~~~~~~~~~
                                     %lld
/usr/local/src/xt_NAT/xt_NAT.c:1552:41: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Related ICMP pkts: %ld\n", atomic64_read(&related_icmp));
                                       ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                       %lld
/usr/local/src/xt_NAT/xt_NAT.c:1553:36: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘s64’ {aka ‘long long int’} [-Wformat=]               
     seq_printf(m, "Active Users: %ld\n", atomic64_read(&users_active));
                                  ~~^     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                  %lld
/usr/local/src/xt_NAT/xt_NAT.c: In function ‘nat_tg_init’:
/usr/local/src/xt_NAT/xt_NAT.c:1664:5: error: implicit declaration of function ‘setup_timer’; did you mean ‘sk_stop_timer’? [-Werror=implicit-function-declaration]              
     setup_timer( &sessions_cleanup_timer, sessions_cleanup_timer_callback, 0 );
     ^~~~~~~~~~~
     sk_stop_timer
cc1: some warnings being treated as errors
make[4]: *** [/usr/src/linux-headers-4.19.0-8-common/scripts/Makefile.build:315: /usr/local/src/xt_NAT/xt_NAT.o] Error 1                                                         
make[3]: *** [/usr/src/linux-headers-4.19.0-8-common/Makefile:1537: _module_/usr/local/src/xt_NAT] Error 2                                                                       
make[2]: *** [Makefile:146: sub-make] Error 2
make[1]: *** [Makefile:8: all] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.19.0-8-amd64'
make: *** [Makefile:11: xt_NAT.ko] Error 2

ld поменять на lld ума хватило, но дальше уже не осилил
:-(

Need Netflow 9 with nat events

i try modul direct on eth0,eth1 and work fine.
but when i try to run over bonding or team driver not work .

Any one to help with fix this problem ?

@andrsharaev , @Stanback

Kernel 5.12.1
400users online.
one user open over 4k socket on udp

10.8.196.171 -> x.x.57.x (tcp: 267, udp: 4093, other: 1)

[81344.832784] xt_NAT: 10.8.196.171 exceed max allowed sessions
[81344.832786] xt_NAT SNAT: Cannot create new session. Dropping packet
[81344.832789] xt_NAT: 10.8.196.171 exceed max allowed sessions
[81344.832790] xt_NAT SNAT: Cannot create new session. Dropping packet
[81344.843236] xt_NAT SNAT: Cannot create new session. Dropping packet
[81344.843483] xt_NAT: 10.8.196.171 exceed max allowed sessions
[81344.981056] xt_NAT: 10.8.196.171 exceed max allowed sessions
[81344.987069] xt_NAT SNAT: Cannot create new session. Dropping packet
[81345.004978] xt_NAT SNAT: Cannot create new session. Dropping packet
[81402.540906] rcu: INFO: rcu_sched self-detected stall on CPU
[81402.540909] rcu: 5-....: (3314 ticks this GP) idle=74e/1/0x4000000000000000 softirq=4979878/4979878 fqs=2554 last_accelerate: a926/c0a0 dyntick_enabled: 1
[81402.540911] (t=6001 jiffies g=7517749 q=44479)
[81402.540913] NMI backtrace for cpu 5
[81402.540914] CPU: 5 PID: 36 Comm: ksoftirqd/5 Tainted: G O 5.12.1 #1
[81402.540916] Hardware name: Supermicro Super Server/X10SRD-F, BIOS 3.3 10/28/2020
[81402.540917] Call Trace:
[81402.540919]
[81402.540920] dump_stack+0x65/0x7d
[81402.540924] ? lapic_can_unplug_cpu+0x70/0x70
[81402.540927] nmi_trigger_cpumask_backtrace.cold+0x40/0x4d
[81402.540929] rcu_dump_cpu_stacks+0xbe/0xec
[81402.540932] rcu_sched_clock_irq.cold+0x195/0x3f1
[81402.540934] ? enqueue_task_fair+0x796/0xbd0
[81402.540938] update_process_times+0x88/0xc0
[81402.540942] tick_sched_timer+0x7f/0x110
[81402.540944] ? tick_nohz_dep_set_task+0x80/0x80
[81402.540945] __hrtimer_run_queues+0x10b/0x1b0
[81402.540947] hrtimer_interrupt+0x10a/0x420
[81402.540949] __sysvec_apic_timer_interrupt+0x47/0x60
[81402.540952] sysvec_apic_timer_interrupt+0x65/0x90
[81402.540955]
[81402.540955] asm_sysvec_apic_timer_interrupt+0xf/0x20
[81402.540959] RIP: 0010:console_unlock+0x366/0x5e0
[81402.540961] Code: ff ff 8b 05 44 5f b2 01 85 c0 75 66 c7 05 3a 5f b2 01 01 00 00 00 e9 0f fd ff ff e8 f4 1c 00 00 48 85 db 74 01 fb 8b 54 24 0c <85> d2 0f 84 4a fd ff ff e8 1d 2b 7c 00 e9 40 fd ff ff 4d 85 ff 74
[81402.540963] RSP: 0018:ffff9dc980203a80 EFLAGS: 00000206
[81402.540964] RAX: 0000000000000000 RBX: 0000000000000200 RCX: 0000000000000000
[81402.540965] RDX: 0000000000000000 RSI: 0000000000000087 RDI: ffffffff82b59898
[81402.540966] RBP: 0000000000000000 R08: ffff9786814db080 R09: 0000000000000000
[81402.540966] R10: ffff9786a85bf260 R11: ffff9786f7bd7cf0 R12: 0000000000000048
[81402.540967] R13: 0000000000000000 R14: 20c49ba5e353f7cf R15: 0000000000000000
[81402.540968] ? common_interrupt+0x14/0xa0
[81402.540969] ? asm_common_interrupt+0x1b/0x40
[81402.540971] vprintk_default+0x5a/0x150
[81402.540972] printk+0x43/0x45
[81402.540975] create_nat_session+0x1c5e/0x1cfd [xt_NAT]
[81402.540978] ipt_do_table+0x2e5/0x670 [ip_tables]
[81402.540980] ? ip_route_input_noref+0xa8/0x1e0
[81402.540983] nf_hook_slow+0x36/0xa0
[81402.540986] ip_forward+0x40d/0x450
[81402.540987] ? ip4_obj_hashfn+0xc0/0xc0
[81402.540989] process_backlog+0x11a/0x230
[81402.540992] __napi_poll+0x1f/0x130
[81402.540994] net_rx_action+0x239/0x2f0
[81402.540996] ? run_timer_softirq+0x730/0x880
[81402.540998] __do_softirq+0xaf/0x1da
[81402.541000] run_ksoftirqd+0x15/0x20
[81402.541004] smpboot_thread_fn+0xb3/0x140
[81402.541006] ? sort_range+0x20/0x20
[81402.541008] kthread+0xea/0x120
[81402.541010] ? kthread_park+0x80/0x80
[81402.541012] ret_from_fork+0x1f/0x30
[81416.300055] rcu: INFO: rcu_sched detected expedited stalls on CPUs/tasks: {
[81476.311498] rcu: INFO: rcu_sched self-detected stall on CPU
[81476.311500] rcu: 3-....: (1 GPs behind) idle=86a/1/0x4000000000000000 softirq=4703397/4703398 fqs=2596 last_accelerate: c5ff/dd71 dyntick_enabled: 1
[81476.311503] (t=6001 jiffies g=7517753 q=82419)
[81476.311505] NMI backtrace for cpu 3
[81476.311506] CPU: 3 PID: 527214 Comm: kworker/3:2 Tainted: G O 5.12.1 #1
[81476.311507] Hardware name: Supermicro Super Server/X10SRD-F, BIOS 3.3 10/28/2020
[81476.311509] Workqueue: rcu_gp wait_rcu_exp_gp
[81476.311512] Call Trace:
[81476.311514]
[81476.311515] dump_stack+0x65/0x7d
[81476.311519] ? lapic_can_unplug_cpu+0x70/0x70
[81476.311521] nmi_trigger_cpumask_backtrace.cold+0x40/0x4d
[81476.311523] rcu_dump_cpu_stacks+0xbe/0xec
[81476.311527] rcu_sched_clock_irq.cold+0x195/0x3f1
[81476.311529] ? timekeeping_advance+0x34e/0x540
[81476.311531] update_process_times+0x88/0xc0
[81476.311534] tick_sched_timer+0x7f/0x110
[81476.311536] ? tick_nohz_dep_set_task+0x80/0x80
[81476.311537] __hrtimer_run_queues+0x10b/0x1b0
[81476.311539] hrtimer_interrupt+0x10a/0x420
[81476.311541] __sysvec_apic_timer_interrupt+0x47/0x60
[81476.311544] sysvec_apic_timer_interrupt+0x65/0x90
[81476.311547]
[81476.311547] asm_sysvec_apic_timer_interrupt+0xf/0x20
[81476.311551] RIP: 0010:console_unlock+0x366/0x5e0
[81476.311554] Code: ff ff 8b 05 44 5f b2 01 85 c0 75 66 c7 05 3a 5f b2 01 01 00 00 00 e9 0f fd ff ff e8 f4 1c 00 00 48 85 db 74 01 fb 8b 54 24 0c <85> d2 0f 84 4a fd ff ff e8 1d 2b 7c 00 e9 40 fd ff ff 4d 85 ff 74
[81476.311555] RSP: 0018:ffff9dc980313cc0 EFLAGS: 00000206
[81476.311556] RAX: 0000000000000000 RBX: 0000000000000200 RCX: 0000000000000000
[81476.311557] RDX: 0000000000000000 RSI: 0000000000000087 RDI: ffffffff82b59898
[81476.311557] RBP: 0000000000000000 R08: ffff9786814db080 R09: 0000000000000000
[81476.311558] R10: ffff9786a85bac10 R11: ffff97872e90acf0 R12: 0000000000000048
[81476.311559] R13: 0000000000000000 R14: 20c49ba5e353f7cf R15: 0000000000000000
[81476.311560] vprintk_default+0x5a/0x150
[81476.311562] printk+0x43/0x45
[81476.311563] synchronize_rcu_expedited_wait.cold+0x20/0x2db
[81476.311565] rcu_exp_wait_wake+0xc/0x110
[81476.311567] process_one_work+0x1ec/0x350
[81476.311569] worker_thread+0x4f/0x4d0
[81476.311570] ? process_one_work+0x350/0x350
[81476.311571] kthread+0xea/0x120
[81476.311573] ? kthread_park+0x80/0x80
[81476.311574] ret_from_fork+0x1f/0x30
[81551.199572] } 19586 jiffies s: 14473 root: 0x0/.

I'm experimenting with this module on a server that has only one public IPv4 address, which I'd like to use for clients behind the NAT server, as well as allowing the NAT server to access the Internet directly using the same IP used by nat_pool.

From what I can tell, the destination NAT (--dnat) prerouting rule always intercepts the packet and it doesn't seem to pass through correctly unless a valid session exists. The session is created when a packet enters the source NAT (--snat), however, since packets in question are originating on the NAT server itself, the PREROUTING/FORWARD/POSTROUTING chains don't seem to be usable.

Waiting to call skb_reset_transport_header until after the session is found seemed to do the trick, but I'm not quite sure what the implications are with this approach. Does that seem sensible or is there an easier/cleaner/better way to accomplish my goal? (Here's the change I made: Stanback@afa5698)

It's more a question.

It says in the feature list:

• User quotas support. Default value is 1000 max connections for each user (for each protocol independently)

I then understand the maximum port each CGNAT IP/User can use is 1000 ports in this case.
Is it possible to allocate initially a smaller block for each user (e.g: 500) and when necessary allocate additional blocks on demand then export the event via Netflow ?

It’s more like a question rather then Issue. What’s the way i can add multiple IP pool and bind specific Private Pool to Specific Live IP Pool.

Would like to use this on my OpenWRT router, but have no idea where to start.