Comments (4)
nik-netlox
commented on June 11, 2024
Hi Stephan,
Thanks for trying LoxiLB! We are looking into your issue.
Since, we would need more information, Request you to connect with us through our member's channel. We will be able to assist you better.
from loxilb.
infinitydon
commented on June 11, 2024
Can you try to create both the loxilb EC2 instance and the worker-nodes in the same subnet?
This is what works for me currently..
My eksctl config:
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: loxilb-sctp
region: us-west-2
version: "1.24"
iam:
withOIDC: true
vpc:
nat:
gateway: Single
availabilityZones: ["us-west-2a", "us-west-2b"]
managedNodeGroups:
- name: workers
instanceType: t3.medium
ssh:
allow: true
publicKeyName: local-key
desiredCapacity: 3
availabilityZones: ["us-west-2a"]
I created the Loxilb in us-west-2a and the same subnet.
[ec2-user@ip-172-31-18-255 manifest]$ kubectl -n open5gs get svc | grep sctp
core5g-amf-sctp LoadBalancer 10.100.57.111 192.168.0.46 31145:32238/SCTP 15m
ubuntu@ip-172-31-7-225:~/UERANSIM/build$ ./nr-gnb -c gnb.yaml
UERANSIM v3.2.6
[2023-03-03 22:38:19.539] [sctp] [info] Trying to establish SCTP connection... (xx.2xx.19.2:31145)
[2023-03-03 22:38:19.543] [sctp] [info] SCTP connection established ([34.222.19.2:31145](xx.2xx.19.2:31145))
[2023-03-03 22:38:19.543] [sctp] [debug] SCTP association setup ascId[7]
[2023-03-03 22:38:19.543] [ngap] [debug] Sending NG Setup Request
[2023-03-03 22:38:19.545] [ngap] [debug] NG Setup Response received
[2023-03-03 22:38:19.545] [ngap] [info] NG Setup procedure is successful
root@core5g-amf-1-deployment-6d457b6888-nvnh8:/#
root@core5g-amf-1-deployment-6d457b6888-nvnh8:/# tcpdump -i any sctp -s0 -nv
tcpdump: data link type LINUX_SLL2
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
22:38:19.542200 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 68)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [INIT] [init tag: 1440608999] [rwnd: 106496] [OS: 10] [MIS: 10] [init TSN: 2670472195]
22:38:19.542249 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 0, offset 0, flags [DF], proto SCTP (132), length 292)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [INIT ACK] [init tag: 3400320274] [rwnd: 106496] [OS: 10] [MIS: 10] [init TSN: 3283050420]
22:38:19.543047 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 264)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [COOKIE ECHO]
22:38:19.543077 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 0, offset 0, flags [DF], proto SCTP (132), length 36)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [COOKIE ACK]
22:38:19.544145 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 120)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [DATA] (B)(E) [TSN: 2670472195] [SID: 0] [SSEQ 0] [PPID 0x3c]
22:38:19.544164 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12175, offset 0, flags [DF], proto SCTP (132), length 48)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [SACK] [cum ack 2670472195] [a_rwnd 106424] [#gap acks 0] [#dup tsns 0]
22:38:19.544417 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12176, offset 0, flags [DF], proto SCTP (132), length 108)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [DATA] (B)(E) [TSN: 3283050420] [SID: 0] [SSEQ 0] [PPID 0x3c]
22:38:19.545042 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 48)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [SACK] [cum ack 3283050420] [a_rwnd 106439] [#gap acks 0] [#dup tsns 0]
22:38:26.122155 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12177, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [HB REQ]
22:38:26.123221 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [HB ACK]
22:38:32.266149 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12178, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [HB REQ]
22:38:32.267026 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [HB ACK]
22:38:38.666151 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12179, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [HB REQ]
22:38:38.667163 eth0 In IP (tos 0x2,ECT(0), ttl 62, id 0, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.13.2.5083 > 192.168.20.99.38412: sctp (1) [HB ACK]
22:38:45.322152 eth0 Out IP (tos 0x2,ECT(0), ttl 64, id 12180, offset 0, flags [DF], proto SCTP (132), length 84)
192.168.20.99.38412 > 192.168.13.2.5083: sctp (1) [HB REQ]
root@ip-192-168-0-46:/# loxicmd get ct
| DESTINATIONIP | SOURCEIP | DESTINATIONPORT | SOURCEPORT | PROTOCOL | STATE | ACT | PACKETS | BYTES |
|---------------|--------------|-----------------|------------|----------|--------|------------------------------------------|---------|-------|
| 192.168.0.46 | 192.168.0.1 | 68 | 67 | udp | closed | | 1 | 351 |
| 192.168.0.46 | 18.xxx.x6.97 | 31145 | 39271 | sctp | est | fdnat-192.168.0.46,192.168.13.2:32238:w0 | 105 | 10418 |
| 192.168.0.46 | 192.168.13.2 | 39271 | 32238 | sctp | est | fsnat-192.168.0.46,18.xxx.x6.97:31145:w0 | 105 | 10406 |
| 192.168.0.1 | 192.168.0.46 | 67 | 68 | udp | closed | | 0 | 0 |
Also I did not disable sctp checksum. I have not tested with the checksum enabled extensively, you can also try it if it works..
from loxilb.
backguynn
commented on June 11, 2024
@samassalt
I configured EKS and tested SCTP as described in the blog post. (EC2 instance and EKS nodes in the same subnet), That scenario worked for me.
While testing with several external clients, there was a case where the firewall prevented me from receiving SCTP response packets like in your case.
Can you check your firewall settings?
If there are no other problems, please let us know your loxilb settings so we can help you further. run this command in loxilb container:
loxicmd get lb -o wide
loxicmd get ct
Thanks.
from loxilb.
UltraInstinct14
commented on June 11, 2024
The issue was finally traced to incompatible linux kernel version. OP's loxilb node kernel version was 5.10 which had problems to handle SCTP (with eBPF). Issue resolved after upgrading to kernel 6.10. Ideally any linux kernel version >= 5.15 should work fine.
from loxilb.
Related Issues (20)
- performance: loxilb starts consuming 100% CPU only after a few seconds HOT 4
- tcplbl3dsrha fail HOT 6
- Support for docker arm builds HOT 2
- Direct host route addition cause traffic forwarding to stop working HOT 11
- Master node IP address changed to one of the Loadbalancer IP addresses after deployment HOT 1
- Issue with BFD Patch for Fast Failover: BFD Not Stable in HA Setup with loxilb HOT 2
- Configure end-point probe-time,retry values etc from kube-loxilb HOT 1
- BFD State Inconsistent [ Both shows MASTER ] at times HOT 2
- [BFD] New loxilb instance not retaining MASTER state after failover HOT 1
- Misleading libbpf FAILED logs
- Build from code and run is not working in a VM with kernel version 5.xxx-generic IF hypervisor (host) has a bigger kernel version like 6.xxx-generic HOT 16
- BPFireOS: cpumap libbpf: map 'cpu_map': failed to create: Argument list too long HOT 20
- BPFireOS: Prog section 'tc_packet_hook0' rejected: Permission denied (13)! R1 type=scalar expected=map_ptr HOT 11
- Ubuntu 22.04 libbpf: prog 'tc_packet_func_fast': BPF program load failed: Argument list too long HOT 4
- Loxilb SYNProxy support? HOT 1
- loxilb go code build error in BPFire chroot build environment HOT 1
- Support for proxy protocol v2
- Support for AWS multi-AZ HOT 3
- SCTP Load Balancing on Kubernetes HOT 1
- Fullnat support pool of local IP addresses assigned to local network interface as source IP HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from loxilb.