Is there a plan to supply the code like other tutorials ?

And I can compile it with command make in the project.

I would like to build an address scanner and portscanner for IPv6 using several different techniques which are more efficient than scanning a /64. I read the following comment in the advanced03 userspace code that builds the structure of the packet to send:

/* Here we sent the packet out of the receive port. Note that
* we allocate one entry and schedule it. Your design would be
* faster if you do batch processing/transmission */

I would like to implement batch transmission, so the SYN TCP packets could be transmitted in the fastest possible way, but I'm not sure how to implement this after reading documentation. After transmitting SYNs, I would like to XDP_DROP all the RST/ACK packets in the XDP program, so only the responses of open ports have to be processed in userspace.

I would really appreciate any pointers on how this could be implemented or improved!

As the title said.

Thanks a lot.

Hi
I'm trying to run Assignment 1 on a VirtualBox VM equipped with Ubuntu 18.04. I upgraded the kernel to version 5.2.13 to make sure it supports AF_XDP operations, at least when the built-in XDP program is being used.

I can compile all programs in the tutorial and set up the test lab environment normally. However, when I try to run the example program for assignment 1 I get this error:

Can't setup AF_XDP socket "Operation not supported"

The error persists even if I try to force "skb-mode" to avoid potential issues with the nic driver. It seems the error originates in the following command, which is part of the "xsk_socket__create" function from libbpf.

err = getsockopt(xsk->fd, SOL_XDP, XDP_OPTIONS, &opts, &optlen);

Any idea on how to solve it?

/sys/kernel/debug/ is empty. what is the problem?
[root@localhost tracing03-xdp-debug-print]# ls /sys/kernel/debug/
[root@localhost tracing03-xdp-debug-print]#

I want make a hash tree with a lot of hash arrays(may be 1000000). But one program can access at most 64 maps. I have tried to use a map with a value size =1000000*size(int), but failed when load the program. Or is there any way that the hash map can add a new element when update an existed element ？
who can help me ?

Hi,

I am working on a Beagle Board and try to compile xdp example but encountered error.
I followed setup_dependencies.org, all installed but bpftool:

# debian@beaglebone:/var/lib/cloud9/xdp-tutorial$ file /usr/sbin/bpftool
/usr/sbin/bpftool: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=d016afa7c9be4e163fb29184fd8323e23ca57fba, not stripped

My system: Debian.

# uname -a
Linux beaglebone 4.14.108-ti-r119 #1 SMP PREEMPT Mon Sep 30 02:30:24 UTC 2019 armv7l GNU/Linux

When I make basic01 program I got this:

# debian@beaglebone:/var/lib/cloud9/xdp-tutorial/basic01-xdp-pass$ make
make[1]: Entering directory '/var/lib/cloud9/xdp-tutorial/libbpf/src'
cc -I. -I../include -I../include/uapi -DCOMPAT_NEED_REALLOCARRAY -fPIC -fvisibility=hidden -g -O2 -Werror -Wall   -c xsk.c -o xsk.o
In file included from xsk.c:22:0:
xsk.c: In function ‘xsk_umem__create’:
../include/uapi/linux/if_xdp.h:87:34: error: overflow in implicit constant conversion [-Werror=overflow]
 #define XDP_UMEM_PGOFF_FILL_RING 0x100000000ULL
                                  ^
xsk.c:198:6: note: in expansion of macro ‘XDP_UMEM_PGOFF_FILL_RING’
      XDP_UMEM_PGOFF_FILL_RING);
      ^~~~~~~~~~~~~~~~~~~~~~~~
../include/uapi/linux/if_xdp.h:88:40: error: overflow in implicit constant conversion [-Werror=overflow]
 #define XDP_UMEM_PGOFF_COMPLETION_RING 0x180000000ULL
                                        ^
xsk.c:215:6: note: in expansion of macro ‘XDP_UMEM_PGOFF_COMPLETION_RING’
      XDP_UMEM_PGOFF_COMPLETION_RING);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Makefile:91: recipe for target 'xsk.o' failed
make[1]: *** [xsk.o] Error 1
make[1]: Leaving directory '/var/lib/cloud9/xdp-tutorial/libbpf/src'
make[1]: Entering directory '/var/lib/cloud9/xdp-tutorial/libbpf/src'
if [ ! -d 'build/usr/include/bpf' ]; then install -d -m 755 'build/usr/include/bpf'; fi; install bpf.h libbpf.h btf.h xsk.h libbpf_util.h -m 644 'build/usr/include/bpf'
make[1]: Leaving directory '/var/lib/cloud9/xdp-tutorial/libbpf/src'
gcc -Wall -I../libbpf/src//build/usr/include/ -g -I/usr/include/x86_64-linux-gnu -I../headers/ -L../libbpf/src/ -o xdp_pass_user ../common//common_params.o \
 xdp_pass_user.c -l:libbpf.a -lelf
/usr/bin/ld: cannot find -l:libbpf.a
collect2: error: ld returned 1 exit status
../common//common.mk:107: recipe for target 'xdp_pass_user' failed
make: *** [xdp_pass_user] Error 1

The tool perf is needed to complete "basic02" lesson. It should be listed among dependencies.

I am using a docker-container with

Linux 4.18.0-25-generic

installed. I can successfully compile your advanced03-AF_XDP example, but the moment I try to deploy it on the target server (Debian 4.19.98-1) via sudo ./af_xdp_user --dev eth20 --force I get:

ibbpf: BPF log buffer:
unknown opcode 66
ERROR: Can't setup AF_XDP socket "Operation not permitted"

The server has all three flags set (verified via /boot/config-4.19.0-8-amd64):

CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_XDP_SOCKETS=y

Any ideas?

On my device wtih Intel i7-6700 CPU and igb NIC driver and centos7.5 kernel 5.4.0, using bpf_redirecct_map() in tutors of packet03, there is always handreds fram losss, no metter the trafic is 500Mbit/s or 10Mbit/s. But there is no fram loss without xdp program loaded , with the same traffic .
With xdp programof bpf_redirecct_map(), the performace can reach about 500Mbit/s, but just loss a little packets, though I reduce the trafic to 10Mbit/s, but still a little packets loss. The above
traffic is produce by hardware tester of SPIRENT. There is no packet loss if the traffic is very little for example of no more than 1000 packes/s .
The packets should not loss in receiving and bpf_redirecct_map processing. Because the rx packets in ifconfig ino and xdp_stats are both right, and the tx packets in ifcong info is not right.
And it also should not result in rx and tx buff or fifo, because the "dropped" and "overrun" in ifconfig info are "0".

So, I do not know what's wrong in bpf redirect action???

The details of my testing:
Threre are 10 000 000 udp packets in my test trafic and 100 arp packets, in both direction. using xdp_router in turor of packet03.
The xdp_stats result is:
XDP-action
XDP_ABORTED 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000460
XDP_DROP 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000471
XDP_PASS 100 pkts ( 0 pps) 6 Kbytes ( 0 Mbits/s) period:2.000478
XDP_TX 0 pkts ( 0 pps) 0 Kbytes ( 0 Mbits/s) period:2.000483
XDP_REDIRECT 10,000,000 pkts ( 0 pps) 600,000 Kbytes ( 0 Mbits/s) period:2.000489

The ifconfig info  before and after transmiting are:

before:
[root@localhost ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::e73:ebff:fe90:80f1 prefixlen 64 scopeid 0x20
ether 0c:73:eb:90:80:f1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7 bytes 826 (826.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xdf000000-df0fffff

[root@localhost ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::e73:ebff:fe90:80f2 prefixlen 64 scopeid 0x20
ether 0c:73:eb:90:80:f2 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 7 bytes 826 (826.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xded00000-dedfffff
after:
[root@localhost ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::e73:ebff:fe90:80f2 prefixlen 64 scopeid 0x20
ether 0c:73:eb:90:80:f2 txqueuelen 1000 (Ethernet)
RX packets 10000100 bytes 600006000 (572.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 9999759 bytes 599984146 (572.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xded00000-dedfffff

[root@localhost ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.1 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::e73:ebff:fe90:80f1 prefixlen 64 scopeid 0x20
ether 0c:73:eb:90:80:f1 txqueuelen 1000 (Ethernet)
RX packets 10000100 bytes 600006000 (572.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 9999727 bytes 599982226 (572.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xdf000000-df0fffff

Some versions of clang creates .ll files, while other versions create .s files.
In particular, the ccache wrapper for clang changes the extension, but the files
are identical.

We should probably just detect this in the Makefile and use whichever one is
there.

Hi everyone,

I'm investigating the AF_XDP technology and I encountered a problem while compiling and trying to use the af_xdp_kern.c source file as-it-is. While compiling the filter triggers no error, trying to load it (x86_64 architecture, at least) will lead to an error of the in-kernel BPF verifier:

libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf: 
0: (61) r1 = *(u32 *)(r1 +16)
1: (63) *(u32 *)(r10 -4) = r1
2: (bf) r2 = r10
3: (07) r2 += -4
4: (18) r1 = 0xffff952a7b59c000
6: (85) call bpf_map_lookup_elem#1
7: (15) if r0 == 0x0 goto pc+7
 R0=map_value(id=0,off=0,ks=4,vs=4,imm=0) R10=fp0,call_-1 fp-8=mmmm????
8: (61) r1 = *(u32 *)(r0 +0)
 R0=map_value(id=0,off=0,ks=4,vs=4,imm=0) R10=fp0,call_-1 fp-8=mmmm????
9: (bf) r2 = r1
10: (07) r2 += 1
11: (63) *(u32 *)(r0 +0) = r2
 R0=map_value(id=0,off=0,ks=4,vs=4,imm=0) R1_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R2_w=inv(id=0,umin_value=1,umax_value=4294967296,var_off=(0x0; 0x1ffffffff)) R10=fp0,call_-1 fp-8=mmmm????
12: (b7) r0 = 2
13: (57) r1 &= 1
14: (55) if r1 != 0x0 goto pc+13
 R0=inv2 R1=inv0 R2=inv(id=0,umin_value=1,umax_value=4294967296,var_off=(0x0; 0x1ffffffff)) R10=fp0,call_-1 fp-8=mmmm????
15: (bf) r2 = r10
16: (07) r2 += -4
17: (18) r1 = 0xffff952a7c78ce00
19: (85) call bpf_map_lookup_elem#1
cannot pass map_type 17 into func bpf_map_lookup_elem#1
processed 18 insns (limit 1000000) max_states_per_insn 0 total_states 3 peak_states 3 mark_read 3

libbpf: -- END LOG --
libbpf: failed to load program 'xdp_sock'
libbpf: failed to load object './af_xdp_kern.o'
ERR: loading BPF-OBJ file(./af_xdp_kern.o) (-22): Invalid argument
ERR: loading file: ./af_xdp_kern.o

Investigation of the kernel verifier and the BPF tools libbpf lead me to the discovery that the filter introduced in the example is similar to the "default" one present in the libbpf library, with one small difference: the one provided in the example lack the xsk_lookup_bpf_maps, which is a necessary component for the library and the XSK subsystem.

In fact in the kernel 5.2.11, file: tools/lib/bpf/xsk/c:448, function xsk_lookup_bpf_maps:

if (xsk->qidconf_map_fd < 0 || xsk->xsks_map_fd < 0) {
        err = -ENOENT;
        xsk_delete_bpf_maps(xsk);
}

force the map to exists in each custom filter using XSK technologies, or the error occurs.

Such error can be fixed by introducing in the af_xdp_kern.c file the following modification:

struct bpf_map_def SEC("maps") qidconf_map = {
	.type = BPF_MAP_TYPE_ARRAY,
	.key_size = sizeof(int),
	.value_size = sizeof(int),
	.max_entries = 64,
};

Cheers,
Kewin R.

Hello. When following the tutorial, in basic 01, I can load xdp_pass_kern.o using the provided ip commands:

shanssian:basic01-xdp-pass $ sudo ip link set dev lo xdpgeneric obj xdp_p
ass_kern.o sec xdp
shanssian:basic01-xdp-pass $
shanssian:basic01-xdp-pass $
shanssian:basic01-xdp-pass $ sudo bpftool net list dev lo
xdp:
lo(1) generic id 40

tc:

flow_dissector:

With the xdp_pass_user program, I was getting a permission denied error:

shanssian:basic01-xdp-pass $ sudo ./xdp_pass_user --dev lo -A -F
libbpf: Error in bpf_object__probe_name():Operation not permitted(1). Couldn't load basic 'r0 = 0' BPF program.
libbpf: Error in bpf_object__probe_global_data():Operation not permitted(1). Couldn't create simple array map.
libbpf: load bpf program failed: Operation not permitted
libbpf: failed to load program 'xdp'
libbpf: failed to load object 'xdp_pass_kern.o'
ERR: loading BPF-OBJ file(xdp_pass_kern.o) (-22): Invalid argument
ERR: loading file: xdp_pass_kern.o

I found this workaround by searching the error message: https://www.spinics.net/lists/netdev/msg548481.html

Applied, it looks like:

diff --git a/basic01-xdp-pass/xdp_pass_user.c b/basic01-xdp-pass/xdp_pass_user.c
index 41b4ba2..1e934c3 100644
--- a/basic01-xdp-pass/xdp_pass_user.c
+++ b/basic01-xdp-pass/xdp_pass_user.c
@@ -6,6 +6,7 @@ static const char *__doc__ = "Simple XDP prog doing XDP_PASS\n";
 #include <string.h>
 #include <errno.h>
 #include <getopt.h>
+#include <sys/resource.h>

 #include <bpf/bpf.h>
 #include <bpf/libbpf.h>
@@ -125,6 +126,13 @@ int xdp_link_attach(int ifindex, __u32 xdp_flags, int prog_fd)

 int main(int argc, char **argv)
 {
+       struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
+
+       if (setrlimit(RLIMIT_MEMLOCK, &r)) {
+               perror("setrlimit(RLIMIT_MEMLOCK)");
+               return 1;
+       }
+

After recompiling, xdp_pass_user works:

shanssian:basic01-xdp-pass $ sudo ./xdp_pass_user --dev lo -A -F
Success: Loading XDP prog name:xdp_prog_simple(id:43) on device:lo(ifindex:1)

I have an XDP program that looks similar to https://github.com/xdp-project/xdp-tutorial/blob/master/packet-solutions/xdp_prog_kern_02.c#L21 - I use the same headers, etc. I'm building and attaching it using gobpf/bcc (if that makes a difference - until now, I've been able to copy/paste most of this tutorial's code and compile/attach it using gobpf/bcc)

If I remove parse_udphdr from the code, it loads fine (attached to lo):

int eth_type, ip_type, map_key;
struct ethhdr *eth;
struct iphdr *iphdr;
struct ipv6hdr *ipv6hdr;
struct udphdr *udphdr;
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
struct hdr_cursor nh = { .pos = data };
long *value;

eth_type = parse_ethhdr(&nh, data_end, &eth);
if (eth_type < 0)
        return XDP_ABORTED;

if (eth_type == ETH_P_IP) {
        ip_type = parse_iphdr(&nh, data_end, &iphdr);
} else if (eth_type == ETH_P_IPV6) {
        ip_type = parse_ip6hdr(&nh, data_end, &ipv6hdr)
} else {
        return XDP_PASS;
}

if (ip_type == IPPROTO_UDP) {
        //if (parse_udphdr(&nh, data_end, &udphdr) < 0)
        //      return XDP_ABORTED;
        return XDP_PASS;
}
return XDP_PASS;

If I uncomment the parse_udphdr lines, I get a pretty enormous error stacktrace, along with what's probably an error for the error message being too large:

from 308 to 324: R0=inv(id=0) R1=pkt(id=0,off=42,r=22,imm=0) R2=pkt(id=0,off=0,r=22,imm=0) R3=inv4294967295 R4=pkt(id=0,off=22,r=22,imm=0) R6=inv2 R7=pkt(id=0,off=22,r=22,imm=0) R8=pkt_end(id=0,off=0,imm=0) R10=fp0,call_-1 fp-8=mmmmmmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=mmmmmmmm fp-192=mmmmmmmm fp-200=mmmmmmmm fp-208=mmmmmmmm fp-216=mmmmmmmm fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm
324: (7b) *(u64 *)(r10 -272) = r3
325: (b7) r9 = 1
326: (63) *(u32 *)(r10 -260) = r9
327: (18) r1 = 0xffff96f623485600
329: (bf) r2 = r10
330: (07) r2 += -260
331: (85) call bpf_map_lookup_elem#1
332: safe

from 303 to 343: safe

from 302 to 317: R0=inv(id=0) R1=inv34525 R2=pkt(id=0,off=0,r=22,imm=0) R3=pkt(id=0,off=26,r=22,imm=0) R4=pkt(id=0,off=22,r=22,imm=0) R6=inv2 R7=pkt(id=0,off=22,r=22,imm=0) R8=pkt_end(id=0,off=0,imm=0) R10=fp0,call_-1 fp-8=mmmmmmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=mmmmmmmm fp-192=mmmmmmmm fp-200=mmmmmmmm fp-208=mmmmmmmm fp-216=mmmmmmmm fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm
317: (18) r3 = 0xffffffff
319: (bf) r1 = r7
320: (07) r1 += 40
321: (2d) if r1 > r8 goto pc+2
 R0=inv(id=0) R1=pkt(id=0,off=62,r=62,imm=0) R2=pkt(id=0,off=0,r=62,imm=0) R3=inv4294967295 R4=pkt(id=0,off=22,r=62,imm=0) R6=inv2 R7=pkt(id=0,off=22,r=62,imm=0) R8=pkt_end(id=0,off=0,imm=0) R10=fp0,call_-1 fp-8=mmmmmmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=mmmmmmmm fp-192=mmmmmmmm fp-200=mmmmmmmm fp-208=mmmmmmmm fp-216=mmmmmmmm fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm
322: (71) r3 = *(u8 *)(r7 +6)
323: (bf) r7 = r1
324: (7b) *(u64 *)(r10 -272) = r3
325: (b7) r9 = 1
326: (63) *(u32 *)(r10 -260) = r9
327: (18) r1 = 0xffff96f623485600
329: (bf) r2 = r10
330: (07) r2 += -260
331: (85) call bpf_map_lookup_elem#1
332: (15) if r0 == 0x0 goto pc+1
 R0=map_value(id=0,off=0,ks=4,vs=8,imm=0) R6=inv2 R7=pkt(id=0,off=62,r=62,imm=0) R8=pkt_end(id=0,off=0,imm=0) R9=inv1 R10=fp0,call_-1 fp-8=mmmmmmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=mmmmmmmm fp-192=mmmmmmmm fp-200=mmmmmmmm fp-208=mmmmmmmm fp-216=mmmmmmmm fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm fp-264=mmmm???? fp-272=mmmmmmmm
333: (db) lock *(u64 *)(r0 +0) += r9
 R0=map_value(id=0,off=0,ks=4,vs=8,imm=0) R6=inv2 R7=pkt(id=0,off=62,r=62,imm=0) R8=pkt_end(id=0,off=0,imm=0) R9=inv1 R10=fp0,call_-1 fp-8=mmmmmmmm fp-16=mmmmmmmm fp-24=mmmmmmmm fp-32=mmmmmmmm fp-40=mmmmmmmm fp-48=mmmmmmmm fp-56=mmmmmmmm fp-64=mmmmmmmm fp-72=mmmmmmmm fp-80=mmmmmmmm fp-88=mmmmmmmm fp-96=mmmmmmmm fp-104=mmmmmmmm fp-112=mmmmmmmm fp-120=mmmmmmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm fp-152=mmmmmmmm fp-160=mmmmmmmm fp-168=mmmmmmmm fp-176=mmmmmmmm fp-184=mmmmmmmm fp-192=mmmmmmmm fp-200=mmmmmmmm fp-208=mmmmmmmm fp-216=mmmmmmmm fp-224=mmmmmmmm fp-232=mmmmmmmm fp-240=mmmmmmmm fp-248=mmmmmmmm fp-256=mmmmmmmm fp-264=mmmm???? fp-272=mmmmmmmm
 R0=map_value(id=
bpf: log_buf size may be insufficient
panic: error loading BPF program: no space left on device

Hi,

I want to use the test lab VM image, where I can get it?

Thanks!

Question: some code seems quite reusable to me, but it's licensed as GPL-2.0 (only) – would you consider changing (some of) these to, say, GPL-2.0-or-later? (My particular project is GPL-3.0-or-later.)

It's not a big deal, and I could write similar functionality anew, but it feels better to avoid that. Most notably parsing_helpers.h seems nice; majority of lines there was written by @tohojo.

Side note: SPDX identifier "GPL-2.0" is deprecated now, apparently in favor of the more clear "GPL-2.0-only".

So I was working through assignment 3 in basic03-map-counter [1] and have the following questions:

Okay, so I changed BPF_MAP_TYPE_ARRAY to BPF_MAP_TYPE_PERCPU_ARRAY and added the map_get_value_percpu_array() function. Everything works as expected.

Question #1:

It says "Thus far, we have used atomic operations to increment our stats counters; however, this is expensive as it inserts memory barriers to make sure different CPUs don’t garble each other’s data. We can avoid this by using another array type that stores its data in per-CPU storage. The drawback of this is that we move the burden of summing to userspace."

How should I change the following code so that it's no longer an atomic operation?

/* LLVM maps __sync_fetch_and_add() as a built-in function to the BPF atomic add
 * instruction (that is BPF_STX | BPF_XADD | BPF_W for word sizes)
 */
#ifndef lock_xadd
#define lock_xadd(ptr, val) ((void) __sync_fetch_and_add(ptr, val))
#endif

Or is LLVM doing this somehow magically for me 'under the covers'?

Question #2:

The per CPU code executes bpf_num_possible_cpus() which returns 128 on my i9 laptop. I guess that's why it's called 'possible CPUs' :-) However, this seems a bit of a waste looping through 100+ possible CPU arrays which will never(?) be written too? If I knew I only had, say, 16 CPUs then could I somehow only loop through the first or last 16 of those 128 possible arrays. How does that work? Or is this something which will become obvious when I finish the tutorial?

    unsigned int nr_cpus = bpf_num_possible_cpus();

[1] https://github.com/xdp-project/xdp-tutorial/tree/master/basic03-map-counter

I'm working through the tutorial and read here [1]:

Besides the ability to transmit packets back from the same interface, there is an option to forward packets to egress ports of other interfaces (if the corresponding driver supports this feature). This can be done using the bpf_redirect or bpf_redirect_map helpers. These helpers will return the XDP_REDIRECT value and this is what should the program return. The bpf_redirect helper actually shouldn’t be used in production as it is slow and can’t be configured from user space.

I also came across this old post [2] which -- presuming xdp_redirect_map is old terminology for bpf_redirect_map() -- also suggests that using bpf_redirect() for XDP_REDIRECT is about half as slow as other XDP return values, or XDP_REDIRECT with bpf_redirect_map().

Questions:

• It's suggested that bpf_redirect() is not okay for production use. Is bpf_redirect_map() okay for production use?
• Are the performance figures in [2] from July 2017 still in the right ball park in Feb. 2020?
• Reading up on bpf_redirect_map() here [3] it's not clear how the packet reaches the other interface. Is the packet copied in memory, and how to find out more about the internal implementation?

[1] https://github.com/xdp-project/xdp-tutorial/tree/master/packet03-redirecting#redirecting-packets-to-other-interfaces
[2] https://lwn.net/Articles/728146/
[3] http://man7.org/linux/man-pages/man7/bpf-helpers.7.html

Hello ,
I don't understand why ./af_xdp_user -d veth-adv03 without --filename af_xdp_kern.o also can load a xdp program and eat all pings.

After ./af_xdp_user -d veth-adv03, using cmd ip a show veth-adv03, the result shows that it have loaded a xdp program truly.

If ./af_xdp_user -d veth-adv03 has loaded a xdp program truly, which xxx.o has been loaded ?

Hello, I tried the rewriting dest port number. but can not load the bpf program.
the Errors are shown below.

sudo ./xdp_loader --dev veth-basic02 --force
libbpf: load bpf program failed: Invalid argument
libbpf: failed to load program 'xdp_port_rewrite'
libbpf: failed to load object 'xdp_prog_kern.o'
ERR: loading BPF-OBJ file(xdp_prog_kern.o) (-22): Invalid argument
ERR: loading file: xdp_prog_kern.o

The parse_udphdr() and xdp_port_rewrite_func() functions are shown. i can compile the C file without problem.
static __always_inline int parse_udphdr(struct hdr_cursor *nh,
void *data_end,
struct udphdr **udphdr)
{
struct udphdr *udph = nh->pos;
if (udph + 1 > data_end)
return -1;

    nh->pos = udph + 1;
    *udphdr = udph;

    //return bpf_ntohs(udph->source);
    return bpf_ntohs(udph->dest);

}

SEC("xdp_port_rewrite")
int xdp_port_rewrite_func(struct xdp_md *ctx)
{
void *data_end = (void *)(long)ctx->data_end;
void *data = (void *)(long)ctx->data;
struct ethhdr *eth;
struct ipv6hdr *ipv6;
struct udphdr *udp;

    struct hdr_cursor nh;
    int nh_type;
    int next_p;
    /* Start next header cursor position at data start */
    nh.pos = data;

    /* Packet parsing in steps: Get each header one at a time, aborting if
     * parsing fails. Each helper function does sanity checking (is the
     * header type in the packet correct?), and bounds checking.
     */
    nh_type = parse_ethhdr(&nh, data_end, &eth);

    /* Assignment additions go below here */

    next_p = parse_ip6hdr(&nh, data_end, &ipv6);


    //xdp_parser_func(struct xdp_md *ctx);
    int udp_source;
    udp_source = parse_udphdr(&nh, data_end, &udp);
   // udp =  nh.pos;
    udp->dest = bpf_htons(bpf_ntohs(udp->dest) - 1);
    return XDP_PASS;

}

Could you kindly help me to check it? Thanks so much

The testenv script should increase the ulimit to make sure we don't get
permission denied errors...

The assignment [1] says:

You can view these with tcpdump:

$ t tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on xdptut-3c93, link-type EN10MB (Ethernet), capture size 262144 bytes
12:54:31.085948 d2:9e:c0:4f:3b:7b > 32:71:5a:a4:74:c1, ethertype IPv6 (0x86dd), length 67: fc00:dead:cafe:1::2.35126 > fc00:dead:cafe:1::1.2000: UDP, length 5

When your program is working correctly, the destination port (2000 near the end of the line) should be 1999 instead.

However, that's not exactly what happens when I run the presumably correctly changed xdp-prog-kern.c source code. Instead I get:

$ sudo ../testenv/testenv.sh setup --legacy-ip --name veth-packet02-1
Setting up new environment 'veth-packet02-1'
Setup environment 'veth-packet02-1' with peer ip fc00:dead:cafe:7::2 and 10.11.7.2.

Running ping from inside test environment:

PING fc00:dead:cafe:7::1(fc00:dead:cafe:7::1) 56 data bytes
64 bytes from fc00:dead:cafe:7::1: icmp_seq=1 ttl=64 time=0.061 ms

--- fc00:dead:cafe:7::1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.061/0.061/0.061/0.000 ms

$ make && sudo ./xdp_loader --dev veth-packet02-1 --auto-mode --force
Success: Loaded BPF-object(xdp_prog_kern.o) and used section(xdp_port_rewrite)
 - XDP prog attached on device:veth-packet02-1(ifindex:33)
 - Pinning maps in /sys/fs/bpf/veth-packet02-1/

$ time sudo ip netns exec veth-packet02-1 sh -c "tcpdump -n -XX & (sleep 1 ; echo 'foo' | socat - 'udp6:[fc00:dead:cafe:7::1]:2000' ; killall tcpdump)"
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on veth0, link-type EN10MB (Ethernet), capture size 262144 bytes
17:49:35.155662 IP6 fc00:dead:cafe:7::2.38004 > fc00:dead:cafe:7::1.2000: UDP, length 4
        0x0000:  5ec9 4de5 7cf9 463f 9fd0 54e4 86dd 600d  ^.M.|.F?..T...`.
        0x0010:  9cca 000c 1140 fc00 dead cafe 0007 0000  .....@..........
        0x0020:  0000 0000 0002 fc00 dead cafe 0007 0000  ................
        0x0030:  0000 0000 0001 9474 07d0 000c 4b89 666f  .......t....K.fo
        0x0040:  6f0a                                     o.
17:49:35.155704 IP6 fc00:dead:cafe:7::1 > fc00:dead:cafe:7::2: ICMP6, destination unreachable, unreachable port, fc00:dead:cafe:7::1 udp port 1999, length 60
        0x0000:  463f 9fd0 54e4 5ec9 4de5 7cf9 86dd 6000  F?..T.^.M.|...`.
        0x0010:  416f 003c 3a40 fc00 dead cafe 0007 0000  Ao.<:@..........
        0x0020:  0000 0000 0001 fc00 dead cafe 0007 0000  ................
        0x0030:  0000 0000 0002 0104 9c36 0000 0000 600d  .........6....`.
        0x0040:  9cca 000c 1140 fc00 dead cafe 0007 0000  .....@..........
        0x0050:  0000 0000 0002 fc00 dead cafe 0007 0000  ................
        0x0060:  0000 0000 0001 9474 07cf 000c 4b89 666f  .......t....K.fo
        0x0070:  6f0a                                     o.
2 packets captured
real    0m1.521s

So tcpdump still says the packet has a destination port 2000, but an ICMP 'destination unreachable' packet is generated which mentions unreachable port 1999.

So presumably port 2000 is correctly rewritten as port 1999 otherwise the ICMP packet would not be generated?

But why does tcpdump not show port 1999 in the first packet captured? Can / does tcpdump snaffle the packets even before XDP? Intuitively I would expect XDP to have its way with the packets before tcpdump gets to see them? Isn't this the idea with XDP that it happens as early as possible?

Kernel:

$ uname -r
5.3.0-29-generic

Disclaimer: I'm trying out the tutorial on a Ubuntu VM running on VMware. And I also didn't use the t alias suggested by the tutorial, but the longer form of the command instead. I've successfully got all the prior tutorial assignments to work as expected :-)

[1] https://github.com/xdp-project/xdp-tutorial/tree/master/packet02-rewriting

Hello, I follow the tuturial, after I install dependencies correctly. I compile the program by "make" in the folder of /xdp-tutorial, but I have the following errors. Thanks.

make -C basic03-map-counter
make[1]: Entering directory '/home/fyan/xdp-tutorial/basic03-map-counter'
make -C ../common/
make[2]: Entering directory '/home/fyan/xdp-tutorial/common'
gcc -g -Wall -I../libbpf/src//root/usr/include/ -c -o common_params.o common_params.c
common_params.c: In function ‘parse_cmdline_args’:
common_params.c:113:23: error: ‘XDP_FLAGS_MODES’ undeclared (first use in this function)
cfg->xdp_flags &= ~XDP_FLAGS_MODES; /* Clear flags /
^
common_params.c:113:23: note: each undeclared identifier is reported only once for each function it appears in
common_params.c:117:22: error: ‘XDP_FLAGS_SKB_MODE’ undeclared (first use in this function)
cfg->xdp_flags |= XDP_FLAGS_SKB_MODE; / Set flag /
^
common_params.c:121:22: error: ‘XDP_FLAGS_DRV_MODE’ undeclared (first use in this function)
cfg->xdp_flags |= XDP_FLAGS_DRV_MODE; / Set flag /
^
common_params.c:125:22: error: ‘XDP_FLAGS_HW_MODE’ undeclared (first use in this function)
cfg->xdp_flags |= XDP_FLAGS_HW_MODE; / Set flag */
^
common_params.c:128:23: error: ‘XDP_FLAGS_UPDATE_IF_NOEXIST’ undeclared (first use in this function)
cfg->xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST;
^
Makefile:13: recipe for target 'common_params.o' failed
make[2]: *** [common_params.o] Error 1
make[2]: Leaving directory '/home/fyan/xdp-tutorial/common'
../common//common.mk:100: recipe for target '../common//common_params.o' failed
make[1]: *** [../common//common_params.o] Error 2
make[1]: Leaving directory '/home/fyan/xdp-tutorial/basic03-map-counter'
Makefile:12: recipe for target 'basic03-map-counter' failed
make: *** [basic03-map-counter] Error 2

Hello,

After following all the previous exercise I had a hard time to make the last one works.
So I just tried to use the solution in the packet-solutions directory and it was still not working with this error when loading the code:

% sudo ./xdp_loader -d enp4s0f0 -A
[sudo] Mot de passe de irevoire :          
libbpf: load bpf program failed: Operation not permitted
libbpf: failed to load program 'xdp_router'
libbpf: failed to load object 'learning_kern.o'
ERR: loading BPF-OBJ file(learning_kern.o) (-22): Invalid argument
ERR: loading file: learning_kern.o

Here I've made a minimal directory with exactly the code I ran: https://github.com/irevoire/xdp-learning
I don't know if I missed something or if it's a problem in my config.

This is in the works. Submitted a pull request last Friday.

#58

Originally posted by @chaudron in #60 (comment)

Could you tell me what system you are using and what kernel you are using to make this tutorial？

For example，I'm using Ubuntu18.04 with kernel 4.19.67.

when I use packet-solutions program to load xdp prog by cmd 'xdp_loader --filename xdp_prog_kern_03.o --progsec xdp_icmp_echo --dev wlp8s0b1 ', it show me "unknown func bpf_csum_diff#28".
My linux is ubuntu18.04 kernel version is 4.15.0.

The VM-image does not show the 'tag' info with 'ip link list':

$ ip link show dev xdp
4: xdp@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 12:90:e6:f2:f0:33 brd ff:ff:ff:ff:ff:ff link-netnsid 0
prog/xdp id 40

Only the 'prog/xdp id 40' is listed, after this there should be a 'tag' like:

prog/xdp id 29 tag 3b185187f1855c4c jited

Hello. I'm having a hard time adapting the advanced03 AF_XDP code to do a similar action to UDP packets - unless I'm really fundamentally misunderstanding the direction of the rx/tx flow.

I have a socat listener on the external ip of my veth-adv03:

sevagh:~ $ socat - UDP6-LISTEN:1234,bind=[fc00:dead:cafe:1::1],fork
hello world 2
hello world 3
hello world 8
hello world 9

I send hello world 1-10 with socat via the internal ip, after running testenv.sh enter:

[root@localhost sevagh]# for x in 1 2 3 4 5 6 7 8 9 10; do echo "hello world ${x}" | socat - UDP6-SENDTO:[fc00:dead:cafe:1::1]:1234; done

Finally, I have XDP code redirecting half the packets to AF_XDP, where I simply want to do a random sleep, and send them, to scramble the packets and have them arrive out of order.

My desired result is:

sevagh:~ $ socat - UDP6-LISTEN:1234,bind=[fc00:dead:cafe:1::1],fork
hello world 2 // XDP_PASS
hello world 3 // XDP_PASS
hello world 8 // XDP_PASS
hello world 9 // XDP_PASS
hello world 4 // AF_XDP + random sleep
hello world 1 // AF_XDP + random sleep
hello world 10 // AF_XDP + random sleep
...

However, in my AF_XDP code, the packet doesn't go anywhere:

static bool process_packet(struct xsk_socket_info *xsk, uint64_t addr,
			   uint32_t len, int udp4_out, int udp6_out)
{
	uint8_t *pkt = xsk_umem__get_data(xsk->umem->buffer, addr);

	int ret;
	uint32_t tx_idx = 0;

	// sleep randomly to scramble
	dawdle();

	ret = xsk_ring_prod__reserve(&xsk->tx, 1, &tx_idx);
	if (ret != 1) {
		/* No more transmit slots, drop the packet */
		return false;
	}

	xsk_ring_prod__tx_desc(&xsk->tx, tx_idx)->addr = addr;
	xsk_ring_prod__tx_desc(&xsk->tx, tx_idx)->len = len;
	xsk_ring_prod__submit(&xsk->tx, 1);
	xsk->outstanding_tx++;

	return true;
}

As a workaround, I can send those AF_XDP rx packets above, on a different, regular AF_INET6 UDP socket, to achieve my desired goal:

static bool process_packet(struct xsk_socket_info *xsk, uint64_t addr,
			   uint32_t len, int udp4_out, int udp6_out)
{
	uint8_t *pkt = xsk_umem__get_data(xsk->umem->buffer, addr);

        ....

        ssize_t sent_bytes;
        struct sockaddr_in6 sin;
        sin.sin6_family = AF_INET6;
        sin.sin6_port = udphdr->dest;
        inet_pton(AF_INET6, "::1", &sin.sin6_addr);
        sent_bytes = sendto(udp6_out, (void *)nh.pos, len, 0,
                                   (struct sockaddr *)&sin, sizeof(sin))

Is using a regular UDP socket the only way to achieve what I need, or should it be possible to do so with AF_XDP?

As the title say.

Firstly execute make command in the advanced03-AF_XDP dir.

$ make

Then I will get the following error.

clang -S \
    -target bpf \
    -D __BPF_TRACING__ \
    -I../libbpf/src//build/usr/include/ -I../headers/ \
    -Wall \
    -Wno-unused-value \
    -Wno-pointer-sign \
    -Wno-compare-distinct-pointer-types \
    -Werror \
    -O2 -emit-llvm -c -g -o af_xdp_kern.ll af_xdp_kern.c
af_xdp_kern.c:5:10: fatal error: 'bpf_helpers.h' file not found
#include "bpf_helpers.h"
         ^~~~~~~~~~~~~~~
1 error generated.
make: *** [../common//common.mk:111: af_xdp_kern.o] Error 1

By the way, when I git clone from https://github.com/chaudron/xdp-tutorial , there is no problem.

When I use the command below to inject the xdp_pass_kern.o to kernel :

ip link set dev eth1 xdpgeneric obj xdp_pass_kern.o sec xdp

it occered that:

BTF debug data section '.BTF' rejected: Invalid argument (22)!
 - Length:       554
Verifier analysis:

magic: 0xeb9f
version: 1
flags: 0x0
hdr_len: 24
type_off: 0
type_len: 256
str_off: 256
str_len: 274
btf_total_size: 554
[1] FUNC_PROTO (anon) return=2 args=(3 (anon))
[2] INT int size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR (anon) type_id=4
[4] STRUCT xdp_md size=20 vlen=5
        data type_id=5 bits_offset=0
        data_end type_id=5 bits_offset=32
        data_meta type_id=5 bits_offset=64
        ingress_ifindex type_id=5 bits_offset=96
        rx_queue_index type_id=5 bits_offset=128
[5] TYPEDEF __u32 type_id=6
[6] INT unsigned int size=4 bits_offset=0 nr_bits=32 encoding=(none)
[7] FUNC xdp_prog_simple type_id=1
[8] INT char size=1 bits_offset=0 nr_bits=8 encoding=SIGNED
[9] ARRAY (anon) type_id=8 index_type_id=10 nr_elems=4
[10] INT __ARRAY_SIZE_TYPE__ size=4 bits_offset=0 nr_bits=32 encoding=(none)
[11] VAR _license type_id=9 linkage=1
[12] DATASEC license size=0 vlen=1 size == 0

And then I use the command below to check whether it's attached to the kernel , the answer is yes.

# ip link show dev eth1
6: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether f8:f2:1e:32:3b:e0 brd ff:ff:ff:ff:ff:ff
    prog/xdp id 131163 tag b8a375b5b20c00
    [...]

SO, why would that Invalid argument(22) happened ?
And if I use libbpf library ,the print log is below (I added some print log)

# ./xdp_pass_user --dev eth1 --skb-mode
libbpf: Error loading ELF section .BTF: -22. Ignored and continue.
BPF-prog is only loaded by the kernel and get the prog_fd.
prog_fd is : 3
start attaching the prog_fd to a kernel hook point
the err code of xdp_link_attach() is 0
xdp_link_attach() success
Success: Loading XDP prog name:xdp_prog_simple(id:131169) on device:eth1(ifindex:6)

it seems the same probelm.
But when I run the samples/bpf in the kernel tree , there's no probelm.
I searched the google but didn't get any useful info. I have no idea how to solve this probelm.
could you give me some advice?

more info:
the kernel version is 5.1.0-rc7+ .
the llvm version is 9.0.0

When I try basic04-pinning-maps with the follow command after Make， there is erros as follow：
[root@localhost basic04-pinning-maps]# sudo ./xdp_loader --dev veth-basic03 --force --progsec xdp_pass
libbpf: BTF is required, but is missing or corrupted.
ERR: loading BPF-OBJ file(xdp_prog_kern.o) (-2): No such file or directory
ERR: loading file: xdp_prog_kern.o

[root@localhost basic04-pinning-maps]# sudo ./xdp_loader --dev lo --force --progsec xdp_pass
libbpf: BTF is required, but is missing or corrupted.
ERR: loading BPF-OBJ file(xdp_prog_kern.o) (-2): No such file or directory
ERR: loading file: xdp_prog_kern.o

[root@localhost basic04-pinning-maps]# ls /sys/fs/
bpf/ cgroup/ pstore/ selinux/ xfs/
[root@localhost basic04-pinning-maps]# ls /sys/fs/bpf/

Hi there

I want to test the XDP program, but the script only provides t ping, which is ICMP, and small size packet, I can modify the ping parameter, but still this is only ICMP.

How can I generate traffic for virtual interfaces to test other types of traffic? (e.g. by iperf3?)

Thanks

The advanced03-AF_XDP example/assignment does not compile on my system. It is missing a header include file named: bpf/xsk.h.

Output is:

advanced03-AF_XDP]$ make
cc -Wall -I../libbpf/src//root/usr/include/ -g -I/usr/include/x86_64-linux-gnu -I../headers/ -L../libbpf/src/ -o af_xdp_user ../common//common_params.o ../common//common_user_bpf_xdp.o \
 af_xdp_user.c -l:libbpf.a -lelf  -lpthread
af_xdp_user.c:19:10: fatal error: bpf/xsk.h: No such file or directory
 #include <bpf/xsk.h>
          ^~~~~~~~~~~
compilation terminated.
make: *** [../common//common.mk:105: af_xdp_user] Error 1

The header file xsk.h is provided by libbpf, but not under subdir bpf/. In my git tree the file is located in: libbpf/src/xsk.h and XDP-tutorial does have this (../libbpf/src/) as object search/link dir in above compile command.

I guess libbpf should have "installed" this header file in: ../libbpf/src//root/usr/include/ but it only contains:

$ ls -1 libbpf/src//root/usr/include/bpf/
bpf.h
btf.h
libbpf.h

Hello,
Simply running af_xdp_user on an updated Stretch (4.19.0-5-amd64):

 # grep XDP /boot/config-4.19.0-5-amd64 
 CONFIG_XDP_SOCKETS=y
 # ./af_xdp_user -d eth20
 ERROR: Can't create umem "Cannot allocate memory"

A bit of gdb shows this happens in xsk_umem_create() :

   map = mmap(NULL, off.cr.desc + umem->config.comp_size * sizeof(__u64),
               PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, umem->fd,
               XDP_UMEM_PGOFF_COMPLETION_RING);

The mmap fails because off.cr.desc is a ridiculous number:

(gdb) print off
$2 = {rx = {producer = 0, consumer = 64, desc = 128, flags = 0}, tx = {producer = 64, consumer = 128, desc = 0, flags = 64}, fr = {producer = 128, consumer = 0, desc = 64, flags = 128},
cr = {producer = 112, consumer = 134112, desc = 140737353639664, flags = 93824992375840}}

Recently I was using xdp/ebpf with kubernetes.
When I want to run an ebpf program for a specific kubernetes pod I use command kubectl exec -it podname bash to enter the pod and install the libraries and dependencies then load the ebpf program. I'm wondering if there's a way to load ebpf from the host without entering the pods and achieve the same goal as above.
In this tutorial the ebpf program loader loads the program to a specific network interface using the interface name(eth0, ens224 etc...). The problem I was facing is that using sriov to attach the network interface from the host into the pods makes the network interface disappear from the host while using commands such as ifconfig to view the interfaces so I cannot specify the interface name in order to load the program.
Is there any workaround?

Would be nice to have aa tutorial for syncookie handling, simple synflood filter with bpf_tcp_check_syncookie function.

Kernel already has this test as a starting point
https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/test_tcp_check_syncookie_kern.c

I'm trying to get the Advanced03_AF_XDP running in Fedora, which runs kernel 5.3.0.rc6.

CONFIG_XDP_SOCKETS=y is correctly configured.

I'm running the following commands as root:

cd advanced03-AF_XPD
make
t setup --name veth-adv03

The last command results in 100% packet loss when running ping. Same happens then for t ping of course.

./af_xdp_user -d veth-adv03

This prints the following error:
ERROR: Can't create umem "Invalid argument"

Any clue how I can fix this? I have also tried compiling it on the released VM, but that version did not include Advanced 3 and I'm not able to compile the new code I pulled. I would appreciate any pointer! :)

First, thank you for these tutorial resources. It's awesome that you're putting this out there.

I am struggling with basic04 assignment 1. After a certain amount of beating my head against the wall, it would be helpful if I could look at a solution (or an example program which solves this problem) to understand the changes and move on. However, the tutorial doesn't have a solution for this step. After a fair amount of digging around, I thought about skipping this assignment and moving on, but the next step in the tutorial (packet01) pulls in our solution from basic04 via Makefile, so it seems like I cannot move on without solving it.

So I am not asking for things empty-handed, here are the things that I have tried:

1. Use bpf_obj_get to retrieve the file descriptor for the pinned map by supplying the pathname to the pinned map file in bpffs. I do not understand why, but each call to bpf_obj_get increments the fd by 1. So if my initial map_fd is 3, bpf_obj_get will return 4, 5, 6, etc... Some other documentation recommended this program for example usage of bpf_obj_get and has a very similar format to our program, but it also does not solve this specific problem (it is a TODO).
2. The assignment suggests to remember the original map_id. So I took the map_id from the info struct populated by the original call to check_map_fd_info in main, and passed it into the stats_poll function. For each poll loop, I use bpf_obj_get_info_by_fd to see if the map_id has changed at that particular file descriptor. However, it never changes, despite me loading a new BPF map with xdp_loader. I assume the problem is I am getting the map_id from the original map file descriptor, and the new map has a different file descriptor. If this is true, that would imply I need to probe when the file descriptor changes, which takes me back to the solution I tried previously and does not work.

Although having a solution in the tutorial would probably be best to ensure the tutorial is self contained, that may be more appropriate in the long term. In the short term, any help is appreciated. Thanks.

The loader will output the function name, which is different from the section
name; explain that this is the case (and why).

[root@localhost packet03-redirecting]# make
...
[root@localhost packet03-redirecting]# t exec -n left -- ./xdp_loader -d veth0 -F --progsec xdp_pass
libbpf: Error in bpf_object__probe_name():Operation not permitted(1). Couldn't load basic 'r0 = 0' BPF program.
libbpf: Error in bpf_object__probe_global_data():Operation not permitted(1). Couldn't create simple array map.
libbpf: failed to create map (name: 'xdp_stats_map'): Operation not permitted(-1)
libbpf: failed to load object 'xdp_prog_kern.o'
ERR: loading BPF-OBJ file(xdp_prog_kern.o) (-22): Invalid argument
ERR: loading file: xdp_prog_kern.o

What I did:

• Setup virtual interface via $ sudo ./testenv.sh setup --name="veth-basic02"
• Constantly pinging the interface via $ sudo ./testenv ping
• Changing af_xdp_kern.c to:

SEC("xdp_sock")
int xdp_sock_prog(struct xdp_md *ctx) {
    return XDP_DROP;
}

• Compiling everything and then $ sudo ./af_xdp_user --dev veth-basic02 --force

As you can see, I drop every packet in the kernel, but for whatever reason, the user-space application is still receiving packets:

AF_XDP RX:             2 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000218
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000218

AF_XDP RX:             4 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000136
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000136

AF_XDP RX:             6 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000119
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000119

AF_XDP RX:             8 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000114
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000114

AF_XDP RX:            10 pkts (         1 pps)           1 Kbytes (     0 Mbits/s) period:2.000111
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000111

If I get the sequence number of the pings via printf("RECEIEVD ICMP6 PACKET WITH SEQUENCE: %d\n", ntohs(icmp->icmp6_sequence)); inside the process_packet-function, I further noticed that they are valid ping requests:

RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2144
RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2145
AF_XDP RX:             2 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000160
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000160

RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2146
RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2147
AF_XDP RX:             4 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000129
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000129

RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2148
RECEIEVD ICMP6 PACKET WITH SEQUENCE: 2149
AF_XDP RX:             6 pkts (         1 pps)           0 Kbytes (     0 Mbits/s) period:2.000114
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000114

In my opinion, the user space application should not receive anything if the kernel BPF-program decides to drop everything.

What did I do incorrectly? Did I understand something wrong?

@tohojo changed parse_ethhdr() to skip VLANs in commit 455cb40 to skip all VLAN headers, and lets the return value be in host byte order.

1. Is it good/expected that VLANs are skipped?

As ethhdr pointer still points to the original, thus callers can still detect this is a VLAN packet, by looking at eth->h_proto, and use proto_is_vlan(eth->h_proto). Which is that @aspsk did in solution to packet02 (packet-solutions/xdp_prog_kern_02.c)

2. h_proto being return as host byte order also seems wrong.

First of all, it's not a constant thus compiler cannot do this compile-time.
Second, cannot use proto_is_vlan() in the return value, as this helper expect network-byte-order.

I complile the project but fails in arm 64. because of libz.so.
so,I solve the problem by the following steps :
################ download zlib-1.2.8.tar.gz################
################ make & install zlib-1.2.8 ####################

./configure

make && make install
headers will be installed in /usr/local/include/ and libz.so.1.2.8 will be installed in /usr/local/lib/
ls /usr/local/include/

ls /usr/local/lib/

################edit common/common.mk ####################

LDFLAGS ?= -L$(LIBBPF_DIR) -L/usr/local/lib

LIBS = -l:libbpf.a -lelf $(USER_LIBS) -lz -l:libz.so.1.2.8

the solution is here:https://github.com/magnate3/xdp

The READMEs claim that I could t ping after setting up the alias using eval $(../testenv/testenv.sh alias). But in fact I have to t exec ping.

Fail to load xdp_prog_kern_03.o with error: invalid bpf_context access off=12 size=4 on SEC('xdp_router')

• ubuntu18.04
• kernel-4.15

With llvm-objdump -S xdp_prog_kern.o , I found that it was caused by line 290fib_params.ifindex = ctx->ingress_ifindex;. And then I found in kernel-4.15 the xdp_md
is defined as below without the field of ingress_ifindex.

struct xdp_md {
	__u32 data;
	__u32 data_end;
	__u32 data_meta;
}

Thus it is not an issue.

DUMP LOG

libbpf: 
0: (bf) r6 = r1
1: (61) r1 = *(u32 *)(r6 +4)
2: (61) r7 = *(u32 *)(r6 +0)
3: (b7) r2 = 0
4: (7b) *(u64 *)(r10 -16) = r2
5: (7b) *(u64 *)(r10 -24) = r2
6: (7b) *(u64 *)(r10 -32) = r2
7: (7b) *(u64 *)(r10 -40) = r2
8: (7b) *(u64 *)(r10 -48) = r2
9: (7b) *(u64 *)(r10 -56) = r2
10: (7b) *(u64 *)(r10 -64) = r2
11: (7b) *(u64 *)(r10 -72) = r2
12: (bf) r8 = r7
13: (07) r8 += 14
14: (b7) r2 = 1
15: (2d) if r8 > r1 goto pc+102
 R1=pkt_end(id=0,off=0,imm=0) R2=inv1 R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=0,r=14,imm=0) R8=pkt(id=0,off=14,r=14,imm=0) R10=fp0
16: (71) r2 = *(u8 *)(r7 +12)
17: (71) r9 = *(u8 *)(r7 +13)
18: (67) r9 <<= 8
19: (4f) r9 |= r2
20: (55) if r9 != 0x8 goto pc+24
 R1=pkt_end(id=0,off=0,imm=0) R2=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=0,r=14,imm=0) R8=pkt(id=0,off=14,r=14,imm=0) R9=inv8 R10=fp0
21: (bf) r3 = r7
22: (07) r3 += 34
23: (b7) r2 = 1
24: (2d) if r3 > r1 goto pc+93
 R1=pkt_end(id=0,off=0,imm=0) R2=inv1 R3=pkt(id=0,off=34,r=34,imm=0) R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=0,r=34,imm=0) R8=pkt(id=0,off=14,r=34,imm=0) R9=inv8 R10=fp0
25: (71) r1 = *(u8 *)(r7 +22)
26: (b7) r2 = 2
27: (2d) if r2 > r1 goto pc+90
 R1=inv(id=0,umin_value=2,umax_value=255,var_off=(0x0; 0xff)) R2=inv2 R3=pkt(id=0,off=34,r=34,imm=0) R6=ctx(id=0,off=0,imm=0) R7=pkt(id=0,off=0,r=34,imm=0) R8=pkt(id=0,off=14,r=34,imm=0) R9=inv8 R10=fp0
28: (b7) r1 = 2
29: (73) *(u8 *)(r10 -72) = r1
30: (71) r1 = *(u8 *)(r7 +15)
31: (73) *(u8 *)(r10 -60) = r1
32: (71) r1 = *(u8 *)(r7 +23)
33: (b7) r2 = 0
34: (6b) *(u16 *)(r10 -68) = r2
35: (6b) *(u16 *)(r10 -70) = r2
36: (73) *(u8 *)(r10 -71) = r1
37: (69) r1 = *(u16 *)(r7 +16)
38: (dc) r1 = be16 r1
39: (6b) *(u16 *)(r10 -66) = r1
40: (61) r1 = *(u32 *)(r7 +26)
41: (63) *(u32 *)(r10 -56) = r1
42: (61) r1 = *(u32 *)(r7 +30)
43: (63) *(u32 *)(r10 -40) = r1
44: (05) goto pc+44
89: (61) r1 = *(u32 *)(r6 +12)
invalid bpf_context access off=12 size=4

libbpf: -- END LOG --

If a program is loaded in generic mode, the loader is not smart enough to unload
it when trying to load a new program in native mode, even when using the --force
parameter.

I was struggling with the assignment of parsing the IP header

See my code below:
static __always_inline int parse_ip6hdr(struct hdr_cursor *nh,
void *data_end,
struct ipv6hdr **ip6hdr)
{
struct ipv6hdr *ip6h = *nexthdr;
if (ip6h + 1 > data_end)
return -1;
}

There is complain of nexthdr, I check the definition of the ipv6.h.
I tried struct ipv6hdr struct ipv6hdr *ip6h = **ip6hdr.nexthdr and struct ipv6hdr *ip6h = &(nh->pos);
it does not work neither.
Could you provide me with a correct sentence of parsing the IP header. Thanks a lot.

How to write a map in user space for kernel space to read？I have not found in any tutorial.

The example in tracing02-xdp-monitor is not showing correct stats for tracepoint xdp:xdp_devmap_xmit

This is the output from the program, where I expected to see info in devmap-xmit:

XDP-event       CPU:to  pps          drop-pps     extra-info
XDP_REDIRECT    0       4,881,438    0            Success
XDP_REDIRECT    1       4,771,660    0            Success
XDP_REDIRECT    3       5,087,495    0            Success
XDP_REDIRECT    4       14,552       0            Success
XDP_REDIRECT    5       5,114,913    0            Success
XDP_REDIRECT    total   19,870,057   0            Success
XDP_REDIRECT    total   0            0            Error
cpumap-kthread  total   0            0            0          
devmap-xmit     total   0            0            0.00       bulk-average drv-err

On the Device Under Test (DUT) the kernel/samples/bpf program called xdp_redirect_map is used like this:

$ sudo ./xdp_redirect_map $(</sys/class/net/mlx5p1/ifindex) $(</sys/class/net/i40e1/ifindex)
input: 8 output: 3
ifindex 3:   24431180 pkt/s
ifindex 3:   19885634 pkt/s
ifindex 3:   19882904 pkt/s
ifindex 3:   19882206 pkt/s

The packet generator is running pktgen_sample05_flow_per_thread.sh with -t 12 threads.