libbpf / libbpf-bootstrap Goto Github PK
View Code? Open in Web Editor NEWScaffolding for BPF application development with libbpf and BPF CO-RE
License: BSD 3-Clause "New" or "Revised" License
Scaffolding for BPF application development with libbpf and BPF CO-RE
License: BSD 3-Clause "New" or "Revised" License
Hi,
When running make minimal
from examples/c folder
, I am getting the following error :
GEN-SKEL .output/minimal.skel.h
Segmentation fault (core dumped)
Makefile:110: recipe for target '.output/minimal.skel.h' failed
make: *** [.output/minimal.skel.h] Error 139
make: *** Deleting file '.output/minimal.skel.h'
The steps I have followed for installation are :
git clone https://github.com/libbpf/libbpf-bootstrap.git
git submodule update --init --recursive
cd examples/c
make minimal
I have also googled for Error 139 and have also tried the solution in the closed issues #69 #70 #72 #15 but couldn't find a solution that works for me.
I and gone through the Makefile line 109 - 110. I tried removing | $(OUTPUT) $(BPFTOOL)
from line 109 $(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL)
but that too didn't work.
The current version of clang and kernel installed on Ubuntu 18.04:
I am not sure what's missing.
When trying to build the project using cmake, I've got the following error:
bash: line 1: /home/mariusz/git/libbpf-bootstrap/examples/c/../../tools/bpftool: No such file or directory
make[2]: *** [CMakeFiles/bootstrap.dir/build.make:74: bootstrap.skel.h] Error 127
make[2]: *** Deleting file 'bootstrap.skel.h'
make[1]: *** [CMakeFiles/Makefile2:145: CMakeFiles/bootstrap.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
Hi team,
I'm learning ebpf, and would like to build an app from scratch. So I'd like to know where to get vmlinux.h
?
BTW, do we have slack?
-- K
I commented on #68 about how I could not get this to run on ubuntu 18.04. I was able to get minimal working, by removing the global variable. I am using kernel version 4.15.0-180-generic. The kprobe example, however, is not running for some other reason:
sudo ./kprobe
libbpf: loading object 'kprobe_bpf' from buffer
libbpf: elf: section(2) kprobe/do_unlinkat, size 152, link 0, flags 6, type=1
libbpf: sec 'kprobe/do_unlinkat': found program 'do_unlinkat' at insn offset 0 (0 bytes), code size 19 insns (152 bytes)
libbpf: elf: section(3) kretprobe/do_unlinkat, size 88, link 0, flags 6, type=1
libbpf: sec 'kretprobe/do_unlinkat': found program 'do_unlinkat_exit' at insn offset 0 (0 bytes), code size 11 insns (88 bytes)
libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
libbpf: license of kprobe_bpf is Dual BSD/GPL
libbpf: elf: section(5) .rodata, size 72, link 0, flags 2, type=1
libbpf: elf: section(6) .BTF, size 1482, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 364, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 240, link 14, flags 0, type=2
libbpf: elf: section(9) .relkprobe/do_unlinkat, size 16, link 8, flags 0, type=9
libbpf: elf: section(10) .relkretprobe/do_unlinkat, size 16, link 8, flags 0, type=9
libbpf: looking for externs among 10 symbols...
libbpf: collected 0 externs total
libbpf: map 'kprobe_b.rodata' (global data): at sec_idx 5, offset 0, flags 480.
libbpf: map 0 is "kprobe_b.rodata"
libbpf: sec '.relkprobe/do_unlinkat': collecting relocation for section(2) 'kprobe/do_unlinkat'
libbpf: sec '.relkprobe/do_unlinkat': relo #0: insn #12 against '.rodata'
libbpf: prog 'do_unlinkat': found data map 0 (kprobe_b.rodata, sec 5, off 0) for insn 12
libbpf: sec '.relkretprobe/do_unlinkat': collecting relocation for section(3) 'kretprobe/do_unlinkat'
libbpf: sec '.relkretprobe/do_unlinkat': relo #0: insn #3 against '.rodata'
libbpf: prog 'do_unlinkat_exit': found data map 0 (kprobe_b.rodata, sec 5, off 0) for insn 3
libbpf: Kernel doesn't support BTF, skipping uploading it.
libbpf: map 'kprobe_b.rodata': skipped auto-creating...
libbpf: sec 'kprobe/do_unlinkat': found 2 CO-RE relocations
libbpf: CO-RE relocating [2] struct pt_regs: found target candidate [14] struct pt_regs in [vmlinux]
libbpf: prog 'do_unlinkat': relo #0: <byte_off> [2] struct pt_regs.si (0:13 @ offset 104)
libbpf: prog 'do_unlinkat': relo #0: non-matching candidate #0 <byte_off> [14] struct pt_regs (0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #0: no matching targets found
libbpf: prog 'do_unlinkat': relo #0: substituting insn #0 w/ invalid insn
libbpf: prog 'do_unlinkat': relo #1: <byte_off> [7] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #1: no matching targets found
libbpf: prog 'do_unlinkat': relo #1: substituting insn #3 w/ invalid insn
libbpf: sec 'kretprobe/do_unlinkat': found 1 CO-RE relocations
libbpf: prog 'do_unlinkat_exit': relo #0: <byte_off> [2] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'do_unlinkat_exit': relo #0: matching candidate #0 <byte_off> [14] struct pt_regs.ax (0:0 @ offset 80)
libbpf: prog 'do_unlinkat_exit': relo #0: patched insn #0 (LDX/ST/STX) off 80 -> 80
libbpf: prog 'do_unlinkat': relo #2: poisoning insn #12 that loads map #0 'kprobe_b.rodata'
libbpf: prog 'do_unlinkat_exit': relo #1: poisoning insn #3 that loads map #0 'kprobe_b.rodata'
libbpf: prog 'do_unlinkat': BPF program load failed: Invalid argument
libbpf: prog 'do_unlinkat': -- BEGIN PROG LOAD LOG --
0: <invalid CO-RE relocation>
failed to resolve CO-RE relocation <byte_off> [2] struct pt_regs.si (0:13 @ offset 104)
-- END PROG LOAD LOG --
libbpf: failed to load program 'do_unlinkat'
libbpf: failed to load object 'kprobe_bpf'
libbpf: failed to load BPF skeleton 'kprobe_bpf': -22
Failed to load and verify BPF skeleton
[root@x src]# yum install clang
[root@x src]# make
MKDIR .output
MKDIR .output/libbpf
LIB libbpf.a
MKDIR staticobjs
CC bpf.o
CC btf.o
CC libbpf.o
CC libbpf_errno.o
CC netlink.o
CC nlattr.o
CC str_error.o
CC libbpf_probes.o
CC bpf_prog_linfo.o
CC xsk.o
CC btf_dump.o
CC hashmap.o
CC ringbuf.o
AR libbpf.a
INSTALL bpf.h libbpf.h btf.h xsk.h libbpf_util.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h libbpf_common.h
INSTALL libbpf.pc
INSTALL libbpf.a
BPF .output/minimal.bpf.o
error: unknown target triple 'bpf', please use -triple or -arch
make: *** [.output/minimal.bpf.o]
I'm using Vagrant (box: bento/ubuntu-20.10
) to compile and run the xdp example. Compilation works fine, but when I run the app I get this error (I added the debug option):
vagrant@vagrant:/vagrant$ sudo ./target/debug/xdppass
libbpf: loading object 'xdppass_bpf' from buffer
libbpf: elf: section(3) xdp, size 120, link 0, flags 6, type=1
libbpf: sec 'xdp': found program 'xdp_pass' at insn offset 0 (0 bytes), code size 15 insns (120 bytes)
libbpf: elf: section(4) .rodata.str1.1, size 16, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(4) .rodata.str1.1
libbpf: elf: section(5) license, size 4, link 0, flags 3, type=1
libbpf: license of xdppass_bpf is GPL
libbpf: elf: section(12) .BTF, size 669, link 0, flags 0, type=1
libbpf: elf: section(14) .BTF.ext, size 220, link 0, flags 0, type=1
libbpf: elf: section(21) .symtab, size 912, link 1, flags 0, type=2
libbpf: looking for externs among 38 symbols...
libbpf: collected 0 externs total
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: sec 'xdp': found 2 CO-RE relocations
libbpf: prog 'xdp_pass': relo #0: kind <byte_off> (0), spec is [2] struct xdp_md.data_end (0:1 @ offset 4)
libbpf: CO-RE relocating [0] struct xdp_md: found target candidate [20548] struct xdp_md in [vmlinux]
libbpf: prog 'xdp_pass': relo #0: matching candidate #0 [20548] struct xdp_md.data_end (0:1 @ offset 4)
libbpf: prog 'xdp_pass': relo #0: patched insn #0 (LDX/ST/STX) off 4 -> 4
libbpf: prog 'xdp_pass': relo #1: kind <byte_off> (0), spec is [2] struct xdp_md.data (0:0 @ offset 0)
libbpf: prog 'xdp_pass': relo #1: matching candidate #0 [20548] struct xdp_md.data (0:0 @ offset 0)
libbpf: prog 'xdp_pass': relo #1: patched insn #1 (LDX/ST/STX) off 0 -> 0
libbpf: prog 'xdp_pass': failed to attach to xdp: Invalid argument
Error: System error, errno: -22
I believe this may be because libbpf needs to use SKB_MODE
when running in a virtualized environment, but I don't believe libbpf-rs allows you to set that flag.
$ xmake f -p android && xmake
[ 34%]: ccache compiling.release ../../libbpf/src/libbpf.c
[ 43%]: ccache compiling.release ../../libbpf/src/usdt.c
[ 43%]: ccache compiling.release ../../libbpf/src/hashmap.c
[ 46%]: ccache compiling.release ../../libbpf/src/strset.c
error: ../../libbpf/src/usdt.c:562:7: error: unknown type name 'GElf_Nhdr'
GElf_Nhdr *nhdr, const char *data, size_t name_off, size_t desc_off,
^
../../libbpf/src/usdt.c:578:11: error: expected ';' after expression
GElf_Nhdr nhdr;
^
;
../../libbpf/src/usdt.c:578:2: error: use of undeclared identifier 'GElf_Nhdr'
GElf_Nhdr nhdr;
^
../../libbpf/src/usdt.c:578:12: error: use of undeclared identifier 'nhdr'; did you mean 'ehdr'?
GElf_Nhdr nhdr;
^~~~
ehdr
../../libbpf/src/usdt.c:577:12: note: 'ehdr' declared here
GElf_Ehdr ehdr;
^
> in ../../libbpf/src/usdt.c
Recently, I am trying to use bpf ringbuf in uprobe example of libbpf. But when running, error occurred which is "libbpf: load bpf program failed: Invalid argument". I have no idea why this happened. Could anyone help? Below is my test code.
Kernel space code: uprobe.bpf.c, define a rb struct, and use bpf_ringbuf_reserve in uprobe code block.
#include <linux/bpf.h>
#include <linux/ptrace.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
char LICENSE[] SEC("license") = "Dual BSD/GPL";
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 256 * 1024);
} rb SEC(".maps");
SEC("uprobe/func")
int BPF_KPROBE(uprobe, int a, int b)
{
__u64* e = bpf_ringbuf_reserve(&rb, sizeof(__u64), 0);
if (!e)
return 0;
bpf_printk("UPROBE ENTRY: a = %d, b = %d\n", a, b);
return 0;
}
SEC("uretprobe/func")
int BPF_KRETPROBE(uretprobe, int ret)
{
bpf_printk("UPROBE EXIT: return = %d\n", ret);
return 0;
}
User space code: uprobe.c
#include <errno.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/resource.h>
#include <bpf/libbpf.h>
#include "uprobe.skel.h"
static int libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
{
return vfprintf(stderr, format, args);
}
static void bump_memlock_rlimit(void)
{
struct rlimit rlim_new = {
.rlim_cur = RLIM_INFINITY,
.rlim_max = RLIM_INFINITY,
};
if (setrlimit(RLIMIT_MEMLOCK, &rlim_new)) {
fprintf(stderr, "Failed to increase RLIMIT_MEMLOCK limit!\n");
exit(1);
}
}
/* Find process's base load address. We use /proc/self/maps for that,
* searching for the first executable (r-xp) memory mapping:
*
* 5574fd254000-5574fd258000 r-xp 00002000 fd:01 668759 /usr/bin/cat
* ^^^^^^^^^^^^ ^^^^^^^^
*
* Subtracting that region's offset (4th column) from its absolute start
* memory address (1st column) gives us the process's base load address.
*/
static long get_base_addr() {
size_t start, offset;
char buf[256];
FILE *f;
f = fopen("/proc/self/maps", "r");
if (!f)
return -errno;
while (fscanf(f, "%zx-%*x %s %zx %*[^\n]\n", &start, buf, &offset) == 3) {
if (strcmp(buf, "r-xp") == 0) {
fclose(f);
return start - offset;
}
}
fclose(f);
return -1;
}
static int handle_event(void *ctx, void *data, size_t data_sz)
{
return 0;
}
/* It's a global function to make sure compiler doesn't inline it. */
int uprobed_function(int a, int b)
{
return a + b;
}
int main(int argc, char **argv)
{
struct ring_buffer *rb = NULL;
struct uprobe_bpf *skel;
long base_addr, uprobe_offset;
int err, i;
/* Set up libbpf errors and debug info callback */
libbpf_set_print(libbpf_print_fn);
/* Bump RLIMIT_MEMLOCK to allow BPF sub-system to do anything */
bump_memlock_rlimit();
/* Load and verify BPF application */
skel = uprobe_bpf__open_and_load();
if (!skel) {
fprintf(stderr, "Failed to open and load BPF skeleton\n");
return 1;
}
base_addr = get_base_addr();
if (base_addr < 0) {
fprintf(stderr, "Failed to determine process's load address\n");
err = base_addr;
goto cleanup;
}
/* uprobe/uretprobe expects relative offset of the function to attach
* to. This offset is relateve to the process's base load address. So
* easy way to do this is to take an absolute address of the desired
* function and substract base load address from it. If we were to
* parse ELF to calculate this function, we'd need to add .text
* section offset and function's offset within .text ELF section.
*/
uprobe_offset = (long)&uprobed_function - base_addr;
/* Attach tracepoint handler */
skel->links.uprobe = bpf_program__attach_uprobe(skel->progs.uprobe,
false /* not uretprobe */,
0 /* self pid */,
"/proc/self/exe",
uprobe_offset);
err = libbpf_get_error(skel->links.uprobe);
if (err) {
fprintf(stderr, "Failed to attach uprobe: %d\n", err);
goto cleanup;
}
/* we can also attach uprobe/uretprobe to any existing or future
* processes that use the same binary executable; to do that we need
* to specify -1 as PID, as we do here
*/
skel->links.uretprobe = bpf_program__attach_uprobe(skel->progs.uretprobe,
true /* uretprobe */,
-1 /* any pid */,
"/proc/self/exe",
uprobe_offset);
err = libbpf_get_error(skel->links.uretprobe);
if (err) {
fprintf(stderr, "Failed to attach uprobe: %d\n", err);
goto cleanup;
}
/* Set up ring buffer polling */
rb = ring_buffer__new(bpf_map__fd(skel->maps.rb), handle_event, NULL, NULL);
if (!rb) {
err = -1;
fprintf(stderr, "Failed to create ring buffer\n");
goto cleanup;
}
printf("Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` "
"to see output of the BPF programs.\n");
for (i = 0; ; i++) {
err = ring_buffer__poll(rb, 100 /* timeout, ms */);
/* trigger our BPF programs */
fprintf(stderr, ".");
uprobed_function(i, i + 1);
sleep(1);
}
cleanup:
ring_buffer__free(rb);
uprobe_bpf__destroy(skel);
return -err;
}
$ xmake f -p android -a arm64-v8a
$ xmake
[ 11%]: compiling.bpf fentry.bpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/bpf_prog_linfo.c
[ 11%]: ccache compiling.release ../../libbpf/src/netlink.c
[ 11%]: ccache compiling.release ../../libbpf/src/bpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/hashmap.c
[ 11%]: ccache compiling.release ../../libbpf/src/ringbuf.c
[ 11%]: ccache compiling.release ../../libbpf/src/btf_dump.c
[ 11%]: ccache compiling.release ../../libbpf/src/str_error.c
[ 11%]: ccache compiling.release ../../libbpf/src/libbpf_probes.c
[ 11%]: compiling.bpf minimal.bpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/linker.c
[ 11%]: ccache compiling.release ../../libbpf/src/btf.c
[ 11%]: ccache compiling.release ../../libbpf/src/libbpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/nlattr.c
[ 11%]: compiling.bpf uprobe.bpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/xsk.c
[ 11%]: compiling.bpf bootstrap.bpf.c
[ 11%]: ccache compiling.release ../../libbpf/src/strset.c
[ 13%]: ccache compiling.release ../../libbpf/src/libbpf_errno.c
libbpf: elf: skipping unrecognized data section(6) .rodata.str1.1
libbpf: elf: skipping unrecognized data section(7) .rodata.str1.1
[ 18%]: ccache compiling.release ../../libbpf/src/gen_loader.c
libbpf: elf: skipping unrecognized data section(6) .rodata.str1.1
[ 58%]: archiving.release libbpf.a
[ 60%]: ccache compiling.release fentry.c
[ 60%]: ccache compiling.release fentry.bpf.c
[ 60%]: ccache compiling.release bootstrap.bpf.c
[ 60%]: ccache compiling.release bootstrap.c
[ 60%]: ccache compiling.release minimal.bpf.c
[ 60%]: ccache compiling.release minimal.c
[ 60%]: ccache compiling.release uprobe.bpf.c
[ 60%]: ccache compiling.release uprobe.c
error: bootstrap.bpf.c:53:12: error: using builtin_preserve_access_index() without -g
e->ppid = BPF_CORE_READ(task, real_parent, tgid);
^
build/bpf/bpf_core_read.h:404:2: note: expanded from macro 'BPF_CORE_READ'
BPF_CORE_READ_INTO(&__r, (src), a, ##__VA_ARGS__); \
^
build/bpf/bpf_core_read.h:311:30: note: expanded from macro 'BPF_CORE_READ_INTO'
___core_read(bpf_core_read, bpf_core_read, \
^
bootstrap.bpf.c:53:12: error: using builtin_preserve_access_index() without -g
build/bpf/bpf_core_read.h:404:2: note: expanded from macro 'BPF_CORE_READ'
BPF_CORE_READ_INTO(&__r, (src), a, ##__VA_ARGS__); \
^
build/bpf/bpf_core_read.h:311:15: note: expanded from macro 'BPF_CORE_READ_INTO'
___core_read(bpf_core_read, bpf_core_read, \
^
11692 warnings and 2 errors generated.
Would it be possible to provide a C version of the XDP example? I'm struggling to figure out the syntax for attaching a program to a particular network interface.
I can compile and run the example posted in issue #35 without errors after including unistd.h
, but the kernel code never triggers no matter which interface I send traffic to. I would expect to be able to replace http_bpf__attach(skel)
with something like http_bpf__attach_xdp(skel, ifindex)
, but no such function gets generated in the skel.h
file and the standard attach function only accepts a single argument. Or is the interface supposed to be set as an attribute of the skeleton before attaching somehow?
Thanks in advance for any help.
/* Load & verify BPF programs */
err = bootstrap_bpf__load(skel);
if (err) {
fprintf(stderr, "Failed to load and verify BPF skeleton\n");
// No need to cleanup, just return, bootstrap_bpf__destroy may work after bootstrap_bpf__load successfully?
goto cleanup;
}
...
cleanup:
/* Clean up */
/*some code path go here, rb may not be ring_buffer__new yet */
ring_buffer__free(rb);
bootstrap_bpf__destroy(skel);
Currently the executable tools/bpftool
is a x86_64
ELF.
How am I suppose to compile the examples/c
on an aarch64
computer?
Could somebody help me please? Thanks.
I pulled down the repo and init'd the submodules
mike@framework:~/Development/libbpf-bootstrap$ git submodule init
mike@framework:~/Development/libbpf-bootstrap$ git submodule update
Cloning into '/home/mike/Development/libbpf-bootstrap/bpftool'...
Cloning into '/home/mike/Development/libbpf-bootstrap/libbpf'...
Submodule path 'bpftool': checked out 'cdd0b425fce958e3d02424c9d0296e6d0f7e323f'
Submodule path 'libbpf': checked out '9c44c8a8e01cf86bc801c3b72324358d5ea99e50'
mike@framework:~/Development/libbpf-bootstrap$ cd examples/c/
mike@framework:~/Development/libbpf-bootstrap/examples/c$ make minimal
MKDIR .output
MKDIR .output/libbpf
... (omitting a bunch of build output)
INSTALL /home/mike/Development/libbpf-bootstrap/examples/c/.output//libbpf/libbpf.a
BPF .output/minimal.bpf.o
MKDIR bpftool/
BPFTOOL bpftool/bpftool
... libbfd: [ OFF ]
... disassembler-four-args: [ OFF ]
... zlib: [ on ]
... libcap: [ OFF ]
... clang-bpf-co-re: [ on ]
MKDIR /home/mike/Development/libbpf-bootstrap/examples/c/.output/bpftool//libbpf/
make[2]: *** /home/mike/Development/libbpf-bootstrap/bpftool/libbpf/src: No such file or directory. Stop.
make[1]: *** [Makefile:44: /home/mike/Development/libbpf-bootstrap/examples/c/.output/bpftool//libbpf/libbpf.a] Error 2
make: *** [Makefile:65: /home/mike/Development/libbpf-bootstrap/examples/c/.output/bpftool/bpftool] Error 2
bpftool takes libbpf as a submodule but never inits it so there's nothing in that dir. Is there any (easy) way to get it so that bpftool just uses the libbpf from this repo or can we just have it init its own submodule in this makefile?
Hello,
I was trying to prepare libbpf-bootstrap on aarch64 using BTF and CORE. When I cloned the GIT, I realized that the bbpftool bin was built for x86. Hence I rebilt the “bpftool” from the kernel tree (v 5.9.0) on aarch64 base board. When I try to move the bpftool to my final board, it missed the shared library (libbfd-2.30-system.so & libopcodes-2.30-system.so), Which I copied from the base board to make the build through. Also I update the ARCH in the libbpf-bootstrap-master/src/Makefile to ARCH := $(shell uname -m | sed 's/x86_64/x86/aarch64')
to consider aarch64
When I try to run, I get the following errors. I am not sure the steps I follow are right or am I missing something.
Any hint would be of great help.
Thankyou for your support.
Tools used:
Clang version: clang version 10.0.1 Target: aarch64-fsl-linux, Thread model: posix, InstalledDir: /usr/bin
Kernel version: 5.4.69
Pahole version used at kernel build: V1.9
.config of the target: attached
config.txt
cc –version : cc (GCC) 9.2.0
Error Message:
~/libbpf-bootstrap-master/src# ./minimal
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) tp/syscalls/sys_enter_write, size 192, link 0, flags 6, type=1
libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 24 insns (192 bytes)
libbpf: elf: section(3) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(4) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(5) .rodata.str1.1, size 28, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
libbpf: elf: section(6) .BTF, size 490, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 160, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 168, link 13, flags 0, type=2
libbpf: elf: section(9) .reltp/syscalls/sys_enter_write, size 16, link 8, flags 0, type=9
libbpf: looking for externs among 7 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 4, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: sec '.reltp/syscalls/sys_enter_write': collecting relocation for section(2) 'tp/syscalls/sys_enter_write'
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #0: insn #2 against 'my_pid'
libbpf: prog 'handle_tp': found data map 0 (minimal_.bss, sec 4, off 0) for insn 2
libbpf: map 'minimal_.bss': created successfully, fd=4
Segmentation fault
~/libbpf-bootstrap-master/src#
~/libbpf-bootstrap-master/src# ./bootstrap -v
libbpf: loading object 'bootstrap_bpf' from buffer
libbpf: elf: section(2) tp/sched/sched_process_exec, size 504, link 0, flags 6, type=1
libbpf: sec 'tp/sched/sched_process_exec': found program 'handle_exec' at insn offset 0 (0 bytes), code size 63 insns (504 bytes)
libbpf: elf: section(3) tp/sched/sched_process_exit, size 680, link 0, flags 6, type=1
libbpf: sec 'tp/sched/sched_process_exit': found program 'handle_exit' at insn offset 0 (0 bytes), code size 85 insns (680 bytes)
libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
libbpf: license of bootstrap_bpf is Dual BSD/GPL
libbpf: elf: section(5) .rodata, size 8, link 0, flags 2, type=1
libbpf: elf: section(6) .maps, size 48, link 0, flags 3, type=1
libbpf: elf: section(7) .BTF, size 24513, link 0, flags 0, type=1
libbpf: elf: section(8) .BTF.ext, size 1356, link 0, flags 0, type=1
libbpf: elf: section(9) .symtab, size 360, link 15, flags 0, type=2
libbpf: elf: section(10) .reltp/sched/sched_process_exec, size 48, link 9, flags 0, type=9
libbpf: elf: section(11) .reltp/sched/sched_process_exit, size 80, link 9, flags 0, type=9
libbpf: looking for externs among 15 symbols...
libbpf: collected 0 externs total
libbpf: map 'exec_start': at sec_idx 6, offset 0.
libbpf: map 'exec_start': found type = 1.
libbpf: map 'exec_start': found max_entries = 8192.
libbpf: map 'exec_start': found key [9], sz = 4.
libbpf: map 'exec_start': found value [12], sz = 8.
libbpf: map 'rb': at sec_idx 6, offset 32.
libbpf: map 'rb': found type = 27.
libbpf: map 'rb': found max_entries = 262144.
libbpf: map 'bootstra.rodata' (global data): at sec_idx 5, offset 0, flags 480.
libbpf: map 2 is "bootstra.rodata"
libbpf: sec '.reltp/sched/sched_process_exec': collecting relocation for section(2) 'tp/sched/sched_process_exec'
libbpf: sec '.reltp/sched/sched_process_exec': relo #0: insn #10 against 'exec_start'
libbpf: prog 'handle_exec': found map 0 (exec_start, sec 6, off 0) for insn #10
libbpf: sec '.reltp/sched/sched_process_exec': relo #1: insn #14 against 'min_duration_ns'
libbpf: prog 'handle_exec': found data map 2 (bootstra.rodata, sec 5, off 0) for insn 14
libbpf: sec '.reltp/sched/sched_process_exec': relo #2: insn #19 against 'rb'
libbpf: prog 'handle_exec': found map 1 (rb, sec 6, off 32) for insn #19
libbpf: sec '.reltp/sched/sched_process_exit': collecting relocation for section(3) 'tp/sched/sched_process_exit'
libbpf: sec '.reltp/sched/sched_process_exit': relo #0: insn #9 against 'exec_start'
libbpf: prog 'handle_exit': found map 0 (exec_start, sec 6, off 0) for insn #9
libbpf: sec '.reltp/sched/sched_process_exit': relo #1: insn #20 against 'min_duration_ns'
libbpf: prog 'handle_exit': found data map 2 (bootstra.rodata, sec 5, off 0) for insn 20
libbpf: sec '.reltp/sched/sched_process_exit': relo #2: insn #26 against 'exec_start'
libbpf: prog 'handle_exit': found map 0 (exec_start, sec 6, off 0) for insn #26
libbpf: sec '.reltp/sched/sched_process_exit': relo #3: insn #29 against 'min_duration_ns'
libbpf: prog 'handle_exit': found data map 2 (bootstra.rodata, sec 5, off 0) for insn 29
libbpf: sec '.reltp/sched/sched_process_exit': relo #4: insn #35 against 'rb'
libbpf: prog 'handle_exit': found map 1 (rb, sec 6, off 32) for insn #35
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: map 'exec_start': created successfully, fd=4
libbpf: map 'rb': failed to create: Invalid argument(-22)
libbpf: failed to load object 'bootstrap_bpf'
libbpf: failed to load BPF skeleton 'bootstrap_bpf': -22
Failed to load and verify BPF skeleton
~/libbpf-bootstrap-master/src#
~/libbpf-bootstrap-master/src#
While generation the “bpftool”, clang-bpf-co-re: [ OFF ] was there, I am not sure either it could cause the problem
BPFTOOL Log:
~/linux-5.9/tools$ make bpf
DESCEND bpf
Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
CC bpf_jit_disasm.o
LINK bpf_jit_disasm
CC bpf_dbg.o
LINK bpf_dbg
CC bpf_asm.o
BISON bpf_exp.yacc.c
CC bpf_exp.yacc.o
FLEX bpf_exp.lex.c
CC bpf_exp.lex.o
LINK bpf_asm
DESCEND bpftool
Auto-detecting system features:
... libbfd: [ on ]
... disassembler-four-args: [ on ]
... zlib: [ on ]
... libcap: [ OFF ]
... clang-bpf-co-re: [ OFF ]
CC map_perf_ring.o
CC xlated_dumper.o
CC iter.o
CC btf.o
CC tracelog.o
CC link.o
CC perf.o
CC prog.o
CC btf_dumper.o
CC net.o
CC struct_ops.o
CC netlink_dumper.o
CC common.o
CC cgroup.o
CC gen.o
CC main.o
CC json_writer.o
CC cfg.o
CC map.o
CC pids.o
CC feature.o
CC jit_disasm.o
CC disasm.o
Auto-detecting system features:
... libelf: [ on ]
... zlib: [ on ]
... bpf: [ on ]
GEN bpf_helper_defs.h
MKDIR staticobjs/
CC staticobjs/libbpf.o
CC staticobjs/bpf.o
CC staticobjs/nlattr.o
CC staticobjs/btf.o
CC staticobjs/libbpf_errno.o
CC staticobjs/str_error.o
CC staticobjs/netlink.o
CC staticobjs/bpf_prog_linfo.o
CC staticobjs/libbpf_probes.o
CC staticobjs/xsk.o
CC staticobjs/hashmap.o
CC staticobjs/btf_dump.o
CC staticobjs/ringbuf.o
LD staticobjs/libbpf-in.o
LINK libbpf.a
LINK bpftool
DESCEND runqslower
MKDIR .output
hi
I am currently trying to write an xdp program by libbpf-bootstrap. But get an error
libbpf: loading object 'http_bpf' from buffer
libbpf: elf: section(2) xdp, size 120, link 0, flags 6, type=1
libbpf: sec 'xdp': found program 'xdp_pass' at insn offset 0 (0 bytes), code size 15 insns (120 bytes)
libbpf: elf: section(3) .rodata.str1.1, size 16, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(3) .rodata.str1.1
libbpf: elf: section(4) license, size 4, link 0, flags 3, type=1
libbpf: license of http_bpf is GPL
libbpf: elf: section(5) .BTF, size 706, link 0, flags 0, type=1
libbpf: elf: section(6) .BTF.ext, size 220, link 0, flags 0, type=1
libbpf: elf: section(7) .symtab, size 120, link 11, flags 0, type=2
libbpf: looking for externs among 5 symbols...
libbpf: collected 0 externs total
libbpf: failed to find valid kernel BTF
libbpf: Error loading vmlinux BTF: -3
libbpf: failed to load object 'http_bpf'
libbpf: failed to load BPF skeleton 'http_bpf': -3
Failed to open BPF skeleton
my http.bpf.c
// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
/* Copyright (c) 2020 Facebook */
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
SEC("xdp")
int xdp_pass(struct xdp_md *ctx)
{
void *data = (void *)(long)ctx->data;
void *data_end = (void *)(long)ctx->data_end;
int pkt_sz = data_end - data;
bpf_printk("packet size: %d", pkt_sz);
return XDP_PASS;
}
char __license[] SEC("license") = "GPL";
my http.c
#include "http.skel.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <getopt.h>
#include <bpf/bpf.h>
#include <net/if.h>
#include <linux/if_link.h> /* depend on kernel-headers installed */
static int libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
{
return vfprintf(stderr, format, args);
}
int main(int argc, char **argv){
struct http_bpf *skel;
int err;
libbpf_set_print(libbpf_print_fn);
/* Open load and verify BPF application */
skel = http_bpf__open_and_load();
if (!skel) {
fprintf(stderr, "Failed to open BPF skeleton\n");
return 1;
}
/* Attach xdp handler */
err = http_bpf__attach(skel);
if (err) {
fprintf(stderr, "Failed to attach BPF skeleton\n");
goto cleanup;
}
while (1) {
fprintf(stderr, ".");
sleep(1);
}
cleanup:
http_bpf__destroy(skel);
return -err;
}
I think that's error in SEC(xdp)
thanks
Hello:
My system uses the ARM architecture, but the bpftool file is x86-64. As a result, the following error message is displayed when the BPF program is compiled:
/bin/sh: /root/tools/libbpf-bootstrap/tools/bpftool: cannot execute binary file: Exec format error
What am I supposed to do with it?
On Ubuntu 20.10 this happens to me:
$ V=1 make minimal
clang -g -O2 -target bpf -D__TARGET_ARCH_x86 -I.output -c minimal.bpf.c -o .output/minimal.bpf.o
In file included from minimal.bpf.c:3:
In file included from /usr/include/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
#include <asm/types.h>
The rub seems to be that /usr/include/linux/types.h
wants asm/types.h
which, on my system, I believe lives in /usr/include/x86_64-linux-gnu/asm/types.h
(which file just reads #include <asm-generic/types.h>
), and /usr/include/x86_64-linux-gnu
does not seem to be among clang's include paths, as demonstrated by the clang -v
output below.
clang -v -g -O2 -target bpf -D__TARGET_ARCH_x86 -I.output -c minimal.bpf.c -o .output/minimal.bpf.o
Ubuntu clang version 11.0.0-2
...
#include "..." search starts here:
#include <...> search starts here:
.output
/usr/local/include`
/usr/lib/llvm-11/lib/clang/11.0.0/include
/usr/include
End of search list.
FWIW, on the gcc side, the preprocessor includes /usr/include/x86_64-linux-gnu
in the cpp -v
output.
I'm not sure who's at fault here, or what the correct fix is, but adding -I/usr/include/x86_64-linux-gnu
makes it work. Changing the source to use vmlinux.h
instead of <linux/bpf.h>
also makes it work.
Although I don't believe it to be the case, it is possible that my system got screwed up by me manually installing kernel headers through the Debian package created by the kernel's deb-pkg
make target. I don't think it's the case because I've verified that /usr/include/linux/types.h
corresponds to the file provided by the Ubuntu's linux-libc-dev
package.
I created a fork and new feature branch at https://github.com/chrispsommers/libbpf-bootstrap/tree/kprobe_netlink-example#kprobe_netlink. I added example kprobe_netlink
to trace netlink messages. The tracing output only shows the kretprobe
messages for some reason. Any advice? You can clone this and build using the normal recipes if you care to try it. See below, thanks:
sudo ./kprobe_netlink
libbpf: loading object 'kprobe_netlink_bpf' from buffer
libbpf: elf: section(2) kprobe/netlink_unicast, size 240, link 0, flags 6, type=1
libbpf: sec 'kprobe/netlink_unicast': found program 'netlink_unicast' at insn offset 0 (0 bytes), code size 30 insns (240 bytes)
libbpf: elf: section(3) kretprobe/netlink_unicast, size 280, link 0, flags 6, type=1
libbpf: sec 'kretprobe/netlink_unicast': found program 'netlink_unicast_exit' at insn offset 0 (0 bytes), code size 35 insns (280 bytes)
libbpf: elf: section(4) kprobe/netlink_broadcast, size 232, link 0, flags 6, type=1
libbpf: sec 'kprobe/netlink_broadcast': found program 'netlink_broadcast' at insn offset 0 (0 bytes), code size 29 insns (232 bytes)
libbpf: elf: section(5) kretprobe/netlink_broadcast, size 248, link 0, flags 6, type=1
libbpf: sec 'kretprobe/netlink_broadcast': found program 'netlink_broadcast_exit' at insn offset 0 (0 bytes), code size 31 insns (248 bytes)
libbpf: elf: section(6) license, size 13, link 0, flags 3, type=1
libbpf: license of kprobe_netlink_bpf is Dual BSD/GPL
libbpf: elf: section(7) .rodata.str1.1, size 201, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(7) .rodata.str1.1
libbpf: elf: section(8) .BTF, size 1641, link 0, flags 0, type=1
libbpf: elf: section(9) .BTF.ext, size 684, link 0, flags 0, type=1
libbpf: elf: section(10) .symtab, size 240, link 14, flags 0, type=2
libbpf: looking for externs among 10 symbols...
libbpf: collected 0 externs total
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: sec 'kprobe/netlink_unicast': found 1 CO-RE relocations
libbpf: prog 'netlink_unicast': relo #0: kind <byte_off> (0), spec is [2] struct pt_regs.dx (0:12 @ offset 96)
libbpf: CO-RE relocating [0] struct pt_regs: found target candidate [221] struct pt_regs in [vmlinux]
libbpf: prog 'netlink_unicast': relo #0: matching candidate #0 [221] struct pt_regs.dx (0:12 @ offset 96)
libbpf: prog 'netlink_unicast': relo #0: patched insn #0 (LDX/ST/STX) off 96 -> 96
libbpf: sec 'kretprobe/netlink_unicast': found 1 CO-RE relocations
libbpf: prog 'netlink_unicast_exit': relo #0: kind <byte_off> (0), spec is [2] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'netlink_unicast_exit': relo #0: matching candidate #0 [221] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'netlink_unicast_exit': relo #0: patched insn #0 (LDX/ST/STX) off 80 -> 80
libbpf: sec 'kprobe/netlink_broadcast': found 1 CO-RE relocations
libbpf: prog 'netlink_broadcast': relo #0: kind <byte_off> (0), spec is [2] struct pt_regs.dx (0:12 @ offset 96)
libbpf: prog 'netlink_broadcast': relo #0: matching candidate #0 [221] struct pt_regs.dx (0:12 @ offset 96)
libbpf: prog 'netlink_broadcast': relo #0: patched insn #0 (LDX/ST/STX) off 96 -> 96
libbpf: sec 'kretprobe/netlink_broadcast': found 1 CO-RE relocations
libbpf: prog 'netlink_broadcast_exit': relo #0: kind <byte_off> (0), spec is [2] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'netlink_broadcast_exit': relo #0: matching candidate #0 [221] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'netlink_broadcast_exit': relo #0: patched insn #0 (LDX/ST/STX) off 80 -> 80
Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` to see output of the BPF programs.
........
Trace output:
$ sudo cat /sys/kernel/debug/tracing/trace_pipe
sudo-31063 [010] d... 90928.229245: bpf_trace_printk: KPROBE netlink_unicast_exit EXIT: pid = 31063, ret = 36
sudo-31063 [010] d... 90928.229252: bpf_trace_printk: KPROBE netlink_unicast_exit EXIT: pid = 31063, ret = 176
sudo-31063 [010] d... 90928.229352: bpf_trace_printk: KPROBE netlink_unicast_exit EXIT: pid = 31063, ret = 36
sudo-31063 [010] d... 90928.229354: bpf_trace_printk: KPROBE netlink_unicast_exit EXIT: pid = 31063, ret = 136
In the vmlinux.h on line 950 appears:
struct hlist_node pid_links[4];
shouldn't it be
struct pid_link pid_links[4];
instead?
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) tp/syscalls/sys_enter_write, size 24, link 0, flags 6, type=1
libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 3 insns (24 bytes)
libbpf: elf: section(3) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(4) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(5) .BTF, size 446, link 0, flags 0, type=1
libbpf: elf: section(6) .BTF.ext, size 96, link 0, flags 0, type=1
libbpf: elf: section(7) .symtab, size 144, link 11, flags 0, type=2
libbpf: looking for externs among 6 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 4, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: Kernel doesn't support BTF, skipping uploading it.
libbpf: kernel doesn't support global data
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -95
Failed to load and verify BPF skeleton
Linux VM-0-13-ubuntu 4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
Hello!
I was wondering why you build & generate the skel manually (using cargo libbpf <...>
) rather than using cargo build
like you did at the end.
These commands generate another .skel.rs
and .mod
files plus keeping the .bpf.o
file which cargo build
discards automatically.
$ cd examples/rust
$ cargo libbpf build
$ cargo libbpf gen
$ cargo build --release
$ sudo ./target/release/xdp 1
<...>
Thanks!
I was running git clone https://github.com/libbpf/libbpf-bootstrap.git
but it looks like it missed libbpf
for obvious reason. I guess documentation should mention that. That would be helpful.
When trying to run make minimal
in the examples/c
directory, getting the following errors - the full output in gist
https://gist.github.com/dmitris/697849eb3c0b80e6f2ca50430cae499b:
skeleton/pid_iter.bpf.c:47:14: error: incomplete definition of type 'struct bpf_perf_link'
[...]
skeleton/pid_iter.bpf.c:44:9: note: forward declaration of 'struct bpf_perf_link'
struct bpf_perf_link *perf_link;
^
skeleton/pid_iter.bpf.c:48:10: error: incomplete definition of type 'struct bpf_perf_link'
event = BPF_CORE_READ(perf_link, perf_file, private_data);
[...]
skeleton/pid_iter.bpf.c:49:30: error: no member named 'bpf_cookie' in 'struct perf_event'
return BPF_CORE_READ(event, bpf_cookie);
Is it related to (or even fixed in) https://lore.kernel.org/bpf/[email protected]/ and
https://lore.kernel.org/bpf/[email protected]/ ?
I got that error with the current rhel8 and Fedora 35 - Linux mybox.example.com 5.14.10-300.fc35.x86_64 #1 SMP Thu Oct 7 20:48:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
After compile and build the minimal
example application, I try to run it but with the following error message:
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) tp/syscalls/sys_enter_write, size 104, link 0, flags 6, type=1
libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 13 insns (104 bytes)
libbpf: elf: section(3) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(4) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(5) .rodata, size 28, link 0, flags 2, type=1
libbpf: elf: section(6) .BTF, size 617, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 160, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 216, link 13, flags 0, type=2
libbpf: elf: section(9) .reltp/syscalls/sys_enter_write, size 32, link 8, flags 0, type=9
libbpf: looking for externs among 9 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 4, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: map 'minimal_.rodata' (global data): at sec_idx 5, offset 0, flags 480.
libbpf: map 1 is "minimal_.rodata"
libbpf: sec '.reltp/syscalls/sys_enter_write': collecting relocation for section(2) 'tp/syscalls/sys_enter_write'
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #0: insn #2 against 'my_pid'
libbpf: prog 'handle_tp': found data map 0 (minimal_.bss, sec 4, off 0) for insn 2
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #1: insn #6 against '.rodata'
libbpf: prog 'handle_tp': found data map 1 (minimal_.rodata, sec 5, off 0) for insn 6
libbpf: Kernel doesn't support BTF, skipping uploading it.
libbpf: prog 'handle_tp': relo #0: kernel doesn't support global data
libbpf: prog 'handle_tp': failed to relocate data references: -95
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -95
Failed to load and verify BPF skeleton
My kernel version is: 4.15.0-96-generic. I run the example on Ubuntu-18.04 system.
For Ubuntu 20.04 I had to install libelf-dev and clang to build examples.
For Ubuntu 16.04 I had to install libz in addition to above.
Greeting,
Thanks for the hard work and great examples.
Since there are more and more SEC
types now supported by the kernel:
https://github.com/libbpf/libbpf/blob/4eb6485c08867edaa5a0a81c64ddb23580420340/src/libbpf.c#L9002-L9081
I think it would be great if we could cover more SEC
types.
I wonder if I can add an example about SEC(socket)
, which can get an insight about how to retrieve data from __sk_buff
, maybe start from most used UDP
and TCP
on top of IP
protocol.
Do you think it's a good idea?
I try build it on my ubuntu x86_64, but it fails.
libbpf-bootstrap/src$ make
MKDIR .output
MKDIR .output/libbpf
LIB libbpf.a
MKDIR staticobjs
CC bpf.o
CC btf.o
CC libbpf.o
CC libbpf_errno.o
CC netlink.o
CC nlattr.o
CC str_error.o
CC libbpf_probes.o
CC bpf_prog_linfo.o
CC xsk.o
CC btf_dump.o
CC hashmap.o
CC ringbuf.o
AR libbpf.a
INSTALL bpf.h libbpf.h btf.h xsk.h libbpf_util.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h libbpf_common.h
INSTALL libbpf.pc
INSTALL libbpf.a
BPF .output/minimal.bpf.o
In file included from minimal.bpf.c:4:
.output/bpf/bpf_helpers.h:99:10: error: unknown register name 'r0' in asm
: "r0", "r1", "r2", "r3", "r4", "r5");
^
1 error generated.
Makefile:59: recipe for target '.output/minimal.bpf.o' failed
make: *** [.output/minimal.bpf.o] Error 1
r0? Why does it generate header files for arm architecture?
Hi there the c bootstrap example does not run. I'm running it on ubuntu 20.04
Error is
teroz@ubuntulsm:~/libbpf-bootstrap/examples/c$ sudo ./bootstrap
[sudo] password for teroz:
libbpf: map 'rb': failed to create: Invalid argument(-22)
libbpf: failed to load object 'bootstrap_bpf'
libbpf: failed to load BPF skeleton 'bootstrap_bpf': -22
Failed to load and verify BPF skeleton
Hello! While playing with libbpf-bootstrap I'm getting unexpected (and strange) function argument for kprobe syscalls. For example for kprobe on close
syscall with int close(inf fd)
signature, I got enormous fd
values like fd=15761240
while expected small int like fd=4
. Reproduced this on Debian 11 x64 (kernel 5.10.0-7-amd64)
and Ubuntu 21.10 x64 (kernel ~5.13)
.
Debug code:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>
char LICENSE[] SEC("license") = "Dual BSD/GPL";
// close accept4 syscall
// int accept4(int sockfd, struct sockaddr *restrict addr, socklen_t *restrict addrlen);
SEC("kretprobe/__x64_sys_accept4")
int BPF_KRETPROBE(accept, int ret) {
u64 id = bpf_get_current_pid_tgid();
u32 pid = id >> 32;
// filter specific pid for simplicity
if (pid != 31114 || ret < 0) {
return 0;
}
// debug returned file descriptor
bpf_printk("opened pid=%d fd=%d", pid, ret);
return 0;
}
// close syscall
// int close(int fd);
SEC("kprobe/__x64_sys_close")
int BPF_KPROBE(close, int fd) {
u64 id = bpf_get_current_pid_tgid();
u32 pid = id >> 32;
// filter specific pid for simplicity
if (pid != 31114) {
return 0;
}
// debug fd arg (expected to be equal to fd returned on accept4)
bpf_printk("closed pid=%d fd=%d", pid, fd);
return 0;
}
Results
$ cat /sys/kernel/debug/tracing/trace_pipe
main-31114 [001] d... 9069.254408: bpf_trace_printk: opened pid=31114 fd=4
main-31114 [001] d... 9069.321946: bpf_trace_printk: closed pid=31114 fd=15761240
I tried to alter vmlinux.h
: at first with vmlinux.h
delivered by libbbpf-bootstrap and then with "native" vmlinux.h
from the instance OS kernel and on both ways I got the issue above.
Also tried to run the same bpf program in BCC way (compiled with bcc at run-time) with kprobes declared without BPF_KPROBE macro, like that:
int syscall__probe_close_entry(struct pt_regs *ctx, int fd) { ... }
and it worked as expected: fd=4
at all the debug points.
Is it a BPF_KPROBE macro bug/incompatibility with the kernel or I'm missing something?
hi,
Recently, I was trying to implement LSM using BPF. but my program has no output.
my ebpf.c
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <errno.h>
char _license[] SEC("license") = "GPL";
SEC("lsm/file_open")
int BPF_PROG(file_open_handler, struct file *file, int ret)
{
bpf_printk("Hello, world\n");
return ret;
}
loader.c
#include <stdio.h>
#include <linux/bpf.h>
#include <bpf/libbpf.h>
#include <bpf/bpf.h>
#include <assert.h>
#include <unistd.h>
#include "lsm_kern.h"
static int libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
{
return vfprintf(stderr, format, args);
}
int main(int argc,char *argv)
{
libbpf_set_print(libbpf_print_fn);
struct lsm_kern *skel = NULL;
int err = 0;
skel = lsm_kern__open_and_load();
if(!skel){
printf("load error!\n");
}
err = lsm_kern__attach(skel);
if(err)
printf("attach error!\n");
while(1){
sleep(100);
}
return 0;
}
Makefile(make output)
clang -g -O2 -target bpf -D__TARGET_ARCH_x86 -I../libbpf/include/uapi -idirafter /usr/lib/llvm-14/lib/clang/14.0.0/include -idirafter /usr/local/include -idirafter /usr/include/x86_64-linux-gnu -idirafter /usr/include -c lsm_kern.c -o lsm_kern.o
bpftool gen skeleton lsm_kern.o > lsm_kern.h
gcc -o loader loader.c /usr/lib64/libbpf.a /usr/lib/x86_64-linux-gnu/libz.a /usr/lib/x86_64-linux-gnu/libelf.a
load and attach out
libbpf: loading object 'lsm_kern' from buffer
libbpf: elf: section(3) lsm/file_open, size 136, link 0, flags 6, type=1
libbpf: sec 'lsm/file_open': found program 'file_open_handler' at insn offset 0 (0 bytes), code size 17 insns (136 bytes)
libbpf: elf: section(4) .rellsm/file_open, size 32, link 26, flags 40, type=9
libbpf: elf: section(5) license, size 4, link 0, flags 3, type=1
libbpf: license of lsm_kern is GPL
libbpf: elf: section(6) .rodata, size 48, link 0, flags 2, type=1
libbpf: elf: section(16) .BTF, size 727, link 0, flags 0, type=1
libbpf: elf: section(18) .BTF.ext, size 144, link 0, flags 0, type=1
libbpf: elf: section(26) .symtab, size 432, link 1, flags 0, type=2
libbpf: looking for externs among 18 symbols...
libbpf: collected 0 externs total
libbpf: map 'lsm_kern.rodata' (global data): at sec_idx 6, offset 0, flags 480.
libbpf: map 0 is "lsm_kern.rodata"
libbpf: sec '.rellsm/file_open': collecting relocation for section(3) 'lsm/file_open'
libbpf: sec '.rellsm/file_open': relo #0: insn #5 against '.rodata'
libbpf: prog 'file_open_handler': found data map 0 (lsm_kern.rodata, sec 6, off 0) for insn 5
libbpf: sec '.rellsm/file_open': relo #1: insn #11 against '.rodata'
libbpf: prog 'file_open_handler': found data map 0 (lsm_kern.rodata, sec 6, off 0) for insn 11
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: map 'lsm_kern.rodata': created successfully, fd=4
but cat /sys/kernel/debug/tracing/trace_pipe
not output
strace -o log ./loader
bpf(BPF_BTF_LOAD, {btf="\237\353\1\0\30\0\0\0\0\0\0\08\0\0\08\0\0\0\t\0\0\0\0\0\0\0\0\0\0\1"..., btf_log_buf=NULL, btf_size=89, btf_log_size=0, btf_log_level=0}, 28) = 3
...
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_LSM, insn_cnt=17, insns=0x56299e5aa620, license="GPL", log_level=0, log_size=0, log_buf=NULL, kern_version=KERNEL_VERSION(5, 15, 30), prog_flags=0, prog_name="file_open_handl", prog_ifindex=0, expected_attach_type=BPF_LSM_MAC, prog_btf_fd=3, func_info_rec_size=8, func_info=0x56299e5a9000, func_info_cnt=1, line_info_rec_size=16, line_info=0x56299e5a8b70, line_info_cnt=5, attach_btf_id=32663, attach_prog_fd=0, fd_array=NULL}, 144) = 5
...
bpf(BPF_RAW_TRACEPOINT_OPEN, {raw_tracepoint={name=NULL, prog_fd=5}}, 144) = 6
libbpf-bootstrap is a great project that allows beginners to quickly get started with bpf program development.
Now I used xmake to port the libbpf-bootstrap program and compiled it to the android platform.
xmake.lua
add_rules("mode.release", "mode.debug")
add_rules("platform.linux.bpf")
add_requires("linux-tools", {configs = {bpftool = true}})
add_requires("libbpf")
if is_plat("android") then
add_requires("ndk >=22.x")
set_toolchains("@ndk", {sdkver = "23"})
else
add_requires("llvm >=10.x")
set_toolchains("@llvm")
add_requires("linux-headers")
end
target("minimal")
set_kind("binary")
add_files("src/*.c")
add_packages("linux-tools", "linux-headers", "libbpf")
set_license("GPL-2.0")
In addition, I ported the libbpf library and other dependent libraries to the android platform. All dependent packages and tool chains can be automatically downloaded and integrated by xmake.
We only need to execute xmake
to compile linux bpf program, or use xmake f -p android; xmake
to compile android bpf program.
If the user has installed the new version of llvm/clang and ndk r22, then xmake will also give priority to using them and will not download them automatically.
It will download llvm toolchain and use it automatically if not found.
we can also install llvm manually. sudo apt install llvm
and xmake will also use it.
$ xmake
It will download ndk toolchain and use it automatically
$ xmake f -p android
$ xmake
We can also set ndk toolchain manually (need ndk >= r22)
$ xmake f -p android --ndk=~/file/android-ndk-r22
$ xmake
Here is a libbpf-bootstrap project that I use xmake to ported. The project directory is very clean. We only need to keep the src source file and the xmake.lua project configuration file.
https://github.com/hack0z/libbpf-bootstrap
And I added libbpf package into our official repository, https://github.com/xmake-io/xmake-repo/blob/master/packages/l/libbpf/xmake.lua
If you are interested, I can open a pr and just add xmake.lua to your project, but I will not delete any other files, otherwise please ignore this issue.
Related issue: xmake-io/xmake#1274
Thanks!
I wrote an Example for helloworld,But pr cannot submit
Creating this issue to decide directory structure and code layout.
Current structure:
.
├── libbpf
├── LICENSE
├── README.md
├── src
│ ├── bootstrap.bpf.c
│ ├── bootstrap.c
│ ├── bootstrap.h
│ ├── fentry.bpf.c
│ ├── fentry.c
│ ├── kprobe.bpf.c
│ ├── kprobe.c
│ ├── Makefile
│ ├── minimal.bpf.c
│ ├── minimal.c
│ ├── uprobe.bpf.c
│ ├── uprobe.c
│ ├── vmlinux_508.h
│ └── vmlinux.h -> vmlinux_508.h
└── tools
├── bpftool
└── gen_vmlinux_h.sh
Proposed structure:
.
├── examples
│ ├── c
│ │ ├── bootstrap
│ │ │ ├── bootstrap.bpf.c
│ │ │ ├── bootstrap.c
│ │ │ ├── bootstrap.h
│ │ │ └── Makefile
│ │ ├── fentry
│ │ │ ├── fentry.bpf.c
│ │ │ ├── fentry.c
│ │ │ └── Makefile
│ │ ├── kprobe
│ │ │ ├── kprobe.bpf.c
│ │ │ ├── kprobe.c
│ │ │ └── Makefile
│ │ ├── Makefile
│ │ ├── minimal
│ │ │ ├── Makefile
│ │ │ ├── minimal.bpf.c
│ │ │ └── minimal.c
│ │ └── uprobe
│ │ ├── Makefile
│ │ ├── uprobe.bpf.c
│ │ └── uprobe.c
│ └── rust
│ ├── Cargo.toml
│ ├── Makefile
│ └── xdp
│ ├── Cargo.toml
│ ├── Makefile
│ └── src
│ ├── bpf
│ │ └── xdp.bpf.c
│ └── main.rs
├── libbpf
├── LICENSE
├── README.md
├── tools
│ ├── bpftool
│ └── gen_vmlinux_h.sh
└── vmlinux
├── Makefile
├── vmlinux_508.h
└── vmlinux.h -> vmlinux_508.h
Basic idea is top level entries (tools/
, vmlinux/
, etc.) are shared between all examples. Anything language / platform specific must be completely contained to the examples/*
directory. Each language/platform has a makefile that does the right thing.
Not sure if each individual example should have its own makefile. Maybe it'd be nice to be able to cd
into any example and just type make
.
Latest (c126589) libbpf-bootstrap gives errors when loading on Debian testing x86_64 with kernel 5.10.17. Actually, already the compilation step is a bit fishy, as it says:
libbpf-bootstrap/src(master)$ make
MKDIR .output
MKDIR .output/libbpf
LIB libbpf.a
MKDIR staticobjs
CC bpf.o
CC btf.o
CC libbpf.o
CC libbpf_errno.o
CC netlink.o
CC nlattr.o
CC str_error.o
CC libbpf_probes.o
CC bpf_prog_linfo.o
CC xsk.o
CC btf_dump.o
CC hashmap.o
CC ringbuf.o
AR libbpf.a
INSTALL bpf.h libbpf.h btf.h xsk.h libbpf_util.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h libbpf_common.h
INSTALL libbpf.pc
INSTALL libbpf.a
BPF .output/minimal.bpf.o
GEN-SKEL .output/minimal.skel.h
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
CC .output/minimal.o
BINARY minimal
BPF .output/bootstrap.bpf.o
GEN-SKEL .output/bootstrap.skel.h
CC .output/bootstrap.o
BINARY bootstrap
BPF .output/uprobe.bpf.o
GEN-SKEL .output/uprobe.skel.h
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
CC .output/uprobe.o
BINARY uprobe
BPF .output/kprobe.bpf.o
GEN-SKEL .output/kprobe.skel.h
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
CC .output/kprobe.o
BINARY kprobe
BPF .output/fentry.bpf.o
GEN-SKEL .output/fentry.skel.h
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
CC .output/fentry.o
BINARY fentry
In all three cases of GEN-SKEL, the output happens in response to
...
llvm-strip -g .output/minimal.bpf.o # strip useless DWARF info
libbpf-bootstrap/tools/bpftool gen skeleton .output/minimal.bpf.o > .output/minimal.skel.h
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
...
But then running 'minimal' fails with
libbpf-bootstrap/src(master)# ./minimal
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) tp/syscalls/sys_enter_write, size 192, link 0, flags 6, type=1
libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 24 insns (192 bytes)
libbpf: elf: section(3) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(4) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(5) .rodata.str1.1, size 28, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
libbpf: elf: section(6) .BTF, size 494, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 160, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 144, link 13, flags 0, type=2
libbpf: elf: section(9) .reltp/syscalls/sys_enter_write, size 16, link 8, flags 0, type=9
libbpf: looking for externs among 6 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 4, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: sec '.reltp/syscalls/sys_enter_write': collecting relocation for section(2) 'tp/syscalls/sys_enter_write'
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #0: insn #2 against 'my_pid'
libbpf: prog 'handle_tp': found data map 0 (minimal_.bss, sec 4, off 0) for insn 2
libbpf: map 'minimal_.bss': created successfully, fd=4
libbpf: load bpf program failed: Invalid argument
libbpf: failed to load program 'handle_tp'
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -22
Failed to load and verify BPF skeleton
The other programs also fail at the exact same point when loading eBPF programs and with the same error. Might this be an issue with the kernel (v5.10.17, compile by me), clang/llvm (Debian testing versions 11.0.1-2 and 11.0.1), Debian (testing), libbpf-bootstrap (c126589) or something else?
I compiled the program but when I try to attach it to an interface with sudo ./target/release/xdp 1
I get this message
libbpf: elf: skipping unrecognized data section(4) .rodata.str1.1
...........
I know this is a warning message but how to fix it?
I would like to know how bpftool
and vmlinux.h
is generated on this repository to it make cross compatible between different kernel versions that have /sys/kernel/btf/vmlinux
exposed.
I can generate vmlinux.h
for kernel 5.15 as follows,
$ uname -r
5.15.0-22-generic
$ git clone --depth 1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git -b v5.15 linux_v5.15
$ cd linux_v5.15/tools/bpf/bpftool/
$ make
$ ./bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
Then I can use vmlinux.h
to compile libbpf programs.
However, if I download kernel source with tag v5.8
I am not able to generate vmlinux.h
on the same machine,
$ git clone --depth 1 git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git -b v5.8 linux_v5.8
$ cd cd linux_v5.8/tools/bpf/bpftool
$ make
$ ./bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h
Error: failed to load BTF from /sys/kernel/btf/vmlinux: Invalid argument
Does it mean that bpftool
must be of the same version (at least major and minor version matching) when generating vmlinux.h
? So, this would explain why bpftool
and vmlinux.h
are included in the version control because, for example, if these files are generated on the fly in a container with mismatched host kernel and bpftool versions, the build can fail?
If yes, how do you go about picking which kernel version to use to compile libbpf program?
My system is:
$ uname -a
Linux localhost.localdomain 5.11.12-300.fc34.x86_64 #1 SMP Wed Apr 7 16:31:13 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
The minimal example:
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
char LICENSE[] SEC("license") = "Dual BSD/GPL";
SEC("tracepoint/syscalls/sys_enter_execve")
int execve(void *ctx)
{
u32 value = 0;
bpf_probe_read(&value, sizeof(value), ctx);
return 0;
}
SEC("tracepoint/kmem/mm_page_alloc_zone_locked")
int page_alloc(void *ctx)
{
u32 value = 0;
bpf_probe_read(&value, sizeof(value), ctx);
return 0;
}
It is a *.bpf.c
file. In userspace do open
, load
and attach
as usual. And entire system is freeze.
Obviously it is a kernel bug.
Hi team,
I failed to run kprobe example n arm64 as below,
root@tegra-ubuntu:/ota# ./kprobe -v
libbpf: loading object 'kprobe_bpf' from buffer
libbpf: elf: section(2) kprobe/do_unlinkat, size 152, link 0, flags 6, type=1
libbpf: sec 'kprobe/do_unlinkat': found program 'do_unlinkat' at insn offset 0 (0 bytes), code size 19 insns (152 bytes)
libbpf: elf: section(3) kretprobe/do_unlinkat, size 88, link 0, flags 6, type=1
libbpf: sec 'kretprobe/do_unlinkat': found program 'do_unlinkat_exit' at insn offset 0 (0 bytes), code size 11 insns (88 bytes)
libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
libbpf: license of kprobe_bpf is Dual BSD/GPL
libbpf: elf: section(5) .rodata, size 72, link 0, flags 2, type=1
libbpf: elf: section(6) .BTF, size 1467, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 364, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 240, link 14, flags 0, type=2
libbpf: elf: section(9) .relkprobe/do_unlinkat, size 16, link 8, flags 0, type=9
libbpf: elf: section(10) .relkretprobe/do_unlinkat, size 16, link 8, flags 0, type=9
libbpf: looking for externs among 10 symbols...
libbpf: collected 0 externs total
libbpf: map 'kprobe_b.rodata' (global data): at sec_idx 5, offset 0, flags 480.
libbpf: map 0 is "kprobe_b.rodata"
libbpf: sec '.relkprobe/do_unlinkat': collecting relocation for section(2) 'kprobe/do_unlinkat'
libbpf: sec '.relkprobe/do_unlinkat': relo #0: insn #12 against '.rodata'
libbpf: prog 'do_unlinkat': found data map 0 (kprobe_b.rodata, sec 5, off 0) for insn 12
libbpf: sec '.relkretprobe/do_unlinkat': collecting relocation for section(3) 'kretprobe/do_unlinkat'
libbpf: sec '.relkretprobe/do_unlinkat': relo #0: insn #3 against '.rodata'
libbpf: prog 'do_unlinkat_exit': found data map 0 (kprobe_b.rodata, sec 5, off 0) for insn 3
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: map 'kprobe_b.rodata': created successfully, fd=4
libbpf: sec 'kprobe/do_unlinkat': found 2 CO-RE relocations
libbpf: CO-RE relocating [2] struct pt_regs: found target candidate [248] struct pt_regs in [vmlinux]
libbpf: prog 'do_unlinkat': relo #0: <byte_off> [2] struct pt_regs.si (0:13 @ offset 104)
libbpf: prog 'do_unlinkat': relo #0: non-matching candidate #0 <byte_off> [248] struct pt_regs (0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #0: no matching targets found
libbpf: prog 'do_unlinkat': relo #0: substituting insn #0 w/ invalid insn
libbpf: CO-RE relocating [7] struct filename: found target candidate [1173] struct filename in [vmlinux]
libbpf: prog 'do_unlinkat': relo #1: <byte_off> [7] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #1: matching candidate #0 <byte_off> [1173] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #1: patched insn #3 (ALU/ALU64) imm 0 -> 0
libbpf: sec 'kretprobe/do_unlinkat': found 1 CO-RE relocations
libbpf: prog 'do_unlinkat_exit': relo #0: <byte_off> [2] struct pt_regs.ax (0:10 @ offset 80)
libbpf: prog 'do_unlinkat_exit': relo #0: non-matching candidate #0 <byte_off> [248] struct pt_regs (0 @ offset 0)
libbpf: prog 'do_unlinkat_exit': relo #0: no matching targets found
libbpf: prog 'do_unlinkat_exit': relo #0: substituting insn #0 w/ invalid insn
libbpf: prog 'do_unlinkat': BPF program load failed: Invalid argument
libbpf: prog 'do_unlinkat': -- BEGIN PROG LOAD LOG --
Unrecognized arg#0 type PTR
; int BPF_KPROBE(do_unlinkat, int dfd, struct filename *name)
0: <invalid CO-RE relocation>
failed to resolve CO-RE relocation <byte_off> [2] struct pt_regs.si (0:13 @ offset 104)
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: failed to load program 'do_unlinkat'
libbpf: failed to load object 'kprobe_bpf'
libbpf: failed to load BPF skeleton 'kprobe_bpf': -22
Failed to open BPF skeleton
Cound someone point out what's wrong here? Thanks.
Hello!
I have a docker container running Ubuntu 22.04 LTS, kernel version is 5.10.47.
I am able to build the minimal example from the cloned repo perfectly: make minimal
.
But when I try to run the binary ./minimal
, I get the following error:
libbpf: Failed to bump RLIMIT_MEMLOCK (err = -1), you might need to do it explicitly!
libbpf: Error in bpf_object__probe_loading():Operation not permitted(1). Couldn't load trivial BPF program. Make sure your kernel supports BPF (CONFIG_BPF_SYSCALL=y) and/or that RLIMIT_MEMLOCK is set to big enough value.
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -1
Failed to load and verify BPF skeleton
I can see the kernel is compiled with the flags correctly by checking the boot file:
cat /boot/config-5.15.0-30-generic
:
CONFIG_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_ARCH_WANT_DEFAULT_BPF_JIT=y
#
# BPF subsystem
#
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_BPF_JIT_DEFAULT_ON=y
CONFIG_BPF_UNPRIV_DEFAULT_OFF=y
CONFIG_USERMODE_DRIVER=y
I can also see that my RLIMIT_MEMLOCK is kinda big enough:
cat /etc/security/limits.conf
#* soft core 1000000
#root hard core 1000000
#* hard rss 1000000
# End of file
Am I missing something here?
Appreciate any help!
hi
I had some problems compiling minimal. I hope to get help
[root@localhost src]# make clean minimal
CLEAN
MKDIR .output
MKDIR .output/libbpf
LIB libbpf.a
MKDIR staticobjs
CC bpf.o
CC btf.o
CC libbpf.o
CC libbpf_errno.o
CC netlink.o
CC nlattr.o
CC str_error.o
CC libbpf_probes.o
CC bpf_prog_linfo.o
CC xsk.o
CC btf_dump.o
CC hashmap.o
CC ringbuf.o
AR libbpf.a
INSTALL bpf.h libbpf.h btf.h xsk.h libbpf_util.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h libbpf_common.h
INSTALL libbpf.pc
INSTALL libbpf.a
BPF .output/minimal.bpf.o
GEN-SKEL .output/minimal.skel.h
libbpf: sec 'tp/syscalls/sys_enter_write': failed to find program symbol at offset 0
Error: failed to open BPF object file: BPF object format invalid
make: *** [.output/minimal.skel.h] 错误 255
make: *** 正在删除文件“.output/minimal.skel.h”
env
[root@localhost src]# uname -a
Linux localhost.localdomain 5.4.108-1.el7.elrepo.x86_64 #1 SMP Mon Mar 22 18:37:08 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost src]# rpm -qa | grep elf
elfutils-libs-0.176-5.el7.x86_64
devtoolset-7-elfutils-libs-0.170-5.el7.x86_64
elfutils-libelf-0.176-5.el7.x86_64
elfutils-libelf-devel-0.176-5.el7.x86_64
elfutils-default-yama-scope-0.172-2.el7.noarch
devtoolset-7-elfutils-libelf-0.170-5.el7.x86_64
devtoolset-7-elfutils-0.170-5.el7.x86_64
[root@localhost src]# rpm -qa | grep zlib
zlib-devel-1.2.7-19.el7_9.x86_64
zlib-1.2.7-19.el7_9.x86_64
[root@localhost src]#
[root@localhost src]# ../tools/bpftool version
../tools/bpftool v5.10.0-rc3
features: libbfd, skeletons
Thanks.
when "cargo build --release", this error (screenshot error1) came at first, I found the "bpf/bpf_helpers.h" was generated in 2 pathes ( find /home/vagrant/libbpf-bootstrap/ -name "bpf_helpers.h" got "screenshot find"), so cop bpf directorys one to /usr/include (in fact I fixed "vmlinux.h not found" error in the same way after being generated by bpftool ), but got another error: screenshot error2, I got no way to continue... BTW. it works well for the C version in /home/vagrant/libbpf-bootstrap/examples/rust/
screenshot error1
text:
Compiling xdp v0.1.0 (/home/vagrant/libbpf-bootstrap/examples/rust/xdp)
error: failed to run custom build command for `xdp v0.1.0 (/home/vagrant/libbpf-bootstrap/examples/rust/xdp)`
Caused by:
process didn't exit successfully: `/home/vagrant/libbpf-bootstrap/examples/rust/target/release/build/xdp-ea3972c5b26f2632/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Build("Failed to compile obj=/tmp/.tmpIAhV9L/xdppass.o with status=exit code: 1\n stdout=\n \n stderr=\n ./src/bpf/xdppass.bpf.c:2:10: fatal error: \'bpf/bpf_helpers.h\' file not found\n#include <bpf/bpf_helpers.h>\n
^~~~~~~~~~~~~~~~~~~\n1 error generated.\n\n")', xdp/build.rs:19:47
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
screenshot find
screenshot error2
text
~/libbpf-bootstrap/examples/rust$ sudo cargo build --release
Compiling tracecon v0.1.0 (/home/vagrant/libbpf-bootstrap/examples/rust/tracecon)
Compiling xdp v0.1.0 (/home/vagrant/libbpf-bootstrap/examples/rust/xdp)
error: failed to run custom build command for `xdp v0.1.0 (/home/vagrant/libbpf-bootstrap/examples/rust/xdp)`
Caused by:
process didn't exit successfully: `/home/vagrant/libbpf-bootstrap/examples/rust/target/release/build/xdp-ea3972c5b26f2632/build-script-build` (exit code: 101)
--- stderr
libbpf: elf: skipping unrecognized data section(4) .rodata.str1.1
libbpf: elf: skipping unrecognized data section(4) .rodata.str1.1
Warning: unrecognized map: license
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Generate("Failed to generate skeleton for /tmp/.tmpPpu8kb/xdppass.o: Failed to spawn rustfmt")', xdp/build.rs:19:47
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
Would it be possible to add an example of how work with uprobes?
This program will compile and run fine on my 64 bit system, but it will fail with a relocation error on 32 bit systems:
SEC("tp/raw_syscalls/sys_enter")
int sys_enter(struct trace_event_raw_sys_enter *ctx) {
long int n = ctx->id;
bpf_printk("hello world %d", n);
return 0;
}
libbpf: prog 'sys_enter': relo #0: insn #0 (LDX/ST/STX) accesses field incorrectly. Make sure you are accessing pointers, unsigned integers, or fields of matching type and size.
libbpf: prog 'sys_enter': BPF program load failed: Invalid argument
libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
R1 type=ctx expected=fp
; long int n = ctx->id;
0: (85) call unknown#195896080
invalid func unknown#195896080
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
-- END PROG LOAD LOG --
libbpf: failed to load program 'sys_enter'
libbpf: failed to load object 'bootstrap_bpf'
libbpf: failed to load BPF skeleton 'bootstrap_bpf': -22
Failed to load and verify BPF skeleton
I'm cross-compiling using a Yocto build. I've reproduced this both with arm and x86.
From my understanding, the issue comes from the long int
in trace_event_raw_sys_enter
, which is 64 bit in the compiled eBPF program, but 32 bit in the target kernel.
struct trace_event_raw_sys_enter {
struct trace_entry ent;
long int id;
long unsigned int args[6];
char __data[0];
} __attribute__((preserve_access_index));
Indeed, manually changing the id
definition in vmlinux.h
will fix the relocation error:
struct trace_event_raw_sys_enter {
u32 id;
} __attribute__((preserve_access_index));
Q: clang flag for target bpf? hints that using a native target could help, but I guess that would completely break CORE relocations since preserve_access_index
is a -target bpf
-specific attribute, right?
Am I missing something? If I had to fix the issue right now I would replace all long definitions in vmlinux.h
to u32 when targeting 32 bit systems. Shouldn't bpftool btf dump
handle this?
We're using eBPF on embedded systems, where 32 bit is still fairly common.
Thanks.
My minimal app executes correctly:
But when I cat the "/sys/kernel/debug/tracing/trace_pipe" file it looks empty:
I'm running a Ubuntu 22.04 LTS docker container on privileged mode.
This is how I have manually mounted my debugfs:
mount -t debugfs none /sys/kernel/debug
Am I missing anything? Appreciate your help.
I am running an arm64 ubuntu VM on my M1 macbook and make kprobe
does not compile. Other examples except fentry work as expected.
$ uname -a
Linux devvm 5.13.0-21-generic #21-Ubuntu SMP Tue Oct 19 09:01:50 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
$ clang -v
Ubuntu clang version 13.0.0-2
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/aarch64-linux-gnu/11
Selected GCC installation: /usr/bin/../lib/gcc/aarch64-linux-gnu/11
Candidate multilib: .;@m64
Selected multilib: .;@m64
Error:
$ make kprobe
BPF .output/kprobe.bpf.o
kprobe.bpf.c:11:5: error: incomplete definition of type 'struct user_pt_regs'
int BPF_KPROBE(do_unlinkat, int dfd, struct filename *name)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.output/bpf/bpf_tracing.h:429:20: note: expanded from macro 'BPF_KPROBE'
return ____##name(___bpf_kprobe_args(args)); \
^~~~~~~~~~~~~~~~~~~~~~~~
.output/bpf/bpf_tracing.h:409:2: note: expanded from macro '___bpf_kprobe_args'
___bpf_apply(___bpf_kprobe_args, ___bpf_narg(args))(args)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.output/bpf/bpf_helpers.h:165:29: note: expanded from macro '___bpf_apply'
#define ___bpf_apply(fn, n) ___bpf_concat(fn, n)
^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
.output/bpf/bpf_tracing.h:401:2: note: expanded from macro '___bpf_kprobe_args2'
___bpf_kprobe_args1(args), (void *)PT_REGS_PARM2(ctx)
^~~~~~~~~~~~~~~~~~~~~~~~~
.output/bpf/bpf_tracing.h:399:33: note: expanded from macro '___bpf_kprobe_args1'
___bpf_kprobe_args0(), (void *)PT_REGS_PARM1(ctx)
^~~~~~~~~~~~~~~~~~
.output/bpf/bpf_tracing.h:195:49: note: expanded from macro 'PT_REGS_PARM1'
#define PT_REGS_PARM1(x) (((PT_REGS_ARM64 *)(x))->regs[0])
~~~~~~~~~~~~~~~~~~~~~~^
kprobe.bpf.c:11:5: note: forward declaration of 'struct user_pt_regs'
.output/bpf/bpf_tracing.h:429:20: note: expanded from macro 'BPF_KPROBE'
return ____##name(___bpf_kprobe_args(args)); \
^
.output/bpf/bpf_tracing.h:409:2: note: expanded from macro '___bpf_kprobe_args'
___bpf_apply(___bpf_kprobe_args, ___bpf_narg(args))(args)
^
.output/bpf/bpf_helpers.h:165:29: note: expanded from macro '___bpf_apply'
#define ___bpf_apply(fn, n) ___bpf_concat(fn, n)
^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
.output/bpf/bpf_tracing.h:399:33: note: expanded from macro '___bpf_kprobe_args1'
___bpf_kprobe_args0(), (void *)PT_REGS_PARM1(ctx)
^
.output/bpf/bpf_tracing.h:195:29: note: expanded from macro 'PT_REGS_PARM1'
#define PT_REGS_PARM1(x) (((PT_REGS_ARM64 *)(x))->regs[0])
^
.output/bpf/bpf_tracing.h:194:45: note: expanded from macro 'PT_REGS_ARM64'
#define PT_REGS_ARM64 const volatile struct user_pt_regs
^
Any ideas what may be going wrong here?
I am successfully building bootstrap, minimal, uprobe examples and they all work on M1 Macbook running an arm64 ubuntu VM but Fentry fails for some odd reason I can't figure out. Any tips on how to debug such issues:
$ uname -a
Linux devvm 5.13.0-21-generic #21-Ubuntu SMP Tue Oct 19 09:01:50 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
$ sudo ./fentry
[sudo] password for sasan:
libbpf: loading object 'fentry_bpf' from buffer
libbpf: elf: section(2) fentry/do_unlinkat, size 200, link 0, flags 6, type=1
libbpf: sec 'fentry/do_unlinkat': found program 'do_unlinkat' at insn offset 0 (0 bytes), code size 25 insns (200 bytes)
libbpf: elf: section(3) fexit/do_unlinkat, size 256, link 0, flags 6, type=1
libbpf: sec 'fexit/do_unlinkat': found program 'do_unlinkat_exit' at insn offset 0 (0 bytes), code size 32 insns (256 bytes)
libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
libbpf: license of fentry_bpf is Dual BSD/GPL
libbpf: elf: section(5) .rodata.str1.1, size 76, link 0, flags 32, type=1
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
libbpf: elf: section(6) .BTF, size 945, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 364, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 144, link 12, flags 0, type=2
libbpf: looking for externs among 6 symbols...
libbpf: collected 0 externs total
libbpf: loading kernel BTF '/sys/kernel/btf/vmlinux': 0
libbpf: sec 'fentry/do_unlinkat': found 1 CO-RE relocations
libbpf: prog 'do_unlinkat': relo #0: kind <byte_off> (0), spec is [6] struct filename.name (0:0 @ offset 0)
libbpf: CO-RE relocating [0] struct filename: found target candidate [1115] struct filename in [vmlinux]
libbpf: prog 'do_unlinkat': relo #0: matching candidate #0 [1115] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat': relo #0: patched insn #16 (LDX/ST/STX) off 0 -> 0
libbpf: sec 'fexit/do_unlinkat': found 1 CO-RE relocations
libbpf: prog 'do_unlinkat_exit': relo #0: kind <byte_off> (0), spec is [6] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat_exit': relo #0: matching candidate #0 [1115] struct filename.name (0:0 @ offset 0)
libbpf: prog 'do_unlinkat_exit': relo #0: patched insn #22 (LDX/ST/STX) off 0 -> 0
libbpf: prog 'do_unlinkat': failed to attach: ERROR: strerror_r(-524)=22
libbpf: failed to auto-attach program 'do_unlinkat': -524
Failed to attach BPF skeleton
in function: int BPF_PROG(do_unlinkat, int dfd, struct filename *name)
I don't know why the function's first argument is "do_unlinkat", I find nothing from all included header files. And i change the "do_unlinkat" to random string, such as "xxxyyyy". And everything is ok. so what is the first argument means?
git clone and git submodule update --init --recursive
run make in example/c dir
BPF .output/usdt.bpf.o
In file included from usdt.bpf.c:6:
.output/bpf/usdt.bpf.h:94:7: error: use of unknown builtin '__builtin_preserve_enum_value' [-Wimplicit-function-declaration]
if (!BPF_USDT_HAS_BPF_COOKIE) {
^
.output/bpf/usdt.bpf.h:39:2: note: expanded from macro 'BPF_USDT_HAS_BPF_COOKIE'
bpf_core_enum_value_exists(enum bpf_func_id___usdt, BPF_FUNC_get_attach_cookie___usdt)
^
.output/bpf/bpf_core_read.h:205:2: note: expanded from macro 'bpf_core_enum_value_exists'
__builtin_preserve_enum_value(*(typeof(enum_type) *)enum_value, BPF_ENUMVAL_EXISTS)
^
1 error generated.
make: *** [Makefile:106: .output/usdt.bpf.o] Error 1
ubuntu 20.04 with 5.13.0-52-generic kernel
I imitated Minimal Makefile and wrote an out-of-kernel tree BPF program which is not CO-RE.
Hence, I include some kernel header which comes form kernel-devel in BPF program. Then I got some error:
/usr/src/kernels/4.19.90-2106.3.0.0095.oe1.x86_64/include/uapi/linux/types.h:10:2: warning: "Attempt to use kernel headers from user space, see http://kernelnewbies.org/KernelHeaders" [-W#warnings]
#warning "Attempt to use kernel headers from user space, see http://kernelnewbies.org/KernelHeaders"
^
readahead_tune.bpf.c:64:24: error: unknown type name 'bool'
static __always_inline bool is_expected_file(void *name)
^
readahead_tune.bpf.c:71:16: error: use of undeclared identifier 'false'; did you mean 'else'?
return false;
^~~~~
else
readahead_tune.bpf.c:71:16: error: expected expression
readahead_tune.bpf.c:85:30: error: use of undeclared identifier 'FMODE_WILLNEED'
rd_ctx->set_f_mode = FMODE_WILLNEED;
^
readahead_tune.bpf.c:91:5: error: use of undeclared identifier 'bool'
bool first = false;
^
readahead_tune.bpf.c:97:9: error: use of undeclared identifier 'first'
first = true;
...
And the Makefile:
OUTPUT := .output
CLANG ?= clang -v
BPFTOOL ?= bpftool
LINUX_HEADER ?= /usr/src/kernels/`uname -r`
LINUX_INCLUDE = -I $(LINUX_HEADER)/include/uapi \
-I $(LINUX_HEADER)/include
...
# Build BPF code
$(OUTPUT)/%.bpf.o: %.bpf.c $(wildcard %.h) | $(OUTPUT)
$(call msg,BPF,$@)
$(Q)$(CLANG) -g -O2 -target bpf \
-D__TARGET_ARCH_$(ARCH) \
$(LINUX_INCLUDE) $(CLANG_BPF_SYS_INCLUDES) -c $(filter %.c,$^) -o $@
The BPF program:
#include <linux/stddef.h> // for true/false
#include <linux/types.h> // for bool
#include <linux/fs.h> // for FMODE_WILLNEED
...
So what do we need something else to make out-of-kernel tree Non-CO-RE BPF program build successfully?
Similarly i can not run minimal example in Centos7,kernel 4.18.9.
But I am able to run the minimal compiled in kernel 5.4(Centos7)
can anybody help me?
`[root@localhost c]# ./minimal
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) tp/syscalls/sys_enter_write, size 104, link 0, flags 6, type=1
libbpf: sec 'tp/syscalls/sys_enter_write': found program 'handle_tp' at insn offset 0 (0 bytes), code size 13 insns (104 bytes)
libbpf: elf: section(3) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(4) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(5) .rodata, size 28, link 0, flags 2, type=1
libbpf: elf: section(6) .BTF, size 591, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 160, link 0, flags 0, type=1
libbpf: elf: section(8) .symtab, size 192, link 13, flags 0, type=2
libbpf: elf: section(9) .reltp/syscalls/sys_enter_write, size 32, link 8, flags 0, type=9
libbpf: looking for externs among 8 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 4, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: map 'minimal_.rodata' (global data): at sec_idx 5, offset 0, flags 480.
libbpf: map 1 is "minimal_.rodata"
libbpf: sec '.reltp/syscalls/sys_enter_write': collecting relocation for section(2) 'tp/syscalls/sys_enter_write'
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #0: insn #2 against 'my_pid'
libbpf: prog 'handle_tp': found data map 0 (minimal_.bss, sec 4, off 0) for insn 2
libbpf: sec '.reltp/syscalls/sys_enter_write': relo #1: insn #6 against '.rodata'
libbpf: prog 'handle_tp': found data map 1 (minimal_.rodata, sec 5, off 0) for insn 6
libbpf: map 'minimal_.bss': skipped auto-creating...
libbpf: map 'minimal_.rodata': skipped auto-creating...
libbpf: prog 'handle_tp': relo #0: poisoning insn #2 that loads map #0 'minimal_.bss'
libbpf: prog 'handle_tp': relo #1: poisoning insn #6 that loads map #1 'minimal_.rodata'
libbpf: prog 'handle_tp': BPF program load failed: Invalid argument
libbpf: prog 'handle_tp': -- BEGIN PROG LOAD LOG --
0: (85) call bpf_get_current_pid_tgid#14
1: (77) r0 >>= 32
2:
BPF map 'minimal_.bss' is referenced but wasn't created
-- END PROG LOAD LOG --
libbpf: failed to load program 'handle_tp'
libbpf: failed to load object 'minimal_bpf'
libbpf: failed to load BPF skeleton 'minimal_bpf': -22
Failed to load and verify BPF skeleton`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.