Code Monkey home page Code Monkey logo

libbpfgo's Introduction

libbpfgo

GitHub release (latest by date) Go Report Card License


libbpfgo is a Go library for Linux's eBPF project. It was created for Tracee, our open source Runtime Security, and eBPF tracing tool, written in Go. If you are interested in eBPF and its applications, check out Tracee at Github: https://github.com/aquasecurity/tracee.

libbpfgo is built around libbpf - the standard library for interacting with eBPF programs from userspace - which is a C library maintained in Linux upstream. We have created libbpfgo as a thin Go wrapper around the libbpf project.

Installing

libbpfgo uses CGO to interop with libbpf and will expect to be linked with libbpf at run or link time. Simply importing libbpfgo is not enough to get started, and you will need to fulfill the required dependency in one of the following ways:

  1. Install libbpf as a shared object in the system. Libbpf may already be packaged for your distribution and, if not, you can build and install from source. More info here.
  2. Embed libbpf into your Go project as a vendored dependency. This means that the libbpf code is statically linked into the resulting binary, and there are no runtime dependencies. Tracee takes this approach.

Building

Currently you will find the following GNU Makefile rules:

Makefile Rule Description
all builds libbpfgo (dynamic)
clean cleans entire tree
selftest builds all selftests (static)
selftest-run runs all selftests (static)
helpers-test-run runs all helpers tests (static)
  • libbpf dynamically linked (libbpf from OS)
Makefile Rule Description
libbpfgo-dynamic builds dynamic libbpfgo (libbpf)
libbpfgo-dynamic-test 'go test' with dynamic libbpfgo
selftest-dynamic build tests with dynamic libbpfgo
selftest-dynamic-run run tests using dynamic libbpfgo
helpers-test-dynamic-run run helpers package unit tests using dynamic libbpfgo
  • statically compiled (libbpf submodule)
Makefile Rule Description
libbpfgo-static builds static libbpfgo (libbpf)
libbpfgo-static-test 'go test' with static libbpfgo
selftest-static build tests with static libbpfgo
selftest-static-run run tests using static libbpfgo
helpers-test-static-run run helpers package unit tests using static libbpfgo
  • examples
$ make libbpfgo-static => libbpfgo statically linked with libbpf
$ make -C selftest/perfbuffers => single selftest build (static libbpf)
$ make -C selftest/perfbuffers run-dynamic => single selftest run (dynamic libbpf)
$ make selftest-static-run => will build & run all static selftests

Note 01: dynamic builds need your OS to have a recent enough libbpf package (and its headers) installed. Sometimes, recent features might require the use of backported OS packages in order for your OS to contain latest libbpf features (sometimes required by libbpfgo). Note 02: static builds need git submodule init first. Make sure to sync the libbpf git submodule before trying to statically compile or test the libbpfgo repository.

Concepts

libbpfgo tries to make it natural for Go developers to use, by abstracting away C technicalities. For example, it will translate low level return codes into Go error, it will organize functionality around Go struct, and it will use channel as to let you consume events.

In a high level, this is a typical workflow for working with the library:

  1. Compile your bpf program into an object file.
  2. Initialize a Module struct - that is a unit of BPF functionality around your compiled object file.
  3. Load bpf programs from the object file using the BPFProg struct.
  4. Attach BPFProg to system facilities, for example to "raw tracepoints" or "kprobes" using the BPFProg's associated functions.
  5. Instantiate and manipulate BPF Maps via the BPFMap struct and it's associated methods.
  6. Instantiate and manipulate Perf Buffer for communicating events from your BPF program to the driving userspace program, using the RingBuffer struct and it's associated objects.

Example

// initializing
import bpf "github.com/aquasecurity/libbpfgo"
...
bpfModule := bpf.NewModuleFromFile(bpfObjectPath)
bpfModule.BPFLoadObject()

// maps
mymap, _ := bpfModule.GetMap("mymap")
mymap.Update(key, value)

// ring buffer
rb, _ := bpfModule.InitRingBuffer("events", eventsChannel, buffSize)
rb.Poll(300)
e := <-eventsChannel

Releases

libbpfgo does not yet have a regular schedule for cutting releases. There has not yet been a major release but API backwards compatibility will be maintained for all releases with the same major release number. Milestones are created when preparing for release.

  • Major releases are cut when backwards compatibility is broken or major milestones are completed, such as reaching parity with libbpf's API.
  • Minor releases are cut to incorporate new support for libbpf APIs.
  • Patch releases are cut to incorporate important individual or groupings of bug fixes.
  • libbpf support numbering indicates the minimum required libbpf version that must be linked in order to ensure libbpfgo compatibility. For example, v0.2.1-libbpf-0.4.0 means that version 0.2.1 of libbpfgo requires v0.4.0 or newer of libbpf.

Note: some distributions might have local changes to their libbpf package and their version might include backports and/or fixes differently than upstream versions. In those cases we recommend that libbpfgo is used statically compiled.

Contributing

To better receive you, libbpfgo makes available GNU Makefile rules for vagrant machines (amd64/arm64) that can be used to compile and test on Linux and Darwin hosts:

Makefile Rule Description
vagrant-up starts and provisions the vagrant environment
vagrant-ssh connects to machine via SSH
vagrant-halt stops the vagrant machine
vagrant-destroy stops and deletes all traces of the vagrant machine

Once connected to the vagrant box you are ready to build libbpfgo (e.g. make libbpfgo-static).

For further information, check Vagrantfile.md.

Learn more

Please check our github milestones for an idea of the project roadmap. The general goal is to fully implement/expose libbpf's API in Go as seamlessly as possible.

libbpfgo's People

Contributors

alonzivony avatar asafeitani avatar aymericdd avatar bobrik avatar brancz avatar chentao-kernel avatar dependabot[bot] avatar derekparker avatar eyakubovich avatar geyslan avatar gitworkflows avatar grantseltzer avatar guyarb avatar hangrymuppet avatar itamarmaouda101 avatar itaysk avatar javierhonduco avatar jeffmahoney avatar josedonizetti avatar kakkoyun avatar lizrice avatar mozillazg avatar mtcherni95 avatar ndstrahilevitz avatar oshaked1 avatar rafaeldtinoco avatar roikol avatar saschagrunert avatar simar7 avatar yanivagman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libbpfgo's Issues

Implement libbpf bpf_object API

We should expose all of the bpf_object api, just being careful about what is deprecated and/or discouraged when 1.0 is released. This would exclude the bpf_object__*skeleton apis since that's c userspace specific.

  • bpf_object__open_file
  • bpf_object__open_mem
  • bpf_object__pin_maps
  • bpf_object__unpin_maps
  • bpf_object__pin_programs
  • bpf_object__unpin_programs
  • bpf_object__pin
  • bpf_object__unpin
  • bpf_object__close
  • bpf_object__load
  • bpf_object__name
  • bpf_object__kversion
  • bpf_object__set_kversion
  • bpf_object__btf
  • bpf_object__btf_fd
  • bpf_object__for_each_program
  • bpf_object__next_program
  • bpf_object__for_each_map
  • bpf_object__next_map
  • bpf_object__find_program_by_name
  • bpf_object__find_map_by_name
  • bpf_object__find_map_fd_by_name
  • bpf_object__gen_loader

Low-level API:

  • bpf_obj_get_info_by_fd

BTF: support providing external kconfig

When running libbpf in an environment where BTF is enabled, it requires having kconfig data which is read from either /proc/config.gz or /boot/config-xxx.
When working from a container in an environment where kconfig is located under /boot and it is not mounted, libbpf might fail to load the BPF object. To avoid this situation, we can expand libbpfgo to support providing extra kconfig values and give it as an argument to libbpf. See https://github.com/libbpf/libbpf/blob/master/src/libbpf.h#L96

Implement libbpf bpf_map API

We should expose all of the bpf_map api, just being careful about what is deprecated and/or discouraged when 1.0 is released.

  • bpf_map__autocreate
  • bpf_map__set_autocreate
  • bpf_map__name
  • bpf_map__type
  • bpf_map__set_type
  • bpf_map__max_entries
  • bpf_map__set_max_entries
  • bpf_map__resize (alias of above, probably shouldn't be exposed)
  • bpf_map__map_flags (missing for BPFMapLow)
  • bpf_map__set_map_flags
  • bpf_map__numa_node
  • bpf_map__set_numa_node
  • bpf_map__key_size
  • bpf_map__set_key_size
  • bpf_map__value_size
  • bpf_map__set_value_size
  • bpf_map__map_extra
  • bpf_map__set_map_extra
  • bpf_map__btf_key_type_id
  • bpf_map__btf_value_type_id
  • bpf_map__ifindex (missing for BPFMapLow)
  • bpf_map__set_ifindex
  • bpf_map__fd
  • bpf_map__reuse_fd
  • bpf_map__initial_value
  • bpf_map__set_initial_value
  • bpf_map__is_internal
  • bpf_map__set_pin_path
  • bpf_map__is_pinned
  • bpf_map__pin
  • bpf_map__unpin
  • bpf_map__inner_map
  • bpf_map__set_inner_map_fd
  • bpf_map__lookup_and_delete_elem
  • bpf_map__get_next_key

Low-level API:

  • bpf_map_get_info_by_fd
  • bpf_map_get_next_id

Implement libbpf bpf_program API

We should expose all of the bpf_program api, just being careful about what is deprecated and/or discouraged when 1.0 is released.

  • bpf_program__set_ifindex
  • bpf_program__name
  • bpf_program__section_name
  • bpf_program__autoload
  • bpf_program__set_autoload
  • bpf_program__fd
  • bpf_program__pin
  • bpf_program__unpin
  • bpf_program__unload
  • bpf_program__attach
  • bpf_program__attach_perf_event
  • bpf_program__attach_perf_event_opts
  • bpf_program__attach_kprobe
  • bpf_program__attach_kprobe_opts
  • bpf_program__attach_uprobe
  • bpf_program__attach_uprobe_opts
  • bpf_program__attach_tracepoint
  • bpf_program__attach_tracepoint_opts
  • bpf_program__attach_raw_tracepoint
  • bpf_program__attach_trace
  • bpf_program__attach_trace_opts
  • bpf_program__attach_lsm
  • bpf_program__attach_cgroup
  • bpf_program__attach_netns
  • bpf_program__attach_xdp
  • bpf_program__attach_freplace
  • bpf_program__attach_iter
  • bpf_program__type
  • bpf_program__set_type
  • bpf_program__expected_attach_type
  • bpf_program__set_expected_attach_type
  • bpf_program__set_attach_target

Low-level API:

  • bpf_prog_get_info_by_fd

Versioning libbpfgo

As libbpfgo depends on libbpf and there are features in libbpfgo that require some minimal version of libbpf, we should add the version of libbpf to be part of the version of libbpfgo.
This can be, for example:
v0.2-libbpf_0.4

Global application configuration

This article section describes how globals can be used to configure a BPF program before loading. I'm curious if we could have some helpers for setting some simple configurations from Go. For my use case, I want to configure a single uint32, for which an entire map and the lookup overhead seem a little drastic. Any ideas on how we could maybe achieve this?

@yanivagman @eyakubovich @grantseltzer

invalid libbpfgo-static target

Prerequisites: no bpftool installed
Steps to reproduce :

  1. run make clean
  2. run make libbpfgo-static: this will fail as we do not have bpftool installed, however it will an create an empty ./output/vmlinux.h file:
mkdir -p ./output
INFO: generating ./output/vmlinux.h from /sys/kernel/btf/vmlinux
/bin/sh: 3: btf: not found
make: *** [Makefile:94: output/vmlinux.h] Error 127
  1. if we run again make libbpfgo-static then compilation will (wrongly) succeed because ./output/vmlinux.h exists (created in step 2):
mkdir -p ./output/libbpf
CC="gcc" CFLAGS="-g -O2 -Wall -fpie" LD_FLAGS="" \
   make -C /home/ubuntu/dev/libbpfgo/libbpf/src \
        BUILD_STATIC_ONLY=1 \
        OBJDIR=/home/ubuntu/dev/libbpfgo/output/libbpf \
        DESTDIR=/home/ubuntu/dev/libbpfgo/output \
        INCLUDEDIR= LIBDIR= UAPIDIR= install
make[1]: Entering directory '/home/ubuntu/dev/libbpfgo/libbpf/src'
  MKDIR    staticobjs
  CC       bpf.o
  CC       btf.o
  CC       libbpf.o
  CC       libbpf_errno.o
  CC       netlink.o
  CC       nlattr.o
  CC       str_error.o
  CC       libbpf_probes.o
  CC       bpf_prog_linfo.o
  CC       xsk.o
  CC       btf_dump.o
  CC       hashmap.o
  CC       ringbuf.o
  CC       strset.o
  CC       linker.o
  CC       gen_loader.o
  CC       relo_core.o
  AR       libbpf.a
  INSTALL  bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h xsk.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h skel_internal.h
  INSTALL  libbpf.pc
  INSTALL  libbpf.a
make[1]: Leaving directory '/home/ubuntu/dev/libbpfgo/libbpf/src'
CC=clang \
        CGO_CFLAGS="-I/home/ubuntu/dev/libbpfgo/output" \
        CGO_LDFLAGS="-lelf -lz /home/ubuntu/dev/libbpfgo/output/libbpf.a" \
        GOOS=linux GOARCH=amd64 \
        go build \
        -tags netgo -ldflags '-w -extldflags "-static"' \
        .

  1. but if we try to run tests then we will fail (I guess relocation issues):
make libbpfgo-static-test                                                                                                                                                      
CC=clang \                                                                                                                                                                                                            
        CGO_CFLAGS="-I/home/ubuntu/dev/libbpfgo/output" \                                                                                                                                                             
        CGO_LDFLAGS="-lelf -lz /home/ubuntu/dev/libbpfgo/output/libbpf.a" \                                                                                                                                           
        GOOS=linux GOARCH=amd64 \                                                                                                                                                                                     
        go build \                                                                                                                                                                                                    
        -tags netgo -ldflags '-w -extldflags "-static"' \                                                                                                                                                             
        .                                                                                                                                                                                                             
make -C ./selftest/build                                                                                                                                                                                              
make[1]: Entering directory '/home/ubuntu/dev/libbpfgo/selftest/build'                                                                                                                                                
make -C /home/ubuntu/dev/libbpfgo vmlinuxh                                                                                                                                                                            
make[2]: Entering directory '/home/ubuntu/dev/libbpfgo'                                                                                                                                                               
make[2]: Leaving directory '/home/ubuntu/dev/libbpfgo'                                                                                                                                                                
clang -g -O2 -Wall -fpie -target bpf -I../../output -c libbpfgo_test.bpf.c -o libbpfgo_test.bpf.o                                                                                                                     
In file included from libbpfgo_test.bpf.c:3:                                                                                                                                                                          
In file included from ../../output/bpf/bpf_helpers.h:11:                                                                                                                                                              
../../output/bpf/bpf_helper_defs.h:72:83: error: unknown type name '__u64'                                                                                                                                            
static long (*bpf_map_update_elem)(void *map, const void *key, const void *value, __u64 flags) = (void *) 2;                                                                                                          
                                                                                  ^                                                                                                                                   
../../output/bpf/bpf_helper_defs.h:96:42: error: unknown type name '__u32'                                                                                                                                            
static long (*bpf_probe_read)(void *dst, __u32 size, const void *unsafe_ptr) = (void *) 4;                                                                                                                            
                                         ^                                                                                                                                                                            
../../output/bpf/bpf_helper_defs.h:108:16: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]                                                                                                        
static __u64 (*bpf_ktime_get_ns)(void) = (void *) 5;                                                                                                                                                                  
               ^                                                                                                                                                                                                      
../../output/bpf/bpf_helper_defs.h:108:8: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]                                                                                                         
static __u64 (*bpf_ktime_get_ns)(void) = (void *) 5;                                                                                                                                                                  
~~~~~~ ^                                                                                                                                                                                                              
../../output/bpf/bpf_helper_defs.h:108:14: error: function cannot return function type 'int (void)'                                                                                                                   
static __u64 (*bpf_ktime_get_ns)(void) = (void *) 5;                                                                                                                                                                  
             ^                                                                                                                                                                                                        
../../output/bpf/bpf_helper_defs.h:171:50: error: unknown type name '__u32'                                                                                                                                           
static long (*bpf_trace_printk)(const char *fmt, __u32 fmt_size, ...) = (void *) 6;                                                                                                                                   
                                                 ^                                                            

IMO we should add check of existence of bpftool, something like:

$(VMLINUXH): $(OUTPUT)
@if [ ! -f $(BTFFILE) ]; then \
echo "ERROR: kernel does not seem to support BTF"; \
exit 1; \
fi
@if [ ! -f $(VMLINUXH) ]; then \
if [ ! $(BPFTOOL) ]; then \
echo "ERROR: could not find bpftool"; \
exit 1; \
fi; \
echo "INFO: generating $(VMLINUXH) from $(BTFFILE)"; \
$(BPFTOOL) btf dump file $(BTFFILE) format c > $(VMLINUXH); \
fi

libbpf submodule is still at v0.5.0

Looks like we missed dumping libbpf to v0.6.1 like the previous release tag suggested. When updating my submodules I still get:

$ git submodule
 5579664205e42194e1921d69d0839f660c801a4d libbpf (v0.5.0)

$ git describe --tags 5579664205e42194e1921d69d0839f660c801a4d
v0.5.0

Improve perf and ring buffers performance

It is well known that cgo has bad performance when calling c code, and even worse when calling go callbacks from c (see, for example, https://about.sourcegraph.com/go/gophercon-2018-adventures-in-cgo-performance/).

This is actually not a problem for most of the use cases of libbpfgo, where we just need to load a program and attach it, or update a map, as these operations are not that frequent. It may become a problem, however, when we need to poll for events coming from the kernel through one of the perf/ring buffers, as these are much more frequent.

Suggestion: for buffers polling (either perf or ring buffers) let's implement the logic in pure go. These functions can be added as an alternative API for the already existing functions, and will offer high performance where needed.

Specifically, these are the libbpf functions that we will need to implement: C.perf_buffer__poll() and C.ring_buffer_poll() (both are called by the PerfBuffer/RingBuffer poll() function)

Add load/unload BPF program API

We would like to have the capability to load and unload BPF program in runtime.
For the Unload API we could use the libbpf bpf_program__unload (https://github.com/libbpf/libbpf/blob/master/src/libbpf.h#L369). Some open questions here that I would like to share:

  1. if the program is attached, should we first detach it? in that case, we should add this capability as well (detaching BPF programs)
  2. if the program is a link, should we remove it from Module.links? do we also handle the link removal from libbpf prospective?

Now, regarding the Load API, we could use the libbpf function bpf_prog_load (https://github.com/libbpf/libbpf/blob/master/src/bpf.h#L144), however for that we should wait till we adopt libbpfv0.6.0.

FYI @grantseltzer @rafaeldtinoco

eBPF CO-RE improvements: Add support for setting global variables in bpf code

libbpf has a feature for setting global variables in bpf code from userspace. This can be done by accessing a map called ".rodata" and setting values there. The setting happens at load time instead of runtime, making it more performant. Users could rely on this instead of maps or compile-time macros in their bpf projects.

Relevant mailing list thread: https://lore.kernel.org/bpf/CAO658oVyB2b+Y6K3--sAhTcXfmPpmPjLhA0z7bbjyjhzDV8kcA@mail.gmail.com/

Implement libbpf bpf_create_map

@parca-dev contributor here. First of all, thanks for the fantastic work you are doing here. We are glad libbpf exists.

To be able to use BPF_MAP_TYPE_ARRAY_OF_MAPS and BPF_MAP_TYPE_HASH_OF_MAPS map types and create maps in user-space we need to access the libbpf#bpf_create_map function (and maybe bpf_map__set_inner_map_fd).

I would be happy to give it a try if you don't have enough capacity. However, I don't know what I don't know, so it would be slow.

bpf_create_map: https://github.com/libbpf/libbpf/blob/5b9d079c7f12d5c0b961623e56fdae17c721debf/src/bpf.c#L166-L178

The interactions I want to replicate can be seen here https://hechao.li/2019/03/19/Use-Map-in-Map-in-BPF-programs-via-Libbpf/

@yanivagman @eyakubovich @grantseltzer

cc @brancz

A good source of features: bpfcc/libbpf-tools

All tools from bpfcc/libbpf-tools project should be portable to libbpfgo if libbpfgo supported all features present in libbpf.

This can be a good source of supportability, tests and tools.

Initial suggestion is to bring:

  • bindsnoop
  • biolatency
  • biopattern

and then move on with the list.

We can sync .bpf.c portion of it from bpfcc/libbpf-tools project and keep userland in go.

NewModuleFromFileArgs should not specify BTF if there isn't one

There is a bug in a logic I added to NewModuleFromFileArgs(). If there are no BTF files specified, I specify "/sys/kernel/btf/vmlinux". In this particular environment my BTF is coming from the debug package (which is hardcoded in libbpf and should work). With this change:

diff --git a/libbpfgo.go b/libbpfgo.go
index 5bc8a80..02093a1 100644
--- a/libbpfgo.go
+++ b/libbpfgo.go
@@ -348,15 +348,15 @@ func NewModuleFromFileArgs(args NewModuleArgs) (*Module, error) {
        if err := bumpMemlockRlimit(); err != nil {
                return nil, err
        }
-       if args.BTFObjPath == "" {
-               args.BTFObjPath = "/sys/kernel/btf/vmlinux"
-       }
-       btfFile := C.CString(args.BTFObjPath)
+       //if args.BTFObjPath == "" {
+       //args.BTFObjPath = "/sys/kernel/btf/vmlinux"
+       //}
+       //btfFile := C.CString(args.BTFObjPath)
        bpfFile := C.CString(args.BPFObjPath)

        opts := C.struct_bpf_object_open_opts{}
        opts.sz = C.sizeof_struct_bpf_object_open_opts
-       opts.btf_custom_path = btfFile // instruct libbpf to use user provided kernel BTF file
+       //opts.btf_custom_path = btfFile // instruct libbpf to use user provided kernel BTF file

        if strings.Compare(args.KConfigFilePath, "") != 0 {
                kConfigFile := C.CString(args.KConfigFilePath)
@@ -370,7 +370,7 @@ func NewModuleFromFileArgs(args NewModuleArgs) (*Module, error) {
        }

        C.free(unsafe.Pointer(bpfFile))
-       C.free(unsafe.Pointer(btfFile))
+       //C.free(unsafe.Pointer(btfFile))

        return &Module{
                obj: obj,

CFLAGS #cgo pragma in libbpfgo.go cannot be ignored for static building

When building a project that uses libbpfgo, the go compiler must be given a linker flag pointing to libbpf. If libbpf is installed in a standard location -lbpf can be passed. They can also point to a specific location. Since go get attempts to build the package without any ldflags, this will cause the errors pasted below.

Users could do something like: CGO_LDFLAGS="-lbpf" go get github.com/aquasecurity/tracee. Alternatively we can add a line for CGO_LDFLAGS in the CGO code.

This shouldn't cause any related issues if not using go get.

For example:

[*] go get github.com/aquasecurity/libbpfgo

go: downloading github.com/aquasecurity/libbpfgo v0.0.0-20210517223445-485e1cdc10ae
go: downloading golang.org/x/sys v0.0.0-20210514084401-e8d321eab015
# github.com/aquasecurity/libbpfgo
/usr/bin/ld: $WORK/b001/_x003.o: in function `init_ring_buf':
../../../../../pkg/mod/github.com/aquasecurity/[email protected]/libbpfgo.go:40: undefined reference to `ring_buffer__new'
/usr/bin/ld: $WORK/b001/_x003.o: in function `init_perf_buf':
../../../../../pkg/mod/github.com/aquasecurity/[email protected]/libbpfgo.go:54: undefined reference to `perf_buffer__new'
/usr/bin/ld: $WORK/b001/_x003.o: in function `attach_kprobe_legacy':
../../../../../pkg/mod/github.com/aquasecurity/[email protected]/libbpfgo.go:158: undefined reference to `bpf_program__attach_perf_event'
/usr/bin/ld: ../../../../../pkg/mod/github.com/aquasecurity/[email protected]/libbpfgo.go:159: undefined reference to `libbpf_get_error'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_link__destroy':
/tmp/go-build/cgo-gcc-prolog:74: undefined reference to `bpf_link__destroy'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__fd':
/tmp/go-build/cgo-gcc-prolog:93: undefined reference to `bpf_map__fd'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__max_entries':
/tmp/go-build/cgo-gcc-prolog:112: undefined reference to `bpf_map__max_entries'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__pin':
/tmp/go-build/cgo-gcc-prolog:132: undefined reference to `bpf_map__pin'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__resize':
/tmp/go-build/cgo-gcc-prolog:153: undefined reference to `bpf_map__resize'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__set_pin_path':
/tmp/go-build/cgo-gcc-prolog:173: undefined reference to `bpf_map__set_pin_path'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map__unpin':
/tmp/go-build/cgo-gcc-prolog:193: undefined reference to `bpf_map__unpin'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map_delete_elem':
/tmp/go-build/cgo-gcc-prolog:214: undefined reference to `bpf_map_delete_elem'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map_lookup_elem':
/tmp/go-build/cgo-gcc-prolog:236: undefined reference to `bpf_map_lookup_elem'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_map_update_elem':
/tmp/go-build/cgo-gcc-prolog:259: undefined reference to `bpf_map_update_elem'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__find_map_by_name':
/tmp/go-build/cgo-gcc-prolog:290: undefined reference to `bpf_object__find_map_by_name'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__find_program_by_name':
/tmp/go-build/cgo-gcc-prolog:309: undefined reference to `bpf_object__find_program_by_name'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__load':
/tmp/go-build/cgo-gcc-prolog:328: undefined reference to `bpf_object__load'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__open':
/tmp/go-build/cgo-gcc-prolog:346: undefined reference to `bpf_object__open'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__open_buffer':
/tmp/go-build/cgo-gcc-prolog:366: undefined reference to `bpf_object__open_buffer'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__attach_kprobe':
/tmp/go-build/cgo-gcc-prolog:387: undefined reference to `bpf_program__attach_kprobe'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__attach_lsm':
/tmp/go-build/cgo-gcc-prolog:405: undefined reference to `bpf_program__attach_lsm'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__attach_raw_tracepoint':
/tmp/go-build/cgo-gcc-prolog:424: undefined reference to `bpf_program__attach_raw_tracepoint'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__attach_tracepoint':
/tmp/go-build/cgo-gcc-prolog:444: undefined reference to `bpf_program__attach_tracepoint'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__fd':
/tmp/go-build/cgo-gcc-prolog:463: undefined reference to `bpf_program__fd'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__get_type':
/tmp/go-build/cgo-gcc-prolog:482: undefined reference to `bpf_program__get_type'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__set_autoload':
/tmp/go-build/cgo-gcc-prolog:503: undefined reference to `bpf_program__set_autoload'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_program__set_tracepoint':
/tmp/go-build/cgo-gcc-prolog:522: undefined reference to `bpf_program__set_tracepoint'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_perf_buffer__poll':
/tmp/go-build/cgo-gcc-prolog:607: undefined reference to `perf_buffer__poll'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_ring_buffer__poll':
/tmp/go-build/cgo-gcc-prolog:661: undefined reference to `ring_buffer__poll'
/usr/bin/ld: $WORK/b001/_x003.o: in function `set_print_fn':
../../../../../pkg/mod/github.com/aquasecurity/[email protected]/libbpfgo.go:35: undefined reference to `libbpf_set_print'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_bpf_object__close':
/tmp/go-build/cgo-gcc-prolog:274: undefined reference to `bpf_object__close'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_perf_buffer__free':
/tmp/go-build/cgo-gcc-prolog:589: undefined reference to `perf_buffer__free'
/usr/bin/ld: $WORK/b001/_x003.o: in function `_cgo_a66f50300837_Cfunc_ring_buffer__free':
/tmp/go-build/cgo-gcc-prolog:643: undefined reference to `ring_buffer__free'
collect2: error: ld returned 1 exit status

errptrError format %s has arg pid of wrong type int

With the selftest PR in place:

#28

I got

$ make libbpfgo-static-test
make -C ./selftest/build
make[1]: Entering directory '/home/rafaeldtinoco/work/sources/ebpf/libbpfgo/selftest/build'
make -C /home/rafaeldtinoco/work/sources/ebpf/libbpfgo vmlinuxh
make[2]: Entering directory '/home/rafaeldtinoco/work/sources/ebpf/libbpfgo'
mkdir -p ./output
make[2]: Leaving directory '/home/rafaeldtinoco/work/sources/ebpf/libbpfgo'
clang -g -O2 -Wall -fpie -target bpf -I../../output -c libbpfgo_test.bpf.c -o libbpfgo_test.bpf.o
make[1]: Leaving directory '/home/rafaeldtinoco/work/sources/ebpf/libbpfgo/selftest/build'
CC=clang \
   CGO_CFLAGS="-I/home/rafaeldtinoco/work/sources/ebpf/libbpfgo/output" \
   CGO_LDFLAGS="-lelf -lz /home/rafaeldtinoco/work/sources/ebpf/libbpfgo/output/libbpf.a" \
   sudo -E go test .
# github.com/aquasecurity/libbpfgo
./libbpfgo.go:790:15: errptrError format %s has arg pid of wrong type int
./libbpfgo.go:802:14: Sprintf format %s has arg offset of wrong type uint32
FAIL	github.com/aquasecurity/libbpfgo [build failed]
FAIL
make: *** [Makefile:79: libbpfgo-static-test] Error 2

from go test. I believe:

10aa311

Fixes the issue

Tracepoint naming isn't libbpf 1:1

The libbpf convention is to split the tracepoint SEC name using slash / as separator.

https://github.com/libbpf/libbpf/blob/a3c0cc19d4b93cb0b7088c5604b0cec1c6863fde/src/libbpf.c#L9433-L9456

In a different way, libbpfgo is using colon : as separator.

libbpfgo/libbpfgo.go

Lines 755 to 762 in 20c6919

func (p *BPFProg) AttachTracepoint(tp string) (*BPFLink, error) {
tpEvent := strings.Split(tp, ":")
if len(tpEvent) != 2 {
return nil, fmt.Errorf("tracepoint must be in 'category:name' format")
}
tpCategory := C.CString(tpEvent[0])
tpName := C.CString(tpEvent[1])
link := C.bpf_program__attach_tracepoint(p.prog, tpCategory, tpName)

Link Detach support is missing

I've just spoke to @rafaeldtinoco and it might there be a need for us to support detach/unloading of specific ebpf programs links without to close the object (and consequently its other relations).

Allow BPFMap.Update() to take arbitrary types as keys/values

Since keys/values can be arbitrary types in bpf code, libbpfgo should also support this. Currently GetUnsafePointer() will fail if what's passed isn't a uint/int/byte slice.

We should be able to just use an unsafe.Pointer, however special care should be taken for testing with arrays, slices, maps, interfaces, deep pointers, etc...

selftests: ringbuffers: failing with timeout

Running selftest/ringbuffers we got [!] ERROR: selftest timeout:

libbpfgo/selftest/ringbuffers on ๐ŸŒฑ main [๐Ÿ“๐ŸŒŸ] via ๐Ÿน v1.17.1
โฏ make run-static
INFO: entering ./selftest/ringbuffers...
make[1]: Entering directory '/home/grg/code/libbpfgo/selftest/ringbuffers'
make -C /home/grg/code/libbpfgo libbpfgo-static
make[2]: Entering directory '/home/grg/code/libbpfgo'
CC=clang \
  CGO_CFLAGS="-I/home/grg/code/libbpfgo/output" \
  CGO_LDFLAGS="-lelf -lz /home/grg/code/libbpfgo/output/libbpf.a" \
  GOOS=linux GOARCH=amd64 \
  go build \
  -tags netgo -ldflags '-w -extldflags "-static"' \
  .
make[2]: Leaving directory '/home/grg/code/libbpfgo'
CC=clang \
  CGO_CFLAGS="-I/home/grg/code/libbpfgo/output" \
  CGO_LDFLAGS="-lelf -lz /home/grg/code/libbpfgo/output/libbpf.a" \
  GOOS=linux GOARCH=amd64 \
  go build \
  -tags netgo -ldflags '-w -extldflags "-static"' \
  -o main-static ./main.go
sudo ./run.sh main-static
[!] ERROR: selftest timeout
make[1]: *** [Makefile:85: run-static] Error 3

Create helpers for checking raw arguments

In tracee PR #1123 parsing of syscall arguments into strings can be turned on with an argument to the output flag. We should provide helpers for checking particular values in the raw

For example, the clone 'flag' argument has a bunch of options, we could provide helpers like bool HasClearTID(flags) to detect if flags contains CLONE_CHILD_CLEARTID

new ParsePtraceRequest() function shouldn't trigger error

While adjusting tracee to use the new libbpfgo v0.2.3-libbpf-0.6.1 I have faced the following errors:

$ sudo ./dist/tracee-ebpf -o format:json -o option:parse-arguments --trace comm=strace | ./dist/tracee-rules --input-tracee file:stdin --input-tracee format:json
Loaded 14 signature(s): [TRC-1 TRC-13 TRC-2 TRC-14 TRC-3 TRC-11 TRC-9 TRC-4 TRC-5 TRC-12 TRC-8 TRC-6 TRC-10 TRC-7]
not a valid ptrace request value: 16910
not a valid ptrace request value: 16910
not a valid ptrace request value: 16910
not a valid ptrace request value: 16910

that happens because we're missing support for:

#define PTRACE_GET_SYSCALL_INFO 0x420e

ptrace flag within ParsePtraceRequest() function.

Fix broken selftests

In #32 I made breaking API changes and failed to update the selftests accordingly. After #28 merges we should follow up immediately with fixing the selftests. Additionally we need to make sure the new map_update selftest is being run as part of the same selftest run target.

ParseSocketType() has a printf leftover from debug

Because ParseSocketType() has a leftover: at https://github.com/aquasecurity/libbpfgo/blob/main/helpers/argumentParsers.go#L1188

fmt.Println(f)

We're facing issues in tracee when trying to parse json:

{"timestamp":1642733775914253604,"processId":16364,"threadId":1536827,"parentProcessId":1,"hostProcessId":16364,"hostThreadId":1536827,"hostParentProcessId":1,"userId":0,"mountNamespace":4026531840,"pidNamespace":4026531836,"processName":"nordvpnd","hostName":"fujitsu","containerId":"","eventId":"41","eventName":"socket","argsNum":3,"returnValue":151,"stackAddresses":null,"args":[{"name":"domain","type":"string","value":"AF_NETLINK"},{"name":"type","type":"string","value":"SOCK_RAW|SOCK_CLOEXEC"},{"name":"protocol","type":"int","value":0}]}
[SOCK_RAW]

as you can see there is a [SOCK_RAW] in the output, causing json not to be in correct format.

Selftests are not run against each version of libbpf

We have the file selftests/.libbpf-versions.txt which should allow us to run each selftest against each version of libbpf which would be checked out in the git submodule. This doesn't appear to be the case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.