Code Monkey home page Code Monkey logo

p4-dpdk-target's Introduction

P4 DPDK Target Components

p4-dpdk-target repo contains the P4-DPDK target specific code that supports the target independent Table Driven Interface (TDI).

p4-dpdk-target works in conjunction with following repos:

Table Driven Interface:

    (https://github.com/p4lang/tdi)
    TDI (Table Driven Interface) provides the target independent frontend
    interface while p4-dpdk-target provides the target dependent backend
    for DPDK SWX pipeline.
    p4-dpdk-target submodules tdi repo.  p4-dpdk-target isn't currently
    defined in p4-dpdk-target/include/bf_rt directory. 

DPDK SWX Pipeline:

    (https://github.com/DPDK/dpdk/blob/main/lib/pipeline)
    Version: 22.07-rc2
    
    P4 DPDK SWX pipeline implements the P4 pipeline in a DPDK software switch.
    The software switch is driven by artifacts generated by the P4 compiler
    for P4 DPDK SWX pipeline.
    p4-dpdk-target submodules dpdk repo to build P4 DPDK pipeline.
    Note about DPDK Patches (p4-dpdk-target/src/lld/dpdk/patch):
       DPDK patches 007-Fix-pipeline-structs-initialization.patch
       is a temporary solution and will be removed. 
       The fixes are being worked upon and will be committed soon.

P4 Compiler DPDK Backend:

    (https://github.com/p4lang/p4c/tree/main/backends/dpdk)
    P4C DPDK backend creates the artifact files used by p4-dpdk-target 
    and DPDK SWX pipeline.
    p4-dpdk-target uses bfrt.json to define the program independent frontend 
    bf_rt interface.
    context.json is used to map the P4 runtime info to the P4 DPDK target
    specific info.
    The P4 DPDK spec file from P4C DPDK, which is used by DPDK SWX pipeline
    to setup the software pipeline, is also loaded through p4-dpdk-target.

P4-OVS:

    (https://github.com/ipdk-io/p4-ovs)
    The remote control plane interfaces like P4 Runtime and OpenConfig are
    supported through P4-OVS.  P4-OVS maps these remote protocol messages 
    and maps them to tdi_rt interface.

Code Organization

The drivers are P4 independent. At the time of device initialization, they take artifacts associated with P4 as inputs.

p4-dpdk-target is organized as follows

include/
    Header files associated with various driver modules
    bf_rt:
        bf_rt defines the interim frontend interface with bfrt.
    tdi_rt:
        tdi_rt defines the frontend interface with TDI.

src/
    Driver source files. Largely they are categorized into these submodules.

    pipe_mgr:
        P4 Pipeline Mgmt API (e.g. tables, actions, stats, meters)
    port_mgr:
        Port Mgmt API
    lld:
        Low Level Driver (e.g. DPDK library)
    dvm:
        Device Mgmt (e.g. device/port instantiation, HA sequences)
    tdi_rt:
        P4 program independent and object driven implementation with TDI.
    bf_rt:
        P4 program independent and object driven implementation.
    bf_pal:
        Platform related code to support device and port management.
    ctx_json:
        Utility used for parsing cJSON structures in context JSON.

bf_switchd/
    Reference implementation of application to exercise the drivers.

third-party
	tdi:
    	Target independent frontend interface.

tools/
    Scripts to help dependencies installation, ports setup and executing.

dependencies/
    Python requirements files used by the driver.    

doc/
    Files used for doxygen.

Building and installing

The building of drivers produces a set of libraries that need to be loaded (or linked to) the application.

Note: SDE is the top level directory with this p4-dpdk-target, syslibs and utils repo cloned in next step

Create install directory under SDE

    sudo -s
    mkdir install

Set environment variables

To set environment variables for SDE, see below :-

    cd p4-dpdk-target/tools/setup
    source p4sde_env_setup.sh <path to SDE directory>

Ensure SDE, SDE_INSTALL and LD_LIBRARY_PATH environment variables are set correctly

Install dependent packages

To Install the dependencies for p4-driver on the platform Fedora 33, see below:-

Note:- Make sure that your yum repository proxy and environment proxies are set properly and you have sudo access.

    pip3 install distro (dependency)
    cd p4-dpdk-target/tools/setup
    python3 install_dep.py

Building P4 DPDK target

   cd $SDE/p4-dpdk-target
   git submodule update --init --recursive --force
   ./autogen.sh
   ./configure --prefix=$SDE_INSTALL (For both bfrt and TDI build)
   ./configure --prefix=$SDE_INSTALL --with-generic-flags=yes (For TDI build)
   make -j
   make install

Artifacts installed

Here're the artifacts that get installed for p4-driver.

Build artifacts:

    header files for driver API to $SDE_INSTALL/include/

    libdriver.[a,la,so] to $SDE_INSTALL/lib/
        driver library to manage the device
    bf_switchd to $SDE_INSTALL/bin
        reference implementation to exercise driver API

Running Reference App

bf_switchd is a reference application that can be configured through a debug CLI.

Running of bf_switchd

    Need to set PYTHONPATH and PYTHONHOME for bfrt_python/tdi_python to work.
    export SDE_INSTALL=<path to install directory>
    export LD_LIBRARY_PATH=$SDE_INSTALL/lib:$SDE_INSTALL/lib64:$SDE_INSTALL/lib/x86_64-linux-gnu
    export PYTHONPATH=$SDE_INSTALL/lib/python3.8/:$SDE_INSTALL/lib/python3.8/lib-dynload:$SDE_INSTALL/lib/python3.8/site-packages
    export PYTHONHOME=$SDE_INSTALL/lib/python3.8
    cd $SDE_INSTALL/bin
    ./bf_switchd --install-dir $SDE_INSTALL --conf-file $SDE_INSTALL/share/dpdk/<P4 program name>/<P4 program name>.conf --init-mode=cold --status-port 7777

Config file used by bf_switchd

bf_switchd uses a config file to initialize.  Following are some of the key fields:
    bfrt-config: Use file generated from the P4C DPDK for P4 pipeline definition.
    port-config: Optional file to describe initial ports topology.
    context:     Use file generated from the P4C DPDK to map P4 to P4 DPDK.
    config:      Use file generated from the P4C DPDK to setup P4 DPDK pipeline.
 {
    "chip_list": [
        {
            "id": "asic-0",
            "chip_family": "dpdk",
            "instance": 0
        }
    ],
    "instance": 0,
    "p4_devices": [
        {
            "device-id": 0,
            "eal-args": "dummy -n 4 -c 3",
            "mempools": [
               {
                   "name": "MEMPOOL0",
                   "buffer_size": 2304,
                   "pool_size": 1024,
                   "cache_size": 256,
                   "numa_node": 0
               }
            ],
            "p4_programs": [
                {
                    "program-name": "<P4 program name>",
                    "bfrt-config": "share/dpdk/<P4 program name>/tdi.json",
                    "port-config": "share/dpdk/<P4 program name>/ports_topology.json",
                    "p4_pipelines": [
                        {
                            "p4_pipeline_name": "pipe",
                            "core_id": 1,
                            "numa_node": 0,
                            "context": "share/dpdk/<P4 program name>/pipe/context.json",
                            "config": "share/dpdk/<P4 program name>/pipe/<P4 program name>.spec",
                            "pipe_scope": [
                                    0,
                                    1,
                                    2,
                                    3
                            ],
                            "path": "share/dpdk/<P4 program name>"
                        }
                    ]
                }
            ]
        }
    ]
}

CLI interface

Once bf_switchd starts running, bfshell is available.

    bfshell> tdi_python

    tdi.<P4 program>.enable
    pipe = tdi.<P4 program>.pipe.ingress
    from netaddr import IPAddress
    pipe.<P4 table>.<P4 action>(<action parameters>)
    eg) pipe.ipv4_lpm.add_with_send(dst_addr=IPAddress('192.168.2.0'),dst_addr_p_length=24, port=1)

p4-dpdk-target's People

Contributors

aman20010sinha avatar asafravid avatar asunkad avatar b3n-l avatar haripriyavemuri avatar hoooao avatar jamescchoi avatar jnfoster avatar nisbhat avatar nupuruttarwar avatar qobilidop avatar riyazahm22 avatar rst0git avatar ruchitg avatar satish153 avatar swaroopsarma avatar vinothcm avatar vsureshkumarp avatar weiqiand avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

p4-dpdk-target's Issues

Why are submodules repeated so often?

Here is the structure of submodules of this repository, and the submodules of those repositories, as of 2021-Dec-21:

+ p4-dpdk-target
  + target-syslibs @ f646177
    + gperftools @ f7c6fb6
    + zlog @ 876099f
  + target-utils @ 61969dd
    + cJSON @ d348621
    + cpython @ 3d8993a
    + target-syslibs @ 580b5e5    Repeat of target-syslibs above.  Why?
    + tommyds @ 5ada275
    + xxHash @ 94e5f23
  + tdi @ 21da36e
    + target-syslibs @ 4f765f5    Repeat of target-syslibs above.  Why?
    + target-utils @ 243a99e      Repeat of target-utils above.    Why?

Is it intentional that so many submodules are repeated? If so, why?

Counters wildcard read doesn't show data

Problem Statement

While using P4Runtime on p4-DPDK-target ovs I want to gather all counter values in single query. This functionality is available in behavioral model and described in P4Runtime spec/proto, which should be target agnostic.

  1. I create CounterEntry ce = counter_entry["someCounter"]
  2. I edit manually some counter entries, intentionally. Example: ce.index=10 ce.packet_count=101 ce.modify
  3. If I want to see only single counter entry, data is visible ce?
  4. But, If i want to see all entries in single query, data is not visible. Commands used: ce.index=None list(ce.read())

Steps to reproduce & Current output

P4 program with some counter (let's say 1024 in size, PACKETS type)
ce = counter_entry["someCounter"]
ce.index=10 ce.packet_count=101 ce.modify
ce.index=11 ce.packet_count=2000 ce.modify
ce.index=None list(ce.read())

output:

...
 counter_id: 317030940 ("someCounter")
index {
  index: 10
}
data {
  
}
,
 counter_id: 317030940 ("someCounter")
index {
  index: 11
}
data {
  
}
...

Expected output

...
 counter_id: 317030940 ("someCounter")
index {
  index: 10
}
data {
   packet_count: 101
}
,
 counter_id: 317030940 ("someCounter")
index {
  index: 11
}
data {
   packet_count: 2000
}
...

Mistake in README for reference application config file

The example json config for the bf_switchd reference application has unnecessary commas on lines 6 and 29 that cause the program to fail to process the file. In the readme the example json given is

    {
        "chip_list": [
            {
                "id": "asic-0",
                "chip_family": "dpdk",
                "instance": 0,
            }
        ],
        "instance": 0,
        "p4_devices": [
            {
                "device-id": 0,
                "p4_programs": [
                    {
                        "program-name": "<P4 program name>",
                        "cpu_numa_node": "0",
                        "bfrt-config": "share/dpdk/<P4 program name>/bf-rt.json",
                        "port-config": "share/dpdk/<P4 program name>/ports_topology.json",
                        "eal-args": "dummy -n 4 -c 3",
                        "p4_pipelines": [
                            {
                                "p4_pipeline_name": "pipe",
                                "context": "share/dpdk/<P4 program name>/pipe/context.json",
                                "config": "share/dpdk/<P4 program name>/pipe/<P4 program name>.spec",
                                "path": "share/dpdk/<P4 program name>"
                            }
                        ]
                    }
                ],
            }
        ]
    }

When it should be

    {
        "chip_list": [
            {
                "id": "asic-0",
                "chip_family": "dpdk",
                "instance": 0
            }
        ],
        "instance": 0,
        "p4_devices": [
            {
                "device-id": 0,
                "p4_programs": [
                    {
                        "program-name": "<P4 program name>",
                        "cpu_numa_node": "0",
                        "bfrt-config": "share/dpdk/<P4 program name>/bf-rt.json",
                        "port-config": "share/dpdk/<P4 program name>/ports_topology.json",
                        "eal-args": "dummy -n 4 -c 3",
                        "p4_pipelines": [
                            {
                                "p4_pipeline_name": "pipe",
                                "context": "share/dpdk/<P4 program name>/pipe/context.json",
                                "config": "share/dpdk/<P4 program name>/pipe/<P4 program name>.spec",
                                "path": "share/dpdk/<P4 program name>"
                            }
                        ]
                    }
                ]
            }
        ]
    }

Note the removed commas after "instance": 0 and the second to last ].

An example for ports.json

Hello, I want to run a simple program, however I get a
ERROR: bf_device_add failed(2) for dev_id 0
ERROR: bf_switchd_device_add failed(2) for dev_id 0
erros.
I suspect it is because ports.json.

I have searched the web for an example but alas, couldn't find any.

I would love a detailed explanation \ guide \ example that works, for how to write my ports.json..

thank you for your time !

Error:

023-01-08 13:29:26.912067 BF_PORT ERROR - port_config_json_parse_ports:507: Failed to add port ens33
2023-01-08 13:29:26.912226 BF_PORT ERROR - parse_port_config_json:586: Failed to parse ports from Port Config JSON
2023-01-08 13:29:26.912395 BF_PORT ERROR - port_mgr_config_import : Error in parsing the port config file
2023-01-08 13:29:26.912580 BF_DVM ERROR - Device add handling failed for dev 0, sts No system resources (2), Client port_mgr 
2023-01-08 13:29:26.912706 BF_DVM ERROR - Device add failed for dev 0, sts No system resources (2)
ERROR: bf_device_add failed(2) for dev_id 0
ERROR: bf_switchd_device_add failed(2) for dev_id 0


Port topology example

Hello,

I have been trying to run some examples in p4-dpdk and I am not able to understand how you bind the interfaces to the switch using a port-config file.

I see that in the bf_switchd config file there exists a port-config file, but I couldn't find any example of such a port-config file somewhere in the repository.

Assigning Parameters for Physical NIC Devices in bf_switchd.

Hello,
Does anyone know if there is any way to pass operating parameters (mtu, bsize ...) to the network card to be linked in the BF_Switchd through the .json file for port configuration?
If yes, how?
Or if there is any other way to do this with bf_Switchd running.
Note: I use an Intel 82599ES 10-Gigabit SFI/SFP+ network card linked in 'link' mode.
My port configuration is the one below, I believe I can assign these parameters in "link_port_attributes", but how/which ones are accepted?
{
"ports": [
{
"dev_port": 0,
"port_name": "LINK0",
"mempool_name": "MEMPOOL0",
"port_dir": "default",
"port_in_id": 0,
"pipe_in": "pipe",
"port_out_id": 0,
"pipe_out": "pipe",
"port_type": "link",
"net_port" : 0,
"link_port_attributes": {
"pcie_bdf": "0000:b3:00.0"
}
}
]

Grateful.

Proper way to start multiple pipeline on 1 device

i would like to start multiple pipelines, each mapped to a single thread, for a single device. Is that possible with bf_switchd? (i think the dpdk's swx pipeline mentioned that feature) And if so, how should i do it?

Problem when running example

Hello,

I encountered a problem while trying to run the example you provide (simple_l2l3_lpm.p4).

I have followed the general README instructions along with the example-specific instructions.

After running the bf_switchd with the example, as instructed, I get the following:

$ sudo ./bf_switchd --install-dir $SDE/install --conf-file $SDE/install/share/examples/simple_l2l3_lpm/simple_l2l3_lpm.conf --init-mode=cold --status-port 7777 --skip-hld mkt

Unknown skip-hld option m 
Unknown skip-hld option k 
Unknown skip-hld option t 
Install dir: $SDE/install (0x56290255f850)
bf_switchd: system services initialized
bf_switchd: loading conf_file $SDE/install/share/examples/simple_l2l3_lpm/simple_l2l3_lpm.conf...
bf_switchd: processing device configuration...
Cannot find chip_list in conf file
bf_switchd: processing P4 configuration...
STUB:port_mgr_platform_init DPDK
bf_switchd: drivers initialized

bf_switchd: initialized 0 devices
bf_switchd: spawning cli server thread
bf_switchd: spawning driver shell
bf_switchd: server started - listening on port 9999
Error: Can't open plugin "tdi": bfshell_plugin_tdi.so: cannot open shared object file: No such file or directory
BFShell: Command tdi_python will be unavailable due to missing components.
BFShell: Command tdi_run_file will be unavailable due to missing components.
Error parsing XML: File $SDE/install/share/cli/xml/tdi.xml
Error: Can't open plugin "debug": bfshell_plugin_debug.so: cannot open shared object file: No such file or directory
BFShell: Command debug_python will be unavailable due to missing components.
Error parsing XML: File $SDE/install/share/cli/xml/debug.xml
Error: Can't open plugin "pipemgr": bfshell_plugin_pipemgr.so: cannot open shared object file: No such file or directory
BFShell: Command access will be unavailable due to missing components.
BFShell: Command ucli will be unavailable due to missing components.
BFShell: Command version will be unavailable due to missing components.
BFShell: Command pipemgr will be unavailable due to missing components.
BFShell: Command end will be unavailable due to missing components.
BFShell: Command show will be unavailable due to missing components.
BFShell: Command show tcams will be unavailable due to missing components.
BFShell: Command show tcam will be unavailable due to missing components.
Error parsing XML: File $SDE/install/share/cli/xml/pipemgr.xml
Error: Can't open plugin "bf_rt": bfshell_plugin_bf_rt.so: cannot open shared object file: No such file or directory
BFShell: Command bfrt_python will be unavailable due to missing components.
BFShell: Command run_file will be unavailable due to missing components.
Error parsing XML: File /home/pfamelis/projects/switch-bench/sw-p4-dpdk/install/share/cli/xml/bf_rt.xml

        ********************************************
        *      WARNING: Authorised Access Only     *
        ********************************************
    

bfshell> ? 
  exit  Exit this CLI session
  help  Display an overview of the CLI syntax
  quit  Exit this CLI session

(When it says $SDE, it is the full path, I just changed it for clarity)

As you can see it seems not to load properly and when hitting question mark, it shows only exit, help, and quit as possible commands.
Do you have any possible idea what could be wrong?
I suspect some python path errors, but I have set the PYTHONPATH as instructed.

P.S.: Also on a related note, in the readme it mentions the simple_l2l3_lpm.conf, while p4c does not create such a file. After various tries, I believe that the simple_l2l3_lpm.conf is the bf-rt.json.

bfshell cause bf_switch to segfault

Hi,
When I use bfshell to send scripts to a running bf_switchd program.
As soon as the bfshell connection is closed, it will cause segfault on the bf_switchd program.
Here is the gdb output:

...
bf_rt cli exited normally.

Thread 13 "bf_cli_smain" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffdeffd640 (LWP 260075)]
0x00007ffff7e32975 in __GI___libc_free (mem=0xffffffffd40017b0) at malloc.c:3288
3288    malloc.c: No such file or directory.
(gdb) bt
#0  0x00007ffff7e32975 in __GI___libc_free (mem=0xffffffffd40017b0) at malloc.c:3288
#1  0x00007ffff7d996a5 in bf_sys_free (ptr=<optimized out>)
    at /P4_SDE/p4-dpdk-target/third-party/target-syslibs/src/bf_sal/linux_usr/bf_sys_mem.c:40
#2  0x00007ffff78a97e4 in clish_shell_fini (this=0x7fffd4000b60)
    at /P4_SDE/p4-dpdk-target/third-party/target-utils/third-party/klish/clish/shell/shell_new.c:195
#3  clish_shell_delete (this=this@entry=0x7fffd4000b60)
    at /P4_SDE/p4-dpdk-target/third-party/target-utils/third-party/klish/clish/shell/shell_new.c:218
#4  0x00007ffff78af808 in cli_thread_start (args=<optimized out>)
    at /P4_SDE/p4-dpdk-target/third-party/target-utils/third-party/klish/clish/shell/shell_thread.c:233
#5  0x00007ffff7f8ad80 in start_thread (arg=0x7fffdeffd640) at pthread_create.c:481
#6  0x00007ffff7ea676f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)

Shell input issues

@lzhzero and I have identified several shell input issues, that might relate to each other. So we decided to use this single issue to report them all at once.

Issue 1: shell corruption after bf_switchd exit

After exiting from a bf_switchd process back to a shell, when you type, there will be nothing shown. After typing ENTER, you shall see the characters typed were recorded. They were just not displayed.

A quick fix to this issue is to type reset followed by ENTER. This will reset shell states to bring it back to normal. But it would be nice not having to do this every time.

Issue 2: bfshell exit leads to bf_switchd segfault

In terminal 1, run bf_switchd. In terminal 2, run bfshell to execute some commands. Now quit bfshell in terminal 2, and you shall see bf_switchd getting a segfault error in terminal 1.

We are not sure if this is the expected behavior though.

Issue 3: messed up input in VS Code integrated terminal

In a VS Code integrated terminal, start bf_switchd, and follow by bfshell> tdi_python. Now start typing, and you shall notice random characters popping up.

This might be an issue on the VS Code side though. We are not sure.

P4runtime usage

Hello, I want to use p4runtime with the P4-DPDK backend.

currently, I compiled the backend with the flag '--enable-grpc',
I also run the bf_switchd with '--p4rt-server' flag
I see that the grpc server started:
'bfruntime gRPC server started on 0.0.0.0:50052'

is there any documentation on how to connect to the server from the p4runtime client, enable the pipeline and add entries?

can I use the p4runtime shell (https://github.com/p4lang/p4runtime-shell) as a client? How can I do that?
I tried it, but it failed to connect to the server.

Thanks.

Problem binding physical NIC in bf_switchd.

Hello,
Has anyone gone through the situation I'm going through to link a physical NIC to p4-dpdk-target .
I already have no idea what to do to successfully execute the binding.
The problem only occurs when using BFRT, I tested it with the DPDK examples and the card links normally.
The error that appears when trying to attach the physical card to bf_switchd is how it cannot read the ports.json file, but this only happens with physical cards.

Log syslog.
Oct 19 23:48:13 master-node dummy[942929]: EAL: Probe PCI driver: net_nfp_pf (19ee:4000) device: 0000:41:00.0 (socket 1)
Oct 19 23:48:13 master-node kernel: [34440.907257] igb_uio 0000:41:00.0: uio device registered with irq 87
MInha dรบvida ao efetuar os teste descritos รฉ:Oct 19 23:58:07 master-node dummy[957429]: 2022-10-19 23:58:07.648645 ERROR (BF_PORT:(file=null):(func=null):0) - port_config_json_parse_ports:507: Failed to add port LINK0
Oct 19 23:59:07 master-node dummy[957429]: 2022-10-19 23:58:07.648667 ERROR (BF_PORT:(file=null):(func=null):0) - parse_port_config_json:586: Failed to parse ports from Port Config JSON
Oct 19 23:59:07 master-node dummy[957429]: 2022-10-19 23:58:07.648705 ERROR (BF_PORT:(file=null):(func=null):0) - port_mgr_config_import : Error in parsing the port config file
Oct 19 23:59:07 master-node dummy[957429]: 2022-10-19 23:58:07.648744 ERROR (BF_DVM:(file=null):(func=null):0) - Device add handling failed for dev 0, sts No system resources (2), Client port_mgr
Oct 19 23:59:07 master-node dummy[957429]: 2022-10-19 23:58:07.648764 ERROR (BF_DVM:(file=null):(func=null):0) - Device add failed for dev 0, sts No system resources (2)

Log bfshell
LINK CREATE: Dev:0000:41:00.0 Args: dev_hotplug_enabled: 0
2022-10-19 09:54:00.196125 BF_LLD ERROR - Creation of Link Port LINK0 failed

2022-10-19 09:54:00.196285 BF_PORT ERROR - port_config_json_parse_ports:507: Failed to add port LINK0
2022-10-19 09:54:00.196322 BF_PORT ERROR - parse_port_config_json:586: Failed to parse ports from Port Config JSON
2022-10-19 09:54:00.196377 BF_PORT ERROR - port_mgr_config_import : Error in parsing the port config file
2022-10-19 09:54:00.196436 BF_DVM ERROR - Device add handling failed for dev 0, sts No system resources (2), Client port_mgr
2022-10-19 09:54:00.196466 BF_DVM ERROR - Device add failed for dev 0, sts No system resources (2)
ERROR: bf_device_add failed(2) for dev_id 0
ERROR: bf_switchd_device_add failed(2) for dev_id 0

bf_switchd: initialized 0 devices
bf_switchd_switchsai_lib_init
starting SAI
bf_switchd: spawning cli server thread
bf_switchd: spawning driver shell
bf_switchd: server started - listening on port 9999

    ********************************************
    *      WARNING: Authorised Access Only     *
    ********************************************

Every help is welcome,
Thanks.

Error when pushing a PNA pipeline

When loading a PNA program (like the PNA simple_lpm) with bf_pal_device_add(dev_id, &device_profile) we are seeing this error message:

2022-11-09 21:55:56.747712 BF_PIPE ERROR - Error -17 at line 23: Struct registration error.
.
2022-11-09 21:55:56.747830 BF_PIPE ERROR - Failed to Build Pipeline pipe
2022-11-09 21:55:56.747854 BF_DVM ERROR - Device add handling failed for dev 0, sts Unexpected error (15), Client pipe-mgr
2022-11-09 21:55:56.747871 BF_DVM ERROR - Device add failed for dev 0, sts Unexpected error (15)

The same build does work with PSA version of the pipeline.

Could be related to #16

Trying to run a simple p4 program

Hello, we're trying to run a simple main.p4 using p4-dpdk-target:
main.p4:

#include <core.p4>
#include <psa.p4>

struct empty_metadata_t {}
struct headers_t {}

struct local_metadata_t {}

parser packet_parser(
  packet_in packet,
  out headers_t headers,
  inout local_metadata_t local_metadata,
  in psa_ingress_parser_input_metadata_t standard_metadata,
  in empty_metadata_t resub_meta,
  in empty_metadata_t recirc_meta) {
    state start {
        transition accept;
    }
}

control packet_deparser(
    packet_out packet,
    out empty_metadata_t clone_i2e_meta,
    out empty_metadata_t resubmit_meta,
    out empty_metadata_t normal_meta,
    inout headers_t headers,
    in local_metadata_t local_metadata,
    in psa_ingress_output_metadata_t istd) {
    apply {
    }
}

control ingress(
  inout headers_t headers,
  inout local_metadata_t local_metadata1,
  in psa_ingress_input_metadata_t piim,
  inout psa_ingress_output_metadata_t piom) {
    apply {
      if (piim.ingress_port == (PortId_t)0) {
        piom.egress_port = (PortId_t)1;
      } else {
        piom.egress_port = (PortId_t)0;
      }
    }
}

control egress(inout headers_t headers, inout local_metadata_t local_metadata, in psa_egress_input_metadata_t istd, inout psa_egress_output_metadata_t ostd) {
    apply {
    }
}

parser egress_parser(packet_in buffer, out headers_t headers, inout local_metadata_t local_metadata, in psa_egress_parser_input_metadata_t istd, in empty_metadata_t normal_meta, in empty_metadata_t clone_i2e_meta, in empty_metadata_t clone_e2e_meta) {
    state start {
        transition accept;
    }
}

control egress_deparser(packet_out packet, out empty_metadata_t clone_e2e_meta, out empty_metadata_t recirculate_meta, inout headers_t headers, in local_metadata_t local_metadata, in psa_egress_output_metadata_t istd, in psa_egress_deparser_input_metadata_t edstd) {
    apply {
    }
}

IngressPipeline(packet_parser(), ingress(), packet_deparser()) ip;

EgressPipeline(egress_parser(), egress(), egress_deparser()) ep;

PSA_Switch(ip, PacketReplicationEngine(), ep, BufferingQueueingEngine()) main;

We downloaded and installed p4c.

we installed p4-dpdk-target using @jafingerhut 's script and @Yi-Tseng 's script, (apparently one of the scripts did not work properly, but we do not know why.

we compile with:

p4c-dpdk main.p4 \
    -o main.spec \
    --arch psa \
    --bf-rt-schema main.bfrt.json \
    --context context.json \
    --p4runtime-files main.pb.txt \
    --p4runtime-format text

we try to run bf_switched with the following:

# Basic environment variables we need
export SDE=$HOME/sde
export SDE_INSTALL=$SDE/install
export LD_LIBRARY_PATH=$SDE_INSTALL/lib:$SDE_INSTALL/lib64:$SDE_INSTALL/lib/x86_64-linux-gnu/:/usr/local/lib64:/usr/local/lib
# We need to put SDE executable path before the system one since the bfrt_python
# needs to use Python libraries from it and use the Python executable from SDE
export PATH=$SDE_INSTALL/bin:$PATH
hash -r

# For security reason, the PATH and LD_LIBRARY_PATH won't pass to root user even if we use "sudo -E"
# We must pass them in sudo to make sure it is correct.
sudo -E PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $SDE_INSTALL/bin/bf_switchd --install-dir $SDE_INSTALL --conf switch_config.json

but here we encounter an error :

bf_switchd: system services initialized
bf_switchd: loading conf_file switch_config.json...
bf_switchd: processing device configuration...
Configuration for dev_id 0
  Family        : dpdk
bf_switchd: processing P4 configuration...
P4 profile for dev_id 0
P4 EAL args dummy -n 2 -c 0xC
Debug CLI disable
num mempool objs 1
  mempool_name: MEMPOOL0
  buffer_size:  2304
  pool_size:    1024
  cache_size:   256
  numa_node:    0
num fixed functions 0
num P4 programs 1
  p4_name: l1switch
  bfrt_config: /home/p4project/p4-dpdk-target-notes/l1switch/tdi.json
  port_config: /home/p4project/p4-dpdk-target-notes/l1switch/ports.json
  p4_pipeline_name: pipe
  core_id: 1
  numa_node: 0
    context: /home/p4project/p4-dpdk-target-notes/l1switch/context.json
    config: /home/p4project/p4-dpdk-target-notes/l1switch/main.spec
  Pipes in scope [0 1 2 3 ]
  Timer values []
  Mirror Config
    n_slots: 4
    n_sessions: 64
    fast_clone: 0
  diag:
  accton diag:
  non_default_port_ppgs: 0
  SAI default initialize: 1
Operational mode set to DPDK MODEL
Initialized the device types using PAL handler registration
STUB:port_mgr_platform_init DPDK
bf_switchd: drivers initialized
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: DPDK is running on a NUMA system, but is compiled without NUMA support.
EAL: This will have adverse consequences for performance and usability.
EAL: Please use --legacy-mem option, or recompile with NUMA support.
EAL: Device 0000:01:00.0 is not NUMA-aware
EAL: Device 0000:01:00.1 is not NUMA-aware
EAL: Device 0000:03:00.0 is not NUMA-aware
EAL: Device 0000:07:00.0 is not NUMA-aware
TELEMETRY: No legacy callbacks, legacy socket not created
RING: Cannot reserve memory
Error: Thread initialization failed (-1)
2022-12-27 16:19:13.789453 BF_PORT ERROR - parse_port_config_json:539: Could not open configuration file: /home/p4project/p4-dpdk-target-notes/l1switch/ports.json.

2022-12-27 16:19:13.789498 BF_PORT ERROR - port_mgr_config_import : Error in parsing the port config file
2022-12-27 16:19:13.789549 BF_DVM ERROR - Device add handling failed for dev 0, sts No system resources (2), Client port_mgr
2022-12-27 16:19:13.789594 BF_DVM ERROR - Device add failed for dev 0, sts No system resources (2)
ERROR: bf_device_add failed(2) for dev_id 0
ERROR: bf_switchd_device_add failed(2) for dev_id 0

bf_switchd: initialized 0 devices
bf_switchd_switchsai_lib_init
 starting SAI
bf_switchd: spawning cli server thread
bf_switchd: spawning driver shell
bf_switchd: server started - listening on port 9999

        ********************************************
        *      WARNING: Authorised Access Only     *
        ********************************************


bfshell>

We tried looking up documentation for ports.json or switch_confiog.json, but alas we could not find any.

any help regarding how should the ports.json and switch_config.json should contain would be helpful.
please know that we are still learning and don't completely understand things that may be obvious to the more experienced.

thank you

Installation problems

Hello, we're trying to build and install but we're experiencing a lot of errors,

we'll be more than happy to get a list of repositories we need to clone and build and prerequisites we need to install.
If there is a complete setup.sh somewhere it would be great.
thank you for your time :)

Consider using fewer parallel processes during build

The -j48 in the ninja command line requires at least about 8 GB of RAM on the system in order to avoid random process crashes due to the Linux kernel OOM process killer:

cd build && ninja -j48 && ninja install)

https://www.kernel.org/doc/gorman/html/understand/understand016.html

It also does not help builds go any faster on a system with relatively few CPU cores, e.g. 4 to 8.

Consider either hard-coding a smaller number (perhaps -j8 ?) or perhaps make it configurable via an environment variable.

DPDK and TAP interface MTU differ, leading to packet truncation

The DPDK port and Linux TAP interface are configured with the same MTU value when calling lld_dpdk_pipeline_tap_port_add.

This causes a truncation problem at runtime, that stems from fact that this MTU is understood differently by DPDK and Linux:

Linux:

Each network device has a Maximum Transfer Unit. The MTU does not
include any link layer protocol overhead. Upper layer protocols must
not pass a socket buffer (skb) to a device to transmit with more data
than the mtu. The MTU does not include link layer header overhead, so
for example on Ethernet if the standard MTU is 1500 bytes used, the
actual skb will contain up to 1514 bytes because of the Ethernet
header. Devices should allow for the 4 byte VLAN header as well.

DPDK:

DPDK treats the MTU as the memory buffer (mbuf) size including ALL headers. This includes Ethernet (14) and the FCS CRC checksum (4).

In my understanding, this means the TAP interface will accept frames that too large for DPDK and thus will get truncated.

Suggestion

I think the TAP interface configuration should happen with MTU-18 to unify behavior:

ifr.ifr_mtu = port_attrib->tap.mtu;

Note: 14 bytes seems to work too, I think we're just cutting off the FCS then. Maybe some DPDK expert can chime in here?

This lower TAP MTU should cause Linux to send an ICMP error back to the sender, hopefully triggering Path MTU discovery there.

Workaround

Until this is fixed in the SDE, there is a hacky workaround in the interim:
If you can ensure that there is never a Ethernet frame larger than the DPDK MTU, this issue will not occur. Either

  • Set the DPDK MTU to your Linux interface MTU + 18
    or
  • Decrease the Linux interface MTU by 18: ip l set mtu 1400 dev h1-eth0

By Linux interface I mean regular, other IP hosts in the network, NOT the DPDK TAP interfaces!

Example pcap with the truncation issue

20:17:55.196347 a6:c0:aa:27:c8:2b > 00:00:00:00:03:14, ethertype IPv4 (0x0800), length 1500: truncated-ip - 14 bytes missing! (tos 0x0, ttl 64, id 1, offset 0, flags [none], proto UDP (17), length 1500)
    10.0.0.1.53 > 10.0.0.2.53: 12336 op6% [12336a] [12336q] [12336n] [12336au][|domain]

The bfrt_python cli of install/bin/bfswitchd

We desire to run the bfrt_python CLI through install/bin/bfswitchd. However, the error occurs:
Traceback (most recent call last):
File "/sde/install/lib/python3.8/bfrtcli.py", line 17, in
from traitlets.config import Config
File "/sde/install/lib/python3.8/site-packages/traitlets/config/init.py", line 6, in
from .application import *
File "/sde/install/lib/python3.8/site-packages/traitlets/config/application.py", line 18, in
from traitlets.config.configurable import Configurable, SingletonConfigurable
File "/sde/install/lib/python3.8/site-packages/traitlets/config/configurable.py", line 11, in
from .loader import Config, LazyConfigValue, DeferredConfig, _is_section_key
File "/sde/install/lib/python3.8/site-packages/traitlets/config/loader.py", line 16, in
from ipython_genutils.path import filefind
File "/sde/install/lib/python3.8/site-packages/ipython_genutils/path.py", line 13, in
import random
File "/sde/install/lib/python3.8/random.py", line 40, in
from cgi import print_form
File "/sde/install/lib/python3.8/cgi.py", line 39, in
from email.parser import FeedParser
File "/sde/install/lib/python3.8/email/parser.py", line 12, in
from email.feedparser import FeedParser, BytesFeedParser
File "/sde/install/lib/python3.8/email/feedparser.py", line 27, in
from email._policybase import compat32
File "/sde/install/lib/python3.8/email/_policybase.py", line 7, in
from email import header
File "/sde/install/lib/python3.8/email/header.py", line 14, in
import binascii
ModuleNotFoundError: No module named 'binascii'
Failed to load bfrtcli python library
bfrt_cli_cmd:201 could not initialize bf_rt for the cli. err: 1

We print the sys.path by hacking the bfrtcli.py and shows that:
bfshell> bfrt_python
['/sde/install/lib/python38.zip', '/sde/install/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/sde/install/lib/python3.8/site-packages']

We use the interactive interface of python3.8 and import the module named 'binascii'. It successes. At the interactive interface, the sys.path is shown as followed:

sys.path
['', '/sde/install/lib/python38.zip', '/sde/install/lib/python3.8', '/sde/install/lib/python3.8/lib-dynload', '/sde/install/lib/python3.8/site-packages']

Issue with running bfswitchd

Hi, after changing the .json to:

Ports:
`{
"ports": [
{
"dev_port": 3,
"port_name": "ens3",
"mempool_name": "MEMPOOL0",
"pipe_name": "pipe",
"pipe_in":"pipe",
"pipe_out":"pipe",
"port_dir": "default",
"port_in_id": 0,
"port_out_id": 0,
"port_type": "tap",
"tap_port_attributes": {
"mtu": 1500
}
},
{
"dev_port": 5,
"port_name": "ens5",
"mempool_name": "MEMPOOL0",
"pipe_name": "pipe",
"port_dir": "default",
"port_in_id": 1,
"pipe_in":"pipe",
"pipe_out":"pipe",
"port_out_id": 1,
"port_type": "tap",
"tap_port_attributes": {
"mtu": 1500
}
}

]

}`

switch config:

{ "chip_list": [ { "id": "asic-0", "chip_family": "dpdk", "instance": 0 } ], "instance": 0, "p4_devices": [ { "device-id": 0, "eal-args": "-n 1 -c 0x1", "mempools": [ { "name": "MEMPOOL0", "buffer_size": 2304, "pool_size": 1024, "cache_size": 256, "numa_node": 0 } ], "p4_programs": [ { "program-name": "l1switch", "bfrt-config": "/home/p4project/l1switch/tdi.json", "port-config": "/home/p4project/l1switch/ports.json", "p4_pipelines": [ { "p4_pipeline_name": "pipe", "core_id": 1, "numa_node": 0, "context": "/home/p4project/l1switch/context.json", "config": "/home/p4project/l1switch/main.spec", "pipe_scope": [ 0, 1, 2, 3 ], "path": "/home/p4project/l1switch/" } ] } ] } ] }

build.sh :

`#!/bin/bash
p4c-dpdk main.p4
-o main.spec
--arch psa
--bf-rt-schema main.bfrt.json
--context context.json
--p4runtime-files main.pb.txt
--p4runtime-format text
--tdi tdi.json

Basic environment variables we need

export SDE=$HOME/sde
export SDE_INSTALL=$SDE/install
export LD_LIBRARY_PATH=$SDE_INSTALL/lib:$SDE_INSTALL/lib64:$SDE_INSTALL/lib/x86_64-linux-gnu/:/usr/local/lib64:/usr/local/lib

We need to put SDE executable path before the system one since the bfrt_python

needs to use Python libraries from it and use the Python executable from SDE

export PATH=$SDE_INSTALL/bin:$PATH
hash -r

For security reason, the PATH and LD_LIBRARY_PATH won't pass to root user even if we use "sudo -E"

We must pass them in sudo to make sure it is correct.

sudo -E PATH=$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $SDE_INSTALL/bin/bf_switchd --install-dir $SDE_INSTALL --conf switch_config.json
`

but sill see issues:

`p4project@p4project-S5520HC:~/l1switch$ ./build.sh
Install dir: /home/p4project/sde/install (0x564c18724c20)
bf_switchd: system services initialized
bf_switchd: loading conf_file switch_config.json...
bf_switchd: processing device configuration...
Configuration for dev_id 0
Family : dpdk
bf_switchd: processing P4 configuration...
P4 profile for dev_id 0
P4 EAL args -n 1 -c 0x1
Debug CLI disable
num mempool objs 1
mempool_name: MEMPOOL0
buffer_size: 2304
pool_size: 1024
cache_size: 256
numa_node: 0
num fixed functions 0
num P4 programs 1
p4_name: l1switch
bfrt_config: /home/p4project/l1switch/tdi.json
port_config: /home/p4project/l1switch/ports.json
p4_pipeline_name: pipe
core_id: 1
numa_node: 0
context: /home/p4project/l1switch/context.json
config: /home/p4project/l1switch/main.spec
Pipes in scope [0 1 2 3 ]
Timer values []
Mirror Config
n_slots: 4
n_sessions: 64
fast_clone: 0
diag:
accton diag:
non_default_port_ppgs: 0
SAI default initialize: 1
Operational mode set to DPDK MODEL
Initialized the device types using PAL handler registration
STUB:port_mgr_platform_init DPDK
bf_switchd: drivers initialized
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 2
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: VFIO support initialized
EAL: DPDK is running on a NUMA system, but is compiled without NUMA support.
EAL: This will have adverse consequences for performance and usability.
EAL: Please use --legacy-mem option, or recompile with NUMA support.
EAL: Device 0000:01:00.0 is not NUMA-aware
EAL: Device 0000:01:00.1 is not NUMA-aware
EAL: Device 0000:03:00.0 is not NUMA-aware
EAL: Using IOMMU type 8 (No-IOMMU)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:03:00.0 (socket -1)
EAL: Device 0000:07:00.0 is not NUMA-aware
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:07:00.0 (socket -1)
TELEMETRY: No legacy callbacks, legacy socket not created

bf_switchd: dev_id 0 initialized

bf_switchd: initialized 1 devices
bf_switchd_switchsai_lib_init
starting SAI
bf_switchd: spawning cli server thread
bf_switchd: spawning driver shell
bf_switchd: server started - listening on port 9999

    ********************************************
    *      WARNING: Authorised Access Only     *
    ********************************************

bfshell> bfrt_python
cwd : /home/p4project/l1switch

We've found 1 p4 programs for device 0:
l1switch
Creating tree for dev 0 and program l1switch

Devices found : [0]
/home/p4project/sde/install/lib/python3.10/site-packages/IPython/core/history.py:226: UserWarning: IPython History requires SQLite, your history will not be saved
warn("IPython History requires SQLite, your history will not be saved")
Python 3.10.8 (aaaf517424:aaaf517424, Dec 27 2022, 12:59:27) [GCC 11.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: bfrt
------> bfrt()
Available symbols:
dump - Command
enable - Command
info - Command
l1switch - Node
port - Node

bfrt> l1switch.enable
----> l1switch.enable()
2022-12-28 18:27:04.276807 BF_PIPE ERROR - error :thread pipeline enable
2022-12-28 18:27:04.276956 BF_PIPE ERROR - Failed to Build Pipeline pipe
2022-12-28 18:27:04.277010 BF_BFRT ERROR - bf_rt_enable_pipeline:120 failed
`

What should we write in switch_config.json and ports.json?
We would appreciate a link to a full tutorial \ guide on how to write these .json files.

Thank you

TDI Runtime Client fails to initialize.

Environment

OS

NAME="Rocky Linux"
VERSION="8.8 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.8 (Green Obsidian)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:8:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2029-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-8"
ROCKY_SUPPORT_PRODUCT_VERSION="8.8"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.8"

Kernel

Linux mev-acc-rl 5.15.0_acc_g0fdad426 #1 SMP Wed May 17 22:54:14 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

Problem

I am trying to run the TDI Runtime Client on a Rocky Linux. I'm using the following configuration file for the P4 program

{
    "chip_list": [
    {
            "id": "asic-0",
            "chip_family": "mev",
            "instance": 0,
             "pcie_sysfs_prefix": "/sys/devices/pci0000:00/0000:00:01.6",
            "pcie_bdf": "0000:00:01.6",
         "pcie_int_mode": 1,
            "iommu_grp_num": 5,
           "sds_fw_path": "share/tofino_sds_fw/avago/firmware"
    }
    ],
    "instance": 0,
    "p4_devices": [
        {
            "device-id": 0,
            "fixed_functions" : [],
            "eal-args": "--lcores=1-2 -a 00:01.6,vport=[0-1] -- -i --rxq=1 --txq=1",
            "p4_programs": [
                {
                    "program-name": "p4_e2100_default_1_0_1_2",
                    "bfrt-config": "/root/runtime_files/e2100-default-1.0.1.2.bfrt.json",
                    "p4_pipelines": [
                        {
                            "p4_pipeline_name": "main",
                            "context": "/root/runtime_files/e2100-default-1.0.1.2.context.json",
                            "config": "/root/runtime_files/e2100-default-1.0.1.2.pkg",
                            "pipe_scope": [
                                0,
                                1,
                                2,
                                3
                            ],
                            "path": "/root/runtime_files"
                        }
                    ]
                }
            ]
        }
    ]
}

After, specifying the configuration file, I run the following command to start bf_switchd

(test_venv) [root@mev-acc-rl ~]# /opt/p4/p4sde/bin/bf_switchd \
>   --background \
>   --install-dir /opt/p4/p4sde \
>   --conf-file /root/runtime_files/mev_sample.conf \
>   --init-mode cold \
>   --status-port 7777 \
>   --skip-hld mkt
Skip-hld-mgrs list is mkt
Unknown skip-hld option m
Unknown skip-hld option k
Unknown skip-hld option t
Install dir: /opt/p4/p4sde (0x20f75f30)
bf_switchd: system services initialized
bf_switchd: loading conf_file /root/runtime_files/mev_sample.conf...
bf_switchd: processing device configuration...
  pci_bdf       : 0000:00:01.6
  iommu_grp_num : 5
  host_id       : 0
  pf_num        : 0
Configuration for dev_id 0
  Family        : mev
bf_switchd: processing P4 configuration...
disable-cache : 0
batch_size    : 1
Num of Cfgqs  : 1
P4 profile for dev_id 0
P4 EAL args: --lcores=1-2 -a 00:01.6,vport=[0-1] -- -i --rxq=1 --txq=1
Debug CLI disable
num mempool objs 0
num fixed functions 0
num P4 programs 1
  p4_name: p4_e2100_default_1_0_1_2
  bfrt_config: /root/runtime_files/e2100-default-1.0.1.2.bfrt.json
  port_config:
  p4_pipeline_name: main
  core_id: 0
  numa_node: 0
    context: /root/runtime_files/e2100-default-1.0.1.2.context.json
    entries:
    config: /root/runtime_files/e2100-default-1.0.1.2.pkg
  Pipes in scope [0 1 2 3 ]
  Timer values []
  Mirror Config
    n_slots: 0
    n_sessions: 0
    fast_clone: 0
  diag:
  accton diag:
  non_default_port_ppgs: 0
  SAI default initialize: 1
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 1
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_cpfl (8086:1453) device: 0000:00:01.6 (socket -1)
cpfl_tdi_init(): tdi module is not initialized
TELEMETRY: No legacy callbacks, legacy socket not created
bf_switchd: drivers initialized
2023-07-10 16:39:48.663411 BF_LLD ERROR - Open failed! Err: 2 (No such file or directory)
2023-07-10 16:39:48.663441 BF_LLD ERROR - LLD FXP mem map init failed

Initialized lld_cpchnl control path
2023-07-10 16:39:48.666407 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666443 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666458 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666474 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666488 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666500 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666515 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666529 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666542 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666553 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666565 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666581 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666593 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666605 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666617 BF_PIPE ERROR - Invalid action:selector
2023-07-10 16:39:48.666628 BF_PIPE ERROR - Invalid action:selector
1 config queues are assigned. Start Qid 385

bf_switchd: dev_id 0 initialized
cpfl_dev_configure(): RSS is not supported.
cpfl_set_rx_function(): Using Split Scalar Rx (port 0).
cpfl_set_tx_function(): Using Split Scalar Tx (port 0).
2023-07-10 16:39:49.950280 BF_PORT ERROR - vPort Info not found. Exiting vport_mgr_set_vport_state
Port 0 MAC: 00 1a 00 01 03 20
cpfl_dev_configure(): RSS is not supported.
cpfl_set_rx_function(): Using Split Scalar Rx (port 1).
cpfl_set_tx_function(): Using Split Scalar Tx (port 1).
2023-07-10 16:39:51.050407 BF_PORT ERROR - vPort Info not found. Exiting vport_mgr_set_vport_state
Port 1 MAC: 00 1b 00 02 03 20

bf_switchd: initialized 1 devices
bf_switchd: spawning cli server thread
bf_switchd: running in background; driver shell is disabled
bf_switchd: server started - listening on port 9999

After starting bf_switch in the background, I tried to run the tdiRtCli.py script, but it failed with the following error:

(test_venv) [root@mev-acc-rl python3.10]# export SDE_INSTALL=/opt/p4/p4sde
(test_venv) [root@mev-acc-rl python3.10]# export LD_LIBRARY_PATH=$SDE_INSTALL/lib
(test_venv) [root@mev-acc-rl python3.10]# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SDE_INSTALL/lib64
(test_venv) [root@mev-acc-rl python3.10]# python tdiRtCli.py
cwd : /opt/p4/p4sde/lib/python3.10

We've found -1 p4 programs for device 0:
Traceback (most recent call last):
  File "/opt/p4/p4sde/lib/python3.10/tdiRtCli.py", line 628, in <module>
    main()
  File "/opt/p4/p4sde/lib/python3.10/tdiRtCli.py", line 625, in main
    start_tdi_rt(0, 1, os.getcwd(), [0])
  File "/opt/p4/p4sde/lib/python3.10/tdiRtCli.py", line 622, in start_tdi_rt
    return tdi_rt_cli.start_tdi(in_fd, out_fd, dev_id_list, udf, interactive)
  File "/opt/p4/p4sde/lib/python3.10/tdicli.py", line 2181, in start_tdi
    sts = self.load_tdi(dev_id_list)
  File "/opt/p4/p4sde/lib/python3.10/tdicli.py", line 2255, in load_tdi
    sts = self.populate_tdi(dev_id_list)
  File "/opt/p4/p4sde/lib/python3.10/tdicli.py", line 2281, in populate_tdi
    cintf = self.cIntf_cls(dev_id, TdiTable, TdiInfo)
  File "/opt/p4/p4sde/lib/python3.10/tdiRtCli.py", line 411, in __init__
    super().__init__(dev_id, TdiRtTable, TdiInfo, self._driver_path)
  File "/opt/p4/p4sde/lib/python3.10/tdicli.py", line 70, in __init__
    sts = self._init_programs()
  File "/opt/p4/p4sde/lib/python3.10/tdicli.py", line 181, in _init_programs
    array_type = c_char_p * num_names.value
ValueError: Array length must be >= 0, not -1

Any help would be appreciated. Thanks!

Link ports Iperf3 TCP disconnection

Hello, i am rather new to raising git issue, please tell me if i should modify anything to make this post better for you.

About a week ago i tried out the project with Yi-Tseng's simple l1switch p4 script on a VEP4600.
I have been able to get it to pass iperf3 UDP traffic through very close to line rate (1Gbs and 10Gbs).
But with iperf3 TCP, for example, on sender's side:

Capture

After which the dpdk switch seems unable to continue passing traffic through, as the two connected hosts can't even ping each other anymore. This happens with scp file transfer as well.
I tried capturing packet at each end points. At the sender's side:
capsend

And at the receiver's side:
caprec

Is the point when the dpdk switch link is broken. Seems like the sender was sending way FASTER than the receiver could respond. At last sender resent the last segment that had been ACK by receiver, and then the link went down.

At this point i've tried all sorts of MTU combinations, reduced the cwnd windows, but the problem persists

What could be going on here? Is this something related to dpdk's mbuf???

Any input is very welcome, thank you in advance

TdiInfo:150 Error creating Table and parseContextJson:184 context Table object not found for

Hello again,

I am trying to run the example that is given. I have successfully compiled it according to the instructions and created a conf-file following the examples given. However, after running the switch I get the following errors:

2022-07-27 20:48:11.005078 BF_BFRT ERROR - TdiInfo:150 Error creating Table:$PORT_FP_IDX_INFO
2022-07-27 20:48:11.005909 BF_BFRT ERROR - TdiInfo:150 Error creating Table:$PORT_HDL_INFO
2022-07-27 20:48:11.006688 BF_BFRT ERROR - TdiInfo:150 Error creating Table:$PORT_STR_INFO
2022-07-27 20:48:11.007621 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "$PORT"
2022-07-27 20:48:11.008474 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "$PORT_STAT"
2022-07-27 20:48:11.009262 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "ip.ingress.ipv4_host"
2022-07-27 20:48:11.010026 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "ip.ingress.ipv4_lpm"
2022-07-27 20:48:11.010804 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "ip.ingress.l2_fwd"
2022-07-27 20:48:11.011562 BF_BFRT ERROR - parseContextJson:184 context Table object not found for "ip.ingress.mymac"

Also when trying to enable the p4 program I get the following

------> bfrt.simple_l2l3_lpm.enable()
2022-07-27 20:51:16.399251 BF_PIPE ERROR - Number of ports added to DPDK Pipeline pipe in input direction should be a power of 2
2022-07-27 20:51:16.400320 BF_PIPE ERROR - Failed to Build Pipeline pipe
2022-07-27 20:51:16.401184 BF_BFRT ERROR - bf_rt_enable_pipeline:120 failed

I am not sure what exactly is the problem and if I am missing some file, or maybe change something?

Does this DPDK target works for PNA programs?

p4c-dpdk can compile PNA programs into spec files. When I try to load it into the bf_switchd target, and run bfrt.<program_name>.enable, I get "Invalid struct field size." error.
I think this is related to some of the PNA metadata fields are 3 bits like bit<3> pna_pre_input_metadata_pass

However, when I hack the .spec file to make them bit<8>, I got "Pipeline build error."

Could you supply some sample programs that can load and run on this DPDK target? Maybe the l3_simple_demo example that is mentioned in the tools folder.

Random failure for so file generation to load to dpdk pipeline

In src/pipe_mgr/shared/dal/dpdk/dal_init.c

	snprintf(buffer, sizeof(buffer), "gcc -shared %s -o %s",
		 o_filepath, so_filepath);
	LOG_TRACE("Running command: %s\n", buffer);
           		status = system(buffer);

status returns error, for which a work around is in place and needs to be fixed properly

Possible enhancement to bf_rt_table_attributes_impl.hpp

After adding:

"#include <array>" to `src/bf_rt/bf_rt_common/bf_rt_table_attributes_impl.hpp`

the package now builds without issue. Can you maybe try this to see if it will fix compilation issues for other people too?

@jamescchoi I see you on the commit log for this source file. This comment is being passed on by me from someone else wanting to use this code commercially.

How to embed p4-dpdk-target with real port / iface?

Hello,
I'm wondering how to integrate p4-dpdk-target with real NIC / port? When using standard dpdk, I need to bind the port to pci-vfio and then configure the EAL.
How about p4-dpdk-target? I think the example in the repo implements a topology with virtual iface. I now have trouble with real NIC. Thanks a lot for your help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.