Comments (20)
Does it work if you use core 0 in your mask/list?
N_QUEUES should be set in FromDPDKDevice I think. The intelligence of FastClick is all about finding the right amount in the output by itself. But you may need to hint the input.
Can you show me the full log? What's the driver?
from fastclick.
Does it work if you use core 0 in your mask/list?
Same error
While initializing 'DPDK0_OUT :: ToDPDKDevice':
So I guess it is about ToDPDK. I get the following if I put N_QUEUES 1 into FromDPDKDevice
click: ../include/click/vector.hh:291: T& Vector::operator [with T = int; Vector::size_type = int]: Assertion `(unsigned) i < (unsigned) vm.n_' failed.
Aborted (core dumped)
Full output. Using virtio_user using UIO driver
echo $EAL_PARAMS
-l 0-7 --master-lcore 5 -n 1 -m 1024 --file-prefix=analyser --no-pci --vdev=virtio_user0,path=/var/run/openvswitch/SRC,mac=00:00:92:00:00:02 --vdev=virtio_user1,path=/var/run/openvswitch/SNK,mac=00:00:92:00:00:03
click --dpdk $EAL_PARAMS -vv -- analyzerbkp.click
EAL: Detected 20 lcore(s)
EAL: RTE Version: 'DPDK 17.11.0'
EAL: RTE Version: 'DPDK 17.11.0'
EAL: Probing VFIO support...
Initializing DPDK
analyzerbkp.click:5: While initializing 'DPDK0_OUT :: ToDPDKDevice':
Port 0 can only use 1 RX queues, use MAXQUEUES to set the maximum number of queues or N_QUEUES to strictly define it.
Router could not be initialized!
from fastclick.
I guess virtio limits the number of queues to 1. Sadely the DPDK elements do not support MT on a single queue as of now. Can you set the number of queues for virtio_user maybe?
from fastclick.
Can you set the number of queues for virtio_user maybe?
Yes. Following is the new EAL params
export EAL_PARAMS="-l 5-10 --master-lcore 5 -n 1 \
-m 1024 --file-prefix=analyser --no-pci \
--vdev=virtio_user0,path=/var/run/openvswitch/SRC,mac=00:00:92:00:00:02,queues=4 \
--vdev=virtio_user1,path=/var/run/openvswitch/SNK,mac=00:00:92:00:00:03,queues=4"
Still the same issue :(
from fastclick.
Can you provide the GDB stack trace of the segfault?
from fastclick.
Can you provide the GDB stack trace of the segfault?
Will do asap. But before that, I guess I am onto something. According to https://software.intel.com/en-us/articles/configure-vhost-user-multiqueue-for-ovs-with-dpdk when setting up multiple queues, a parameter called vectors has to be configured at the node connecting to OVS. In the example vectors is configurable in QEMU settings of the VM.
Do you think this is a paramter that needs to be configured in the from/todevice elements, or provided by EAL?
from fastclick.
The segfault is because of the opened pull requests actually. But there may be a second problem with the queue...
from fastclick.
Was painful to get the core from within the container, hope it is the right stuff.
https://drive.google.com/open?id=1Y_J8wWJ0d9zbuiAnSP8ht7kog9sU7JB3
Following is from OVS side once click coredumps
2018-03-28T16:30:38.093Z|03549|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
2018-03-28T16:30:38.093Z|03550|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 1
2018-03-28T16:30:38.204Z|03551|dpdk|ERR|VHOST_CONFIG: recvmsg failed
2018-03-28T16:30:38.204Z|03552|dpdk|INFO|VHOST_CONFIG: vhost peer closed
This vectors parameter has to be configurable somewhere. It has to match OVS's setting, for example vectors has to be set as 6 at the connecting end if the queue count is 2.
from fastclick.
Can you try with the last rebase? I don't have the right setup to test with virtio_user...
from fastclick.
Can you try with the last rebase? I don't have the right setup to test with virtio_user...
Sorry I guess I am missing it, which rebase?
from fastclick.
from fastclick.
Same issue. I asked the following to the developer that released the multiqueue patch on virtio
Hello,
I am working on connecting VNFs in docker containers to OVS over virtio user using multi-queues and I found out that you have pushed the patch http://dpdk.org/ml/archives/dev/2016-May/038353.html for it.
In qemu for using virtio, the vectors parameter is configurable in addition to the queues in vdev settings. I was wondering if you are planning to have configurable vectors as an addition to queues in your patch.
and got the following response
Hi
We now only allow 1:1 vec to rx queue mapping which is hardcoded; each vec is connected with a callfd.
In qemu context, similarly, each queue would have a callfd, it’s just multiple callfds are translated to one interrupt vec in KVM.
For vector number, in QEMU’s case, we suggest 2N+2 vectors, N for tx queues, N for rx queues, 1 for config, and one for possible control vq.
So this makes me think that FromDPDK/ToDPDK element should have configurable vector size.
from fastclick.
Some more feedback from Intel
The design of virtio-user is to supersede the QEMU with a vdev layer embedded in the virtio pmd; if we go back a little bit, people also suggest to have a QEMU in qtest mode.
So basically, all parameters for virtio-net-device of QEMU shall be ported to this vdev (virtio-user) specific parameters.
And I’m not sure I understand “a configurable parameter in the EAL for vdev” correctly. IMO, there is no such parameter from EAL for a vdev in DPDK. The device-specific parameters always comes from devargs when we do dev_attach().
from fastclick.
I would prefer the stacktrace in text format, copy/paste the "where" output of GDB.
from fastclick.
So the setup is inside a KVM/QEMU VM which has two virtio ports that are attached to OVS, right?
Can you provide a full install step maybe ?
from fastclick.
Apologies for the late response, I got hold up with the paper side of things.
I am working on a full tutorial but not finished yet. But following is a summary.
The setup is on the hardware directly. No VM/QEMU involved
- Install OVS with DPDK support (link)
- Install Docker CE (link)
- Save the following in a file named Dockerfile
FROM ubuntu:latest
######################
#Initial Preparations#
######################
RUN echo "Initial Preparations ..."
RUN apt-get update && apt-get install -y \
build-essential automake git gcc tar wget python-pip libcap-ng-dev libnuma-dev \
gawk pciutils linux-headers-$(uname -a | awk '{print $3}') nano kmod apt-utils \
iproute2 zlib1g-dev
RUN pip install -U pip six
#################
#Installing DPDK#
#################
RUN echo "Installing DPDK ..."
ENV INSTALL_PATH "/usr/src"
ENV RTE_SDK "$INSTALL_PATH/dpdk-17.11"
ENV RTE_TARGET "x86_64-native-linuxapp-gcc"
ENV DPDK_BUILD "$RTE_SDK/$RTE_TARGET"
RUN cd $INSTALL_PATH; \
wget http://fast.dpdk.org/rel/dpdk-17.11.tar.xz; \
tar xf dpdk-17.11.tar.xz; \
rm dpdk-17.11.tar.xz; \
cd $RTE_SDK; \
make install -j4 T=$RTE_TARGET DESTDIR=install
######################
#Installing fastclick#
######################
RUN echo "Installing fastclick ..."
ENV CLICK_DIR "$INSTALL_PATH/fastclick"
ENV GIT_URL "https://github.com/kthkaya/fastclick"
RUN cd $INSTALL_PATH; \
git clone $GIT_URL; \
cd $CLICK_DIR; \
trframework/build_click.sh
######
#Done#
######
RUN echo "Done."
CMD ["/bin/bash"]
then in the folder Dockerfile exists, run the command
docker build -t fastclick .
- Instantiate the docker image named fastclick with the following command
docker run -ti --name="instancename" -v /dev/hugepages:/dev/hugepages -v /usr/local/var/run/openvswitch:/var/run/openvswitch --privileged fastclick:latest
- Press enter, now you are in the container. Navigate to /usr/src/fastclick
Now in the steps I provided #55; follow the ones until dpdkbridge.click on the server itself, and the rest of the steps in the container prompt.
from fastclick.
Ok thanks. I can more or less reproduce, except that it tells me that the device only supports 0 queues :p If I remove the check the device fails to initialize... So the error is somehow logical.
I'll look in more details at the virtio driver. Seems like a bug in DPDK, or an unexpected behaviour (maybe documented in the limbs of the DPDK documentation).
from fastclick.
FYI, I would sugget adding the following before installing DPDK :
sed -i 's/CONFIG_RTE_EAL_IGB_UIO=y/CONFIG_RTE_EAL_IGB_UIO=n/' config/common_linuxapp; \
sed -i 's/CONFIG_RTE_KNI_KMOD=y/CONFIG_RTE_KNI_KMOD=n/' config/common_linuxapp; \
Which will avoid dependency on the Linux headers, by not building KNI and igb_uio. WHen using virtio, they are not necessary. In my case I ran pure debian, and the headers are not available on Ubuntu for the same kernel.
from fastclick.
Thanks, will do!
from fastclick.
A bad merge some time ago removed part of the message, making it misleading. It's now maybe even too much bla bla. But it should be clear enough.
I think following the instructions, this will solve your problem.
The core of the problem is that FastClick will not automatically rely on locking in the ToDPDKDevice to use more threads than queues. Therfore you must pass the argument by yourself, in the ToDPDKDevice.
If it's not fixed after merging and following the message, we'll have at least a much better view of the problem...
from fastclick.
Related Issues (20)
- Create Element HOT 2
- Secondary DPDK with FastClick HOT 2
- Access useful_kcycles() HOT 3
- Input aggregation element HOT 1
- --with-netmap does not compile HOT 3
- FastClick can't identify DPDK PORT,whatever Integer or PCI address.
- port 0 Rx queue allocation failed: Cannot allocate memory HOT 7
- Illegal instruction (core dumped) when using ToDPDKDevice() HOT 3
- rte_mempool segmentation fault. HOT 8
- storeing data in packet HOT 2
- How to enable scattered mode in fastclick? HOT 7
- issue in make package element HOT 3
- How to use replayUnqueue or MultiReplayUnqueue to limit the tx rate? HOT 3
- Automatically update wiki doc with CI
- middleclick configuration HOT 1
- compile error about context HOT 1
- Allow loop in FromDump HOT 4
- Non-existent librairies when parsing llvm-config's output
- FromDPDKDevice(FLOW_RULES_FILE) only work with DPDK<=21.11 HOT 2
- About address of IIO LLC WAYS in Ice lake HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastclick.