Port 0 can only use 1 RX queues, use MAXQUEUES to set the maximum number of queues or N_QUEUES to strictly define it. about fastclick HOT 20 CLOSED

Does it work if you use core 0 in your mask/list?

N_QUEUES should be set in FromDPDKDevice I think. The intelligence of FastClick is all about finding the right amount in the output by itself. But you may need to hint the input.

Can you show me the full log? What's the driver?

from fastclick.

Does it work if you use core 0 in your mask/list?

Same error

While initializing 'DPDK0_OUT :: ToDPDKDevice':
So I guess it is about ToDPDK. I get the following if I put N_QUEUES 1 into FromDPDKDevice

click: ../include/click/vector.hh:291: T& Vector::operator [with T = int; Vector::size_type = int]: Assertion `(unsigned) i < (unsigned) vm.n_' failed.
Aborted (core dumped)

Full output. Using virtio_user using UIO driver

echo $EAL_PARAMS
-l 0-7 --master-lcore 5 -n 1 -m 1024 --file-prefix=analyser --no-pci --vdev=virtio_user0,path=/var/run/openvswitch/SRC,mac=00:00:92:00:00:02 --vdev=virtio_user1,path=/var/run/openvswitch/SNK,mac=00:00:92:00:00:03

click --dpdk $EAL_PARAMS -vv -- analyzerbkp.click
EAL: Detected 20 lcore(s)
EAL: RTE Version: 'DPDK 17.11.0'
EAL: RTE Version: 'DPDK 17.11.0'
EAL: Probing VFIO support...
Initializing DPDK
analyzerbkp.click:5: While initializing 'DPDK0_OUT :: ToDPDKDevice':
  Port 0 can only use 1 RX queues, use MAXQUEUES to set the maximum number of queues or N_QUEUES to strictly define it.
Router could not be initialized!

from fastclick.

I guess virtio limits the number of queues to 1. Sadely the DPDK elements do not support MT on a single queue as of now. Can you set the number of queues for virtio_user maybe?

from fastclick.

Can you set the number of queues for virtio_user maybe?

Yes. Following is the new EAL params

export EAL_PARAMS="-l 5-10 --master-lcore 5 -n 1 \
-m 1024 --file-prefix=analyser --no-pci \
--vdev=virtio_user0,path=/var/run/openvswitch/SRC,mac=00:00:92:00:00:02,queues=4 \
--vdev=virtio_user1,path=/var/run/openvswitch/SNK,mac=00:00:92:00:00:03,queues=4"

Still the same issue :(

from fastclick.

Can you provide the GDB stack trace of the segfault?

from fastclick.

Can you provide the GDB stack trace of the segfault?

Will do asap. But before that, I guess I am onto something. According to https://software.intel.com/en-us/articles/configure-vhost-user-multiqueue-for-ovs-with-dpdk when setting up multiple queues, a parameter called vectors has to be configured at the node connecting to OVS. In the example vectors is configurable in QEMU settings of the VM.

Do you think this is a paramter that needs to be configured in the from/todevice elements, or provided by EAL?

from fastclick.

The segfault is because of the opened pull requests actually. But there may be a second problem with the queue...

from fastclick.

Was painful to get the core from within the container, hope it is the right stuff.
https://drive.google.com/open?id=1Y_J8wWJ0d9zbuiAnSP8ht7kog9sU7JB3

Following is from OVS side once click coredumps

2018-03-28T16:30:38.093Z|03549|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE
2018-03-28T16:30:38.093Z|03550|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 1
2018-03-28T16:30:38.204Z|03551|dpdk|ERR|VHOST_CONFIG: recvmsg failed
2018-03-28T16:30:38.204Z|03552|dpdk|INFO|VHOST_CONFIG: vhost peer closed

This vectors parameter has to be configurable somewhere. It has to match OVS's setting, for example vectors has to be set as 6 at the connecting end if the queue count is 2.

from fastclick.

Can you try with the last rebase? I don't have the right setup to test with virtio_user...

from fastclick.

Can you try with the last rebase? I don't have the right setup to test with virtio_user...

Sorry I guess I am missing it, which rebase?

from fastclick.

from fastclick.

Same issue. I asked the following to the developer that released the multiqueue patch on virtio

Hello,

I am working on connecting VNFs in docker containers to OVS over virtio user using multi-queues and I found out that you have pushed the patch http://dpdk.org/ml/archives/dev/2016-May/038353.html for it.
In qemu for using virtio, the vectors parameter is configurable in addition to the queues in vdev settings. I was wondering if you are planning to have configurable vectors as an addition to queues in your patch.

and got the following response

Hi 

We now only allow 1:1 vec to rx queue mapping which is hardcoded; each vec is connected with a callfd.
In qemu context, similarly, each queue would have a callfd, it’s just multiple callfds are translated to one interrupt vec in KVM.

For vector number, in QEMU’s case, we suggest 2N+2 vectors, N for tx queues, N for rx queues, 1 for config, and one for possible control vq.

So this makes me think that FromDPDK/ToDPDK element should have configurable vector size.

from fastclick.

Some more feedback from Intel

The design of virtio-user is to supersede the QEMU with a vdev layer embedded in the virtio pmd; if we go back a little bit, people also suggest to have a QEMU in qtest mode.

So basically, all parameters for virtio-net-device of QEMU shall be ported to this vdev (virtio-user) specific parameters.

And I’m not sure I understand “a configurable parameter in the EAL for vdev” correctly. IMO, there is no such parameter from EAL for a vdev in DPDK. The device-specific parameters always comes from devargs when we do dev_attach().     

from fastclick.

I would prefer the stacktrace in text format, copy/paste the "where" output of GDB.

from fastclick.

So the setup is inside a KVM/QEMU VM which has two virtio ports that are attached to OVS, right?
Can you provide a full install step maybe ?

from fastclick.

Apologies for the late response, I got hold up with the paper side of things.
I am working on a full tutorial but not finished yet. But following is a summary.

The setup is on the hardware directly. No VM/QEMU involved

1. Install OVS with DPDK support (link)
2. Install Docker CE (link)
3. Save the following in a file named Dockerfile

FROM ubuntu:latest

######################
#Initial Preparations#
######################

RUN echo "Initial Preparations ..."

RUN apt-get update && apt-get install -y \
        build-essential automake git gcc tar wget python-pip libcap-ng-dev libnuma-dev \
        gawk pciutils linux-headers-$(uname -a | awk '{print $3}') nano kmod apt-utils \
        iproute2 zlib1g-dev
RUN pip install -U pip six

#################
#Installing DPDK#
#################
RUN echo "Installing DPDK ..."

ENV INSTALL_PATH "/usr/src"
ENV RTE_SDK "$INSTALL_PATH/dpdk-17.11"
ENV RTE_TARGET "x86_64-native-linuxapp-gcc"
ENV DPDK_BUILD "$RTE_SDK/$RTE_TARGET"

RUN cd $INSTALL_PATH; \
wget http://fast.dpdk.org/rel/dpdk-17.11.tar.xz; \
tar xf dpdk-17.11.tar.xz; \
rm dpdk-17.11.tar.xz; \
cd $RTE_SDK; \
make install -j4 T=$RTE_TARGET DESTDIR=install

######################
#Installing fastclick#
######################
RUN echo "Installing fastclick ..."

ENV CLICK_DIR "$INSTALL_PATH/fastclick"
ENV GIT_URL "https://github.com/kthkaya/fastclick"

RUN cd $INSTALL_PATH; \
git clone $GIT_URL; \
cd $CLICK_DIR; \
trframework/build_click.sh

######
#Done#
######
RUN echo "Done."
CMD ["/bin/bash"]

then in the folder Dockerfile exists, run the command

docker build -t fastclick .

4. Instantiate the docker image named fastclick with the following command

docker run -ti --name="instancename" -v /dev/hugepages:/dev/hugepages -v /usr/local/var/run/openvswitch:/var/run/openvswitch --privileged fastclick:latest

5. Press enter, now you are in the container. Navigate to /usr/src/fastclick

Now in the steps I provided #55; follow the ones until dpdkbridge.click on the server itself, and the rest of the steps in the container prompt.

from fastclick.

Ok thanks. I can more or less reproduce, except that it tells me that the device only supports 0 queues :p If I remove the check the device fails to initialize... So the error is somehow logical.
I'll look in more details at the virtio driver. Seems like a bug in DPDK, or an unexpected behaviour (maybe documented in the limbs of the DPDK documentation).

from fastclick.

FYI, I would sugget adding the following before installing DPDK :

sed -i 's/CONFIG_RTE_EAL_IGB_UIO=y/CONFIG_RTE_EAL_IGB_UIO=n/' config/common_linuxapp; \
sed -i 's/CONFIG_RTE_KNI_KMOD=y/CONFIG_RTE_KNI_KMOD=n/' config/common_linuxapp; \

Which will avoid dependency on the Linux headers, by not building KNI and igb_uio. WHen using virtio, they are not necessary. In my case I ran pure debian, and the headers are not available on Ubuntu for the same kernel.

from fastclick.

Thanks, will do!

from fastclick.

A bad merge some time ago removed part of the message, making it misleading. It's now maybe even too much bla bla. But it should be clear enough.
I think following the instructions, this will solve your problem.

The core of the problem is that FastClick will not automatically rely on locking in the ToDPDKDevice to use more threads than queues. Therfore you must pass the argument by yourself, in the ToDPDKDevice.

If it's not fixed after merging and following the message, we'll have at least a much better view of the problem...

from fastclick.