Comments (10)
Add PushNull element after RatedSources. That will improve the performance.
from fastclick.
If CPU is not at 100% on any CPU it means the problem is not the cycles.
Counter use an EWMA, it may just lag a little. AverageCounter will give you the rate between the first and last packet.
from fastclick.
Tried with Counter too. I believe something is not right with the RatedSource element, or I don't properly understand its purpose. Following is a lean test of RatedSource.
//--------Program Start----------
rs :: RatedSource(LENGTH 64, RATE 25000, LIMIT 500000, STOP true)
-> NumberPacket(OFFSET 0)
-> record1 :: RecordTimestamp()
-> CheckNumberPacket(OFFSET 0, COUNT 500000)
-> diff1 :: TimestampDiff(OFFSET 0, N 500000, RECORDER record1)
-> c1 :: Counter()
-> Discard;
DriverManager(wait, read diff1.average, read c1.count, read c1.rate)
//--------Program End----------
Results
diff1.average:
2.172142
c1.count:
500000
c1.rate:
16722.656
After some exploration, I found the RatedUnqueue element, which I believe what I was looking for. And I crafted the following test with InfiniteSource and RatedUnqueue.
//--------Program Start----------
is :: InfiniteSource(LENGTH 64, LIMIT 525000, STOP true)
-> Queue(25007)
-> RatedUnqueue (RATE 25000)
-> NumberPacket(OFFSET 0)
-> record :: RecordTimestamp()
-> CheckNumberPacket(OFFSET 0, COUNT 500000)
-> diff1 :: TimestampDiff(OFFSET 0, N 500000, RECORDER record)
-> c1 :: Counter()
-> Discard;
DriverManager(wait, read diff1.average, read c1.count, read c1.rate)
//--------Program End----------
Results
diff1.average:
0.117046
c1.count:
500000
c1.rate:
25007.8125
I got the counter rate that I was looking for (a little higher than what I was expecting). If my understanding of RatedUnqueue and Counter.rate in the last experiment is correct, then something is wrong with the RatedSource. What do you think?
from fastclick.
RatedSource should work.
The solution you have is performance-wise horrible. And horrible. Maybe batching is the issue here? It would break the algorithm maybe. Can you try once RatedSource solution by compiling with --disable-batch ?
from fastclick.
Wow! It indeed was the issue. The CPU utilization now hit %100 and following is the result.
diff1.average:
0.002588
c1.count:
500000
c1.rate:
25007.8125
Perhaps it could be a good idea to pop a warning, or completely lock RatedSource when batching is not disabled?
from fastclick.
No we have to fix the issue :p
I'm not sure why it is at 100% though...
from fastclick.
Idk if it matters but fastclick is running in a Docker container in all those tests.
from fastclick.
Can you provide the full config?
from fastclick.
Dockerfile
FROM ubuntu:17.10
######################
#Initial Preparations#
######################
RUN echo "Initial Preparations ..."
RUN apt-get update && apt-get install -y \
build-essential automake git gcc tar wget python-pip libcap-ng-dev libnuma-dev \
gawk pciutils linux-headers-$(uname -a | awk '{print $3}') nano kmod apt-utils \
iproute2 zlib1g-dev
RUN pip install -U pip six
#################
#Installing DPDK#
#################
RUN echo "Installing DPDK ..."
ENV INSTALL_PATH "/usr/src"
ENV RTE_SDK "$INSTALL_PATH/dpdk-17.11"
ENV RTE_TARGET "x86_64-native-linuxapp-gcc"
ENV DPDK_BUILD "$RTE_SDK/$RTE_TARGET"
RUN cd $INSTALL_PATH; \
wget http://fast.dpdk.org/rel/dpdk-17.11.tar.xz; \
tar xf dpdk-17.11.tar.xz; \
rm dpdk-17.11.tar.xz; \
cd $RTE_SDK; \
make install -j4 T=$RTE_TARGET DESTDIR=install
######################
#Installing fastclick#
######################
RUN echo "Installing fastclick ..."
ENV CLICK_DIR "$INSTALL_PATH/fastclick"
ENV GIT_URL "https://github.com/kthkaya/fastclick"
RUN cd $INSTALL_PATH; \
git clone $GIT_URL; \
cd $CLICK_DIR; \
./configure --enable-multithread --enable-intel-cpu --enable-user-multithread --enable-ip6; \
make && make install
######
#Done#
######
RUN echo "Done."
CMD ["/bin/bash"]
Build an image
Save above in a file named Dockerfile. In the same folder, run the following
docker build -t fastClick .
Instantiate a container and attach to it.
docker run -ti --name="trafGen" --privileged fastClick:latest
Run the packetgen Click code
Fastclick is in /usr/src/fastclick. Save the RatedSource generator code at my first post and run it.
from fastclick.
No activity, and it should be possible. The OP probably misconfigured.
from fastclick.
Related Issues (20)
- Secondary DPDK with FastClick HOT 2
- Access useful_kcycles() HOT 3
- Input aggregation element HOT 1
- --with-netmap does not compile HOT 3
- FastClick can't identify DPDK PORT,whatever Integer or PCI address.
- port 0 Rx queue allocation failed: Cannot allocate memory HOT 7
- Illegal instruction (core dumped) when using ToDPDKDevice() HOT 3
- rte_mempool segmentation fault. HOT 8
- storeing data in packet HOT 2
- How to enable scattered mode in fastclick? HOT 7
- issue in make package element HOT 3
- How to use replayUnqueue or MultiReplayUnqueue to limit the tx rate? HOT 3
- Automatically update wiki doc with CI
- middleclick configuration HOT 1
- compile error about context HOT 1
- Allow loop in FromDump HOT 4
- Non-existent librairies when parsing llvm-config's output
- FromDPDKDevice(FLOW_RULES_FILE) only work with DPDK<=21.11 HOT 2
- About address of IIO LLC WAYS in Ice lake HOT 3
- Deadlock possible between BalancedThreadSched and RatedUnqueue HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastclick.