dataartisans / flink-benchmarks Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Currently, the statebanchmark did not delete the temp directory after the benchmark teardown, so we'll have many empty directory in /tmp
,
Our latest flink benchmark Jenkins builds are failing with below error:
[ERROR] Failed to execute goal on project flink-hackathon-benchmarks: Could not resolve dependencies for project org.apache.flink.benchmark:flink-hackathon-benchmarks:jar:0.1: Failure to find org.apache.flink:flink-tests_2.11:jar:tests:1.10-SNAPSHOT in https://repository.apache.org/content/repositories/snapshots/
Which is due to the branching of 1.9 has updated our flink project version to 1.10 while still no 1.10 snapshot deployed yet. I tried to help deploy but it turned out with no access. We need someone with the privilege help deploy the snapshot for the new version.
What's more, to prevent such issue happen again, we need to document it somewhere that a deploy of snapshot is necessary when branching new releases and update the snapshot version.
I wonder if these benchmarks only work with local mode, or we could modify and apply them on yarn cluster or other deploy environment.
Results of ListStateBenchmark
are not stable:
# Benchmark: org.apache.flink.state.benchmark.ListStateBenchmark.listAddAll
# Parameters: (backendType = HEAP)
# Run progress: 5.56% complete, ETA 00:35:16
# Fork: 1 of 3
# Warmup Iteration 1: 763.432 ops/ms
# Warmup Iteration 2: 114.673 ops/ms
# Warmup Iteration 3: 169.797 ops/ms
# Warmup Iteration 4: 805.482 ops/ms
# Warmup Iteration 5: 346.139 ops/ms
# Warmup Iteration 6: 1292.193 ops/ms
# Warmup Iteration 7: 23.363 ops/ms
# Warmup Iteration 8: 756.512 ops/ms
# Warmup Iteration 9: 845.994 ops/ms
# Warmup Iteration 10: 86.222 ops/ms
Iteration 1: 725.139 ops/ms
Iteration 2: 1397.114 ops/ms
Iteration 3: 311.249 ops/ms
Iteration 4: 312.853 ops/ms
Iteration 5: 6.339 ops/ms
Iteration 6: 1179.413 ops/ms
Iteration 7: 1380.267 ops/ms
Iteration 8: 1391.004 ops/ms
Iteration 9: 571.861 ops/ms
Iteration 10: 31.509 ops/ms
The same applies for RocksDB, which is visible in averages:
http://codespeed.dak8s.net:8000/timeline/#/?exe=3&ben=listAddAll.ROCKSDB&env=2&revs=1000&equid=off&quarts=on&extr=on
Does this benchmark have a GC issue?
CC @carp84
Hi all
From my understanding, the benchmark for state-backend should only compare the performance of combined state operations, but the operation to benchmark within JMH contains the time of submitting job, deploying tasks and so on.
Why not just compare the standalone state-backend performance without influence from other Flink components?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.