Comments (9)
- clap: it looks awesome! I haven't used it, but if you like it, let's stick with it!
- required options: none
- option groups: none, but warn when
!(min <= median && median <= max)
, (the log + env_logger crates are a nice first pick for outputting stuff, and we can decorate the logs later with things like machine resource utilization stats with a custom logger etc...) - default threads: number of cpu cores
- operations: 1 million (doctor evil.jpg)
- reads -> get, writes -> set (cas is a write, scan is a read, so we should be specific to the operation on the tree)
- freshness-bias: valid values: old, new, random
--no-present-key-chance
maybe should be--non-present-key-chance
- defaults: 80% get + 15% set, + 4% scan + 1% cas. 64 byte keys, 512 byte values, with min/max/median all being the same. number of cpu cores on the machine for thread count. 50 scan iterations.
from sled.
@pmuens no need for percentages, we can just sum all of the proportions together, use that as a max number to feed a random number generator, (in your example, 80 is the max). Say it spits back 22. We see get is 10, which is less than 22, so we chop off 10 then go to the next. set is 20, and now we're at 12, so we decide that this operation will be a set.
yeah, exactly 64 / 512
from sled.
Thanks for the writeup @spacejam 👍
Jumped into this today. Here are just some quick questions / notes from my side which came up during the implementation.
I decided to use the clap crate which is quite popular for such use-cases to implement the CLI application. Any objections with that choice? Is it maybe too bloated? Here are some other crates we could consider.
However so far I'm pretty happy with clap
since it's really easy to use and quite powerful as well.
Could you get into more detail regarding the options we want to use here?
- Which options are required?
- Which options from "option groups" can be used together and which ones are XOR (e.g. "key size min, max, median" <-- should only one argument usage be valid here or can the user pass arguments for all of them?)
Here's what I came up with so far:
- number of threads
--num-threads
-u64
- (defaults to X (TBD))
- number of total operations
--num-operations
-u64
- (defaults to X (TBD))
- proportion of reads, writes, cas, del, and scan operations among the operations
--prop-reads
-u64
--prop-writes
-u64
--prop-cas
-u64
--prop-del
-u64
--prop-scan
-u64
- freshness bias (prefer recent/likely to be in cache, prefer old/not likely to be in cache, no preference)
--freshness-bias
-String
- (defaults to "no preference")
- non-present-key chance (the chance that a request for cas/get/set/del may be sent for a key that does not exist)
--no-present-key-chance
-bool
- key size min, max, median
--key-size-min
-u64
--key-size-max
-u64
--key-size-median
-u64
- value size min, max, median
--value-size-min
-u64
--value-size-max
-u64
--value-size-median
-u64
- scan iterations min, max, median
--scan-iter-min
-u64
--scan-iter-max
-u64
--scan-iter-median
-u64
Thanks in advance!
from sled.
for number of cpu cores, I already included the num_cpus
crate, so we can use that to get the number
from sled.
@spacejam could you please get into more detail about the following comments?
defaults: 80% get + 15% set, + 4% scan + 1% cas.
Could you maybe provide an example CLI input with the get
, set
, scan
, delete
and cas
arguments and how they're translated to percentages? I understand it in a way that you e.g. say --get 10 --set 20 --scan 50 ...
. But how would we be able to translate that to percentages like the ones described above since the user provides the numbers (or am I missing smth. obvious here?)?
64 byte keys, 512 byte values, with min/max/median all being the same.
Does this mean that the key we use should be exactly 64 byte long and the value always 512 bytes?
Thanks in advance!
from sled.
@spacejam thanks for the comment 👍
That makes sense! I'll link the outcome of the conversation here in #56 where this will be implemented!
from sled.
@spacejam just one quick question regarding this:
@pmuens no need for percentages, we can just sum all of the proportions together, use that as a max number to feed a random number generator, (in your example, 80 is the max). Say it spits back 22. We see get is 10, which is less than 22, so we chop off 10 then go to the next. set is 20, and now we're at 12, so we decide that this operation will be a set.
Unfortunately I'm stuck understanding the random number generation usage here.
Could you provide a quick example how the proportions for a set of given Tree operations would be calculated using this? Thanks in advance! 👍
from sled.
so, if any of the tree op types are provided, the defaults for all of the others should become 0.
bench --set=5 --del=2
for each iteration of each thread that is running commands:
sum = 5 + 2
ops = vec![(Op::Set, 5), (Op::Del, 2)];
let mut choice = rand::gen_range::<usize>(0, sum);
for (op, weight) in ops {
if weight >= choice {
return op;
}
choice -= weight;
}
from sled.
Thanks for the explanation and the code-snippet @spacejam 👍
from sled.
Related Issues (20)
- `IoBuf::get_mut_range` unsoundly extends arbitrary lifetime to 'static HOT 2
- panic: cannot find pid in view_for_key, looking for key
- Hi I wanted to know if this project is dead? HOT 13
- Is there any GUI program to explore sled db
- Does Tree::transaction really need Fn instead of FnMut? HOT 3
- Data is not written durably HOT 1
- If the assertion goes into unwinding, the value will be leaked inside a panic.
- memory leak?
- Modify issues related to atomic::ordering usage
- support ttl or some callback mechanism to remove data automatically
- Storage issues when using high count of trees HOT 1
- Less unsafe, in more controlled instances HOT 2
- document the fact zstd support is currently halfway removed
- VFS
- Is it safe to drop `Db` after `Tree`?
- Can `K` in `Tree::Range` be made `?Sized`?
- Add MSRV to `Cargo.toml`
- Preserve the insertion ordering HOT 1
- Bloodstone `Db::tree_names`
- Weak Db handle
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sled.