Hello,

For the newbies can you please update the document on how to install it on a closed environment where github.com is not accessible. Was trying to download the repository on different host and pushed to closed environment but still its looking for some dependency object in github, can you please combine all dependency in one package or correct this url http://ps-tools.s3.amazonaws.com/basho-bench-0.10.0.53.g0e15158-1.el7.centos.x86_64.rpm. Thank you.

`make all
/app/scripts/basho_bench_bkp/rebar get-deps
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> basho_bench_bkp (get-deps)
Pulling node_package from {git,"git://github.com/basho/node_package",
{tag,"2.0.0"}}
fatal: unable to connect to github.com:
github.com[0: ip]: errno=Connection timed out
github.com[1: ip]: errno=Connection timed out

Cloning into 'node_package'...
ERROR: sh(git clone -n git://github.com/basho/node_package node_package)
failed with return code 128 and the following output:
fatal: unable to connect to github.com:
github.com[0: ip]: errno=Connection timed out
github.com[1: ip]: errno=Connection timed out

Cloning into 'node_package'...

ERROR: 'get-deps' failed while processing /app/scripts/basho_bench_bkp: rebar_abort
make: *** [deps] Error 1
`

The following config was used to populate a riak node with concurrent workers and a partitioned_sequential_int key generator. Keys at the end of the range are never inserted and I have to do a manual run with a single worker to fill in the hole for some reason:

{mode, max}.
{driver, basho_bench_driver_2i}.
{duration, 60}.
{concurrent, 12}.
{key_generator, {partitioned_sequential_int, 0, 101000}}.
{value_generator, {exponential_bin, 1024, 50000}}.
{pb_ips, [{"127.0.0.1", 10017}]}.
{pb_replies, 2}.
{rng_seed, {1, 2, 3}}.
{operations, [{ { put_pb, 5 }, 1}] }.

18:17:02.269 [error] The on_load function for module e2qc_nif returned {{badmatch,{error,{bad_lib,"Library version (2.11) not compatible (with 2.4)."}}},[{e2qc_nif,init,0,[{file,...},{...}]},{code_server,'-handle_on_load/4-fun-0-',1,[{...}|...]}]}

It seemed the rpm one containing some nif not compatible with app have nif compiled with R19.2

and i also can not compile it with the source under contos 7+ R19.2

WARN:  Missing plugins: [rebar_lock_deps_plugin]
WARN:  Missing plugins: [rebar_lock_deps_plugin]
==> node_package (compile)
ERROR: OTP release 19 does not match required regex R16|17|18
ERROR: compile failed while processing /mnt/hgfs/share/devtest/basho_bench/deps/node_package: rebar_abort
make: *** [compile] Error 1

Right now, generating fixed_bin values can result in non UTF-8 compatible characters. It would be helpful to provide a fixed_char to generate a certain length random character value that is UTF-8 clean.

I have installed basho_bench the first time using RPM (previously i used it the source only). I was running into this issue:

$ basho_bench -d tests /etc/basho_bench/http.config
escript: exception error: no match of right hand side value {error,eacces}
  in function  basho_bench:test_dir/2 (src/basho_bench.erl, line 178)
  in call from basho_bench:main/1 (src/basho_bench.erl, line 47)
  in call from escript:run/2 (escript.erl, line 747)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

I was wondering what is going ton and after poking around I have seen #56.

I have tried it with sudo.

[vagrant@localhost ~]$ sudo basho_bench -d tests /etc/basho_bench/http.config
16:45:47.105 [debug] Lager installed handler {lager_file_backend,
                            "/usr/lib64/basho_bench/lib/basho_bench-0.10.0/ebin/tests/20150408_164547/error.log"} into lager_event
16:45:47.105 [debug] Lager installed handler {lager_file_backend,
                            "/usr/lib64/basho_bench/lib/basho_bench-0.10.0/ebin/tests/20150408_164547/console.log"} into lager_event

I guess the following function prefixes the parameter that was passed in with -d with the CWD that is the "/usr/lib64/basho_bench/lib/basho_bench-0.10.0/ebin" in my case if the path is not a full path.

https://github.com/basho/basho_bench/blob/0.10.0/src/basho_bench.erl#L172

It would be nice to make it explicit that the path has to be absolute and relative paths will fail if the user has no access to the CWD of the basho_bench binary.

What do you guys think?

Does the current riakc_pb driver support username & password authentication from Riak 2.0?
If yes, let's add documentation.
If not, let's implement.

This is mostly a documentation issue, but the README states that "Erlang/OTP R13B03 or later" is required.

I have installed Erlang R14B04 (from EPEL on Oracle Linux 6.x) and get this make error:

$ make all
./rebar get-deps

=ERROR REPORT==== 2-Feb-2016::10:14:01 ===
beam/beam_load.c(1365): Error loading module rebar:
  use of opcode 153; this emulator supports only up to 152

After some searching, I found #154 from 2014 in which I read:

The version of Erlang/OTP that you're using to try to build basho_bench appears to be R14, whereas basho_bench requires R15 or newer ... including the rebar executable that's bundled with basho_bench.

Can you please fix the README to reflect this.

R.

The current implementation of 2i MapReduce uses the KeyGen to pick a starting key, and picks the range by adding the number of keys that it was given. This can result in M/R queries over keys that don't exist if KeyGen returns an edge number.

For instance, with a M/R job querying 20 keys, if KeyGen is set to generate 1,000 keys, it could return 990, and the M/R job would be short by 10 non-existing keys.

• more recent otp versions
• build and run tests on travicis.

Hi! I'm using basho bench to do some performance evaluation and I found that it seems that when a get operation can't get the right response(like a 404 in http test), no errors will be reported.Are there any ways or configurations that can support the reporting of situation?
Thanks U.

make
...
...

ERROR: Failed to get ebin/ directory for erlcql escript_incl_apps.ERROR: escriptize failed while processing /tmp/basho_bench: rebar_abort

After installing kerl, erlang, R and the necessary dependencies, I still can't run make on this. Heres the errors I get. This is on an AWS instance, which was built off of the Basho instance from the AWS marketplace.

# make
./rebar get-deps

=ERROR REPORT==== 23-Nov-2014::02:47:40 ===
Loading of /home/ec2-user/basho_bench/rebar/rebar/ebin/rebar.beam failed: badfile
escript: exception error: undefined function rebar:main/1
  in function  escript:run/2
  in call from escript:start/1
  in call from init:start_it/1
  in call from init:start_em/1

=ERROR REPORT==== 23-Nov-2014::02:47:40 ===
beam/beam_load.c(1365): Error loading module rebar:
  use of opcode 153; this emulator supports only up to 152

make: *** [deps] Error 127

mailmark thread: http://riak.markmail.org/thread/hlzuyhpbrmezk4kg

the http_raw driver doesn't seem to be working for me

  root@staging:/root/basho_bench# erl 
  Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]
  Eshell V5.8.4  (abort with ^G)

  root@staging:/root/basho_bench# curl -v http://10.1.1.101:8089/riak/test
  * About to connect() to 10.1.1.101 port 8089 (#0)
  *   Trying 10.1.1.101... connected
  * Connected to 10.1.1.101 (10.1.1.101) port 8089 (#0)
  > GET /riak/test HTTP/1.1
  > User-Agent: curl/7.21.3 (x86_64-pc-linux-gnu) libcurl/7.21.3 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
  > Host: 10.1.1.101:8089
  > Accept: */*
  > 
  < HTTP/1.1 200 OK
  < Vary: Accept-Encoding
  < Server: MochiWeb/1.1 WebMachine/1.9.0 (someone had painted it blue)
  < Date: Wed, 07 Mar 2012 00:24:38 GMT
  < Content-Type: application/json
  < Content-Length: 422
  < 
  * Connection #0 to host 10.1.1.101 left intact
  * Closing connection #0
  {"props":{"name":"test","allow_mult":false,"basic_quorum":false,"big_vclock":50,"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"dw":"quorum","last_write_wins":false,"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"n_val":3,"notfound_ok":true,"old_vclock":86400,"postcommit":[],"pr":0,"precommit":[],"pw":0,"r":"quorum","rw":"quorum","small_vclock":50,"w":"quorum","young_vclock":20}}

Error:

=ERROR REPORT==== 7-Mar-2012::00:16:23 ===
** Generic server <0.74.0> terminating 
** Last message in was {send_req,
                           {{url,undefined,"10.1.1.101",8089,undefined,
                                undefined,"/riak/test/3900",undefined},
                            [],get,[],
                            [{response_format,binary}],
                            5000}}
** When Server state == {state,"10.1.1.101",8089,undefined,#Ref<0.0.0.188>,
                               false,undefined,[],false,undefined,false,[],
                               {[],[]},
                               undefined,idle,undefined,<<>>,0,0,[],undefined,
                               undefined,undefined,undefined,false,undefined,
                               undefined,<<>>,undefined,false,undefined,0,
                               undefined}
** Reason for termination == 
** {function_clause,
       [{ibrowse_http_client,send_req_1,
            [{<0.64.0>,#Ref<0.0.0.192>},
             {url,undefined,"10.1.1.101",8089,undefined,undefined,
                 "/riak/test/3900",undefined},
             [],get,[],
             [{response_format,binary}],
             5000,
             {state,"10.1.1.101",8089,undefined,#Ref<0.0.0.188>,false,
                 undefined,[],false,undefined,false,[],
                 {[],[]},
                 undefined,idle,undefined,<<>>,0,0,[],undefined,undefined,
                 undefined,undefined,false,undefined,undefined,<<>>,undefined,
                 false,undefined,0,undefined}]},
        {gen_server,handle_msg,5},
        {proc_lib,init_p_do_apply,3}]}

test.config:

{mode, max}.
{duration, 1}.
{concurrent, 3}.
{driver, basho_bench_driver_http_raw}.
{key_generator, {int_to_str, {uniform_int, 10000}}}.
{value_generator, {fixed_bin, 10000}}.
{http_raw_ips, ["10.1.1.101"]}.
{http_raw_port, 8089}.
{http_raw_path, "/riak/test"}.
{operations, [{get, 1}, {update, 1}]}. 

Any thoughts?

Hi

I run in to this problem:

9:52:59.439 [error] gen_server <0.135.0> terminated with reason: {'module could not be loaded',[{riak_pb_codec,encode,[{rpbgetreq,<<"test">>,<<0,0,20,71>>,1,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined}],[]},{riakc_pb_socket,encode_request_message,1

I guess it's the same problem as this issue.
basho/riak-erlang-client#151

I don't remember running it to this a couple of weeks ago.

Steps to reproduce:

• Write a driver Driver with new and run callbacks
• Start Basho Bench in standalone mode with at least one worker using the driver
• (Invocation of Driver:new succeeds)
• Cause an invocation of Driver:run to crash e.g. badmatch (write run callback craftily for this to happen)
• (Basho Bench tries to restart the driver) Cause the invocation of Driver:new to crash e.g. badmatch
• Observe that basho Bench does not stop

I feel (I have not double checked) the root cause of the non-termination is the interaction between:

• The way module basho_bench_worker handles crashes, and
• The init:stop call introduced in #117.

Drivers: security
Requestor: customer request
Internal contact: Alex B or John W

I'm using riemann to monitor Riak clusters under benchmarking, and I figured it may be a good idea to push the periodic measurements and latencies to Riemann, instead of into a CSV file. So I thought that making basho_bench_stats a little pluggable, this would become much easier to accomplish, and would open up other opportunities for near-real-time monitoring.

I plan to work on the feature soonish, but since I'm fairly new to Erlang, I wanted to give a little heads-up, and see what others think about the idea.

(I'd make the system somewhat similar to how generators are done, I found that very neat.)

The fix is here:
#194

This rmp basho-bench-0.10.0.53-1.el7.centos.x86_64.rpm still has this problem #178 and all paths i typed in were abs-path;
with the exception error
exception error: no match of right hand side value ..........
in function basho_bench:test_dir/2 (src/basho_bench.erl, line 180

Workaround:

./priv/gp_throughput.sh -t dumb -P | gnuplot

When set terminal dumb, gnuplot -persist does not output any result.
Without -persist option gnuplot outputs graph successfully.

mochijson2 is required for the 2i driver, yet is not included.

All of the .r scripts in priv/ are broken. The all start with the following shebang line:

!/usr/bin/env Rscript --vanilla

This causes the system to look in the path for a binary called 'Rscript --vanilla'.

This behavior is mentioned in the wikipedia article located at http://en.wikipedia.org/wiki/Shebang_(Unix)

Here is the relevant snip:

Another portability problem is the interpretation of the command arguments. Some systems, including Linux, do not split up the arguments;[11] for example, when running the script with the first line like,

!/usr/bin/env python -c

That is, python -c will be passed as one argument to /usr/bin/env, rather than two arguments. Cygwin also behaves this way.

This can be resolve in multiple ways, and perhaps the easiest is to simply modify the make file and instead of calling the .r scripts directly, call 'Rscript --vanilla priv/script.r

basho_bench_driver_http.erl can call basho_bench_valgen:new with same id many times.

This crash with badarg since it creates a ets table every time.

build_generators([{Name, {value_generator, ValGenSpec}}|Rest], Generators, Id) ->
ValGen = basho_bench_valgen:new(ValGenSpec, Id),
build_generators(Rest, [{Name, ValGen}|Generators], Id);

Seems to be introduced by:
44ee51f

Hi all,
I am trying to build basho-bench from the source code as given in basho-bench docs to benchmark riak 3.0 as shown below.
These are the commands used to build basho bench.

git clone git://github.com/basho/basho_bench.git
cd basho_bench
make all

But it is failing with the following error.

[root@localhost basho_bench]# make
/home/mac/Projects/basho_bench/rebar get-deps
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> basho_bench (get-deps)
Pulling node_package from {git,"git://github.com/basho/node_package",
{tag,"2.0.0"}}
Cloning into 'node_package'...
Pulling folsom from {git,"git://github.com/basho/folsom.git",
{branch,"boundary-0.7.1+basho-bench-float"}}
Cloning into 'folsom'...
Pulling lager from {git,"git://github.com/basho/lager",{tag,"2.1.0"}}
Cloning into 'lager'...
Pulling ibrowse from {git,"git://github.com/cmullaparthi/ibrowse.git",
{tag,"v4.0.2"}}
Cloning into 'ibrowse'...
Pulling riakc from {git,"git://github.com/basho/riak-erlang-client",
{branch,"master"}}
Cloning into 'riakc'...
Pulling mochiweb from {git,"git://github.com/basho/mochiweb",{tag,"v2.9.0"}}
Cloning into 'mochiweb'...
Pulling getopt from {git,"git://github.com/jcomellas/getopt",{tag,"v0.8.2"}}
Cloning into 'getopt'...
Pulling casbench from {git,"git://github.com/basho/casbench",
"95ed55b494551577870984aeb1e0f683631a326f"}
Cloning into 'casbench'...
Pulling cqerl from {git,"git://github.com/matehat/cqerl.git",
"16021593c866ca314acd0ba6858194ed44a362b3"}
Cloning into 'cqerl'...
Pulling katja from {git,"git://github.com/nifoc/katja.git",{branch,"master"}}
Cloning into 'katja'...
Pulling rebar_lock_deps_plugin from {git,
"git://github.com/seth/rebar_lock_deps_plugin.git",
{tag,"3.1.0"}}
Cloning into 'rebar_lock_deps_plugin'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> node_package (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> folsom (get-deps)
Pulling bear from {git,"git://github.com/boundary/bear.git",{tag,"0.1.1"}}
Cloning into 'bear'...
Pulling meck from {git,"git://github.com/basho/meck.git",{tag,"0.8.2"}}
Cloning into 'meck'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> bear (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> meck (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> lager (get-deps)
Pulling goldrush from {git,"git://github.com/DeadZen/goldrush.git",
{tag,"0.1.6"}}
Cloning into 'goldrush'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> goldrush (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> ibrowse (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> riakc (get-deps)
Pulling riak_pb from {git,"https://github.com/basho/riak_pb",{tag,"2.3.2"}}
Cloning into 'riak_pb'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> riak_pb (get-deps)
Pulling hamcrest from {git,"https://github.com/basho/hamcrest-erlang.git",
{tag,"0.4.1"}}
Cloning into 'hamcrest'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> hamcrest (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> mochiweb (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> getopt (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> casbench (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> cqerl (get-deps)
Pulling snappy from {git,"https://github.com/fdmanana/snappy-erlang-nif.git"}
Cloning into 'snappy'...
Pulling lz4 from {git,"https://github.com/szktty/erlang-lz4.git"}
Cloning into 'lz4'...
Pulling semver from {git,"https://github.com/nebularis/semver.git"}
Cloning into 'semver'...
Pulling uuid from {git,"https://github.com/okeuday/uuid.git"}
Cloning into 'uuid'...
Pulling pooler from {git,"https://github.com/seth/pooler.git"}
Cloning into 'pooler'...
WARN: Missing plugins: [pc,rebar_lock_deps_plugin]
==> snappy (get-deps)
WARN: Missing plugins: [pc,rebar_lock_deps_plugin]
==> lz4 (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> semver (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> uuid (get-deps)
Pulling quickrand from {git,"https://github.com/okeuday/quickrand.git",
{branch,"master"}}
Cloning into 'quickrand'...
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> quickrand (get-deps)
WARN: Missing plugins: [rebar_lock_deps_plugin]
==> pooler (get-deps)
Uncaught error in rebar_core: {'EXIT',
{function_clause,
[{code,which,
[{rebar3_gpb_plugin,"2.12.5"}],
[{file,"code.erl"},{line,790}]},
{rebar_core,'-plugin_modules/3-lc$^0/1-0-',
1,
[{file,"src/rebar_core.erl"},{line,573}]},
{rebar_core,plugin_modules,3,
[{file,"src/rebar_core.erl"},{line,573}]},
{rebar_core,process_dir1,7,
[{file,"src/rebar_core.erl"},{line,244}]},
{rebar_core,process_each,5,
[{file,"src/rebar_core.erl"},{line,351}]},
{rebar_core,process_dir1,7,
[{file,"src/rebar_core.erl"},{line,271}]},
{rebar_core,process_commands,2,
[{file,"src/rebar_core.erl"},{line,93}]},
{rebar,main,1,
[{file,"src/rebar.erl"},{line,58}]}]}}
make: *** [deps] Error 1

Seems like missing some dependency plugins, can you help us with this error.

-Rama krishna.

It would better emulate production behaviour, where there is a default retry 3 times strategy.

When doing massive inserts in leveldb (slow) using partitioned keyspace between workers, its a bummer to have a worker crash because of timeout and restart from scratch. Not only is a lot of work lost, but it also changes the entire load job from being pure insert into empty database to being insert+replace some of the data. It is difficult to make meaningful performance meassurements of load jobs or predict insert time, when workers crash and restart all the time.

Using 0f9b539

Pulling casbench from {git,"git://github.com/basho/casbench",
                           "95ed55b494551577870984aeb1e0f683631a326f"}
Cloning into casbench...
error: pathspec 'origin/95ed55b494551577870984aeb1e0f683631a326f' did not match any file(s) known to git.
ERROR: git checkout -q origin/95ed55b494551577870984aeb1e0f683631a326f failed with error: 1

The commit in question does exist though - basho/casbench@95ed55b

• http_raw_ips can not be [{{A,B,C,D}, Port =/= 80}]. [{"A.B.C.D", Port}] works.
• there may be off-by-one error in http_raw_disconnect_frequency when set to {ops, 0} or {ops, 1}.
  (further confirmation is needed)
• handling of {error, connection_closed} needed, because the error can occur by race between client-side keep alive and server side active close.
• summary may be zero for very low rate like {rate, 1} even there are real ops executed.

This breaks basho_bench in master as it requires the basho_bench_log module that currently lives in slf-custom-units.

Crash:

2012-10-02 17:35:30.788 [error] <0.78.0> CRASH REPORT Process <0.78.0> with 0 neighbours exited with reason: call to undefined function basho_bench_log:log(info, "~s value gen props: ~p\n", [basho_bench_driver_moss,[{file_size,216330242},{max_rate_per_chunk,99999999}]]) in gen_server:init_it/6 line 328

For some datasets having points on the graph makes it much harder to read than looking at the trendlines only (and the points obscure information). It'd be nice to have a flag on summary.r to easily disable them.

Muti-user Multi-bucket test for further testing including multi-age

[Created in JIRA by Kota Uenishi]

Minor error, but summary.r fails the first time its run, and then works the second time.

08 ** inst
309 ** preparing package for lazy loading
310 ** help
311 *** installing help indices
312 ** building package indices ...
313 ** testing if installed package can be loaded
314
315 * DONE (ggplot2)
316
317 The downloaded packages are in
318 ‘/tmp/RtmpEKjJwX/downloaded_packages’
319 Loading required package: methods
320 Error: could not find function "getopt"
321 Execution halted
322 root@ec2app2:~/rovio_bench# Rscript deps/basho_bench/priv/summary.r -i tests/current323 [1] "plyr"
324 Loading required package: plyr325 [1] "grid"
326 Loading required package: grid327 [1] "getopt"
328 Loading required package: getopt329 [1] "proto"
330 Loading required package: proto331 [1] "ggplot2"
332 Loading required package: ggplot2333 Loading required package: methods
334 [1] 0335 [1] 7874
336 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.337 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
338 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.339 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
340 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.341 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
342 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.343 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
344 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.345 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
346 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
347 geom_smooth: method="auto" and size of largest group is <1000, so using loess. Use 'method = x' to change the smoothing method.
348 null device349 1

See also: brianfrankcooper/YCSB#214 (though that branch appears to be merged into the https://github.com/YCSB/YCSB repo now, despite the status of that ticket)?

It would be awfully nice if we could have basho_bench avoid measurements due to coordinated omission. Gil Tene and others have been very helpful in pointing out this problem in many tools. YCSB has recently fixed it; b_b should do so also.

Hello Team,

1. I am using the Dockerfile of this repo to create a new docker image.
2. After that, I created a docker container from that docker image. Inside the docker container there is a path /srv/basho_bench/_build/default/bin/ in that path basho_bench binary will be created.
3. I use that basho_bench binary to run load-test on riak with below config file.

{driver,
    basho_bench_driver_2i}.

{operations, [
    {get_pb, 1},
    {{put_pb, 5}, 20},
    {{query_http, 10}, 1},
    {{query_mr,   10}, 1},
    {{query_pb,   10}, 1}
]}.

{measurement_driver,
    basho_bench_measurement_erlangvm}.

{measurements, [
    {memory, 1000},
    {cpu, 1000},
    {processes, 1000},
    {filehandles, 1000}
]}.

%%% LOAD SETTINGS %%%

{mode, {rate, 5}}.
%%{mode, max}.
{duration, 1}.
{concurrent, 3}.

%%% DATA SHAPE %%%

{key_generator, uuid_v4}.
{value_generator, {uniform_bin, 300,1000}}.

{pb_ips, [{"*-riak1.*", 18087}]}.
{pb_replies, 1}.

{http_ips, ["*-riak1.*"]}.
{http_port, 18098}.

%% Timeout units are milliseconds, default = 30000
{pb_timeout_general, 30000}.
{http_timeout_general, 30000}.

%%It can also be configured for devrel!
%%{pb_ips, [{"*-riak1.*", 8087},
%%          {"*-riak2.*", 8087},
%%          {"*-riak3.*", 8087}
%%]}.
%%{pb_replies, 1}.
%%
%%{http_ips, [{"*-riak1.*", 8098},
%%            {"*-riak2.*", 8098},
%%            {"*-riak3.*", 8098}
%%]}.

{rng_seed, {1, 2, 3}}.

%% enforce_keyrange is for use with sequential_int
%% Will error if keys in a range are missing.
%%{enforce_keyrange, 10000}.

%%% MEASUREMENT SETTINGS %%%

{nodes, ['*-riak1.*','*-riak2.*','*-riak3.*']}.
{cookie, riak}.

4. I am running this inside docker container and that docker container is part of k8s pod which is part of k8s cluster.
5. Riak are configured as Azure VM which I can access via SSH from pod which contain my container with basho_bench script.
6. while running the Performance test from that docker container I am getting below error.
7. I am using this basho_bench script for the performance test first time.
8. can someone please help me, with this error and how can I resolve this ?
9. Any help will be really appreciated.

12:22:01.211 [error] Failed to start net_kernel for basho_bench_measurement_erlangvm: {{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}},{child,undefined,net_sup_dynamic,{erl_distribution,start_link,[#{clean_halt => false,name => basho_bench,name_domain => longnames,net_tickintensity => 4,net_ticktime => 60,supervisor => net_sup_dynamic}]},permanent,false,1000,supervisor,[erl_distribution]}}

Hi,

I run the sample command but got following log? Did I miss something?

./basho_bench examples/riakc_pb.config

03:04:02.285 [debug] Lager installed handler {lager_file_backend,
"/home/ubuntu/basho_bench/tests/20181212_030402/error.log"} into lager_event
03:04:02.285 [debug] Lager installed handler {lager_file_backend,
"/home/ubuntu/basho_bench/tests/20181212_030402/console.log"} into lager_event
03:04:02.285 [debug] Lager installed handler error_logger_lager_h into error_logger
03:04:02.338 [debug] Supervisor gr_param_sup started gr_param:start_link(gr_lager_default_tracer_params) at pid <0.60.0>
03:04:02.338 [debug] Supervisor gr_counter_sup started gr_counter:start_link(gr_lager_default_tracer_counters) at pid <0.61.0>
03:04:02.338 [debug] Supervisor gr_manager_sup started gr_manager:start_link(gr_lager_default_tracer_params_mgr, gr_lager_default_tracer_params, []) at pid <0.62.0>
03:04:02.338 [debug] Supervisor gr_manager_sup started gr_manager:start_link(gr_lager_default_tracer_counters_mgr, gr_lager_default_tracer_counters, [{input,0},{filter,0},{output,0}]) at pid <0.63.0>
03:04:02.750 [debug] Lager installed handler lager_backend_throttle into lager_event
03:04:03.317 [info] Application lager started on node nonode@nohost
03:04:03.319 [notice] Changed loglevel of /home/ubuntu/basho_bench/tests/20181212_030402/console.log to debug
03:04:03.330 [info] No dimension available for key generator: {int_to_bin_bigendian,{uniform_int,10000}}
03:04:03.406 [debug] Supervisor sasl_safe_sup started alarm_handler:start_link() at pid <0.73.0>
03:04:03.407 [debug] Supervisor sasl_safe_sup started overload:start_link() at pid <0.74.0>
03:04:03.407 [debug] Supervisor sasl_sup started supervisor:start_link({local,sasl_safe_sup}, sasl, safe) at pid <0.72.0>
03:04:03.422 [debug] Supervisor sasl_sup started release_handler:start_link() at pid <0.75.0>
03:04:03.422 [info] Application sasl started on node nonode@nohost
03:04:03.442 [info] Application crypto started on node nonode@nohost
03:04:03.482 [debug] Supervisor folsom_sup started folsom_sample_slide_sup:start_link() at pid <0.87.0>
03:04:03.487 [debug] Supervisor folsom_sup started folsom_meter_timer_server:start_link() at pid <0.88.0>
03:04:03.492 [debug] Supervisor folsom_sup started folsom_metrics_histogram_ets:start_link() at pid <0.89.0>
03:04:03.493 [info] Application folsom started on node nonode@nohost
03:04:03.569 [info] module=basho_bench_stats_writer_csv event=start stats_sink=csv
03:04:03.571 [debug] Supervisor basho_bench_sup started basho_bench_stats:start_link() at pid <0.82.0>
03:04:03.597 [info] Random source: calling crypto:rand_bytes(100663296) (override with the 'value_generator_source_size' config option
03:04:06.536 [info] Random source: finished crypto:rand_bytes(100663296)
03:04:06.602 [info] Using target {127,0,0,1}:8087 for worker 1
03:04:06.644 [info] Using target {127,0,0,1}:8087 for worker 2
03:04:06.645 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_1, 1) at pid <0.92.0>
03:04:06.645 [info] Using target {127,0,0,1}:8087 for worker 3
03:04:06.646 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_2, 2) at pid <0.96.0>
03:04:06.646 [info] Using target {127,0,0,1}:8087 for worker 4
03:04:06.648 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_3, 3) at pid <0.99.0>
03:04:06.648 [info] Using target {127,0,0,1}:8087 for worker 5
03:04:06.649 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_4, 4) at pid <0.102.0>
03:04:06.649 [info] Using target {127,0,0,1}:8087 for worker 6
03:04:06.650 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_5, 5) at pid <0.105.0>
03:04:06.650 [info] Using target {127,0,0,1}:8087 for worker 7
03:04:06.651 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_6, 6) at pid <0.108.0>
03:04:06.651 [info] Using target {127,0,0,1}:8087 for worker 8
03:04:06.652 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_7, 7) at pid <0.111.0>
03:04:06.653 [info] Using target {127,0,0,1}:8087 for worker 9
03:04:06.653 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_8, 8) at pid <0.114.0>
03:04:06.654 [info] Using target {127,0,0,1}:8087 for worker 10
03:04:06.654 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_9, 9) at pid <0.117.0>
03:04:06.660 [debug] Supervisor basho_bench_sup started basho_bench_worker:start_link(basho_bench_worker_10, 10) at pid <0.120.0>
03:04:06.661 [debug] Supervisor kernel_safe_sup started timer:start_link() at pid <0.123.0>
03:04:06.668 [info] Starting max worker: <0.118.0> on nonode@nohost
03:04:06.668 [info] Starting max worker: <0.121.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.115.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.106.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.103.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.112.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.100.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.97.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.109.0> on nonode@nohost
03:04:06.669 [info] Starting max worker: <0.94.0> on nonode@nohost
03:04:06.670 [info] Application basho_bench started on node nonode@nohost
03:04:07.672 [warning] No data for op: {get,get}
03:04:07.672 [warning] No data for op: {update,update}
03:04:08.661 [warning] No data for op: {get,get}
03:04:08.661 [warning] No data for op: {update,update}
03:04:09.661 [warning] No data for op: {get,get}
03:04:09.661 [warning] No data for op: {update,update}
03:04:10.661 [warning] No data for op: {get,get}
03:04:10.661 [warning] No data for op: {update,update}
03:04:11.661 [warning] No data for op: {get,get}
03:04:11.661 [warning] No data for op: {update,update}
03:04:12.661 [warning] No data for op: {get,get}
03:04:12.661 [warning] No data for op: {update,update}
03:04:13.661 [warning] No data for op: {get,get}
03:04:13.661 [warning] No data for op: {update,update}
^C
[email protected]:~/ejabberd/basho_bench#

It would be nice to have this feature in order that we can show/measure the performance of CRDT counters. It should have crdt_counter_incr and crdt_counter_get operations, or something like those.

When I run basho_bench as root there is no problem,
but if I run it as another user I get an error that is not very helpful (see below)

I suggest that this common error is reported a bit more informative so users can resolve this
problem faster in the future.

david@some_machine$ ./basho_bench
escript: exception error: no match of right hand side value {error,enoent}
in function basho_bench:test_dir/2 (src/basho_bench.erl, line 136)
in call from basho_bench:main/1 (src/basho_bench.erl, line 43)
in call from escript:run/2 (escript.erl, line 727)
in call from escript:start/1 (escript.erl, line 277)
in call from init:start_it/1
in call from init:start_em/1

Code line 136 in basho_bench.erl:
ok = filelib:ensure_dir(filename:join(TestDir, "foobar")),

Hi, I'm trying to do some test put test with basho_bench_driver_riakc_pb, but in the configuration files under the examples folder, I cannot find any way to insert a custom string as a value(I only found value generators that support random strings).So how can I insert a custom string like a json with the driver? Please tell me the way, thanks in advance.

You have the rebar.config with

{escript_incl_apps,
  [
    lager, getopt, bear, folsom, ibrowse,
    riakc, riak_pb, mochiweb, protobuffs, velvet, goldrush
  ]
}.

and with the old version of the riakc

{riakc, ".*",
 {git, "git://github.com/basho/riak-erlang-client", {tag, "1.4.1"}}},

which does not contains the riak_pb dependency.
And when I build the basho_bench I get an error

ERROR: Failed to get ebin/ directory for riak_pb escript_incl_apps.
ERROR: escriptize failed while processing /home/a/basho_bench: rebar_abort

There are no tests.

We have an old requirement from State Farm that we have already addressed
Self-contained Basho Bench for customers who don't have access to external packaging systems like EPEL or R's CRAN.
We need to make sure that Docs is updated so that we can point to packages that customers can download without needing to build or using external packages.
http://docs.basho.com/riak/latest/ops/building/benchmarking/

Documents links were broken, shoulde be http://docs.basho.com/riak/latest/ops/building/benchmarking/

I worked from two dev hosts with basho bench and just noticed that one host had deps/yaws_json2 but another didn't have. Probably it is a dependency of a dependency but it would be good if it is in the deps and in the escriptized libraries. Since basho bench is perfect for sending http requests and process json responses. So in that case I wouldn't need to fork basho bench, and it is easier to include it in multi-person projects.