3pillarlabs / hailstorm-sdk Goto Github PK

Distribute Apache JMeter tests with Amazon EC2 instances or data-center machines, optionally collect application infrastructure metrics and generate editable reports.

Home Page: https://3pillarlabs.github.io/hailstorm-sdk/

License: MIT License

Shell 0.42% Ruby 52.44% Gherkin 1.23% HTML 3.01% Java 4.81% JavaScript 0.72% CSS 0.01% Dockerfile 0.16% Makefile 1.19% TypeScript 35.70% SCSS 0.32%

jmeter aws nmon reports cli

hailstorm-sdk's Introduction

Hailstorm SDK

A cloud-aware library and applications for distributed load testing using JMeter and support for server monitoring.

Hailstorm uses JMeter to generate load on the system under test. You create your JMeter test plans/scripts using JMeter GUI interface. Hailstorm uses these test plans to generate load. Hailstorm uses Amazon EC2 to create load agents. Each load agent is pre-installed with JMeter. The application executes your test plans in non-GUI mode using these load agents. Hailstorm can also work with containers or virtual machines or physical machines in your data center. Hailstorm can monitor server side resources, though at the moment, the server side monitoring is limited to UNIX hosts with nmon.

Installing Hailstorm

Software Prerequisites

Setup

🎉 Download the latest Hailstorm release.

Every release consists of three files:

hailstorm-web/docker-compose.yml
hailstorm-cli/docker-compose.yml
hailstorm-cli/Makefile

The release is a tar+gz file. Unpack to any directory on your filesystem.

Web Interface

This is the recommended approach for most users.

To start the web interface:

$ cd /path/to/unpacked/release
$ cd hailstorm-web
$ docker-compose up

This starts downloading the containers, and starting them one by one. This should complete within a minute. You will get output similar to this towards the end.

hailstorm-api_1    | The signal INFO is in use by the JVM and will not work correctly on this platform
hailstorm-api_1    | Puma starting in single mode...
hailstorm-api_1    | * Version 4.3.3 (jruby 9.1.17.0 - ruby 2.3.3), codename: Mysterious Traveller
hailstorm-api_1    | * Min threads: 0, max threads: 16
hailstorm-api_1    | * Environment: development
hailstorm-api_1    | * Listening on tcp://0.0.0.0:8080
hailstorm-api_1    | Use Ctrl-C to stop
web_1              | 2020/04/04 17:03:23 Received 200 from http://hailstorm-api:8080
web_1              | 2020-04-04 17:03:23: (server.c.1521) server started (lighttpd/1.4.54)

The Received 200 from http://... message indicates that the system is available.

Open browser to http://localhost:8080

You should see a wizard to create a new project.

To bring down the container setup, exit with CTRL+C if docker-compose is running in foreground. If you daemonized it, or want to clean up completely, execute: docker-compose down.

Daemonized Mode

To start the web interface as a daemon:

$ docker-compose up -d

It takes up to sixty seconds for the system to initialize completely. If you get a connection reset message in the browser, wait for a few seconds and refresh the browser. While you are waiting, you can see the logs with docker-compose logs -f.

CLI Interface

The CLI is meant for advanced users who need low level customization and/or server monitoring.

Additional Prerequisite

make - Available on most Linux distributions and MacOSX. For Windows, try installing with Chocolatey: choco install make.

Running the CLI

$ cd /path/to/unpacked/release
$ cd hailstorm-cli
$ docker-compose up -d
$ make

The CLI will wait for the docker containers to be available. It should take less than a minute. When the CLI starts, it shows a prompt:

hailstorm@ab7ecdeac102:/hailstorm$

The current directory on the host is mapped to /hailstorm in the container. Any files saved to this location in the container will persist across container restarts.

Create a CLI project

hailstorm@ab7ecdeac102:/hailstorm$ hailstorm shopping_cart

Truncated output...

    wrote shopping_cart/config/environment.rb
    wrote shopping_cart/config/database.properties
    wrote shopping_cart/config/progressive.rb
    wrote shopping_cart/config/boot.rb

Done!

Start the CLI

hailstorm@ab7ecdeac102:/hailstorm$ cd shopping_cart
hailstorm@ab7ecdeac102:/hailstorm/shopping_cart$ ./script/hailstorm

Welcome to the Hailstorm (version 5.0.0) shell.
Type help to get started...
hs > _

To bring down the containers, exit the CLI container, and execute on the host: docker-compose down.

License

The source code is distributed under the MIT License.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/3pillarlabs/hailstorm-sdk/. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

Code of Conduct

Everyone interacting in the Hailstorm project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.

About this project

Hailstorm is developed, and maintained by open source volunteers at 3Pillar Global. Hailstorm is not an official 3Pillar Global product (experimental or otherwise), but 3Pillar Global owns the code.

hailstorm-sdk's People

Contributors

Stargazers

Watchers

Forkers

aptimyze isabella232

hailstorm-sdk's Issues

Use ~/.ssh keys if available

Increase unit test coverage to >= 80%

Add hailstorm-api

API accepts commands and returns output as the CLI would do.

Automatic test site setup (with AWS Lambda?)

Unexpected Exception on setup command

Stacktrace -

ERROR 2018-05-30 18:04:19,311 main /vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:68:in `handle_error' - undefined method `jmeter' for #<Hailstorm::Model::Project:0x37fef327>
DEBUG 2018-05-30 18:04:19,312 main /vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:69:in `handle_error' - 
org/jruby/RubyBasicObject.java:1653:in `method_missing'
/usr/local/rvm/gems/jruby-9.1.7.0@hailstorm/gems/activemodel-4.2.8/lib/active_model/attribute_methods.rb:433:in `method_missing'
/vagrant/hailstorm-gem/lib/hailstorm/cli/view_renderer.rb:22:in `render_setup'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:36:in `execute_method_args'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:26:in `interpret_execute'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:44:in `process_cmd_line'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:122:in `start_cmd_loop'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:38:in `process_commands'
./script/hailstorm:13:in `<main>'

Merge regions from API and AMI map from AWS SDK

Clear log in web console

Rework how failures are handled in start and stop. Need a "retry" or "auto-retry" capability.

For example, sometimes logs fail to download from the Web interface (exception is in a thread making it hard to track down); but there's no apparent issue.

Other times, when an operation has failed across all instances, it should be regarded as error (at the end) instead of continuing with the next step.

[INFO] Provisioning 2 Hailstorm::Model::DataCenter machines...
[WARN] Either JMeter is not installed or required version '5.2.1' is not available on one of one of the machines specified. Please make sure:
 1) Required JMeter version is installed
 2) JMETER_HOME and required path variable are set and accessible

[WARN] Either JMeter is not installed or required version '5.2.1' is not available on one of one of the machines specified. Please make sure:
 1) Required JMeter version is installed
 2) JMETER_HOME and required path variable are set and accessible
[INFO] Setting up targets...

In the above example, Hailstorm should not try to set up targets.

Integrate container structure tests

https://github.com/GoogleContainerTools/container-structure-test

Generated Makefile for CLI release should look for matching network instead of prefixing directory name

Format rspec output to show logs only for failed examples

Replace references to Hailstorm.root to project.root_path in core libraries

Indicate that a project has live resources (how to tell if a project is terminated or not)

Show notifications in notification area

Update Wiki for releases/5.0.0

Release - fix release name and asset name

Remove database and model access from hailstorm-cli spec

Indicate hourly price has updated when the region is changed on AWS cluster page

Update default JMeter to latest version

Modify unit tests
Modify integration test to create a new AMI
AWS load generation tests should pass

Add integration tests to CI

Speed up Travis build

Ability to use AWS spot instances, blocks and fleet as load agents

Check parallelism of data center operations (upload scripts, execution)

Max size of log console (2000 lines)

Error while importing results from default location

hs > results import
[ERROR] wrong number of arguments (0 for 2)

ERROR 2019-09-24 10:12:57,566 main hailstorm-cli/lib/hailstorm/controller/cli.rb:78:in `handle_error' - wrong number of arguments (0 for 2)
DEBUG 2019-09-24 10:12:57,567 main hailstorm-cli/lib/hailstorm/controller/cli.rb:79:in `handle_error' - 
hailstorm-cli/lib/hailstorm/cli/cmd_executor.rb:33:in `find_files'
hailstorm-cli/lib/hailstorm/cli/cmd_executor.rb:33:in `execute_method_args'
hailstorm-cli/lib/hailstorm/cli/cmd_executor.rb:26:in `interpret_execute'
hailstorm-cli/lib/hailstorm/controller/cli.rb:54:in `process_cmd_line'
hailstorm-cli/lib/hailstorm/controller/cli.rb:132:in `start_cmd_loop'
hailstorm-cli/lib/hailstorm/controller/cli.rb:42:in `process_commands'
./script/hailstorm:13:in `<main>'

Upload to rubygems

Enable hailstorm-api to handle multiple projects

Refactor multi table inheritance in clusters.

Update JMeter plan in a project

Web - show errors received in log stream as error messages

Exception on 'show' or 'show active' or 'show [arg] active'

hs > show
[ERROR] undefined method `active' for #<Hailstorm::Model::Cluster::ActiveRecord_Associations_CollectionProxy:0x3421debd>
Did you mean?  acts_like?
[DEBUG] 
org/jruby/RubyBasicObject.java:1653:in `method_missing'
/usr/local/rvm/gems/jruby-9.1.7.0@hailstorm/gems/activerecord-4.2.8/lib/active_record/relation/delegation.rb:136:in `method_missing'
/usr/local/rvm/gems/jruby-9.1.7.0@hailstorm/gems/activerecord-4.2.8/lib/active_record/relation/delegation.rb:99:in `method_missing'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:52:in `block in query_relations_map'
org/jruby/RubyArray.java:1733:in `each'
org/jruby/RubyEnumerable.java:936:in `inject'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:51:in `query_relations_map'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:45:in `show'
/vagrant/hailstorm-gem/lib/hailstorm/cli/cmd_executor.rb:22:in `interpret_execute'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:44:in `process_cmd_line'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:122:in `start_cmd_loop'
/vagrant/hailstorm-gem/lib/hailstorm/controller/cli.rb:38:in `process_commands'
./script/hailstorm:13:in `<main>'

Remove references to static methods from hailstorm-gem in Hailstorm module
Remove direct file system access from hailstorm-gem such that is agnostic of the project structure, and the storage mechanism.
CLI should create new project in single database, and not create a new database per project
spec related to CLI should be in hailstorm-cli
Align transfer_file / set_defaults as validation
Check for memoization opportunities in workspace
features related to CLI should be in hailstorm-cli
expand scope of static code analysis and CI to CLI

Some data is not refreshed or loaded after setup

With a new (only happens on a new project) CLI project:

Edit config/environment.rb and add

Hailstorm.application.config do |config|
  config.jmeter do |jmeter|
    jmeter.properties do |props|
      props['NumUsers'] = 100 
      props['Duration'] = 600 
    end 
  end 
end

Open Hailstorm CLI and execute setup, then show, and show all

hs > setup
# output clipped
hs > show jmeter
Active JMeter Plans
===============================================================================
# Properties listed here
hs > show jmeter all
All JMeter Plans
===============================================================================
**End of output - the JMeter properties are not shown**

Exit the CLI, and reopen it. Execute show jmeter all, and the properties are displayed.

Web - move to open project dialog if there's only one project. Actual screen depends on project status. If incomplete - it opens in wizard else the dashboard

==> dev: Running provisioner: hailstorm_web (shell)...
    dev: Running: /var/folders/j2/kpb8q5q568g83tf4mzphgl6w4__sd2/T/vagrant-shell20180213-12751-1rpkxi0.sh
    dev: stdin: is not a tty
    dev: Reading package lists...
    dev: Building dependency tree...
    dev: Reading state information...
    dev: libmysqlclient-dev is already the newest version.
    dev: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
    dev: Reading package lists...
    dev: Building dependency tree...
    dev: Reading state information...
    dev: nodejs is already the newest version.
    dev: 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
    dev: Required ruby-2.1.4 is not installed.
    dev: To install do: 'rvm install "ruby-2.1.4"'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/bin/bundle
    dev: Required ruby-2.1.4 is not installed.
    dev: To install do: 'rvm install "ruby-2.1.4"'
    dev: Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
    dev: installing your bundle as root will break this application for all non-root
    dev: users on this machine.
    dev: Fetching gem metadata from https://rubygems.org/
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: Fetching gem metadata from https://rubygems.org/
    dev: .
    dev: Resolving dependencies...
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: .
    dev: Using rake 12.0.0
    dev: Using concurrent-ruby 1.0.5 (java)
    dev: Using aasm 4.12.1
    dev: Using i18n 0.8.6
    dev: Using json 1.8.6 (java)
    dev: Using minitest 5.10.3
    dev: Using thread_safe 0.3.6 (java)
    dev: Using tzinfo 1.2.3
    dev: Using activesupport 4.1.6
    dev: Using builder 3.2.3
    dev: Using erubis 2.7.0
    dev: Using actionview 4.1.6
    dev: Using rack 1.5.5
    dev: Using rack-test 0.6.3
    dev: Using actionpack 4.1.6
    dev: Using mime-types-data 3.2016.0521
    dev: Using mime-types 3.1
    dev: Using mail 2.6.6
    dev: Using actionmailer 4.1.6
    dev: Using activemodel 4.1.6
    dev: Using arel 5.0.1.20140414130214
    dev: Using activerecord 4.1.6
    dev: Using execjs 2.7.0
    dev: Using autoprefixer-rails 7.1.2.3
    dev: Using bundler 1.16.1
    dev: Using climate_control 0.2.0
    dev: Using cocaine 0.5.8
    dev: Using connection_pool 2.2.1
    dev: Using thor 0.19.4
    dev: Using railties 4.1.6
    dev: Using font-awesome-rails 4.3.0.0
    dev: Using hike 1.2.3
    dev: Using multi_json 1.12.1
    dev: Using jbuilder 2.6.4
    dev: Using jquery-rails 3.1.4
    dev: Fetching kgio 2.11.0
    dev: Installing kgio 2.11.0 with native extensions
    dev: Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
    dev: 
    dev: current directory:
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/kgio-2.11.0/ext/kgio
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/bin/jruby -r
    dev: ./siteconf20180213-29667-6i3ptq.rb extconf.rb
    dev: checking for CLOCK_MONOTONIC in time.h... RuntimeError: The compiler failed to
    dev: generate an executable file.
    dev: You have to install development tools first.
    dev: 
    dev: try_do at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:456
    dev: block in try_compile at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:571
    dev: with_werror at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:524
    dev: try_compile at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:571
    dev: macro_defined? at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:839
    dev: block in have_macro at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:974
    dev: block in checking_for at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:942
    dev: block in postpone at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:350
    dev: open at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:320
    dev: block in postpone at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:350
    dev: open at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:320
    dev: postpone at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:346
    dev: checking_for at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:941
    dev: have_macro at
    dev: /usr/local/rvm/rubies/jruby-9.1.7.0/lib/ruby/stdlib/mkmf.rb:973
    dev:                  <main> at extconf.rb:5
    dev: *** extconf.rb failed ***
    dev: Could not create Makefile due to some reason, probably lack of necessary
    dev: libraries and/or headers.  Check the mkmf.log file for more details.  You may
    dev: need configuration options.
    dev: 
    dev: Provided configuration options:
    dev: 	--with-opt-dir
    dev: 	--without-opt-dir
    dev: 	--with-opt-include
    dev: 	--without-opt-include=${opt-dir}/include
    dev: 	--with-opt-lib
    dev: 	--without-opt-lib=${opt-dir}/lib
    dev: 	--with-make-prog
    dev: 	--without-make-prog
    dev: 	--srcdir=.
    dev: 	--curdir
    dev: 	--ruby=/usr/local/rvm/rubies/jruby-9.1.7.0/bin/jruby
    dev: 
    dev: To see why this extension failed to compile, please check the mkmf.log which can
    dev: be found here:
    dev: 
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/extensions/universal-java-1.8/2.3.0/kgio-2.11.0/mkmf.log
    dev: 
    dev: extconf failed, exit code 1
    dev: 
    dev: Gem files will remain installed in
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/kgio-2.11.0 for inspection.
    dev: Results logged to
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/extensions/universal-java-1.8/2.3.0/kgio-2.11.0/gem_make.out
    dev: 
    dev: An error occurred while installing kgio (2.11.0), and Bundler cannot continue.
    dev: Make sure that `gem install kgio -v '2.11.0'` succeeds before bundling.
    dev: 
    dev: In Gemfile:
    dev:   unicorn was resolved to 5.3.0, which depends on
    dev:     kgio
    dev: rake aborted!
    dev: Bundler::GemNotFound: Could not find kgio-2.11.0 in any of the sources
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/spec_set.rb:88:in `block in materialize'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/spec_set.rb:82:in `materialize'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/definition.rb:170:in `specs'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/definition.rb:237:in `specs_for'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/definition.rb:226:in `requested_specs'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/runtime.rb:108:in `block in requested_specs'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/runtime.rb:20:in `setup'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler.rb:107:in `setup'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/gems/bundler-1.16.1/lib/bundler/setup.rb:20:in `<main>'
    dev: /usr/local/lib/hailstorm-web/config/boot.rb:1:in `<main>'
    dev: /usr/local/lib/hailstorm-web/config/boot.rb:4:in `<main>'
    dev: /usr/local/lib/hailstorm-web/config/application.rb:1:in `<main>'
    dev: /usr/local/lib/hailstorm-web/config/application.rb:1:in `(root)'
    dev: /usr/local/lib/hailstorm-web/Rakefile:1:in `(root)'
    dev: /usr/local/lib/hailstorm-web/Rakefile:4:in `block in (root)'
    dev: /usr/local/rvm/gems/jruby-9.1.7.0/bin/jruby_executable_hooks:15:in `<main>'
    dev: (See full trace by running task with --trace)
    dev: start: Job failed to start

CLI: Generate config/log4j.xml for new projects

When a new project is created, create a config/log4j.xml file for letting the developer control the application logging.