gongo / turnip_formatter Goto Github PK

View Code? Open in Web Editor NEW

34.0 7.0 26.0 1.96 MB

A pretty formatter for Turnip

Home Page: http://gongo.hatenablog.com/entry/2013/04/18/203426

License: MIT License

Ruby 81.16% JavaScript 1.19% CSS 0.03% HTML 8.65% Logos 0.20% Gherkin 4.10% SCSS 4.67%

turnip rspec-formatter

turnip_formatter's Introduction

RSpec::Core::Formatters::TurnipFormatter

TurnipFormatter is a pretty custom formatter for Turnip.

Requirements

Ruby
- 2.2.0 or later

Installation

RubyGems

$ gem install turnip_formatter

Bundler

Add this line to your application's Gemfile:

group :test do
  gem 'turnip_formatter'
end

And then execute:

$ bundle install

Usage

Run this command.

$ rspec -r turnip_formatter --format RSpecTurnipFormatter --out report.html

Example

See:

Feature and Steps
- https://github.com/gongo/turnip_formatter/tree/master/example
Report
- http://gongo.github.io/turnip_formatter/examples/report.html

Configuration

TurnipFormatter.configure do |config|
  config.title = 'My Report'                                 # <h1>My Report</h1>
  config.add_stylesheet File.dirname(__FILE__) + '/foo.css'  # <style>..</style>
  config.add_javascript File.dirname(__FILE__) + '/bar.js'   # <script>..</script>

  config.add_stylesheet 'http://cdn.example.com/hoge.css'    # <link rel="..">
  config.add_javascript '//cdn.example.com/fuga.js'          # <script src="..">
end

Add-on

Gnawrnip
- Gnawrnip is a TurnipFormatter Add-on that provides put a screen shot to report use Capybara
- https://github.com/gongo/gnawrnip

License

see LICENSE.txt

turnip_formatter's People

Contributors

Stargazers

Watchers

turnip_formatter's Issues

Tag Statistics

Pakuri of https://github.com/masterthought/jenkins-cucumber-jvm-reports-plugin-java

Background がある場合のシナリオでエラーステップの番号がずれる

Background:
    When login # (1)

Scenario:
    When type 'name' # (2)
     And click 'button' # (3)

みたいなシナリオがあった場合、もし (2) でエラーになると

期待値: When type 'name' が赤く表示される
現状: When login が赤く表示される

現状、ext/turnip/rspec.rb で steps を with_index で回してそれをステップ番号としているが、
backgrounds の分を加算していなかったため、エラーステップ番号が 2 になるべきなのに
Scenario を基点として 1 が返ってきてしまってた

JRuby support, undefined method `[]' in turnip_formatter/scenario.rb

Is JRuby supposed to be fully supported?

I'm running into several issues. Here's what I've done so far..

$ bundle exec rspec -r turnip_formatter --format RSpecTurnipFormatter --out report.html spec/integration/

Specs are running and at the end I see:

NoMethodError: undefined method `[]' for nil:NilClass
     feature_name at /.rvm/gems/jruby-1.7.10/gems/turnip_formatter-0.2.11/lib/turnip_formatter/scenario.rb:62
        print_out at /.rvm/gems/jruby-1.7.10/gems/turnip_formatter-0.2.11/lib/turnip_formatter/printer/tab_feature_statistics.rb:13
             each at org/jruby/RubyArray.java:1613
         group_by at org/jruby/RubyEnumerable.java:1724
        print_out at /.rvm/gems/jruby-1.7.10/gems/turnip_formatter-0.2.11/lib/turnip_formatter/printer/tab_feature_statistics.rb:13
           result at (erb):53
             eval at org/jruby/RubyKernel.java:1119
           result at /.rvm/rubies/jruby-1.7.10/lib/ruby/1.9/erb.rb:838
  render_template at /.rvm/gems/jruby-1.7.10/gems/turnip_formatter-0.2.11/lib/turnip_formatter/printer.rb:11
....

report.html is not generated.

When I patched TurnipFormatter::Scenario#feature_name to rescue that error and return "Unknown" text in that case, report.html is generated. All tabs looks good besides Steps, which contains various TurnipFormatter::Scenario::NotExistStepsInformationError or NoExistPendingStepInformationError errors rendered in html.

I've cloned repository and run bundle; rspec. I've got 4 errors when running on JRuby 1.7.10:

`Finished in 1.33 seconds
49 examples, 4 failures

Failed examples:

rspec ./spec/turnip_formatter/step/pending_spec.rb:54 # TurnipFormatter::Step::Failure#attention? 
rspec ./spec/turnip_formatter/step/pending_spec.rb:59 # TurnipFormatter::Step::Failure#status 
rspec ./spec/turnip_formatter/step_template/exception_spec.rb:18 # TurnipFormatter::StepTemplate::Exception.build 
rspec ./spec/turnip_formatter/step_template/source_spec.rb:14 # TurnipFormatter::StepTemplate::Source.build should match "require 'turnip_formatter/step_template/source'"

シナリオに Permalink

リーダー said:

「このシナリオ失敗してるけど?」って IRC に貼りやすくなるんじゃないかな

add_template 使いやすくしよう

今、module TurnipFormatter::Template 内で Step class 定義しないといけないし
クラス指定できたらいいよねっていうかんあつーか使いづらいので直します

Change flow that create report (test -> json -> html)

Now

Run tests
- Continues to have the test results in RSpec::Core::Formatters::TurnipFormatter.{passed,failed,pending}_scenarios until end of all tests.
Create report

Problem above:

Can not cope with distributed processing (multiple report files are created.)
- parallel_tests
- rrrspec
Memory usage
- with gnawrnip

Run tests
Create test_result.json each test
- e.g. sample_test_1.json sample_test_3.json sample_test_3.json ..
Report was created by fusing these files

英語飽きた

今まではテスト全て完了するまでは結果持ち続けててここで最終的に1個の HTML が作成される。これはこれでいいんだけど

RRRSpec や parallel tests みたいに分散実行されるやつだと、そのプロセスやマシン毎にレポートファイルができる。
- parallel_rspec -n 10 とかだと report1 〜 report10 までできるみたいなイメージ
gnawrnip 使ってると、画像ファイル開きぱなしという仕様のため、fd 枯渇したりいろいろ圧迫してよろしくない

個々はそれほどでもないんだけど、やっぱり「レポート一個にしたい」というのもあるので、各プロセス上で HTML が作成されるのはよろしくない。

というわけで

$ rspec

実行すると .turnip_formatter/ 以下に各テスト結果が json で落ちていく。
全部のテストが終わったらそれら json を一個ずつ読んで HTML ファイルに追記していくみたいなイメージ。

1ホスト環境だと今迄と変わらないけど、分散環境にそれぞれできた json ファイルをかき集めることで、ちゃんと1個のレポートファイルとしてできるのではないか。ねむい。

で、rrrspec とか使って各マシン上に json ファイルができる場合、それらを「かきあつめる」という処理も turnip formatter に任せるのはさすがによろしくないので

$ rrrspec start
$ rakeup.sh # テスト実行された各サーバから json ファイルをもってきて ./.turnip_formatter/ 以下に配置するスクリプト自作しよう
$ turnip_formatter generate # ./.turnip_formatter/ 以下にあるやつを使ってHTML生成

ということができる CLI command を一個用意するとか。

cucumber compatible json output

How does turnip_formatter compare to existing report builders:
eg. this one for jenkins:

https://github.com/jan-molak/jenkins-cucumber-jvm-reports-plugin-java

And can this gem be used to produce cucumber compatible json output?

ie. of the form:
https://www.relishapp.com/cucumber/cucumber/docs/json-output-formatter#one-feature,-one-passing-scenario,-one-failing-scenario

which could then be plugged in to the cucumber report builders?

Or on the other hand can this gem generate reports in jenkins based on the cucumber json created by some other job?

execjs の runtime がない言われる

/home/gongo/Develop/working/webmagic/test/feature/vendor/bundle/ruby/1.9.1/gems/execjs-1.4.0/lib/execjs/runtimes.rb:51:
in `autodetect': Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available
 runtimes. (ExecJS::RuntimeUnavailable)

therubyracer を使おう

Feature Statistics

Pakuri of https://github.com/masterthought/jenkins-cucumber-jvm-reports-plugin-java

RSpec::Core::Formatters::TurnipFormatter って書くのめんどい

https://github.com/sj26/rspec_junit_formatter/blob/master/lib/rspec_junit_formatter.rb#L8 をパクって

RSpecTurnipFormatter = RSpec::Core::Formatters::TurnipFormatter

とかしておこう

scss 見直そう

Encoding::CompatibilityError on language:ja feature file

こんにちは。日本語で Issue 書いていいのか分かりませんが・・。

Turnipのfeatureファイルで日本語書式にするため冒頭に# language: jaと書くと、レポート生成時に Encoding::CompatibilityError エラーが出力されます。

[feature file]

# encoding: utf-8
# language: ja
...

[error]

/turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/template/index.slim:87:in `join': incompatible character encodings: US-ASCII and UTF-8 (Encoding::CompatibilityError)
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/template/index.slim:87:in `block in singleton class'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/template/index.slim:-6:in `instance_eval'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/template/index.slim:-6:in `singleton class'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/template/index.slim:-8:in `__tilt_70112817494420'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.1/lib/tilt/template.rb:155:in `call'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.1/lib/tilt/template.rb:155:in `evaluate'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/tilt-2.0.1/lib/tilt/template.rb:96:in `render'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/printer.rb:11:in `render_template'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/turnip_formatter/printer/index.rb:14:in `print_out'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/rspec/core/formatters/turnip_formatter.rb:34:in `output_html'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/turnip_formatter-0.3.4/lib/rspec/core/formatters/turnip_formatter/for_rspec3.rb:19:in `dump_summary'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:184:in `block in notify'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:183:in `each'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:183:in `notify'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:162:in `block in finish'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:170:in `close_after'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:151:in `finish'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/reporter.rb:79:in `report'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:113:in `run_specs'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:89:in `run'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:73:in `run'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/lib/rspec/core/runner.rb:41:in `invoke'
        from /turnip/vendor/bundle/ruby/2.2.0/gems/rspec-core-3.3.2/exe/rspec:4:in `<top (required)>'
        from /turnip/vendor/bundle/ruby/2.2.0/bin/rspec:23:in `load'
        from /turnip/vendor/bundle/ruby/2.2.0/bin/rspec:23:in `<main>'

日本語でfeatureファイル書きたいのです・・！
http://magazine.rubyist.net/?0042-FromCucumberToTurnip

よろしくご確認お願いします。

Passed, Failed, Pending を選択して表示

HTMLFormatter のパクリ

Exception Causes the Entire Report Not to Generate

So I've had tests fail due to Watir Not Being able to find an element, that has caused some cascading failures in my before/after hooks.

Below is an example of the failure:

3.1) Failure/Error: @site.create_strategy.vhs_select.select_value @vhs_id

      Watir::Exception::UnknownObjectException:
        unable to locate element, using {:id=>"VHS_ID", :tag_name=>"select"}
      # ./spec/support/steps/insight/strategy/create_strategy_steps.rb:28:in `block in <module:CreateStrategySteps>'
      # ./spec/isolation/insight/sharing/create_replace_strategy_vhs.feature:18:in `run_step'
      # ./spec/isolation/insight/sharing/create_replace_strategy_vhs.feature:9:in `block (4 levels) in run_feature'
      # ./spec/isolation/insight/sharing/create_replace_strategy_vhs.feature:8:in `each'
      # ./spec/isolation/insight/sharing/create_replace_strategy_vhs.feature:8:in `block (3 levels) in run_feature'
      # ./spec/isolation/insight/sharing/create_replace_strategy_vhs.feature:18:in `I Select a VHS Setting'

 3.2) Failure/Error: login.log_out.click

      Selenium::WebDriver::Error::UnknownError:
        unknown error: Element is not clickable at point (1867, 15). Other element would receive the click: <div id="logopanelgrey" class="greyedoutmodal" style="height: 78px; display: block;"></div>
          (Session info: chrome=49.0.2623.112)
          (Driver info: chromedriver=2.14.313457 (3d645c400edf2e2c500566c9aa096063e707c9cf),platform=Windows NT 6.1 SP1 x86_64)
      # ./lib/sites/insight/flows/logout.rb:8:in `logout_flow'
      # ./spec/spec_helper.rb:225:in `block (2 levels) in <top (required)>'

When the report tries to generate it creates the following error and the report fails to generate:
C:/Ruby200/lib/ruby/gems/2.0.0/gems/turnip_formatter-0.5.0/lib/turnip_formatter/scenario/failure.rb:36:in failed_line_number': undefined methodfind' for nil:NilClass (NoMethodError)
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/turnip_formatter-0.5.0/lib/turnip_formatter/scenario/failure.rb:25:in validation' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/turnip_formatter-0.5.0/lib/turnip_formatter/scenario/base.rb:15:ininitialize'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/turnip_formatter-0.5.0/lib/rspec/core/formatters/turnip_formatter.rb:49:in new' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/turnip_formatter-0.5.0/lib/rspec/core/formatters/turnip_formatter.rb:49:inexample_failed'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/reporter.rb:189:in block in notify' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/reporter.rb:188:ineach'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/reporter.rb:188:in notify' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/reporter.rb:140:inexample_failed'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example.rb:452:in finish' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example.rb:410:infail_with_exception'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:550:in block in run' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:591:ineach'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:591:in for_filtered_examples' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:595:inblock in for_filtered_examples'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:593:in each' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:593:infor_filtered_examples'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:550:in rescue in run' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/example_group.rb:556:inrun'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:119:in block (3 levels) in run_specs' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:119:inmap'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:119:in block (2 levels) in run_specs' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/configuration.rb:1680:inwith_suite_hooks'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:118:in block in run_specs' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/reporter.rb:77:inreport'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:117:in run_specs' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:93:inrun'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:78:in run' from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/lib/rspec/core/runner.rb:45:ininvoke'
from C:/Ruby200/lib/ruby/gems/2.0.0/gems/rspec-core-3.4.4/exe/rspec:4:in `

I'm not sure what the expected behavior is in this case.

#15 Tag Statistics
#16 Feature Statistics
#17 Speed ranking

Failure:
RSpec::Core::MultipleExceptionError
Backtrace:

But this class has @all_exceptions property with a array of all exceptions.
It would be nice print all exceptions in a html report instead RSpec::Core::MultipleExceptionError.

save_screenshot でエラー時の画像を埋め込む

※ これは Turnip というよりは Capybara の機能なので、実装するとしても別 gem

RSpec.configure do |config|
  config.after :each do
    if example.exception
      page.save_screenshot('/tmp/feature_error_' + Time.now.strftime("%Y%m%d%H%M%S") + '.png')
    end
  end
end

とすると、エラー時のスクリーンショットを /tmp 以下に保存してくれる
この機能を利用して、エラースクリーンショットを base64 で report.html に埋め込むことが可能なのでは？

Speed ranking

ステップの実行時間ランキング表示

各シナリオを折りたためるように

リーダー said:

折りたためるといいですね

Add plain text format

HTML で出力するほどでもないけど、RSpec 標準の text format だと見づれーわって時があるので、
なんかいい感じで step を載せるようなフォーマット

ascii table みたいな感じでやるものいいですね。