Code Monkey home page Code Monkey logo

testrail-rspec's Introduction

testrail-rspec

Gem Version

Sync Rspec test results with your testrail suite. Discover an example with Capybara in this gem source.

Features

  • Update test results in the existing test run
  • Create dynamic test run and update test results in it
  • Update multi-testrail cases from a single automation scenario
  • Delete/clean all the existing test runs in a project's suite before test run
  • Skip specific test-runs from deletion, when clean_testrun is set true
  • Static status comments on all the scenarios
  • Support for RSpec shared examples
  • Disable testrail-rspec execution on-demand
  • Support for environment variables to pass testrail config values

Installation

Add this line to your application's Gemfile:

gem 'testrail-rspec'

And then execute:

$ bundle

Or install it yourself as:

$ gem install testrail-rspec

Import the library in your spec_helper.rb file

require 'testrail-rspec'

Usage outline

Update one case at a time

Prefix TestRail Case ID on start of your rspec scenario; say, C845

  describe 'Verify Google Home Page' do
    
    scenario 'C845 verify the Google home page' do
      expect(page).to have_content('Google')
    end
  
    scenario 'C847 verify the Google home page to fail' do
      expect(page).to have_content('Goo gle')
    end
    
    scenario 'C853 verify the Google home page to skip' do
      skip "skipping this test"
    end
  
  end

Update multi-cases at a time

Prefix multiple testrail case id(s) on start of your rspec scenario; say, C845 C845 your scenario description

  describe 'Verify Google Home Page' do
    
    scenario 'C847 C846 C845 verify the Google home page' do
      expect(page).to have_content('Google')
    end
  
    scenario 'C848 C849 verify the Google home page to fail' do
      expect(page).to have_content('Goo gle')
    end
    
    scenario 'verify the Google home page to fail' do
      expect(page).to have_content('Goo gle')
    end
    
  end

Use context blocks to update cases

Context blocks with testrail case id(s) to make use of Shared Examples

shared_examples 'log in' do
  it 'logs in'
end

shared_examples 'log out' do
  it 'logs out'
end

describe 'User login' do
  context 'Regular user' do
    let(:user) { create(:regular_user) }
    context 'C845 single case' do
      include_examples 'log in'
    end

    context 'C847' do
      include_examples 'log out'
    end
  end

  context 'Admin user' do
    let(:user) { create(:admin_user) }
    context 'C850 C853 multi cases' do
      include_examples 'log in'
    end

    context 'nothing to update in test-rail' do
      include_examples 'log out'
    end
  end
end

Configuration

  1. Create a config file, testrail_config.yml in the project's parent folder

  2. Enter the testrail details based on demand

  3. To execute tests against the existing Test Run,

    testrail:
      url: https://your_url.testrail.io/
      user: [email protected]
      password: ******
      run_id: 111

    Here, run_id is the dynamically generated id from your testrail account (say, run_id: 111)

  4. To create a dynamic Test Run and to update results,

    testrail:
      url: https://your_url.testrail.io/
      user: [email protected]
      password: ******
      project_id: 10
      suite_id: 110

    Here, project_id and suite_id are the dynamically generated id from your testrail account; run_id is optional in this case.

  5. To delete all test-runs before execution,

    testrail:
      url: https://your_url.testrail.io/
      user: [email protected]
      password: ******
      clean_testrun: true
      project_id: 10
      suite_id: 110

    Set, clean_testrun: false if you don't want to clean the existing test runs; but this keyword is optional.

  6. Skip specific test-runs from deletion: set clean_testrun: true & skip_testrun_ids: value, ...

    testrail:
      url: https://your_url.testrail.io/
      user: [email protected]
      password: ******
      clean_testrun: true
      skip_testrun_ids: 473, 475
      project_id: 10
      suite_id: 110

    Here, skip_testrun_ids: value is optional.

  7. Disable testrail-rspec execution: set allow: yes

    testrail:
      url: https://your_url.testrail.io/
      user: [email protected]
      password: ******
      run_id: 111
      allow: no

    Here, allow: yes is optional.

  8. Use Environment variables to pass testrail config values

    testrail:
      url: ENV['URL']
      user: ENV['TESTRAIL_USER']
      password: ENV['TESTRAIL_PASSWORD']
      run_id: ENV['RUN_ID']
      clean_testrun: false
      project_id: 10
      suite_id: 110

    Example, rake ./demo_spec.rb [email protected] TESTRAIL_PASSWORD=****** RUN_ID=564 URL=https://your_url.testrail.io/

Hooks

Update the results through Hooks on end of each test

config.after(:each) do |scenario|
    TestrailRSpec::UpdateTestRails.new(scenario).upload_result
end

Is there any demo available for this gem?

Yes, you can use this capybara demo as an example, https://github.com/prashanth-sams/testrail-rspec

bundle install
rake test

testrail-rspec's People

Contributors

erikdahlinghaus avatar prashanth-sams avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

testrail-rspec's Issues

Use full_description instead of description to capture tags on context blocks

We are using shared_examples blocks which are difficult to tag because if we tag the inner examples, they will be marked multiple times and potentially on the wrong cases.

I think using full_description instead of description at update-testrails.rb#L25 will fix this issue because it will capture the tags on surrounding context blocks.

For example, given a set of TestRail cases

  • C1232: A regular user can log in
  • C1233: A regular user can log out
  • C1234: An admin user can log in
  • C1235: An admin user can log out

This rspec file is difficult to tag correctly.

shared_examples 'log in / log out' do
  it 'logs in'
  it 'logs out'
end

describe 'basic user stuff' do
  context 'A regular user can' do
    include_examples 'log in / log out'
  end

  context 'An admin user can' do
    include_examples 'log in / log out'
  end
end

With a little refactoring and my proposed change

shared_examples 'log in' do
  it 'logs in'
end

shared_examples 'log out' do
  it 'logs out'
end

describe 'basic user stuff' do
  context 'A regular user can' do
    context 'C1232' do
      include_examples 'log in'
    end

    context 'C1233' do
      include_examples 'log out'
    end
  end

  context 'An admin user can' do
    context 'C1234' do
      include_examples 'log in'
    end

    context 'C1235' do
      include_examples 'log out'
    end
  end
end

we can capture all of the functionality in the right cases.

Dynamic test run name with timestamp

Hello,

I want to create the "test run" dynamically, I am able to do that as per your instruction but in my case, it always creates with a static name "master", could please help me create a test run with a customized name? like "Regression suite - 13/07/2021 min-hr"

upload_result error

I'm getting this error:

Failure/Error: TestrailRSpec::UpdateTestRails.new(scenario).upload_result
TypeError: no implicit conversion of String into Integer

Weird thing is the test-case I'm running passes when run without TestrailRspec, and shows up as passed in TestRail

Edit: I was able to get it working with rspec-retry, looks like it was only happening on first connection attempt, worked on subsequent attempts. It still happens pretty consistently on that first connection, though.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.