Code Monkey home page Code Monkey logo

testerbot-public's Introduction

testerbot

Automatic Front-End Testing

Maintainability Test Coverage CircleCI PRs Welcome npm version npm NSP Status license Join the chat at https://gitter.im/theoutlander/testerbot

Purpose

During development, there isn't enough time to write tests, so our only option is manual testing. We don't need a reminder of how inefficient, time-consuming, error-prone and boring that can get. When we get around to writing test-automations, we spend a lot of time writing tests from scratch. However, most of the test-cases for testing web-applications can be automated and reused.

We created Testerbot to automatically test web-applications during the development process via Puppeteer which talks to Headless Google Chrome.

The basic framework is ready and we have implemented a few test-cases, but we need help in adding more reusable test-cases.

Note: I started this project several months ago and built a SaaS product, but realized that an open-source test automation package will help catch issues quickly during development. I recently stumbled upon https://frontendchecklist.io/ and created github issues to automate as many of those tests as possible into Testerbot.

Install

npm i testerbot -g

NPM

Quick Start

Running testerbot without any arguments defaults to http://localhost:3000

testerbot 

Specify a url via the --url argument

testerbot --url http://localhost:5000

Specify multiple comma seoparated urls via the --urls argument

testerbot --urls http://localhost:5000,http://localhost:5000/toc.html

Here is what the output looks like:

screenshot 2018-01-01 16 18 21

Command Line Options

  Usage: testerbot [options]


  Options:

    -c, --config <path>  Specify Testerbot Config file
    -d, --dash           Display results in dashboard
    -n, --no-verbose     Hide verbose output (default: true)
    -s, --silent         Hide console output from tests
    -u, --url <url>      Url to run tests against
    -U, --urls <urls>    Url to run tests against
    -v, --version        Output the version number
    -h, --help           output usage information

Configuration File

For better configuration control, you can run Testerbot by providing a config file which allows you to skip or run specific tests.

By default, Testerbot looks for testerbot.config.js in your project root:

// Testerbot Run Configuration

module.exports = [{
  url:
    'http://localhost:5000',

  tests: {
  
    // Test name(s) go here
    // These tests will be skipped
    skip: [
      'Open Graph'
    ],

    // Test name(s) go here
    // These are the only tests to be run
    filter: [
      'Viewport' 
    ]
  }
}]

You can also specify a path to the config file via:

testerbot --config ./config/testerbot.config.js

Basic Features

  • A comprehensive set of pre-defined test cases
  • Support for multiple URL's in a test run
  • Ability to skip tests or run only specific tests
  • Integrated dashboard
  • Platform and language agnostic
  • Ability to run in CI build process

Todo

  • Authentication

  • Custom Scenarios (workflows)

  • Support for filtering and skipping tests by Tags

  • Custom Reporter

  • Automatic Crawling

  • Filling Forms

Contribution

This is a work-in-progress and we would like your help. Please consider contributing to this project on one of the following:

License

MIT License

FAQ

Why does installation fail sometimes with an EACCES error?

This is an npm permissions related issue. You will need to change the directory where npm is installed. You can follow instructions at https://docs.npmjs.com/getting-started/fixing-npm-permissions.

testerbot-public's People

Contributors

purplebracket avatar theoutlander avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

gitter-badger

testerbot-public's Issues

Fix CI issues

#!/bin/bash -eo pipefail
npm test

[email protected] test /home/circleci/repo
concurrently 'npm run static' 'jest --config ./src/jest.config.js'

[0]
[0] > [email protected] static /home/circleci/repo
[0] > static ./tests -H '{"Cache-Control": "no-cache, must-revalidate"}' &
[0]
[0] serving "./tests" at http://127.0.0.1:8080
[1] Setup Puppeteer Environment.
[1] Error: kill ESRCH
[1] at Object._errnoException (util.js:1024:11)
[1] at process.kill (internal/process.js:183:18)
[1] at forceKillChrome (/home/circleci/repo/node_modules/puppeteer/lib/Launcher.js:162:19)
[1] at killChrome (/home/circleci/repo/node_modules/puppeteer/lib/Launcher.js:147:9)
[1] at Function.launch (/home/circleci/repo/node_modules/puppeteer/lib/Launcher.js:137:7)
[1] at
[1] at process._tickCallback (internal/process/next_tick.js:188:7)
[1] jest --config ./src/jest.config.js exited with code 1
Too long with no output (exceeded 10m0s)

Circle CI Test Summary is incorrect

#Issue Details
This is what CircleCI is showing at the moment:

Your build ran 23 tests in unknown with 0 failures

We need to fix it so that it displays all details accurately.

Introduce the concept of various types of processors and possibly a pipeline

These can be one of the following (only listing what I can think of at the moment):

  • Crawler
  • Extractor
  • Validator?

To accomplish the above, we may have to disconnect from Jest. However, I think the main issue with Jest is the reporter. We can create a custom reporter that integrates with stuff outside of Jest as well but need to re-evaluate the role that Jest plays in this tool.

Originally, the idea was to use Jest so it handles test execution as well as reporting. However, that is not too difficult for us to implement. It appears that Jest is sort of a crutch in this project because of the many restrictions, especially not being able to create dynamic tests - that has complicated the workflow more than it needs to.

Also, the assertions are very basic. We might be able to just use an assertion library and get away without the rest of the test framework.

Perhaps, one advantage of keeping Jest is so people can write additional tests in a familiar pattern. But, I think at this stage it needs serious re-evaluation before we get too invested in it.

Create custom reporter for jest-testerbot

#Issue Details
We need to create a custom jest reporter that outputs a better report
We would like to ignore all the test failure output because it just clutters everything and is unnecessary.

Add command line parameters so user can specify level of testing - basic, full, advanced, etc.

We should allow users to use pre-defined configuration files. These would be checked in to a config folder. We can instruct testerbot to execute one of those files via a command line param.

For example, I could create a config file which runs some very basic tests essential for a web application. Facebook and Twitter OG might be unnecessary for some sites. In that case, we can choose to ignore them without having to create config entry (by using a pre-defined config file).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.