isaak-malers / obverse Goto Github PK

Automatically build protractor test-rigs. Adds rich logging, debugging, and human readable test instructions

License: MIT License

JavaScript 89.67% HTML 7.49% CSS 2.84%

protractor protractor-e2e-test protractor-tests protractor-async-await protractortest jasmine jasmine-tests webdriver webdriverjs selenium

obverse's Introduction

Obverse

Obverse is a library and style guide for protractor automated testing, and is comprised of the following parts:

The major additions to protractor are:

Adds "step" function works similarly to jasmine "it" function, adding granularity to tests.
Includes rich logging, debugging, and manual test-replication tools.
Test rigs are built quickly using a data-entry tool which is added to chrome as an extension.

Chrome Extension:

The most visible part of an obverse test is the "map" of the website to test. A chrome extension which adds a data-entry tool to the developer tools enables a fast developer workflow for creating and updating test-rigs.

Test Rig Data:

Test rigs are stored in a rigid directory structure. The entire directory is considered to be a part of the test rig, but there is no database or other complex data-store.

/data-model
  /images
  /app-map-backups
  app-map.json
  .gitignore

The images folder stores image metadata about test elements.
The app-map-backups directory contains a copy of app-map.json at every single save point
app-map.json is the primary data used by the chrome extension to store test rig data
.gitignore is included by default so that the large images and app-map-backup directories don't wind up in your repository!

Test Library:

The Obverse run time library uses the data stored in the "data-model" directory to build test rigs on the fly at run time. Below is an example of a test rig being created and used in a language test for Wikipedia:

An Obverse object is created to be used for the test, and is concisely named "ui".
Obverse objects allow test specific timing, logging, and debugging options.
Obverse.step (ui.step) is used to denote actions which occur in a single browser state.
Obverse.at (ui.at) is used to force a browser context check/wait.
Obverse.el (ui.el) is used to locate a page element based on its name in the data-model and the current page location.
Obverse.finalize (ui.finalize) compiles failure reasons and fails or passes the test.

var Obverse = require('../../../src/obverse');
//global test variables can go here, or they could be read in from an external file.
var url = "https://www.wikipedia.org/";

describe("Basic Language selection tests for wikipedia", function(){

	it("should be able to change the language from default to polish", function(done){
		//construct an obverse object for this test case.
		var ui = new Obverse("example");
		ui.setLogLevel("debug");
		ui.suppressDumps(true);//failure reasons will not be automatically logged to console.
		ui.setDelay(1);//additionall delay between actions, setting to 0 breaks protractor.
		browser.waitForAngularEnabled(false);//disable protractors built in waiting methods.

		ui.step("load the test url: " + url, function(){
			return browser.get(url);
		});

		ui.step("change the language from default to polish", function(){
			return ui.at("search-page")
			.then(ui.select("language", "Polski"));
		});

		//this assertion should pass
		ui.step("verify the link to the english version is labled 'English'", function(){
			return expect(ui.el("english").getText()).toBe("English");
		});

		//this assertion should fail to showcase error reporting.
		ui.step("verify the link to the english version is labled 'PL', this should fail", function(){
			return expect(ui.el("english").getText()).toBe("PL");
		});

		//compile results, then call jasmines (done) function to complete the test.
		ui.finalize(function(){
			done();
		});

	});
});

Obverse also provides built in "shorthand" actions for the following browser actions:

Clicking
Typing into a field
Drop-down selections
Clearing a field

Logging and Output:

Obverse provides in depth logging, and can be used to generate human readable instructions for failing tests. Failing tests can have their instructions compiled and then sent automatically to manual QA personnel for a second check. This saves developer time.

Getting started:

Pre-requisites:

Node 8.xx
Protractor installed and working
Familiarity with Jasmine/Protractor/Node.js

Setup

Download and unzip a stable version from the "releases" tab on this GitHub page.
Open chromes extension tools by going typing: chrome://extensions into the address bar.
Clicking load unpacked near the top.
Navigate to the the directory where you un-zipped the release, and then into: obverse/bin/ui
Select "open"
The obverse plugin will now appear in your developer tools, but requires the back end to be running to function fully.
Navigate to: obverse/tests in the directory that was just unzipped.
Copy the demo directory, and name the copy to whatever you want the new test area to be called.
Open a terminal window and CD into the following directory: obverse/tests
Run the command: node init.js to start the back end on your computer.
Open the developer tools in chrome (this must be done AFTER starting the back end), and select your new test area from the app-map drop down.
The copied obverse test area has all the boilerplate code you need to get started. Refer to API documentation and the examples for more in depth guidance.

obverse's People

Contributors

Stargazers

Watchers

obverse's Issues

Error message for malformed xpaths are missleading

A malformed xpath can cause protractor to throw a useless exception. It should be caught, and re-thrown with a useful hint to the user.

calls to console.log() in javascript code running on the chrome extension result in an error

all instances of console.log must be purged from code that runs on the front end.

"highlight overlay cannot be applied to page" has the wrong logging level

the "highlight overlay cannot be applied to page" error messages should be logged as minor, not critical.

obverse object should write a message when it gets required in.

obverse should print a logging message when it gets required in.

null error when opening obverse in chrome extension

When opening the obverse tab in the chrome extension, the following error is displayed.

Error: ENOENT: no such file or directory, open 'c:\automationTesting\obverse\tests\undefined\data-model\app-map.json'
    at Object.fs.openSync (fs.js:646:18)
    at Object.fs.readFileSync (fs.js:551:33)
    at c:\automationTesting\obverse\chrome-extension\app.js:47:19
    at Layer.handle [as handle_request] (c:\automationTesting\obverse\node_modules\express\lib\router\layer.js:95:5)
    at next (c:\automationTesting\obverse\node_modules\express\lib\router\route.js:137:13)
    at Route.dispatch (c:\automationTesting\obverse\node_modules\express\lib\router\route.js:112:3)
    at Layer.handle [as handle_request] (c:\automationTesting\obverse\node_modules\express\lib\router\layer.js:95:5)
    at c:\automationTesting\obverse\node_modules\express\lib\router\index.js:281:22
    at param (c:\automationTesting\obverse\node_modules\express\lib\router\index.js:354:14)
    at param (c:\automationTesting\obverse\node_modules\express\lib\router\index.js:365:14)

Search

The search/filiter bar will have the following behaviour:

Hide completely (ass oppose to collapse) tree branches that don't match the search term, OR contain a child which matches the term exactly. Given the following tree definition:

root
- login
  - username
  - password
  - ok
    *newUser
    *username
    *password
    *password2
    *ok
    *userProfile
    *bio
    *settings

Search: user

root
- login
  - username
    *newUser
    *username
    *userProfile
    *bio
    *settings

Search: username

root
- login
  - username
    *newUser
    *username

Search: login.userName

root
- login
  - username

Evaluate 3rd party logging solutions, or pull the logger out into a git sub-module to reduce code base size.

Evaluate a 3rd party logging library so that the logger doesn't need to be maintained. If there are none that are sufficient, pull that code out into a different repository/project to reduce codebase size.

Unified Description

Make all the "description" fields on all of the different systems match, off the top of my head this will/does include:

Git-hub description
Chrome Web Store Extension
NPM description

Allow configurable directory structure.

The chrome extension AND the test runtime code need to be able to pull in, save, and manipulate datasets from arbitrary disk locations. The data set will always have the format:

data-model/
- images/
- backups/
- app-map.json

Required for this task:

Code that can recognize when a directory matches this structure, which will be used on the front-end and the back-end.

Obverse constructor should take a path, which will be used to find this code.

Chrome extension will need a "save" and "save-as" button to save to a stored location, as well as to save to a new location. I think the chrome extension should take over the responsibility of creating new, "empty" data directories.

Check for Duplicate names in the Hooks Array

obverse-tree.js should:

On serialization AND de-serialization

Ensure that there are no duplicate names in the "hooks"/"children" array.

screenshot re-design.

I want to re-work how screenshots are stored:

I want to store them in a base64 encoded strings, in a JSON. Originally I wanted them as PNG's so that they were easy to view, but viewing 10000 screenshots in a directory crashes my file explorer and isn't helpful. Additionally it is more difficult for the code to auto-organize.

I think that the best way to do this is to have a JSON for every unique item in the tree. This JSON will look like:

{
  "path" : "login.username",
  "referance-into": {
      "image": "BASE64",
      "time": "timestamp",
      "left": "left margin",
      "right": "right margin",
      "top": "top margin",
      "bottom": "bottom margin"
    },
  "referance-outof": {},
  "recent": []
}

path is the path
reference-into is a reference image of the element on the page.
reference-outof is a referance image of the browser state AFTER the element is clicked/typed/whatever. this will match referance-into in structure.
recent will be an array of the same structure as referance-into of the last 10 images recorded.

git-hub pages API guide document.

wrtie api-guide.md for github pages.

chrome Icon isn't loading correctly

the caliper asset needs to be set as the chrome icon for obverse

create a helper function to disable CSS transitions

this stack overflow comment is helpfull:

https://stackoverflow.com/questions/32097973/protractor-how-to-handle-site-that-is-heavily-animated

this would be done by adding a CSS class to the page dynamically at start:

.notransition * {
-webkit-transition: none !important;
-moz-transition: none !important;
-o-transition: none !important;
-ms-transition: none !important;
transition: none !important;
}
... and in protractor, I've got something like:

_self.disableCssAnimations = function() {
return browser.executeScript("document.body.className += ' notransition';");
};

This task should be rolled into the "screenshot redesign" task, and they can be accomplished in one branch.

app-map runtime tags

Add functionality for "tagging" A node on the app-map with information about its past run-time.

this will look something like:

meta:{
  "runtimeTags": {
     "lastUsed": "date",
     "lastError": "error string",
     "lastErrorDate": "date",
     "useCount": "an incremented value"
  }
}

This will allow the "search" function in the UI to do some fancy stuff such as, automatically showing errors, and automatically showing "depricated" nodes, which maybe haven't been used in quite a while.

github pages style guide

write a style guide.md document for github pages

fix readme with markdown code examples

chrome extension workflow

This is mostly a comment thread to record good ideas about how the frontend workflow can be made faster.

Graphics re-design

The chrome extension graphics looked good when it was standalone in a web page, but they do not match the chrome dev tools style. It would be nice if it were a more cohesive look and feel.