dwyl / learn-istanbul Goto Github PK

View Code? Open in Web Editor NEW

340.0 138.0 41.0 3.02 MB

:checkered_flag: Learn how to use the Istanbul JavaScript Code Coverage Tool

JavaScript 100.00%

coverage javascript codecov testing tutorial

learn-istanbul's Introduction

Learn Istanbul

Learn how to use Istanbul to check/track Code Coverage in your JavaScript projects.

Why?

Like the road sign that is "Not In Use" most of the code being written never gets executed.

There are a few obvious issues with this:

if un-tested code remains in the codebase i t can contain unknown behaviour e.g. bugs.
untested features are more difficult to maintain without introducing breaking changes.
un-tested code can clutter a project and accumulates technical debt that wastes time.

What?

Code coverage tells you when code you have written is being executed so you can decide if un-covered lines are superfluous (and can be removed) or require additional testing.

The rest of this page will focus on practical usage example, so if you are completely new to Code Coverage we recommend you read the wikipedia article: https://en.wikipedia.org/wiki/Code_coverage first.

Istanbul is a code coverage analysis script you run when executing your unit tests: https://github.com/gotwarlost/istanbul/ we like it because it's simple and prints out nice html reports (see below)

How?

Installation

We prefer to install istanbul as a "devDependencies" in each of our projects:

npm install istanbul --save-dev

to check if the installation worked, (copy-paste and) run the following command in your terminal:

node_modules/.bin/istanbul help

Simple Example

For our first example create a file called test.js.

vi test.js

type (or copy-paste) the following code in the test.js file:

x = 42;
if(false)
     x =-1;

Now run the istanbul command to generate a coverage report:

node ./node_modules/.bin/istanbul cover test.js

Alternatively you can insert the line

"coverage": "istanbul cover ./test.js"

into the scripts section of your package.json and run the following command in your terminal:

npm run coverage

This will create a directory in your project called coverage where you will find the generated coverage reports. In our case: learn-istanbul/coverage/lcov-report/learning-istanbul/test1.js.html

If you open the test1.js**.html** file in your web browser you will see a visual coverage report:

Istanbul gives us four code coverage metrics:

Statements: How many of the statements in you code are executed.
Branches: Conditional statements create branches of code which may not be executed (e.g. if/else). This metric tells you how many of your branches have been executed.
Functions: The proportion of the functions you have defined which have been called.
Lines: The proportion of lines of code which have been executed.

when you click test.js to view the coverage for the file you see:

Two things to note in the example above:

we only get 66.67% coverage because the only 2/3 of the code is being run
the 3rd line never gets executed because false is always false!

This may be a trivial example but it shows exactly where the useless code is.

A more "Real World" Example

Try executing the mischief.js file by running npm test:

What is wrong with the following picture?

There are plenty of developers/organisations that can only dream about getting 96% code coverage! and yet when we inspect the detail, there's something big slipping through the net!

We have 100% functional code coverage, but only 50% "Branch" Coverage. This means one or more conditional execution branches is not being executed.

Most of the time it will be something innocuous but what if a disgruntled person slipped in something like:

if(employee.status === 'terminated' && employee.left - today() > 90) {
	selfDestruct();
}

The 97% Coverage is not looking so hot anymore ...

What if we add a Test that follows the branch containing the rogue code? We reach our mythical 100% Coverage:

And if we simply allow this code to be promoted without further checks, the rogue code will be in production and soon forgotten.

The solution here is to not rely (solely) on tools such as Istanbul to check code. Its essential would advocate a separation between the people writing the tests and the developers who write the code.

And there is still no substitute for Code Review!

Tracking Coverage as-a-Service

Knowing the coverage locally is nice, but it's way more useful to a team to track coverage over time using an independent service. Thankfully, there are a few you can chose from:

Codecov: https://codecov.io/#features
Coveralls: https://coveralls.io/features
Codeclimate: https://codeclimate.com/features
Codacy: https://codacy.com/product

We have used all the services and found that Codecov i s the clear winner for 3 reasons:

Easy to integrate with existing Continuous Integration (CI) setup e.g. Travis-CI
Great stats and visualisations
Pull request message informs if coverage has changed (see below)

Setup

Sign-up to use Codecov (Free) using your GitHub account: https://codecov.io/
Select the repository you want to track coverage for from your list.
Add a few lines to your CI configuration file e.g:

before_install:
  - pip install --user codecov
after_success:
  - codecov --file coverage/lcov.info --disable search

This installs the Codecov reporter tool on your CI and sends the lcov.info report (which gets generated by Istanbul) to Codecov where it's kept safely.

Real-world example in .travis.yml file: https://github.com/dwyl/hapi-auth-jwt2/blob/master/.travis.yml#L7-L10

Now when you create a pull request your CI will send a coverage report to Codecov and Codecov will leave a comment on the PR:

Yes, it's "possible" to have a "large" project with 100% Test Coverage. (ALL DWYL projects do!)

So you can see at a glance if new code is being added without corresponding tests ...

When the coverage is lower the Pull Request "fails":

Note: if you want to prevent people (your team) from creating Pull Requests with less than an agreed level of coverage (e.g 100%!!), add a coverage checking script and corresponding pre-commit hook. More on pre-commit hooks: https://github.com/dwyl/learn-pre-commit

Add a "Badge" to your Readme (Optional/Recommended)

Click on the Settings tab for your chosen repo, click on Badge and click Copy e.g:

Then paste the markdown into your Readme.

More about badges: https://github.com/dwyl/repo-badges

Background Reading

Ariay's basic tutorial: https://ariya.io/2012/12/javascript-code-coverage-with-istanbul
Jasmine Test Coverage: https://dzone.com/articles/code-coverage-jasmine-tests

Istanbul (the JavaScript Code Coverage tool) https://github.com/gotwarlost/istanbul should not to be confused with istanbul the desktop screen recorder, they are totally diferent animals! Shame about the name collision... :-(

learn-istanbul's People

Contributors

Stargazers

Watchers

Forkers

mskchaitanya guotechfin fullstackforger keertimaan fullcrimp rykon lior-amsalem renatoargh ellerbrock roppa nicholas-b-carter damianmcnulty 17812 isakib khaosdoctor signorini macysre particle4dev jblackcat jordaaash walter2 tim-kos durja alu0100946499 enshaednhiker noro07 vdyachkov manjudr adamwads gcaliene donavon ashishepost 135lou lutece calebisc miketomato13 sprabhu1705 temirtator qas52 andrwlt lipanpan03

learn-istanbul's Issues

Migrate from istanbul to nyc ?

istanbul has recently been deprecated https://www.npmjs.com/package/istanbul

Should we switch from istanbul to nyc? https://www.npmjs.com/package/nyc
See: https://github.com/istanbuljs/nyc#readme
I expect that it will take a couple of hours to update this tutorial/example and then a few minutes for each of our JS repos which are using istanbul ...
Thoughts? Anyone want to volunteer to do this?

broken Hitcount

Tidy up, Update Dependencies and Cut down on intro text

This doc needs some TLC.

Unable to run whole project

/usr/bin/istanbul --include-all-sources cover --recursive ./src/talos/wwwroot/panojs/pyramid_Zoomify.js
I am unable to run whole project
can anyone guide me ?
The above image only for one single file

test.js and test1.js appear to be identical is this intentional?

https://github.com/dwyl/learn-istanbul/blob/d8ebff630bdb7fdfcbd7a67cf3e56b66654ff358/test.js
and
https://github.com/dwyl/learn-istanbul/blob/d8ebff630bdb7fdfcbd7a67cf3e56b66654ff358/test1.js
appear to be identical. is this intentional? do we need both?

(ES6) Missing description about branches originated by default value for function parameters

I'm using istanbul 0.4.5 (with istanbul-lib-coverage 1.2.0 and istanbul-lib-instrument 1.10.1) to check coverage of my jasmine unit tests over my ES6 source, and I found a branches count which is not mentioned in your document.

If my function has a default value for input parameter, like this (this is a class used to create an AngularJS service):

class UserDataService {
    getClientData() {
        return this.client;
    }
    setClientData(obj = { foo: 'bar' }) {
        this.client = obj;
    }
}
export default UserDataService;

Notice the default value for setClientData parameter obj.

If my test suite is:

import moduleUnderTesting from './userData';

describe('Service: UserDataService', () => {
    let service;

    beforeEach(() => {
        angular.mock.module(moduleUnderTesting);
    });

    beforeEach(inject((_UserDataService_) => {
        service = _UserDataService_;
    }));

    describe('method setClientData and getClientData', () => {
        it('should set/get clientData correctly', () => {
            const cData = { one: 1, two: 2 };
            service.setClientData(cData);
            expect(service.getClientData()).toBe(cData);
        });
    });
});

The test run reports 1 branch not covered:

While, if I add a test case that makes setClientData use the default value for obj parameter, like this:

import moduleUnderTesting from './userData';

describe('Service: UserDataService', () => {
    let service;

    beforeEach(() => {
        angular.mock.module(moduleUnderTesting);
    });

    beforeEach(inject((_UserDataService_) => {
        service = _UserDataService_;
    }));

    describe('method setClientData and getClientData', () => {
        it('should set/get clientData correctly', () => {
            const cData = { one: 1, two: 2 };
            service.setClientData(cData);
            expect(service.getClientData()).toBe(cData);
        });

        it('should set/get clientData correctly using default value', () => {
            service.setClientData();
            expect(service.getClientData()).toEqual({ foo: 'bar' });
        });
    });
});

Test run reports full coverage:

This is an issue of istanbul itself, because the coverage should show what's the missing branch; anyway, I'm reporting it here as well in order for it to be pointed out in your document.

Advice needed :-) on implementation of Codecov without key on Travis

I'm trying to use Codecov on Travis without using the key as you told me it can be done without and so in the name of efficiency I want to learn how it's done!
My .travis.yml and package.json look like yours:
https://github.com/Cleop/triumph-website/blob/master/.travis.yml#L3-L6
https://github.com/Cleop/triumph-website/blob/master/package.json#L10

And I'm registered on Codecov and have selected the repo.

However I'm not getting any data on Codecov when I merge PR or on the badge on my README.

Any advice would be greatly appreciated 😄 Thank you!

How to ignore files (e.g: libraries) from coverage using a .istanbul.yml file?

Sometimes you want to ignore a specific file or even a folder from being checked by Istanbul
e.g you include a 3rd party library or code in your project (instead of putting it on a CDN)

You can define a list of excludes files in a .istanbul.yml at the root of your project/repo.

Its not an especially well-documented feature... There's only one reference to .istanbul.yml in the entire Istanbul project, see: https://github.com/gotwarlost/istanbul/search?l=markdown&q=.istanbul.yml&utf8=%E2%9C%93

There are examples on the StackOverflows if you search for them. e.g: http://stackoverflow.com/a/34685175/1148249

instrumentation:
    root: .
    extensions:
        - .js
    default-excludes: true
    excludes: ['**/tinymce/**', '**/lib/**', '**/tools/**', '**/build/**']
    include-all-sources: true

a more detailed sample file: https://github.com/vesln/todo/blob/master/.istanbul.yml

@nikhilaravi needed this today in our project, it reminded me we should add it to the readme.

Update all screenshots to the latest Istanbul UI

Update the screenshots and remove the images from the repo to reduce bloat.

update readme links

http=>https

Check in

Hi,

I am not sure if this was intended in the first place or a typo. I was learning from your article and the first time I ran code coverage on mischief.js, it had a 100% coverage. After scratching my head for sometime, I realized it's doing what the test asked it to do. I had to update the 20000 to number < 10000 to have the right output.

console.log("\nExpect Account1 Opening Balance "+getAccountBalance(account1) +" === 0 \u2713 ");
creditAccount(account1, 2000, 'Add Funds');
transferMoney(account1,account2, 100, 'Give money to friend');
transferMoney(account2,account1, 10, 'Transfer back 10');