alex-seville / blanket Goto Github PK

Another issue we will have is the support of logic operators.

var empty;

var result = empty || "default";

Do we plan to support and see if we passed through the left or the right of this statement ?

One way we could do this, it is to apply an additional operator.

var result = empty || TRACKING CODE[NumberCol, NumberCol] || "default";

if it is an || the tracking code return false in order to go to "default".
If it is an && the tracking code return true in order to be seamless in the condition.

I think it is a more advance feature and we could mark the line as parsed for now :)

So I tried to run the grunt jscover command inside the grunt folder and I get this error message:

TypeError: Cannot call method 'split' of undefined
at Object.module.exports.instrument (E:\sandbox\jsCover\lib\jsCover.js:7:24)
at Object.module.exports (E:\sandbox\jsCover\grunt\tasks\jscover.js:30:20)
at Task.helper (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\lib\util\task.js:117:19)
at Object.module.exports (E:\sandbox\jsCover\grunt\tasks\jscover.js:22:34)
at Object.task.registerTask.thisTask.fn (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\lib\grunt\task.js:58:16)
at Task. (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\lib\util\task.js:341:36)
at Task.start (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\lib\util\task.js:357:5)
at Object.grunt.tasks (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\lib\grunt.js:143:8)
at Object.module.exports as cli
at Object. (C:\Users\pde\AppData\Roaming\npm\node_modules\grunt\bin\grunt:19:14)

The documentation of running blanket in combination with mocha appears to be incorrect. The require('blanket') call returns an object and not a function.
In order to set the correct filter, the setFilter method should be called with the correct parameter (regex, string etc like in the documentation). just adding require('blanket') to my testcode gets a 0% coverage of 0 lines, because it does not see what source it needs to scan.

Am I doing something wrong or is the documentation outdated?

Hi guys,

In node with mocha, is the correct usage to include 'require('blanket')(path)' to every test file?

From what I can tell it looks like blanket shims the require method (looking at https://github.com/Migrii/blanket/blob/live/lib/node.js)

However does it then shim for every blanket call? i.e. if I add require('blanket')(path) to the top of every test file, then for a mocha run of multiple test files will blanket add shims on shims for each test file processed?

If you are only supposed to include and call blanket once, to which test file can you add it as does mocha guarantee the order which test files are processed from the test directory?

Sorry if I have missed something obvious, otherwise thanks for this!

I think, all the result should be already there before passing through the reporter.

We should be able to click on the file names and see the full source with highlighted lines and a count of the number of times the line was covered.

I think it should slide down when you click on the filename. Other options are to open a new browser window, but I don't like that. I think it would be more intuitive to have it hide/show.

Using underscore I will put the css and html of the report in a template.

For now it is a separate file but for the builder it will need to be compiled and integrated inside the lib.

So to show the result, I will need to execute some basic mathematics.

Do you think, we could add these function under the blanket object ?

Or maybe just create another object which could be available whatever the test runner that I could use inside my output :)

for example GetPercentage()... which for now is very easy but later when we will track inline if else and so on would be a bit difficult ?

Should point to: https://raw.github.com/Migrii/blanket/live/dist/blanket_qunit.js

Need documentation, need to manage the script resources (falafel, esprima, requirejs) better, need to clean up that script, other command line arguments.

I'm trying to integrate Blanket in Chutzpah, and one of the test files I'm using for the integration tests contains an error (on purpose):

var errorThing = {
    thing: ["thing" + someUndefinedVariable]
};

When I open the test harness that Chutzpah generates in Chrome, the console shows the following:

Uncaught ReferenceError: someUndefinedVariable is not defined code.js:23
waiting for blanket... blanket_jasmine.js:4561
Error parsing instrumented code: ReferenceError: someUndefinedVariable is not defined blanket_jasmine.js:4431
Uncaught Error: Load timeout for modules: blanket_0,blanket_1
http://requirejs.org/docs/errors.html#timeout blanket_jasmine.js:4202
G blanket_jasmine.js:4202
w blanket_jasmine.js:4209
(anonymous function)

If I remove Blanket from the test harness, I still get the first ReferenceError, but Jasmine runs all tests as expected.

I'm using Blanket 1.0.0.

Since the philosophy of the project is be 100% javascript, we should in theory replace the Makefile with a grunt.js file.

Need to copy the koans example for test on live to examples on gh-pages since it's now more accurate.

jscoverage has a feature where you can indicate that some code shouldn't be covered.
Maybe we use something like

/*blanket-start-nocover*/  
... the source code you want ignored...  
/*blanket-end-nocover*/  

I've noticed that the instrumentation doesn't take into account lines that cannot be reached, like in the case of multi-line variable declarations. I think this is already handled in jscoverage (or so the mocha reporters imply) by some mechanism. This affects the quality of the results, so it needs to be a priority.

I guess jsCover will run all tests :) with some runners.

I think we should define the testing framework we want to support.

I think Mocha, Jasmine and Qunit is a must.

?

Each part of the build should be modularized. Right now everything is tossed into the global scope and that's a recipe for disaster/incompatibility.

Also, all of our requirejs stuff especially needs to be reassigned to different variables to avoid conflicting, not only with existing requirejs (which we override for the moment, not safe), but also to avoid test runners that mock out the amd references (i.e. jquery test suite).

Create FAQ.

-Can't use chrome with file:/// protocol, for example

Not sure how this will work on the node side.

Some thoughts:

Parser and node-falafel will run the same. The test runner part, lib/qunit.js, will need to be modified, but when we make the mocha one it will probably handle that.

The live instrumenting will be another issue. We'll need to make a special require script for node that finds the existing scripts (not sure how we'll indicate which ones are tests, but that can be determined later), remove them from the require cache (unrequire), then load them in and instrument them. How likely is this? I imagine it's possible, but it'll require some testing.

Line 12 of blanket require could be changed to create a script element for the instrumented files:

var script = document.createElement("script");
script.type = "text/javascript";
script.text = "alert('voila!');"
document.body.appendChild(script);​

The readme mentions requirejs. Is there an example somewhere with the recommended usage with requirejs?

More tests for blanket.js
reporter.js
blanketrequire.js

Sorry, this is probably a stupid question. Couldn't find an IRC channel or mailing list, so I will open an issue here, hope that is OK.

I'm not familiar with QUnit. I'm running tests for Django using Selenium. Django basically just starts a LiveWebserver for Selenium, Selenium then goes to a certain URL which displays a view with all the scripts and HTML elements that I need to test. On that view, Selenium will click around and see if everything works as it should.

That view, of course, could also just be opened in the browser and I could click around manually. I was hoping that, since I have included blanket.js, after some clicking, the coverage result would magically show up somewhere on the page? (I know, that's naive) :)

I have even tried to use the simple_json_reporter.js, which to me looks like it would append something to the page, but of course it never seems to be triggered.

Is my usecase completely wrong or is there any way to "record coverage" by clicking around on a page and then see the result somewhere?

Require runs the tests two or three times, so I think there's an issue related to the starting and stopping of the test runner, and how files are instrumented maybe too.

The koans works in the browser but fails with no coverage data under phantom.

Scripts with relative paths get wrong urls and confuse require.js loading, the script can't be loaded.

For example:

<script src="../../src/myscript1.js" data-cover></script>
<script src="../../src/myscript2.js" data-cover></script>

Debugging shows url for myscript2 is converted to ./../src/../src/myscript2.js

Blanket doesn't seem to support Mocha in the browser. According to baeda5c a dist/mocha/blanket.js used to exist, but not anymore.

Adding the current dist file generates a require.js error:

Uncaught Error: Mismatched anonymous define() module: function () {
    return definition();
}
http://requirejs.org/docs/errors.html#mismatch 

It also seems to prevent chai from exporting itself to window.chai.

Make an example where the source file uses something like jQuery as a depency, and then run the coverage on the dependency, rather than on the source.

The result will show you the amount of jQuery (in this example) used by your source code. This could help you with refactoring or removing dependencies that are unused.

This should be possible right now, I just need to review the default reporter because I think it's unreporting the stats.

Credit to this tweet by @tobie for the inspiration, and @paulirish for bringing it to our attention.

I think all the output should be in json.

I also think, we should clearly split the "task" creating the data with the JsCover execution and reporting the data.

I could built a basic HTML report based on the JSON we output :)

I've already started making some changes on my branch to handle the case where requirejs is already being used by the test runner.

1. Added a conditional in the bundle template, so the requirejs bundled code is not executed if requirejs already exists (might need to specify that blanket has to be declared after the requirejs)
2. In that case it might be better to bundle a requirejs plugin that allows you to specify which files are to be covered, think:

require: {
   'sourcefile1': 'cov!sourcefile1.js'  

or something similar. I think there are examples I can base it off already, like text

I'm using Bower to manage client-side dependencies. My unit tests run in the browser, so I'd like to install Blanket as a client-side dependency (as opposed to installing it via NPM).

Bower uses tags to track versions, and if the repo has no tags it uses the latest commit in master.

Since Blanket's master branch isn't stable, it would be great if you could tag stable versions. Or even better, define Blanket as a Bower package!

In place of this

module.exports.instrument = function(config, next){
    var inFile = config.inputFile,
        inFileName = config.inputFileName;

    var lines = inFile.split("\n");
    var instrumented = "";
    //set up namespace
    instrumented += "if (typeof _$blanket === 'undefined') { _$blanket = {}; }\n";
    //initialize file object
    instrumented += "_$blanket['"+inFileName+"']=[];\n";
    //initialize array values
    for (var j=0;j<lines.length;j++){
      instrumented += "_$blanket['"+inFileName+"']["+j+"]=0;\n";
    }
    for (var i=0;i<lines.length;i++){
      instrumented += "_$blanket['"+inFileName+"']["+i+"]++;\n";
      instrumented += lines[i]+"\n";
    }
    next(instrumented);
};

Maybe we should use a template engine to ease the maintenance :)

Might be interesting to host this project on travis-ci as a way of ensuring that it works with continuous integration.

Now that we are using live instrumenting, don't you think we should separate clearly the report and blanket and communicating between them by events.

For example. when the blanket.js has finished their job. you could trigger an event "reporting-time".

Like this I could listen for it and output the result to the end of the page.

Currently, both are quite tied together which is not easy for me because I fear to break your code :)

So, I was trying to fix the grunt task and I realized that maybe trying to play with the tree structure of the user won't be a good thing.

I mean, as a dev, I would be really mad if a tool will add some folder under my source file.

Maybe we should put everything under a cover folder.

If blanket is required and called, I see unexpected identifier exceptions where there are escaped single quotes in single quoted strings ('it isn't ok') such as:

Error parsing instrumented code: SyntaxError: Unexpected identifier
method(\'This isn\\'t working 
                                 ^

Version 0.9.7

Replace:
https://github.com/Migrii/blanket/blob/live/lib/blanketRequire.js#L110

with the more efficient:

document.querySelectorAll("script[data-cover]")

I noticed js-coverage and blanket report slightly different SLOC and % coverage

When comparing side-by-side outputs from the HTMLCov reporter it looks like some single line comments are counted as lines of code in blanket

Eg from js-coverage: (file reports 29 SLOC)

70                  }
71                  // the comment
72  2               _.each(filters, function (promise) {
73  8                   if (promise.data) {

from blanket: (file reports 33 SLOC)

70                  }
71  1               // the comment
72  2               _.each(filters, function (promise) {
73  7                   if (promise.data) {

Version 0.9.7

Instead of writing:

_$blanket['testname'][0]=0; _$blanket['testname'][1]=0; ...

For n lines of source code, we could write:

for(var _$bFor=0;_$bFor<n;_$bFor++){ _$blanket['testname'][_$bFor]=0; }

But I'm not 100% sure whether this will adversely affect things in any way...

Currently we have

if (typeof _$blanket === 'undefined') { _$blanket = {}; }
_$blanket['example/test samples/src/testLib.js']=[];
_$blanket['example/test samples/src/testLib.js'][0]=0;
_$blanket['example/test samples/src/testLib.js'][1]=0;
_$blanket['example/test samples/src/testLib.js'][2]=0;
_$blanket['example/test samples/src/testLib.js'][3]=0;
_$blanket['example/test samples/src/testLib.js'][4]=0;
_$blanket['example/test samples/src/testLib.js'][5]=0;
_$blanket['example/test samples/src/testLib.js'][0]++;
var testObj = null;
_$blanket['example/test samples/src/testLib.js'][1]++;
if (testObj){
_$blanket['example/test samples/src/testLib.js'][2]++;
    testObj = "2";
_$blanket['example/test samples/src/testLib.js'][3]++;
}else{
_$blanket['example/test samples/src/testLib.js'][4]++;
    testObj = "1";
_$blanket['example/test samples/src/testLib.js'][5]++;
}

Would it be possible to have something like :

if (typeof _$blanket === 'undefined') { _$blanket = {}; }
 var blanket = _$blanket['example/test samples/src/testLib.js']=[];
  blanket[0]=0;
  blanket[1]=0;
  blanket[2]=0;
  ...
var testObj = null;
  blanket[1];
if (testObj){
    blanket[2]++;
    testObj = "2";
    blanket[3]++;
}else{
   blanket[4]++;
    testObj = "1";
    blanket[5]++;
}

I think it would be more clear and leads to less bytes inside the file :)

For the browser:

I'll still include the default reporter (even though it's not reporting accurately right now) but create a mechanism for specifying alternative reporters. I think maybe using another data attribute on the blanket script reference, like:

<Script src="blanket.js" data-cover-reporter="mocha-reporter.js"></script>

which would load the mocha-reporter.js file for use in reporting the results in browser (for mocha in browser tests).

To create your own API, I'll publish some documentation. Basically the reporter should export a function (not unlike mocha reporters in node) and it should expect a coverage variable from blanket.

That coverage variable is of this format:

{ 
    filenamekey: {
            [null,0,1,0,1,1,1], //line coverage
            ['this','is','the','source code;'],
          }
  , // you can also include test data
}

I'll create a simple example in the coming days, that way people can contribute their own reporters. The only thing standing in the way of supporting the various different test runners is the command to stop/start tests and the reporting.

I think it could be good to move the live branch to the master branch and use the master by default :)

I could be cool if we could setup the success rate, for example, 60% in the attribute of blanket.js.

For example,

If coverage is less than 60%, it show error css else success css ?

What do you think ?

Covered/Total stms. header is misplaced.

Do you plan to distribute 3 versions of Blanket ? I saw you name it blanket_qunit.js ?

I think, if we really want to be seamless we will need to have the 3 working on the same files. I know it will be heavy and that it would maybe add some "hacky" code but we will keep a freaking easy configuration :)

Maybe you just did that for test purpose ?

Need to add test duration, and other details as found in jscoverage json results.

make a jasmine.js file similar to lib/qunit.js.

We should probably refactor reporter.js for each test runner as well and make a qunit and jasmine version.

Look at techniques for reducing the size of the _$blanket variable.

Maybe local storage, saving source out of order, saving source to the DOM (sounds like a bad idea, but might not be)...

So, one of the problem of our code is the "none" support for inline if.

So in place of just having the line counted, I think we also need the number of column we have in the code.

So if we have a code like this:

  if(toto) { return true; } else { return false; }

We could do:

• Load the line, parse the line and check if it is an inline if (I think looking at jslint could help us for this part). Now we must instrumented the code in order to know it is a "inline" code.

  [INLINE] if(toto) {[COL 10]return true;[col 22]} else {[COL 34]return false;[col 47]}

When we run the coverage, if we pass the if (only). We mark as "passed" the if and inside the { /* --- */ }

The result could be like:

Line 34 : Inline code -> passed: [0,10], [10,22]

The require statements get executed int he global context so the paths are incorrect. I think I need to look at changing the module.paths for the module being eval'd.

There are a few issues that need to be addressed with the way we load source files:

1.With the typical Jasmine test runner, tests are executed on window.load. Our scripts are fetched for instrumentation on window.load (which is done before Jamsine starts) but it is done asynchronously so before it is finished the test runner begins.

2.Since files are fetched asynchronously, they don't end up respecting the order placed in the HTML file. For example, in backbone koans, if you add data-cover to jquery and to templates.js, you'll notice that template.js (a jquery plugin) is smaller and is therefore loaded before jquery (to which it depends on). This applies only to situations where the test author is not already using requirejs.

Thoughts:
1.There are some simple ways to address this issue, but because we want Blanket to be as seamless as possible (fewest changes to the existing test runner) we have less options.
I think it could work if I override the jasmine.getEnv().execute() function when I attach the jasmine-blanket adapter and basically mock it out. Then we can control exactly when the test starts. This would all be done within the jasmine-blanket adapter, but I'm undecided if this is a good approach. For one it would make the adapter less modular, since we'll need to access the global jasmine object, rather than a local version (as is being done presently).

2.The quick and dirty way to resolve this could be to nest require statements based on order. So, for example, if we have 3 scripts in the HTML page:

<script src="a.js"></script>    
<script src="b.js"></script>     
<script src="c.js"><script>    

Then our blanket code would dynamically generate the following loading approach:

 require(["a.js"],function(){   
    require(["b.js"],function(){    
     require(["c.js"],function(){    
        //do callback, probably start test   
     }  
   }  
 }  

Any ideas or suggestions would be appreciated. Thanks.