If you need a preprocessor (coffee), then it's also good to remove the compiled js files.
If not when the whole tests have been run, at least when quitting testem.

on_exit: rm -rf src/*.js

*Update: I've changed the code example to on_exit so as to not confuse more people.

{
    "launchers": {
        "tap": {
            "command": "npm test"
            , "protocol": "tap"
        }
    }
    , "src_files": [
        "./**/*.js"
    ]
    , "launch_in_dev": [
        "tap"
    ]
}

Strips the stack traces off of my tap output thus making debugging a pain in the ass.

I've got many javascript modules, each with their own qunit test file. So putting them in the test_page is rather cumbersome, and I'd like to use the src_files config to specify a glob that grabs them all, the same way the framework runners do. And then I'd like to run testem ci to run all the tests.

If the test_page were rendered as a template in a similar fashion to the framework runners templates are, then testem could better integrate with projects that use a dependency loader, such as StealJS or RequireJS.

Here's my test_page so far: https://gist.github.com/3991896

This page works with testem's watching/reloading feature nicely, but you can see the manual entries for 2 qunit tests.

You can see that with StealJS, dependencies (including testem.js) are loaded via a pattern that uses a JS call:

steal('lib/first.js', 'lib/second.js');

instead of HTML:

<script src="lib/first.js"></script>
<script src="lib/second.js"></script>

I'd like to have my src_files set to lib/**/*_test.js and then I would finally be able to do testem ci in a Jenkins job.

hey, first of thanks for creating this.

i'm trying to run this on a virtual box vm ( ubuntu 10.04 ) with phantomjs installed. Now, if I start the test suite it just hangs at:

vagrant@lucid32:/vagrant_data$ testem ci
# Open the URL below in a browser to connect.
# http://*.*.*.*:3580
Waiting for 0 more browsers...

is there anything else I need to consider, like starting xvfb and launching a display or will phantomjs work this out by itself?

This is not an issue but a question, Is it possible to use testem witha headleass browser like PhantomJS?

I was using another runner before testem called Testacular (http://vojtajina.github.com/testacular/) and recently switched ti Test'em.

I love Test'em and thank you for all your hard work on it, but I wish it ran in the JetBrains IDE's properly. Here is a screencast of Testacular running in the JetBrains IDE, WebStorm: https://www.youtube.com/watch?v=MVw8N3hTfCI&#t=7m39s

So I figured I could probably do the same with Test'em, but I get errors. See below.

/usr/local/bin/node ../../../../usr/local/bin/testem
tty.setRawMode: Use `process.stdin.setRawMode()` instead.

tty.js:37
    throw new Error('can\'t set raw mode on non-tty');
          ^
Error: can't set raw mode on non-tty
    at Object.<anonymous> (tty.js:37:11)
    at Object.deprecated (util.js:75:15)
    at EventEmitter.exports.Charm (/usr/local/lib/node_modules/testem/node_modules/charm/index.js:43:13)
    at module.exports (/usr/local/lib/node_modules/testem/node_modules/charm/index.js:29:12)
    at initCharm (/usr/local/lib/node_modules/testem/lib/ui/screen.js:29:23)
    at Object.<anonymous> (/usr/local/lib/node_modules/testem/lib/ui/screen.js:43:18)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

Process finished with exit code 1

The strange thing is, if I run the command on the first line from my Terminal

/usr/local/bin/node ../../../../usr/local/bin/testem

it works just fine. It only errors when it runs within the IDE. Any ideas?

as an alternative faster scroll.

The testem runner sometimes has multiple opera tabs because something is going wrong (probably websocket reconnects)

not sure where the bug is coming from or how to solve it though

I've updated to Testem 0.2.1 (and using Node 0.8.11 on Mac OSX 10.8.2) and whenever I run testem ci the process never completes. The tests execute properly in all launchers but the CI mode process just hangs.

I cloned the Testem repo and tested this with a few of the project configurations that you have inside of examples and it seems to hang all the time.

Can you reproduce this on your machine?

I would add some parameter that would exclude automatic phantom launch if its not needed, for example
no_phantom: true

What do you think?

That is a kind of idea to think through.

As I described earlier in the issue (#35) I use testem to run tests suites depending on location hash. By test suite I mean a set of files including tests and sources. (for example browser with hash #test=models.my_model runs tests from test/models/my_model.test.js)

So, browsers (maybe different tabs in one browser) can attach to testem server to run different test suites.

The questions that rise when using such approach:

1. How to identify browsers attached in a testem console (besides browsers name and version, maybe title?)
2. FileWatch problem: To rerun tests (reload page) for particular test suite only if sources files that are included in that test suite have been changed. At the moment testem will reload all attached browsers if any of source files changed.

What do you think, or this is too sophisticated?

I'm sure I'm just missing a configuration option, but I'm seeing something odd.

I run testem with this command:

$ ./node_modules/testem/testem.js ci -f /config/spec.json

which is this config:

{
  "framework" : "jasmine",
  "launch_in_dev" : ["PhantomJS"],
  "launch_in_ci" : ["PhantomJS"],
  "src_files" : [
    "generated/js/app.js",
    "generated/js/spec.js"
  ]
}

And then I get this result:

1..76
# tests 76
# pass  75
# fail  1

So I immediately echo $? and find that exited with code 0, which is causing my build to stay green when it should go red.

$ echo $?
0

Hi, I was testing out testem and tried to configure it to run based on a complex file structure in an existing app, my config.yml looks like this:

framework: jasmine
src_files:
- health/public/ui/default/js/vendor/jquery.min.js
- health/public/ui/default/js/vendor/jquery.datatables.min.js
- health/public/ui/default/js/vendor/jquery.peity.min.js
- health/public/ui/default
- health/public/tests/spec/helpers/*.js
- health/public/tests/spec/*-spec.js

I can connect browsers but testem can't find any files other than the ones directly in the same path as config.yml. Am I doing something wrong?

using the config file to auto launch in chrome, some chrome processes start in task manager, chrome tab appears in cli, but chrome does not open. Also if I open chrome manually testem seems to be watching files and running tests but does not send the changes to chrome. If I run live-reload in chrome it auto refreshes and picks up the changes. So it seems as if everything is working on the testem side but commands are not being passed to chrome.

I started with chrome v21 and then updated to v24. Firefox works perfectly without live-reload.

Could this be a security setting on my PC not allowing commands? I tried with 'run as administrator' and also tried disabling AV shields.

Thanks

When i use a "test_page" with mocha 1.4 or 1.6, the results are shown in the browser but not in the console.

if i include the file "public/testem/mocha.js" in my "test_page" everything works fine.

What version of mocha is public/testem/mocha.js?

Now testem express app just redirects to custom test page from site root "/". I think it would be convenient to render the page as template and pass there for example express app options as model data available for the template.

Not an issue per se, I just could not find it in the docs a way to hook up my existing qunit tests, to start using testem. Not sure this is the right forum for help, but i'll give it a shot...

I have a large application built using javascriptmvc and TDD. As part of the javascriptmvc framework we use stealjs, instead of requirejs for modular code. The HTML driver page loads stealjs, followed by the main qunit.js file (not qunit itself).

HTML file:

<script type='text/javascript' src='../steal/steal.js?corp/test/qunit'></script>

corp.test.qunit.qunit.js loaded by the statement above:

steal(  'funcunit/qunit' )
.then(
        'corp/common/test/qunit',
        'corp/subscription_manager/test/qunit',
        'corp/jquery/test/qunit',
        'corp/util/test/qunit'
);

The first statement on the snippet above loads in qunit itself.

• my tests live in corp/test/qunit/qunit.js

• i created a testem.yml in corp/testem.yml

• in the yml file I entered:

  framework: qunit
  src_files: - test/qunit/qunit.js

• when I run testem from corp/, it loads localhost:7357/testem/qunit.js, and not corp/test/qunit/qunit.js (which includes qunit itself, as described above)

What am I doing wrong? Any help/input appreciated...

Guilherme

whenever I add:

    <script>
if (location.hash === '#testem')
    document.write('<script src="/testem.js"></'+'script>')
</script>

to my test page all I get is Uncaught SyntaxError: Unexpected token < when I run my test page in the browser.

the odd thing is that the error location is line 1 so I'm wondering what this could be.

Using OSX 10.8.2 and firefox 16.0.1.

When the tests are run with either testem, testem -l Firefox or testem ci then firefox always asks to be set as default browser.

The tests are not executed again, when I press "enter" in development mode. Used version: 0.2.8.

My current testem.yml has just one line:

test_page: specs/runner.html

Currently we have all kinds of services requiring to add config files to the root folder of the project repository: grunt, travis-ci, npm, etc, etc...

My root folder could be a little bit clearer if i could run the following command line:

testem ci --launch phantomjs --test_page specs/runner.html

I have opera installed on my system. Testem ci -l does not list opera as one of the valid browsers, but when I open opera and navigate to the testem runner page, opera is detected as a connected browser. Opera is installed on my windows box at C:\Program Files (x86)\Opera

In case of CI mode for jenkins.
I want to execute many tests and output single test tap.
like a below specified json.

{
'test_page' : [
'test/1.html',
'test/2.html',
'*/1.html'
]
}

Currently, I temporarily developed the testem-multi that wrap testem for this case.
https://github.com/sideroad/testem-multi

Will you be fixing this case into testem in the future?

lib/server.js

•    // Force xhr-polling if you have issues with
  

  102

•    // reserved bits in Chrome and websockets.
  

  103

•    //this.io.set('transports', ['xhr-polling'])
  

This is a work around, but we might want to let people specify the transports supported.

I'm trying out Testem on a fairly large codebase (about 900 files incl tests) and I ran into the EMFILE error for too many open files pretty quickly (on OS X 10.7.5) when watching all those files.

I'm not very experienced with node, so I had to dig around quite a lot to fix it with this OS setting (found here):

sudo echo "limit maxfiles 4096 400000" >> /etc/launchd.conf

The default setting for the soft limit was 256 - far too low. The problem was that the testem-crash was quite cryptic, I could only see fragments of an error message in my Terminal window, like this:

em/node_modules/rimraf/rimraf.js:67:409:5)ncher.js:82:17)  

I think it would be helpful for newcomers if you could document this somewhere obvious, or ideally work around the issue (I don't know if it's possible).

Maybe the existing issue #29 for folder watching would solve it?

My config looks something like this:

"src_files": [
    "src/**/*.js",
    "lib/**/*.js",
    "test-fixtures/**/*.html"
],
"serve_files": [
    "test/**/*.js"
]

Only the tests are served. They pull in their dependencies, so the dependencies don't need to be served but watched.

Hi there,

I've found testem and I'm a little confused about the project. So if you don't mind, here are my questions.

• Is it possible to integrate testem with jenkins or hudson?
• Is there a way to get coverage reports?
• What is the main difference between testem and other distributed runners like js-test-driver?

Thanks, cheers
Miguel

testem ci -t test/mocha-1.6.0/index.html -p 8080 -l Safari

Does everything it's supposed to as far as opening the browser and running the tests.

After it runs Safari it just hangs:

Launching Safari

.

TAP version 13
ok 1 - Safari Should return getsBored should return gets bored

I have to ctrl c to get out.

Also - I noticed that if I run:

testem ci -t test/mocha-1.6.0/index.html

Does the same with Chrome:

Launching Chrome

.

TAP version 13
ok 1 - Chrome Should return getsBored should return gets bored

I tried setting a --timeout 3:

.
timers.js:96
if (!process.listeners('uncaughtException').length) throw e;
^
ReferenceError: browser is not defined
at Object._onTimeout (/usr/local/lib/node_modules/testem/lib/ci_mode_app.js:132:53)
at Timer.ontimeout (timers.js:94:19)

It would also be nice to specify where you want your testem.log file to go when debugging.

I'm on Lion.

I don't know what I am doing wrong, but running testem launchers does not find Chrome ?
I am using the node installer ( npm install testem -g).
I have testem running on a Windows 7 machine, as well as a Windows 8 machine, and both are not finding Chrome.
Chrome version is Version 22.0.1229.79 m

Also, running testem in ci mode hangs Opera, even though connecting to the local testem server works find.

new version is sweet apart from this nitpick.

This appears to be related to the following issue in socket.io:

socketio/socket.io-client#413

The issue is reported as being in IE6, and it's possible my work's quirky IE setup is to blame for the fact it's not working under IE7 standards mode. However, the issue was fixed for version 0.9.10 of Socket.io:

socketio/socket.io-client@335e8b2

Any chance you can do an upgrade to use Socket.io 0.9.10?

Run testem with phantom JS. Connect to page with Opera, Chrome, Firefox, and IE. Rerun tests on testem screen with either saving a file or hitting enter. Testem stops and throws this error:
"C:\Users\AppData\Roaming\npm\node_modules\testem\lib\appview.js:547
.write(Array(this.appview.get('cols') - startCol + 1).join(Chars.h
^
RangeError: Invalid array length
at [object Object].renderLine (C:\Users\AppData\Roaming\npm\node_modules\testem\lib\appview.js:547:20)
at [object Object].render (C:\Users\AppData\Roaming\npm\node_modules\testem\lib\appview.js:541:18)
at [object Object]. (C:\Users\AppData\Roaming\npm\node_modules\testem\lib\appview.js:595:25)
at [object Object].trigger (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:163:27)
at [object Object]._onModelEvent (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:844:20)
at [object Object].trigger (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:170:27)
at [object Object].add (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:631:15)
at [object Object].push (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:662:12)
at [object Object]. (C:\Users\AppData\Roaming\npm\node_modules\testem\lib\appview.js:585:25)
at [object Object].trigger (C:\Users\AppData\Roaming\npm\node_modules\testem\node_modules\backbone\backbone.js:163:27)"

Example output

# Open the URL below in a browser to connect.
# http://192.168.1.130:3580
# Ok! Starting tests with browsers: PhantomJS 1.5
# ....

ok 1 - PhantomJS 1.5  hello should say hello.
ok 2 - PhantomJS 1.5  hello level 2 should not say not hello.
ok 3 - PhantomJS 1.5  hello level 2 should throw another exception.
not ok 4 - PhantomJS 1.5  hello level 2 level 3 should throw exception.
  ---
    message: TypeError: 'undefined' is not an object
  ...

1..4
# tests 4
# pass  3
# fail  1

Expected output

# Open the URL below in a browser to connect.
# http://192.168.1.130:3580
# Ok! Starting tests with browsers: PhantomJS 1.5
# ....

ok 1 - PhantomJS 1.5  hello should say hello.
ok 2 - PhantomJS 1.5  hello level 2 should not say not hello.
ok 3 - PhantomJS 1.5  hello level 2 should throw another exception.
not ok 4 - PhantomJS 1.5  hello level 2 level 3 should throw exception.
  ---
    message: "TypeError: 'undefined' is not an object"
  ...

1..4
# tests 4
# pass  3
# fail  1

This is related to this isaacs/yamlish@dd495bb and isaacs/yamlish#2

norunner.html would be a simple

<!doctype html>
<html>
<head>
{{#scripts}}<script src="{{.}}"></script>{{/scripts}}
</head>
<body>
</body>
</html>

Red in the error tab is annoying me. I want "no colors" so that my own colouring of logging output shines through.

Also red is hard to read on my background which is reddish.

/home/raynos/Documents/chain-stream/node_modules/testem/lib/runners.js:245
        this.tapConsumer.removeAllListeners()
                         ^ test /home/raynos/Documents/chain-stream
TypeError: Cannot call method 'removeAllListeners' of null
    at Backbone.Model.extend.onTapEnd (/home/raynos/Documents/chain-stream/node_modules/testem/lib/runners.js:245:26)
    at TapConsumer.EventEmitter.emit (events.js:115:20)
    at TapConsumer.end (/home/raynos/Documents/chain-stream/node_modules/tap/lib/tap-consumer.js:98:8)
    at Socket.onend (stream.js:66:10)
    at Socket.EventEmitter.emit (events.js:115:20)
    at Pipe.onread (net.js:416:51)

Not sure why this is caused. Havn't looked into it. It clearly happens when something weird happens with my tap process.

It doesn't handle watching new files in an elegant fashion.

Ideally we want to add a watcher to each folder, detect new files added and then add any new files to the watcher

The workflow is

• most of the time when I save a test file I want you to only run that test file.
• sometimes when I save a test file I want to run the entire test suite.

Would be nice to have a clever file watcher that can run a sub set of my test suite.

upnode has a more stable reconnecting system.

The upnode test-runner to testem-server communication API is also a lot better then your custom socket.io based events

I was testing out the testem continuous integration feature, and if I try to test firefox, the prompt returns the following:

testem ci -b Firefox

Launching Firefox

1..0

tests 0

ok

but when I run from the browser all 17 tests I have in the folder display. If I run testem ci with all browsers, firefox works as intended.

Not accessible files from parent directory

serve_files:
- "../js/mycode.js"

I modified testem to allow additional configuration for express server application (in separate node module) to make it possible to expose for example REST services needed for tests or *.less compilation or any other custom server stuff.

Didn't you think about such feature?

If I try to specify
port: XXX
in testem.yml it does not work because it is progOptions and by default its 7357.

Is it supposed behavior?
I understand that command line options should probably override static settings in config, but shouldn't settings in config override default options?

Alex

In an ideal world I would rather not have to specify served_files vs src_files inside the testem.yml/json configuration file; Testem would just be smart enough to figure out that it should auto compile any coffee files it finds in src_files and serve those up automatically inside the generated test runners.

Any thoughts on making it easier to support simplifying this kind of workflow?

Hi,

this is the error I get, after I installed testem with npm and startet it on the cli: Can't set headers after they are sent.

Stacktrace:

http.js:644
    throw new Error('Can\'t set headers after they are sent.');
          ^
Error: Can't set headers after they are sent.
    at ServerResponse.OutgoingMessage.setHeader (http.js:644:11)
    at ServerResponse.res.setHeader (/usr/local/lib/node_modules/testem/node_modules/express/node_modules/connect/lib/patch.js:62:20)
    at exports.send (/usr/local/lib/node_modules/testem/node_modules/express/node_modules/connect/lib/middleware/static.js:168:11)
    at Object.oncomplete (fs.js:308:15)
    at process.startup.processMakeCallback.process._makeCallback (node.js:248:20)

Steps to reproduce:

• sudo npm install testem -g
• cd /tmp && mkdir test && cd test
• testem -l firefox
• => new tab is created, blank page, stacktrace is displayed
• or:
• testem
• => then I open http://localhost:7357/ in the browser and the exception is immediately displayed in the console, and a blank page in the browser

Test with testem 0.2.23 and 0.2.24.

I have no idea what's wrong.

If set config option

launch_in_dev:
- PhantomJS

but not install Phantom, it fails:

C:\Users\ekaragodin\AppData\Roaming\npm\node_modules\testem\lib\ci_mode_app.js:67                
            console.log("# Launching " + launcher.name)                                          
                                                 ^                                               
TypeError: Cannot read property 'name' of undefined                                              
    at App.runAllTheTests (C:\Users\ekaragodin\AppData\Roaming\npm\node_modules\testem\lib\ci_mod
e_app.js:67:50)         
...                                                                         

Need more clear error message.

Some unit tests are written in such a way that they can be run in either node or the browser (consider testling for example).

What would be the cleanest way the spawn a process that runs the tests in node and then prints their output in a testem tab.

[email protected] install /usr/local/lib/node_modules/testem/node_modules/socket.io/node_modules/socket.io-client/node_modules/ws
node install.js

sh: node: command not found
npm http GET https://registry.npmjs.org/traverse
npm http GET https://registry.npmjs.org/uglify-js
npm ERR! [email protected] install: node install.js
npm ERR! sh "-c" "node install.js" failed with 127
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the ws package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node install.js
npm ERR! You can get their info via:
npm ERR! npm owner ls ws
npm ERR! There is likely additional logging output above.

npm ERR! System Darwin 11.4.2
npm ERR! command "node" "/usr/local/bin/npm" "install" "testem" "-g"
npm ERR! cwd /Users/mbenin
npm ERR! node -v v0.8.6
npm ERR! npm -v 1.1.48
npm ERR! code ELIFECYCLE
npm http 304 https://registry.npmjs.org/traverse
npm http 304 https://registry.npmjs.org/uglify-js
npm ERR! Error: ENOENT, lstat '/usr/local/lib/node_modules/testem/node_modules/socket.io/node_modules/socket.io-client/node_modules/active-x-obfuscator/node_modules/zeparser/benchmark.html'
npm ERR! If you need help, you may report this log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]

npm ERR! System Darwin 11.4.2
npm ERR! command "node" "/usr/local/bin/npm" "install" "testem" "-g"
npm ERR! cwd /Users/mbenin
npm ERR! node -v v0.8.6
npm ERR! npm -v 1.1.48
npm ERR! path /usr/local/lib/node_modules/testem/node_modules/socket.io/node_modules/socket.io-client/node_modules/active-x-obfuscator/node_modules/zeparser/benchmark.html
npm ERR! fstream_path /usr/local/lib/node_modules/testem/node_modules/socket.io/node_modules/socket.io-client/node_modules/active-x-obfuscator/node_modules/zeparser/benchmark.html
npm ERR! fstream_type File
npm ERR! fstream_class FileWriter
npm ERR! code ENOENT
npm ERR! errno 34
npm ERR! fstream_stack Writer._finish.er.fstream_finish_call (/usr/local/lib/node_modules/npm/node_modules/fstream/lib/writer.js:284:26)
npm ERR! fstream_stack Object.oncomplete (fs.js:297:15)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /Users/mbenin/npm-debug.log
npm ERR! not ok code 0

Using Test'em I run 5000+ unit tests, each of which requires 3 additional files which are loaded using XMLHttpRequest. After a couple of hundred tests, Node crashes with the EMFILE error.

Adding the following line to server.js prevents this crash:

fs.watch = function(){};

I don't care about automatically refreshing. In a continuous integration environment it doesn't even make sense. A flag to disable the watching of files would be greatly appreciated.

When I try to run the spec along with the function in the same file, I got an error:

https://gist.github.com/2925403

function hello() {
  return "hello world";
}

describe("Hello World", function() {
  it("should say hello world", function() {
    expect(hello()).toBe("hello world");
  });
});

thanks,
Alberto.