Unequal number of tests in each partition about ember-exam HOT 7 CLOSED

Partitioning is currently done by file, or, more accurately, by AMD module. Ideally, you are correct and partitioning would be done at the test level, but this is currently not possible as once a test() call is evaluated there is no way to remove that test from the queue. The current implementation thus relies on the TestLoader to not load test modules in order to create partitions.

In the future, I think this could be possible if the partitioning logic is pushed upstream into QUnit itself, but this is a long way off and likely not to be completed any time in the near future.

I think a note about this in the README might be beneficial.

from ember-exam.

We see Chrome error: browser disconnected when running tests. It is not related to a specific test, it always happens after a successful test. It and looks like a memory leak that piles up as more tests are run.

So far, we have no idea how to debug the memory leak. I'm looking into ways of working around it.

An obvious way would be to split the test into many partitions, but it doesn't help. Regardless of the amount of partitions (I tried --split=10, --split=100, --split=1000, --split=9999), the first partition runs 340-380 tests until it reaches a browser disconnect. Then the next partition runs exact same cases and crashes in the same place ± 1-2 tests.

Also, --parallel does not seem to parallelize. It runs partitions sequentially. If I remove --parallel, the tests will finish after the first browser disconnect.

Please advise.

from ember-exam.

I just ran ember exam --split=9999 --parallel --random, and the first partition ran 827 tests before it crashes. On our non-failing branch we have less than 2000 tests total.

It looks like splitting doesn't work at all, please help.

from ember-exam.

We're running Ember CLI 2.15.

from ember-exam.

I had the same issue today. In testem.js I had parallel: '3' and I would see every partition running all the tests. I set parallel: '-1' and I think that's what got the split + parallel working for me. It may not exactly solve the "browser disconnects" but it seemed to help for me :S

from ember-exam.

@lolmaus it sounds like you have several distinct issues. It would be tremendously helpful if you could (1) open new issues for each distinct issue you are experiencing (as they don't seem to be related to this issue), and (2) provide some form of a reproduction. Thanks!

from ember-exam.

@trentmwillis Here you go: #108.

from ember-exam.