Proper SenTestObserver log output about kiwi HOT 10 CLOSED

I've been looking into adding logging support to Kiwi, I'm just about to work on this so if you'd like to have some input into how it should work, do leave a reply.

I'm thinking that building in some logging hooks should be enough to allow people to write their own loggers by implementing a protocol. Some default loggers can be provided too.

The issue with using SenTestObserver is that all kiwi specs in a single context run within a single SenTest test case which would mean the SenTestObserver hooks aren't fine-grained enough.

from kiwi.

Incidentally, one of the reasons I wanted to do this is to aid writing a command-line utility for running specs; if spec details could be spat out in an easily parsable format (e.g. JSON) then it would be simple to write a script that runs xcodebuild and parses the JSON data on the fly to provide cleaner spec input (and perhaps similar output options to RSpec).

from kiwi.

Oh, so I just tweeted at somebody about this. I thought it was a bug, but because all my specs were coming up with:

Executed 1 test, with 0 failures (0 unexpected) in 0.004 (0.005) seconds

But that must just be the way its implemented. I'm pretty keen to get on this. Any pointers?

from kiwi.

Sounds like a good idea, the reason SenTest was 'hijacked' in the first place was because it reduced the barrier to entry, but I fought with getting 'proper' and customizable logging from day one. Never was satisfied in the end.

from kiwi.

I think building on top of SenTest is a good thing; it maintains Xcode integration and saves a lot of unnecessary work.

from kiwi.

Keith: you're right, because each KW_SPEC block essentially becomes a single SenTest test case, it's hard to report errors/failures properly using the built-in mechanism that SenTest provides.

I've already started work on improving this; the SenTest log output is quite important to Xcode as it relies on it to report failures within the GUI but I'm going to be working on a separate command-line runner that will be much more fully featured.

from kiwi.

Hi Luke -

I wanted to know where things stand with the commandline runner and if you could use some help on the effort. I developed a CLI runner for UISpec's test harness last year that works quite well (https://github.com/twotoasters/UISpecRunner). The runner works by wrapping xcodebuild and then supplying testing targets via environment variables (UISPEC_SPEC, UISPEC_EXAMPLE, etc). The tests are run via WaxSim, a shell script (headlessly) or AppleScript. Obviously some things would have to be reworked for SenTest, but there is probably useful overlap.

I am keen to fully adopt Kiwi for all my iOS projects going forward, but I am really missing the ability easily target subsets of the suite easily via the CLI.

from kiwi.

Luke, any info on how you are progressing on this? I have been thinking a bit about this recently.

from kiwi.

No progress on a command-line runner but it should be made significantly easier by my latest refactoring (see the one-test-per-example branch). More importantly, each spec is now a separate test case and each gets logged even on success. Check it out, I will merge to master soon.

from kiwi.

Merged into master.

from kiwi.