Teamcity native support about tester HOT 6 CLOSED

Good idea 👍, but unfortunately I'm not using TC.

from tester.

My personal take would be to enrich the API with some sort of testStarted callback (similar to what I have done here), or a timing property as you suggested. That would enable anybody improve whatever reporter they are using. It could even enable some improvements of CLI reporter, as you could see which tests are in progress.

Also having more outputs would be useful, possibly just by passing more -o arguments (eg. -o machine.xml -o human.html).

As for the TeamCity OutputHandler, I'm not sure if Teamcity is so widely used that it is beneficial to bake its OutputHandler directly into Tester. Maybe it would be better to make a list or a repository of "recommended" output handlers. That would hopefully enable some decentralisation so that you wouldn't have to maintain an OutputHandler for each CI/CD solution, but rather "outsource" that to individual maintainers.

from tester.

Seems to me better to record start/stop event time as close to reality as possible. Moreover, adding new OutputHandler interface method is a big one BC break.

Also having more outputs would be useful, possibly just by passing more -o arguments (eg. -o machine.xml -o human.html).

Are you sure about -o? As I read your article, multiple --coverage would be handy. Anyway, for multiple -o is there #329 PR.

And about integrating TC into Tester directly... that's always a question. You mentioned, that Jiří Pudil uses it for PhpStorm plugin (@jiripudil Is that correct, that PhpStorm uses TC format internally too?). I think that better support for such wide-used tool as PhpStorm is worth it.

from tester.

Seems to me better to record start/stop event time as close to reality as possible. Moreover, adding new OutputHandler interface method is a big one BC break.

I don't know much timing error would change between callback and property approach. My guess is that most of the time, you'd want a precision of only tens of miliseconds, below that, you probably would get a lot of noise anyway, because of varying system load etc.
I used an opt-in interface in my solution so that back-compatibility wouldn't be broken, as old OutputHandlers could still use the old interface. Of course, it's one more near-useless interface that way...

Are you sure about -o?
You're right, I should be thinking more and typing less :-) Multiple --coverage is what would be useful in this scenario.

Good point about PHPStorm. That might make it worthwhile to integrate the output into the core.

from tester.

Is that correct, that PhpStorm uses TC format internally too?

That is indeed correct. Test duration is a nice addition too 👍 I agree with @veproza that reporting the actual start of the test would pave the way for a better UX, but I also agree with @milo that it's an unnecessary BC break too soon after the last major release. I'd postpone that until a 3.0 release is considered, and stick with a property in Test for now.

from tester.

I closing for lack of interest.

from tester.