Comments (1)
From #968 (comment) users should have an ability to see test results in their preferred format.
For that, there could be templates that receive stream of test result JSON and format the output accordingly.
There should be an ability to create reports in multiple formats without rerunning the test suite. So the stream of test result objects or events could be saved as JSONL for later processing.
A bit of clarification WRT how tmt does things: we are talking about two places where "result reporting" happens. One is a progress report, emitted by execute
plugin while it's running as part of tmt run
, this one is now a stream of lines; the other is report
step, executed when all testing is done. The former is short-lived, run-time only, and cannot be replayed without re-running tests - run a test, collect its results, report the simplified outcome, move on to next test. The latter can be re-run many times because it depends on already collected results.
First thoughts:
- we can extend the
execute
step and plugins to use templates when rendering the progress reporting. Today the format is hardcoded, switching it to a template - and letting the user change the template - should provide us a way to introduce different formats of progress reporting intoexecute
. We can have a default template, we can provide TAP template or a template convertingtmt.result.Result
instances into JSON/YAML, and so on. - we can add a
report
plugin that would read collected results and use a given template to render their reporting. Similar to whathtml
already does, but with the ability to emit tostdout
.
And both places can use the very same template. The default/TAP/JSON/YAML/whatever else would be them emitted by execute
to stderr
while tmt run
is in action and by tmt run --last report -h new-plugin
any time later, for the same price. I like it.
from tmt.
Related Issues (20)
- Change the default unit of memory HW requirement HOT 8
- exit-first does not apply across discover phase boundaries
- Use relative `import`
- autopep8 generates invalid Python 3.9 multi-line f-string formatting
- Export `TMT_SOURCE_DIR` in prepare/finish steps as well HOT 1
- tmt reportportal plugin not reporting multi-host test results properly HOT 2
- Please print from where tmt is loading beakerlib libraries HOT 2
- Test discovery fails when `path` is used with `prune`
- Limit the maximum log size in reportportal plugin
- bug: tmt try multihost plan breaks terminal HOT 14
- tmt clean: 'NoneType' object is not iterable HOT 4
- tmt run failed with "Failed to connect in 120s" if user is offered(virtual only)
- /var/tmp/tmt result dir is removed if it is symlink and replaced by regular directory HOT 1
- [packit] Fedora Koji build failed to be triggered
- Add way to ignore any tmt tests in extracted sources for `discover -h fmf --dist-git-source` HOT 1
- tmt: can't disable lint check G001 HOT 1
- Add support for `environment` keyword in discover step for running tests with different arguments HOT 1
- tmt will try to clean all guests when users run tmt clean guests with --id specified
- can't use several instances of tmt run in parallel HOT 3
- tmt does not install well on Fedora 41 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tmt.