Comments (11)
If I recall correctly, the timeout is being thrown in the infrastructure code, which is a layer above the main test body (which I think is also being run in another thread). I think you can wrap the entire test (new MuDut()) { ... }
construct in a try-catch, and it might work?
Otherwise, can you give me an example of the use case? I'm not sure how this is best handled especially given the multithreaded nature of tests. Currently, uncaught exceptions in any thread are propagated up to the main thread, then all other threads interrupted and terminated. If you wanted to catch the TimeoutException, in which thread should it fire? Just the main thread? Or is it fine to it have it as an infrastructure-level exception that can't be caught within the test body?
Also, the intent behind TimeoutException is to allow recovery from infinite loops without killing and restarting sbt - it's essentially a last-ditch save. We could also think of other mechanisms to support your use case.
from chiseltest.
The short-term problem which I ran into is the case of writing a test library. I wanted a way to present a friendlier message to the user about some more details of the test that failed, with the current stacktrace it's not immediately obvious where the fault came from. (i.e. yes, the test failed, but what was the test library doing that led to it?)
Code wise I wanted to do something like: (not super fixed on this API, just need some way to express this intent)
try {
fork {
...
} .fork {
...
} .join
} catch {
case e: TimeoutException: reportError("Your foo bar test failed")
}
from chiseltest.
One idea for debugging was to allow named timescopes, which would be useful in localizing assertions (so you could get something like "assertion x != 1 failure in MyTransaction: SPI: bit 0: high", and not just a bunch of line numbers / stack trace). That might also provide more context for timeout failures.
That being said, this strategy certainly can't provide as much information as catch-rethrow, but it's also more lightweight. If you're already using timescopes, you just need to tack a name onto them, and you get assertion and timeout context essentially free. I think wrapping everything in try-catch is a bit cumbersome.
Anyways, I still don't think catching a TimeoutException is a good pattern (feel free to convince me otherwise?) - instead, if something is expected to finish within some time, that should be expressed with more intent-exposing constructs, eg expectWithin.
from chiseltest.
I'm fine with using named timescopes; then, is there a way to get the extra information out of it? It would make test libraries more pleasant to use if test libraries can help provide extra information. The main thing is to make the user aware of what the test library was doing that caused this.
val t = timescope(name="my_foo_bar", timeout=1000) {
fork { ...} .fork { ... } .join
}
if (t.hasExceptions) {
// get exception and report it
}
from chiseltest.
The idea is for the named timescope to provide information on what the library was doing (eg, SPI transaction, on bit 0, on the clock-high half-cycle). It wouldn't allow you to put in logic after-the-fact (eg, post-processing error messages) as you might when catching an exception, but you can have generated timescope names beforehand since the timescope name would take a string argument.
from chiseltest.
Is it possible to expand the named timescope idea to take a function or some user logic? e.g.
timescope(reportingFunc=myReportingFunc, timeout=1000) { ... }
It could also be a trait, object, or something else. Which ever API is fine, as long as there's a relatively easy way for test library writers to add information beyond just a string.
from chiseltest.
Also, if there's an experimental
workaround where it's possible to just catch the TimeoutException
directly, that's fine too. I understand developing a clean API takes time and thought.
from chiseltest.
So in my view there's two orthogonal issues here:
- Should TimeoutException be catchable in user test code? And which thread should get the exception if there are multiple threads? Note if all threads get the exception, and nothing in the thread catches it, you will get console spam indicating uncaught exception(s), which could just add to the confusion.
- How should the timescope name stack (or other context-dependent testers2 data) be obtained for the purposes of reporting exceptions? Is this sufficiently beneficial to outweigh the cost of adding special-case constructs? One possibility that I think is relatively clean would be to have a special Testers2Exception (which could be extended by more specific exceptions) that captures the current context-specific testers2 data that is known, like timescope name stack. I'm not convinced adding a reporting function is the best solution, since it doesn't seem super composable - it seems like it's replacing a try/catch block without looking like a try/catch block.
from chiseltest.
Can we have TimeoutException
as a subclass of Testers2Exception
?
from chiseltest.
Sure, but you still need to determine which thread gets the TimeoutException
, or, if it stays an infrastructure-level exception not visible from user test code, but rather the code invoking the driver, which thread's context does it capture from.
from chiseltest.
Typing up resolutions from two weeks ago...
When a TimeoutException fires, dump debugging info from all the threads.
It should provide good debugging trace information (especially if we have waitFor as in #39), though its won't be catchable from within a test.
from chiseltest.
Related Issues (20)
- JRE detects `EXCEPTION_ACCESS_VIOLATION` when trying to use Verilator as Chiseltest's backend HOT 2
- Report assert message with `FailedBoundedCheckException` HOT 1
- Generate waveform file in real-time HOT 1
- chiseltest gets the signal name wrong when trying to peek, poke, or expect an OpaqueType HOT 3
- Solver Chosen Constants for Formal Verification HOT 3
- scala.NotImplementedError: TODO: convert ThrowOnFirstErrorAnnotation HOT 3
- Bundle literal construction outside test() is not allowed in Chiseltest 5.0.0 (works in 0.5.4) HOT 2
- assertion failed: The Chisel compiler plugin is now required for compiling Chisel code HOT 1
- The waveform doesn't reflect changes in the input port until io.clock.peek HOT 1
- scala.NotImplementedError: TODO: convert DecodeTableAnnotatio HOT 7
- Will there be a chiseltest 6.0.0? HOT 16
- Frequent crash on macOS with the threaded Verilator backend HOT 6
- AXI4RAM test failed on chiseltest 5.0.2 HOT 2
- Cant ```import chiseltest._``` HOT 1
- Bitwuzla has changed it's command line argument format HOT 1
- [WARNING] Unsupported annotation: SRAMAnnotation
- [Help]A TLRAM test failed log HOT 8
- What are the future use cases of ChiselTest if it is replaced by ChiselSim? HOT 4
- Who is the copyright holder of chiseltest and what is the license? HOT 1
- one step takes extremely long time to complete HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chiseltest.