Comments (9)
For the failing pinmux tests, PR #22754 should get them running again.
from opentitan.
@rswarbrick: How much effort is left just to get pinmux numbers above the limit, what do you think?
from opentitan.
I spent some time looking at this yesterday and the answer is "quite a bit, for coverage" and also "but maybe the limit is wrong".
In more detail, across all the FPV tests in the issue:
- Pass rates are mostly high.
- The
pinmux_fpv
pass rates from last weekend were low, but these should have been fixed by #23216 (which was merged on Tuesday) - Other pass rates are high, except for
prim_fifo_sync_fpv
andprim_fifo_async_sram_adapter_fpv
. These are both tooling issues: probably worth working around in each case, but neither shows meaningful missing test coverage.
- The
- Coverage numbers for the IP FPV results (including for pinmux) are currently well under the 90% limit
- I think they are currently better than they were when we taped out the ES chip(!), so we probably weren't too worried then...
- There is some sensible work that we could do to include the coverage numbers, but it can probably reasonably go an the back burner.
from opentitan.
Thanks for your investigations @rswarbrick. IIRC, FPV is run weekly, so we'll get an update for #23216 next Monday.
IIRC you've also investigated the coverage gaps and reported that most of these gaps were due the COIs of our assertions not touching the signals in question, in particualr inside u_reg
, and further that this most likely hasn't been covered before. Further, you've also been checking the Earlgrey-M2.5.2-RC0
tag again and the results look better now.
Based on this, I am supportive to conclude this work for Earlgrey-PROD / close this M4 issue. It would probably be worth to document the open gaps and outline the next steps for closing these. This would give us a starting point for the future and will speed up things when we decide we need to more work here. Let's discuss this in the issue triage meeting.
from opentitan.
Discussed in triage: No regressions w.r.t. Earlgrey-ES. We also have added top-level test cases to cover pinmux permutations. prims are a known gap. The PROVE_COV
from prim_onehot_check
looks concerning, though. @rswarbrick What are your thoughts on that one?
from opentitan.
It's not all that bad:
|+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++|
| Instance Name | Formal | Stimuli | Checker |
| | Coverage | Coverage | Coverage |
|+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++|
|prim_onehot_check |100.00% (3/3) |100.00% (3/3) |0.00% (0/0) |
|+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++|
Maybe 0/0
should appear in grey in some way, rather than being a red 0%
. But there's no uncovered item here.
from opentitan.
@vogelpi to take a look
@rswarbrick future work would probably be to work around tooling issues (about prim). For pinmux it's just coverage.
@moidx much better than what we had before!
from opentitan.
The latest regression results look as follows.
As previously mentioned, we're done here. I am closing this issue and will create a new one to track potential future work (low priority). Thanks @rswarbrick for the hard work here!
from opentitan.
The new issue is here #23499
from opentitan.
Related Issues (20)
- [csrng/rtl] Discuss whether we should remove the `INVALID_GEN_CMD` status error response
- [test-triage] KMAC shake test vectors regression HOT 4
- [test-triage] `edn_sec_cm` failures HOT 1
- [entropy_src] Reference to main_sm_state in Stub mode uses incorrect path HOT 1
- [rom_ext] Do not honor boot service requests on deep sleep resume HOT 3
- [rom_ext] Add new boot service message to set both Primary and Next boot slot together
- Use Github Squash and Merge strategy for PRs HOT 9
- [questa] known-good output from passing sims
- [test-triage] i2c_target_hrst HOT 1
- [cryptolib] Allow garbage collection when linking OTBN binaries to reduce code size HOT 11
- [ot_certs,opentitantool] enable generating certificates in CBOR format HOT 3
- [hmac,dv] Break up NIST test vector HOT 1
- [spi_host] Missing TXEMPTY interrupt on Z1 silicon HOT 3
- [dvsim] Automatically generating reproduction commands in regression logs
- [test-triage] flash_ctrl - Check failed on interface flash_ctrl_eflash_reg_block
- [usbdev] vcs simulation error HOT 2
- [rom_ctrl,dv] Fix invalid check in scoreboard
- [rom_ctrl,dv] The stress_all_with_rand_reset sequence mostly fails
- [rom_ctrl,dv] Tweak the mask for invalid digests in rom_ctrl_base_vseq
- [rom_ctrl,dv] Make some assertions terminate
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opentitan.