Skip to content

Code coverage issues, performance and discrepancies #1151

@Blebowski

Description

@Blebowski

A follow-up of: #1150

I see that the run-time of simple test (the device_id as in the previous issue) differs a lot when coverage
is enabled vs. the case when it is disabled (e.g. by factor of 4).

Majority of time is consumed due to the elaboration. When coverage is enabled, the code is always generated
AOT. E.g. progress bar with coverage:

** Note: initialising [9ms +53600kB]
** Note: loading top-level unit [17ms +6192kB]
** Note: elaborating design [3921ms 1.1x +1326692kB]
** Note: dumping coverage data [15ms +160kB]
** Note: code generation for 9641 units [12342ms 4.6x +1583648kB]
** Note: linking shared library [566ms +0kB]

vs without coverage:

** Note: initialising [9ms +53760kB]
** Note: loading top-level unit [18ms +6196kB]
** Note: elaborating design [3850ms 1.1x +1312896kB]
... simulation already starts

In my case DEFAULT_JIT=true. I tried to modify nvc.c not to enforce AOT generation when coverage is enabled,
and with coverage and AOT not enforced, the elab time with coverage is more-less similar as without coverage.
Is there a reason why with coverage AOT is enforced ?

I am surprised by such huge difference, most of the code for the TB needs to be generated anyway for the TB.
Maybe the difference is the code for those huge constant arrays reference_data_set_* that are not used in
this test, so that is where the savings are from (we discussed this in VUnit/vunit#1036).

There is one other issue I get. When running without coverage (therefore with JIT), I get simulation log flooded
with:

** Warning: 0ms+0: NUMERIC_STD.TO_INTEGER: metavalue detected, returning 0
   Function TO_INTEGER [UNRESOLVED_UNSIGNED return NATURAL] at ../lib/ieee.08/numeric_std-body.vhdl:3025
   Process :tb_top_ctu_can_fd:dut:rx_buffer_inst:rx_buffer_ram_inst:rx_buf_ram_inst:_p0 at /WORK/ctu_can_fd_clean/src/common_blocks/inf_ram_wrapper.vhd:191

at many simulation times. This is expected since I drive DUT inputs to X besides SETUP+HOLD around
the edge when DUT should sample them, so the metavalues do propagate to many places. I know I can disable this
with --ieee-warnings. What is strange is that I do not get these warnings fired when the coverage is enabled
(and code is generated AOT). I would expect equal behavior regardless of the compilation mode.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions