Check Library¶
Introduction¶
The check library is an assertion library for VHDL providing the more commonly used assertions as a number of check procedures and functions.
Architecture¶
The check subprograms are mainly a number conditional log messages. If the condition represents a failing check then an error message is issued using the VUnit log package.
Every check you do is handled by a checker. There is a default checker that is used when none is specified but you can also create multiple custom checkers. For example
check(re = '1', "Expected active read enable at this point");
will use the default checker while
check(my_checker, re = '1', "Expected active read enable at this point");
will use the custom my_checker. A checker is just a (shared)
variable
variable my_checker : checker_t;
All procedures presented in this user guide are available for both the
default checker and custom checkers. The difference is the first
checker parameter which only exists for custom checker procedures.
To make the user guide more compact we present this as an optional
parameter using brackets. For example
procedure checker_init (
[variable checker : inout checker_t;]
constant default_level : in log_level_t := error;
variable logger : inout logger_t);
The full verbose API description can always be found in check_api.vhd.
There are also a number of functions in the check library. These are
only available for the default checker and the reason is that
checker_t is defined as a protected type in the VHDL 2002 and 2008
implementations. Protected types cannot be input to a function so in
these cases the check package will provide a procedure which will return
the same result in an output parameter.
Checker Initialization¶
Since a check is a conditional log most of the checker initialization
parameters are the same as you’ll find for logger initialization. These
parameters are described in the log user
guide and are used to initialize the
checker internal logger that outputs any error message. However, the
default values for file_name and display_format are different
and there is also a default_level parameter unique to
checker_init. default_level is used to specify the log level for
the error message unless specified in the check subprogram as explained
in the following chapters.
procedure checker_init (
[variable checker : inout checker_t;]
constant default_level : in log_level_t := error;
constant default_src : in string := "";
constant file_name : in string := "error.csv";
constant display_format : in log_format_t := level;
constant file_format : in log_format_t := off;
constant stop_level : in log_level_t := failure;
constant separator : in character := ',';
constant append : in boolean := false);
It is also possible to initialize a checker to use the configuration of an already defined logger using the following procedure
procedure checker_init (
[variable checker : inout checker_t;]
constant default_level : in log_level_t := error;
variable logger : inout logger_t);
Note that the default checker with its default parameters covers most use cases such that
you don’t need to call checker_init.
Check¶
The check library provides a basic check procedure which is similar
to the VHDL assert statement
check(re = '1', "Expected active read enable at this point");
The first parameter is the boolean expression to check and the second parameter is the error message issued if the expression is false. Assuming this check fails and you haven’t changed the default settings for the default checker the error message will be
ERROR: Expected active read enable at this point
If you wish to have another log level than the default one set by
checker_init you can override this for each check call. For example
check(re = '1', "Expected active read enable at this point", failure);
A failing check is always counted as a failing check regardless of the
level but the level affects whether or not the simulation stops as
controlled by the stop_level parameter to checker_init.
Note that when using the VUnit Python test runner the default checker
stop_level is set to error when calling test_runner_setup.
The reason is that the Python test runner has the ability to restart the
simulation with the next test case so that all test cases are run
despite of an error while guaranteeing that the error state of the
failing test case doesn’t propagate into the next. If you’re not using
the Python test runner and have stop_level at failure as a way
to continue on error you don’t have this guarantee.
Logging Passing Checks¶
The provided message in a check call can also be logged when the check passes. The typical use case is to quickly create a debug trace to investigate what happened before a bug occured. This feature is disabled by default but can be enabled for all output handlers or just for one output handler.
enable_pass_msg;
enable_pass_msg(file_handler);
enable_pass_msg(display_handler);
disable_pass_msg with or without an output handler parameter can be used to disable
the feature again.
The difference between a passing check log message and a failing check log message is the log level used. A passing check like this
check(re = '1', "Checking that read enable is active");
will result in a log entry like this
PASS: Checking that read enable is active
Note that a message that reads well for both the pass and the fail cases was used. Note
also that PASS isn’t a standard log level but a custom log level defined by renaming
one of the extra log levels, debug_low2, provided by the logging library for the purpose
of creating custom levels like this.
A number of check subprograms perform several checks for every call, each of which can fail
and generate an error message. However, there will only be one pass message for such a call
to avoid confusion. For example, check_stable checks the stability of a signal for every
clock cycle in a window. If the window is 100 clock cycles there will be 100 checks for
stability but there will only be one pass message, not 100, if the signal is stable.
Message Format¶
In the previous examples the outputs from passing and failing checks were the messages provided by
the user with the addition of a log level. If we change the log format to raw there would be no
additions at all, just the user message. However, the check subprograms may also add information to the user
message before the log format additions are applied. For example, checking a pixel value after an image
processing operation can be done like this:
check_equal(output_pixel, reference_model(y)(x), "Comparing output pixel with reference model");
Resulting in an error message like this:
ERROR: Comparing output pixel with reference model - Got 1111_1010 (250). Expected 1111_1001 (249).
The last part of the message provides an error context to help debugging. Such a context is only given
if that provides extra information. In the case of a failing check we know that the input boolean is
false so there is no need to provide that information. The context may also be different between pass and
error messages. For example, a pass message from check_equal looks like this:
PASS: Comparing output pixel with reference model - Got 1111_1010 (250).
Redundancy is avoided by excluding the expected value which is the same as the value received.
So far I’ve used a message that reads well in both the passing and the failing case. The check library
also provides another way of doing this using the result function. The call
check_equal(output_pixel, reference_model(y)(x), result("for output pixel"));
gives the following messages:
ERROR: Equality check failed for output pixel - Got 1111_1010 (250). Expected 1111_1001 (249).
and
PASS: Equality check passed for output pixel - Got 1111_1010 (250).
The result function prepends the provided string with the check type (equality check in this case)
and passed/failed depending on the result. The result function is also used as the default value for
check calls so
check_equal(output_pixel, reference_model(y)(x));
gives the following messages:
ERROR: Equality check failed - Got 1111_1010 (250). Expected 1111_1001 (249).
and
PASS: Equality check passed - Got 1111_1010 (250).
If you look at the default value for the user message in the check subprogram APIs you will see that the
result function isn’t used. This is a workaround for one of the supported simulators which exposes the
internal implementation of the result function (a magic constant prepending the user message).
You shouldn’t use the magic constant yourself since that implementation may change at any time. For that reason
I’m also keeping the result function in the APIs presented in this user guide.
Check Location¶
The check procedures described in the previous sections have two
additional parameters, line_num and file_name. These are
normally not set by the user but by the location preprocessor such that
the location of a failing check is included in the error message. The
location preprocessor is further described in the log user
guide. Preprocessor parameters are always
placed at the end of the parameter list and they have “good” default
values such that the function behaves nicely even if the preprocessor
isn’t used.
procedure check(
[variable checker : inout checker_t;]
constant expr : in boolean;
constant msg : in string := result(".");
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "");
Acting on Failing Checks¶
The check procedure described so far doesn’t reveal whether the
check passed or not. If you want that information to control the flow of
your test and your testbench is setup to continue on a failing check you
have a number of options. You can use this procedure where the pass
output is false on a failing check
procedure check(
[variable checker : inout checker_t;]
variable pass : out boolean;
constant expr : in boolean;
constant msg : in string := result(".");
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "");
or you can use this function which returns the same information
impure function check(
constant expr : in boolean;
constant msg : in string := result(".");
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "")
return boolean;
or you can see if there has been any errors so far
procedure checker_found_errors (
[variable checker : inout checker_t;]
variable result : out boolean);
impure function checker_found_errors
return boolean;
or you can use any of the following subprograms to get more details.
procedure get_checker_stat (
[variable checker : inout checker_t;]
variable stat : out checker_stat_t);
impure function get_checker_stat
return checker_stat_t;
checker_stat_t is a record containing pass/fail information.
type checker_stat_t is record
n_checks : natural;
n_failed : natural;
n_passed : natural;
end record;
Note that a check subprogram with many internal checks may
generate several error messages if the simulation isn’t stopped by an error.
Each such error will add one to n_checks and n_failed. However, if
the check pass n_checks and n_passed will only be increase by one.
The reason for this is the same as for the single pass message approach, that
is to avoid mismatch between the pass statistics and the number of passing
check subprogram calls.
Managing Checker Statistics¶
A checker will continuously update its statistics counters as new check
subprograms are called. If you want to collect the statistics for parts
of your test you can make intermediate readouts using the
get_checker_stat subprograms and then reset the counters to zero
using
procedure reset_checker_stat [(
variable checker : inout checker_t)];
Another way of collecting statistics for different parts is to use several separate checkers.
Variables of type checker_stat_t can be added to or subtracted from
each other using the normal - and + operators. There is also a
to_string function defined to allow for logging/reporting of
statistics, for example
info(to_string(get_checker_stat));
Check Types¶
In addition to the basic check subprograms the check library also
provides a number of more specialized checks. These checks can be
divided into four different types
- Point checks
- Relation checks
- Sequential checks
- Unconditional checks
These types and the checks belonging to each type are described in the following chapters.
Point Checks¶
Common to all point checks is that the condition for failure is
evaluated at a single point in time, either when the subprogram is
called as part of sequential code or synchronous to a clock in a clocked
and usually concurrent procedure call. There are five unclocked versions
of each point check and they correspond to the function and four
procedures previously described for check. The only difference to the
parameter lists is that the boolean expr parameter is replaced by
one or more parameters specific to the point check.
The unclocked procedures have the following format. The four variants comes from the different combinations of using the two first optional parameters.
procedure check<_name>(
[variable checker : inout checker_t;]
[variable pass : out boolean;]
<specific parameters>
constant msg : in string := result<(".")>;
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "");
The function has the following format.
impure function check<_name>(
<specific parameters>
constant msg : in string := result<(".")>;
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "")
return boolean;
The clocked procedures come from the following format with and without
the optional parameter. These procedures are also available for check.
procedure check<_name>(
[variable checker : inout checker_t;]
signal clock : in std_logic;
signal en : in std_logic;
<specific parameters>
constant msg : in string := result<(".")>;
constant level : in log_level_t := dflt;
constant active_clock_edge : in edge_t := rising_edge;
constant line_num : in natural := 0;
constant file_name : in string := "");
edge_t is an enumerated type:
type edge_t is (rising_edge, falling_edge, both_edges);
The condition for failure is continuously evaluated on the clock edge(s)
specified by active_clock_edge as long as en = '1'.
The figure below shows an example using the concurrent version of
check.
expr is evaluated on every rising clock edge except for edge 3 where
en is low. This means that the check will pass despite the false expr in
the third clock cycle.
(True) Check (check and check_true)¶
| Specific Parameter | Type |
|---|---|
| expr | boolean or std_logic |
check_true is a more verbose version of check which
emphasises that we’re expecting expr to be true/1/H.
The extra verbosity is also present when the result function is used.
check(false, result("for my data.");
will result in
ERROR: Check failed for my data.
while
check_true(false, result("for my data.");
will result in
ERROR: True check failed for my data.
False Check (check_false)¶
| Specific Parameter | Type |
|---|---|
| expr | boolean or std_logic |
check_false passes when expr is false/0/L.
Implication Check (check_implication)¶
| Specific Parameter | Type |
|---|---|
| antecedent_expr | boolean or std_logic |
| consequent_expr | boolean or std_logic |
The unclocked subprograms use boolean parameters while the clocked
procedures use std_logic.
check_implication checks logical implication and passes unless
antecedent_expr is true/1/H and consequent_expr is
false/0/L.
Not Unknown Check (check_not_unknown)¶
| Specific Parameter | Type |
|---|---|
| expr | std_logic_vector or std_logic |
check_not_unknown passes when expr contains none of the
metavalues U, X, Z, W, or -.
Zero One-Hot Check (check_zero_one_hot)¶
| Specific Parameter | Type |
|---|---|
| expr | std_logic_vector |
check_zero_one_hot passes when expr contains none of the
metavalues U, X, Z, W, or - and there are zero or
one bit equal to 1 or H .
One-Hot Check (check_one_hot)¶
| Specific Parameter | Type |
|---|---|
| expr | std_logic_vector |
check_one_hot passes when expr contains none of the metavalues
U, X, Z, W, or - and there is exactly one bit equal
to 1 or H .
Relation Checks¶
Relation checks are used to check whether or not a relation holds
between two expressions, for example if (a + b) = c. They support
the following five unclocked formats.
procedure check_<name>(
[variable checker : inout checker_t;]
[variable pass : out boolean;]
<specific parameters>
constant msg : in string := result;
constant level : in log_level_t := dflt;
<preprocessor parameters>);
impure function check_<name>(
<specific parameters>
constant msg : in string := result;
constant level : in log_level_t := dflt;
<preprocessor parameters>)
return boolean;
Equality Check (check_equal)¶
| Specific Parameter |
|---|
| got |
| expected |
The got and expected parameters can have the following
combination of types
| got | expected |
|---|---|
| unsigned | unsigned |
| natural | unsigned |
| unsigned | natural |
| natural | std_logic_vector |
| std_logic_vector | natural |
| std_logic_vector | std_logic_vector |
| std_logic_vector | unsigned |
| unsigned | std_logic_vector |
| signed | signed |
| integer | signed |
| signed | integer |
| integer | integer |
| std_logic | std_logic |
| boolean | std_logic |
| std_logic | boolean |
| boolean | boolean |
| time | time |
| string | string |
| Preprocessor Parameter | Type | Default Value |
|---|---|---|
| line_num | natural | 0 |
| file_name | string | “” |
check_equal passes when got equals expected. When comparing
std_logic values with boolean values 1 equals true and
0 equals false. Note that the std_logic don’t care (-)
only equals itself. If you want an equality like "0011" = "00--" to
pass you should use check_relation with the matching equality
operator (?=) or check_match instead.
If a check fails you will get a context on the following format.
Got <got value>. Expected <expected value>.
When you compare bit vectors, integer and natural type of values
the error message will output the values on both formats. For example,
here is a context when a check_equal between an integer and a signed
value fails.
Got 17 (0001_0001). Expected 0001_0000 (16).
Relation Check (check_relation)¶
| Specific Parameter | Type |
|---|---|
| expr | boolean, std_ulogic, or bit |
| Preprocessor Parameter | Type | Default Value |
|---|---|---|
| context_msg | string | “” |
| line_num | natural | 0 |
| file_name | string | “” |
expr is intended to be a relational expression and three different
types are supported. In case a matching relational operator is used the
relation will return a std_ulogic or bit depending on the
operands. All other relations will return a boolean.
check_relation passes when expr evaluates to true in the
boolean case and to 1 in the std_ulogic and bit cases. This
means that the boolean case behaves just like check and check_true. The
additional value of this check comes when you enable the check
preprocessor in your VUnit run script.
ui = VUnit.from_argv()
ui.enable_check_preprocessing()
The check preprocessor scans your code for calls to check_relation
and then parses expr as a VHDL relation. From that it will generate
a context (context_msg parameter) describing how the relation failed.
For example, the check
check_relation(real_time_clock <= timeout, "Response too late.");
will generate the following error message if it fails.
ERROR: Response too late - Expected real_time_clock <= timeout. Left is 23:15:06. Right is 23:15:04.
This works for any type of relation between any types as long as
the operator and the to_string function are defined for the types
involved. In the example the operands are of a custom clock_t type for
which both the <= operator and the to_string function have been
defined.
Note that context_msg is the empty
string by default so without the check preprocessor the error message
will be just the msg provided by the user.
Relations with Side Effects¶
The left and right hand sides of the relation are evaluated twice, once when the relation is evaluated and once to create the error message so if you have a call like this
check_relation(counter_to_verify = get_and_increment_reference_counter(increment_with => 3));
The reference counter will be incremented with 6 which is not what you
expect by just looking at the code before the preprocessor has generated
the context_msg which will be a string containing
to_string(get_and_increment_reference_counter(increment_with => 3)).
Conclusion: Do not use impure functions in your expression. If you have a case like this you can do something like
ref_cnt := get_and_increment_reference_counter(increment_with => 3);
check_relation(counter_to_verify = ref_cnt);
or since this is an equality relation, probably between standard
countable types, use check_equal instead. check_equal has the
left and right hand operands separated in the call itself so in that
case there is no need for a second evaluation in order to create the
error message.
Fooling the Parser¶
The check preprocessor has a simplified parser to determine what the relation operator in the expression is and what the left and right hand operands are. For example, it knows that this is an inequality since that is the only relational operator on the “top-level”.
check_relation((a = b) /= (c = d));
It also knows that this isn’t a relation since there’s no relational operator on the top-level.
check_relation((a = b) and c);
This will result in a syntax error from the check preprocessor
SyntaxError: Failed to find relation in check_relation((a = b) and c)
However, its knowledge about precedence is limited to parenthesis so it will not understand that this identical expression isn’t a relation.
check_relation(a = b and c);
If this logical expression returns false the check will generate an
error message claiming that a relation failed and that to_string(a)
was the left value and to_string(b and c) was the right value.
Conclusion: Use check_relation for relations as intended!
It should also be noted that the parser can handle that there are
relational operators within the check call but outside of the expr
parameter. For example, it won’t be fooled by the relational operators
appearing within strings and comments of this call.
check_relation(len("""Heart"" => <3") = -- The string contains <, so does
-- this comment
12, "Incorrect length of ""<3 string"".");
Match Check (check_match)¶
| Specific Parameter |
|---|
| got |
| expected |
The got and expected parameters can have the following
combination of types
| got | expected |
|---|---|
| unsigned | unsigned |
| std_logic_vector | std_logic_vector |
| signed | signed |
| std_logic | std_logic |
| Preprocessor Parameter | Type | Default Value |
|---|---|---|
| line_num | natural | 0 |
| file_name | string | “” |
check_match passes when got equals expected but differs from
check_equal in that a don’t care (-) bit equals anything.
Sequence Checks¶
Sequence checks are checks that use several clock cycles to determine whether or not the desired property holds.
Stability Check (check_stable)¶
check_stable supports four different clocked formats. The expr
parameter can be std_logic or std_logic_vector and the call can
be made with or without the initial custom checker parameter.
procedure check_stable(
[variable checker : inout checker_t;]
signal clock : in std_logic;
signal en : in std_logic;
signal start_event : in std_logic;
signal end_event : in std_logic;
signal expr : in std_logic or std_logic_vector;
constant msg : in string := result;
constant level : in log_level_t := dflt;
constant active_clock_edge : in edge_t := rising_edge;
constant line_num : in natural := 0;
constant file_name : in string := "");
check_stable passes if the expr parameter is stable in the
window defined by the start_event and end_event parameters. The
window starts at an active (according to active_clock_edge) and
enabled (en = '1') clock edge for which start_event = '1' and it
ends at the next active and enabled clock edge for which
end_event = '1'. expr is sampled for a reference value at the
start event and is considered stable if it keeps that reference value at
all enabled active clock edges within the window, including the clock
edge for the end event. Bits within expr may change drive strength
(between '0' and 'L' or between '1' and 'H') and still be considered
stable. Below is an example with two windows that will pass.
Here are two examples of failing checks. Note that any unknown value
(U, X, Z, W, or -) will cause the check to fail even
if the unknown value is constant. The check will also fail if
start_event or end_event in an active window has an unknown
value.
check_stable can handle one clock cycle windows and back-to-back
windows.
When allow_restart is false check_stable will ignore additional start events in the window.
When allow_restart is true a new window is started if a new start event appears before the end event.
The previous window is implicitly closed in the clock cycle before the new start event. An end event will
still close the window if it appears before a second start event.
Next Check (check_next)¶
check_next supports two different formats. One with and one without
the initial custom checker parameter.
procedure check_next(
[variable checker : inout checker_t;]
signal clock : in std_logic;
signal en : in std_logic;
signal start_event : in std_logic;
signal expr : in std_logic;
constant msg : in string := result;
constant num_cks : in natural := 1;
constant allow_overlapping : in boolean := true;
constant allow_missing_start : in boolean := true;
constant level : in log_level_t := dflt;
constant active_clock_edge : in edge_t := rising_edge;
constant line_num : in natural := 0;
constant file_name : in string := "");
check_next passes if expr = '1' num_cks active (according to
active_clock_edge) and enabled (en = '1') clock edges after a
start event. The start event is defined by an active and enabled clock
edge for which start_event = '1'. Below is an example of a passing
check. The start event is sampled at clock edge two. expr is
expected to be high four enabled clock edges after that which is at
clock edge seven due to en being low at clock edge five.
When allow_overlapping is true check_next will allow a new
start event before the check based on the previous start event has been
completed. Here is an example with two overlapping and passing
sequences.
In case allow_overlapping is false check_next will fail at
the second start event
When allow_missing_start is true check_next will allow
expr = '1' when there is no corresponding start event. When
allow_missing_start is false such a situation will lead to a
failure.
Any unknown value (U, X, Z, W, or -) on start_event
will cause an error.
check_next will handle the weak values L and H in the same
way as 0 and 1, respectively.
Sequence Check (check_sequence)¶
check_sequence supports two different formats. One with and one
without the initial custom checker parameter.
procedure check_sequence(
[variable checker : inout checker_t;]
signal clock : in std_logic;
signal en : in std_logic;
signal event_sequence : in std_logic_vector;
constant msg : in string := result;
constant trigger_event : in trigger_event_t := penultimate;
constant level : in log_level_t := dflt;
constant active_clock_edge : in edge_t := rising_edge;
constant line_num : in natural := 0;
constant file_name : in string := "");
check_sequence passes if a number of events, represented by the bits
in the event_sequence parameter, are activated (bit = '1' or
'H') in sequence at consecutive active (according to
active_clock_edge) and enabled (en = '1') clock edges.
check_sequence supports three different modes of operation as
controlled by the trigger_event parameter:
first_pipe- The sequence is started when the leftmost bit ofevent_sequenceis activated. This will also triggercheck_sequenceto verify that the remaining bits are activated at the following active and enabled clock edges.check_sequencewill also verify new sequences starting before the first is completed.
The figure below shows two overlapping sequences that pass.
In this example the sequence is started but not completed and the check fails.
first_no_pipe- Same asfirst_pipewith the exception that only one sequence is verified at a time. New sequences starting before the previous is verified will be ignored.
In this example we have two sequences, the first is completed while the second is interrupted. However, since only one sequence is handled at a time the second is ignored and the check pass.
penultimate- The difference with the previous modes is thatcheck_sequenceonly verifies the last event (the rightmost bit) when all the preceding events in the sequence have been activated. This means that a started sequence that is interrupted before the second to last bit is activated will pass.check_sequencewill also verify new sequences starting before the first is completed.
The figure below shows two overlapping sequences which pass and then an
early interrupted sequence that doesn’t cause a failure in this mode
(which it did in the example for the first_pipe mode.
In this example the sequence is interrupted after the second to last bit is activated and the check fails.
Any unknown values (U, X, Z, W, or -) in event_sequence
will lead to a an error. This is regardless of the mode of operation.
Unconditional Checks¶
The check library has two unconditional checks, check_passed and
check_failed, that contains no expression parameter to evaluate.
They are used when the pass/fail status is already given by the program
flow. For example,
if <some condition> then
<do something>
check_passed;
else
<do something else>
check_failed("This was not expected");
end if;
With no expr parameter there are also fewer usable formats for these
checkers.
procedure check_passed(
[variable checker : inout checker_t;]
constant msg : in string := result(".");
constant line_num : in natural := 0;
constant file_name : in string := "");
procedure check_failed(
[variable checker : inout checker_t;]
constant msg : in string := result(".");
constant level : in log_level_t := dflt;
constant line_num : in natural := 0;
constant file_name : in string := "");
