Improved Support for VHDL Configurations and OSVVM¶

For quite some time, several initiatives have been underway to improve the integration between VUnit and OSVVM. Examples of these efforts are the external logging framework integration feature and the OSVVM pull request #81.

Another example is the introduction of support for top-level VHDL configurations which serves several purposes, for example:

Enabling the selection of the Device Under Test (DUT) to be used in a VUnit testbench.
Direct support for running conventional OSVVM testbenches within the VUnit framework.

In this blog, we will primarily focus on top-level configurations but before delving into the specifics of these use cases, we will describe how VUnit addressed these issues in the past.

Selecting DUT Using Generics¶

Sometimes the VHDL DUT comes in different variants (architectures) and there is a need to verify all of these with the same testbench. It could be an FPGA and an ASIC implementation or an RTL and a behavioral architecture. Before supporting VHDL configurations, VUnit addressed this issue with a combination of generics and an if-generate statement as showed in the example below. For the purpose of this blog, we have removed the complexities typically found in real-world designs and chosen to focus on the fundamental principles. Thus, we will use a simple variable-width flip-flop as the DUT for our demonstrations.

entity tb_selecting_dut_with_generics is
  generic(
    runner_cfg : string;
    width : positive;
    dff_arch : string
  );
end entity;

architecture tb of tb_selecting_dut_with_generics is
  ...
begin
  test_runner : process
  begin
    test_runner_setup(runner, runner_cfg);

    while test_suite loop
      if run("Test reset") then
        ...
      elsif run("Test state change") then
        ...
      end if;
    end loop;

    test_runner_cleanup(runner);
  end process;

  test_runner_watchdog(runner, 10 * clk_period);

  test_fixture : block is
  begin
    clk <= not clk after clk_period / 2;

    dut_selection : if dff_arch = "rtl" generate
      dut : entity work.dff(rtl)
        generic map(
          width => width
        )
        port map(
          clk => clk,
          reset => reset,
          d => d,
          q => q
        );

    elsif dff_arch = "behavioral" generate
      dut : entity work.dff(behavioral)
        generic map(
          width => width
        )
        port map(
          clk => clk,
          reset => reset,
          d => d,
          q => q
        );

    else generate
      error("Unknown DFF architecture");
    end generate;
  end block;
end architecture;

This approach is straightforward: simply copy and paste the flip-flop instantiation, but modify the architecture to use based on the dut_arch generic. While the approach is simple it also introduces code duplication which can be a bit dangerous. In this case, since the copies are placed adjacent to each other, the risk of inadvertently changing one without updating the other is somewhat mitigated.

If your DUT has numerous ports, you can consider leveraging the VHDL-2019 interface construct as a means to raise the level of abstraction and reduce code duplication. This approach allows for a more concise representation of the design, provided your simulator supports the latest VHDL standard.

Note

There is a proposed update to the VHDL standard related to this topic as it would fully remove the code duplication. Issue #235 proposes that a string should be possible to use when specifying the architecture in an entity instantiation, i.e. "rtl" or "behavioral" rather than rtl or behavioral. In our example we would simply have a single entity instantiation which architecture is specified with the dut_arch generic.

The various settings of the dut_arch generic are handled with a VUnit configuration in the Python run script. Initially, the use of both VUnit and VHDL configuration concepts may appear confusing, but we will soon see that a VHDL configuration is a special case of the broader VUnit configuration concept. In this example, we are also testing the DUT with multiple width settings. Note how we can use the product function from itertools to iterate over all combinations of dut_arch and width. This is equivalent to two nested loops over these generics but scales better as the number of generics to combine increases.

tb = lib.test_bench("tb_selecting_dut_with_generics")

for dut_arch, width in itertools.product(["rtl", "behavioral"], [8, 16]):
    tb.add_config(
        name=f"{dut_arch}_{width}",
        generics=dict(width=width, dff_arch=dut_arch),
    )

If we list all the tests, we will see that there are four for each test case in the testbench, one for each combination of dut_arch and width:

> python run.py --list
lib.tb_selecting_dut_with_generics.rtl_8.Test reset
lib.tb_selecting_dut_with_generics.rtl_16.Test reset
lib.tb_selecting_dut_with_generics.behavioral_8.Test reset
lib.tb_selecting_dut_with_generics.behavioral_16.Test reset
lib.tb_selecting_dut_with_generics.rtl_8.Test state change
lib.tb_selecting_dut_with_generics.rtl_16.Test state change
lib.tb_selecting_dut_with_generics.behavioral_8.Test state change
lib.tb_selecting_dut_with_generics.behavioral_16.Test state change
Listed 8 tests

Selecting DUT Using VHDL Configurations¶

When using VHDL configurations we need three ingredients in our testbench

A component declaration for the DUT. In the example below it has been placed in the declarative part of the testbench architecture but it can also be placed in a separate package.
A component instantiation of the declared component. Note that the component keyword is optional and can be excluded.
A configuration declaration for each DUT architecture

entity tb_selecting_dut_with_vhdl_configuration is
  generic(
    runner_cfg : string;
    width : positive
  );
end entity;

architecture tb of tb_selecting_dut_with_vhdl_configuration is
  ...

  -- Component declaration
  component dff is
    generic(
      width : positive := width
    );
    port(
      clk : in std_logic;
      reset : in std_logic;
      d : in std_logic_vector(width - 1 downto 0);
      q : out std_logic_vector(width - 1 downto 0)
    );
  end component;
begin
  test_runner : process
  begin
    test_runner_setup(runner, runner_cfg);

    while test_suite loop
      if run("Test reset") then
        ...
      elsif run("Test state change") then
        ...
      end if;
    end loop;

    test_runner_cleanup(runner);
  end process;

  test_runner_watchdog(runner, 10 * clk_period);

  test_fixture : block is
  begin
    clk <= not clk after clk_period / 2;

    -- Component instantiation
    dut : component dff
      generic map(
        width => width
      )
      port map(
        clk => clk,
        reset => reset,
        d => d,
        q => q
      );
  end block;
end architecture;

-- Configuration declarations
configuration rtl of tb_selecting_dut_with_vhdl_configuration is
  for tb
    for test_fixture
      for dut : dff
        use entity work.dff(rtl);
      end for;
    end for;
  end for;
end;

configuration behavioral of tb_selecting_dut_with_vhdl_configuration is
  for tb
    for test_fixture
      for dut : dff
        use entity work.dff(behavioral);
      end for;
    end for;
  end for;
end;

Instead of assigning a generic to select our architecture, we now specify which VHDL configuration our VUnit configuration should use:

tb = lib.test_bench("tb_selecting_dut_with_vhdl_configuration")

for dut_arch, width in itertools.product(["rtl", "behavioral"], [8, 16]):
    tb.add_config(
        name=f"{dut_arch}_{width}",
        generics=dict(width=width),
        vhdl_configuration_name=dut_arch,
    )

Incorporating VHDL configurations within VUnit configurations brings forth another advantage. From a VHDL point of view, VHDL configurations are linked to entities, such as the testbench entity in our scenario. However, a VUnit configuration can also be applied to specific test cases, opening up the possibility of using VHDL configurations at that finer level of granularity. For instance, consider a situation where we have an FPGA and an ASIC implementation/architecure that differ only in the memory IPs they use. In such a case, it might be sufficient to simulate only one of the architectures for the test cases not involving memory operations.

To illustrate this using the flip-flop example, let’s create a test where we set width to 32 and exclusively simulate it using the RTL architecture:

tb.test("Test reset").add_config(name="rtl_32", generics=dict(width=32), vhdl_configuration_name="rtl")

Now, we have an additional entry in our list of tests:

> python run.py --list
lib.tb_selecting_dut_with_vhdl_configuration.rtl_8.Test reset
lib.tb_selecting_dut_with_vhdl_configuration.rtl_16.Test reset
lib.tb_selecting_dut_with_vhdl_configuration.behavioral_8.Test reset
lib.tb_selecting_dut_with_vhdl_configuration.behavioral_16.Test reset
lib.tb_selecting_dut_with_vhdl_configuration.rtl_32.Test reset
lib.tb_selecting_dut_with_vhdl_configuration.rtl_8.Test state change
lib.tb_selecting_dut_with_vhdl_configuration.rtl_16.Test state change
lib.tb_selecting_dut_with_vhdl_configuration.behavioral_8.Test state change
lib.tb_selecting_dut_with_vhdl_configuration.behavioral_16.Test state change
Listed 9 tests

Choosing between VHDL configurations and generics is primarily a matter of personal preference. The generic approach led us to multiple direct entity instantiations and code duplication. However, the configuration approach demands a component declaration, which essentially duplicates the DUT entity declaration. Additionally, VHDL configuration declarations are also necessary.

Selecting Test Runner Using VHDL Configurations¶

In the previous examples, the VUnit test cases were located in a process called test_runner residing alongside the DUT. This is the most straightforward arrangement, as it provides the test cases with direct access to the DUT’s interface. An alternative approach involves encapsulating test_runner within an entity, which is subsequently instantiated within the testbench. Such a test_runner entity needs access to the runner_cfg and width generics, in addition to the clk_period constant and the interface ports of the DUT.

entity test_runner is
  generic(
    clk_period : time;
    width : positive;
    nested_runner_cfg : string
  );
  port(
    reset : out std_logic;
    clk : in std_logic;
    d : out std_logic_vector(width - 1 downto 0);
    q : in std_logic_vector(width - 1 downto 0)
  );
end entity;

Note that the runner configuration generic is called nested_runner_cfg and not runner_cfg. The reason is that runner_cfg is the signature used to identify a testbench, the top-level of a simulation. The test_runner entity is not a simulation top-level and must not be mistaken as such.

We can now replace the testbench test_runner process and watchdog with an instantiation of this component:

test_runner_inst : component test_runner
  generic map(
    clk_period => clk_period,
    width => width,
    nested_runner_cfg => runner_cfg
  )
  port map(
    reset => reset,
    clk => clk,
    d => d,
    q => q
  );

Having relocated test_runner into an entity, we can have VHDL configurations selecting which test runner to use, and let each such test runner represent a single test. This setup is the conventional methodology seen in OSVVM testbenches. With VUnit’s extended support for VHDL configurations, it becomes possible to keep that structure when adding VUnit capabilities. For example, this is the architecture for the reset test:

architecture test_reset_architecture of test_runner is
begin
  main : process
  begin
    test_runner_setup(runner, nested_runner_cfg);

    -- Test code here

    test_runner_cleanup(runner);
  end process;

  test_runner_watchdog(runner, 10 * clk_period);
end;

Note

When using several configurations to select what test runner to use, each test runner can only contain a single test, i.e. no test cases specified by the use of the run function are allowed.

Below are the two configurations that select this particular test along with one of the rtl and behavioral architectures for the DUT:

configuration test_reset_behavioral of tb_selecting_test_runner_with_vhdl_configuration is
  for tb
    for test_runner_inst : test_runner
      use entity work.test_runner(test_reset_architecture);
    end for;

    for test_fixture
      for dut : dff
        use entity work.dff(behavioral);
      end for;
    end for;
  end for;
end;

configuration test_reset_rtl of tb_selecting_test_runner_with_vhdl_configuration is
  for tb
    for test_runner_inst : test_runner
      use entity work.test_runner(test_reset_architecture);
    end for;

    for test_fixture
      for dut : dff
        use entity work.dff(rtl);
      end for;
    end for;
  end for;
end;

This example highlights a drawback of VHDL configurations: every combination of architectures to use in a test has to be manually created. When we use generics and if generate statements to select architectures, we create all combinations programatically in the Python script using the itertools.product function. Despite this, Python can continue to play a role in alleviating certain aspects of the combinatorial workload:

tb = lib.test_bench("tb_selecting_test_runner_with_vhdl_configuration")

for dut_arch, width, test_case_name in itertools.product(
    ["rtl", "behavioral"], [8, 16], ["test_reset", "test_state_change"]
):
    vhdl_configuration_name = f"{test_case_name}_{dut_arch}"
    tb.add_config(
        name=f"{vhdl_configuration_name}_{width}",
        generics=dict(width=width),
        vhdl_configuration_name=vhdl_configuration_name,
    )

> python run.py --list
lib.tb_selecting_test_runner_with_vhdl_configuration.test_reset_rtl_8
lib.tb_selecting_test_runner_with_vhdl_configuration.test_state_change_rtl_8
lib.tb_selecting_test_runner_with_vhdl_configuration.test_reset_rtl_16
lib.tb_selecting_test_runner_with_vhdl_configuration.test_state_change_rtl_16
lib.tb_selecting_test_runner_with_vhdl_configuration.test_reset_behavioral_8
lib.tb_selecting_test_runner_with_vhdl_configuration.test_state_change_behavioral_8
lib.tb_selecting_test_runner_with_vhdl_configuration.test_reset_behavioral_16
lib.tb_selecting_test_runner_with_vhdl_configuration.test_state_change_behavioral_16
Listed 8 tests

That concludes our discussion for now. As always, we highly value your feedback and appreciate any insights you might have to offer.