SystemVerilog For Verification Training

Verification Planning and Management

Session 9

Sameh El-Ashry

Hardware Verification Engineer

14 Oct 2015

Verification Planning and Management

14 Oct 2015

PHASES OF VERIFICATION

Analyze Coverage

Integrating Code Coverage

Writing Tests

Building Testbench

Verification Plan

14 Oct 2015

Detailed Phases Of Verification

14 Oct 2015

Verification Plan Concept

In test plan, we prepare a road map for how do achieve the goal,

it is a living document. Test plan includes, introduction,

assumptions, list of test cases, list of features to be tested,

approach, deliverables, resources, risks and scheduling, entry and

exit criteria. Test plan helps verification engineer to understand

how the verification should be done. A test plan could come in

many forms, such as a spreadsheet, a document or a simple text

file. Sometimes, test plan simply reside in the engineer's head

which is dangerous in which the process cannot be properly

measured and controlled. Test plan also contains the descriptions

of TestBench architecture and description of each component and its functionality.

14 Oct 2015

Building Testbench

In this phase, the verification environment is

developed. Each verification component can

be developed one by one or if more than one

engineer is working it can be developed

parallel. Writing the coverage module can be

done at any time. It is preferred to write down

the coverage module first as it gives some idea of the verification progress.

14 Oct 2015

Writing Tests

After the TestBench is built and integrated to DUT, it's

time for validating the DUT. Initially in Constrained

Driven Verification, the test are ran randomly till some

70 % of coverage is reached or no improvement in the

coverage for 1 day simulation. By analyzing the

coverage reports, new tests are written to cover the

holes. In these tests, randomization is directed to

cover the holes. Then finally, the hard to reach

scenarios, called as corner cases have to be written in

directed verification fashion. Of course, debugging is done in parallel and DUT fixes are done.

14 Oct 2015

Integrating Code Coverage && Analyze Coverage Once you have achieved certain level of functional

coverage, integrate the code coverage. For doing code

coverage, the code coverage tools have option to

switch it on. And then do the simulation, the tool will

provide the report.

Finally analyze both functional coverage and code

coverage reports and take necessary steps to achieve

coverage goals. Run simulation again with a different

seed, all the while collecting functional coverage information.

14 Oct 2015

Verification Plan

Here the start

14 Oct 2015

Case Study: ONES COUNTER EXAMPLE

Following example is TestBench for ones counter.

It has some verification components which are required.

The intention of showing this example is to make you familiar with some steps required while building verification environment and to help you to understand the flow of System Verilog environment.

14 Oct 2015

Specification of ones counter

Ones Counter is a Counter which counts the number of one's coming in serial stream.

The Minimum value of the count is "0" and count starts by incriminating one till "15". After "15" the counter rolls back to "0".

Reset is also provided to reset the counter value to "0". Reset signal is active negedge.

Input is 1 bit port for which the serial stream enters. Out bit is 4 bit port from where the count values can be taken. Reset and clock pins also provided.

14 Oct 2015

4 bit counter using four D flip-flops

Count[0]

Count[1]

din

14 Oct 2015

The following is the RTL code of ones counter with bugs

module dff(clk,reset,din,dout); input clk,reset,din; output dout; logic dout; always@(posedge clk,negedge reset) if(!reset) dout <= 0; else dout <= din; endmodule module ones_counter(clk,reset,data,count); input clk,reset,data; output [0:3] count; dff d1(clk,reset,data,count[0]); dff d2(count[0],reset,~count[1],count[1]); dff d3(count[1],reset,~count[2],count[2]); dff d4(count[2],reset,~count[3],count[3]); endmodule

14 Oct 2015

Verification Test Plan Template

Testcase

Name

Description Section # in

standard

Type

(positive/ne

gative)

Status

(pass/fail/

hang)

14 Oct 2015

Test Plan For Ones Counter

Testcase

Name

Description Section # in

standard

Type

(positive/nega

tive)

Status

(pass/fail/

hang)

test1.v Count should

increment from "0"

to "15".

( Coverage item)

test2.v Count should

roolover to "0" after

"15".

(Coverage

transition)

test3.v Reset should

make the output

count to "0", when

the count values is

non "0".

( Assertion

coverage)

14 Oct 2015

Verification Environment Hierarchy

TOP |-- Clock generator |-- Dut Instance |-- Interface |-- Assertion block instance ( assertion coverage) |-- Testcase instance |-- Environment |-- Driver | |-- Stimulus | |-- Covergroup |-- Monitor |-- Scoreboard

14 Oct 2015

SystemVerilog Environment For Ones Counter

DUT

SystemVerilog Interface

Scoreboard

stimulus Driver Monitor

Assertion monitor

Env

Test case

Top

reset Data Count

Count

14 Oct 2015

COVERAGE DRIVEN CONSTRAINT RANDOM VERIFICATION ARCHITECTURE (CDRV)

Basic functionality of CDRV Environment:

Input side of DUT :

-- Generating traffic streams

-- Driving traffic into the design (stimuli)

Output side of DUT:

-- Checking these data streams

-- Checking protocols and timing

Collecting both the functional coverage and code coverage information.

Writing deterministic tests and random tests to achieve 100% coverage.

14 Oct 2015

Verification Components Required For CDRV

14 Oct 2015

Stimulus

When building a verification environment, the verification

engineer often starts by modeling the device input stimulus.

In Verilog, the verification engineer is limited in how to model

this stimulus because of the lack of high-level data structures.

Typically, the verification engineer will create a array/memory

to store the stimuli. SystemVerilog provides high-level data

structures and the notion of dynamic data types for modeling

stimulus.

Using SystemVerilog randomization, stimulus is generated

automatically. Stimulus is also processed in other verification

components. SystemVerilog high-level data structures helps in storing and processing of stimulus in an efficient way.

14 Oct 2015

Stimulus Generator

The generator component generates stimulus which are sent to

DUT by driver. Stimulus generation is modeled to generate the

stimulus based on the specification.

For simple memory stimulus generator generates read, write

operations, address and data to be stored in the address if its

write operation.

Scenarios like generate alternate read/write operations are

specified in scenario generator.

SystemVerilog provided construct to control the random

generation distribution and order.

Generally generator should be able to generate every possible

scenario and the user should be able to control the generation

from directed and directed random testcases

14 Oct 2015

Transactor

Transactor does the high level operations like burst-operations

into individual commands, sub-layer protocol in layered protocol

like PciExpress Transaction layer over PciExpress Data Link Layer,

TCP/IP over Ethernet etc.

It also handles the DUT configuration operations. This layer also

provides necessary information to coverage model about the

stimulus generated.

Stimulus generated in generator is high level like Packet is with

good crc, length is 5 and da is 8~Rh0. This high level stimulus is

converted into low level data using packing. This low level data is

just a array of bits or bytes. Packing is an operation in which the

high level stimulus values scalars, strings, array elements and struct are concatenated in the specified manner.

14 Oct 2015

Driver

The drivers translate the operations produced by the

generator into the actual inputs for the design under

verification. Generators create inputs at a high level

of abstraction namely, as transactions like read write

operation. The drivers convert this input into actual

design inputs, as defined in the specification of the

designs interface. If the generator generates read

operation, then read task is called, in that, the DUT input pin "read_write" is asserted.

14 Oct 2015

Monitor

Monitor reports the protocol violation and identifies all

the transactions. Monitors are two types, Passive and

active. Passive monitors do not drive any signals.

Active monitors can drive the DUT signals. Sometimes

this is also refered as receiver. Monitor converts the

state of the design and its outputs to a transaction

abstraction level so it can be stored in a 'score-boards'

database to be checked later on. Monitor converts the pin level activities in to high level.

14 Oct 2015

Assertion Based Monitor

Assertions are used to check time based

protocols, also known as temporal checks.

Assertions are a necessary compliment to

transaction based testing as they describe the

pin level, cycle by cycle, protocols of the

design. Assertions are also used for functional coverage.

14 Oct 2015

Data Checker

The monitor only monitors the interface protocol. It doesn't check

the whether the data is same as expected data or not, as

interface has nothing to do with the date. Checker converts the

low level data to high level data and validated the data. This

operation of converting low level data to high level data is called

Unpacking which is reverse of packing operation. For example, if

data is collected from all the commands of the burst operation

and then the data is converted in to raw data , and all the sub

fields information are extracted from the data and compared

against the expected values. The comparison state is sent to scoreboard.

14 Oct 2015

Scoreboard

Scoreboard is sometimes referred as tracker. Scoreboard stores the

expected DUT output.

The stimulus generator generated the random vectors and is sent to the

DUT using drivers. These stimuli are stored in scoreboard until the output

comes out of the DUT.

When a write operation is done on a memory with address 101 and data

202, after some cycles, if a read is done at address 101, what should be

the data?.The score board recorded the address and data when write

operation is done. Get the data stored at address of 101 in scoreboard

and compare with the output of the DUT in checker module.

Scoreboard also has expected logic if needed. Take a 2 inputs and gate.

The expect logic does the "and " operation on the two inputs and stores the output".

14 Oct 2015

Coverage

This component has all the coverage related to the functional coverage groups.

Utilities

Utilities are set of global tasks which are not related

to any protocol. So this module can be reused across

projects without any modification to code. Tasks such

as global timeout, printing messages control, seeding

control, test pass/fail conditions, error counters etc.

The tasks defined in utilities are used by all other components of the TestBench.

14 Oct 2015

Environment

Environment contains the instances of all the

verification component and Component connectivity is

also done. Steps required for execution of each

component is done in this.

Tests

Tests contain the code to control the TestBench

features. Tests can communicate with all the

TestBench components. Once the TestBench is in

place, the verification engineer now needs to focus on

writing tests to verify that the device behaves according to specification.

14 Oct 2015

Regression Concept

Regression is re-running previously run tests and checking whether previously fixed faults have re-emerged.

New bugs may come out due to new changes in RTL or DUT to unmasking of previously hidden bugs due to new changes.

Each time, when design is changed, regression is done. One more important aspect of regression is testing by generation new vectors. Usually the seed to generate stimulus is the system time.

Whenever a regression is done, it will take the current system time and generate new vectors than earlier tested.

This way testbench can reach corners of DUT.

14 Oct 2015

How much regression testing?

14 Oct 2015

Coverage Model

Code coverage + Functional coverage

Example of Functional Coverage Result

14 Oct 2015

Example of Code Coverage Result

14 Oct 2015

Code Coverage

To check whether the Testbench has satisfactory exercised the design or not? Coverage is used. It will measure the efficiency of your verification implementation. Code coverage answers the questions like Have all the lines of the DUT has been exercised? Have all the states in the FSM has been entered? Have all the paths within a block have been exercised? Have all the branches in Case have been entered? Have all the conditions in an if statement is simulated?

14 Oct 2015

Functional Coverage

Functional coverage answers questions like Have all the packets length between 64 to 1518 are used? Did the DUT got exercised with alternate packets with good and bad crc? Did the monitor observe that the result comes with 4 clock cycles after read operation? Did the fifos are filled completely?

Summary of functional coverage advantages: Functional coverage helps determine how much of your specification was covered. Functional coverage qualifies the testbenchs. Considered as stopping criteria for unit level verification. Gives feedback about the untested features. Gives the information about the redundant tests which consume valuable cycle. Guides to reach the goals earlier based on grading.

14 Oct 2015

Verification IP

How there are companies working in verification services only ?!

Benefits ?

14 Oct 2015

VIP Example

14 Oct 2015

Tracking the Simulation Process

14 Oct 2015

Testsuite optimization in verification

Research task.

Thank You !

Presented by Sameh El-Ashry

samehelashry@ieee.org

https://eg.linkedin.com/pub/sameh-el-ashry/3b/560/22b