top-level model coverage report
if you simulate your model using the run button, simulink®
coverage™ creates a model coverage report for the specified model named
. the model coverage report
is also opened automatically in the coverage details pane. the model
coverage report contains several sections:model_name
_cov.html
to access the sldemo_fuelsys
model, execute the following command in the
matlab® command
window:
openexample('modelingafaulttolerantfuelcontrolsystemexample');
analysis information
the analysis information section contains basic information about the model being analyzed:
coverage data information
model information
harness information (appears if you record coverage from a simulink test™ harness)
simulation optimization options
coverage options
aggregated tests
the aggregated tests section appears if you:
record aggregated coverage results for at least two test cases through the simulink test manager and produce a coverage report for the aggregated results, or
produce a coverage report for cumulative coverage results in the results explorer.
if you run test cases through the simulink test manager, the aggregated tests section links to the associated test cases in the simulink test manager.
if you aggregate test case results through the results explorer, the aggregated tests section links to the corresponding cvdata node in the results explorer.
for each run in the aggregated tests section, there is a link to the corresponding results in the simulink test manager or the results explorer.
aggregated unit tests
if you record coverage for one or more subsystem harnesses, the aggregated tests section lists each unit test run, and the description section displays the description given to the aggregated coverage data. you can see and edit this description by going to the coverage results explorer and clicking current cumulative data.
each unit under test receives an ordinal number n, and each test for
a unit under test receives an ordinal number m in the style
u
n.m.
coverage summary
the coverage summary has two subsections:
tests — the simulation start and stop time of each test case and any setup commands that preceded the simulation. the heading for each test case includes any test case label specified using the
cvtest
command. this section only shows when the report does not contain an aggregated tests section.summary — summaries of the subsystem results. to see detailed results for a specific subsystem, in the summary subsection, click the subsystem name.
the summary section contains a column for each requested coverage metric,
even for metrics that are not applicable to the model or model objects analyzed. for example,
in the sldemo_fuelsys
model, if you select the objectives and
constraints coverage metric, you get columns titled test
objective, proof objective, test
condition, and proof assumption, even though the model does
not contain blocks that simulink
coverage can analyze for these metrics.
details
the details section reports the detailed model coverage results. each section of the detailed report summarizes the results for the metrics that test each object in the model:
you can also access a model element details subsection as follows:
right-click a simulink element.
in the context menu, select coverage > report.
filtered objects
the filtered objects section lists all the objects in the model that were filtered from coverage recording, and the rationale you specified for filtering those objects. if the filter rule specifies that all blocks of a certain type be filtered, all those blocks are listed here.
in the following graphic, several blocks, subsystems, and transitions were filtered. two library-linked blocks, protected division and protected division1, were filtered because their block library was filtered.
model details
the details section contains a results summary for the model as a whole, followed by a list of elements. click the model element name to see its coverage results.
the following graphic shows the
details section for the sldemo_fuelsys
example
model.
subsystem details
each subsystem details section contains a summary of the test coverage results for the subsystem and a list of the subsystems it contains. the overview is followed by sections for blocks, charts, and matlab functions, one for each object that contains a decision point in the subsystem.
the following graphic shows the coverage results for the engine gas
dynamics subsystem in the sldemo_fuelsys
example model.
block details
the following graphic shows decision coverage results for the minmax block in
the mixing & combustion subsystem of the engine gas dynamics subsystem in the
sldemo_fuelsys
example model.
the uncovered links element first appears in the block details section of the first block in the model hierarchy that does not achieve 100% coverage. the first uncovered links element has an arrow that links to the block details section in the report of the next block that does not achieve 100% coverage.
subsequent blocks that do not achieve 100% coverage have links to the block details sections in the report of the previous and next blocks that do not achieve 100% coverage.
chart details
the following graphic shows the
coverage results for the stateflow® chart control_logic
in the
sldemo_fuelsys
example model.
for more information about model coverage reports for stateflow charts and their objects, see .
coverage details for matlab functions and simulink design verifier functions
by default, simulink coverage records coverage for all matlab functions in a model. matlab functions are in matlab function blocks, stateflow charts, or external matlab files.
note
for a detailed example of coverage reports for external matlab files, see .
to record simulink
design verifier™ coverage for sldv.*
functions called by matlab functions, and any
simulink
design verifier blocks, select objectives and constraints on the
coverage pane of the configuration parameters dialog
box.
the following example shows coverage details for a matlab function, hfcnsinexternaleml
, that calls four simulink
design verifier functions. in this example, the code for hfcnsinexternaleml
resides in an external file.
this example also shows simulink design verifier coverage details for the following functions:
(simulink design verifier)
(simulink design verifier)
(simulink design verifier)
(simulink design verifier)
in the coverage results, code that achieves 100% coverage is green. code that achieves less than 100% coverage is red.
coverage for the hfcnsinexternaleml
function and the
sldv.*
calls is:
line 1, the function declaration for
hfcnsinexternaleml
is green because the simulation executes that function at least once.fcn
callshfcnsinexternaleml
11 times during simulation.line 4,
sldv.assume(u1 > u2)
, achieves 0% coverage becauseu1 > u2
never evaluates to true.line 5,
sldv.condition(u1 == 0)
, achieves 100% coverage becauseu1 == 0
evaluates to true for at least one time step.line 6,
switch u1
, achieves 25% coverage because only one of the four outcomes in theswitch
statement (case 0
) occurs during simulation.line 17,
sldv.test(y > u1); sldv.test (y == 4)
achieves 50% coverage. the firstsldv.test
call achieves 100% coverage, but the secondsldv.test
call achieves 0% coverage.
for more information about coverage for matlab functions, see .
for more information about coverage for simulink design verifier functions, see objectives and constraints coverage.
requirement testing details
if you run at least two test cases in simulink test that are linked to requirements in requirements toolbox™, the aggregated coverage report details the links between model elements, test cases, and linked requirements.
the requirement testing details section includes:
implemented requirements — which requirements are linked to the model element.
verified by tests — which tests verify the requirement.
associated runs — which runs are associated with each verification test.
for an example of how to trace coverage results to requirements in a coverage report, see .
cyclomatic complexity in the model coverage report
you can specify that the model coverage report include cyclomatic complexity numbers in two locations in the report:
the summary section contains the cyclomatic complexity numbers for each object in the model hierarchy. for a subsystem or stateflow chart, that number includes the cyclomatic complexity numbers for all their descendants.
the details sections for each object list the cyclomatic complexity numbers for all individual objects.
decisions analyzed
the decisions analyzed table lists possible outcomes for a decision and the number of times that an outcome occurred in each test simulation. outcomes that did not occur are in red highlighted table rows.
the following graphic shows the decisions analyzed table for the saturate
block in the throttle & manifold subsystem of the engine gas dynamics subsystem in the
sldemo_fuelsys
example model.
to display and highlight the block in question, click the block name at the top of the section containing the block’s decisions analyzed table.
conditions analyzed
the conditions analyzed table lists the number of occurrences of true and false conditions on each input port of the corresponding block.
mcdc analysis
the mcdc analysis table lists the mcdc input condition cases represented by the corresponding block and the extent to which the reported test cases cover the condition cases.
each row of the mcdc analysis table represents a condition case for a
particular input to the block. a condition case for input n
of a block is a
combination of input values. input n
is called the deciding
input of the condition case. changing the value of input n
alone changes the value of the block's output.
the mcdc analysis table shows a condition case expression to represent a condition case. a condition case expression is a character string where:
the position of a character in the string corresponds to the input port number.
the character at the position represents the value of the input. (
t
meanstrue
;f
meansfalse
).a boldface character corresponds to the value of the deciding input.
for example, f
t
f
represents a condition case for a three-input block where
the second input is the deciding input.
the
decision/condition column specifies the deciding input for an input
condition case. the true out column specifies the deciding input value
that causes the block to output a true
value for a condition case. the
true out entry uses a condition case expression, for example, f
f
, to express the values of
all the inputs to the block, with the value of the
deciding variable in bold.
parentheses around the expression indicate that the specified combination of inputs did not occur during the first (or only) test case included in this report. in other words, the test case did not cover the corresponding condition case. the false out column specifies the deciding input value that causes the block to output a false value and whether the value actually occurred during the first (or only) test case included in the report.
some model elements achieve less mcdc coverage depending on the mcdc definition used during analysis. for more information on how the mcdc definition used during analysis affects the coverage results, see .
if you select treat simulink logic blocks as
short-circuited in the coverage pane in the configuration
parameters dialog box, mcdc coverage analysis does not verify whether short-circuited inputs
actually occur. the mcdc analysis table uses an x
in a condition expression
(for example, tfxxx
) to indicate short-circuited inputs that were not
analyzed by the tool.
if you disable this feature and logic blocks are not short-circuited while collecting model coverage, you might not be able to achieve 100% coverage for that block.
select the treat simulink logic blocks as short-circuited option for where you want the mcdc coverage analysis to approximate the degree of coverage that your test cases achieve for the generated code (most high-level languages short-circuit logic expressions).
cumulative coverage
after you record successive coverage results, you can from within the coverage results explorer. by default, the results of each simulation are saved and recorded cumulatively in the report.
if you select show cumulative progress report in the section of the configuration parameters, the results located in the right-most area in all tables of the cumulative coverage report reflect the running total value. the report is organized so that you can easily compare the additional coverage from the most recent run with the coverage from all prior runs in the session.
a cumulative coverage report contains information about:
current run — the coverage results of the simulation just completed.
delta — percentage of coverage added to the cumulative coverage achieved with the simulation just completed. if the previous simulation's cumulative coverage and the current coverage are nonzero, the delta may be 0 if the new coverage does not add to the cumulative coverage.
cumulative — the total coverage collected for the model up to, and including, the simulation just completed.
after running three test cases, the summary report shows how much additional coverage the third test case achieved and the cumulative coverage achieved for the first two test cases.
the decisions analyzed table for cumulative coverage contains three columns of data about decision outcomes that represent the current run, the delta since the last run, and the cumulative data, respectively.
the conditions analyzed table uses column headers #n t and #n f to indicate results for individual test cases. the table uses tot t and tot f for the cumulative results. you can identify the true and false conditions on each input port of the corresponding block for each test case.
the mcdc analysis #n true out and #n false out columns show the condition cases for each test case. the total out t and total out f column show the cumulative results.
note
you can calculate cumulative coverage for reusable subsystems and stateflow constructs at the command line. for more information, see .
n-dimensional lookup table
the following interactive chart summarizes the extent to which elements of a lookup table are accessed. in this example, two sine wave blocks generate x and y indices that access a 2-d lookup table block of 10-by-10 elements filled with random values.
in this model, the lookup table indices are 1, 2,..., 10 in each direction. the sine wave 2 block is out of phase with the sine wave 1 block by pi/2 radians. this generates x and y numbers for the edge of a circle, which you see when you examine the resulting lookup table coverage.
the report contains a two-dimensional table representing the elements of the lookup table. the element indices are represented by the cell border grid lines, which number 10 in each dimension. areas where the lookup table interpolates between table values are represented by the cell areas. areas of extrapolation left of element 1 and right of element 10 are represented by cells at the edge of the table, which have no outside border.
note
the coverage report only generates the look-up table details image for lookup tables that have 400 or fewer interpolation or extrapolation intervals.
the number of values interpolated or extrapolated for each cell (execution counts) during testing is represented by a shade of green assigned to the cell. each of six levels of green shading and the range of execution counts represented are displayed on one side of the table.
if you click an individual table cell, you see a dialog box that displays the index location of the cell and the exact number of execution counts generated for it during testing. the following example shows the contents of a color-shaded cell on the right edge of the circle.
the selected cell is outlined in red. you can also click the extrapolation cells on the edge of the table.
a bold grid line indicates that at least one block input equal to its exact index value occurred during the simulation. click the border to display the exact number of hits for that index value.
the following example model uses an n-d lookup table block of 10-by-10-by-5 elements filled with random values.
both the x and y table axes have the indices 1, 2,..., 10. the z axis has the indices 10, 20,..., 50. lookup table values are accessed with x and y indices that the two sine wave blocks generated, in the preceding example, and a z index that a ramp block generates.
after simulation, you see the following lookup table report.
instead of a two-dimensional table, the link force map generation
displays the following tables:
lookup table coverage for a three-dimensional lookup table block is reported as a set of two-dimensional tables.
the vertical bars represent the exact z index values: 10, 20, 30, 40, 50. if a vertical bar is bold, this indicates that at least one block input was equal to the exact index value it represents during the simulation. click a bar to get a coverage report for the exact index value that bar represents.
you can report lookup table coverage for lookup tables of any dimension. coverage for four-dimensional tables is reported as sets of three-dimensional sets, like those in the preceding example. five-dimensional tables are reported as sets of sets of three-dimensional sets, and so on.
block reduction
all model coverage reports indicate the status of the simulink block reduction parameter at the beginning of the report. in the following example, you set force block reduction off.
in the next example, you enabled the simulink block reduction parameter, and you did not set force block reduction off.
consider the following model where the simulation does not execute the minmax1 block
because there is only one input — in3
.
if you set force block reduction off, the report contains
no coverage data for this block because the minimum input
to the minmax1 block is always
1
.
if you do not set force block reduction off, the report contains no coverage data for reduced blocks.
relational boundary
on the of the configuration parameters dialog box, if you select the relational boundary coverage metric, the software creates a relational boundary table in the model coverage report for each model object that is supported for this coverage. the table applies to the explicit or implicit relational operation involved in the model object. for more information, see:
the relational boundary column in .
the tables below show the relational boundary coverage report for the relation
input1 <= input2
. the appearance of the tables depend on the operand
data type.
integers
if both operands are integers (or if one operand is an integer and the other a boolean), the table appears as follows.
for a relational operation such as
:operand_1
<=
operand_2
the first row states the two operands in the form
.operand_1
-operand_2
the second row states the number of times during the simulation that
is equal to -1.operand_1
-operand_2
the third row states the number of times during the simulation that
is equal tooperand_1
.operand_2
the fourth row states the number of times during the simulation that
is equal to 1.operand_1
-operand_2
fixed point
if one of the operands has fixed-point
type and the other operand is either a fixed point or an integer, the table appears as
follows. lsb
represents the value of the least significant bit. for more
information, see (fixed-point designer). if the two operands have different
precision, the smaller value of precision is used.
for a relational operation such as
:operand_1
<=
operand_2
the first row states the two operands in the form
.operand_1
-operand_2
the second row states the number of times during the simulation that
is equal tooperand_1
-operand_2
-lsb
.the third row states the number of times during the simulation that
is equal tooperand_1
.operand_2
the fourth row states the number of times during the simulation that
is equal tooperand_1
-operand_2
lsb
.
floating point
if one of the operands has
floating-point type, the table appears as follows. tol
represents a value
computed using the input values and a tolerance that you specify. if you do not specify a
tolerance, the default values are used. for more information, see relational boundary coverage.
for a relational operation such as
:operand_1
<=
operand_2
the first row states the two operands in the form
.operand_1
-operand_2
the second row states the number of times during the simulation that
has values in the rangeoperand_1
-operand_2
[-tol..0]
.the third row states the number of times during the simulation that
has values in the rangeoperand_1
-operand_2
(0..tol]
during the simulation.
the appearance of this table changes according to the relational
operator in the block. depending on the relational operator, the value of
equal to 0 is either: operand_1
-
operand_2
excluded from relational boundary coverage.
included in the region above the relational boundary.
included in the region below the relational boundary.
relational operator | report format | explanation |
---|---|---|
== | [-tol..0) | 0 is excluded. |
(0..tol] | ||
!= | [-tol..0) | 0 is excluded. |
(0..tol] | ||
<= | [-tol..0] | 0 is included in the region below the relational boundary. |
(0..tol] | ||
< | [-tol..0) | 0 is included in the region above the relational boundary. |
[0..tol] | ||
>= | [-tol..0) | 0 is included in the region above the relational boundary. |
[0..tol] | ||
> | [-tol..0] | 0 is included in the region below the relational boundary. |
(0..tol] |
0 is included below the relational boundary for <=
but above the relational boundary for <
. this rule is consistent with
decision coverage. for instance:
for the relation
input1 <= input2
, the decision is true ifinput1
is less than or equal toinput2
.<
and=
are grouped together. therefore, 0 lies in the region below the relational boundary.for the relation
input1 < input2
, the decision is true only ifinput1
is less thaninput2
.>
and=
are grouped together. therefore, 0 lies in the region above the relational boundary.
saturate on integer overflow analysis
on the of the configuration parameters dialog box, if you select the saturate on integer overflow coverage metric, the software creates a saturation on overflow analyzed table in the model coverage report. the software creates the table for each block with the saturate on integer overflow parameter selected.
the saturation on overflow analyzed table lists the number of times a block saturates on integer overflow, indicating a true decision. if the block does not saturate on integer overflow, the table indicates a false decision. outcomes that do not occur are in red highlighted table rows.
the following graphic shows the saturation on
overflow analyzed table for the minmax block in the mixing & combustion
subsystem of the engine gas dynamics subsystem in the sldemo_fuelsys
example model.
to display and highlight the block in question, click the block name at the top of the section containing the block’s saturation on overflow analyzed table.
signal range analysis
if you select the signal range coverage metric, the software creates a signal range analysis section at the bottom of the model coverage report. this section lists the maximum and minimum signal values for each output signal in the model measured during simulation.
access the signal range analysis report quickly with the
signal ranges link in the nonscrolling region at the top of the model
coverage report, as shown below in the sldemo_fuelsys
example model
report.
each block is reported in hierarchical fashion; child blocks appear
directly under parent blocks. each block name in the signal ranges report
is a link. for example, select the ego sensor
link to display this block
highlighted in its native diagram.
signal size coverage for variable-dimension signals
if you select signal size, the software creates a variable signal widths section after the signal ranges data in the model coverage report. this section lists the maximum and minimum signal sizes for all output ports in the model that have variable-size signals. it also lists the memory that simulink allocated for that signal, as measured during simulation. this list does not include signals whose size does not vary during simulation.
the following example shows the variable signal widths section in a coverage report. in this example, the abs block signal size varied from 2 to 5, with an allocation of 5.
each block is reported in hierarchical fashion; child blocks appear directly under parent blocks. each block name in the variable signal widths list is a link. clicking on the link highlights the corresponding block in the simulink editor. after the analysis, the variable-size signals have a wider line design.
simulink design verifier coverage
if you select objectives and constraints, the analysis collects coverage data for all simulink design verifier blocks in your model.
for an example of how this works, open the
sldvdemo_debounce_testobjblks
model.
this model contains two test objective blocks:
the true block defines a property that the signal have a value of
2
.the edge block, inside the masked objective subsystem, describes the property where the output of the and block in the masked objective subsystem changes from
2
to1
.
the simulink design verifier software analyzes this model and produces a harness model that contains test cases that achieve certain test objectives. to see if the original model achieves those objectives, simulate the harness model and collect model coverage data. the model coverage tool analyzes any decision points or values within an interval that you specify in the test objective block.
in this example, the
coverage report shows that you achieved 100% coverage of the true block because the signal
value was 2
at least once. the signal value was 2
in 6
out of 14 time steps.
the input signal to the edge block achieved a value of
true
once out of 14 time steps.