1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
|
1
00:00:00,270 --> 00:00:03,080
Now that we saw this overview of white-box testing, I'd like
2
00:00:03,080 --> 00:00:06,270
to start talking about specific coverage criterion. And I'm going to start
3
00:00:06,270 --> 00:00:09,010
with the first one, which is Statement Coverage. This criterion is
4
00:00:09,010 --> 00:00:12,355
going to be characterized by two aspects, the first one is which
5
00:00:12,355 --> 00:00:15,580
are the Test requirements for the criteria and the second one
6
00:00:15,580 --> 00:00:18,830
is how we measure Coverage for that criteria. In the case
7
00:00:18,830 --> 00:00:22,200
of statement coverage, these test requirements are all the statements in
8
00:00:22,200 --> 00:00:25,804
the program. So this is the basic, the first the, the simplest
9
00:00:25,804 --> 00:00:29,860
coverage criteria in the white-box arena. Let me remind you the assumption
10
00:00:29,860 --> 00:00:33,280
that we made at the beginning. White-box testing is based on the assumption
11
00:00:33,280 --> 00:00:35,990
that if there isn't a faulty element in the code, we need
12
00:00:35,990 --> 00:00:38,390
to exercise it. We need to execute it, in order to find the
13
00:00:38,390 --> 00:00:41,260
fault. And that's exactly what statement coverage does. If there is a
14
00:00:41,260 --> 00:00:44,450
statement that is faulty in the code, we need to exercise it, in
15
00:00:44,450 --> 00:00:47,490
order to find the fault. And therefore, a good measure of how well
16
00:00:47,490 --> 00:00:51,480
we exercise the code, is the ratio of the number of executed statements.
17
00:00:51,480 --> 00:00:54,950
So all the statements that my test cases executed, to the total
18
00:00:54,950 --> 00:00:58,570
number of statements in the program. The higher this number, the better
19
00:00:58,570 --> 00:01:01,870
I exercise my code. And we can also look at coverage criterion
20
00:01:01,870 --> 00:01:04,290
in terms of questions. So what is the questions they were trying
21
00:01:04,290 --> 00:01:06,940
to answer when we look at a specific set of test cases
22
00:01:06,940 --> 00:01:09,870
and we assess the statement coverage that they achieved. And the question
23
00:01:09,870 --> 00:01:13,440
is whether each statement in the program has been executed. So, statement
24
00:01:13,440 --> 00:01:16,920
coverage is satisfied when all the statements in the program have been executed.
25
00:01:16,920 --> 00:01:19,640
And we can satisfy to different degrees and the degrees to which it's
26
00:01:19,640 --> 00:01:23,320
satisfied is measured by this value. So now let's go ahead and measure
27
00:01:23,320 --> 00:01:27,250
statement coverage on our printSum example. What I'm going to show down here is
28
00:01:27,250 --> 00:01:30,500
this progress bar in which we show the amount of coverage, the percentage of
29
00:01:30,500 --> 00:01:33,060
coverage achieved. So what this means is that the, if I get to
30
00:01:33,060 --> 00:01:36,680
this point I've covered 25% of the statements in the code. And my goal
31
00:01:36,680 --> 00:01:39,160
is to get up here to cover all the statements in the code.
32
00:01:39,160 --> 00:01:42,000
We have two test cases for this code. The first one that we just
33
00:01:42,000 --> 00:01:45,700
saw, consists of the inputs a equal to 3 and b equal
34
00:01:45,700 --> 00:01:48,200
to 9, and the second one has the inputs a is equal to
35
00:01:48,200 --> 00:01:50,840
minus 5 and b is equal to minus 8. So now let's see
36
00:01:50,840 --> 00:01:53,450
what happens when we run this test case. When we run this test
37
00:01:53,450 --> 00:01:56,920
case, I'm going to show you by highlighting in the code the parts that
38
00:01:56,920 --> 00:02:00,430
we cover when we start executing the code. We cover the first statement,
39
00:02:00,430 --> 00:02:01,910
then we always execute the second
40
00:02:01,910 --> 00:02:04,180
statement, which computes the result, we continue
41
00:02:04,180 --> 00:02:07,070
the execution, we get to the if statement. If the result is greater
42
00:02:07,070 --> 00:02:10,038
than zero, in this case our result is 12 because we
43
00:02:10,038 --> 00:02:12,360
are working with the inputs 3 and 9, and therefore we
44
00:02:12,360 --> 00:02:15,710
execute the true part of the if, we execute the statement.
45
00:02:15,710 --> 00:02:18,850
And at this point, we just jump to the end. Because we
46
00:02:18,850 --> 00:02:21,530
do not execute the else part of the statement, since we
47
00:02:21,530 --> 00:02:24,240
have executed a true one, and therefore, we cover this final
48
00:02:24,240 --> 00:02:27,534
statement. So at the end of the execution of this test
49
00:02:27,534 --> 00:02:32,313
case, we cover one, two, three, four, five statement out of seven
50
00:02:32,313 --> 00:02:36,299
which is roughly speaking 71%. So we can mark in here
51
00:02:36,299 --> 00:02:39,779
that we more or less got to 71% of coverage for
52
00:02:39,779 --> 00:02:42,719
this code. Now let's look at what happens when we execute
53
00:02:42,719 --> 00:02:45,700
test case number two. In this case again, we execute the
54
00:02:45,700 --> 00:02:48,660
first statement, the second statement, the third statement. In this case
55
00:02:48,660 --> 00:02:52,010
though, the first statement, when it evaluates the value of result,
56
00:02:52,010 --> 00:02:54,250
it sees that the result is not greater than zero because
57
00:02:54,250 --> 00:02:57,590
our inputs are minus five and minus eight. Therefore, you will execute
58
00:02:57,590 --> 00:03:00,090
line number five. And because the result is less than
59
00:03:00,090 --> 00:03:02,810
zero, you will also execute line number six. So, at
60
00:03:02,810 --> 00:03:05,230
this point, all of the statements in our code are
61
00:03:05,230 --> 00:03:09,360
executed and therefore, we achieved a 100% statement coverage, which
62
00:03:09,360 --> 00:03:13,090
was our goal. Before looking at other kinds of coverage,
63
00:03:13,090 --> 00:03:16,060
let's see how our statement coverage is used in practice.
64
00:03:16,060 --> 00:03:19,170
First of all, statement coverage is the most used kind
65
00:03:19,170 --> 00:03:22,830
of coverage criterion in industry. Normally for company that uses
66
00:03:22,830 --> 00:03:27,200
statement coverage, the typical coverage target is 80-90%, which
67
00:03:27,200 --> 00:03:29,690
mean the outcome of the test should be such
68
00:03:29,690 --> 00:03:32,770
that 80-90% of the statements are exercised at the
69
00:03:32,770 --> 00:03:34,880
end of testing. So at this point, you might
70
00:03:34,880 --> 00:03:36,850
be wondering, why don't we just shoot for 100%?
71
00:03:36,850 --> 00:03:38,580
Why don't we try to cover all of the
72
00:03:38,580 --> 00:03:40,050
code? We just saw that we could do it.
73
00:03:40,050 --> 00:03:41,730
And so I'm going to ask you the same question.
|
