1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
|
1
00:00:00,160 --> 00:00:03,050
And before I start talking about specific testing techniques, there's
2
00:00:03,050 --> 00:00:05,490
something else that I want to discuss, which is Testing
3
00:00:05,490 --> 00:00:09,090
Granularity Levels. So let's consider a software system, a system
4
00:00:09,090 --> 00:00:12,310
made out of components that interact with one another. So the
5
00:00:12,310 --> 00:00:15,000
first level that we consider in testing is called Unit
6
00:00:15,000 --> 00:00:18,490
Testing, which is the testing of the individual units or modules
7
00:00:18,490 --> 00:00:20,690
in isolation. The next step, is to see there are
8
00:00:20,690 --> 00:00:25,340
multiple modules and their interactions. And this is called Integration Testing.
9
00:00:25,340 --> 00:00:28,650
So, integration testing is the testing of the interactions among
10
00:00:28,650 --> 00:00:31,460
different modules. And it can be performed according to different
11
00:00:31,460 --> 00:00:34,260
strategies. Depending on the order in which the modules are
12
00:00:34,260 --> 00:00:37,570
integrated and on whether we integrate one module at a time
13
00:00:37,570 --> 00:00:40,510
or multiple modules together, all at once. And in this
14
00:00:40,510 --> 00:00:43,240
latter case, we call this kind of integration testing, the one that
15
00:00:43,240 --> 00:00:47,640
integrates all the modules at once, Big Bang integration testing.
16
00:00:47,640 --> 00:00:50,520
And after performing integration testing, the next step is to test
17
00:00:50,520 --> 00:00:52,750
the complete system as a whole. And this level of
18
00:00:52,750 --> 00:00:56,190
testing is normally called, System Testing. So system testing in the
19
00:00:56,190 --> 00:00:59,560
testing of the complete system and it includes both functional and
20
00:00:59,560 --> 00:01:03,250
non functional testing. We will discuss functional and non functional testing
21
00:01:03,250 --> 00:01:05,575
in details in the next two lessons. But I just
22
00:01:05,575 --> 00:01:08,330
want to give you an idea of what they are intuitively. Functional
23
00:01:08,330 --> 00:01:12,250
tests are the test that aim to verify the functionality provided
24
00:01:12,250 --> 00:01:15,680
by the system. For example if you consider the function double
25
00:01:15,680 --> 00:01:17,840
value that we saw earlier in the lesson, a
26
00:01:17,840 --> 00:01:20,660
functional test will try to assess that that function
27
00:01:20,660 --> 00:01:23,970
is producing the right value given a specific input.
28
00:01:23,970 --> 00:01:26,826
Conversely, no functional test are the one that target, as
29
00:01:26,826 --> 00:01:30,540
surprisingly, no functional properties of the system. For example,
30
00:01:30,540 --> 00:01:34,060
no functional test will include performance tests, load tests,
31
00:01:34,060 --> 00:01:37,310
robustness tests. In general, no functional tests will try
32
00:01:37,310 --> 00:01:41,410
to assess different qualities of the system, such as reliability,
33
00:01:41,410 --> 00:01:45,970
maintainability, usability, so basically, all the ilities that you can
34
00:01:45,970 --> 00:01:49,760
think about. In addition to these three basic testing levels, there
35
00:01:49,760 --> 00:01:51,830
are two more levels that I want to consider and that
36
00:01:51,830 --> 00:01:55,150
I want to discuss. And they both involve the whole system.
37
00:01:55,150 --> 00:01:57,610
And the first one is Acceptance Testing which is the
38
00:01:57,610 --> 00:02:01,320
validation of the software against the Customer requirements. So this is
39
00:02:01,320 --> 00:02:04,090
the testing that makes sure that the system does what the
40
00:02:04,090 --> 00:02:06,720
customer wants it to do. And the last type of testing
41
00:02:06,720 --> 00:02:09,729
that I want to mention is Regression Testing. And regression testing
42
00:02:09,729 --> 00:02:13,260
is the type of testing or retesting, that we perform every time
43
00:02:13,260 --> 00:02:15,540
that we change our system. And we need to make sure
44
00:02:15,540 --> 00:02:19,060
that the changes behave as intended and that the unchanged code is
45
00:02:19,060 --> 00:02:22,680
not negatively affected by the modification, by these changes. In fact,
46
00:02:22,680 --> 00:02:25,070
what can happen when you modify the code is that parts of
47
00:02:25,070 --> 00:02:28,120
the code that are related to the changes, are actually affected
48
00:02:28,120 --> 00:02:32,110
by the changes, and start misbehaving. And we call those regression errors.
49
00:02:32,110 --> 00:02:35,080
And regression errors, are very common. For example, you're probably
50
00:02:35,080 --> 00:02:38,070
familiar with the situation in which, one software update is
51
00:02:38,070 --> 00:02:41,350
released, and just a few days later, another software update
52
00:02:41,350 --> 00:02:44,640
is released. In many cases that happens because the first update
53
00:02:44,640 --> 00:02:47,760
was containing regression errors. So the changes in the code
54
00:02:47,760 --> 00:02:51,100
that broke some functionality, that resulted in failures on the user's
55
00:02:51,100 --> 00:02:53,940
machine and in bug reports and therefore that caused further
56
00:02:53,940 --> 00:02:57,090
maintenance, further bug fixes, and a release on a new version.
57
00:02:57,090 --> 00:02:59,790
Something else I'd like to mention about regression testing, is that
58
00:02:59,790 --> 00:03:02,535
regression testing is one of the main causes why software maintenance is
59
00:03:02,535 --> 00:03:05,679
so expensive. And that's also why researchers have invested a great
60
00:03:05,679 --> 00:03:07,481
deal of effort into refining regression
61
00:03:07,481 --> 00:03:09,495
testing techniques that can make regression
62
00:03:09,495 --> 00:03:12,599
testing more effective and more efficient. So let me leave you,
63
00:03:12,599 --> 00:03:15,129
with a little piece of advice which is try to automate as
64
00:03:15,129 --> 00:03:19,470
much as possible regression testing. For example use scripts, use tools, make
65
00:03:19,470 --> 00:03:22,660
sure to save your harness, make sure to save your input, and
66
00:03:22,660 --> 00:03:24,820
outputs for the test, because you want to be able
67
00:03:24,820 --> 00:03:27,680
to rerun your test, at a push of a button as
68
00:03:27,680 --> 00:03:29,750
much as possible every time you change your code, to
69
00:03:29,750 --> 00:03:32,560
avoid the presence of regression errors in the code you release.
|
