1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
|
1
00:00:00,310 --> 00:00:02,250
So then maybe what we can do is just to
2
00:00:02,250 --> 00:00:05,720
pick our test inputs randomly so to do what is called
3
00:00:05,720 --> 00:00:08,850
random testing. And what that means is that we pick the
4
00:00:08,850 --> 00:00:11,720
inputs to test just as we pick a number by rolling
5
00:00:11,720 --> 00:00:15,410
a set of dice randomly. And this will have several advantages.
6
00:00:15,410 --> 00:00:18,780
First, we will pick inputs uniformly. So if we use a
7
00:00:18,780 --> 00:00:21,790
uniform distribution as the basis for our random testing, we will
8
00:00:21,790 --> 00:00:25,540
make no preferences. In other words, all inputs will be considered
9
00:00:25,540 --> 00:00:28,324
equal, of equal value. And what that means in turn, is
10
00:00:28,324 --> 00:00:32,640
that random testing eliminates designer bias. So what does designer bias
11
00:00:32,640 --> 00:00:36,030
mean? Designer bias is the problem of making the same assumption,
12
00:00:36,030 --> 00:00:38,570
when we read the specification and we interpret it and when we
13
00:00:38,570 --> 00:00:42,100
develop test cases. Which means that the developer might develop code,
14
00:00:42,100 --> 00:00:44,930
assuming a given behavior of the user. And we may write
15
00:00:44,930 --> 00:00:47,520
tests, making the same assumptions. And the problem, of course, is
16
00:00:47,520 --> 00:00:50,690
even worse if it's the same person that develops the code and
17
00:00:50,690 --> 00:00:53,730
writes the test cases. With random testing, the problem is gone,
18
00:00:53,730 --> 00:00:57,440
because we just pick randomly what our inputs will be. So
19
00:00:57,440 --> 00:01:00,180
why not do in random? The problem is that when testing,
20
00:01:00,180 --> 00:01:03,610
we are looking for a needle in a haystack. Actually, multiple
21
00:01:03,610 --> 00:01:06,620
needles in multiple haystacks, if we want to be precise. So,
22
00:01:06,620 --> 00:01:09,500
random approaches are not necessarily the best way to go about
23
00:01:09,500 --> 00:01:12,430
it, because we might just be looking in all the wrong
24
00:01:12,430 --> 00:01:15,760
places. So let me show you this, using a different representation
25
00:01:15,760 --> 00:01:18,000
for the haystack. What I'm showing here is a grid, and
26
00:01:18,000 --> 00:01:22,130
imagine this grid just expanding indefinitely outside the screen, and this grid
27
00:01:22,130 --> 00:01:26,120
represents the domain for the program, so each box in the grid,
28
00:01:26,120 --> 00:01:29,050
each square in the grid, it's a possible input. So what happens
29
00:01:29,050 --> 00:01:32,670
with bugs is that bugs are very scarce in this grid. Maybe
30
00:01:32,670 --> 00:01:35,070
there is a bug here, so that means that there is a
31
00:01:35,070 --> 00:01:38,090
bug, than an input, in this point we'll reveal. And maybe there
32
00:01:38,090 --> 00:01:40,820
is another bug that will be triggered by an input over here.
33
00:01:40,820 --> 00:01:44,570
So imagine this spread out over this infinite grid. Its very unlikely
34
00:01:44,570 --> 00:01:47,440
that just by picking randomly that we will be able to get to
35
00:01:47,440 --> 00:01:50,910
these two points. Fortunately not all is lost, there is a silver lining.
36
00:01:50,910 --> 00:01:53,410
So we need to look a little more in depth into this grid.
|
