-In this demo, we're going to do exactly what we - -2
-did just now in the lesson. We're going to use - -3
-the category partition method to go from a high-level - -4
-description of a piece of software of a program to - -5
-a set of test cases for that program. To - -6
-do that, we're going to use a simple tool. So - -7
-I'm going to show you here the tool that is - -8
-called a tsl generator right here. This tool is available - -9
-to you, so you can look in the class notes to - -10
-see information on how to download it. And together with the tool, - -11
-we are also going to provide a manual for the tool, and - -12
-a set of files that I'm going to use in this demo. So - -13
-you should be able to do exactly what I'm doing. So - -14
-again, all of those are available from the class notes. So specifically, - -15
-today we're going to write test cases for the grep program. So in - -16
-case you're familiar with the grep utility, this is a simplified version - -17
-of that utility. So basically the grep utility allows you - -18
-to search a file for the occurrences of a given - -19
-pattern. So you can invoke it, as it's shown here - -20
-in the synopsis, by executing grep, the pattern that you're - -21
-looking for, and the filename in which you want to - -22
-look for the pattern. And let me read the description - -23
-of the grep utility. The grep utility searches files for - -24
-a pattern and brings all lines that contain that pattern - -25
-on the standard output. A line that contains multiple occurrences of - -26
-the pattern is printed only once. The pattern is any sequence - -27
-of characters. To include a blank in the pattern, the entire - -28
-pattern must be enclosed in single quotes. To include a quote - -29
-sign in the pattern, the quote sign must be escaped, which - -30
-means that we have to put a slash in front of - -31
-the quotes sign. And in general, it is safest to enclose - -32
-the entire pattern in single quotes. So this is our high - -33
-level description for the program, for the softer system, that - -34
-we need to test. So now let me show you - -35
-what a possible set of categories and partitions could be - -36
-for this program. So what I have here is a - -37
-file, a textual file, which contains all the categories and - -38
-partitions for the elements that are relevant for my program. - -39
-In particular, when we look at the file, we can - -40
-see that the file can be characterized by its size. - -41
-And in this case, I've got two choices. The file can - -42
-be empty or not empty. The second characteristic of the file - -43
-that I'm considering is the number of occurrences of the pattern - -44
-in the file. And I'm considering that the pattern might not occur - -45
-in the file or it might occur once, or multiple times. - -46
-I'm not going to go through the rest of the file because we - -47
-already covered how to apply the category partition method in the - -48
-lesson. So if you had doubts about that, about the method and - -49
-how to apply, you might want to go back and watch again the - -50
-lesson. What I want to show you here is how you can go - -51
-from this information that you have here, that we have derived by - -52
-applying the, the first steps of the method, to a set of - -53
-test frames, and then, a set of test packs. So to do - -54
-that we're going to use the tool that I just mentioned. So let - -55
-me bring back my terminal. So first of all, let's see how - -56
-we can run the tool. So you have a manual that will explain - -57
-all the details on how to build the file that we're - -58
-going to feed the tool. So what is the format and so - -59
-on. Here I'm just going to see how I can run the - -60
-tool. So first of all, let me point out that this was - -61
-developed together by professors from the University of California Irvine and - -62
-Oregon State University. And as you can see, we can run - -63
-TSL generator and specify that we want to see the main - -64
-page. So in this case if we run it this, this way, - -65
-you'll have some basic information on how to run - -66
-the tool. And from the main page you can see - -67
-that you can specify the minus c flag and in - -68
-this case the TSL generator will report the number of - -69
-test frames generated without writing them to output. For - -70
-example, you might want to use this as we will do - -71
-to see how many tests that you will generate with - -72
-a current set of category partitions and choices. The minus - -73
-s option will bring the result of the TSL - -74
-generator on the standard output. And finally, you can - -75
-use minus o to specify an output file, where - -76
-to put the output of the program. So let's - -77
-at first run our TSL generator by specifying the - -78
-minus c option and by bypassing our current set - -79
-of category partitions and choices. Okay, so let me - -80
-remind you that what the, the tool will do - -81
-is what we will do manually. Otherwise, which is - -82
-to combine all these choices so as to have one - -83
-test case for each combination. So if we do - -84
-that, you can see that the tool tells us that - -85
-we will generate 7776 test frames in this case. - -86
-And this seems to be a little too much for - -87
-a program as small as the one that we are - -88
-testing. And assume for instance that we don't have the - -89
-resources to run this many test cases for, for - -90
-the grep program. In addition, consider that in this case, - -91
-we're computing all possible combinations of choices. And there's going to - -92
-be some combination that do not make sense as we - -93
-discussed in the lesson. So what we might want to - -94
-do in this case is to go back to our - -95
-spec and start adding constraints to eliminate this meaningless combination. - -96
-So I'm going to show you the result of doing that. - -97
-And I'm going to show you a few examples. - -98
-For example here, when the file is empty, I'm going to - -99
-define this property empty file. And how am I - -100
-going to use this property? Well for example here, it - -101
-doesn't make sense to consider the case in which - -102
-we have one or many occurrences of the pattern in - -103
-the file if the file is empty. Therefore I'm - -104
-going to tell the tool that it should consider this specific - -105
-choice only if the file is not empty, only if - -106
-empty file is not defined. And that will skip, for - -107
-example, all of the combinations in which the file is - -108
-empty. And I'm trying to generate the test case that has - -109
-one occurrence of the pattern in the file, which is - -110
-simply not possible. For another example, in case I have - -111
-an empty pattern, I define the property empty pattern. And - -112
-then I avoid the choices that involve the pattern in case - -113
-the pattern is empty. because, for example, I cannot - -114
-have quotes in a pattern that is empty. For example, - -115
-it doesn't make sense to have blanks. So, one or - -116
-more blanks if the pattern is empty. So I'm going to - -117
-specify again that this choice should be considered only if - -118
-we don't have an empty pattern. And so on and - -119
-so forth. So now after I edit these constraints, I - -120
-can go back and compute again the number of test - -121
-frames and therefore the test cases that will be - -122
-generated with these constraints. So let me go again - -123
-to my terminal. Okay, so now I'm going to run - -124
-my TSL generator again, and I'm going to run it on - -125
-the second version of this file. And you can - -126
-see that I reduced the, the number of test frames - -127
-from about 7800 to about 1700. So it's quite - -128
-a, quite a big reduction by eliminating all these combinations - -129
-that do not make sense. But let's assume again that - -130
-we want to reduce this further so that we don't want to - -131
-generate those many test frames and therefore test cases. So - -132
-what can we do? We go back to our spec. And - -133
-in this case, we start adding error constraints. So if - -134
-you remember what we said in the lesson, error constraints are - -135
-constraints that indicate a choice that has to do with an - -136
-erroneous behaviour. For example, an erroneous input provided to the problem. - -137
-So here for instance, we're indicating the presence - -138
-of incorrectly enclosing quotes as an error choice. Same - -139
-thing if there's no file corresponding to the - -140
-name that we provide to the tool, we say - -141
-that this corresponds to an error. So how - -142
-is the tool going to use this information? It uses - -143
-this information by producing only one combination that involves - -144
-error choices, instead of combining them with other choices. - -145
-So let's see what happens after we added this - -146
-error constraints. So we go back to our console - -147
-once more. And in this case, we want to run - -148
-the TSL generator with the version of the, of my - -149
-file that contains the area of constraints. And again, - -150
-I reduce quite a bit the number of test frames. - -151
-So now I have only 562 test frames that - -152
-will be generated by using the file that I provided. - -153
-So for the last time, let's assume that we really want - -154
-to cut down the number of test frames or the number of - -155
-test cases. So once more, we go back to our file, and - -156
-at this point what we can add is the final type of - -157
-constraints that we have, which are - -158
-single constraints. And single constraints are - -159
-basically indicated choices that we don't want to combine with other choices. - -160
-So they have the same effect of the error constraints, but they - -161
-have a different meaning, so they do not indicate choices that corresponds - -162
-to an error. In other words, I can use a - -163
-single constraints to identify choices that I want to test only once. - -164
-So for example in this case, I might decide that I - -165
-want to have only one test frame that tests my program - -166
-with a file being empty and I can do the - -167
-same for other choices. So basically I can continue adding this - -168
-single constraint until I get down to the number of test - -169
-frames and therefore the number of test cases that I want. - -170
-So now let's go back once more to our console. And so now if we run - -171
-using this file as input, you can see that we have 35 test frames generated. So - -172
-this is a fairly low number of test cases, so we might decide that we want to - -173
-go ahead and write these test frames to a file. So now let's open this file - -174
-that we just generated. And as you can see here, I have exactly 35 test frames, - -175
-as expected. Some of those correspond to the single and error cases. So in this - -176
-case, the only choice that I have indicated - -177
-is the one that corresponds to the single - -178
-or error constraint. What is for the other - -179
-ones? I actually have the whole test spec. - -180
-So let's pick one just to give you an example. - -181
-In this case, that's frame number 15 that will correspond to - -182
-test case number 15. And here you can see that - -183
-we have all the information. So this is a test specification. - -184
-All the information that we need to generate the corresponding - -185
-test. We know that we need a file that is not - -186
-empty. That we need to have one occurrence of the pattern - -187
-in the file. One occurrence of the pattern in one line. - -188
-The position of the pattern in the file can - -189
-be any position. The length of the pattern must - -190
-be more than one character. The pattern should not - -191
-be enclosed in quotes. There should be one white - -192
-space, one quote within the pattern, and finally the - -193
-file that would pass through the program should exist. - -194
-So the file should be present. So I can - -195
-easily transform all of this into an actual test case. - -196
-And notice that even though we're not, we're not - -197
-going to do it here. In cases like this, it might - -198
-even be possible to automatically generate the test cases - -199
-from the test specifications because, here for example, here it - -200
-should be relatively straight forward to parse these test - -201
-specifications and generate test cases accordingly. So, just to summarize, - -202
-what we have done is to go from one high-level - -203
-description of a program to a set of categories, partitions, - -204
-and choices for that program. Then we have combined them - -205
-in different ways, adding more and more constraints to reduce the - -206
-number of combinations until we ended up with the right number - -207
-of test cases, so the number of test cases that we - -208
-were fine generating. We generated the corresponding test specifications. And at - -209
-that point, we could just go ahead, generate the test case, - -210
-and test our application. So, and you can see how this - -211
-can result in a much more thorough testing of your application. - -212
-Because instead of reading this description and just trying to come up with test - -213
-cases for it, we can break down the process in steps that are easy to perform - -214
-individually. They can be automated as much - -215
-as possible. And they will end up with - -216
-a set of test cases that will test - -217
-all the interests and aspects of your application. -- cgit 1.4.1