diff options
Diffstat (limited to 'usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt')
| -rw-r--r-- | usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt | 867 |
1 files changed, 867 insertions, 0 deletions
diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt new file mode 100644 index 0000000..51c2e52 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt @@ -0,0 +1,867 @@ +1 +00:00:01,290 --> 00:00:03,210 +In this demo, we're going to do exactly what we + +2 +00:00:03,210 --> 00:00:06,370 +did just now in the lesson. We're going to use + +3 +00:00:06,370 --> 00:00:09,250 +the category partition method to go from a high-level + +4 +00:00:09,250 --> 00:00:12,930 +description of a piece of software of a program to + +5 +00:00:12,930 --> 00:00:15,512 +a set of test cases for that program. To + +6 +00:00:15,512 --> 00:00:17,640 +do that, we're going to use a simple tool. So + +7 +00:00:17,640 --> 00:00:21,040 +I'm going to show you here the tool that is + +8 +00:00:21,040 --> 00:00:26,380 +called a tsl generator right here. This tool is available + +9 +00:00:26,380 --> 00:00:28,520 +to you, so you can look in the class notes to + +10 +00:00:28,520 --> 00:00:31,900 +see information on how to download it. And together with the tool, + +11 +00:00:31,900 --> 00:00:34,890 +we are also going to provide a manual for the tool, and + +12 +00:00:34,890 --> 00:00:37,680 +a set of files that I'm going to use in this demo. So + +13 +00:00:37,680 --> 00:00:40,420 +you should be able to do exactly what I'm doing. So + +14 +00:00:40,420 --> 00:00:45,110 +again, all of those are available from the class notes. So specifically, + +15 +00:00:45,110 --> 00:00:48,390 +today we're going to write test cases for the grep program. So in + +16 +00:00:48,390 --> 00:00:51,970 +case you're familiar with the grep utility, this is a simplified version + +17 +00:00:51,970 --> 00:00:55,060 +of that utility. So basically the grep utility allows you + +18 +00:00:55,060 --> 00:00:58,410 +to search a file for the occurrences of a given + +19 +00:00:58,410 --> 00:01:01,552 +pattern. So you can invoke it, as it's shown here + +20 +00:01:01,552 --> 00:01:05,570 +in the synopsis, by executing grep, the pattern that you're + +21 +00:01:05,570 --> 00:01:08,310 +looking for, and the filename in which you want to + +22 +00:01:08,310 --> 00:01:10,300 +look for the pattern. And let me read the description + +23 +00:01:10,300 --> 00:01:13,650 +of the grep utility. The grep utility searches files for + +24 +00:01:13,650 --> 00:01:17,110 +a pattern and brings all lines that contain that pattern + +25 +00:01:17,110 --> 00:01:20,900 +on the standard output. A line that contains multiple occurrences of + +26 +00:01:20,900 --> 00:01:24,350 +the pattern is printed only once. The pattern is any sequence + +27 +00:01:24,350 --> 00:01:27,700 +of characters. To include a blank in the pattern, the entire + +28 +00:01:27,700 --> 00:01:31,060 +pattern must be enclosed in single quotes. To include a quote + +29 +00:01:31,060 --> 00:01:34,420 +sign in the pattern, the quote sign must be escaped, which + +30 +00:01:34,420 --> 00:01:36,260 +means that we have to put a slash in front of + +31 +00:01:36,260 --> 00:01:39,290 +the quotes sign. And in general, it is safest to enclose + +32 +00:01:39,290 --> 00:01:42,210 +the entire pattern in single quotes. So this is our high + +33 +00:01:42,210 --> 00:01:45,420 +level description for the program, for the softer system, that + +34 +00:01:45,420 --> 00:01:47,270 +we need to test. So now let me show you + +35 +00:01:47,270 --> 00:01:50,600 +what a possible set of categories and partitions could be + +36 +00:01:50,600 --> 00:01:53,770 +for this program. So what I have here is a + +37 +00:01:53,770 --> 00:01:58,080 +file, a textual file, which contains all the categories and + +38 +00:01:58,080 --> 00:02:02,760 +partitions for the elements that are relevant for my program. + +39 +00:02:02,760 --> 00:02:05,240 +In particular, when we look at the file, we can + +40 +00:02:05,240 --> 00:02:07,809 +see that the file can be characterized by its size. + +41 +00:02:08,889 --> 00:02:12,160 +And in this case, I've got two choices. The file can + +42 +00:02:12,160 --> 00:02:16,050 +be empty or not empty. The second characteristic of the file + +43 +00:02:16,050 --> 00:02:19,490 +that I'm considering is the number of occurrences of the pattern + +44 +00:02:19,490 --> 00:02:22,320 +in the file. And I'm considering that the pattern might not occur + +45 +00:02:22,320 --> 00:02:25,780 +in the file or it might occur once, or multiple times. + +46 +00:02:25,780 --> 00:02:28,264 +I'm not going to go through the rest of the file because we + +47 +00:02:28,264 --> 00:02:31,234 +already covered how to apply the category partition method in the + +48 +00:02:31,234 --> 00:02:34,226 +lesson. So if you had doubts about that, about the method and + +49 +00:02:34,226 --> 00:02:36,952 +how to apply, you might want to go back and watch again the + +50 +00:02:36,952 --> 00:02:40,040 +lesson. What I want to show you here is how you can go + +51 +00:02:40,040 --> 00:02:43,670 +from this information that you have here, that we have derived by + +52 +00:02:43,670 --> 00:02:47,020 +applying the, the first steps of the method, to a set of + +53 +00:02:47,020 --> 00:02:50,650 +test frames, and then, a set of test packs. So to do + +54 +00:02:50,650 --> 00:02:53,240 +that we're going to use the tool that I just mentioned. So let + +55 +00:02:53,240 --> 00:02:56,670 +me bring back my terminal. So first of all, let's see how + +56 +00:02:56,670 --> 00:02:59,570 +we can run the tool. So you have a manual that will explain + +57 +00:02:59,570 --> 00:03:02,180 +all the details on how to build the file that we're + +58 +00:03:02,180 --> 00:03:04,350 +going to feed the tool. So what is the format and so + +59 +00:03:04,350 --> 00:03:07,780 +on. Here I'm just going to see how I can run the + +60 +00:03:07,780 --> 00:03:10,828 +tool. So first of all, let me point out that this was + +61 +00:03:10,828 --> 00:03:15,028 +developed together by professors from the University of California Irvine and + +62 +00:03:15,028 --> 00:03:18,020 +Oregon State University. And as you can see, we can run + +63 +00:03:18,020 --> 00:03:20,968 +TSL generator and specify that we want to see the main + +64 +00:03:20,968 --> 00:03:24,361 +page. So in this case if we run it this, this way, + +65 +00:03:24,361 --> 00:03:27,723 +you'll have some basic information on how to run + +66 +00:03:27,723 --> 00:03:30,720 +the tool. And from the main page you can see + +67 +00:03:30,720 --> 00:03:33,520 +that you can specify the minus c flag and in + +68 +00:03:33,520 --> 00:03:37,360 +this case the TSL generator will report the number of + +69 +00:03:37,360 --> 00:03:41,410 +test frames generated without writing them to output. For + +70 +00:03:41,410 --> 00:03:43,828 +example, you might want to use this as we will do + +71 +00:03:43,828 --> 00:03:46,308 +to see how many tests that you will generate with + +72 +00:03:46,308 --> 00:03:49,630 +a current set of category partitions and choices. The minus + +73 +00:03:49,630 --> 00:03:52,620 +s option will bring the result of the TSL + +74 +00:03:52,620 --> 00:03:55,620 +generator on the standard output. And finally, you can + +75 +00:03:55,620 --> 00:03:58,330 +use minus o to specify an output file, where + +76 +00:03:58,330 --> 00:04:01,010 +to put the output of the program. So let's + +77 +00:04:01,010 --> 00:04:05,070 +at first run our TSL generator by specifying the + +78 +00:04:05,070 --> 00:04:08,620 +minus c option and by bypassing our current set + +79 +00:04:08,620 --> 00:04:11,860 +of category partitions and choices. Okay, so let me + +80 +00:04:11,860 --> 00:04:15,140 +remind you that what the, the tool will do + +81 +00:04:15,140 --> 00:04:17,790 +is what we will do manually. Otherwise, which is + +82 +00:04:17,790 --> 00:04:20,380 +to combine all these choices so as to have one + +83 +00:04:20,380 --> 00:04:23,305 +test case for each combination. So if we do + +84 +00:04:23,305 --> 00:04:27,412 +that, you can see that the tool tells us that + +85 +00:04:27,412 --> 00:04:32,383 +we will generate 7776 test frames in this case. + +86 +00:04:32,383 --> 00:04:34,660 +And this seems to be a little too much for + +87 +00:04:34,660 --> 00:04:36,868 +a program as small as the one that we are + +88 +00:04:36,868 --> 00:04:40,341 +testing. And assume for instance that we don't have the + +89 +00:04:40,341 --> 00:04:43,149 +resources to run this many test cases for, for + +90 +00:04:43,149 --> 00:04:46,518 +the grep program. In addition, consider that in this case, + +91 +00:04:46,518 --> 00:04:50,356 +we're computing all possible combinations of choices. And there's going to + +92 +00:04:50,356 --> 00:04:52,384 +be some combination that do not make sense as we + +93 +00:04:52,384 --> 00:04:54,945 +discussed in the lesson. So what we might want to + +94 +00:04:54,945 --> 00:04:57,051 +do in this case is to go back to our + +95 +00:04:57,051 --> 00:05:03,120 +spec and start adding constraints to eliminate this meaningless combination. + +96 +00:05:03,120 --> 00:05:05,980 +So I'm going to show you the result of doing that. + +97 +00:05:05,980 --> 00:05:08,690 +And I'm going to show you a few examples. + +98 +00:05:08,690 --> 00:05:11,670 +For example here, when the file is empty, I'm going to + +99 +00:05:11,670 --> 00:05:15,010 +define this property empty file. And how am I + +100 +00:05:15,010 --> 00:05:18,490 +going to use this property? Well for example here, it + +101 +00:05:18,490 --> 00:05:20,760 +doesn't make sense to consider the case in which + +102 +00:05:20,760 --> 00:05:24,660 +we have one or many occurrences of the pattern in + +103 +00:05:24,660 --> 00:05:27,020 +the file if the file is empty. Therefore I'm + +104 +00:05:27,020 --> 00:05:31,650 +going to tell the tool that it should consider this specific + +105 +00:05:31,650 --> 00:05:35,780 +choice only if the file is not empty, only if + +106 +00:05:35,780 --> 00:05:38,660 +empty file is not defined. And that will skip, for + +107 +00:05:38,660 --> 00:05:41,330 +example, all of the combinations in which the file is + +108 +00:05:41,330 --> 00:05:44,171 +empty. And I'm trying to generate the test case that has + +109 +00:05:44,171 --> 00:05:46,364 +one occurrence of the pattern in the file, which is + +110 +00:05:46,364 --> 00:05:49,355 +simply not possible. For another example, in case I have + +111 +00:05:49,355 --> 00:05:52,824 +an empty pattern, I define the property empty pattern. And + +112 +00:05:52,824 --> 00:05:56,725 +then I avoid the choices that involve the pattern in case + +113 +00:05:56,725 --> 00:05:59,820 +the pattern is empty. because, for example, I cannot + +114 +00:05:59,820 --> 00:06:03,900 +have quotes in a pattern that is empty. For example, + +115 +00:06:03,900 --> 00:06:06,760 +it doesn't make sense to have blanks. So, one or + +116 +00:06:06,760 --> 00:06:10,250 +more blanks if the pattern is empty. So I'm going to + +117 +00:06:10,250 --> 00:06:14,180 +specify again that this choice should be considered only if + +118 +00:06:14,180 --> 00:06:16,140 +we don't have an empty pattern. And so on and + +119 +00:06:16,140 --> 00:06:20,080 +so forth. So now after I edit these constraints, I + +120 +00:06:20,080 --> 00:06:21,890 +can go back and compute again the number of test + +121 +00:06:21,890 --> 00:06:23,970 +frames and therefore the test cases that will be + +122 +00:06:23,970 --> 00:06:27,530 +generated with these constraints. So let me go again + +123 +00:06:27,530 --> 00:06:30,381 +to my terminal. Okay, so now I'm going to run + +124 +00:06:30,381 --> 00:06:34,061 +my TSL generator again, and I'm going to run it on + +125 +00:06:34,061 --> 00:06:37,389 +the second version of this file. And you can + +126 +00:06:37,389 --> 00:06:40,546 +see that I reduced the, the number of test frames + +127 +00:06:40,546 --> 00:06:43,889 +from about 7800 to about 1700. So it's quite + +128 +00:06:43,889 --> 00:06:46,967 +a, quite a big reduction by eliminating all these combinations + +129 +00:06:46,967 --> 00:06:49,540 +that do not make sense. But let's assume again that + +130 +00:06:49,540 --> 00:06:52,040 +we want to reduce this further so that we don't want to + +131 +00:06:52,040 --> 00:06:55,610 +generate those many test frames and therefore test cases. So + +132 +00:06:55,610 --> 00:06:58,660 +what can we do? We go back to our spec. And + +133 +00:06:58,660 --> 00:07:02,280 +in this case, we start adding error constraints. So if + +134 +00:07:02,280 --> 00:07:05,200 +you remember what we said in the lesson, error constraints are + +135 +00:07:05,200 --> 00:07:08,310 +constraints that indicate a choice that has to do with an + +136 +00:07:08,310 --> 00:07:11,980 +erroneous behaviour. For example, an erroneous input provided to the problem. + +137 +00:07:11,980 --> 00:07:15,210 +So here for instance, we're indicating the presence + +138 +00:07:15,210 --> 00:07:20,060 +of incorrectly enclosing quotes as an error choice. Same + +139 +00:07:20,060 --> 00:07:22,270 +thing if there's no file corresponding to the + +140 +00:07:22,270 --> 00:07:23,970 +name that we provide to the tool, we say + +141 +00:07:23,970 --> 00:07:26,760 +that this corresponds to an error. So how + +142 +00:07:26,760 --> 00:07:29,130 +is the tool going to use this information? It uses + +143 +00:07:29,130 --> 00:07:33,980 +this information by producing only one combination that involves + +144 +00:07:33,980 --> 00:07:37,270 +error choices, instead of combining them with other choices. + +145 +00:07:37,270 --> 00:07:39,780 +So let's see what happens after we added this + +146 +00:07:39,780 --> 00:07:43,370 +error constraints. So we go back to our console + +147 +00:07:43,370 --> 00:07:46,920 +once more. And in this case, we want to run + +148 +00:07:46,920 --> 00:07:50,910 +the TSL generator with the version of the, of my + +149 +00:07:50,910 --> 00:07:53,900 +file that contains the area of constraints. And again, + +150 +00:07:53,900 --> 00:07:56,390 +I reduce quite a bit the number of test frames. + +151 +00:07:56,390 --> 00:07:59,110 +So now I have only 562 test frames that + +152 +00:07:59,110 --> 00:08:02,660 +will be generated by using the file that I provided. + +153 +00:08:02,660 --> 00:08:05,460 +So for the last time, let's assume that we really want + +154 +00:08:05,460 --> 00:08:07,780 +to cut down the number of test frames or the number of + +155 +00:08:07,780 --> 00:08:10,380 +test cases. So once more, we go back to our file, and + +156 +00:08:10,380 --> 00:08:12,980 +at this point what we can add is the final type of + +157 +00:08:12,980 --> 00:08:14,170 +constraints that we have, which are + +158 +00:08:14,170 --> 00:08:17,245 +single constraints. And single constraints are + +159 +00:08:17,245 --> 00:08:21,360 +basically indicated choices that we don't want to combine with other choices. + +160 +00:08:21,360 --> 00:08:24,210 +So they have the same effect of the error constraints, but they + +161 +00:08:24,210 --> 00:08:28,120 +have a different meaning, so they do not indicate choices that corresponds + +162 +00:08:28,120 --> 00:08:29,860 +to an error. In other words, I can use a + +163 +00:08:29,860 --> 00:08:35,280 +single constraints to identify choices that I want to test only once. + +164 +00:08:35,280 --> 00:08:38,510 +So for example in this case, I might decide that I + +165 +00:08:38,510 --> 00:08:42,520 +want to have only one test frame that tests my program + +166 +00:08:42,520 --> 00:08:44,420 +with a file being empty and I can do the + +167 +00:08:44,420 --> 00:08:47,370 +same for other choices. So basically I can continue adding this + +168 +00:08:47,370 --> 00:08:50,400 +single constraint until I get down to the number of test + +169 +00:08:50,400 --> 00:08:53,410 +frames and therefore the number of test cases that I want. + +170 +00:08:53,410 --> 00:08:57,770 +So now let's go back once more to our console. And so now if we run + +171 +00:08:59,060 --> 00:09:04,450 +using this file as input, you can see that we have 35 test frames generated. So + +172 +00:09:04,450 --> 00:09:07,750 +this is a fairly low number of test cases, so we might decide that we want to + +173 +00:09:07,750 --> 00:09:13,380 +go ahead and write these test frames to a file. So now let's open this file + +174 +00:09:15,990 --> 00:09:25,500 +that we just generated. And as you can see here, I have exactly 35 test frames, + +175 +00:09:26,670 --> 00:09:30,900 +as expected. Some of those correspond to the single and error cases. So in this + +176 +00:09:30,900 --> 00:09:33,330 +case, the only choice that I have indicated + +177 +00:09:33,330 --> 00:09:35,690 +is the one that corresponds to the single + +178 +00:09:35,690 --> 00:09:38,310 +or error constraint. What is for the other + +179 +00:09:38,310 --> 00:09:42,170 +ones? I actually have the whole test spec. + +180 +00:09:42,170 --> 00:09:45,530 +So let's pick one just to give you an example. + +181 +00:09:45,530 --> 00:09:48,440 +In this case, that's frame number 15 that will correspond to + +182 +00:09:48,440 --> 00:09:51,910 +test case number 15. And here you can see that + +183 +00:09:51,910 --> 00:09:55,280 +we have all the information. So this is a test specification. + +184 +00:09:55,280 --> 00:09:57,560 +All the information that we need to generate the corresponding + +185 +00:09:57,560 --> 00:09:59,760 +test. We know that we need a file that is not + +186 +00:09:59,760 --> 00:10:03,810 +empty. That we need to have one occurrence of the pattern + +187 +00:10:03,810 --> 00:10:07,580 +in the file. One occurrence of the pattern in one line. + +188 +00:10:08,680 --> 00:10:10,360 +The position of the pattern in the file can + +189 +00:10:10,360 --> 00:10:13,740 +be any position. The length of the pattern must + +190 +00:10:13,740 --> 00:10:16,640 +be more than one character. The pattern should not + +191 +00:10:16,640 --> 00:10:20,140 +be enclosed in quotes. There should be one white + +192 +00:10:20,140 --> 00:10:24,460 +space, one quote within the pattern, and finally the + +193 +00:10:24,460 --> 00:10:27,230 +file that would pass through the program should exist. + +194 +00:10:27,230 --> 00:10:29,680 +So the file should be present. So I can + +195 +00:10:29,680 --> 00:10:33,950 +easily transform all of this into an actual test case. + +196 +00:10:33,950 --> 00:10:35,540 +And notice that even though we're not, we're not + +197 +00:10:35,540 --> 00:10:38,420 +going to do it here. In cases like this, it might + +198 +00:10:38,420 --> 00:10:42,190 +even be possible to automatically generate the test cases + +199 +00:10:42,190 --> 00:10:45,020 +from the test specifications because, here for example, here it + +200 +00:10:45,020 --> 00:10:48,150 +should be relatively straight forward to parse these test + +201 +00:10:48,150 --> 00:10:52,450 +specifications and generate test cases accordingly. So, just to summarize, + +202 +00:10:52,450 --> 00:10:55,910 +what we have done is to go from one high-level + +203 +00:10:55,910 --> 00:10:58,880 +description of a program to a set of categories, partitions, + +204 +00:10:58,880 --> 00:11:01,820 +and choices for that program. Then we have combined them + +205 +00:11:01,820 --> 00:11:04,930 +in different ways, adding more and more constraints to reduce the + +206 +00:11:04,930 --> 00:11:07,600 +number of combinations until we ended up with the right number + +207 +00:11:07,600 --> 00:11:09,650 +of test cases, so the number of test cases that we + +208 +00:11:09,650 --> 00:11:14,630 +were fine generating. We generated the corresponding test specifications. And at + +209 +00:11:14,630 --> 00:11:17,340 +that point, we could just go ahead, generate the test case, + +210 +00:11:17,340 --> 00:11:20,660 +and test our application. So, and you can see how this + +211 +00:11:20,660 --> 00:11:23,720 +can result in a much more thorough testing of your application. + +212 +00:11:23,720 --> 00:11:27,890 +Because instead of reading this description and just trying to come up with test + +213 +00:11:27,890 --> 00:11:33,900 +cases for it, we can break down the process in steps that are easy to perform + +214 +00:11:33,900 --> 00:11:36,960 +individually. They can be automated as much + +215 +00:11:36,960 --> 00:11:38,600 +as possible. And they will end up with + +216 +00:11:38,600 --> 00:11:40,380 +a set of test cases that will test + +217 +00:11:40,380 --> 00:11:42,790 +all the interests and aspects of your application. |