diff options
Diffstat (limited to 'usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt')
| -rw-r--r-- | usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt | 867 |
1 files changed, 0 insertions, 867 deletions
diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt deleted file mode 100644 index 51c2e52..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt +++ /dev/null @@ -1,867 +0,0 @@ -1 -00:00:01,290 --> 00:00:03,210 -In this demo, we're going to do exactly what we - -2 -00:00:03,210 --> 00:00:06,370 -did just now in the lesson. We're going to use - -3 -00:00:06,370 --> 00:00:09,250 -the category partition method to go from a high-level - -4 -00:00:09,250 --> 00:00:12,930 -description of a piece of software of a program to - -5 -00:00:12,930 --> 00:00:15,512 -a set of test cases for that program. To - -6 -00:00:15,512 --> 00:00:17,640 -do that, we're going to use a simple tool. So - -7 -00:00:17,640 --> 00:00:21,040 -I'm going to show you here the tool that is - -8 -00:00:21,040 --> 00:00:26,380 -called a tsl generator right here. This tool is available - -9 -00:00:26,380 --> 00:00:28,520 -to you, so you can look in the class notes to - -10 -00:00:28,520 --> 00:00:31,900 -see information on how to download it. And together with the tool, - -11 -00:00:31,900 --> 00:00:34,890 -we are also going to provide a manual for the tool, and - -12 -00:00:34,890 --> 00:00:37,680 -a set of files that I'm going to use in this demo. So - -13 -00:00:37,680 --> 00:00:40,420 -you should be able to do exactly what I'm doing. So - -14 -00:00:40,420 --> 00:00:45,110 -again, all of those are available from the class notes. So specifically, - -15 -00:00:45,110 --> 00:00:48,390 -today we're going to write test cases for the grep program. So in - -16 -00:00:48,390 --> 00:00:51,970 -case you're familiar with the grep utility, this is a simplified version - -17 -00:00:51,970 --> 00:00:55,060 -of that utility. So basically the grep utility allows you - -18 -00:00:55,060 --> 00:00:58,410 -to search a file for the occurrences of a given - -19 -00:00:58,410 --> 00:01:01,552 -pattern. So you can invoke it, as it's shown here - -20 -00:01:01,552 --> 00:01:05,570 -in the synopsis, by executing grep, the pattern that you're - -21 -00:01:05,570 --> 00:01:08,310 -looking for, and the filename in which you want to - -22 -00:01:08,310 --> 00:01:10,300 -look for the pattern. And let me read the description - -23 -00:01:10,300 --> 00:01:13,650 -of the grep utility. The grep utility searches files for - -24 -00:01:13,650 --> 00:01:17,110 -a pattern and brings all lines that contain that pattern - -25 -00:01:17,110 --> 00:01:20,900 -on the standard output. A line that contains multiple occurrences of - -26 -00:01:20,900 --> 00:01:24,350 -the pattern is printed only once. The pattern is any sequence - -27 -00:01:24,350 --> 00:01:27,700 -of characters. To include a blank in the pattern, the entire - -28 -00:01:27,700 --> 00:01:31,060 -pattern must be enclosed in single quotes. To include a quote - -29 -00:01:31,060 --> 00:01:34,420 -sign in the pattern, the quote sign must be escaped, which - -30 -00:01:34,420 --> 00:01:36,260 -means that we have to put a slash in front of - -31 -00:01:36,260 --> 00:01:39,290 -the quotes sign. And in general, it is safest to enclose - -32 -00:01:39,290 --> 00:01:42,210 -the entire pattern in single quotes. So this is our high - -33 -00:01:42,210 --> 00:01:45,420 -level description for the program, for the softer system, that - -34 -00:01:45,420 --> 00:01:47,270 -we need to test. So now let me show you - -35 -00:01:47,270 --> 00:01:50,600 -what a possible set of categories and partitions could be - -36 -00:01:50,600 --> 00:01:53,770 -for this program. So what I have here is a - -37 -00:01:53,770 --> 00:01:58,080 -file, a textual file, which contains all the categories and - -38 -00:01:58,080 --> 00:02:02,760 -partitions for the elements that are relevant for my program. - -39 -00:02:02,760 --> 00:02:05,240 -In particular, when we look at the file, we can - -40 -00:02:05,240 --> 00:02:07,809 -see that the file can be characterized by its size. - -41 -00:02:08,889 --> 00:02:12,160 -And in this case, I've got two choices. The file can - -42 -00:02:12,160 --> 00:02:16,050 -be empty or not empty. The second characteristic of the file - -43 -00:02:16,050 --> 00:02:19,490 -that I'm considering is the number of occurrences of the pattern - -44 -00:02:19,490 --> 00:02:22,320 -in the file. And I'm considering that the pattern might not occur - -45 -00:02:22,320 --> 00:02:25,780 -in the file or it might occur once, or multiple times. - -46 -00:02:25,780 --> 00:02:28,264 -I'm not going to go through the rest of the file because we - -47 -00:02:28,264 --> 00:02:31,234 -already covered how to apply the category partition method in the - -48 -00:02:31,234 --> 00:02:34,226 -lesson. So if you had doubts about that, about the method and - -49 -00:02:34,226 --> 00:02:36,952 -how to apply, you might want to go back and watch again the - -50 -00:02:36,952 --> 00:02:40,040 -lesson. What I want to show you here is how you can go - -51 -00:02:40,040 --> 00:02:43,670 -from this information that you have here, that we have derived by - -52 -00:02:43,670 --> 00:02:47,020 -applying the, the first steps of the method, to a set of - -53 -00:02:47,020 --> 00:02:50,650 -test frames, and then, a set of test packs. So to do - -54 -00:02:50,650 --> 00:02:53,240 -that we're going to use the tool that I just mentioned. So let - -55 -00:02:53,240 --> 00:02:56,670 -me bring back my terminal. So first of all, let's see how - -56 -00:02:56,670 --> 00:02:59,570 -we can run the tool. So you have a manual that will explain - -57 -00:02:59,570 --> 00:03:02,180 -all the details on how to build the file that we're - -58 -00:03:02,180 --> 00:03:04,350 -going to feed the tool. So what is the format and so - -59 -00:03:04,350 --> 00:03:07,780 -on. Here I'm just going to see how I can run the - -60 -00:03:07,780 --> 00:03:10,828 -tool. So first of all, let me point out that this was - -61 -00:03:10,828 --> 00:03:15,028 -developed together by professors from the University of California Irvine and - -62 -00:03:15,028 --> 00:03:18,020 -Oregon State University. And as you can see, we can run - -63 -00:03:18,020 --> 00:03:20,968 -TSL generator and specify that we want to see the main - -64 -00:03:20,968 --> 00:03:24,361 -page. So in this case if we run it this, this way, - -65 -00:03:24,361 --> 00:03:27,723 -you'll have some basic information on how to run - -66 -00:03:27,723 --> 00:03:30,720 -the tool. And from the main page you can see - -67 -00:03:30,720 --> 00:03:33,520 -that you can specify the minus c flag and in - -68 -00:03:33,520 --> 00:03:37,360 -this case the TSL generator will report the number of - -69 -00:03:37,360 --> 00:03:41,410 -test frames generated without writing them to output. For - -70 -00:03:41,410 --> 00:03:43,828 -example, you might want to use this as we will do - -71 -00:03:43,828 --> 00:03:46,308 -to see how many tests that you will generate with - -72 -00:03:46,308 --> 00:03:49,630 -a current set of category partitions and choices. The minus - -73 -00:03:49,630 --> 00:03:52,620 -s option will bring the result of the TSL - -74 -00:03:52,620 --> 00:03:55,620 -generator on the standard output. And finally, you can - -75 -00:03:55,620 --> 00:03:58,330 -use minus o to specify an output file, where - -76 -00:03:58,330 --> 00:04:01,010 -to put the output of the program. So let's - -77 -00:04:01,010 --> 00:04:05,070 -at first run our TSL generator by specifying the - -78 -00:04:05,070 --> 00:04:08,620 -minus c option and by bypassing our current set - -79 -00:04:08,620 --> 00:04:11,860 -of category partitions and choices. Okay, so let me - -80 -00:04:11,860 --> 00:04:15,140 -remind you that what the, the tool will do - -81 -00:04:15,140 --> 00:04:17,790 -is what we will do manually. Otherwise, which is - -82 -00:04:17,790 --> 00:04:20,380 -to combine all these choices so as to have one - -83 -00:04:20,380 --> 00:04:23,305 -test case for each combination. So if we do - -84 -00:04:23,305 --> 00:04:27,412 -that, you can see that the tool tells us that - -85 -00:04:27,412 --> 00:04:32,383 -we will generate 7776 test frames in this case. - -86 -00:04:32,383 --> 00:04:34,660 -And this seems to be a little too much for - -87 -00:04:34,660 --> 00:04:36,868 -a program as small as the one that we are - -88 -00:04:36,868 --> 00:04:40,341 -testing. And assume for instance that we don't have the - -89 -00:04:40,341 --> 00:04:43,149 -resources to run this many test cases for, for - -90 -00:04:43,149 --> 00:04:46,518 -the grep program. In addition, consider that in this case, - -91 -00:04:46,518 --> 00:04:50,356 -we're computing all possible combinations of choices. And there's going to - -92 -00:04:50,356 --> 00:04:52,384 -be some combination that do not make sense as we - -93 -00:04:52,384 --> 00:04:54,945 -discussed in the lesson. So what we might want to - -94 -00:04:54,945 --> 00:04:57,051 -do in this case is to go back to our - -95 -00:04:57,051 --> 00:05:03,120 -spec and start adding constraints to eliminate this meaningless combination. - -96 -00:05:03,120 --> 00:05:05,980 -So I'm going to show you the result of doing that. - -97 -00:05:05,980 --> 00:05:08,690 -And I'm going to show you a few examples. - -98 -00:05:08,690 --> 00:05:11,670 -For example here, when the file is empty, I'm going to - -99 -00:05:11,670 --> 00:05:15,010 -define this property empty file. And how am I - -100 -00:05:15,010 --> 00:05:18,490 -going to use this property? Well for example here, it - -101 -00:05:18,490 --> 00:05:20,760 -doesn't make sense to consider the case in which - -102 -00:05:20,760 --> 00:05:24,660 -we have one or many occurrences of the pattern in - -103 -00:05:24,660 --> 00:05:27,020 -the file if the file is empty. Therefore I'm - -104 -00:05:27,020 --> 00:05:31,650 -going to tell the tool that it should consider this specific - -105 -00:05:31,650 --> 00:05:35,780 -choice only if the file is not empty, only if - -106 -00:05:35,780 --> 00:05:38,660 -empty file is not defined. And that will skip, for - -107 -00:05:38,660 --> 00:05:41,330 -example, all of the combinations in which the file is - -108 -00:05:41,330 --> 00:05:44,171 -empty. And I'm trying to generate the test case that has - -109 -00:05:44,171 --> 00:05:46,364 -one occurrence of the pattern in the file, which is - -110 -00:05:46,364 --> 00:05:49,355 -simply not possible. For another example, in case I have - -111 -00:05:49,355 --> 00:05:52,824 -an empty pattern, I define the property empty pattern. And - -112 -00:05:52,824 --> 00:05:56,725 -then I avoid the choices that involve the pattern in case - -113 -00:05:56,725 --> 00:05:59,820 -the pattern is empty. because, for example, I cannot - -114 -00:05:59,820 --> 00:06:03,900 -have quotes in a pattern that is empty. For example, - -115 -00:06:03,900 --> 00:06:06,760 -it doesn't make sense to have blanks. So, one or - -116 -00:06:06,760 --> 00:06:10,250 -more blanks if the pattern is empty. So I'm going to - -117 -00:06:10,250 --> 00:06:14,180 -specify again that this choice should be considered only if - -118 -00:06:14,180 --> 00:06:16,140 -we don't have an empty pattern. And so on and - -119 -00:06:16,140 --> 00:06:20,080 -so forth. So now after I edit these constraints, I - -120 -00:06:20,080 --> 00:06:21,890 -can go back and compute again the number of test - -121 -00:06:21,890 --> 00:06:23,970 -frames and therefore the test cases that will be - -122 -00:06:23,970 --> 00:06:27,530 -generated with these constraints. So let me go again - -123 -00:06:27,530 --> 00:06:30,381 -to my terminal. Okay, so now I'm going to run - -124 -00:06:30,381 --> 00:06:34,061 -my TSL generator again, and I'm going to run it on - -125 -00:06:34,061 --> 00:06:37,389 -the second version of this file. And you can - -126 -00:06:37,389 --> 00:06:40,546 -see that I reduced the, the number of test frames - -127 -00:06:40,546 --> 00:06:43,889 -from about 7800 to about 1700. So it's quite - -128 -00:06:43,889 --> 00:06:46,967 -a, quite a big reduction by eliminating all these combinations - -129 -00:06:46,967 --> 00:06:49,540 -that do not make sense. But let's assume again that - -130 -00:06:49,540 --> 00:06:52,040 -we want to reduce this further so that we don't want to - -131 -00:06:52,040 --> 00:06:55,610 -generate those many test frames and therefore test cases. So - -132 -00:06:55,610 --> 00:06:58,660 -what can we do? We go back to our spec. And - -133 -00:06:58,660 --> 00:07:02,280 -in this case, we start adding error constraints. So if - -134 -00:07:02,280 --> 00:07:05,200 -you remember what we said in the lesson, error constraints are - -135 -00:07:05,200 --> 00:07:08,310 -constraints that indicate a choice that has to do with an - -136 -00:07:08,310 --> 00:07:11,980 -erroneous behaviour. For example, an erroneous input provided to the problem. - -137 -00:07:11,980 --> 00:07:15,210 -So here for instance, we're indicating the presence - -138 -00:07:15,210 --> 00:07:20,060 -of incorrectly enclosing quotes as an error choice. Same - -139 -00:07:20,060 --> 00:07:22,270 -thing if there's no file corresponding to the - -140 -00:07:22,270 --> 00:07:23,970 -name that we provide to the tool, we say - -141 -00:07:23,970 --> 00:07:26,760 -that this corresponds to an error. So how - -142 -00:07:26,760 --> 00:07:29,130 -is the tool going to use this information? It uses - -143 -00:07:29,130 --> 00:07:33,980 -this information by producing only one combination that involves - -144 -00:07:33,980 --> 00:07:37,270 -error choices, instead of combining them with other choices. - -145 -00:07:37,270 --> 00:07:39,780 -So let's see what happens after we added this - -146 -00:07:39,780 --> 00:07:43,370 -error constraints. So we go back to our console - -147 -00:07:43,370 --> 00:07:46,920 -once more. And in this case, we want to run - -148 -00:07:46,920 --> 00:07:50,910 -the TSL generator with the version of the, of my - -149 -00:07:50,910 --> 00:07:53,900 -file that contains the area of constraints. And again, - -150 -00:07:53,900 --> 00:07:56,390 -I reduce quite a bit the number of test frames. - -151 -00:07:56,390 --> 00:07:59,110 -So now I have only 562 test frames that - -152 -00:07:59,110 --> 00:08:02,660 -will be generated by using the file that I provided. - -153 -00:08:02,660 --> 00:08:05,460 -So for the last time, let's assume that we really want - -154 -00:08:05,460 --> 00:08:07,780 -to cut down the number of test frames or the number of - -155 -00:08:07,780 --> 00:08:10,380 -test cases. So once more, we go back to our file, and - -156 -00:08:10,380 --> 00:08:12,980 -at this point what we can add is the final type of - -157 -00:08:12,980 --> 00:08:14,170 -constraints that we have, which are - -158 -00:08:14,170 --> 00:08:17,245 -single constraints. And single constraints are - -159 -00:08:17,245 --> 00:08:21,360 -basically indicated choices that we don't want to combine with other choices. - -160 -00:08:21,360 --> 00:08:24,210 -So they have the same effect of the error constraints, but they - -161 -00:08:24,210 --> 00:08:28,120 -have a different meaning, so they do not indicate choices that corresponds - -162 -00:08:28,120 --> 00:08:29,860 -to an error. In other words, I can use a - -163 -00:08:29,860 --> 00:08:35,280 -single constraints to identify choices that I want to test only once. - -164 -00:08:35,280 --> 00:08:38,510 -So for example in this case, I might decide that I - -165 -00:08:38,510 --> 00:08:42,520 -want to have only one test frame that tests my program - -166 -00:08:42,520 --> 00:08:44,420 -with a file being empty and I can do the - -167 -00:08:44,420 --> 00:08:47,370 -same for other choices. So basically I can continue adding this - -168 -00:08:47,370 --> 00:08:50,400 -single constraint until I get down to the number of test - -169 -00:08:50,400 --> 00:08:53,410 -frames and therefore the number of test cases that I want. - -170 -00:08:53,410 --> 00:08:57,770 -So now let's go back once more to our console. And so now if we run - -171 -00:08:59,060 --> 00:09:04,450 -using this file as input, you can see that we have 35 test frames generated. So - -172 -00:09:04,450 --> 00:09:07,750 -this is a fairly low number of test cases, so we might decide that we want to - -173 -00:09:07,750 --> 00:09:13,380 -go ahead and write these test frames to a file. So now let's open this file - -174 -00:09:15,990 --> 00:09:25,500 -that we just generated. And as you can see here, I have exactly 35 test frames, - -175 -00:09:26,670 --> 00:09:30,900 -as expected. Some of those correspond to the single and error cases. So in this - -176 -00:09:30,900 --> 00:09:33,330 -case, the only choice that I have indicated - -177 -00:09:33,330 --> 00:09:35,690 -is the one that corresponds to the single - -178 -00:09:35,690 --> 00:09:38,310 -or error constraint. What is for the other - -179 -00:09:38,310 --> 00:09:42,170 -ones? I actually have the whole test spec. - -180 -00:09:42,170 --> 00:09:45,530 -So let's pick one just to give you an example. - -181 -00:09:45,530 --> 00:09:48,440 -In this case, that's frame number 15 that will correspond to - -182 -00:09:48,440 --> 00:09:51,910 -test case number 15. And here you can see that - -183 -00:09:51,910 --> 00:09:55,280 -we have all the information. So this is a test specification. - -184 -00:09:55,280 --> 00:09:57,560 -All the information that we need to generate the corresponding - -185 -00:09:57,560 --> 00:09:59,760 -test. We know that we need a file that is not - -186 -00:09:59,760 --> 00:10:03,810 -empty. That we need to have one occurrence of the pattern - -187 -00:10:03,810 --> 00:10:07,580 -in the file. One occurrence of the pattern in one line. - -188 -00:10:08,680 --> 00:10:10,360 -The position of the pattern in the file can - -189 -00:10:10,360 --> 00:10:13,740 -be any position. The length of the pattern must - -190 -00:10:13,740 --> 00:10:16,640 -be more than one character. The pattern should not - -191 -00:10:16,640 --> 00:10:20,140 -be enclosed in quotes. There should be one white - -192 -00:10:20,140 --> 00:10:24,460 -space, one quote within the pattern, and finally the - -193 -00:10:24,460 --> 00:10:27,230 -file that would pass through the program should exist. - -194 -00:10:27,230 --> 00:10:29,680 -So the file should be present. So I can - -195 -00:10:29,680 --> 00:10:33,950 -easily transform all of this into an actual test case. - -196 -00:10:33,950 --> 00:10:35,540 -And notice that even though we're not, we're not - -197 -00:10:35,540 --> 00:10:38,420 -going to do it here. In cases like this, it might - -198 -00:10:38,420 --> 00:10:42,190 -even be possible to automatically generate the test cases - -199 -00:10:42,190 --> 00:10:45,020 -from the test specifications because, here for example, here it - -200 -00:10:45,020 --> 00:10:48,150 -should be relatively straight forward to parse these test - -201 -00:10:48,150 --> 00:10:52,450 -specifications and generate test cases accordingly. So, just to summarize, - -202 -00:10:52,450 --> 00:10:55,910 -what we have done is to go from one high-level - -203 -00:10:55,910 --> 00:10:58,880 -description of a program to a set of categories, partitions, - -204 -00:10:58,880 --> 00:11:01,820 -and choices for that program. Then we have combined them - -205 -00:11:01,820 --> 00:11:04,930 -in different ways, adding more and more constraints to reduce the - -206 -00:11:04,930 --> 00:11:07,600 -number of combinations until we ended up with the right number - -207 -00:11:07,600 --> 00:11:09,650 -of test cases, so the number of test cases that we - -208 -00:11:09,650 --> 00:11:14,630 -were fine generating. We generated the corresponding test specifications. And at - -209 -00:11:14,630 --> 00:11:17,340 -that point, we could just go ahead, generate the test case, - -210 -00:11:17,340 --> 00:11:20,660 -and test our application. So, and you can see how this - -211 -00:11:20,660 --> 00:11:23,720 -can result in a much more thorough testing of your application. - -212 -00:11:23,720 --> 00:11:27,890 -Because instead of reading this description and just trying to come up with test - -213 -00:11:27,890 --> 00:11:33,900 -cases for it, we can break down the process in steps that are easy to perform - -214 -00:11:33,900 --> 00:11:36,960 -individually. They can be automated as much - -215 -00:11:36,960 --> 00:11:38,600 -as possible. And they will end up with - -216 -00:11:38,600 --> 00:11:40,380 -a set of test cases that will test - -217 -00:11:40,380 --> 00:11:42,790 -all the interests and aspects of your application. |