From b2d80610db6beda38573890ed169815e495bc663 Mon Sep 17 00:00:00 2001
From: Nguyễn Gia Phong <mcsinyx@disroot.org>
Date: Sun, 24 May 2020 16:34:31 +0700
Subject: [usth/ICT2.7] Engineer software

---
 .../23 - Category Partition Demo - lang_en_vs4.srt | 867 +++++++++++++++++++++
 1 file changed, 867 insertions(+)
 create mode 100644 usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt

(limited to 'usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt')

diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt
new file mode 100644
index 0000000..51c2e52
--- /dev/null
+++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt	
@@ -0,0 +1,867 @@
+1
+00:00:01,290 --> 00:00:03,210
+In this demo, we're going to do exactly what we
+
+2
+00:00:03,210 --> 00:00:06,370
+did just now in the lesson. We're going to use
+
+3
+00:00:06,370 --> 00:00:09,250
+the category partition method to go from a high-level
+
+4
+00:00:09,250 --> 00:00:12,930
+description of a piece of software of a program to
+
+5
+00:00:12,930 --> 00:00:15,512
+a set of test cases for that program. To
+
+6
+00:00:15,512 --> 00:00:17,640
+do that, we're going to use a simple tool. So
+
+7
+00:00:17,640 --> 00:00:21,040
+I'm going to show you here the tool that is
+
+8
+00:00:21,040 --> 00:00:26,380
+called a tsl generator right here. This tool is available
+
+9
+00:00:26,380 --> 00:00:28,520
+to you, so you can look in the class notes to
+
+10
+00:00:28,520 --> 00:00:31,900
+see information on how to download it. And together with the tool,
+
+11
+00:00:31,900 --> 00:00:34,890
+we are also going to provide a manual for the tool, and
+
+12
+00:00:34,890 --> 00:00:37,680
+a set of files that I'm going to use in this demo. So
+
+13
+00:00:37,680 --> 00:00:40,420
+you should be able to do exactly what I'm doing. So
+
+14
+00:00:40,420 --> 00:00:45,110
+again, all of those are available from the class notes. So specifically,
+
+15
+00:00:45,110 --> 00:00:48,390
+today we're going to write test cases for the grep program. So in
+
+16
+00:00:48,390 --> 00:00:51,970
+case you're familiar with the grep utility, this is a simplified version
+
+17
+00:00:51,970 --> 00:00:55,060
+of that utility. So basically the grep utility allows you
+
+18
+00:00:55,060 --> 00:00:58,410
+to search a file for the occurrences of a given
+
+19
+00:00:58,410 --> 00:01:01,552
+pattern. So you can invoke it, as it's shown here
+
+20
+00:01:01,552 --> 00:01:05,570
+in the synopsis, by executing grep, the pattern that you're
+
+21
+00:01:05,570 --> 00:01:08,310
+looking for, and the filename in which you want to
+
+22
+00:01:08,310 --> 00:01:10,300
+look for the pattern. And let me read the description
+
+23
+00:01:10,300 --> 00:01:13,650
+of the grep utility. The grep utility searches files for
+
+24
+00:01:13,650 --> 00:01:17,110
+a pattern and brings all lines that contain that pattern
+
+25
+00:01:17,110 --> 00:01:20,900
+on the standard output. A line that contains multiple occurrences of
+
+26
+00:01:20,900 --> 00:01:24,350
+the pattern is printed only once. The pattern is any sequence
+
+27
+00:01:24,350 --> 00:01:27,700
+of characters. To include a blank in the pattern, the entire
+
+28
+00:01:27,700 --> 00:01:31,060
+pattern must be enclosed in single quotes. To include a quote
+
+29
+00:01:31,060 --> 00:01:34,420
+sign in the pattern, the quote sign must be escaped, which
+
+30
+00:01:34,420 --> 00:01:36,260
+means that we have to put a slash in front of
+
+31
+00:01:36,260 --> 00:01:39,290
+the quotes sign. And in general, it is safest to enclose
+
+32
+00:01:39,290 --> 00:01:42,210
+the entire pattern in single quotes. So this is our high
+
+33
+00:01:42,210 --> 00:01:45,420
+level description for the program, for the softer system, that
+
+34
+00:01:45,420 --> 00:01:47,270
+we need to test. So now let me show you
+
+35
+00:01:47,270 --> 00:01:50,600
+what a possible set of categories and partitions could be
+
+36
+00:01:50,600 --> 00:01:53,770
+for this program. So what I have here is a
+
+37
+00:01:53,770 --> 00:01:58,080
+file, a textual file, which contains all the categories and
+
+38
+00:01:58,080 --> 00:02:02,760
+partitions for the elements that are relevant for my program.
+
+39
+00:02:02,760 --> 00:02:05,240
+In particular, when we look at the file, we can
+
+40
+00:02:05,240 --> 00:02:07,809
+see that the file can be characterized by its size.
+
+41
+00:02:08,889 --> 00:02:12,160
+And in this case, I've got two choices. The file can
+
+42
+00:02:12,160 --> 00:02:16,050
+be empty or not empty. The second characteristic of the file
+
+43
+00:02:16,050 --> 00:02:19,490
+that I'm considering is the number of occurrences of the pattern
+
+44
+00:02:19,490 --> 00:02:22,320
+in the file. And I'm considering that the pattern might not occur
+
+45
+00:02:22,320 --> 00:02:25,780
+in the file or it might occur once, or multiple times.
+
+46
+00:02:25,780 --> 00:02:28,264
+I'm not going to go through the rest of the file because we
+
+47
+00:02:28,264 --> 00:02:31,234
+already covered how to apply the category partition method in the
+
+48
+00:02:31,234 --> 00:02:34,226
+lesson. So if you had doubts about that, about the method and
+
+49
+00:02:34,226 --> 00:02:36,952
+how to apply, you might want to go back and watch again the
+
+50
+00:02:36,952 --> 00:02:40,040
+lesson. What I want to show you here is how you can go
+
+51
+00:02:40,040 --> 00:02:43,670
+from this information that you have here, that we have derived by
+
+52
+00:02:43,670 --> 00:02:47,020
+applying the, the first steps of the method, to a set of
+
+53
+00:02:47,020 --> 00:02:50,650
+test frames, and then, a set of test packs. So to do
+
+54
+00:02:50,650 --> 00:02:53,240
+that we're going to use the tool that I just mentioned. So let
+
+55
+00:02:53,240 --> 00:02:56,670
+me bring back my terminal. So first of all, let's see how
+
+56
+00:02:56,670 --> 00:02:59,570
+we can run the tool. So you have a manual that will explain
+
+57
+00:02:59,570 --> 00:03:02,180
+all the details on how to build the file that we're
+
+58
+00:03:02,180 --> 00:03:04,350
+going to feed the tool. So what is the format and so
+
+59
+00:03:04,350 --> 00:03:07,780
+on. Here I'm just going to see how I can run the
+
+60
+00:03:07,780 --> 00:03:10,828
+tool. So first of all, let me point out that this was
+
+61
+00:03:10,828 --> 00:03:15,028
+developed together by professors from the University of California Irvine and
+
+62
+00:03:15,028 --> 00:03:18,020
+Oregon State University. And as you can see, we can run
+
+63
+00:03:18,020 --> 00:03:20,968
+TSL generator and specify that we want to see the main
+
+64
+00:03:20,968 --> 00:03:24,361
+page. So in this case if we run it this, this way,
+
+65
+00:03:24,361 --> 00:03:27,723
+you'll have some basic information on how to run
+
+66
+00:03:27,723 --> 00:03:30,720
+the tool. And from the main page you can see
+
+67
+00:03:30,720 --> 00:03:33,520
+that you can specify the minus c flag and in
+
+68
+00:03:33,520 --> 00:03:37,360
+this case the TSL generator will report the number of
+
+69
+00:03:37,360 --> 00:03:41,410
+test frames generated without writing them to output. For
+
+70
+00:03:41,410 --> 00:03:43,828
+example, you might want to use this as we will do
+
+71
+00:03:43,828 --> 00:03:46,308
+to see how many tests that you will generate with
+
+72
+00:03:46,308 --> 00:03:49,630
+a current set of category partitions and choices. The minus
+
+73
+00:03:49,630 --> 00:03:52,620
+s option will bring the result of the TSL
+
+74
+00:03:52,620 --> 00:03:55,620
+generator on the standard output. And finally, you can
+
+75
+00:03:55,620 --> 00:03:58,330
+use minus o to specify an output file, where
+
+76
+00:03:58,330 --> 00:04:01,010
+to put the output of the program. So let's
+
+77
+00:04:01,010 --> 00:04:05,070
+at first run our TSL generator by specifying the
+
+78
+00:04:05,070 --> 00:04:08,620
+minus c option and by bypassing our current set
+
+79
+00:04:08,620 --> 00:04:11,860
+of category partitions and choices. Okay, so let me
+
+80
+00:04:11,860 --> 00:04:15,140
+remind you that what the, the tool will do
+
+81
+00:04:15,140 --> 00:04:17,790
+is what we will do manually. Otherwise, which is
+
+82
+00:04:17,790 --> 00:04:20,380
+to combine all these choices so as to have one
+
+83
+00:04:20,380 --> 00:04:23,305
+test case for each combination. So if we do
+
+84
+00:04:23,305 --> 00:04:27,412
+that, you can see that the tool tells us that
+
+85
+00:04:27,412 --> 00:04:32,383
+we will generate 7776 test frames in this case.
+
+86
+00:04:32,383 --> 00:04:34,660
+And this seems to be a little too much for
+
+87
+00:04:34,660 --> 00:04:36,868
+a program as small as the one that we are
+
+88
+00:04:36,868 --> 00:04:40,341
+testing. And assume for instance that we don't have the
+
+89
+00:04:40,341 --> 00:04:43,149
+resources to run this many test cases for, for
+
+90
+00:04:43,149 --> 00:04:46,518
+the grep program. In addition, consider that in this case,
+
+91
+00:04:46,518 --> 00:04:50,356
+we're computing all possible combinations of choices. And there's going to
+
+92
+00:04:50,356 --> 00:04:52,384
+be some combination that do not make sense as we
+
+93
+00:04:52,384 --> 00:04:54,945
+discussed in the lesson. So what we might want to
+
+94
+00:04:54,945 --> 00:04:57,051
+do in this case is to go back to our
+
+95
+00:04:57,051 --> 00:05:03,120
+spec and start adding constraints to eliminate this meaningless combination.
+
+96
+00:05:03,120 --> 00:05:05,980
+So I'm going to show you the result of doing that.
+
+97
+00:05:05,980 --> 00:05:08,690
+And I'm going to show you a few examples.
+
+98
+00:05:08,690 --> 00:05:11,670
+For example here, when the file is empty, I'm going to
+
+99
+00:05:11,670 --> 00:05:15,010
+define this property empty file. And how am I
+
+100
+00:05:15,010 --> 00:05:18,490
+going to use this property? Well for example here, it
+
+101
+00:05:18,490 --> 00:05:20,760
+doesn't make sense to consider the case in which
+
+102
+00:05:20,760 --> 00:05:24,660
+we have one or many occurrences of the pattern in
+
+103
+00:05:24,660 --> 00:05:27,020
+the file if the file is empty. Therefore I'm
+
+104
+00:05:27,020 --> 00:05:31,650
+going to tell the tool that it should consider this specific
+
+105
+00:05:31,650 --> 00:05:35,780
+choice only if the file is not empty, only if
+
+106
+00:05:35,780 --> 00:05:38,660
+empty file is not defined. And that will skip, for
+
+107
+00:05:38,660 --> 00:05:41,330
+example, all of the combinations in which the file is
+
+108
+00:05:41,330 --> 00:05:44,171
+empty. And I'm trying to generate the test case that has
+
+109
+00:05:44,171 --> 00:05:46,364
+one occurrence of the pattern in the file, which is
+
+110
+00:05:46,364 --> 00:05:49,355
+simply not possible. For another example, in case I have
+
+111
+00:05:49,355 --> 00:05:52,824
+an empty pattern, I define the property empty pattern. And
+
+112
+00:05:52,824 --> 00:05:56,725
+then I avoid the choices that involve the pattern in case
+
+113
+00:05:56,725 --> 00:05:59,820
+the pattern is empty. because, for example, I cannot
+
+114
+00:05:59,820 --> 00:06:03,900
+have quotes in a pattern that is empty. For example,
+
+115
+00:06:03,900 --> 00:06:06,760
+it doesn't make sense to have blanks. So, one or
+
+116
+00:06:06,760 --> 00:06:10,250
+more blanks if the pattern is empty. So I'm going to
+
+117
+00:06:10,250 --> 00:06:14,180
+specify again that this choice should be considered only if
+
+118
+00:06:14,180 --> 00:06:16,140
+we don't have an empty pattern. And so on and
+
+119
+00:06:16,140 --> 00:06:20,080
+so forth. So now after I edit these constraints, I
+
+120
+00:06:20,080 --> 00:06:21,890
+can go back and compute again the number of test
+
+121
+00:06:21,890 --> 00:06:23,970
+frames and therefore the test cases that will be
+
+122
+00:06:23,970 --> 00:06:27,530
+generated with these constraints. So let me go again
+
+123
+00:06:27,530 --> 00:06:30,381
+to my terminal. Okay, so now I'm going to run
+
+124
+00:06:30,381 --> 00:06:34,061
+my TSL generator again, and I'm going to run it on
+
+125
+00:06:34,061 --> 00:06:37,389
+the second version of this file. And you can
+
+126
+00:06:37,389 --> 00:06:40,546
+see that I reduced the, the number of test frames
+
+127
+00:06:40,546 --> 00:06:43,889
+from about 7800 to about 1700. So it's quite
+
+128
+00:06:43,889 --> 00:06:46,967
+a, quite a big reduction by eliminating all these combinations
+
+129
+00:06:46,967 --> 00:06:49,540
+that do not make sense. But let's assume again that
+
+130
+00:06:49,540 --> 00:06:52,040
+we want to reduce this further so that we don't want to
+
+131
+00:06:52,040 --> 00:06:55,610
+generate those many test frames and therefore test cases. So
+
+132
+00:06:55,610 --> 00:06:58,660
+what can we do? We go back to our spec. And
+
+133
+00:06:58,660 --> 00:07:02,280
+in this case, we start adding error constraints. So if
+
+134
+00:07:02,280 --> 00:07:05,200
+you remember what we said in the lesson, error constraints are
+
+135
+00:07:05,200 --> 00:07:08,310
+constraints that indicate a choice that has to do with an
+
+136
+00:07:08,310 --> 00:07:11,980
+erroneous behaviour. For example, an erroneous input provided to the problem.
+
+137
+00:07:11,980 --> 00:07:15,210
+So here for instance, we're indicating the presence
+
+138
+00:07:15,210 --> 00:07:20,060
+of incorrectly enclosing quotes as an error choice. Same
+
+139
+00:07:20,060 --> 00:07:22,270
+thing if there's no file corresponding to the
+
+140
+00:07:22,270 --> 00:07:23,970
+name that we provide to the tool, we say
+
+141
+00:07:23,970 --> 00:07:26,760
+that this corresponds to an error. So how
+
+142
+00:07:26,760 --> 00:07:29,130
+is the tool going to use this information? It uses
+
+143
+00:07:29,130 --> 00:07:33,980
+this information by producing only one combination that involves
+
+144
+00:07:33,980 --> 00:07:37,270
+error choices, instead of combining them with other choices.
+
+145
+00:07:37,270 --> 00:07:39,780
+So let's see what happens after we added this
+
+146
+00:07:39,780 --> 00:07:43,370
+error constraints. So we go back to our console
+
+147
+00:07:43,370 --> 00:07:46,920
+once more. And in this case, we want to run
+
+148
+00:07:46,920 --> 00:07:50,910
+the TSL generator with the version of the, of my
+
+149
+00:07:50,910 --> 00:07:53,900
+file that contains the area of constraints. And again,
+
+150
+00:07:53,900 --> 00:07:56,390
+I reduce quite a bit the number of test frames.
+
+151
+00:07:56,390 --> 00:07:59,110
+So now I have only 562 test frames that
+
+152
+00:07:59,110 --> 00:08:02,660
+will be generated by using the file that I provided.
+
+153
+00:08:02,660 --> 00:08:05,460
+So for the last time, let's assume that we really want
+
+154
+00:08:05,460 --> 00:08:07,780
+to cut down the number of test frames or the number of
+
+155
+00:08:07,780 --> 00:08:10,380
+test cases. So once more, we go back to our file, and
+
+156
+00:08:10,380 --> 00:08:12,980
+at this point what we can add is the final type of
+
+157
+00:08:12,980 --> 00:08:14,170
+constraints that we have, which are
+
+158
+00:08:14,170 --> 00:08:17,245
+single constraints. And single constraints are
+
+159
+00:08:17,245 --> 00:08:21,360
+basically indicated choices that we don't want to combine with other choices.
+
+160
+00:08:21,360 --> 00:08:24,210
+So they have the same effect of the error constraints, but they
+
+161
+00:08:24,210 --> 00:08:28,120
+have a different meaning, so they do not indicate choices that corresponds
+
+162
+00:08:28,120 --> 00:08:29,860
+to an error. In other words, I can use a
+
+163
+00:08:29,860 --> 00:08:35,280
+single constraints to identify choices that I want to test only once.
+
+164
+00:08:35,280 --> 00:08:38,510
+So for example in this case, I might decide that I
+
+165
+00:08:38,510 --> 00:08:42,520
+want to have only one test frame that tests my program
+
+166
+00:08:42,520 --> 00:08:44,420
+with a file being empty and I can do the
+
+167
+00:08:44,420 --> 00:08:47,370
+same for other choices. So basically I can continue adding this
+
+168
+00:08:47,370 --> 00:08:50,400
+single constraint until I get down to the number of test
+
+169
+00:08:50,400 --> 00:08:53,410
+frames and therefore the number of test cases that I want.
+
+170
+00:08:53,410 --> 00:08:57,770
+So now let's go back once more to our console. And so now if we run
+
+171
+00:08:59,060 --> 00:09:04,450
+using this file as input, you can see that we have 35 test frames generated. So
+
+172
+00:09:04,450 --> 00:09:07,750
+this is a fairly low number of test cases, so we might decide that we want to
+
+173
+00:09:07,750 --> 00:09:13,380
+go ahead and write these test frames to a file. So now let's open this file
+
+174
+00:09:15,990 --> 00:09:25,500
+that we just generated. And as you can see here, I have exactly 35 test frames,
+
+175
+00:09:26,670 --> 00:09:30,900
+as expected. Some of those correspond to the single and error cases. So in this
+
+176
+00:09:30,900 --> 00:09:33,330
+case, the only choice that I have indicated
+
+177
+00:09:33,330 --> 00:09:35,690
+is the one that corresponds to the single
+
+178
+00:09:35,690 --> 00:09:38,310
+or error constraint. What is for the other
+
+179
+00:09:38,310 --> 00:09:42,170
+ones? I actually have the whole test spec.
+
+180
+00:09:42,170 --> 00:09:45,530
+So let's pick one just to give you an example.
+
+181
+00:09:45,530 --> 00:09:48,440
+In this case, that's frame number 15 that will correspond to
+
+182
+00:09:48,440 --> 00:09:51,910
+test case number 15. And here you can see that
+
+183
+00:09:51,910 --> 00:09:55,280
+we have all the information. So this is a test specification.
+
+184
+00:09:55,280 --> 00:09:57,560
+All the information that we need to generate the corresponding
+
+185
+00:09:57,560 --> 00:09:59,760
+test. We know that we need a file that is not
+
+186
+00:09:59,760 --> 00:10:03,810
+empty. That we need to have one occurrence of the pattern
+
+187
+00:10:03,810 --> 00:10:07,580
+in the file. One occurrence of the pattern in one line.
+
+188
+00:10:08,680 --> 00:10:10,360
+The position of the pattern in the file can
+
+189
+00:10:10,360 --> 00:10:13,740
+be any position. The length of the pattern must
+
+190
+00:10:13,740 --> 00:10:16,640
+be more than one character. The pattern should not
+
+191
+00:10:16,640 --> 00:10:20,140
+be enclosed in quotes. There should be one white
+
+192
+00:10:20,140 --> 00:10:24,460
+space, one quote within the pattern, and finally the
+
+193
+00:10:24,460 --> 00:10:27,230
+file that would pass through the program should exist.
+
+194
+00:10:27,230 --> 00:10:29,680
+So the file should be present. So I can
+
+195
+00:10:29,680 --> 00:10:33,950
+easily transform all of this into an actual test case.
+
+196
+00:10:33,950 --> 00:10:35,540
+And notice that even though we're not, we're not
+
+197
+00:10:35,540 --> 00:10:38,420
+going to do it here. In cases like this, it might
+
+198
+00:10:38,420 --> 00:10:42,190
+even be possible to automatically generate the test cases
+
+199
+00:10:42,190 --> 00:10:45,020
+from the test specifications because, here for example, here it
+
+200
+00:10:45,020 --> 00:10:48,150
+should be relatively straight forward to parse these test
+
+201
+00:10:48,150 --> 00:10:52,450
+specifications and generate test cases accordingly. So, just to summarize,
+
+202
+00:10:52,450 --> 00:10:55,910
+what we have done is to go from one high-level
+
+203
+00:10:55,910 --> 00:10:58,880
+description of a program to a set of categories, partitions,
+
+204
+00:10:58,880 --> 00:11:01,820
+and choices for that program. Then we have combined them
+
+205
+00:11:01,820 --> 00:11:04,930
+in different ways, adding more and more constraints to reduce the
+
+206
+00:11:04,930 --> 00:11:07,600
+number of combinations until we ended up with the right number
+
+207
+00:11:07,600 --> 00:11:09,650
+of test cases, so the number of test cases that we
+
+208
+00:11:09,650 --> 00:11:14,630
+were fine generating. We generated the corresponding test specifications. And at
+
+209
+00:11:14,630 --> 00:11:17,340
+that point, we could just go ahead, generate the test case,
+
+210
+00:11:17,340 --> 00:11:20,660
+and test our application. So, and you can see how this
+
+211
+00:11:20,660 --> 00:11:23,720
+can result in a much more thorough testing of your application.
+
+212
+00:11:23,720 --> 00:11:27,890
+Because instead of reading this description and just trying to come up with test
+
+213
+00:11:27,890 --> 00:11:33,900
+cases for it, we can break down the process in steps that are easy to perform
+
+214
+00:11:33,900 --> 00:11:36,960
+individually. They can be automated as much
+
+215
+00:11:36,960 --> 00:11:38,600
+as possible. And they will end up with
+
+216
+00:11:38,600 --> 00:11:40,380
+a set of test cases that will test
+
+217
+00:11:40,380 --> 00:11:42,790
+all the interests and aspects of your application.
-- 
cgit 1.4.1