diff options
| author | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-05-24 16:34:31 +0700 |
|---|---|---|
| committer | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-05-24 16:34:31 +0700 |
| commit | b2d80610db6beda38573890ed169815e495bc663 (patch) | |
| tree | 176e1bca6fe644c619d53cf1c24682c244b79cf6 /usth/ICT2.7/P4L2 Black-Box Testing Subtitles | |
| parent | 49376ab97c7427f1c1eca64072d1a934c2e52f50 (diff) | |
| download | cp-b2d80610db6beda38573890ed169815e495bc663.tar.gz | |
[usth/ICT2.7] Engineer software
Diffstat (limited to 'usth/ICT2.7/P4L2 Black-Box Testing Subtitles')
28 files changed, 3700 insertions, 0 deletions
diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt new file mode 100644 index 0000000..09a8f10 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt @@ -0,0 +1,47 @@ +1 +00:00:00,320 --> 00:00:02,620 +In the previous lesson, we discussed the fundamental + +2 +00:00:02,620 --> 00:00:06,380 +concepts behind software verification in general, and software testing + +3 +00:00:06,380 --> 00:00:09,470 +in particular. In this lesson, we will discuss + +4 +00:00:09,470 --> 00:00:12,310 +one of the two main testing approaches. Black box + +5 +00:00:12,310 --> 00:00:16,180 +testing, also called functional testing. We will cover + +6 +00:00:16,180 --> 00:00:19,180 +the main characteristic of black box testing, its pros + +7 +00:00:19,180 --> 00:00:22,260 +and cons, and discuss some commonly used black + +8 +00:00:22,260 --> 00:00:25,350 +box testing techniques. We will conclude the lesson with + +9 +00:00:25,350 --> 00:00:28,520 +a practical exercise in which we will apply a specific black + +10 +00:00:28,520 --> 00:00:32,520 +box testing technique to a real program. We will derive test + +11 +00:00:32,520 --> 00:00:34,980 +cases for the program and assess how the use of a + +12 +00:00:34,980 --> 00:00:38,690 +systematic approach, in contrast to a brute force approach, can help. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt new file mode 100644 index 0000000..418ed91 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt @@ -0,0 +1,79 @@ +1 +00:00:00,025 --> 00:00:02,900 +Okay, so now we're going to answer the question. So if we want + +2 +00:00:02,900 --> 00:00:06,520 +to consider all these inputs, and run them all on the software, + +3 +00:00:06,520 --> 00:00:09,490 +let's see how it will work. Let's assume that these are, 32 + +4 +00:00:09,490 --> 00:00:12,280 +bit integers. So at this point what we will have is, a + +5 +00:00:12,280 --> 00:00:15,130 +number of combination, which is 2 to the 32, times 2 to + +6 +00:00:15,130 --> 00:00:18,390 +the 32. They're two integers. This is equal to 2 to the + +7 +00:00:18,390 --> 00:00:21,910 +64, which in turn, is more or less equal, to 10 to + +8 +00:00:21,910 --> 00:00:25,110 +the 19. So 10 to the 19 is the number of tests that + +9 +00:00:25,110 --> 00:00:27,960 +we need to run to cover the whole domain. Now let's assume + +10 +00:00:27,960 --> 00:00:31,400 +that we can run one test per nanosecond. So what that means + +11 +00:00:31,400 --> 00:00:34,290 +is that we can run 10 to the 9 tests per second, + +12 +00:00:34,290 --> 00:00:37,240 +and that's a lot. If we do the math, that results in 10 + +13 +00:00:37,240 --> 00:00:40,750 +to the 10 seconds over all, because we have 10 to the + +14 +00:00:40,750 --> 00:00:43,760 +19 tests, we could run 10 to the 9 tests per second + +15 +00:00:43,760 --> 00:00:46,630 +so, we do the math, and we can run all these tests + +16 +00:00:46,630 --> 00:00:50,340 +in 10 to the 10 seconds. And what that corresponds to, it's about + +17 +00:00:50,340 --> 00:00:54,470 +600 years, so a lot of time. So even for such + +18 +00:00:54,470 --> 00:00:57,710 +a simple problem, a problem that takes two integers and adds them, + +19 +00:00:57,710 --> 00:01:00,990 +it will take more than 500 years to test it exhaustively. So + +20 +00:01:00,990 --> 00:01:04,690 +the bottom line here is that we just can't do exhaustive testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt new file mode 100644 index 0000000..a91af45 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt @@ -0,0 +1,143 @@ +1 +00:00:00,310 --> 00:00:02,250 +So then maybe what we can do is just to + +2 +00:00:02,250 --> 00:00:05,720 +pick our test inputs randomly so to do what is called + +3 +00:00:05,720 --> 00:00:08,850 +random testing. And what that means is that we pick the + +4 +00:00:08,850 --> 00:00:11,720 +inputs to test just as we pick a number by rolling + +5 +00:00:11,720 --> 00:00:15,410 +a set of dice randomly. And this will have several advantages. + +6 +00:00:15,410 --> 00:00:18,780 +First, we will pick inputs uniformly. So if we use a + +7 +00:00:18,780 --> 00:00:21,790 +uniform distribution as the basis for our random testing, we will + +8 +00:00:21,790 --> 00:00:25,540 +make no preferences. In other words, all inputs will be considered + +9 +00:00:25,540 --> 00:00:28,324 +equal, of equal value. And what that means in turn, is + +10 +00:00:28,324 --> 00:00:32,640 +that random testing eliminates designer bias. So what does designer bias + +11 +00:00:32,640 --> 00:00:36,030 +mean? Designer bias is the problem of making the same assumption, + +12 +00:00:36,030 --> 00:00:38,570 +when we read the specification and we interpret it and when we + +13 +00:00:38,570 --> 00:00:42,100 +develop test cases. Which means that the developer might develop code, + +14 +00:00:42,100 --> 00:00:44,930 +assuming a given behavior of the user. And we may write + +15 +00:00:44,930 --> 00:00:47,520 +tests, making the same assumptions. And the problem, of course, is + +16 +00:00:47,520 --> 00:00:50,690 +even worse if it's the same person that develops the code and + +17 +00:00:50,690 --> 00:00:53,730 +writes the test cases. With random testing, the problem is gone, + +18 +00:00:53,730 --> 00:00:57,440 +because we just pick randomly what our inputs will be. So + +19 +00:00:57,440 --> 00:01:00,180 +why not do in random? The problem is that when testing, + +20 +00:01:00,180 --> 00:01:03,610 +we are looking for a needle in a haystack. Actually, multiple + +21 +00:01:03,610 --> 00:01:06,620 +needles in multiple haystacks, if we want to be precise. So, + +22 +00:01:06,620 --> 00:01:09,500 +random approaches are not necessarily the best way to go about + +23 +00:01:09,500 --> 00:01:12,430 +it, because we might just be looking in all the wrong + +24 +00:01:12,430 --> 00:01:15,760 +places. So let me show you this, using a different representation + +25 +00:01:15,760 --> 00:01:18,000 +for the haystack. What I'm showing here is a grid, and + +26 +00:01:18,000 --> 00:01:22,130 +imagine this grid just expanding indefinitely outside the screen, and this grid + +27 +00:01:22,130 --> 00:01:26,120 +represents the domain for the program, so each box in the grid, + +28 +00:01:26,120 --> 00:01:29,050 +each square in the grid, it's a possible input. So what happens + +29 +00:01:29,050 --> 00:01:32,670 +with bugs is that bugs are very scarce in this grid. Maybe + +30 +00:01:32,670 --> 00:01:35,070 +there is a bug here, so that means that there is a + +31 +00:01:35,070 --> 00:01:38,090 +bug, than an input, in this point we'll reveal. And maybe there + +32 +00:01:38,090 --> 00:01:40,820 +is another bug that will be triggered by an input over here. + +33 +00:01:40,820 --> 00:01:44,570 +So imagine this spread out over this infinite grid. Its very unlikely + +34 +00:01:44,570 --> 00:01:47,440 +that just by picking randomly that we will be able to get to + +35 +00:01:47,440 --> 00:01:50,910 +these two points. Fortunately not all is lost, there is a silver lining. + +36 +00:01:50,910 --> 00:01:53,410 +So we need to look a little more in depth into this grid. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt new file mode 100644 index 0000000..b91462c --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt @@ -0,0 +1,67 @@ +1 +00:00:00,110 --> 00:00:02,570 +So let me use a slightly expanded version of this + +2 +00:00:02,570 --> 00:00:05,560 +grid. Although we're indeed looking at a needle in a haystack. + +3 +00:00:05,560 --> 00:00:09,560 +And failing inputs are generally sparse, very sparse, in the input + +4 +00:00:09,560 --> 00:00:12,890 +domain. However, they tend to be dense in some parts of + +5 +00:00:12,890 --> 00:00:15,860 +the domain. Like here or here. So how can we leverage + +6 +00:00:15,860 --> 00:00:18,920 +this? The fact that the failures are dense in some subdomains? + +7 +00:00:18,920 --> 00:00:22,290 +As it turns out, the domain is naturally split into partitions. + +8 +00:00:22,290 --> 00:00:25,340 +Where partitions are areas of the domain that are treated homogeneously + +9 +00:00:25,340 --> 00:00:28,070 +by the software. And this is what happens, that normally, + +10 +00:00:28,070 --> 00:00:31,020 +failures tend to be dense in this partitions. So the way + +11 +00:00:31,020 --> 00:00:34,000 +to leverage this characteristic of failures, is that we don't know + +12 +00:00:34,000 --> 00:00:36,950 +want to pick inputs randomly, in the input domain. Just here + +13 +00:00:36,950 --> 00:00:39,460 +and there. Rather we want to do two things. First we + +14 +00:00:39,460 --> 00:00:43,300 +want to identify partitions of our domain. And second we want + +15 +00:00:43,300 --> 00:00:46,950 +to select inputs from each partition. And by doing so, we + +16 +00:00:46,950 --> 00:00:50,300 +can dramatically increase our chances to reveal faults in the code. + +17 +00:00:50,300 --> 00:00:54,170 +So the name that is normally used for this process, is partition testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt new file mode 100644 index 0000000..6738346 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt @@ -0,0 +1,167 @@ +1 +00:00:00,120 --> 00:00:02,969 +So let's look at how this will work with an example. + +2 +00:00:02,969 --> 00:00:06,210 +I'm going to use this simple program that takes two inputs. The + +3 +00:00:06,210 --> 00:00:08,790 +first input is a string, str, and the second one is + +4 +00:00:08,790 --> 00:00:11,575 +an integer, size. And the problem is called split. And as + +5 +00:00:11,575 --> 00:00:14,363 +the name says what it does is to take this string, + +6 +00:00:14,363 --> 00:00:17,491 +str, and split it into sub string, into chunks of size + +7 +00:00:17,491 --> 00:00:21,550 +characters each. So how do we identify some possible partitions for + +8 +00:00:21,550 --> 00:00:25,620 +this program? If we consider the input size, we can identify + +9 +00:00:25,620 --> 00:00:29,630 +three neutral partitions which are size less than 0. For example, + +10 +00:00:29,630 --> 00:00:32,259 +we want to test how the program behaves. But if we pass an + +11 +00:00:32,259 --> 00:00:36,100 +incorrect size, size equal to 0, which is also a partition. In + +12 +00:00:36,100 --> 00:00:39,390 +this case, a partition with a single element. And the third case + +13 +00:00:39,390 --> 00:00:42,540 +is size greater than 0, which I will consider to be + +14 +00:00:42,540 --> 00:00:44,960 +kind of the standard case. And actually let me do a, you + +15 +00:00:44,960 --> 00:00:48,220 +know, slight aggression so when I was talking about designer bias. So + +16 +00:00:48,220 --> 00:00:50,630 +this is a case in which designer bias might not make you + +17 +00:00:50,630 --> 00:00:53,050 +think of using size less than 0 because you read the + +18 +00:00:53,050 --> 00:00:56,210 +spec. And you sort of assume that the size will be positive. + +19 +00:00:56,210 --> 00:00:58,556 +Whereas the right thing to do when we test is to consider + +20 +00:00:58,556 --> 00:01:01,700 +the complete domain rather than just parts of it. So now let's + +21 +00:01:01,700 --> 00:01:04,760 +look at string, str, and let's see what kind of sub + +22 +00:01:04,760 --> 00:01:06,538 +domains we could identify for this + +23 +00:01:06,538 --> 00:01:08,670 +parameter. And notice another important aspect + +24 +00:01:08,670 --> 00:01:12,290 +here is that we treat each different part of the input independently, + +25 +00:01:12,290 --> 00:01:15,760 +which also helps breaking down the problem. One interesting sub domain is + +26 +00:01:15,760 --> 00:01:18,980 +the domain that includes all the strings whose length is less than + +27 +00:01:18,980 --> 00:01:22,310 +size. So all the strings that will not be displayed. Another subdomain + +28 +00:01:22,310 --> 00:01:25,000 +is all the strings with length which is between the value of + +29 +00:01:25,000 --> 00:01:28,350 +size and twice the value of size. A third subdomain is the one + +30 +00:01:28,350 --> 00:01:31,820 +including all the strings whose length is greater than twice the value + +31 +00:01:31,820 --> 00:01:35,140 +of size. And we can continue and identify more and more subdomain. + +32 +00:01:35,140 --> 00:01:38,350 +The key thing here is that we have to do that based + +33 +00:01:38,350 --> 00:01:41,180 +on the domain. So we need to adapt what we just did here + +34 +00:01:41,180 --> 00:01:44,620 +based on, on the specific domain involved and on the type + +35 +00:01:44,620 --> 00:01:47,190 +of data in this domain. So at this point we said that + +36 +00:01:47,190 --> 00:01:49,630 +there were two steps. One was to identify the subdomains and + +37 +00:01:49,630 --> 00:01:52,990 +the second one was to pick values in this subdomain. The values + +38 +00:01:52,990 --> 00:01:55,320 +that we'll actually use for the testing. In this case, we + +39 +00:01:55,320 --> 00:01:58,218 +do not want to just pick any value. Rather we want to + +40 +00:01:58,218 --> 00:01:59,871 +pick values that are particularly + +41 +00:01:59,871 --> 00:02:02,710 +interesting, particularly representative. So what does + +42 +00:02:02,710 --> 00:02:05,800 +that mean? Well, we're going to do that based on an intuitive idea. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt new file mode 100644 index 0000000..3d54ad4 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt @@ -0,0 +1,55 @@ +1 +00:00:00,150 --> 00:00:02,590 +So let's go back again to our domain, with + +2 +00:00:02,590 --> 00:00:06,580 +all the sub-domains identified. And the basic idea, or the + +3 +00:00:06,580 --> 00:00:09,750 +intuitive idea I was talking about, is that errors tend + +4 +00:00:09,750 --> 00:00:12,425 +to occur at the boundary of a domain, or a + +5 +00:00:12,425 --> 00:00:15,510 +sub-domain. Like in this case. And why? Because these + +6 +00:00:15,510 --> 00:00:19,360 +are the cases that are less understood by the developers. + +7 +00:00:19,360 --> 00:00:22,155 +Like for example, the last iteration of a loop, or + +8 +00:00:22,155 --> 00:00:25,150 +a special value like zero for integers. So if this + +9 +00:00:25,150 --> 00:00:29,590 +is true, what we want to do is to select inputs at these + +10 +00:00:29,590 --> 00:00:32,189 +boundaries. And this is complementary to partition + +11 +00:00:32,189 --> 00:00:34,270 +testing, in the sense that partition testing + +12 +00:00:34,270 --> 00:00:38,160 +will identify the partitions in which we want to select inputs, and boundary + +13 +00:00:38,160 --> 00:00:40,570 +testing. So the selection of boundary values + +14 +00:00:40,570 --> 00:00:43,060 +will help select inputs in these partitions. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt new file mode 100644 index 0000000..b344802 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt @@ -0,0 +1,99 @@ +1 +00:00:00,200 --> 00:00:03,320 +So now let's go back to our split example. Let me + +2 +00:00:03,320 --> 00:00:05,790 +rearrange things a little bit to make more room. So now I'm + +3 +00:00:05,790 --> 00:00:09,160 +going to put the domains for size and for strength one next to + +4 +00:00:09,160 --> 00:00:12,210 +the other. So let's look at what some possible inputs will be + +5 +00:00:12,210 --> 00:00:15,140 +for the sub domains that we identified when we use the idea + +6 +00:00:15,140 --> 00:00:17,790 +of selecting input of the boundary. If we look at the first + +7 +00:00:17,790 --> 00:00:21,860 +subdomain, size less than zero, one reasonable input is, size equals to + +8 +00:00:21,860 --> 00:00:25,360 +minus 1, because minus 1 is the boundary value for the domain + +9 +00:00:25,360 --> 00:00:28,530 +of the integers less than zero. If we look at the third + +10 +00:00:28,530 --> 00:00:32,119 +subdomain, possibly interesting case is the one of size of equal to + +11 +00:00:32,119 --> 00:00:34,900 +1, for the same reasoning that we used for the previous subdomain, + +12 +00:00:34,900 --> 00:00:37,670 +for size less than zero. And, let's try to select another one + +13 +00:00:37,670 --> 00:00:40,870 +for this subdomain, for the integers greater than zero. If there is + +14 +00:00:40,870 --> 00:00:44,200 +a concept of maximal integer, we can select that one as our + +15 +00:00:44,200 --> 00:00:47,140 +boundary value. And of course we could select much more, but this + +16 +00:00:47,140 --> 00:00:50,630 +is just to give you an idea. Other possible inputs. One interesting + +17 +00:00:50,630 --> 00:00:53,310 +example for the first one, string with length less than + +18 +00:00:53,310 --> 00:00:57,690 +size will be a string with length size minus one. Again + +19 +00:00:57,690 --> 00:01:00,240 +this is the boundary value for this domain. And we could + +20 +00:01:00,240 --> 00:01:03,500 +continue in this way like for example selecting a string who's + +21 +00:01:03,500 --> 00:01:07,310 +length is exactly size as a boundary value for this + +22 +00:01:07,310 --> 00:01:10,490 +other domain. Instant one, and we look back actually to this + +23 +00:01:10,490 --> 00:01:12,940 +example and look at it in a more extensive way when + +24 +00:01:12,940 --> 00:01:15,690 +we actually talk about a specific method for doing this kind + +25 +00:01:15,690 --> 00:01:16,340 +of process. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt new file mode 100644 index 0000000..5164f1a --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt @@ -0,0 +1,199 @@ +1 +00:00:00,140 --> 00:00:02,790 +Now, let's go back to our systematic functional testing + +2 +00:00:02,790 --> 00:00:05,540 +approach and all the steps in this process. So + +3 +00:00:05,540 --> 00:00:07,680 +far we've seen the first step and the second + +4 +00:00:07,680 --> 00:00:10,340 +step. Now we're going to look at this step in which, + +5 +00:00:10,340 --> 00:00:13,290 +once we have identified the values of interest, we + +6 +00:00:13,290 --> 00:00:18,110 +derive test case specifications for these values, or using these + +7 +00:00:18,110 --> 00:00:21,300 +values. And the test case specification defines how the + +8 +00:00:21,300 --> 00:00:25,230 +values should be put together when actually testing the system. + +9 +00:00:25,230 --> 00:00:29,110 +And test case specification describe how these values should be put + +10 +00:00:29,110 --> 00:00:32,360 +together when testing the system. So let me go back one more + +11 +00:00:32,360 --> 00:00:34,780 +time to our split program, so that we can use the + +12 +00:00:34,780 --> 00:00:37,450 +information that we already computed. At this point what we have is + +13 +00:00:37,450 --> 00:00:41,670 +some possible inputs for "string," our first parameter, and for "size," + +14 +00:00:41,670 --> 00:00:44,410 +our second parameter. And we want to put them together, to generate + +15 +00:00:44,410 --> 00:00:47,080 +the description of what the test case should be. So let + +16 +00:00:47,080 --> 00:00:50,420 +me once more rearrange this a little bit. I first remove the + +17 +00:00:50,420 --> 00:00:53,360 +description of the subdomains, because we won't use them in this step. + +18 +00:00:53,360 --> 00:00:55,800 +And I moved out the set of all our possible inputs, that we're + +19 +00:00:55,800 --> 00:00:59,470 +going to combine to create the test case specification. And one possible way + +20 +00:00:59,470 --> 00:01:03,320 +of doing that is simply to combine the values for the first parameter, + +21 +00:01:03,320 --> 00:01:06,370 +and the values for the second parameter. So the Cartesian product. So + +22 +00:01:06,370 --> 00:01:09,060 +if we do that, what we will obtain is, for example, if we + +23 +00:01:09,060 --> 00:01:12,470 +consider the first possible input, size is equal to minus 1, we can + +24 +00:01:12,470 --> 00:01:15,510 +combine it with these two possible inputs for string, and we will get + +25 +00:01:15,510 --> 00:01:18,680 +size is equal to minus 1 string with length minus 2, or + +26 +00:01:18,680 --> 00:01:21,680 +size is equal to minus 1 string with length minus 1. And we'll + +27 +00:01:21,680 --> 00:01:24,200 +go back in a second to see what this means. Now if we + +28 +00:01:24,200 --> 00:01:27,510 +consider the second possible value for size, size is equal to one, we + +29 +00:01:27,510 --> 00:01:30,260 +also have two cases so the first one in this case that will + +30 +00:01:30,260 --> 00:01:34,030 +be considered a string with length zero. So the antistring. And we can + +31 +00:01:34,030 --> 00:01:37,410 +continue combining this value, but one thing I want to point out is + +32 +00:01:37,410 --> 00:01:40,570 +that if we just go in this straight forward and brute force sort + +33 +00:01:40,570 --> 00:01:43,390 +of way, we will obtain many combinations that don't make any sense, + +34 +00:01:43,390 --> 00:01:46,500 +like for example, this combination which doesn't make any sense because we can + +35 +00:01:46,500 --> 00:01:50,410 +not create the string with length minus 2. Similar for this combination, because + +36 +00:01:50,410 --> 00:01:53,190 +then by the same token, we cannot raise things with length minus 1. + +37 +00:01:53,190 --> 00:01:55,730 +And so there's a lot of cases that we will have to eliminate + +38 +00:01:55,730 --> 00:01:59,380 +afterwards. So what we're going to see in a few minutes is a possible + +39 +00:01:59,380 --> 00:02:02,970 +way in which we can avoid producing these meaningless cases. And at the + +40 +00:02:02,970 --> 00:02:06,380 +same time, keep under control, the number of test cases that we generate. + +41 +00:02:06,380 --> 00:02:09,070 +So lets go back for the last time to our steps + +42 +00:02:09,070 --> 00:02:11,980 +for systematic functional testing. What we just did was to derive + +43 +00:02:11,980 --> 00:02:15,040 +test case specification from a set of relevant inputs. The following + +44 +00:02:15,040 --> 00:02:18,420 +step is to use these test case specifications to generate actual test + +45 +00:02:18,420 --> 00:02:21,170 +cases. And this is normally a fairly mechanical step in the + +46 +00:02:21,170 --> 00:02:23,900 +sense that we just have to instantiate what is in the test + +47 +00:02:23,900 --> 00:02:27,970 +case specification as actual test cases. And it's really dependent on + +48 +00:02:27,970 --> 00:02:32,300 +the specific type of partitions and values identified on the specific context. + +49 +00:02:32,300 --> 00:02:35,100 +So instead of looking at that here in the, in the abstract, + +50 +00:02:35,100 --> 00:02:37,480 +I'm going to show you with an example later on, in the lesson. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt new file mode 100644 index 0000000..938e80c --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt @@ -0,0 +1,99 @@ +1 +00:00:00,200 --> 00:00:03,690 +What we will discuss next is a specific black-box testing + +2 +00:00:03,690 --> 00:00:07,240 +approach. So a specific instance of the general approach that + +3 +00:00:07,240 --> 00:00:10,814 +we just saw. And this approach is the category-partition method, + +4 +00:00:10,814 --> 00:00:13,418 +and was defined by Ostrand & Balcer in 1988 in + +5 +00:00:13,418 --> 00:00:16,760 +an article to the peer [UNKNOWN] communication of the ACM. + +6 +00:00:16,760 --> 00:00:19,490 +So this is a method for going from a specification, + +7 +00:00:19,490 --> 00:00:21,370 +a description of the system, to a set of test + +8 +00:00:21,370 --> 00:00:26,170 +cases like any other black-box testing approach by following six steps. + +9 +00:00:26,170 --> 00:00:28,370 +So let's look at what these steps are. The first + +10 +00:00:28,370 --> 00:00:31,850 +step is to identify independently testable features and this is + +11 +00:00:31,850 --> 00:00:33,870 +a step that we are familiar with because its exactly + +12 +00:00:33,870 --> 00:00:36,950 +the same step that we performed in the generic black box + +13 +00:00:36,950 --> 00:00:39,480 +testing approach that we just discussed. The second step is + +14 +00:00:39,480 --> 00:00:42,250 +to identify categories. Then the next step is to partition + +15 +00:00:42,250 --> 00:00:45,070 +categories into choices. Identify constraints + +16 +00:00:45,070 --> 00:00:47,530 +among choices. Produce and evaluate + +17 +00:00:47,530 --> 00:00:51,330 +test case specifications. And finally, the sixth step is to generate + +18 +00:00:51,330 --> 00:00:54,100 +test cases from test case specifications. So two of + +19 +00:00:54,100 --> 00:00:56,970 +the key elements in these six steps are the two + +20 +00:00:56,970 --> 00:00:59,430 +that give the name to the technique so the identification + +21 +00:00:59,430 --> 00:01:02,240 +of the categories and the partition of these categories into + +22 +00:01:02,240 --> 00:01:04,810 +choices. What we're going to do next is to go + +23 +00:01:04,810 --> 00:01:07,670 +and look at each one of the steps independently, except + +24 +00:01:07,670 --> 00:01:10,285 +for the first one. Because we're already familiar with that + +25 +00:01:10,285 --> 00:01:12,650 +step, and this method doesn't really add much to it. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt new file mode 100644 index 0000000..a1eb4dd --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt @@ -0,0 +1,127 @@ +1 +00:00:00,140 --> 00:00:02,980 +In the second step of the category partition technique, the goal is + +2 +00:00:02,980 --> 00:00:05,630 +to Identify Categories. Where categories are + +3 +00:00:05,630 --> 00:00:08,650 +characteristics of each input element. So + +4 +00:00:08,650 --> 00:00:11,490 +let me illustrate what that means using an example. And to do + +5 +00:00:11,490 --> 00:00:14,440 +that I'm going to use again the example of the split program, as + +6 +00:00:14,440 --> 00:00:16,830 +we are already familiar with it and we kind of already played + +7 +00:00:16,830 --> 00:00:20,110 +with it. When we were talking about the generic black box approach. + +8 +00:00:20,110 --> 00:00:22,293 +So let me bring back the program, and let me remind you + +9 +00:00:22,293 --> 00:00:25,440 +that what the program does is to take two inputs, a string and + +10 +00:00:25,440 --> 00:00:28,560 +the size, and it breaks down the string into chunks, + +11 +00:00:28,560 --> 00:00:31,300 +whose length is size. If we look at the split program + +12 +00:00:31,300 --> 00:00:34,330 +there are two input elements, str and size so we going to + +13 +00:00:34,330 --> 00:00:37,930 +identify categories for these two. So starting from str, what are + +14 +00:00:37,930 --> 00:00:41,228 +the interesting characteristics of the string? In, in this step + +15 +00:00:41,228 --> 00:00:44,948 +you're going to use your domain knowledge, your understanding of what a + +16 +00:00:44,948 --> 00:00:47,986 +string is, and for example we might identify the length of + +17 +00:00:47,986 --> 00:00:50,528 +the string and the content of the string as the two + +18 +00:00:50,528 --> 00:00:53,840 +main characteristics that we want to focus on. If we now + +19 +00:00:53,840 --> 00:00:57,540 +move our focus to size, the only characteristic I can really + +20 +00:00:57,540 --> 00:01:00,340 +think of for an integer is its value. So that's what + +21 +00:01:00,340 --> 00:01:02,470 +I'm going to mark here. So at the end of the step + +22 +00:01:02,470 --> 00:01:05,030 +what we have is that we have two categories. So two + +23 +00:01:05,030 --> 00:01:08,930 +interesting characteristics for the string input str, which are the length + +24 +00:01:08,930 --> 00:01:12,940 +and the content. And one category for the integer input size + +25 +00:01:12,940 --> 00:01:16,480 +which is its value. And notice that there's not only one solution. + +26 +00:01:16,480 --> 00:01:18,200 +So there's not only one possibility. + +27 +00:01:18,200 --> 00:01:20,384 +So that the specific characteristics that you + +28 +00:01:20,384 --> 00:01:22,800 +will identify are somehow subjective. But the + +29 +00:01:22,800 --> 00:01:25,630 +important point is that you identify characteristics + +30 +00:01:25,630 --> 00:01:29,259 +that are meaningful and they sort of cover the main aspects of the + +31 +00:01:29,259 --> 00:01:31,200 +inputs, which is the case for the + +32 +00:01:31,200 --> 00:01:33,440 +categories that we've identified in this example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt new file mode 100644 index 0000000..a411c30 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt @@ -0,0 +1,151 @@ +1 +00:00:00,230 --> 00:00:03,250 +Now we move to the next step, which involves partitioning the + +2 +00:00:03,250 --> 00:00:07,680 +categories that we just identified into choices. And these choices are the + +3 +00:00:07,680 --> 00:00:11,930 +interesting cases for each category. So the interesting subdomains for each one + +4 +00:00:11,930 --> 00:00:14,810 +of these categories. So once more, lets look at that using our + +5 +00:00:14,810 --> 00:00:18,110 +example, the split program. So lets start by considering length. What are + +6 +00:00:18,110 --> 00:00:20,710 +the interesting cases when we think about the length of a string? + +7 +00:00:20,710 --> 00:00:23,220 +Some of those we already saw, one interesting case is the case + +8 +00:00:23,220 --> 00:00:26,190 +of the length of size zero, so a string with no characters. + +9 +00:00:26,190 --> 00:00:29,030 +Another interesting case is the one in which the length of the + +10 +00:00:29,030 --> 00:00:31,780 +string is size minus one, so the string is just one character + +11 +00:00:31,780 --> 00:00:34,610 +short of the size at which it will be cut by the + +12 +00:00:34,610 --> 00:00:38,160 +split program. And we can continue along these lines, so we will select + +13 +00:00:38,160 --> 00:00:42,950 +size, size plus one, size twice the value of size minus one, + +14 +00:00:42,950 --> 00:00:45,560 +and so on and so forth. But even without listing all of + +15 +00:00:45,560 --> 00:00:47,500 +those, I'm sure you get the idea of what it means to + +16 +00:00:47,500 --> 00:00:49,780 +identify this interesting cases. Let's see + +17 +00:00:49,780 --> 00:00:51,702 +the movements that are considering the content. + +18 +00:00:51,702 --> 00:00:53,990 +So without the interesting cases when we think about the content + +19 +00:00:53,990 --> 00:00:56,520 +of the string. So possible interesting case is the string that + +20 +00:00:56,520 --> 00:01:00,660 +contains only spaces. Why? Well maybe because a split is written + +21 +00:01:00,660 --> 00:01:04,290 +spaces in a special way. Similarly a string that contains special + +22 +00:01:04,290 --> 00:01:06,780 +characters, like non printable characters, + +23 +00:01:06,780 --> 00:01:09,020 +like tabulation characters, new line might + +24 +00:01:09,020 --> 00:01:12,240 +also be an interesting case, something that we want to test. Also + +25 +00:01:12,240 --> 00:01:14,280 +in this case we could continue and go on and on. + +26 +00:01:14,280 --> 00:01:17,250 +So basically here you just want to put all the interesting cases + +27 +00:01:17,250 --> 00:01:20,010 +that you can think of when you consider the content + +28 +00:01:20,010 --> 00:01:22,440 +of a string. Now let's move to the value as the + +29 +00:01:22,440 --> 00:01:25,740 +next category. So the value of the input size. And + +30 +00:01:25,740 --> 00:01:29,280 +here we might want to consider a size zero, special case, a + +31 +00:01:29,280 --> 00:01:33,640 +normal situation, like size greater than zero, another special case, + +32 +00:01:33,640 --> 00:01:36,420 +size less than zero or maxint. And these are, if you + +33 +00:01:36,420 --> 00:01:39,420 +remember, I accepted the cases that we consider when we look + +34 +00:01:39,420 --> 00:01:42,350 +at this example, before. And also here we can continue and + +35 +00:01:42,350 --> 00:01:43,970 +go on and on. So, at the end of the + +36 +00:01:43,970 --> 00:01:46,860 +step, what we have is a set of interesting cases + +37 +00:01:46,860 --> 00:01:49,220 +for each one of the categories, and now we can + +38 +00:01:49,220 --> 00:01:51,150 +start to think about how we want to combine them. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt new file mode 100644 index 0000000..2af56c5 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt @@ -0,0 +1,103 @@ +1 +00:00:00,450 --> 00:00:03,400 +As we said at the end of the previous lesson, black-box testing is + +2 +00:00:03,400 --> 00:00:06,230 +the testing of the software when we look at it as a black box, + +3 +00:00:06,230 --> 00:00:09,110 +as a closed box, without looking at it inside, without looking at the + +4 +00:00:09,110 --> 00:00:10,750 +code. And there are several advantages in + +5 +00:00:10,750 --> 00:00:12,680 +using black-box testing. So let me recap + +6 +00:00:12,680 --> 00:00:15,590 +those advantages, some of which we already mentioned, and let me also expand + +7 +00:00:15,590 --> 00:00:18,220 +on that a little bit. The first advantage of when I mentioned is that + +8 +00:00:18,220 --> 00:00:22,760 +black box focuses on the domain, on the input domain of the software. And + +9 +00:00:22,760 --> 00:00:25,750 +as such, we can use it to make sure that we are actually covering + +10 +00:00:25,750 --> 00:00:28,400 +this domain, that we are actually covering the important behaviors of + +11 +00:00:28,400 --> 00:00:32,200 +the software. A second advantage is that black box testing does not + +12 +00:00:32,200 --> 00:00:35,402 +need the code. What that means is that you can perform early + +13 +00:00:35,402 --> 00:00:38,801 +test design. So you can start designing and writing your test cases, + +14 +00:00:38,801 --> 00:00:41,423 +even before writing your code, so that when the code is + +15 +00:00:41,423 --> 00:00:44,520 +ready, we can test it right away. And that helps prevent a + +16 +00:00:44,520 --> 00:00:48,054 +problem that is very typical in real life software development, which is + +17 +00:00:48,054 --> 00:00:50,790 +getting an idea of the project and having no time to create + +18 +00:00:50,790 --> 00:00:53,040 +the tests. In this way, we already have the tests, so + +19 +00:00:53,040 --> 00:00:56,170 +we just have to run them. Another advantage is that black-box testing + +20 +00:00:56,170 --> 00:00:59,530 +can catch logic defects, because it focuses on the description of what + +21 +00:00:59,530 --> 00:01:02,440 +the software should do, and therefore on its logic. If we derive + +22 +00:01:02,440 --> 00:01:05,250 +test cases from such description, then we can catch these kind of + +23 +00:01:05,250 --> 00:01:07,510 +problems. And finally, black-box testing is + +24 +00:01:07,510 --> 00:01:09,900 +applicable at all granularity levels, which + +25 +00:01:09,900 --> 00:01:13,490 +means that we can use black-box testing in unit testing, integration testing, + +26 +00:01:13,490 --> 00:01:16,290 +system testing, and so on. We can use it at all levels. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt new file mode 100644 index 0000000..0278cc8 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt @@ -0,0 +1,239 @@ +1 +00:00:00,160 --> 00:00:02,080 +Something that we saw when we were looking at the split + +2 +00:00:02,080 --> 00:00:05,760 +program before is that if we just combine the interesting values that + +3 +00:00:05,760 --> 00:00:08,490 +we identify, we might end up with a lot of cases. + +4 +00:00:08,490 --> 00:00:11,030 +And I mentioned that we, we're going to look at some way of + +5 +00:00:11,030 --> 00:00:13,720 +addressing that problem. And this is exactly what happens in the + +6 +00:00:13,720 --> 00:00:17,420 +next step of the category partition method, in which we identify constraints + +7 +00:00:17,420 --> 00:00:20,510 +among choices. And why do we identify these constraints? We do + +8 +00:00:20,510 --> 00:00:23,440 +that to eliminate meaningless combinations of + +9 +00:00:23,440 --> 00:00:25,460 +inputs. If you remember, for example, + +10 +00:00:25,460 --> 00:00:27,260 +we had the case in which we were trying to + +11 +00:00:27,260 --> 00:00:30,110 +create a string with a size less than 0, which + +12 +00:00:30,110 --> 00:00:32,930 +doesn't make any sense. And very importantly, we also do + +13 +00:00:32,930 --> 00:00:37,070 +that to reduce the number of test cases. Because every time + +14 +00:00:37,070 --> 00:00:40,930 +we constrain one of the possible choices, we eliminate possible + +15 +00:00:40,930 --> 00:00:43,250 +test cases, so we can use it to keep under + +16 +00:00:43,250 --> 00:00:45,960 +control the number of tests that we generate. There are + +17 +00:00:45,960 --> 00:00:47,571 +three types of properties. The + +18 +00:00:47,571 --> 00:00:50,610 +pair property...if, error properties, and properties + +19 +00:00:50,610 --> 00:00:52,610 +of type single. So we're going to look at what these + +20 +00:00:52,610 --> 00:00:56,230 +properties mean, using, once more, our example of the split program. + +21 +00:00:56,230 --> 00:00:58,750 +In particular, we're going to use some of the choices that we + +22 +00:00:58,750 --> 00:01:02,080 +identified earlier. So let's look, for example, at choice 0, for + +23 +00:01:02,080 --> 00:01:04,599 +category length of the string. All we can say is that, + +24 +00:01:04,599 --> 00:01:07,860 +if the length is 0, this define a special property of + +25 +00:01:07,860 --> 00:01:11,160 +the string. And that was specified in this way by saying + +26 +00:01:11,160 --> 00:01:15,610 +that this identifies property zerovalue. So every time that we use + +27 +00:01:15,610 --> 00:01:19,080 +this choice, zerovalue is defined. At this point, we can use + +28 +00:01:19,080 --> 00:01:20,550 +this to exclude some meaningless + +29 +00:01:20,550 --> 00:01:23,170 +combinations. For instance, consider special characters. + +30 +00:01:23,170 --> 00:01:25,130 +If we have a string of length 0, which means a + +31 +00:01:25,130 --> 00:01:26,850 +string with no characters. Obviously, there + +32 +00:01:26,850 --> 00:01:28,760 +cannot be special characters. So, considering + +33 +00:01:28,760 --> 00:01:31,020 +this combination will just be a waste of time. So what + +34 +00:01:31,020 --> 00:01:33,770 +we do is that we specify next to this choice, that we + +35 +00:01:33,770 --> 00:01:36,360 +will only consider this if length is not 0. And we + +36 +00:01:36,360 --> 00:01:41,110 +do this by saying that we consider this only if not zerovalue. + +37 +00:01:41,110 --> 00:01:43,970 +So, if zerovalue is not defined. So this pair is an + +38 +00:01:43,970 --> 00:01:47,640 +example of a property...if case. Define a property and use that + +39 +00:01:47,640 --> 00:01:49,770 +property. Now let's look at a case in which we might + +40 +00:01:49,770 --> 00:01:52,580 +want to use an error property. For instance, when we look at + +41 +00:01:52,580 --> 00:01:56,480 +the category value for the input size, the choice value less + +42 +00:01:56,480 --> 00:01:59,510 +than 0 is an erroneous choice. So it's a choice that + +43 +00:01:59,510 --> 00:02:02,630 +we selected to test a possibly erroneous situation, so we can + +44 +00:02:02,630 --> 00:02:06,160 +mark this as an error property. And what that means is that + +45 +00:02:06,160 --> 00:02:09,949 +when generating a combination of choices, we will consider this only + +46 +00:02:09,949 --> 00:02:12,990 +once because we assume that we just want to test this error condition + +47 +00:02:12,990 --> 00:02:16,260 +once. Finally, the single property is a property that we use + +48 +00:02:16,260 --> 00:02:19,050 +when we want to limit the number of test cases. And it's + +49 +00:02:19,050 --> 00:02:21,900 +similar as an effect to error. It just has a different + +50 +00:02:21,900 --> 00:02:24,410 +meaning. So what we do when we use the single property is + +51 +00:02:24,410 --> 00:02:27,420 +that we're saying that this choice, we want to use in only + +52 +00:02:27,420 --> 00:02:31,120 +one combination. So don't combine it multiple times. And that, of course, + +53 +00:02:31,120 --> 00:02:34,260 +given the combinatorial nature of the problem, cuts down dramatically + +54 +00:02:34,260 --> 00:02:36,370 +the number of test cases. And we might use, for + +55 +00:02:36,370 --> 00:02:39,500 +instance, the single property for maxint, which means that we + +56 +00:02:39,500 --> 00:02:41,890 +will only have one test case in which the size is + +57 +00:02:41,890 --> 00:02:44,300 +equal to maxint. We're actually going to see a demo + +58 +00:02:44,300 --> 00:02:46,880 +on this topic so we'll have more chances of seeing how + +59 +00:02:46,880 --> 00:02:49,370 +properties work in practice and how good they are at + +60 +00:02:49,370 --> 00:02:53,022 +eliminating meaningless combinations and at reducing the number of test cases. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt new file mode 100644 index 0000000..ebf8098 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt @@ -0,0 +1,111 @@ +1 +00:00:00,130 --> 00:00:03,200 +Before getting to our demo, we still have two steps to consider. + +2 +00:00:03,200 --> 00:00:06,570 +The first step corresponds to the identification of the test case specifications + +3 +00:00:06,570 --> 00:00:09,890 +in our general systematic approach. And in fact, it's called produce and + +4 +00:00:09,890 --> 00:00:13,410 +evaluate test case specifications. This is a step than can be completely + +5 +00:00:13,410 --> 00:00:16,590 +automated given the results of the previous steps. And the final result + +6 +00:00:16,590 --> 00:00:19,860 +of this step is the production of a set of test frames. + +7 +00:00:19,860 --> 00:00:22,820 +Where a test frame is the specification of a test. Let me + +8 +00:00:22,820 --> 00:00:25,330 +show you an example of this. What we are looking at here + +9 +00:00:25,330 --> 00:00:28,380 +is a test frame for the program split. Test frames are normally + +10 +00:00:28,380 --> 00:00:31,460 +identified by a sequence number. But in this case we are looking at + +11 +00:00:31,460 --> 00:00:34,290 +the 30th six test frame. And what they do is simply to + +12 +00:00:34,290 --> 00:00:38,280 +specify the characteristic of the inputs for that test. In this case, since + +13 +00:00:38,280 --> 00:00:40,920 +we have two inputs, we have two entries, the first one for + +14 +00:00:40,920 --> 00:00:44,010 +string str tells us that the length of the string has to be + +15 +00:00:44,010 --> 00:00:47,840 +size minus 1, and that the string has to contain special characters. And + +16 +00:00:47,840 --> 00:00:50,380 +for size, it tells us that the value of size has to be + +17 +00:00:50,380 --> 00:00:53,340 +greater than zero. As the title says, this step is meant + +18 +00:00:53,340 --> 00:00:56,790 +to produce but also evaluate the case specification. What does it mean + +19 +00:00:56,790 --> 00:01:00,070 +to evaluate? One of the advantages of this approach is that we + +20 +00:01:00,070 --> 00:01:03,420 +can easily use it to assess how many test frames and therefore + +21 +00:01:03,420 --> 00:01:06,120 +how many test cases we will generate with the current least + +22 +00:01:06,120 --> 00:01:09,340 +of categories, choices and constraints. And the beauty of this is that + +23 +00:01:09,340 --> 00:01:12,320 +if the number is too large we can just add additional constraints + +24 +00:01:12,320 --> 00:01:15,840 +and reduce it. And given then the step is automated we just + +25 +00:01:15,840 --> 00:01:19,030 +add constraints push a button and we get our new set of + +26 +00:01:19,030 --> 00:01:21,580 +test frames. And again we can have a wait it either go + +27 +00:01:21,580 --> 00:01:24,560 +hat or add more constraints if we need to further reduce it + +28 +00:01:24,560 --> 00:01:26,320 +and this is something else that we will see in our demo. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt new file mode 100644 index 0000000..7d74ae3 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt @@ -0,0 +1,91 @@ +1 +00:00:00,135 --> 00:00:01,935 +So we get to the last step of the technique in + +2 +00:00:01,935 --> 00:00:05,142 +which once we have generated test case specifications. We create test + +3 +00:00:05,142 --> 00:00:09,180 +cases starting from this specifications. This step mainly consists in a + +4 +00:00:09,180 --> 00:00:12,840 +simple instantiation of frames and it's final result is a set of + +5 +00:00:12,840 --> 00:00:16,840 +concrete tests. For our example, test frame number 36 that we + +6 +00:00:16,840 --> 00:00:19,270 +just saw, this will be the resulting test case, which has + +7 +00:00:19,270 --> 00:00:21,490 +the same ID, so that we can track it and will + +8 +00:00:21,490 --> 00:00:25,520 +specify to concrete values, not just the specification for the input elements. + +9 +00:00:25,520 --> 00:00:29,130 +So string STR will have this value. And the integer size + +10 +00:00:29,130 --> 00:00:31,720 +will have this value. And these two values satisfy what this + +11 +00:00:31,720 --> 00:00:35,260 +test case specification was. Which was, having a string contain special + +12 +00:00:35,260 --> 00:00:38,760 +characters. Here, we have two special characters, like the new line + +13 +00:00:38,760 --> 00:00:41,900 +and the tab. And, we have a size which is greater + +14 +00:00:41,900 --> 00:00:44,430 +than zero, in particular, okay? And this is a test case + +15 +00:00:44,430 --> 00:00:46,880 +that we can actually run on our code. That we can + +16 +00:00:46,880 --> 00:00:50,730 +run on the split program. So, to summarize, we perform six steps + +17 +00:00:50,730 --> 00:00:54,400 +in which we went from a high level description of what the program does, to a + +18 +00:00:54,400 --> 00:00:56,070 +set of concrete test cases. And this is + +19 +00:00:56,070 --> 00:00:57,880 +one of those test cases. So what, what we're + +20 +00:00:57,880 --> 00:01:00,210 +going to do next, we're going to do a mini-demo, + +21 +00:01:00,210 --> 00:01:02,260 +in which we do this for real. We take + +22 +00:01:02,260 --> 00:01:05,440 +the program, we identify categories, choices, constraints, and + +23 +00:01:05,440 --> 00:01:07,800 +we actually generate test frames and then test cases. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt new file mode 100644 index 0000000..51c2e52 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt @@ -0,0 +1,867 @@ +1 +00:00:01,290 --> 00:00:03,210 +In this demo, we're going to do exactly what we + +2 +00:00:03,210 --> 00:00:06,370 +did just now in the lesson. We're going to use + +3 +00:00:06,370 --> 00:00:09,250 +the category partition method to go from a high-level + +4 +00:00:09,250 --> 00:00:12,930 +description of a piece of software of a program to + +5 +00:00:12,930 --> 00:00:15,512 +a set of test cases for that program. To + +6 +00:00:15,512 --> 00:00:17,640 +do that, we're going to use a simple tool. So + +7 +00:00:17,640 --> 00:00:21,040 +I'm going to show you here the tool that is + +8 +00:00:21,040 --> 00:00:26,380 +called a tsl generator right here. This tool is available + +9 +00:00:26,380 --> 00:00:28,520 +to you, so you can look in the class notes to + +10 +00:00:28,520 --> 00:00:31,900 +see information on how to download it. And together with the tool, + +11 +00:00:31,900 --> 00:00:34,890 +we are also going to provide a manual for the tool, and + +12 +00:00:34,890 --> 00:00:37,680 +a set of files that I'm going to use in this demo. So + +13 +00:00:37,680 --> 00:00:40,420 +you should be able to do exactly what I'm doing. So + +14 +00:00:40,420 --> 00:00:45,110 +again, all of those are available from the class notes. So specifically, + +15 +00:00:45,110 --> 00:00:48,390 +today we're going to write test cases for the grep program. So in + +16 +00:00:48,390 --> 00:00:51,970 +case you're familiar with the grep utility, this is a simplified version + +17 +00:00:51,970 --> 00:00:55,060 +of that utility. So basically the grep utility allows you + +18 +00:00:55,060 --> 00:00:58,410 +to search a file for the occurrences of a given + +19 +00:00:58,410 --> 00:01:01,552 +pattern. So you can invoke it, as it's shown here + +20 +00:01:01,552 --> 00:01:05,570 +in the synopsis, by executing grep, the pattern that you're + +21 +00:01:05,570 --> 00:01:08,310 +looking for, and the filename in which you want to + +22 +00:01:08,310 --> 00:01:10,300 +look for the pattern. And let me read the description + +23 +00:01:10,300 --> 00:01:13,650 +of the grep utility. The grep utility searches files for + +24 +00:01:13,650 --> 00:01:17,110 +a pattern and brings all lines that contain that pattern + +25 +00:01:17,110 --> 00:01:20,900 +on the standard output. A line that contains multiple occurrences of + +26 +00:01:20,900 --> 00:01:24,350 +the pattern is printed only once. The pattern is any sequence + +27 +00:01:24,350 --> 00:01:27,700 +of characters. To include a blank in the pattern, the entire + +28 +00:01:27,700 --> 00:01:31,060 +pattern must be enclosed in single quotes. To include a quote + +29 +00:01:31,060 --> 00:01:34,420 +sign in the pattern, the quote sign must be escaped, which + +30 +00:01:34,420 --> 00:01:36,260 +means that we have to put a slash in front of + +31 +00:01:36,260 --> 00:01:39,290 +the quotes sign. And in general, it is safest to enclose + +32 +00:01:39,290 --> 00:01:42,210 +the entire pattern in single quotes. So this is our high + +33 +00:01:42,210 --> 00:01:45,420 +level description for the program, for the softer system, that + +34 +00:01:45,420 --> 00:01:47,270 +we need to test. So now let me show you + +35 +00:01:47,270 --> 00:01:50,600 +what a possible set of categories and partitions could be + +36 +00:01:50,600 --> 00:01:53,770 +for this program. So what I have here is a + +37 +00:01:53,770 --> 00:01:58,080 +file, a textual file, which contains all the categories and + +38 +00:01:58,080 --> 00:02:02,760 +partitions for the elements that are relevant for my program. + +39 +00:02:02,760 --> 00:02:05,240 +In particular, when we look at the file, we can + +40 +00:02:05,240 --> 00:02:07,809 +see that the file can be characterized by its size. + +41 +00:02:08,889 --> 00:02:12,160 +And in this case, I've got two choices. The file can + +42 +00:02:12,160 --> 00:02:16,050 +be empty or not empty. The second characteristic of the file + +43 +00:02:16,050 --> 00:02:19,490 +that I'm considering is the number of occurrences of the pattern + +44 +00:02:19,490 --> 00:02:22,320 +in the file. And I'm considering that the pattern might not occur + +45 +00:02:22,320 --> 00:02:25,780 +in the file or it might occur once, or multiple times. + +46 +00:02:25,780 --> 00:02:28,264 +I'm not going to go through the rest of the file because we + +47 +00:02:28,264 --> 00:02:31,234 +already covered how to apply the category partition method in the + +48 +00:02:31,234 --> 00:02:34,226 +lesson. So if you had doubts about that, about the method and + +49 +00:02:34,226 --> 00:02:36,952 +how to apply, you might want to go back and watch again the + +50 +00:02:36,952 --> 00:02:40,040 +lesson. What I want to show you here is how you can go + +51 +00:02:40,040 --> 00:02:43,670 +from this information that you have here, that we have derived by + +52 +00:02:43,670 --> 00:02:47,020 +applying the, the first steps of the method, to a set of + +53 +00:02:47,020 --> 00:02:50,650 +test frames, and then, a set of test packs. So to do + +54 +00:02:50,650 --> 00:02:53,240 +that we're going to use the tool that I just mentioned. So let + +55 +00:02:53,240 --> 00:02:56,670 +me bring back my terminal. So first of all, let's see how + +56 +00:02:56,670 --> 00:02:59,570 +we can run the tool. So you have a manual that will explain + +57 +00:02:59,570 --> 00:03:02,180 +all the details on how to build the file that we're + +58 +00:03:02,180 --> 00:03:04,350 +going to feed the tool. So what is the format and so + +59 +00:03:04,350 --> 00:03:07,780 +on. Here I'm just going to see how I can run the + +60 +00:03:07,780 --> 00:03:10,828 +tool. So first of all, let me point out that this was + +61 +00:03:10,828 --> 00:03:15,028 +developed together by professors from the University of California Irvine and + +62 +00:03:15,028 --> 00:03:18,020 +Oregon State University. And as you can see, we can run + +63 +00:03:18,020 --> 00:03:20,968 +TSL generator and specify that we want to see the main + +64 +00:03:20,968 --> 00:03:24,361 +page. So in this case if we run it this, this way, + +65 +00:03:24,361 --> 00:03:27,723 +you'll have some basic information on how to run + +66 +00:03:27,723 --> 00:03:30,720 +the tool. And from the main page you can see + +67 +00:03:30,720 --> 00:03:33,520 +that you can specify the minus c flag and in + +68 +00:03:33,520 --> 00:03:37,360 +this case the TSL generator will report the number of + +69 +00:03:37,360 --> 00:03:41,410 +test frames generated without writing them to output. For + +70 +00:03:41,410 --> 00:03:43,828 +example, you might want to use this as we will do + +71 +00:03:43,828 --> 00:03:46,308 +to see how many tests that you will generate with + +72 +00:03:46,308 --> 00:03:49,630 +a current set of category partitions and choices. The minus + +73 +00:03:49,630 --> 00:03:52,620 +s option will bring the result of the TSL + +74 +00:03:52,620 --> 00:03:55,620 +generator on the standard output. And finally, you can + +75 +00:03:55,620 --> 00:03:58,330 +use minus o to specify an output file, where + +76 +00:03:58,330 --> 00:04:01,010 +to put the output of the program. So let's + +77 +00:04:01,010 --> 00:04:05,070 +at first run our TSL generator by specifying the + +78 +00:04:05,070 --> 00:04:08,620 +minus c option and by bypassing our current set + +79 +00:04:08,620 --> 00:04:11,860 +of category partitions and choices. Okay, so let me + +80 +00:04:11,860 --> 00:04:15,140 +remind you that what the, the tool will do + +81 +00:04:15,140 --> 00:04:17,790 +is what we will do manually. Otherwise, which is + +82 +00:04:17,790 --> 00:04:20,380 +to combine all these choices so as to have one + +83 +00:04:20,380 --> 00:04:23,305 +test case for each combination. So if we do + +84 +00:04:23,305 --> 00:04:27,412 +that, you can see that the tool tells us that + +85 +00:04:27,412 --> 00:04:32,383 +we will generate 7776 test frames in this case. + +86 +00:04:32,383 --> 00:04:34,660 +And this seems to be a little too much for + +87 +00:04:34,660 --> 00:04:36,868 +a program as small as the one that we are + +88 +00:04:36,868 --> 00:04:40,341 +testing. And assume for instance that we don't have the + +89 +00:04:40,341 --> 00:04:43,149 +resources to run this many test cases for, for + +90 +00:04:43,149 --> 00:04:46,518 +the grep program. In addition, consider that in this case, + +91 +00:04:46,518 --> 00:04:50,356 +we're computing all possible combinations of choices. And there's going to + +92 +00:04:50,356 --> 00:04:52,384 +be some combination that do not make sense as we + +93 +00:04:52,384 --> 00:04:54,945 +discussed in the lesson. So what we might want to + +94 +00:04:54,945 --> 00:04:57,051 +do in this case is to go back to our + +95 +00:04:57,051 --> 00:05:03,120 +spec and start adding constraints to eliminate this meaningless combination. + +96 +00:05:03,120 --> 00:05:05,980 +So I'm going to show you the result of doing that. + +97 +00:05:05,980 --> 00:05:08,690 +And I'm going to show you a few examples. + +98 +00:05:08,690 --> 00:05:11,670 +For example here, when the file is empty, I'm going to + +99 +00:05:11,670 --> 00:05:15,010 +define this property empty file. And how am I + +100 +00:05:15,010 --> 00:05:18,490 +going to use this property? Well for example here, it + +101 +00:05:18,490 --> 00:05:20,760 +doesn't make sense to consider the case in which + +102 +00:05:20,760 --> 00:05:24,660 +we have one or many occurrences of the pattern in + +103 +00:05:24,660 --> 00:05:27,020 +the file if the file is empty. Therefore I'm + +104 +00:05:27,020 --> 00:05:31,650 +going to tell the tool that it should consider this specific + +105 +00:05:31,650 --> 00:05:35,780 +choice only if the file is not empty, only if + +106 +00:05:35,780 --> 00:05:38,660 +empty file is not defined. And that will skip, for + +107 +00:05:38,660 --> 00:05:41,330 +example, all of the combinations in which the file is + +108 +00:05:41,330 --> 00:05:44,171 +empty. And I'm trying to generate the test case that has + +109 +00:05:44,171 --> 00:05:46,364 +one occurrence of the pattern in the file, which is + +110 +00:05:46,364 --> 00:05:49,355 +simply not possible. For another example, in case I have + +111 +00:05:49,355 --> 00:05:52,824 +an empty pattern, I define the property empty pattern. And + +112 +00:05:52,824 --> 00:05:56,725 +then I avoid the choices that involve the pattern in case + +113 +00:05:56,725 --> 00:05:59,820 +the pattern is empty. because, for example, I cannot + +114 +00:05:59,820 --> 00:06:03,900 +have quotes in a pattern that is empty. For example, + +115 +00:06:03,900 --> 00:06:06,760 +it doesn't make sense to have blanks. So, one or + +116 +00:06:06,760 --> 00:06:10,250 +more blanks if the pattern is empty. So I'm going to + +117 +00:06:10,250 --> 00:06:14,180 +specify again that this choice should be considered only if + +118 +00:06:14,180 --> 00:06:16,140 +we don't have an empty pattern. And so on and + +119 +00:06:16,140 --> 00:06:20,080 +so forth. So now after I edit these constraints, I + +120 +00:06:20,080 --> 00:06:21,890 +can go back and compute again the number of test + +121 +00:06:21,890 --> 00:06:23,970 +frames and therefore the test cases that will be + +122 +00:06:23,970 --> 00:06:27,530 +generated with these constraints. So let me go again + +123 +00:06:27,530 --> 00:06:30,381 +to my terminal. Okay, so now I'm going to run + +124 +00:06:30,381 --> 00:06:34,061 +my TSL generator again, and I'm going to run it on + +125 +00:06:34,061 --> 00:06:37,389 +the second version of this file. And you can + +126 +00:06:37,389 --> 00:06:40,546 +see that I reduced the, the number of test frames + +127 +00:06:40,546 --> 00:06:43,889 +from about 7800 to about 1700. So it's quite + +128 +00:06:43,889 --> 00:06:46,967 +a, quite a big reduction by eliminating all these combinations + +129 +00:06:46,967 --> 00:06:49,540 +that do not make sense. But let's assume again that + +130 +00:06:49,540 --> 00:06:52,040 +we want to reduce this further so that we don't want to + +131 +00:06:52,040 --> 00:06:55,610 +generate those many test frames and therefore test cases. So + +132 +00:06:55,610 --> 00:06:58,660 +what can we do? We go back to our spec. And + +133 +00:06:58,660 --> 00:07:02,280 +in this case, we start adding error constraints. So if + +134 +00:07:02,280 --> 00:07:05,200 +you remember what we said in the lesson, error constraints are + +135 +00:07:05,200 --> 00:07:08,310 +constraints that indicate a choice that has to do with an + +136 +00:07:08,310 --> 00:07:11,980 +erroneous behaviour. For example, an erroneous input provided to the problem. + +137 +00:07:11,980 --> 00:07:15,210 +So here for instance, we're indicating the presence + +138 +00:07:15,210 --> 00:07:20,060 +of incorrectly enclosing quotes as an error choice. Same + +139 +00:07:20,060 --> 00:07:22,270 +thing if there's no file corresponding to the + +140 +00:07:22,270 --> 00:07:23,970 +name that we provide to the tool, we say + +141 +00:07:23,970 --> 00:07:26,760 +that this corresponds to an error. So how + +142 +00:07:26,760 --> 00:07:29,130 +is the tool going to use this information? It uses + +143 +00:07:29,130 --> 00:07:33,980 +this information by producing only one combination that involves + +144 +00:07:33,980 --> 00:07:37,270 +error choices, instead of combining them with other choices. + +145 +00:07:37,270 --> 00:07:39,780 +So let's see what happens after we added this + +146 +00:07:39,780 --> 00:07:43,370 +error constraints. So we go back to our console + +147 +00:07:43,370 --> 00:07:46,920 +once more. And in this case, we want to run + +148 +00:07:46,920 --> 00:07:50,910 +the TSL generator with the version of the, of my + +149 +00:07:50,910 --> 00:07:53,900 +file that contains the area of constraints. And again, + +150 +00:07:53,900 --> 00:07:56,390 +I reduce quite a bit the number of test frames. + +151 +00:07:56,390 --> 00:07:59,110 +So now I have only 562 test frames that + +152 +00:07:59,110 --> 00:08:02,660 +will be generated by using the file that I provided. + +153 +00:08:02,660 --> 00:08:05,460 +So for the last time, let's assume that we really want + +154 +00:08:05,460 --> 00:08:07,780 +to cut down the number of test frames or the number of + +155 +00:08:07,780 --> 00:08:10,380 +test cases. So once more, we go back to our file, and + +156 +00:08:10,380 --> 00:08:12,980 +at this point what we can add is the final type of + +157 +00:08:12,980 --> 00:08:14,170 +constraints that we have, which are + +158 +00:08:14,170 --> 00:08:17,245 +single constraints. And single constraints are + +159 +00:08:17,245 --> 00:08:21,360 +basically indicated choices that we don't want to combine with other choices. + +160 +00:08:21,360 --> 00:08:24,210 +So they have the same effect of the error constraints, but they + +161 +00:08:24,210 --> 00:08:28,120 +have a different meaning, so they do not indicate choices that corresponds + +162 +00:08:28,120 --> 00:08:29,860 +to an error. In other words, I can use a + +163 +00:08:29,860 --> 00:08:35,280 +single constraints to identify choices that I want to test only once. + +164 +00:08:35,280 --> 00:08:38,510 +So for example in this case, I might decide that I + +165 +00:08:38,510 --> 00:08:42,520 +want to have only one test frame that tests my program + +166 +00:08:42,520 --> 00:08:44,420 +with a file being empty and I can do the + +167 +00:08:44,420 --> 00:08:47,370 +same for other choices. So basically I can continue adding this + +168 +00:08:47,370 --> 00:08:50,400 +single constraint until I get down to the number of test + +169 +00:08:50,400 --> 00:08:53,410 +frames and therefore the number of test cases that I want. + +170 +00:08:53,410 --> 00:08:57,770 +So now let's go back once more to our console. And so now if we run + +171 +00:08:59,060 --> 00:09:04,450 +using this file as input, you can see that we have 35 test frames generated. So + +172 +00:09:04,450 --> 00:09:07,750 +this is a fairly low number of test cases, so we might decide that we want to + +173 +00:09:07,750 --> 00:09:13,380 +go ahead and write these test frames to a file. So now let's open this file + +174 +00:09:15,990 --> 00:09:25,500 +that we just generated. And as you can see here, I have exactly 35 test frames, + +175 +00:09:26,670 --> 00:09:30,900 +as expected. Some of those correspond to the single and error cases. So in this + +176 +00:09:30,900 --> 00:09:33,330 +case, the only choice that I have indicated + +177 +00:09:33,330 --> 00:09:35,690 +is the one that corresponds to the single + +178 +00:09:35,690 --> 00:09:38,310 +or error constraint. What is for the other + +179 +00:09:38,310 --> 00:09:42,170 +ones? I actually have the whole test spec. + +180 +00:09:42,170 --> 00:09:45,530 +So let's pick one just to give you an example. + +181 +00:09:45,530 --> 00:09:48,440 +In this case, that's frame number 15 that will correspond to + +182 +00:09:48,440 --> 00:09:51,910 +test case number 15. And here you can see that + +183 +00:09:51,910 --> 00:09:55,280 +we have all the information. So this is a test specification. + +184 +00:09:55,280 --> 00:09:57,560 +All the information that we need to generate the corresponding + +185 +00:09:57,560 --> 00:09:59,760 +test. We know that we need a file that is not + +186 +00:09:59,760 --> 00:10:03,810 +empty. That we need to have one occurrence of the pattern + +187 +00:10:03,810 --> 00:10:07,580 +in the file. One occurrence of the pattern in one line. + +188 +00:10:08,680 --> 00:10:10,360 +The position of the pattern in the file can + +189 +00:10:10,360 --> 00:10:13,740 +be any position. The length of the pattern must + +190 +00:10:13,740 --> 00:10:16,640 +be more than one character. The pattern should not + +191 +00:10:16,640 --> 00:10:20,140 +be enclosed in quotes. There should be one white + +192 +00:10:20,140 --> 00:10:24,460 +space, one quote within the pattern, and finally the + +193 +00:10:24,460 --> 00:10:27,230 +file that would pass through the program should exist. + +194 +00:10:27,230 --> 00:10:29,680 +So the file should be present. So I can + +195 +00:10:29,680 --> 00:10:33,950 +easily transform all of this into an actual test case. + +196 +00:10:33,950 --> 00:10:35,540 +And notice that even though we're not, we're not + +197 +00:10:35,540 --> 00:10:38,420 +going to do it here. In cases like this, it might + +198 +00:10:38,420 --> 00:10:42,190 +even be possible to automatically generate the test cases + +199 +00:10:42,190 --> 00:10:45,020 +from the test specifications because, here for example, here it + +200 +00:10:45,020 --> 00:10:48,150 +should be relatively straight forward to parse these test + +201 +00:10:48,150 --> 00:10:52,450 +specifications and generate test cases accordingly. So, just to summarize, + +202 +00:10:52,450 --> 00:10:55,910 +what we have done is to go from one high-level + +203 +00:10:55,910 --> 00:10:58,880 +description of a program to a set of categories, partitions, + +204 +00:10:58,880 --> 00:11:01,820 +and choices for that program. Then we have combined them + +205 +00:11:01,820 --> 00:11:04,930 +in different ways, adding more and more constraints to reduce the + +206 +00:11:04,930 --> 00:11:07,600 +number of combinations until we ended up with the right number + +207 +00:11:07,600 --> 00:11:09,650 +of test cases, so the number of test cases that we + +208 +00:11:09,650 --> 00:11:14,630 +were fine generating. We generated the corresponding test specifications. And at + +209 +00:11:14,630 --> 00:11:17,340 +that point, we could just go ahead, generate the test case, + +210 +00:11:17,340 --> 00:11:20,660 +and test our application. So, and you can see how this + +211 +00:11:20,660 --> 00:11:23,720 +can result in a much more thorough testing of your application. + +212 +00:11:23,720 --> 00:11:27,890 +Because instead of reading this description and just trying to come up with test + +213 +00:11:27,890 --> 00:11:33,900 +cases for it, we can break down the process in steps that are easy to perform + +214 +00:11:33,900 --> 00:11:36,960 +individually. They can be automated as much + +215 +00:11:36,960 --> 00:11:38,600 +as possible. And they will end up with + +216 +00:11:38,600 --> 00:11:40,380 +a set of test cases that will test + +217 +00:11:40,380 --> 00:11:42,790 +all the interests and aspects of your application. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt new file mode 100644 index 0000000..3d41cd9 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt @@ -0,0 +1,83 @@ +1 +00:00:00,370 --> 00:00:02,950 +What we just saw with the category-partition method, is a + +2 +00:00:02,950 --> 00:00:05,530 +specific instance of this systematic + +3 +00:00:05,530 --> 00:00:08,230 +functional testing approach. So specific instance + +4 +00:00:08,230 --> 00:00:10,900 +of the steps that we represented here. And, as I + +5 +00:00:10,900 --> 00:00:13,100 +mentioned earlier on, this is not the only way in which + +6 +00:00:13,100 --> 00:00:16,700 +you can generate test cases, starting from a functional specification. + +7 +00:00:16,700 --> 00:00:20,190 +In particular, this step, in which we identified relevant inputs and + +8 +00:00:20,190 --> 00:00:23,390 +then we combine them to generate test case specifications, can also + +9 +00:00:23,390 --> 00:00:25,490 +be done in different ways. And, we're going to look at one + +10 +00:00:25,490 --> 00:00:28,170 +of these ways. Which is through the construction of a + +11 +00:00:28,170 --> 00:00:31,040 +model. And, the reason why I want to talk about models. + +12 +00:00:31,040 --> 00:00:34,090 +Is because, model based testing is also, fairly popular in + +13 +00:00:34,090 --> 00:00:37,680 +industry. And fairly used in practice. In model based testing, the + +14 +00:00:37,680 --> 00:00:41,150 +way in which we go from specifications, to test cases, + +15 +00:00:41,150 --> 00:00:44,220 +is through the construction of a model. Where a model is + +16 +00:00:44,220 --> 00:00:47,670 +an abstract representation of the software under test. Also in + +17 +00:00:47,670 --> 00:00:50,860 +this case there are many possible models, that we can use. + +18 +00:00:50,860 --> 00:00:52,180 +And what we're going to do, we're going to focus + +19 +00:00:52,180 --> 00:00:54,370 +on a specific kind of model. And I'll + +20 +00:00:54,370 --> 00:00:57,010 +just point you to additional sources of information, + +21 +00:00:57,010 --> 00:00:59,070 +in case you're interested in seeing other examples. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt new file mode 100644 index 0000000..26f2fd7 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt @@ -0,0 +1,103 @@ +1 +00:00:00,160 --> 00:00:02,320 +The model that we will consider, is a very well + +2 +00:00:02,320 --> 00:00:05,330 +known one. Which is finite state machines. And you might have + +3 +00:00:05,330 --> 00:00:08,320 +seen them before. At a high level, a state machine is + +4 +00:00:08,320 --> 00:00:11,990 +a graph in which nodes represent states of the system. For + +5 +00:00:11,990 --> 00:00:15,650 +example, in this case, state 1, state 2, and state 3. + +6 +00:00:15,650 --> 00:00:19,950 +Edges represent transitions between states. For instance, in this case we + +7 +00:00:19,950 --> 00:00:22,850 +have one edge from state 1, to state 2. That means + +8 +00:00:22,850 --> 00:00:25,640 +that the system can go from state 1, to state 2. + +9 +00:00:25,640 --> 00:00:29,530 +And finally, the labels on the edges represent events + +10 +00:00:29,530 --> 00:00:32,800 +and actions. For example, what this label means is that + +11 +00:00:32,800 --> 00:00:35,400 +the system goes from state three to state two + +12 +00:00:35,400 --> 00:00:39,140 +when event five occurs. And when going from state three + +13 +00:00:39,140 --> 00:00:42,190 +to state two, it generates action four. And does + +14 +00:00:42,190 --> 00:00:45,430 +reacher model, sir reacher's kind of state machines, but we're + +15 +00:00:45,430 --> 00:00:48,160 +just going to stick to this ones which are enough. For + +16 +00:00:48,160 --> 00:00:50,760 +our purpose. So how do we build such a final + +17 +00:00:50,760 --> 00:00:53,530 +state machine starting from a specification? The first thing + +18 +00:00:53,530 --> 00:00:56,660 +we need to do is to identify the system's boundaries + +19 +00:00:56,660 --> 00:00:59,250 +and the input and output to the system. Once we + +20 +00:00:59,250 --> 00:01:01,960 +have done that, we can identify, within the boundaries of + +21 +00:01:01,960 --> 00:01:06,070 +the system, the relevant states and transitions. So we split + +22 +00:01:06,070 --> 00:01:09,670 +this single state We'll refine it into several states. And + +23 +00:01:09,670 --> 00:01:12,640 +we also identify how the system can go from one + +24 +00:01:12,640 --> 00:01:16,830 +state to another. Including which inputs cause which transition, and + +25 +00:01:16,830 --> 00:01:19,350 +which result in outputs we can obtain. To + +26 +00:01:19,350 --> 00:01:21,810 +better illustrate that, let's look at a concrete example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt new file mode 100644 index 0000000..f889057 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt @@ -0,0 +1,251 @@ +1 +00:00:00,140 --> 00:00:03,010 +In this example, we're going to start from an informal specification, and the + +2 +00:00:03,010 --> 00:00:06,670 +specification is the one shown here in file spec.txt. This is the + +3 +00:00:06,670 --> 00:00:10,210 +specification for the maintenance function in a specific system. So what we're + +4 +00:00:10,210 --> 00:00:13,630 +doing is that we're taking the description of the functionality of a system, + +5 +00:00:13,630 --> 00:00:16,219 +and we're building a model, in this case a final state machine + +6 +00:00:16,219 --> 00:00:18,710 +for it. And there is no need to look at all the details + +7 +00:00:18,710 --> 00:00:21,350 +for this specification, but I want to point out that if you + +8 +00:00:21,350 --> 00:00:25,640 +look at the way the specification is written, we can identify specific cases + +9 +00:00:25,640 --> 00:00:28,210 +that we need to take into account. Like here if something + +10 +00:00:28,210 --> 00:00:31,920 +happens, something else will follow. Again, if something happens something else + +11 +00:00:31,920 --> 00:00:35,690 +will follow. So we have multiple choices here. Here will determine + +12 +00:00:35,690 --> 00:00:38,140 +the next steps and so on. So all we have to + +13 +00:00:38,140 --> 00:00:42,420 +do is to go through this process, identify these cases and + +14 +00:00:42,420 --> 00:00:45,300 +then build a machine that represents these cases. For the spec + +15 +00:00:45,300 --> 00:00:47,830 +that we just consider this is the state machine that will + +16 +00:00:47,830 --> 00:00:50,960 +result. Again there is no need to go through all the details, + +17 +00:00:50,960 --> 00:00:53,090 +but what I want to point out is that we have + +18 +00:00:53,090 --> 00:00:55,710 +a set of states. So for instance, we have state zero, + +19 +00:00:55,710 --> 00:00:58,380 +which is no maintenance, and if a request comes in, the + +20 +00:00:58,380 --> 00:01:01,350 +system will move, and the system wait for pickup. Then if + +21 +00:01:01,350 --> 00:01:04,400 +the pickup actually occurs, the system will move to the repair + +22 +00:01:04,400 --> 00:01:07,540 +state, and so on and so forth. So this is just + +23 +00:01:07,540 --> 00:01:13,070 +a more systematic representation of what was in the former specification. + +24 +00:01:13,070 --> 00:01:16,160 +And I will argue that this is much easier to understand at + +25 +00:01:16,160 --> 00:01:19,170 +least for somebody who has to develop tests for this system. In + +26 +00:01:19,170 --> 00:01:21,770 +fact what we're going to see next is how we can go from that + +27 +00:01:21,770 --> 00:01:24,790 +representation to a set of test cases. And the way which we do + +28 +00:01:24,790 --> 00:01:28,950 +it is by covering the behaviors represented by defining state machine. And we + +29 +00:01:28,950 --> 00:01:31,500 +can decide how we want to cover them. For example we might want + +30 +00:01:31,500 --> 00:01:35,080 +to cover all the states. So we might want to identify paths in + +31 +00:01:35,080 --> 00:01:38,310 +the state machine that go through all the states in the machine. Like + +32 +00:01:38,310 --> 00:01:41,840 +the one I just draw or this one, this one and this one. + +33 +00:01:41,840 --> 00:01:44,900 +So if we consider these four test cases, we can see that all the + +34 +00:01:44,900 --> 00:01:48,470 +states in my system or at least all the states that I have identified + +35 +00:01:48,470 --> 00:01:51,450 +are covered. I might want to go a little further, and decide that I + +36 +00:01:51,450 --> 00:01:54,210 +don't only want to cover all of the states, but I want to cover, all + +37 +00:01:54,210 --> 00:01:57,930 +of the transitions, because, it makes sense to visit a state, when coming from + +38 +00:01:57,930 --> 00:02:00,380 +different states. And, if I want to do that, and I look at the + +39 +00:02:00,380 --> 00:02:03,440 +test cases that I generated so far, I can see that there is one + +40 +00:02:03,440 --> 00:02:06,910 +transition, the one here, that is not covered. And, the same can be said for + +41 +00:02:06,910 --> 00:02:09,210 +the two transitions here. So what I can decide to do is + +42 +00:02:09,210 --> 00:02:13,370 +to generate another test case, that covers those or extend an existing one. + +43 +00:02:13,370 --> 00:02:16,500 +For instance, I could extend this test case by adding a visit to + +44 +00:02:16,500 --> 00:02:20,760 +the state, before going back to these two. Alternatively, I could also generate + +45 +00:02:20,760 --> 00:02:24,390 +new test cases, such as this one. To cover the missing transitions. + +46 +00:02:24,390 --> 00:02:26,350 +And once I have these test cases, I can express them in a + +47 +00:02:26,350 --> 00:02:29,860 +clearer way by simply specifying what are the states that they cover. I'm + +48 +00:02:29,860 --> 00:02:31,990 +just going to give you a couple of examples. Say, if we look + +49 +00:02:31,990 --> 00:02:34,280 +at the last one that I added, which will be test case + +50 +00:02:34,280 --> 00:02:37,190 +number five, I just need to specify that it will go through state + +51 +00:02:37,190 --> 00:02:41,090 +zero, which is this one, five, which is this one, six, and + +52 +00:02:41,090 --> 00:02:43,130 +then back to zero. And I can do the same for the other + +53 +00:02:43,130 --> 00:02:46,500 +test cases. So this will be my complete set of test cases. + +54 +00:02:46,500 --> 00:02:50,060 +So the bottom line here is that it is much harder to build + +55 +00:02:50,060 --> 00:02:53,080 +a set of test cases that will cover the behavior of an informal + +56 +00:02:53,080 --> 00:02:56,970 +description. But by going through a model, so by building in this case, + +57 +00:02:56,970 --> 00:03:01,700 +a finite state machine for that description, we can, in a much easier way, see + +58 +00:03:01,700 --> 00:03:04,100 +what the behaviors of interest of the system + +59 +00:03:04,100 --> 00:03:05,960 +are, and try to cover them. And there + +60 +00:03:05,960 --> 00:03:07,400 +is again in the spirit of breaking + +61 +00:03:07,400 --> 00:03:09,950 +down a complex problem into smaller steps that + +62 +00:03:09,950 --> 00:03:11,620 +we can better manage, which in the end, + +63 +00:03:11,620 --> 00:03:14,450 +results in a more efficient and effective testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt new file mode 100644 index 0000000..64c2c00 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt @@ -0,0 +1,99 @@ +1 +00:00:00,100 --> 00:00:03,060 +There are some important considerations I want to make on final state + +2 +00:00:03,060 --> 00:00:06,410 +machines. And more in general, on model based testing. The first one + +3 +00:00:06,410 --> 00:00:10,600 +is about applicability. Testing based on final state machines is a very + +4 +00:00:10,600 --> 00:00:13,890 +general approach, that we can apply in a number of contexts. And in + +5 +00:00:13,890 --> 00:00:16,510 +particular, if you are working with UML, you have state machines for + +6 +00:00:16,510 --> 00:00:19,720 +free. Because state charts are nothing else but a special kind of + +7 +00:00:19,720 --> 00:00:22,210 +state machine. So you can apply the technique that we just saw + +8 +00:00:22,210 --> 00:00:26,010 +directly on state charts, and try to cover their states and their transitions. + +9 +00:00:26,010 --> 00:00:29,980 +Another important point is that abstraction is key. You have to find the + +10 +00:00:29,980 --> 00:00:33,280 +right level of abstraction. The bigger the system, the more you have to + +11 +00:00:33,280 --> 00:00:36,130 +abstract if you want to represent it with a model, and in particular, with + +12 +00:00:36,130 --> 00:00:38,710 +the final state machine. So it's like having a slider, and you have + +13 +00:00:38,710 --> 00:00:41,840 +to decide where you want to move on that slider. The more you represent, + +14 +00:00:41,840 --> 00:00:44,700 +the more complex your system is going to be and the more thorough your + +15 +00:00:44,700 --> 00:00:48,110 +testing is going to be but also more expensive. The less you represent the + +16 +00:00:48,110 --> 00:00:51,330 +less expensive testing is going to be, but also testing might not be as + +17 +00:00:51,330 --> 00:00:53,500 +thorough as it would be otherwise. So you have to find + +18 +00:00:53,500 --> 00:00:56,150 +the right balance between abstracting the weight too much and abstracting + +19 +00:00:56,150 --> 00:00:59,840 +the weight too little. And finally there are many other approaches. + +20 +00:00:59,840 --> 00:01:02,840 +So we just scratched the surface, and we just saw one possible + +21 +00:01:02,840 --> 00:01:05,760 +approach. But for instance, other models that you can use are + +22 +00:01:05,760 --> 00:01:09,780 +decision tables, flow graphs and even historical models. Models that can + +23 +00:01:09,780 --> 00:01:13,560 +guide your testing based on problems that occurred in your system + +24 +00:01:13,560 --> 00:01:16,500 +in the past. And also, in this case, I'm going to put pointers + +25 +00:01:16,500 --> 00:01:18,460 +to additional materials in the class notes. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt new file mode 100644 index 0000000..1b68b9f --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt @@ -0,0 +1,67 @@ +1 +00:00:00,110 --> 00:00:01,600 +Now we are at the end of this lesson, and I + +2 +00:00:01,600 --> 00:00:04,450 +just want to wrap it up by summarizing what we've seen. We talked + +3 +00:00:04,450 --> 00:00:06,340 +about black-box testing, the testing of + +4 +00:00:06,340 --> 00:00:08,760 +software based on a functional specification, + +5 +00:00:08,760 --> 00:00:11,510 +a description of the software rather than its code. We saw a + +6 +00:00:11,510 --> 00:00:15,500 +systematic way of doing that, that allows for breaking down the problem + +7 +00:00:15,500 --> 00:00:19,140 +of testing software, so the problem of going from this functional specification + +8 +00:00:19,140 --> 00:00:22,540 +to a set of test cases into smaller steps, more manageable steps. + +9 +00:00:22,540 --> 00:00:25,466 +And we saw two main ways of doing this. One by identifying + +10 +00:00:25,466 --> 00:00:28,304 +relevant inputs for the main features in the system + +11 +00:00:28,304 --> 00:00:31,802 +and then deriving test case specifications and test cases from + +12 +00:00:31,802 --> 00:00:34,180 +this set of inputs. And the second way by + +13 +00:00:34,180 --> 00:00:36,780 +building a model for the main features of the system + +14 +00:00:36,780 --> 00:00:39,210 +and then using this model to decide how to + +15 +00:00:39,210 --> 00:00:41,680 +test the system. In the next lesson, we are going + +16 +00:00:41,680 --> 00:00:44,440 +to discuss, how to do testing by looking inside the + +17 +00:00:44,440 --> 00:00:47,716 +box? So, how to do testing in a white-box fashion. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt new file mode 100644 index 0000000..75da7e8 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt @@ -0,0 +1,179 @@ +1 +00:00:00,140 --> 00:00:02,690 +So what is the starting point of black box testing? Black + +2 +00:00:02,690 --> 00:00:06,010 +box testing start from a description of the software or as we + +3 +00:00:06,010 --> 00:00:09,590 +call it, a functional specification. And the final result of black + +4 +00:00:09,590 --> 00:00:12,760 +box testing is a set of test cases, a set of actual + +5 +00:00:12,760 --> 00:00:16,410 +inputs and corresponding outputs that we can use to exercise our + +6 +00:00:16,410 --> 00:00:19,030 +code and to try to find defects in our code. So the + +7 +00:00:19,030 --> 00:00:22,060 +question is, how do we get from functional specification to test + +8 +00:00:22,060 --> 00:00:25,510 +cases? Doing these derivations, so going from this description to a concrete + +9 +00:00:25,510 --> 00:00:28,550 +set of tests, is a very complex analytical process. + +10 +00:00:28,550 --> 00:00:31,220 +And normally brute force generation is not a good + +11 +00:00:31,220 --> 00:00:34,430 +idea because it's inefficient and ineffective. What we want + +12 +00:00:34,430 --> 00:00:37,540 +to do instead is to have a systematic approach to + +13 +00:00:37,540 --> 00:00:40,250 +derive test cases from a functional specification. What a + +14 +00:00:40,250 --> 00:00:43,970 +systematic approach does is to simplify the overall problem by + +15 +00:00:43,970 --> 00:00:48,320 +dividing the process into elementary steps. In particular, in + +16 +00:00:48,320 --> 00:00:50,520 +this case, we will perform three main steps. The first + +17 +00:00:50,520 --> 00:00:55,790 +step is to identify independently testable features. Individual features in + +18 +00:00:55,790 --> 00:00:57,600 +the soft hood that we can test. And we're going to + +19 +00:00:57,600 --> 00:00:59,990 +expand on each one of these steps in the next + +20 +00:00:59,990 --> 00:01:02,490 +part of the lesson. The following step is once we have + +21 +00:01:02,490 --> 00:01:06,000 +these independently testable features to identify what are the relevant + +22 +00:01:06,000 --> 00:01:08,590 +inputs. So what are the inputs or the behavior that is + +23 +00:01:08,590 --> 00:01:11,610 +worth testing for these features. Next once we have these + +24 +00:01:11,610 --> 00:01:13,020 +inputs, we're going to derive test + +25 +00:01:13,020 --> 00:01:15,770 +specifications. And test case specifications are + +26 +00:01:15,770 --> 00:01:19,490 +description of the test cases that we can then use + +27 +00:01:19,490 --> 00:01:23,270 +to generate actual test cases. And proceeding in this way, + +28 +00:01:23,270 --> 00:01:26,050 +by this steps, has many advantages. It allows for the + +29 +00:01:26,050 --> 00:01:29,920 +coupling different activities. It allows for dividing brain intensive steps from + +30 +00:01:29,920 --> 00:01:32,240 +steps that can be automated, which is a great advantage. + +31 +00:01:32,240 --> 00:01:34,980 +And also we will see, it allows you for monitoring + +32 +00:01:34,980 --> 00:01:38,040 +the testing process. So to figure out whether your testing + +33 +00:01:38,040 --> 00:01:41,000 +process is going as expected, for example, if you're generating too + +34 +00:01:41,000 --> 00:01:44,160 +many test cases. Or you're generating the number of test cases that your + +35 +00:01:44,160 --> 00:01:47,880 +amount of resources available allows you to run. So let's start by looking + +36 +00:01:47,880 --> 00:01:51,230 +at the first step of this process in which our goal is to + +37 +00:01:51,230 --> 00:01:54,820 +go from a Functional Specification to a set of features that we can + +38 +00:01:54,820 --> 00:01:57,700 +test in the software. So what we want to do is to identify all + +39 +00:01:57,700 --> 00:02:00,290 +of the feature of the software. And why do we want to do this? + +40 +00:02:00,290 --> 00:02:02,650 +Well you know, in the spirit of breaking down the complexity of the + +41 +00:02:02,650 --> 00:02:06,170 +problem, it does not make sense to just try to devise test cases for + +42 +00:02:06,170 --> 00:02:08,750 +all the features of the software at once. For any non-trivial + +43 +00:02:08,750 --> 00:02:11,980 +software, that's a humongous problem, and something that we cannot really + +44 +00:02:11,980 --> 00:02:16,300 +handle effectively. A much better way is to identify independently testable + +45 +00:02:16,300 --> 00:02:19,100 +features and consider one of them at a time when generating tests. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt new file mode 100644 index 0000000..dda3144 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt @@ -0,0 +1,35 @@ +1 +00:00:00,740 --> 00:00:02,780 +So, now I want to do a little quiz about + +2 +00:00:02,780 --> 00:00:05,530 +identifying testable features. Let's consider + +3 +00:00:05,530 --> 00:00:07,700 +this simple problem called printSum. + +4 +00:00:07,700 --> 00:00:09,680 +We won't see the implementation because we are doing black-box + +5 +00:00:09,680 --> 00:00:12,780 +testing. And all we need to know is that printSum takes + +6 +00:00:12,780 --> 00:00:15,550 +two integers, a and b, and prints the sum of + +7 +00:00:15,550 --> 00:00:17,540 +these two numbers. So what I want to ask, is + +8 +00:00:17,540 --> 00:00:20,700 +how many independently testable features do we have here? Do + +9 +00:00:20,700 --> 00:00:24,860 +we have one, two, three features? Or more than three features? diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt new file mode 100644 index 0000000..5503f98 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt @@ -0,0 +1,15 @@ +1 +00:00:00,360 --> 00:00:03,760 +Sum is a very simple program, that only does one thing, + +2 +00:00:03,760 --> 00:00:07,540 +summing two number, adding two numbers. So the answer in this + +3 +00:00:07,540 --> 00:00:10,640 +case, it's one. There's only one feature that we can test + +4 +00:00:10,640 --> 00:00:14,060 +in PrintSum. So let's look at the slightly more interesting example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt new file mode 100644 index 0000000..96c643e --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt @@ -0,0 +1,19 @@ +1 +00:00:00,150 --> 00:00:02,469 +Let's look at this spreadsheet. I'm pretty sure most of you + +2 +00:00:02,469 --> 00:00:05,990 +are familiar with what a spreadsheet is and have used them before. + +3 +00:00:05,990 --> 00:00:08,119 +So now I'm going to ask the same question, which is I'd + +4 +00:00:08,119 --> 00:00:10,480 +like you to identify three possible + +5 +00:00:10,480 --> 00:00:13,190 +independently testable features for a spreadsheet. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt new file mode 100644 index 0000000..88e7cd5 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt @@ -0,0 +1,71 @@ +1 +00:00:00,190 --> 00:00:03,030 +In this case there's not really a right answer because there's + +2 +00:00:03,030 --> 00:00:05,750 +many, many features that you could identify in a piece of + +3 +00:00:05,750 --> 00:00:08,655 +software as complex as a, a spreadsheet. So I'm just going + +4 +00:00:08,655 --> 00:00:11,900 +to give you three examples. So one could be the cell merging + +5 +00:00:11,900 --> 00:00:14,950 +operation. So the operation in which we merge two cells in + +6 +00:00:14,950 --> 00:00:18,550 +the spreadsheet. Another example could be chart creation, so I might want + +7 +00:00:18,550 --> 00:00:21,370 +to test the feature that allows you to create charts in + +8 +00:00:21,370 --> 00:00:23,310 +your spreadsheets. Yet another example could + +9 +00:00:23,310 --> 00:00:25,540 +be the test of statistical functions, + +10 +00:00:25,540 --> 00:00:29,350 +so the function that allows you to do various statistical calculations on + +11 +00:00:29,350 --> 00:00:32,650 +the numbers in your cells. And as I said there's many, many + +12 +00:00:32,650 --> 00:00:35,900 +more example that we could use. But the key thing I want + +13 +00:00:35,900 --> 00:00:38,770 +to convey here is the fact that there is no way you + +14 +00:00:38,770 --> 00:00:42,020 +can look at a spreadsheet with all the functionality that it provides + +15 +00:00:42,020 --> 00:00:44,160 +and just go and test it. The first step, what you need + +16 +00:00:44,160 --> 00:00:47,430 +to do first, is to identify which ones are the pieces of + +17 +00:00:47,430 --> 00:00:50,620 +functionality that I can test individually. So that's why this is the + +18 +00:00:50,620 --> 00:00:52,310 +first step in black-box testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt new file mode 100644 index 0000000..51aaffe --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt @@ -0,0 +1,103 @@ +1 +00:00:00,230 --> 00:00:03,550 +Once we have identified Independently Testable Features, the next step is to + +2 +00:00:03,550 --> 00:00:07,720 +identify the Relevant Inputs for each one of these features. And there are + +3 +00:00:07,720 --> 00:00:10,420 +many ways to do that. So, what we're going to do, instead of + +4 +00:00:10,420 --> 00:00:13,770 +looking at them all, is that we're just going to focus on two different + +5 +00:00:13,770 --> 00:00:16,239 +ways of doing it. And they are fairly general ways. So, they + +6 +00:00:16,239 --> 00:00:19,910 +are applicable to a number of situations. And in addition, what I will + +7 +00:00:19,910 --> 00:00:22,190 +do, I will point you to other sources in which you can + +8 +00:00:22,190 --> 00:00:25,530 +look at different ways of doing this in the class notes. The problem + +9 +00:00:25,530 --> 00:00:28,390 +of identifying relevant inputs for some Software or some feature + +10 +00:00:28,390 --> 00:00:31,930 +of it is called Test Data Selection and can be expressed + +11 +00:00:31,930 --> 00:00:35,040 +as followed. Let's consider our software as usual we have + +12 +00:00:35,040 --> 00:00:38,190 +our Input Domain, which is the set of inputs for all + +13 +00:00:38,190 --> 00:00:40,980 +the software. And again as usual, we have our Output + +14 +00:00:40,980 --> 00:00:44,050 +Domain, which is the set of corresponding outlets for these inputs. + +15 +00:00:44,050 --> 00:00:47,450 +So the question here is, how can we select a meaningful + +16 +00:00:47,450 --> 00:00:50,920 +set of inputs in my domain? And of course corresponding outputs + +17 +00:00:50,920 --> 00:00:53,240 +because we know that test cases are an input, plus an + +18 +00:00:53,240 --> 00:00:56,900 +expected output. So how can we select interesting inputs for our + +19 +00:00:56,900 --> 00:01:00,040 +software? So a set of inputs that, after we run them + +20 +00:01:00,040 --> 00:01:02,950 +on the software, if the software behaves correctly, we'll have enough + +21 +00:01:02,950 --> 00:01:07,060 +confidence that the software is correctly implemented. So one possible idea + +22 +00:01:07,060 --> 00:01:09,830 +is, hey, why don't we just test them all? We just + +23 +00:01:09,830 --> 00:01:12,800 +do exhaustive testing. We do all the inputs, nowadays we have + +24 +00:01:12,800 --> 00:01:16,290 +powerful machines, we have a lot of computational power in the cloud. + +25 +00:01:16,290 --> 00:01:17,890 +Why not just doing it? So to answer that + +26 +00:01:17,890 --> 00:01:20,480 +question, what I'm going to do? I'm going to use another quiz. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt new file mode 100644 index 0000000..9ffead6 --- /dev/null +++ b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt @@ -0,0 +1,31 @@ +1 +00:00:00,200 --> 00:00:02,180 +So I'm going to ask you something. Which is if + +2 +00:00:02,180 --> 00:00:04,750 +we consider again our function print sum, the one + +3 +00:00:04,750 --> 00:00:06,760 +that takes two integers and prints the sum. How + +4 +00:00:06,760 --> 00:00:09,300 +long would it take to exhaustively test this function? And + +5 +00:00:09,300 --> 00:00:10,950 +this is a very simple one. There's just two + +6 +00:00:10,950 --> 00:00:13,240 +inputs, right? So we can just enumerate them all. + +7 +00:00:13,240 --> 00:00:15,950 +And put them throw them at the computer form, + +8 +00:00:15,950 --> 00:00:18,440 +and wait for the results. How long would that take? |