diff options
| author | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-07-19 20:34:40 +0700 |
|---|---|---|
| committer | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-07-19 20:34:40 +0700 |
| commit | 8a7dfa0972c83fd811a4296e7373574bea4a28d0 (patch) | |
| tree | 16d37247e8b909ce5f885affd2b2473faab891fd /usth/ICT2.7/P4L2 Black-Box Testing Subtitles | |
| parent | dc6f57c3af35f599abab2c4bac950654282cb519 (diff) | |
| download | cp-8a7dfa0972c83fd811a4296e7373574bea4a28d0.tar.gz | |
[usth/ICT2.7] Remove Udacity transcribes
Diffstat (limited to 'usth/ICT2.7/P4L2 Black-Box Testing Subtitles')
28 files changed, 0 insertions, 3700 deletions
diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt deleted file mode 100644 index 09a8f10..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt +++ /dev/null @@ -1,47 +0,0 @@ -1 -00:00:00,320 --> 00:00:02,620 -In the previous lesson, we discussed the fundamental - -2 -00:00:02,620 --> 00:00:06,380 -concepts behind software verification in general, and software testing - -3 -00:00:06,380 --> 00:00:09,470 -in particular. In this lesson, we will discuss - -4 -00:00:09,470 --> 00:00:12,310 -one of the two main testing approaches. Black box - -5 -00:00:12,310 --> 00:00:16,180 -testing, also called functional testing. We will cover - -6 -00:00:16,180 --> 00:00:19,180 -the main characteristic of black box testing, its pros - -7 -00:00:19,180 --> 00:00:22,260 -and cons, and discuss some commonly used black - -8 -00:00:22,260 --> 00:00:25,350 -box testing techniques. We will conclude the lesson with - -9 -00:00:25,350 --> 00:00:28,520 -a practical exercise in which we will apply a specific black - -10 -00:00:28,520 --> 00:00:32,520 -box testing technique to a real program. We will derive test - -11 -00:00:32,520 --> 00:00:34,980 -cases for the program and assess how the use of a - -12 -00:00:34,980 --> 00:00:38,690 -systematic approach, in contrast to a brute force approach, can help. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 418ed91..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/10 - Test Data Selection Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,79 +0,0 @@ -1 -00:00:00,025 --> 00:00:02,900 -Okay, so now we're going to answer the question. So if we want - -2 -00:00:02,900 --> 00:00:06,520 -to consider all these inputs, and run them all on the software, - -3 -00:00:06,520 --> 00:00:09,490 -let's see how it will work. Let's assume that these are, 32 - -4 -00:00:09,490 --> 00:00:12,280 -bit integers. So at this point what we will have is, a - -5 -00:00:12,280 --> 00:00:15,130 -number of combination, which is 2 to the 32, times 2 to - -6 -00:00:15,130 --> 00:00:18,390 -the 32. They're two integers. This is equal to 2 to the - -7 -00:00:18,390 --> 00:00:21,910 -64, which in turn, is more or less equal, to 10 to - -8 -00:00:21,910 --> 00:00:25,110 -the 19. So 10 to the 19 is the number of tests that - -9 -00:00:25,110 --> 00:00:27,960 -we need to run to cover the whole domain. Now let's assume - -10 -00:00:27,960 --> 00:00:31,400 -that we can run one test per nanosecond. So what that means - -11 -00:00:31,400 --> 00:00:34,290 -is that we can run 10 to the 9 tests per second, - -12 -00:00:34,290 --> 00:00:37,240 -and that's a lot. If we do the math, that results in 10 - -13 -00:00:37,240 --> 00:00:40,750 -to the 10 seconds over all, because we have 10 to the - -14 -00:00:40,750 --> 00:00:43,760 -19 tests, we could run 10 to the 9 tests per second - -15 -00:00:43,760 --> 00:00:46,630 -so, we do the math, and we can run all these tests - -16 -00:00:46,630 --> 00:00:50,340 -in 10 to the 10 seconds. And what that corresponds to, it's about - -17 -00:00:50,340 --> 00:00:54,470 -600 years, so a lot of time. So even for such - -18 -00:00:54,470 --> 00:00:57,710 -a simple problem, a problem that takes two integers and adds them, - -19 -00:00:57,710 --> 00:01:00,990 -it will take more than 500 years to test it exhaustively. So - -20 -00:01:00,990 --> 00:01:04,690 -the bottom line here is that we just can't do exhaustive testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt deleted file mode 100644 index a91af45..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/11 - Why Not Random Testing? - lang_en_vs4.srt +++ /dev/null @@ -1,143 +0,0 @@ -1 -00:00:00,310 --> 00:00:02,250 -So then maybe what we can do is just to - -2 -00:00:02,250 --> 00:00:05,720 -pick our test inputs randomly so to do what is called - -3 -00:00:05,720 --> 00:00:08,850 -random testing. And what that means is that we pick the - -4 -00:00:08,850 --> 00:00:11,720 -inputs to test just as we pick a number by rolling - -5 -00:00:11,720 --> 00:00:15,410 -a set of dice randomly. And this will have several advantages. - -6 -00:00:15,410 --> 00:00:18,780 -First, we will pick inputs uniformly. So if we use a - -7 -00:00:18,780 --> 00:00:21,790 -uniform distribution as the basis for our random testing, we will - -8 -00:00:21,790 --> 00:00:25,540 -make no preferences. In other words, all inputs will be considered - -9 -00:00:25,540 --> 00:00:28,324 -equal, of equal value. And what that means in turn, is - -10 -00:00:28,324 --> 00:00:32,640 -that random testing eliminates designer bias. So what does designer bias - -11 -00:00:32,640 --> 00:00:36,030 -mean? Designer bias is the problem of making the same assumption, - -12 -00:00:36,030 --> 00:00:38,570 -when we read the specification and we interpret it and when we - -13 -00:00:38,570 --> 00:00:42,100 -develop test cases. Which means that the developer might develop code, - -14 -00:00:42,100 --> 00:00:44,930 -assuming a given behavior of the user. And we may write - -15 -00:00:44,930 --> 00:00:47,520 -tests, making the same assumptions. And the problem, of course, is - -16 -00:00:47,520 --> 00:00:50,690 -even worse if it's the same person that develops the code and - -17 -00:00:50,690 --> 00:00:53,730 -writes the test cases. With random testing, the problem is gone, - -18 -00:00:53,730 --> 00:00:57,440 -because we just pick randomly what our inputs will be. So - -19 -00:00:57,440 --> 00:01:00,180 -why not do in random? The problem is that when testing, - -20 -00:01:00,180 --> 00:01:03,610 -we are looking for a needle in a haystack. Actually, multiple - -21 -00:01:03,610 --> 00:01:06,620 -needles in multiple haystacks, if we want to be precise. So, - -22 -00:01:06,620 --> 00:01:09,500 -random approaches are not necessarily the best way to go about - -23 -00:01:09,500 --> 00:01:12,430 -it, because we might just be looking in all the wrong - -24 -00:01:12,430 --> 00:01:15,760 -places. So let me show you this, using a different representation - -25 -00:01:15,760 --> 00:01:18,000 -for the haystack. What I'm showing here is a grid, and - -26 -00:01:18,000 --> 00:01:22,130 -imagine this grid just expanding indefinitely outside the screen, and this grid - -27 -00:01:22,130 --> 00:01:26,120 -represents the domain for the program, so each box in the grid, - -28 -00:01:26,120 --> 00:01:29,050 -each square in the grid, it's a possible input. So what happens - -29 -00:01:29,050 --> 00:01:32,670 -with bugs is that bugs are very scarce in this grid. Maybe - -30 -00:01:32,670 --> 00:01:35,070 -there is a bug here, so that means that there is a - -31 -00:01:35,070 --> 00:01:38,090 -bug, than an input, in this point we'll reveal. And maybe there - -32 -00:01:38,090 --> 00:01:40,820 -is another bug that will be triggered by an input over here. - -33 -00:01:40,820 --> 00:01:44,570 -So imagine this spread out over this infinite grid. Its very unlikely - -34 -00:01:44,570 --> 00:01:47,440 -that just by picking randomly that we will be able to get to - -35 -00:01:47,440 --> 00:01:50,910 -these two points. Fortunately not all is lost, there is a silver lining. - -36 -00:01:50,910 --> 00:01:53,410 -So we need to look a little more in depth into this grid. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt deleted file mode 100644 index b91462c..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/12 - Partition Testing - lang_en_vs4.srt +++ /dev/null @@ -1,67 +0,0 @@ -1 -00:00:00,110 --> 00:00:02,570 -So let me use a slightly expanded version of this - -2 -00:00:02,570 --> 00:00:05,560 -grid. Although we're indeed looking at a needle in a haystack. - -3 -00:00:05,560 --> 00:00:09,560 -And failing inputs are generally sparse, very sparse, in the input - -4 -00:00:09,560 --> 00:00:12,890 -domain. However, they tend to be dense in some parts of - -5 -00:00:12,890 --> 00:00:15,860 -the domain. Like here or here. So how can we leverage - -6 -00:00:15,860 --> 00:00:18,920 -this? The fact that the failures are dense in some subdomains? - -7 -00:00:18,920 --> 00:00:22,290 -As it turns out, the domain is naturally split into partitions. - -8 -00:00:22,290 --> 00:00:25,340 -Where partitions are areas of the domain that are treated homogeneously - -9 -00:00:25,340 --> 00:00:28,070 -by the software. And this is what happens, that normally, - -10 -00:00:28,070 --> 00:00:31,020 -failures tend to be dense in this partitions. So the way - -11 -00:00:31,020 --> 00:00:34,000 -to leverage this characteristic of failures, is that we don't know - -12 -00:00:34,000 --> 00:00:36,950 -want to pick inputs randomly, in the input domain. Just here - -13 -00:00:36,950 --> 00:00:39,460 -and there. Rather we want to do two things. First we - -14 -00:00:39,460 --> 00:00:43,300 -want to identify partitions of our domain. And second we want - -15 -00:00:43,300 --> 00:00:46,950 -to select inputs from each partition. And by doing so, we - -16 -00:00:46,950 --> 00:00:50,300 -can dramatically increase our chances to reveal faults in the code. - -17 -00:00:50,300 --> 00:00:54,170 -So the name that is normally used for this process, is partition testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt deleted file mode 100644 index 6738346..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/13 - Partition Testing Example - lang_en_vs4.srt +++ /dev/null @@ -1,167 +0,0 @@ -1 -00:00:00,120 --> 00:00:02,969 -So let's look at how this will work with an example. - -2 -00:00:02,969 --> 00:00:06,210 -I'm going to use this simple program that takes two inputs. The - -3 -00:00:06,210 --> 00:00:08,790 -first input is a string, str, and the second one is - -4 -00:00:08,790 --> 00:00:11,575 -an integer, size. And the problem is called split. And as - -5 -00:00:11,575 --> 00:00:14,363 -the name says what it does is to take this string, - -6 -00:00:14,363 --> 00:00:17,491 -str, and split it into sub string, into chunks of size - -7 -00:00:17,491 --> 00:00:21,550 -characters each. So how do we identify some possible partitions for - -8 -00:00:21,550 --> 00:00:25,620 -this program? If we consider the input size, we can identify - -9 -00:00:25,620 --> 00:00:29,630 -three neutral partitions which are size less than 0. For example, - -10 -00:00:29,630 --> 00:00:32,259 -we want to test how the program behaves. But if we pass an - -11 -00:00:32,259 --> 00:00:36,100 -incorrect size, size equal to 0, which is also a partition. In - -12 -00:00:36,100 --> 00:00:39,390 -this case, a partition with a single element. And the third case - -13 -00:00:39,390 --> 00:00:42,540 -is size greater than 0, which I will consider to be - -14 -00:00:42,540 --> 00:00:44,960 -kind of the standard case. And actually let me do a, you - -15 -00:00:44,960 --> 00:00:48,220 -know, slight aggression so when I was talking about designer bias. So - -16 -00:00:48,220 --> 00:00:50,630 -this is a case in which designer bias might not make you - -17 -00:00:50,630 --> 00:00:53,050 -think of using size less than 0 because you read the - -18 -00:00:53,050 --> 00:00:56,210 -spec. And you sort of assume that the size will be positive. - -19 -00:00:56,210 --> 00:00:58,556 -Whereas the right thing to do when we test is to consider - -20 -00:00:58,556 --> 00:01:01,700 -the complete domain rather than just parts of it. So now let's - -21 -00:01:01,700 --> 00:01:04,760 -look at string, str, and let's see what kind of sub - -22 -00:01:04,760 --> 00:01:06,538 -domains we could identify for this - -23 -00:01:06,538 --> 00:01:08,670 -parameter. And notice another important aspect - -24 -00:01:08,670 --> 00:01:12,290 -here is that we treat each different part of the input independently, - -25 -00:01:12,290 --> 00:01:15,760 -which also helps breaking down the problem. One interesting sub domain is - -26 -00:01:15,760 --> 00:01:18,980 -the domain that includes all the strings whose length is less than - -27 -00:01:18,980 --> 00:01:22,310 -size. So all the strings that will not be displayed. Another subdomain - -28 -00:01:22,310 --> 00:01:25,000 -is all the strings with length which is between the value of - -29 -00:01:25,000 --> 00:01:28,350 -size and twice the value of size. A third subdomain is the one - -30 -00:01:28,350 --> 00:01:31,820 -including all the strings whose length is greater than twice the value - -31 -00:01:31,820 --> 00:01:35,140 -of size. And we can continue and identify more and more subdomain. - -32 -00:01:35,140 --> 00:01:38,350 -The key thing here is that we have to do that based - -33 -00:01:38,350 --> 00:01:41,180 -on the domain. So we need to adapt what we just did here - -34 -00:01:41,180 --> 00:01:44,620 -based on, on the specific domain involved and on the type - -35 -00:01:44,620 --> 00:01:47,190 -of data in this domain. So at this point we said that - -36 -00:01:47,190 --> 00:01:49,630 -there were two steps. One was to identify the subdomains and - -37 -00:01:49,630 --> 00:01:52,990 -the second one was to pick values in this subdomain. The values - -38 -00:01:52,990 --> 00:01:55,320 -that we'll actually use for the testing. In this case, we - -39 -00:01:55,320 --> 00:01:58,218 -do not want to just pick any value. Rather we want to - -40 -00:01:58,218 --> 00:01:59,871 -pick values that are particularly - -41 -00:01:59,871 --> 00:02:02,710 -interesting, particularly representative. So what does - -42 -00:02:02,710 --> 00:02:05,800 -that mean? Well, we're going to do that based on an intuitive idea. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt deleted file mode 100644 index 3d54ad4..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/14 - Boundary Values - lang_en_vs4.srt +++ /dev/null @@ -1,55 +0,0 @@ -1 -00:00:00,150 --> 00:00:02,590 -So let's go back again to our domain, with - -2 -00:00:02,590 --> 00:00:06,580 -all the sub-domains identified. And the basic idea, or the - -3 -00:00:06,580 --> 00:00:09,750 -intuitive idea I was talking about, is that errors tend - -4 -00:00:09,750 --> 00:00:12,425 -to occur at the boundary of a domain, or a - -5 -00:00:12,425 --> 00:00:15,510 -sub-domain. Like in this case. And why? Because these - -6 -00:00:15,510 --> 00:00:19,360 -are the cases that are less understood by the developers. - -7 -00:00:19,360 --> 00:00:22,155 -Like for example, the last iteration of a loop, or - -8 -00:00:22,155 --> 00:00:25,150 -a special value like zero for integers. So if this - -9 -00:00:25,150 --> 00:00:29,590 -is true, what we want to do is to select inputs at these - -10 -00:00:29,590 --> 00:00:32,189 -boundaries. And this is complementary to partition - -11 -00:00:32,189 --> 00:00:34,270 -testing, in the sense that partition testing - -12 -00:00:34,270 --> 00:00:38,160 -will identify the partitions in which we want to select inputs, and boundary - -13 -00:00:38,160 --> 00:00:40,570 -testing. So the selection of boundary values - -14 -00:00:40,570 --> 00:00:43,060 -will help select inputs in these partitions. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt deleted file mode 100644 index b344802..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/15 - Boundary Values Example - lang_en_vs4.srt +++ /dev/null @@ -1,99 +0,0 @@ -1 -00:00:00,200 --> 00:00:03,320 -So now let's go back to our split example. Let me - -2 -00:00:03,320 --> 00:00:05,790 -rearrange things a little bit to make more room. So now I'm - -3 -00:00:05,790 --> 00:00:09,160 -going to put the domains for size and for strength one next to - -4 -00:00:09,160 --> 00:00:12,210 -the other. So let's look at what some possible inputs will be - -5 -00:00:12,210 --> 00:00:15,140 -for the sub domains that we identified when we use the idea - -6 -00:00:15,140 --> 00:00:17,790 -of selecting input of the boundary. If we look at the first - -7 -00:00:17,790 --> 00:00:21,860 -subdomain, size less than zero, one reasonable input is, size equals to - -8 -00:00:21,860 --> 00:00:25,360 -minus 1, because minus 1 is the boundary value for the domain - -9 -00:00:25,360 --> 00:00:28,530 -of the integers less than zero. If we look at the third - -10 -00:00:28,530 --> 00:00:32,119 -subdomain, possibly interesting case is the one of size of equal to - -11 -00:00:32,119 --> 00:00:34,900 -1, for the same reasoning that we used for the previous subdomain, - -12 -00:00:34,900 --> 00:00:37,670 -for size less than zero. And, let's try to select another one - -13 -00:00:37,670 --> 00:00:40,870 -for this subdomain, for the integers greater than zero. If there is - -14 -00:00:40,870 --> 00:00:44,200 -a concept of maximal integer, we can select that one as our - -15 -00:00:44,200 --> 00:00:47,140 -boundary value. And of course we could select much more, but this - -16 -00:00:47,140 --> 00:00:50,630 -is just to give you an idea. Other possible inputs. One interesting - -17 -00:00:50,630 --> 00:00:53,310 -example for the first one, string with length less than - -18 -00:00:53,310 --> 00:00:57,690 -size will be a string with length size minus one. Again - -19 -00:00:57,690 --> 00:01:00,240 -this is the boundary value for this domain. And we could - -20 -00:01:00,240 --> 00:01:03,500 -continue in this way like for example selecting a string who's - -21 -00:01:03,500 --> 00:01:07,310 -length is exactly size as a boundary value for this - -22 -00:01:07,310 --> 00:01:10,490 -other domain. Instant one, and we look back actually to this - -23 -00:01:10,490 --> 00:01:12,940 -example and look at it in a more extensive way when - -24 -00:01:12,940 --> 00:01:15,690 -we actually talk about a specific method for doing this kind - -25 -00:01:15,690 --> 00:01:16,340 -of process. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt deleted file mode 100644 index 5164f1a..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/16 - Deriving Test Case Specifications - lang_en_vs4.srt +++ /dev/null @@ -1,199 +0,0 @@ -1 -00:00:00,140 --> 00:00:02,790 -Now, let's go back to our systematic functional testing - -2 -00:00:02,790 --> 00:00:05,540 -approach and all the steps in this process. So - -3 -00:00:05,540 --> 00:00:07,680 -far we've seen the first step and the second - -4 -00:00:07,680 --> 00:00:10,340 -step. Now we're going to look at this step in which, - -5 -00:00:10,340 --> 00:00:13,290 -once we have identified the values of interest, we - -6 -00:00:13,290 --> 00:00:18,110 -derive test case specifications for these values, or using these - -7 -00:00:18,110 --> 00:00:21,300 -values. And the test case specification defines how the - -8 -00:00:21,300 --> 00:00:25,230 -values should be put together when actually testing the system. - -9 -00:00:25,230 --> 00:00:29,110 -And test case specification describe how these values should be put - -10 -00:00:29,110 --> 00:00:32,360 -together when testing the system. So let me go back one more - -11 -00:00:32,360 --> 00:00:34,780 -time to our split program, so that we can use the - -12 -00:00:34,780 --> 00:00:37,450 -information that we already computed. At this point what we have is - -13 -00:00:37,450 --> 00:00:41,670 -some possible inputs for "string," our first parameter, and for "size," - -14 -00:00:41,670 --> 00:00:44,410 -our second parameter. And we want to put them together, to generate - -15 -00:00:44,410 --> 00:00:47,080 -the description of what the test case should be. So let - -16 -00:00:47,080 --> 00:00:50,420 -me once more rearrange this a little bit. I first remove the - -17 -00:00:50,420 --> 00:00:53,360 -description of the subdomains, because we won't use them in this step. - -18 -00:00:53,360 --> 00:00:55,800 -And I moved out the set of all our possible inputs, that we're - -19 -00:00:55,800 --> 00:00:59,470 -going to combine to create the test case specification. And one possible way - -20 -00:00:59,470 --> 00:01:03,320 -of doing that is simply to combine the values for the first parameter, - -21 -00:01:03,320 --> 00:01:06,370 -and the values for the second parameter. So the Cartesian product. So - -22 -00:01:06,370 --> 00:01:09,060 -if we do that, what we will obtain is, for example, if we - -23 -00:01:09,060 --> 00:01:12,470 -consider the first possible input, size is equal to minus 1, we can - -24 -00:01:12,470 --> 00:01:15,510 -combine it with these two possible inputs for string, and we will get - -25 -00:01:15,510 --> 00:01:18,680 -size is equal to minus 1 string with length minus 2, or - -26 -00:01:18,680 --> 00:01:21,680 -size is equal to minus 1 string with length minus 1. And we'll - -27 -00:01:21,680 --> 00:01:24,200 -go back in a second to see what this means. Now if we - -28 -00:01:24,200 --> 00:01:27,510 -consider the second possible value for size, size is equal to one, we - -29 -00:01:27,510 --> 00:01:30,260 -also have two cases so the first one in this case that will - -30 -00:01:30,260 --> 00:01:34,030 -be considered a string with length zero. So the antistring. And we can - -31 -00:01:34,030 --> 00:01:37,410 -continue combining this value, but one thing I want to point out is - -32 -00:01:37,410 --> 00:01:40,570 -that if we just go in this straight forward and brute force sort - -33 -00:01:40,570 --> 00:01:43,390 -of way, we will obtain many combinations that don't make any sense, - -34 -00:01:43,390 --> 00:01:46,500 -like for example, this combination which doesn't make any sense because we can - -35 -00:01:46,500 --> 00:01:50,410 -not create the string with length minus 2. Similar for this combination, because - -36 -00:01:50,410 --> 00:01:53,190 -then by the same token, we cannot raise things with length minus 1. - -37 -00:01:53,190 --> 00:01:55,730 -And so there's a lot of cases that we will have to eliminate - -38 -00:01:55,730 --> 00:01:59,380 -afterwards. So what we're going to see in a few minutes is a possible - -39 -00:01:59,380 --> 00:02:02,970 -way in which we can avoid producing these meaningless cases. And at the - -40 -00:02:02,970 --> 00:02:06,380 -same time, keep under control, the number of test cases that we generate. - -41 -00:02:06,380 --> 00:02:09,070 -So lets go back for the last time to our steps - -42 -00:02:09,070 --> 00:02:11,980 -for systematic functional testing. What we just did was to derive - -43 -00:02:11,980 --> 00:02:15,040 -test case specification from a set of relevant inputs. The following - -44 -00:02:15,040 --> 00:02:18,420 -step is to use these test case specifications to generate actual test - -45 -00:02:18,420 --> 00:02:21,170 -cases. And this is normally a fairly mechanical step in the - -46 -00:02:21,170 --> 00:02:23,900 -sense that we just have to instantiate what is in the test - -47 -00:02:23,900 --> 00:02:27,970 -case specification as actual test cases. And it's really dependent on - -48 -00:02:27,970 --> 00:02:32,300 -the specific type of partitions and values identified on the specific context. - -49 -00:02:32,300 --> 00:02:35,100 -So instead of looking at that here in the, in the abstract, - -50 -00:02:35,100 --> 00:02:37,480 -I'm going to show you with an example later on, in the lesson. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt deleted file mode 100644 index 938e80c..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/17 - Category Partition Method - lang_en_vs4.srt +++ /dev/null @@ -1,99 +0,0 @@ -1 -00:00:00,200 --> 00:00:03,690 -What we will discuss next is a specific black-box testing - -2 -00:00:03,690 --> 00:00:07,240 -approach. So a specific instance of the general approach that - -3 -00:00:07,240 --> 00:00:10,814 -we just saw. And this approach is the category-partition method, - -4 -00:00:10,814 --> 00:00:13,418 -and was defined by Ostrand & Balcer in 1988 in - -5 -00:00:13,418 --> 00:00:16,760 -an article to the peer [UNKNOWN] communication of the ACM. - -6 -00:00:16,760 --> 00:00:19,490 -So this is a method for going from a specification, - -7 -00:00:19,490 --> 00:00:21,370 -a description of the system, to a set of test - -8 -00:00:21,370 --> 00:00:26,170 -cases like any other black-box testing approach by following six steps. - -9 -00:00:26,170 --> 00:00:28,370 -So let's look at what these steps are. The first - -10 -00:00:28,370 --> 00:00:31,850 -step is to identify independently testable features and this is - -11 -00:00:31,850 --> 00:00:33,870 -a step that we are familiar with because its exactly - -12 -00:00:33,870 --> 00:00:36,950 -the same step that we performed in the generic black box - -13 -00:00:36,950 --> 00:00:39,480 -testing approach that we just discussed. The second step is - -14 -00:00:39,480 --> 00:00:42,250 -to identify categories. Then the next step is to partition - -15 -00:00:42,250 --> 00:00:45,070 -categories into choices. Identify constraints - -16 -00:00:45,070 --> 00:00:47,530 -among choices. Produce and evaluate - -17 -00:00:47,530 --> 00:00:51,330 -test case specifications. And finally, the sixth step is to generate - -18 -00:00:51,330 --> 00:00:54,100 -test cases from test case specifications. So two of - -19 -00:00:54,100 --> 00:00:56,970 -the key elements in these six steps are the two - -20 -00:00:56,970 --> 00:00:59,430 -that give the name to the technique so the identification - -21 -00:00:59,430 --> 00:01:02,240 -of the categories and the partition of these categories into - -22 -00:01:02,240 --> 00:01:04,810 -choices. What we're going to do next is to go - -23 -00:01:04,810 --> 00:01:07,670 -and look at each one of the steps independently, except - -24 -00:01:07,670 --> 00:01:10,285 -for the first one. Because we're already familiar with that - -25 -00:01:10,285 --> 00:01:12,650 -step, and this method doesn't really add much to it. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt deleted file mode 100644 index a1eb4dd..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/18 - Identify Categories - lang_en_vs4.srt +++ /dev/null @@ -1,127 +0,0 @@ -1 -00:00:00,140 --> 00:00:02,980 -In the second step of the category partition technique, the goal is - -2 -00:00:02,980 --> 00:00:05,630 -to Identify Categories. Where categories are - -3 -00:00:05,630 --> 00:00:08,650 -characteristics of each input element. So - -4 -00:00:08,650 --> 00:00:11,490 -let me illustrate what that means using an example. And to do - -5 -00:00:11,490 --> 00:00:14,440 -that I'm going to use again the example of the split program, as - -6 -00:00:14,440 --> 00:00:16,830 -we are already familiar with it and we kind of already played - -7 -00:00:16,830 --> 00:00:20,110 -with it. When we were talking about the generic black box approach. - -8 -00:00:20,110 --> 00:00:22,293 -So let me bring back the program, and let me remind you - -9 -00:00:22,293 --> 00:00:25,440 -that what the program does is to take two inputs, a string and - -10 -00:00:25,440 --> 00:00:28,560 -the size, and it breaks down the string into chunks, - -11 -00:00:28,560 --> 00:00:31,300 -whose length is size. If we look at the split program - -12 -00:00:31,300 --> 00:00:34,330 -there are two input elements, str and size so we going to - -13 -00:00:34,330 --> 00:00:37,930 -identify categories for these two. So starting from str, what are - -14 -00:00:37,930 --> 00:00:41,228 -the interesting characteristics of the string? In, in this step - -15 -00:00:41,228 --> 00:00:44,948 -you're going to use your domain knowledge, your understanding of what a - -16 -00:00:44,948 --> 00:00:47,986 -string is, and for example we might identify the length of - -17 -00:00:47,986 --> 00:00:50,528 -the string and the content of the string as the two - -18 -00:00:50,528 --> 00:00:53,840 -main characteristics that we want to focus on. If we now - -19 -00:00:53,840 --> 00:00:57,540 -move our focus to size, the only characteristic I can really - -20 -00:00:57,540 --> 00:01:00,340 -think of for an integer is its value. So that's what - -21 -00:01:00,340 --> 00:01:02,470 -I'm going to mark here. So at the end of the step - -22 -00:01:02,470 --> 00:01:05,030 -what we have is that we have two categories. So two - -23 -00:01:05,030 --> 00:01:08,930 -interesting characteristics for the string input str, which are the length - -24 -00:01:08,930 --> 00:01:12,940 -and the content. And one category for the integer input size - -25 -00:01:12,940 --> 00:01:16,480 -which is its value. And notice that there's not only one solution. - -26 -00:01:16,480 --> 00:01:18,200 -So there's not only one possibility. - -27 -00:01:18,200 --> 00:01:20,384 -So that the specific characteristics that you - -28 -00:01:20,384 --> 00:01:22,800 -will identify are somehow subjective. But the - -29 -00:01:22,800 --> 00:01:25,630 -important point is that you identify characteristics - -30 -00:01:25,630 --> 00:01:29,259 -that are meaningful and they sort of cover the main aspects of the - -31 -00:01:29,259 --> 00:01:31,200 -inputs, which is the case for the - -32 -00:01:31,200 --> 00:01:33,440 -categories that we've identified in this example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt deleted file mode 100644 index a411c30..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/19 - Partition Categories into Choices - lang_en_vs4.srt +++ /dev/null @@ -1,151 +0,0 @@ -1 -00:00:00,230 --> 00:00:03,250 -Now we move to the next step, which involves partitioning the - -2 -00:00:03,250 --> 00:00:07,680 -categories that we just identified into choices. And these choices are the - -3 -00:00:07,680 --> 00:00:11,930 -interesting cases for each category. So the interesting subdomains for each one - -4 -00:00:11,930 --> 00:00:14,810 -of these categories. So once more, lets look at that using our - -5 -00:00:14,810 --> 00:00:18,110 -example, the split program. So lets start by considering length. What are - -6 -00:00:18,110 --> 00:00:20,710 -the interesting cases when we think about the length of a string? - -7 -00:00:20,710 --> 00:00:23,220 -Some of those we already saw, one interesting case is the case - -8 -00:00:23,220 --> 00:00:26,190 -of the length of size zero, so a string with no characters. - -9 -00:00:26,190 --> 00:00:29,030 -Another interesting case is the one in which the length of the - -10 -00:00:29,030 --> 00:00:31,780 -string is size minus one, so the string is just one character - -11 -00:00:31,780 --> 00:00:34,610 -short of the size at which it will be cut by the - -12 -00:00:34,610 --> 00:00:38,160 -split program. And we can continue along these lines, so we will select - -13 -00:00:38,160 --> 00:00:42,950 -size, size plus one, size twice the value of size minus one, - -14 -00:00:42,950 --> 00:00:45,560 -and so on and so forth. But even without listing all of - -15 -00:00:45,560 --> 00:00:47,500 -those, I'm sure you get the idea of what it means to - -16 -00:00:47,500 --> 00:00:49,780 -identify this interesting cases. Let's see - -17 -00:00:49,780 --> 00:00:51,702 -the movements that are considering the content. - -18 -00:00:51,702 --> 00:00:53,990 -So without the interesting cases when we think about the content - -19 -00:00:53,990 --> 00:00:56,520 -of the string. So possible interesting case is the string that - -20 -00:00:56,520 --> 00:01:00,660 -contains only spaces. Why? Well maybe because a split is written - -21 -00:01:00,660 --> 00:01:04,290 -spaces in a special way. Similarly a string that contains special - -22 -00:01:04,290 --> 00:01:06,780 -characters, like non printable characters, - -23 -00:01:06,780 --> 00:01:09,020 -like tabulation characters, new line might - -24 -00:01:09,020 --> 00:01:12,240 -also be an interesting case, something that we want to test. Also - -25 -00:01:12,240 --> 00:01:14,280 -in this case we could continue and go on and on. - -26 -00:01:14,280 --> 00:01:17,250 -So basically here you just want to put all the interesting cases - -27 -00:01:17,250 --> 00:01:20,010 -that you can think of when you consider the content - -28 -00:01:20,010 --> 00:01:22,440 -of a string. Now let's move to the value as the - -29 -00:01:22,440 --> 00:01:25,740 -next category. So the value of the input size. And - -30 -00:01:25,740 --> 00:01:29,280 -here we might want to consider a size zero, special case, a - -31 -00:01:29,280 --> 00:01:33,640 -normal situation, like size greater than zero, another special case, - -32 -00:01:33,640 --> 00:01:36,420 -size less than zero or maxint. And these are, if you - -33 -00:01:36,420 --> 00:01:39,420 -remember, I accepted the cases that we consider when we look - -34 -00:01:39,420 --> 00:01:42,350 -at this example, before. And also here we can continue and - -35 -00:01:42,350 --> 00:01:43,970 -go on and on. So, at the end of the - -36 -00:01:43,970 --> 00:01:46,860 -step, what we have is a set of interesting cases - -37 -00:01:46,860 --> 00:01:49,220 -for each one of the categories, and now we can - -38 -00:01:49,220 --> 00:01:51,150 -start to think about how we want to combine them. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt deleted file mode 100644 index 2af56c5..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt +++ /dev/null @@ -1,103 +0,0 @@ -1 -00:00:00,450 --> 00:00:03,400 -As we said at the end of the previous lesson, black-box testing is - -2 -00:00:03,400 --> 00:00:06,230 -the testing of the software when we look at it as a black box, - -3 -00:00:06,230 --> 00:00:09,110 -as a closed box, without looking at it inside, without looking at the - -4 -00:00:09,110 --> 00:00:10,750 -code. And there are several advantages in - -5 -00:00:10,750 --> 00:00:12,680 -using black-box testing. So let me recap - -6 -00:00:12,680 --> 00:00:15,590 -those advantages, some of which we already mentioned, and let me also expand - -7 -00:00:15,590 --> 00:00:18,220 -on that a little bit. The first advantage of when I mentioned is that - -8 -00:00:18,220 --> 00:00:22,760 -black box focuses on the domain, on the input domain of the software. And - -9 -00:00:22,760 --> 00:00:25,750 -as such, we can use it to make sure that we are actually covering - -10 -00:00:25,750 --> 00:00:28,400 -this domain, that we are actually covering the important behaviors of - -11 -00:00:28,400 --> 00:00:32,200 -the software. A second advantage is that black box testing does not - -12 -00:00:32,200 --> 00:00:35,402 -need the code. What that means is that you can perform early - -13 -00:00:35,402 --> 00:00:38,801 -test design. So you can start designing and writing your test cases, - -14 -00:00:38,801 --> 00:00:41,423 -even before writing your code, so that when the code is - -15 -00:00:41,423 --> 00:00:44,520 -ready, we can test it right away. And that helps prevent a - -16 -00:00:44,520 --> 00:00:48,054 -problem that is very typical in real life software development, which is - -17 -00:00:48,054 --> 00:00:50,790 -getting an idea of the project and having no time to create - -18 -00:00:50,790 --> 00:00:53,040 -the tests. In this way, we already have the tests, so - -19 -00:00:53,040 --> 00:00:56,170 -we just have to run them. Another advantage is that black-box testing - -20 -00:00:56,170 --> 00:00:59,530 -can catch logic defects, because it focuses on the description of what - -21 -00:00:59,530 --> 00:01:02,440 -the software should do, and therefore on its logic. If we derive - -22 -00:01:02,440 --> 00:01:05,250 -test cases from such description, then we can catch these kind of - -23 -00:01:05,250 --> 00:01:07,510 -problems. And finally, black-box testing is - -24 -00:01:07,510 --> 00:01:09,900 -applicable at all granularity levels, which - -25 -00:01:09,900 --> 00:01:13,490 -means that we can use black-box testing in unit testing, integration testing, - -26 -00:01:13,490 --> 00:01:16,290 -system testing, and so on. We can use it at all levels. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt deleted file mode 100644 index 0278cc8..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/20 - Identify Constraints Among Choices - lang_en_vs4.srt +++ /dev/null @@ -1,239 +0,0 @@ -1 -00:00:00,160 --> 00:00:02,080 -Something that we saw when we were looking at the split - -2 -00:00:02,080 --> 00:00:05,760 -program before is that if we just combine the interesting values that - -3 -00:00:05,760 --> 00:00:08,490 -we identify, we might end up with a lot of cases. - -4 -00:00:08,490 --> 00:00:11,030 -And I mentioned that we, we're going to look at some way of - -5 -00:00:11,030 --> 00:00:13,720 -addressing that problem. And this is exactly what happens in the - -6 -00:00:13,720 --> 00:00:17,420 -next step of the category partition method, in which we identify constraints - -7 -00:00:17,420 --> 00:00:20,510 -among choices. And why do we identify these constraints? We do - -8 -00:00:20,510 --> 00:00:23,440 -that to eliminate meaningless combinations of - -9 -00:00:23,440 --> 00:00:25,460 -inputs. If you remember, for example, - -10 -00:00:25,460 --> 00:00:27,260 -we had the case in which we were trying to - -11 -00:00:27,260 --> 00:00:30,110 -create a string with a size less than 0, which - -12 -00:00:30,110 --> 00:00:32,930 -doesn't make any sense. And very importantly, we also do - -13 -00:00:32,930 --> 00:00:37,070 -that to reduce the number of test cases. Because every time - -14 -00:00:37,070 --> 00:00:40,930 -we constrain one of the possible choices, we eliminate possible - -15 -00:00:40,930 --> 00:00:43,250 -test cases, so we can use it to keep under - -16 -00:00:43,250 --> 00:00:45,960 -control the number of tests that we generate. There are - -17 -00:00:45,960 --> 00:00:47,571 -three types of properties. The - -18 -00:00:47,571 --> 00:00:50,610 -pair property...if, error properties, and properties - -19 -00:00:50,610 --> 00:00:52,610 -of type single. So we're going to look at what these - -20 -00:00:52,610 --> 00:00:56,230 -properties mean, using, once more, our example of the split program. - -21 -00:00:56,230 --> 00:00:58,750 -In particular, we're going to use some of the choices that we - -22 -00:00:58,750 --> 00:01:02,080 -identified earlier. So let's look, for example, at choice 0, for - -23 -00:01:02,080 --> 00:01:04,599 -category length of the string. All we can say is that, - -24 -00:01:04,599 --> 00:01:07,860 -if the length is 0, this define a special property of - -25 -00:01:07,860 --> 00:01:11,160 -the string. And that was specified in this way by saying - -26 -00:01:11,160 --> 00:01:15,610 -that this identifies property zerovalue. So every time that we use - -27 -00:01:15,610 --> 00:01:19,080 -this choice, zerovalue is defined. At this point, we can use - -28 -00:01:19,080 --> 00:01:20,550 -this to exclude some meaningless - -29 -00:01:20,550 --> 00:01:23,170 -combinations. For instance, consider special characters. - -30 -00:01:23,170 --> 00:01:25,130 -If we have a string of length 0, which means a - -31 -00:01:25,130 --> 00:01:26,850 -string with no characters. Obviously, there - -32 -00:01:26,850 --> 00:01:28,760 -cannot be special characters. So, considering - -33 -00:01:28,760 --> 00:01:31,020 -this combination will just be a waste of time. So what - -34 -00:01:31,020 --> 00:01:33,770 -we do is that we specify next to this choice, that we - -35 -00:01:33,770 --> 00:01:36,360 -will only consider this if length is not 0. And we - -36 -00:01:36,360 --> 00:01:41,110 -do this by saying that we consider this only if not zerovalue. - -37 -00:01:41,110 --> 00:01:43,970 -So, if zerovalue is not defined. So this pair is an - -38 -00:01:43,970 --> 00:01:47,640 -example of a property...if case. Define a property and use that - -39 -00:01:47,640 --> 00:01:49,770 -property. Now let's look at a case in which we might - -40 -00:01:49,770 --> 00:01:52,580 -want to use an error property. For instance, when we look at - -41 -00:01:52,580 --> 00:01:56,480 -the category value for the input size, the choice value less - -42 -00:01:56,480 --> 00:01:59,510 -than 0 is an erroneous choice. So it's a choice that - -43 -00:01:59,510 --> 00:02:02,630 -we selected to test a possibly erroneous situation, so we can - -44 -00:02:02,630 --> 00:02:06,160 -mark this as an error property. And what that means is that - -45 -00:02:06,160 --> 00:02:09,949 -when generating a combination of choices, we will consider this only - -46 -00:02:09,949 --> 00:02:12,990 -once because we assume that we just want to test this error condition - -47 -00:02:12,990 --> 00:02:16,260 -once. Finally, the single property is a property that we use - -48 -00:02:16,260 --> 00:02:19,050 -when we want to limit the number of test cases. And it's - -49 -00:02:19,050 --> 00:02:21,900 -similar as an effect to error. It just has a different - -50 -00:02:21,900 --> 00:02:24,410 -meaning. So what we do when we use the single property is - -51 -00:02:24,410 --> 00:02:27,420 -that we're saying that this choice, we want to use in only - -52 -00:02:27,420 --> 00:02:31,120 -one combination. So don't combine it multiple times. And that, of course, - -53 -00:02:31,120 --> 00:02:34,260 -given the combinatorial nature of the problem, cuts down dramatically - -54 -00:02:34,260 --> 00:02:36,370 -the number of test cases. And we might use, for - -55 -00:02:36,370 --> 00:02:39,500 -instance, the single property for maxint, which means that we - -56 -00:02:39,500 --> 00:02:41,890 -will only have one test case in which the size is - -57 -00:02:41,890 --> 00:02:44,300 -equal to maxint. We're actually going to see a demo - -58 -00:02:44,300 --> 00:02:46,880 -on this topic so we'll have more chances of seeing how - -59 -00:02:46,880 --> 00:02:49,370 -properties work in practice and how good they are at - -60 -00:02:49,370 --> 00:02:53,022 -eliminating meaningless combinations and at reducing the number of test cases. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt deleted file mode 100644 index ebf8098..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/21 - Produce and Evaluate Test Case Specifications - lang_en_vs4.srt +++ /dev/null @@ -1,111 +0,0 @@ -1 -00:00:00,130 --> 00:00:03,200 -Before getting to our demo, we still have two steps to consider. - -2 -00:00:03,200 --> 00:00:06,570 -The first step corresponds to the identification of the test case specifications - -3 -00:00:06,570 --> 00:00:09,890 -in our general systematic approach. And in fact, it's called produce and - -4 -00:00:09,890 --> 00:00:13,410 -evaluate test case specifications. This is a step than can be completely - -5 -00:00:13,410 --> 00:00:16,590 -automated given the results of the previous steps. And the final result - -6 -00:00:16,590 --> 00:00:19,860 -of this step is the production of a set of test frames. - -7 -00:00:19,860 --> 00:00:22,820 -Where a test frame is the specification of a test. Let me - -8 -00:00:22,820 --> 00:00:25,330 -show you an example of this. What we are looking at here - -9 -00:00:25,330 --> 00:00:28,380 -is a test frame for the program split. Test frames are normally - -10 -00:00:28,380 --> 00:00:31,460 -identified by a sequence number. But in this case we are looking at - -11 -00:00:31,460 --> 00:00:34,290 -the 30th six test frame. And what they do is simply to - -12 -00:00:34,290 --> 00:00:38,280 -specify the characteristic of the inputs for that test. In this case, since - -13 -00:00:38,280 --> 00:00:40,920 -we have two inputs, we have two entries, the first one for - -14 -00:00:40,920 --> 00:00:44,010 -string str tells us that the length of the string has to be - -15 -00:00:44,010 --> 00:00:47,840 -size minus 1, and that the string has to contain special characters. And - -16 -00:00:47,840 --> 00:00:50,380 -for size, it tells us that the value of size has to be - -17 -00:00:50,380 --> 00:00:53,340 -greater than zero. As the title says, this step is meant - -18 -00:00:53,340 --> 00:00:56,790 -to produce but also evaluate the case specification. What does it mean - -19 -00:00:56,790 --> 00:01:00,070 -to evaluate? One of the advantages of this approach is that we - -20 -00:01:00,070 --> 00:01:03,420 -can easily use it to assess how many test frames and therefore - -21 -00:01:03,420 --> 00:01:06,120 -how many test cases we will generate with the current least - -22 -00:01:06,120 --> 00:01:09,340 -of categories, choices and constraints. And the beauty of this is that - -23 -00:01:09,340 --> 00:01:12,320 -if the number is too large we can just add additional constraints - -24 -00:01:12,320 --> 00:01:15,840 -and reduce it. And given then the step is automated we just - -25 -00:01:15,840 --> 00:01:19,030 -add constraints push a button and we get our new set of - -26 -00:01:19,030 --> 00:01:21,580 -test frames. And again we can have a wait it either go - -27 -00:01:21,580 --> 00:01:24,560 -hat or add more constraints if we need to further reduce it - -28 -00:01:24,560 --> 00:01:26,320 -and this is something else that we will see in our demo. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt deleted file mode 100644 index 7d74ae3..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/22 - Generate Test Cases from Test Case Specifications - lang_en_vs4.srt +++ /dev/null @@ -1,91 +0,0 @@ -1 -00:00:00,135 --> 00:00:01,935 -So we get to the last step of the technique in - -2 -00:00:01,935 --> 00:00:05,142 -which once we have generated test case specifications. We create test - -3 -00:00:05,142 --> 00:00:09,180 -cases starting from this specifications. This step mainly consists in a - -4 -00:00:09,180 --> 00:00:12,840 -simple instantiation of frames and it's final result is a set of - -5 -00:00:12,840 --> 00:00:16,840 -concrete tests. For our example, test frame number 36 that we - -6 -00:00:16,840 --> 00:00:19,270 -just saw, this will be the resulting test case, which has - -7 -00:00:19,270 --> 00:00:21,490 -the same ID, so that we can track it and will - -8 -00:00:21,490 --> 00:00:25,520 -specify to concrete values, not just the specification for the input elements. - -9 -00:00:25,520 --> 00:00:29,130 -So string STR will have this value. And the integer size - -10 -00:00:29,130 --> 00:00:31,720 -will have this value. And these two values satisfy what this - -11 -00:00:31,720 --> 00:00:35,260 -test case specification was. Which was, having a string contain special - -12 -00:00:35,260 --> 00:00:38,760 -characters. Here, we have two special characters, like the new line - -13 -00:00:38,760 --> 00:00:41,900 -and the tab. And, we have a size which is greater - -14 -00:00:41,900 --> 00:00:44,430 -than zero, in particular, okay? And this is a test case - -15 -00:00:44,430 --> 00:00:46,880 -that we can actually run on our code. That we can - -16 -00:00:46,880 --> 00:00:50,730 -run on the split program. So, to summarize, we perform six steps - -17 -00:00:50,730 --> 00:00:54,400 -in which we went from a high level description of what the program does, to a - -18 -00:00:54,400 --> 00:00:56,070 -set of concrete test cases. And this is - -19 -00:00:56,070 --> 00:00:57,880 -one of those test cases. So what, what we're - -20 -00:00:57,880 --> 00:01:00,210 -going to do next, we're going to do a mini-demo, - -21 -00:01:00,210 --> 00:01:02,260 -in which we do this for real. We take - -22 -00:01:02,260 --> 00:01:05,440 -the program, we identify categories, choices, constraints, and - -23 -00:01:05,440 --> 00:01:07,800 -we actually generate test frames and then test cases. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt deleted file mode 100644 index 51c2e52..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/23 - Category Partition Demo - lang_en_vs4.srt +++ /dev/null @@ -1,867 +0,0 @@ -1 -00:00:01,290 --> 00:00:03,210 -In this demo, we're going to do exactly what we - -2 -00:00:03,210 --> 00:00:06,370 -did just now in the lesson. We're going to use - -3 -00:00:06,370 --> 00:00:09,250 -the category partition method to go from a high-level - -4 -00:00:09,250 --> 00:00:12,930 -description of a piece of software of a program to - -5 -00:00:12,930 --> 00:00:15,512 -a set of test cases for that program. To - -6 -00:00:15,512 --> 00:00:17,640 -do that, we're going to use a simple tool. So - -7 -00:00:17,640 --> 00:00:21,040 -I'm going to show you here the tool that is - -8 -00:00:21,040 --> 00:00:26,380 -called a tsl generator right here. This tool is available - -9 -00:00:26,380 --> 00:00:28,520 -to you, so you can look in the class notes to - -10 -00:00:28,520 --> 00:00:31,900 -see information on how to download it. And together with the tool, - -11 -00:00:31,900 --> 00:00:34,890 -we are also going to provide a manual for the tool, and - -12 -00:00:34,890 --> 00:00:37,680 -a set of files that I'm going to use in this demo. So - -13 -00:00:37,680 --> 00:00:40,420 -you should be able to do exactly what I'm doing. So - -14 -00:00:40,420 --> 00:00:45,110 -again, all of those are available from the class notes. So specifically, - -15 -00:00:45,110 --> 00:00:48,390 -today we're going to write test cases for the grep program. So in - -16 -00:00:48,390 --> 00:00:51,970 -case you're familiar with the grep utility, this is a simplified version - -17 -00:00:51,970 --> 00:00:55,060 -of that utility. So basically the grep utility allows you - -18 -00:00:55,060 --> 00:00:58,410 -to search a file for the occurrences of a given - -19 -00:00:58,410 --> 00:01:01,552 -pattern. So you can invoke it, as it's shown here - -20 -00:01:01,552 --> 00:01:05,570 -in the synopsis, by executing grep, the pattern that you're - -21 -00:01:05,570 --> 00:01:08,310 -looking for, and the filename in which you want to - -22 -00:01:08,310 --> 00:01:10,300 -look for the pattern. And let me read the description - -23 -00:01:10,300 --> 00:01:13,650 -of the grep utility. The grep utility searches files for - -24 -00:01:13,650 --> 00:01:17,110 -a pattern and brings all lines that contain that pattern - -25 -00:01:17,110 --> 00:01:20,900 -on the standard output. A line that contains multiple occurrences of - -26 -00:01:20,900 --> 00:01:24,350 -the pattern is printed only once. The pattern is any sequence - -27 -00:01:24,350 --> 00:01:27,700 -of characters. To include a blank in the pattern, the entire - -28 -00:01:27,700 --> 00:01:31,060 -pattern must be enclosed in single quotes. To include a quote - -29 -00:01:31,060 --> 00:01:34,420 -sign in the pattern, the quote sign must be escaped, which - -30 -00:01:34,420 --> 00:01:36,260 -means that we have to put a slash in front of - -31 -00:01:36,260 --> 00:01:39,290 -the quotes sign. And in general, it is safest to enclose - -32 -00:01:39,290 --> 00:01:42,210 -the entire pattern in single quotes. So this is our high - -33 -00:01:42,210 --> 00:01:45,420 -level description for the program, for the softer system, that - -34 -00:01:45,420 --> 00:01:47,270 -we need to test. So now let me show you - -35 -00:01:47,270 --> 00:01:50,600 -what a possible set of categories and partitions could be - -36 -00:01:50,600 --> 00:01:53,770 -for this program. So what I have here is a - -37 -00:01:53,770 --> 00:01:58,080 -file, a textual file, which contains all the categories and - -38 -00:01:58,080 --> 00:02:02,760 -partitions for the elements that are relevant for my program. - -39 -00:02:02,760 --> 00:02:05,240 -In particular, when we look at the file, we can - -40 -00:02:05,240 --> 00:02:07,809 -see that the file can be characterized by its size. - -41 -00:02:08,889 --> 00:02:12,160 -And in this case, I've got two choices. The file can - -42 -00:02:12,160 --> 00:02:16,050 -be empty or not empty. The second characteristic of the file - -43 -00:02:16,050 --> 00:02:19,490 -that I'm considering is the number of occurrences of the pattern - -44 -00:02:19,490 --> 00:02:22,320 -in the file. And I'm considering that the pattern might not occur - -45 -00:02:22,320 --> 00:02:25,780 -in the file or it might occur once, or multiple times. - -46 -00:02:25,780 --> 00:02:28,264 -I'm not going to go through the rest of the file because we - -47 -00:02:28,264 --> 00:02:31,234 -already covered how to apply the category partition method in the - -48 -00:02:31,234 --> 00:02:34,226 -lesson. So if you had doubts about that, about the method and - -49 -00:02:34,226 --> 00:02:36,952 -how to apply, you might want to go back and watch again the - -50 -00:02:36,952 --> 00:02:40,040 -lesson. What I want to show you here is how you can go - -51 -00:02:40,040 --> 00:02:43,670 -from this information that you have here, that we have derived by - -52 -00:02:43,670 --> 00:02:47,020 -applying the, the first steps of the method, to a set of - -53 -00:02:47,020 --> 00:02:50,650 -test frames, and then, a set of test packs. So to do - -54 -00:02:50,650 --> 00:02:53,240 -that we're going to use the tool that I just mentioned. So let - -55 -00:02:53,240 --> 00:02:56,670 -me bring back my terminal. So first of all, let's see how - -56 -00:02:56,670 --> 00:02:59,570 -we can run the tool. So you have a manual that will explain - -57 -00:02:59,570 --> 00:03:02,180 -all the details on how to build the file that we're - -58 -00:03:02,180 --> 00:03:04,350 -going to feed the tool. So what is the format and so - -59 -00:03:04,350 --> 00:03:07,780 -on. Here I'm just going to see how I can run the - -60 -00:03:07,780 --> 00:03:10,828 -tool. So first of all, let me point out that this was - -61 -00:03:10,828 --> 00:03:15,028 -developed together by professors from the University of California Irvine and - -62 -00:03:15,028 --> 00:03:18,020 -Oregon State University. And as you can see, we can run - -63 -00:03:18,020 --> 00:03:20,968 -TSL generator and specify that we want to see the main - -64 -00:03:20,968 --> 00:03:24,361 -page. So in this case if we run it this, this way, - -65 -00:03:24,361 --> 00:03:27,723 -you'll have some basic information on how to run - -66 -00:03:27,723 --> 00:03:30,720 -the tool. And from the main page you can see - -67 -00:03:30,720 --> 00:03:33,520 -that you can specify the minus c flag and in - -68 -00:03:33,520 --> 00:03:37,360 -this case the TSL generator will report the number of - -69 -00:03:37,360 --> 00:03:41,410 -test frames generated without writing them to output. For - -70 -00:03:41,410 --> 00:03:43,828 -example, you might want to use this as we will do - -71 -00:03:43,828 --> 00:03:46,308 -to see how many tests that you will generate with - -72 -00:03:46,308 --> 00:03:49,630 -a current set of category partitions and choices. The minus - -73 -00:03:49,630 --> 00:03:52,620 -s option will bring the result of the TSL - -74 -00:03:52,620 --> 00:03:55,620 -generator on the standard output. And finally, you can - -75 -00:03:55,620 --> 00:03:58,330 -use minus o to specify an output file, where - -76 -00:03:58,330 --> 00:04:01,010 -to put the output of the program. So let's - -77 -00:04:01,010 --> 00:04:05,070 -at first run our TSL generator by specifying the - -78 -00:04:05,070 --> 00:04:08,620 -minus c option and by bypassing our current set - -79 -00:04:08,620 --> 00:04:11,860 -of category partitions and choices. Okay, so let me - -80 -00:04:11,860 --> 00:04:15,140 -remind you that what the, the tool will do - -81 -00:04:15,140 --> 00:04:17,790 -is what we will do manually. Otherwise, which is - -82 -00:04:17,790 --> 00:04:20,380 -to combine all these choices so as to have one - -83 -00:04:20,380 --> 00:04:23,305 -test case for each combination. So if we do - -84 -00:04:23,305 --> 00:04:27,412 -that, you can see that the tool tells us that - -85 -00:04:27,412 --> 00:04:32,383 -we will generate 7776 test frames in this case. - -86 -00:04:32,383 --> 00:04:34,660 -And this seems to be a little too much for - -87 -00:04:34,660 --> 00:04:36,868 -a program as small as the one that we are - -88 -00:04:36,868 --> 00:04:40,341 -testing. And assume for instance that we don't have the - -89 -00:04:40,341 --> 00:04:43,149 -resources to run this many test cases for, for - -90 -00:04:43,149 --> 00:04:46,518 -the grep program. In addition, consider that in this case, - -91 -00:04:46,518 --> 00:04:50,356 -we're computing all possible combinations of choices. And there's going to - -92 -00:04:50,356 --> 00:04:52,384 -be some combination that do not make sense as we - -93 -00:04:52,384 --> 00:04:54,945 -discussed in the lesson. So what we might want to - -94 -00:04:54,945 --> 00:04:57,051 -do in this case is to go back to our - -95 -00:04:57,051 --> 00:05:03,120 -spec and start adding constraints to eliminate this meaningless combination. - -96 -00:05:03,120 --> 00:05:05,980 -So I'm going to show you the result of doing that. - -97 -00:05:05,980 --> 00:05:08,690 -And I'm going to show you a few examples. - -98 -00:05:08,690 --> 00:05:11,670 -For example here, when the file is empty, I'm going to - -99 -00:05:11,670 --> 00:05:15,010 -define this property empty file. And how am I - -100 -00:05:15,010 --> 00:05:18,490 -going to use this property? Well for example here, it - -101 -00:05:18,490 --> 00:05:20,760 -doesn't make sense to consider the case in which - -102 -00:05:20,760 --> 00:05:24,660 -we have one or many occurrences of the pattern in - -103 -00:05:24,660 --> 00:05:27,020 -the file if the file is empty. Therefore I'm - -104 -00:05:27,020 --> 00:05:31,650 -going to tell the tool that it should consider this specific - -105 -00:05:31,650 --> 00:05:35,780 -choice only if the file is not empty, only if - -106 -00:05:35,780 --> 00:05:38,660 -empty file is not defined. And that will skip, for - -107 -00:05:38,660 --> 00:05:41,330 -example, all of the combinations in which the file is - -108 -00:05:41,330 --> 00:05:44,171 -empty. And I'm trying to generate the test case that has - -109 -00:05:44,171 --> 00:05:46,364 -one occurrence of the pattern in the file, which is - -110 -00:05:46,364 --> 00:05:49,355 -simply not possible. For another example, in case I have - -111 -00:05:49,355 --> 00:05:52,824 -an empty pattern, I define the property empty pattern. And - -112 -00:05:52,824 --> 00:05:56,725 -then I avoid the choices that involve the pattern in case - -113 -00:05:56,725 --> 00:05:59,820 -the pattern is empty. because, for example, I cannot - -114 -00:05:59,820 --> 00:06:03,900 -have quotes in a pattern that is empty. For example, - -115 -00:06:03,900 --> 00:06:06,760 -it doesn't make sense to have blanks. So, one or - -116 -00:06:06,760 --> 00:06:10,250 -more blanks if the pattern is empty. So I'm going to - -117 -00:06:10,250 --> 00:06:14,180 -specify again that this choice should be considered only if - -118 -00:06:14,180 --> 00:06:16,140 -we don't have an empty pattern. And so on and - -119 -00:06:16,140 --> 00:06:20,080 -so forth. So now after I edit these constraints, I - -120 -00:06:20,080 --> 00:06:21,890 -can go back and compute again the number of test - -121 -00:06:21,890 --> 00:06:23,970 -frames and therefore the test cases that will be - -122 -00:06:23,970 --> 00:06:27,530 -generated with these constraints. So let me go again - -123 -00:06:27,530 --> 00:06:30,381 -to my terminal. Okay, so now I'm going to run - -124 -00:06:30,381 --> 00:06:34,061 -my TSL generator again, and I'm going to run it on - -125 -00:06:34,061 --> 00:06:37,389 -the second version of this file. And you can - -126 -00:06:37,389 --> 00:06:40,546 -see that I reduced the, the number of test frames - -127 -00:06:40,546 --> 00:06:43,889 -from about 7800 to about 1700. So it's quite - -128 -00:06:43,889 --> 00:06:46,967 -a, quite a big reduction by eliminating all these combinations - -129 -00:06:46,967 --> 00:06:49,540 -that do not make sense. But let's assume again that - -130 -00:06:49,540 --> 00:06:52,040 -we want to reduce this further so that we don't want to - -131 -00:06:52,040 --> 00:06:55,610 -generate those many test frames and therefore test cases. So - -132 -00:06:55,610 --> 00:06:58,660 -what can we do? We go back to our spec. And - -133 -00:06:58,660 --> 00:07:02,280 -in this case, we start adding error constraints. So if - -134 -00:07:02,280 --> 00:07:05,200 -you remember what we said in the lesson, error constraints are - -135 -00:07:05,200 --> 00:07:08,310 -constraints that indicate a choice that has to do with an - -136 -00:07:08,310 --> 00:07:11,980 -erroneous behaviour. For example, an erroneous input provided to the problem. - -137 -00:07:11,980 --> 00:07:15,210 -So here for instance, we're indicating the presence - -138 -00:07:15,210 --> 00:07:20,060 -of incorrectly enclosing quotes as an error choice. Same - -139 -00:07:20,060 --> 00:07:22,270 -thing if there's no file corresponding to the - -140 -00:07:22,270 --> 00:07:23,970 -name that we provide to the tool, we say - -141 -00:07:23,970 --> 00:07:26,760 -that this corresponds to an error. So how - -142 -00:07:26,760 --> 00:07:29,130 -is the tool going to use this information? It uses - -143 -00:07:29,130 --> 00:07:33,980 -this information by producing only one combination that involves - -144 -00:07:33,980 --> 00:07:37,270 -error choices, instead of combining them with other choices. - -145 -00:07:37,270 --> 00:07:39,780 -So let's see what happens after we added this - -146 -00:07:39,780 --> 00:07:43,370 -error constraints. So we go back to our console - -147 -00:07:43,370 --> 00:07:46,920 -once more. And in this case, we want to run - -148 -00:07:46,920 --> 00:07:50,910 -the TSL generator with the version of the, of my - -149 -00:07:50,910 --> 00:07:53,900 -file that contains the area of constraints. And again, - -150 -00:07:53,900 --> 00:07:56,390 -I reduce quite a bit the number of test frames. - -151 -00:07:56,390 --> 00:07:59,110 -So now I have only 562 test frames that - -152 -00:07:59,110 --> 00:08:02,660 -will be generated by using the file that I provided. - -153 -00:08:02,660 --> 00:08:05,460 -So for the last time, let's assume that we really want - -154 -00:08:05,460 --> 00:08:07,780 -to cut down the number of test frames or the number of - -155 -00:08:07,780 --> 00:08:10,380 -test cases. So once more, we go back to our file, and - -156 -00:08:10,380 --> 00:08:12,980 -at this point what we can add is the final type of - -157 -00:08:12,980 --> 00:08:14,170 -constraints that we have, which are - -158 -00:08:14,170 --> 00:08:17,245 -single constraints. And single constraints are - -159 -00:08:17,245 --> 00:08:21,360 -basically indicated choices that we don't want to combine with other choices. - -160 -00:08:21,360 --> 00:08:24,210 -So they have the same effect of the error constraints, but they - -161 -00:08:24,210 --> 00:08:28,120 -have a different meaning, so they do not indicate choices that corresponds - -162 -00:08:28,120 --> 00:08:29,860 -to an error. In other words, I can use a - -163 -00:08:29,860 --> 00:08:35,280 -single constraints to identify choices that I want to test only once. - -164 -00:08:35,280 --> 00:08:38,510 -So for example in this case, I might decide that I - -165 -00:08:38,510 --> 00:08:42,520 -want to have only one test frame that tests my program - -166 -00:08:42,520 --> 00:08:44,420 -with a file being empty and I can do the - -167 -00:08:44,420 --> 00:08:47,370 -same for other choices. So basically I can continue adding this - -168 -00:08:47,370 --> 00:08:50,400 -single constraint until I get down to the number of test - -169 -00:08:50,400 --> 00:08:53,410 -frames and therefore the number of test cases that I want. - -170 -00:08:53,410 --> 00:08:57,770 -So now let's go back once more to our console. And so now if we run - -171 -00:08:59,060 --> 00:09:04,450 -using this file as input, you can see that we have 35 test frames generated. So - -172 -00:09:04,450 --> 00:09:07,750 -this is a fairly low number of test cases, so we might decide that we want to - -173 -00:09:07,750 --> 00:09:13,380 -go ahead and write these test frames to a file. So now let's open this file - -174 -00:09:15,990 --> 00:09:25,500 -that we just generated. And as you can see here, I have exactly 35 test frames, - -175 -00:09:26,670 --> 00:09:30,900 -as expected. Some of those correspond to the single and error cases. So in this - -176 -00:09:30,900 --> 00:09:33,330 -case, the only choice that I have indicated - -177 -00:09:33,330 --> 00:09:35,690 -is the one that corresponds to the single - -178 -00:09:35,690 --> 00:09:38,310 -or error constraint. What is for the other - -179 -00:09:38,310 --> 00:09:42,170 -ones? I actually have the whole test spec. - -180 -00:09:42,170 --> 00:09:45,530 -So let's pick one just to give you an example. - -181 -00:09:45,530 --> 00:09:48,440 -In this case, that's frame number 15 that will correspond to - -182 -00:09:48,440 --> 00:09:51,910 -test case number 15. And here you can see that - -183 -00:09:51,910 --> 00:09:55,280 -we have all the information. So this is a test specification. - -184 -00:09:55,280 --> 00:09:57,560 -All the information that we need to generate the corresponding - -185 -00:09:57,560 --> 00:09:59,760 -test. We know that we need a file that is not - -186 -00:09:59,760 --> 00:10:03,810 -empty. That we need to have one occurrence of the pattern - -187 -00:10:03,810 --> 00:10:07,580 -in the file. One occurrence of the pattern in one line. - -188 -00:10:08,680 --> 00:10:10,360 -The position of the pattern in the file can - -189 -00:10:10,360 --> 00:10:13,740 -be any position. The length of the pattern must - -190 -00:10:13,740 --> 00:10:16,640 -be more than one character. The pattern should not - -191 -00:10:16,640 --> 00:10:20,140 -be enclosed in quotes. There should be one white - -192 -00:10:20,140 --> 00:10:24,460 -space, one quote within the pattern, and finally the - -193 -00:10:24,460 --> 00:10:27,230 -file that would pass through the program should exist. - -194 -00:10:27,230 --> 00:10:29,680 -So the file should be present. So I can - -195 -00:10:29,680 --> 00:10:33,950 -easily transform all of this into an actual test case. - -196 -00:10:33,950 --> 00:10:35,540 -And notice that even though we're not, we're not - -197 -00:10:35,540 --> 00:10:38,420 -going to do it here. In cases like this, it might - -198 -00:10:38,420 --> 00:10:42,190 -even be possible to automatically generate the test cases - -199 -00:10:42,190 --> 00:10:45,020 -from the test specifications because, here for example, here it - -200 -00:10:45,020 --> 00:10:48,150 -should be relatively straight forward to parse these test - -201 -00:10:48,150 --> 00:10:52,450 -specifications and generate test cases accordingly. So, just to summarize, - -202 -00:10:52,450 --> 00:10:55,910 -what we have done is to go from one high-level - -203 -00:10:55,910 --> 00:10:58,880 -description of a program to a set of categories, partitions, - -204 -00:10:58,880 --> 00:11:01,820 -and choices for that program. Then we have combined them - -205 -00:11:01,820 --> 00:11:04,930 -in different ways, adding more and more constraints to reduce the - -206 -00:11:04,930 --> 00:11:07,600 -number of combinations until we ended up with the right number - -207 -00:11:07,600 --> 00:11:09,650 -of test cases, so the number of test cases that we - -208 -00:11:09,650 --> 00:11:14,630 -were fine generating. We generated the corresponding test specifications. And at - -209 -00:11:14,630 --> 00:11:17,340 -that point, we could just go ahead, generate the test case, - -210 -00:11:17,340 --> 00:11:20,660 -and test our application. So, and you can see how this - -211 -00:11:20,660 --> 00:11:23,720 -can result in a much more thorough testing of your application. - -212 -00:11:23,720 --> 00:11:27,890 -Because instead of reading this description and just trying to come up with test - -213 -00:11:27,890 --> 00:11:33,900 -cases for it, we can break down the process in steps that are easy to perform - -214 -00:11:33,900 --> 00:11:36,960 -individually. They can be automated as much - -215 -00:11:36,960 --> 00:11:38,600 -as possible. And they will end up with - -216 -00:11:38,600 --> 00:11:40,380 -a set of test cases that will test - -217 -00:11:40,380 --> 00:11:42,790 -all the interests and aspects of your application. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt deleted file mode 100644 index 3d41cd9..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/24 - Model Based Testing - lang_en_vs4.srt +++ /dev/null @@ -1,83 +0,0 @@ -1 -00:00:00,370 --> 00:00:02,950 -What we just saw with the category-partition method, is a - -2 -00:00:02,950 --> 00:00:05,530 -specific instance of this systematic - -3 -00:00:05,530 --> 00:00:08,230 -functional testing approach. So specific instance - -4 -00:00:08,230 --> 00:00:10,900 -of the steps that we represented here. And, as I - -5 -00:00:10,900 --> 00:00:13,100 -mentioned earlier on, this is not the only way in which - -6 -00:00:13,100 --> 00:00:16,700 -you can generate test cases, starting from a functional specification. - -7 -00:00:16,700 --> 00:00:20,190 -In particular, this step, in which we identified relevant inputs and - -8 -00:00:20,190 --> 00:00:23,390 -then we combine them to generate test case specifications, can also - -9 -00:00:23,390 --> 00:00:25,490 -be done in different ways. And, we're going to look at one - -10 -00:00:25,490 --> 00:00:28,170 -of these ways. Which is through the construction of a - -11 -00:00:28,170 --> 00:00:31,040 -model. And, the reason why I want to talk about models. - -12 -00:00:31,040 --> 00:00:34,090 -Is because, model based testing is also, fairly popular in - -13 -00:00:34,090 --> 00:00:37,680 -industry. And fairly used in practice. In model based testing, the - -14 -00:00:37,680 --> 00:00:41,150 -way in which we go from specifications, to test cases, - -15 -00:00:41,150 --> 00:00:44,220 -is through the construction of a model. Where a model is - -16 -00:00:44,220 --> 00:00:47,670 -an abstract representation of the software under test. Also in - -17 -00:00:47,670 --> 00:00:50,860 -this case there are many possible models, that we can use. - -18 -00:00:50,860 --> 00:00:52,180 -And what we're going to do, we're going to focus - -19 -00:00:52,180 --> 00:00:54,370 -on a specific kind of model. And I'll - -20 -00:00:54,370 --> 00:00:57,010 -just point you to additional sources of information, - -21 -00:00:57,010 --> 00:00:59,070 -in case you're interested in seeing other examples. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt deleted file mode 100644 index 26f2fd7..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/25 - Finite State Machines - lang_en_vs4.srt +++ /dev/null @@ -1,103 +0,0 @@ -1 -00:00:00,160 --> 00:00:02,320 -The model that we will consider, is a very well - -2 -00:00:02,320 --> 00:00:05,330 -known one. Which is finite state machines. And you might have - -3 -00:00:05,330 --> 00:00:08,320 -seen them before. At a high level, a state machine is - -4 -00:00:08,320 --> 00:00:11,990 -a graph in which nodes represent states of the system. For - -5 -00:00:11,990 --> 00:00:15,650 -example, in this case, state 1, state 2, and state 3. - -6 -00:00:15,650 --> 00:00:19,950 -Edges represent transitions between states. For instance, in this case we - -7 -00:00:19,950 --> 00:00:22,850 -have one edge from state 1, to state 2. That means - -8 -00:00:22,850 --> 00:00:25,640 -that the system can go from state 1, to state 2. - -9 -00:00:25,640 --> 00:00:29,530 -And finally, the labels on the edges represent events - -10 -00:00:29,530 --> 00:00:32,800 -and actions. For example, what this label means is that - -11 -00:00:32,800 --> 00:00:35,400 -the system goes from state three to state two - -12 -00:00:35,400 --> 00:00:39,140 -when event five occurs. And when going from state three - -13 -00:00:39,140 --> 00:00:42,190 -to state two, it generates action four. And does - -14 -00:00:42,190 --> 00:00:45,430 -reacher model, sir reacher's kind of state machines, but we're - -15 -00:00:45,430 --> 00:00:48,160 -just going to stick to this ones which are enough. For - -16 -00:00:48,160 --> 00:00:50,760 -our purpose. So how do we build such a final - -17 -00:00:50,760 --> 00:00:53,530 -state machine starting from a specification? The first thing - -18 -00:00:53,530 --> 00:00:56,660 -we need to do is to identify the system's boundaries - -19 -00:00:56,660 --> 00:00:59,250 -and the input and output to the system. Once we - -20 -00:00:59,250 --> 00:01:01,960 -have done that, we can identify, within the boundaries of - -21 -00:01:01,960 --> 00:01:06,070 -the system, the relevant states and transitions. So we split - -22 -00:01:06,070 --> 00:01:09,670 -this single state We'll refine it into several states. And - -23 -00:01:09,670 --> 00:01:12,640 -we also identify how the system can go from one - -24 -00:01:12,640 --> 00:01:16,830 -state to another. Including which inputs cause which transition, and - -25 -00:01:16,830 --> 00:01:19,350 -which result in outputs we can obtain. To - -26 -00:01:19,350 --> 00:01:21,810 -better illustrate that, let's look at a concrete example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt deleted file mode 100644 index f889057..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/26 - Finite State Machines Example - lang_en_vs4.srt +++ /dev/null @@ -1,251 +0,0 @@ -1 -00:00:00,140 --> 00:00:03,010 -In this example, we're going to start from an informal specification, and the - -2 -00:00:03,010 --> 00:00:06,670 -specification is the one shown here in file spec.txt. This is the - -3 -00:00:06,670 --> 00:00:10,210 -specification for the maintenance function in a specific system. So what we're - -4 -00:00:10,210 --> 00:00:13,630 -doing is that we're taking the description of the functionality of a system, - -5 -00:00:13,630 --> 00:00:16,219 -and we're building a model, in this case a final state machine - -6 -00:00:16,219 --> 00:00:18,710 -for it. And there is no need to look at all the details - -7 -00:00:18,710 --> 00:00:21,350 -for this specification, but I want to point out that if you - -8 -00:00:21,350 --> 00:00:25,640 -look at the way the specification is written, we can identify specific cases - -9 -00:00:25,640 --> 00:00:28,210 -that we need to take into account. Like here if something - -10 -00:00:28,210 --> 00:00:31,920 -happens, something else will follow. Again, if something happens something else - -11 -00:00:31,920 --> 00:00:35,690 -will follow. So we have multiple choices here. Here will determine - -12 -00:00:35,690 --> 00:00:38,140 -the next steps and so on. So all we have to - -13 -00:00:38,140 --> 00:00:42,420 -do is to go through this process, identify these cases and - -14 -00:00:42,420 --> 00:00:45,300 -then build a machine that represents these cases. For the spec - -15 -00:00:45,300 --> 00:00:47,830 -that we just consider this is the state machine that will - -16 -00:00:47,830 --> 00:00:50,960 -result. Again there is no need to go through all the details, - -17 -00:00:50,960 --> 00:00:53,090 -but what I want to point out is that we have - -18 -00:00:53,090 --> 00:00:55,710 -a set of states. So for instance, we have state zero, - -19 -00:00:55,710 --> 00:00:58,380 -which is no maintenance, and if a request comes in, the - -20 -00:00:58,380 --> 00:01:01,350 -system will move, and the system wait for pickup. Then if - -21 -00:01:01,350 --> 00:01:04,400 -the pickup actually occurs, the system will move to the repair - -22 -00:01:04,400 --> 00:01:07,540 -state, and so on and so forth. So this is just - -23 -00:01:07,540 --> 00:01:13,070 -a more systematic representation of what was in the former specification. - -24 -00:01:13,070 --> 00:01:16,160 -And I will argue that this is much easier to understand at - -25 -00:01:16,160 --> 00:01:19,170 -least for somebody who has to develop tests for this system. In - -26 -00:01:19,170 --> 00:01:21,770 -fact what we're going to see next is how we can go from that - -27 -00:01:21,770 --> 00:01:24,790 -representation to a set of test cases. And the way which we do - -28 -00:01:24,790 --> 00:01:28,950 -it is by covering the behaviors represented by defining state machine. And we - -29 -00:01:28,950 --> 00:01:31,500 -can decide how we want to cover them. For example we might want - -30 -00:01:31,500 --> 00:01:35,080 -to cover all the states. So we might want to identify paths in - -31 -00:01:35,080 --> 00:01:38,310 -the state machine that go through all the states in the machine. Like - -32 -00:01:38,310 --> 00:01:41,840 -the one I just draw or this one, this one and this one. - -33 -00:01:41,840 --> 00:01:44,900 -So if we consider these four test cases, we can see that all the - -34 -00:01:44,900 --> 00:01:48,470 -states in my system or at least all the states that I have identified - -35 -00:01:48,470 --> 00:01:51,450 -are covered. I might want to go a little further, and decide that I - -36 -00:01:51,450 --> 00:01:54,210 -don't only want to cover all of the states, but I want to cover, all - -37 -00:01:54,210 --> 00:01:57,930 -of the transitions, because, it makes sense to visit a state, when coming from - -38 -00:01:57,930 --> 00:02:00,380 -different states. And, if I want to do that, and I look at the - -39 -00:02:00,380 --> 00:02:03,440 -test cases that I generated so far, I can see that there is one - -40 -00:02:03,440 --> 00:02:06,910 -transition, the one here, that is not covered. And, the same can be said for - -41 -00:02:06,910 --> 00:02:09,210 -the two transitions here. So what I can decide to do is - -42 -00:02:09,210 --> 00:02:13,370 -to generate another test case, that covers those or extend an existing one. - -43 -00:02:13,370 --> 00:02:16,500 -For instance, I could extend this test case by adding a visit to - -44 -00:02:16,500 --> 00:02:20,760 -the state, before going back to these two. Alternatively, I could also generate - -45 -00:02:20,760 --> 00:02:24,390 -new test cases, such as this one. To cover the missing transitions. - -46 -00:02:24,390 --> 00:02:26,350 -And once I have these test cases, I can express them in a - -47 -00:02:26,350 --> 00:02:29,860 -clearer way by simply specifying what are the states that they cover. I'm - -48 -00:02:29,860 --> 00:02:31,990 -just going to give you a couple of examples. Say, if we look - -49 -00:02:31,990 --> 00:02:34,280 -at the last one that I added, which will be test case - -50 -00:02:34,280 --> 00:02:37,190 -number five, I just need to specify that it will go through state - -51 -00:02:37,190 --> 00:02:41,090 -zero, which is this one, five, which is this one, six, and - -52 -00:02:41,090 --> 00:02:43,130 -then back to zero. And I can do the same for the other - -53 -00:02:43,130 --> 00:02:46,500 -test cases. So this will be my complete set of test cases. - -54 -00:02:46,500 --> 00:02:50,060 -So the bottom line here is that it is much harder to build - -55 -00:02:50,060 --> 00:02:53,080 -a set of test cases that will cover the behavior of an informal - -56 -00:02:53,080 --> 00:02:56,970 -description. But by going through a model, so by building in this case, - -57 -00:02:56,970 --> 00:03:01,700 -a finite state machine for that description, we can, in a much easier way, see - -58 -00:03:01,700 --> 00:03:04,100 -what the behaviors of interest of the system - -59 -00:03:04,100 --> 00:03:05,960 -are, and try to cover them. And there - -60 -00:03:05,960 --> 00:03:07,400 -is again in the spirit of breaking - -61 -00:03:07,400 --> 00:03:09,950 -down a complex problem into smaller steps that - -62 -00:03:09,950 --> 00:03:11,620 -we can better manage, which in the end, - -63 -00:03:11,620 --> 00:03:14,450 -results in a more efficient and effective testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt deleted file mode 100644 index 64c2c00..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/27 - Finite State Machines Considerations - lang_en_vs4.srt +++ /dev/null @@ -1,99 +0,0 @@ -1 -00:00:00,100 --> 00:00:03,060 -There are some important considerations I want to make on final state - -2 -00:00:03,060 --> 00:00:06,410 -machines. And more in general, on model based testing. The first one - -3 -00:00:06,410 --> 00:00:10,600 -is about applicability. Testing based on final state machines is a very - -4 -00:00:10,600 --> 00:00:13,890 -general approach, that we can apply in a number of contexts. And in - -5 -00:00:13,890 --> 00:00:16,510 -particular, if you are working with UML, you have state machines for - -6 -00:00:16,510 --> 00:00:19,720 -free. Because state charts are nothing else but a special kind of - -7 -00:00:19,720 --> 00:00:22,210 -state machine. So you can apply the technique that we just saw - -8 -00:00:22,210 --> 00:00:26,010 -directly on state charts, and try to cover their states and their transitions. - -9 -00:00:26,010 --> 00:00:29,980 -Another important point is that abstraction is key. You have to find the - -10 -00:00:29,980 --> 00:00:33,280 -right level of abstraction. The bigger the system, the more you have to - -11 -00:00:33,280 --> 00:00:36,130 -abstract if you want to represent it with a model, and in particular, with - -12 -00:00:36,130 --> 00:00:38,710 -the final state machine. So it's like having a slider, and you have - -13 -00:00:38,710 --> 00:00:41,840 -to decide where you want to move on that slider. The more you represent, - -14 -00:00:41,840 --> 00:00:44,700 -the more complex your system is going to be and the more thorough your - -15 -00:00:44,700 --> 00:00:48,110 -testing is going to be but also more expensive. The less you represent the - -16 -00:00:48,110 --> 00:00:51,330 -less expensive testing is going to be, but also testing might not be as - -17 -00:00:51,330 --> 00:00:53,500 -thorough as it would be otherwise. So you have to find - -18 -00:00:53,500 --> 00:00:56,150 -the right balance between abstracting the weight too much and abstracting - -19 -00:00:56,150 --> 00:00:59,840 -the weight too little. And finally there are many other approaches. - -20 -00:00:59,840 --> 00:01:02,840 -So we just scratched the surface, and we just saw one possible - -21 -00:01:02,840 --> 00:01:05,760 -approach. But for instance, other models that you can use are - -22 -00:01:05,760 --> 00:01:09,780 -decision tables, flow graphs and even historical models. Models that can - -23 -00:01:09,780 --> 00:01:13,560 -guide your testing based on problems that occurred in your system - -24 -00:01:13,560 --> 00:01:16,500 -in the past. And also, in this case, I'm going to put pointers - -25 -00:01:16,500 --> 00:01:18,460 -to additional materials in the class notes. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt deleted file mode 100644 index 1b68b9f..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/28 - Summary - lang_en_vs4.srt +++ /dev/null @@ -1,67 +0,0 @@ -1 -00:00:00,110 --> 00:00:01,600 -Now we are at the end of this lesson, and I - -2 -00:00:01,600 --> 00:00:04,450 -just want to wrap it up by summarizing what we've seen. We talked - -3 -00:00:04,450 --> 00:00:06,340 -about black-box testing, the testing of - -4 -00:00:06,340 --> 00:00:08,760 -software based on a functional specification, - -5 -00:00:08,760 --> 00:00:11,510 -a description of the software rather than its code. We saw a - -6 -00:00:11,510 --> 00:00:15,500 -systematic way of doing that, that allows for breaking down the problem - -7 -00:00:15,500 --> 00:00:19,140 -of testing software, so the problem of going from this functional specification - -8 -00:00:19,140 --> 00:00:22,540 -to a set of test cases into smaller steps, more manageable steps. - -9 -00:00:22,540 --> 00:00:25,466 -And we saw two main ways of doing this. One by identifying - -10 -00:00:25,466 --> 00:00:28,304 -relevant inputs for the main features in the system - -11 -00:00:28,304 --> 00:00:31,802 -and then deriving test case specifications and test cases from - -12 -00:00:31,802 --> 00:00:34,180 -this set of inputs. And the second way by - -13 -00:00:34,180 --> 00:00:36,780 -building a model for the main features of the system - -14 -00:00:36,780 --> 00:00:39,210 -and then using this model to decide how to - -15 -00:00:39,210 --> 00:00:41,680 -test the system. In the next lesson, we are going - -16 -00:00:41,680 --> 00:00:44,440 -to discuss, how to do testing by looking inside the - -17 -00:00:44,440 --> 00:00:47,716 -box? So, how to do testing in a white-box fashion. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt deleted file mode 100644 index 75da7e8..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/3 - Systematic Functional Testing Approach - lang_en_vs4.srt +++ /dev/null @@ -1,179 +0,0 @@ -1 -00:00:00,140 --> 00:00:02,690 -So what is the starting point of black box testing? Black - -2 -00:00:02,690 --> 00:00:06,010 -box testing start from a description of the software or as we - -3 -00:00:06,010 --> 00:00:09,590 -call it, a functional specification. And the final result of black - -4 -00:00:09,590 --> 00:00:12,760 -box testing is a set of test cases, a set of actual - -5 -00:00:12,760 --> 00:00:16,410 -inputs and corresponding outputs that we can use to exercise our - -6 -00:00:16,410 --> 00:00:19,030 -code and to try to find defects in our code. So the - -7 -00:00:19,030 --> 00:00:22,060 -question is, how do we get from functional specification to test - -8 -00:00:22,060 --> 00:00:25,510 -cases? Doing these derivations, so going from this description to a concrete - -9 -00:00:25,510 --> 00:00:28,550 -set of tests, is a very complex analytical process. - -10 -00:00:28,550 --> 00:00:31,220 -And normally brute force generation is not a good - -11 -00:00:31,220 --> 00:00:34,430 -idea because it's inefficient and ineffective. What we want - -12 -00:00:34,430 --> 00:00:37,540 -to do instead is to have a systematic approach to - -13 -00:00:37,540 --> 00:00:40,250 -derive test cases from a functional specification. What a - -14 -00:00:40,250 --> 00:00:43,970 -systematic approach does is to simplify the overall problem by - -15 -00:00:43,970 --> 00:00:48,320 -dividing the process into elementary steps. In particular, in - -16 -00:00:48,320 --> 00:00:50,520 -this case, we will perform three main steps. The first - -17 -00:00:50,520 --> 00:00:55,790 -step is to identify independently testable features. Individual features in - -18 -00:00:55,790 --> 00:00:57,600 -the soft hood that we can test. And we're going to - -19 -00:00:57,600 --> 00:00:59,990 -expand on each one of these steps in the next - -20 -00:00:59,990 --> 00:01:02,490 -part of the lesson. The following step is once we have - -21 -00:01:02,490 --> 00:01:06,000 -these independently testable features to identify what are the relevant - -22 -00:01:06,000 --> 00:01:08,590 -inputs. So what are the inputs or the behavior that is - -23 -00:01:08,590 --> 00:01:11,610 -worth testing for these features. Next once we have these - -24 -00:01:11,610 --> 00:01:13,020 -inputs, we're going to derive test - -25 -00:01:13,020 --> 00:01:15,770 -specifications. And test case specifications are - -26 -00:01:15,770 --> 00:01:19,490 -description of the test cases that we can then use - -27 -00:01:19,490 --> 00:01:23,270 -to generate actual test cases. And proceeding in this way, - -28 -00:01:23,270 --> 00:01:26,050 -by this steps, has many advantages. It allows for the - -29 -00:01:26,050 --> 00:01:29,920 -coupling different activities. It allows for dividing brain intensive steps from - -30 -00:01:29,920 --> 00:01:32,240 -steps that can be automated, which is a great advantage. - -31 -00:01:32,240 --> 00:01:34,980 -And also we will see, it allows you for monitoring - -32 -00:01:34,980 --> 00:01:38,040 -the testing process. So to figure out whether your testing - -33 -00:01:38,040 --> 00:01:41,000 -process is going as expected, for example, if you're generating too - -34 -00:01:41,000 --> 00:01:44,160 -many test cases. Or you're generating the number of test cases that your - -35 -00:01:44,160 --> 00:01:47,880 -amount of resources available allows you to run. So let's start by looking - -36 -00:01:47,880 --> 00:01:51,230 -at the first step of this process in which our goal is to - -37 -00:01:51,230 --> 00:01:54,820 -go from a Functional Specification to a set of features that we can - -38 -00:01:54,820 --> 00:01:57,700 -test in the software. So what we want to do is to identify all - -39 -00:01:57,700 --> 00:02:00,290 -of the feature of the software. And why do we want to do this? - -40 -00:02:00,290 --> 00:02:02,650 -Well you know, in the spirit of breaking down the complexity of the - -41 -00:02:02,650 --> 00:02:06,170 -problem, it does not make sense to just try to devise test cases for - -42 -00:02:06,170 --> 00:02:08,750 -all the features of the software at once. For any non-trivial - -43 -00:02:08,750 --> 00:02:11,980 -software, that's a humongous problem, and something that we cannot really - -44 -00:02:11,980 --> 00:02:16,300 -handle effectively. A much better way is to identify independently testable - -45 -00:02:16,300 --> 00:02:19,100 -features and consider one of them at a time when generating tests. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt deleted file mode 100644 index dda3144..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/4 - Overview Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,35 +0,0 @@ -1 -00:00:00,740 --> 00:00:02,780 -So, now I want to do a little quiz about - -2 -00:00:02,780 --> 00:00:05,530 -identifying testable features. Let's consider - -3 -00:00:05,530 --> 00:00:07,700 -this simple problem called printSum. - -4 -00:00:07,700 --> 00:00:09,680 -We won't see the implementation because we are doing black-box - -5 -00:00:09,680 --> 00:00:12,780 -testing. And all we need to know is that printSum takes - -6 -00:00:12,780 --> 00:00:15,550 -two integers, a and b, and prints the sum of - -7 -00:00:15,550 --> 00:00:17,540 -these two numbers. So what I want to ask, is - -8 -00:00:17,540 --> 00:00:20,700 -how many independently testable features do we have here? Do - -9 -00:00:20,700 --> 00:00:24,860 -we have one, two, three features? Or more than three features? diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 5503f98..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/5 - Overview Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,15 +0,0 @@ -1 -00:00:00,360 --> 00:00:03,760 -Sum is a very simple program, that only does one thing, - -2 -00:00:03,760 --> 00:00:07,540 -summing two number, adding two numbers. So the answer in this - -3 -00:00:07,540 --> 00:00:10,640 -case, it's one. There's only one feature that we can test - -4 -00:00:10,640 --> 00:00:14,060 -in PrintSum. So let's look at the slightly more interesting example. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt deleted file mode 100644 index 96c643e..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/6 - Overview Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,19 +0,0 @@ -1 -00:00:00,150 --> 00:00:02,469 -Let's look at this spreadsheet. I'm pretty sure most of you - -2 -00:00:02,469 --> 00:00:05,990 -are familiar with what a spreadsheet is and have used them before. - -3 -00:00:05,990 --> 00:00:08,119 -So now I'm going to ask the same question, which is I'd - -4 -00:00:08,119 --> 00:00:10,480 -like you to identify three possible - -5 -00:00:10,480 --> 00:00:13,190 -independently testable features for a spreadsheet. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 88e7cd5..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/7 - Overview Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,71 +0,0 @@ -1 -00:00:00,190 --> 00:00:03,030 -In this case there's not really a right answer because there's - -2 -00:00:03,030 --> 00:00:05,750 -many, many features that you could identify in a piece of - -3 -00:00:05,750 --> 00:00:08,655 -software as complex as a, a spreadsheet. So I'm just going - -4 -00:00:08,655 --> 00:00:11,900 -to give you three examples. So one could be the cell merging - -5 -00:00:11,900 --> 00:00:14,950 -operation. So the operation in which we merge two cells in - -6 -00:00:14,950 --> 00:00:18,550 -the spreadsheet. Another example could be chart creation, so I might want - -7 -00:00:18,550 --> 00:00:21,370 -to test the feature that allows you to create charts in - -8 -00:00:21,370 --> 00:00:23,310 -your spreadsheets. Yet another example could - -9 -00:00:23,310 --> 00:00:25,540 -be the test of statistical functions, - -10 -00:00:25,540 --> 00:00:29,350 -so the function that allows you to do various statistical calculations on - -11 -00:00:29,350 --> 00:00:32,650 -the numbers in your cells. And as I said there's many, many - -12 -00:00:32,650 --> 00:00:35,900 -more example that we could use. But the key thing I want - -13 -00:00:35,900 --> 00:00:38,770 -to convey here is the fact that there is no way you - -14 -00:00:38,770 --> 00:00:42,020 -can look at a spreadsheet with all the functionality that it provides - -15 -00:00:42,020 --> 00:00:44,160 -and just go and test it. The first step, what you need - -16 -00:00:44,160 --> 00:00:47,430 -to do first, is to identify which ones are the pieces of - -17 -00:00:47,430 --> 00:00:50,620 -functionality that I can test individually. So that's why this is the - -18 -00:00:50,620 --> 00:00:52,310 -first step in black-box testing. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt deleted file mode 100644 index 51aaffe..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/8 - Test Data Selection - lang_en_vs4.srt +++ /dev/null @@ -1,103 +0,0 @@ -1 -00:00:00,230 --> 00:00:03,550 -Once we have identified Independently Testable Features, the next step is to - -2 -00:00:03,550 --> 00:00:07,720 -identify the Relevant Inputs for each one of these features. And there are - -3 -00:00:07,720 --> 00:00:10,420 -many ways to do that. So, what we're going to do, instead of - -4 -00:00:10,420 --> 00:00:13,770 -looking at them all, is that we're just going to focus on two different - -5 -00:00:13,770 --> 00:00:16,239 -ways of doing it. And they are fairly general ways. So, they - -6 -00:00:16,239 --> 00:00:19,910 -are applicable to a number of situations. And in addition, what I will - -7 -00:00:19,910 --> 00:00:22,190 -do, I will point you to other sources in which you can - -8 -00:00:22,190 --> 00:00:25,530 -look at different ways of doing this in the class notes. The problem - -9 -00:00:25,530 --> 00:00:28,390 -of identifying relevant inputs for some Software or some feature - -10 -00:00:28,390 --> 00:00:31,930 -of it is called Test Data Selection and can be expressed - -11 -00:00:31,930 --> 00:00:35,040 -as followed. Let's consider our software as usual we have - -12 -00:00:35,040 --> 00:00:38,190 -our Input Domain, which is the set of inputs for all - -13 -00:00:38,190 --> 00:00:40,980 -the software. And again as usual, we have our Output - -14 -00:00:40,980 --> 00:00:44,050 -Domain, which is the set of corresponding outlets for these inputs. - -15 -00:00:44,050 --> 00:00:47,450 -So the question here is, how can we select a meaningful - -16 -00:00:47,450 --> 00:00:50,920 -set of inputs in my domain? And of course corresponding outputs - -17 -00:00:50,920 --> 00:00:53,240 -because we know that test cases are an input, plus an - -18 -00:00:53,240 --> 00:00:56,900 -expected output. So how can we select interesting inputs for our - -19 -00:00:56,900 --> 00:01:00,040 -software? So a set of inputs that, after we run them - -20 -00:01:00,040 --> 00:01:02,950 -on the software, if the software behaves correctly, we'll have enough - -21 -00:01:02,950 --> 00:01:07,060 -confidence that the software is correctly implemented. So one possible idea - -22 -00:01:07,060 --> 00:01:09,830 -is, hey, why don't we just test them all? We just - -23 -00:01:09,830 --> 00:01:12,800 -do exhaustive testing. We do all the inputs, nowadays we have - -24 -00:01:12,800 --> 00:01:16,290 -powerful machines, we have a lot of computational power in the cloud. - -25 -00:01:16,290 --> 00:01:17,890 -Why not just doing it? So to answer that - -26 -00:01:17,890 --> 00:01:20,480 -question, what I'm going to do? I'm going to use another quiz. diff --git a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt deleted file mode 100644 index 9ffead6..0000000 --- a/usth/ICT2.7/P4L2 Black-Box Testing Subtitles/9 - Test Data Selection Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,31 +0,0 @@ -1 -00:00:00,200 --> 00:00:02,180 -So I'm going to ask you something. Which is if - -2 -00:00:02,180 --> 00:00:04,750 -we consider again our function print sum, the one - -3 -00:00:04,750 --> 00:00:06,760 -that takes two integers and prints the sum. How - -4 -00:00:06,760 --> 00:00:09,300 -long would it take to exhaustively test this function? And - -5 -00:00:09,300 --> 00:00:10,950 -this is a very simple one. There's just two - -6 -00:00:10,950 --> 00:00:13,240 -inputs, right? So we can just enumerate them all. - -7 -00:00:13,240 --> 00:00:15,950 -And put them throw them at the computer form, - -8 -00:00:15,950 --> 00:00:18,440 -and wait for the results. How long would that take? |