diff options
| author | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-07-19 20:34:40 +0700 |
|---|---|---|
| committer | Nguyễn Gia Phong <mcsinyx@disroot.org> | 2020-07-19 20:34:40 +0700 |
| commit | 8a7dfa0972c83fd811a4296e7373574bea4a28d0 (patch) | |
| tree | 16d37247e8b909ce5f885affd2b2473faab891fd /usth/ICT2.7/P4L3 White-Box Testing Subtitles | |
| parent | dc6f57c3af35f599abab2c4bac950654282cb519 (diff) | |
| download | cp-8a7dfa0972c83fd811a4296e7373574bea4a28d0.tar.gz | |
[usth/ICT2.7] Remove Udacity transcribes
Diffstat (limited to 'usth/ICT2.7/P4L3 White-Box Testing Subtitles')
30 files changed, 0 insertions, 2946 deletions
diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt deleted file mode 100644 index 64f1576..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/1 - Lesson Overview - lang_en_vs4.srt +++ /dev/null @@ -1,47 +0,0 @@ -1 -00:00:00,110 --> 00:00:02,570 -In the previous lesson we discussed black box - -2 -00:00:02,570 --> 00:00:05,120 -testing, which is the testing performed based on a - -3 -00:00:05,120 --> 00:00:08,760 -description of the software, but without considering any of - -4 -00:00:08,760 --> 00:00:12,440 -software's internal details. In this lesson, we will discuss - -5 -00:00:12,440 --> 00:00:15,490 -the counterpart of black box testing, which is called - -6 -00:00:15,490 --> 00:00:20,590 -unsurprisingly white box testing or structural testing. And just - -7 -00:00:20,590 --> 00:00:22,780 -like we did for black box testing, we will - -8 -00:00:22,780 --> 00:00:25,740 -cover the main characteristics of white box testing and - -9 -00:00:25,740 --> 00:00:30,620 -this casts the most commonly used white box testing techniques. We - -10 -00:00:30,620 --> 00:00:33,650 -will conclude the lesson with the discussion of the main strengths and - -11 -00:00:33,650 --> 00:00:37,540 -limitations of white box testing. So that you will know, when to - -12 -00:00:37,540 --> 00:00:40,950 -use it? How to use it? And what to expect from it? diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/10 - Statement Coverage Quiz Solution - lang_en_vs7.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/10 - Statement Coverage Quiz Solution - lang_en_vs7.srt deleted file mode 100644 index c18d579..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/10 - Statement Coverage Quiz Solution - lang_en_vs7.srt +++ /dev/null @@ -1,31 +0,0 @@ -1 -00:00:00,210 --> 00:00:01,900 -To get the answer to this question, I'm - -2 -00:00:01,900 --> 00:00:03,540 -going to ask you to be a little patient, and - -3 -00:00:03,540 --> 00:00:05,120 -to wait until the end of the lesson. - -4 -00:00:05,120 --> 00:00:06,792 -Because there are a couple of more topics that - -5 -00:00:06,792 --> 00:00:08,244 -I want to cover, before I actually get in - -6 -00:00:08,244 --> 00:00:10,210 -to this. Nevertheless, I wanted to ask you right - -7 -00:00:10,210 --> 00:00:11,880 -away, because I wanted you to think about - -8 -00:00:11,880 --> 00:00:13,560 -this, before you see the rest of the lesson. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/11 - Control Flow Graphs - lang_en_vs5.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/11 - Control Flow Graphs - lang_en_vs5.srt deleted file mode 100644 index 83d5860..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/11 - Control Flow Graphs - lang_en_vs5.srt +++ /dev/null @@ -1,167 +0,0 @@ -1 -00:00:00,320 --> 00:00:03,450 -Let's look at the code for PrintSum in a slightly different way - -2 -00:00:03,450 --> 00:00:06,140 -by making something explicit. If we go through the code, we can - -3 -00:00:06,140 --> 00:00:08,578 -see that the, the code does something in that case, if the - -4 -00:00:08,578 --> 00:00:11,720 -result greater then zero, does something else if the result is not - -5 -00:00:11,720 --> 00:00:14,890 -greater than zero but is less than zero, and otherwise in the - -6 -00:00:14,890 --> 00:00:17,800 -case in which neither of these two conditions is true. Nothing really - -7 -00:00:17,800 --> 00:00:19,090 -happens. So we're going to make that - -8 -00:00:19,090 --> 00:00:21,290 -explicit, we're going to say here, otherwise - -9 -00:00:21,290 --> 00:00:25,430 -do nothing, which is exactly our problem, the code does nothing, in this - -10 -00:00:25,430 --> 00:00:28,380 -case where it should do something. So now, let's look again in - -11 -00:00:28,380 --> 00:00:31,470 -our test cases, let's consider the first one, and I'm going to go a - -12 -00:00:31,470 --> 00:00:34,410 -little faster in this case, because we already saw what happens If - -13 -00:00:34,410 --> 00:00:37,890 -we execute the first test case, we get to this point, we execute - -14 -00:00:37,890 --> 00:00:40,710 -this statement, and then we just jump to the end, as we - -15 -00:00:40,710 --> 00:00:44,020 -saw. Now we, if we execute the second test case, we do the - -16 -00:00:44,020 --> 00:00:46,740 -same, we get to the else statement, the condition for the if is - -17 -00:00:46,740 --> 00:00:50,770 -true, and therefore we execute this statement. And we never reached this point - -18 -00:00:50,770 --> 00:00:53,320 -for either of the test cases. So how can we express - -19 -00:00:53,320 --> 00:00:56,470 -this? In order to do that, I'm going to introduce a very useful - -20 -00:00:56,470 --> 00:01:00,450 -concept. The concept of control flow graphs. The control flow graphs - -21 -00:01:00,450 --> 00:01:03,310 -is just a representation for the code that is very convenient when - -22 -00:01:03,310 --> 00:01:06,030 -we run our reason about the code and its structure. And - -23 -00:01:06,030 --> 00:01:09,910 -it's a fairly simple one that represents statement with notes and the - -24 -00:01:09,910 --> 00:01:12,900 -flow of control within the code with edges. So here's an - -25 -00:01:12,900 --> 00:01:16,180 -example of control flow graph for this code. There is the entry - -26 -00:01:16,180 --> 00:01:19,790 -point of the code right here, then our statement in which we - -27 -00:01:19,790 --> 00:01:23,320 -assign the result of A plus B to variable result. Our if - -28 -00:01:23,320 --> 00:01:25,720 -statement and as you can see the if statement it's got two - -29 -00:01:25,720 --> 00:01:28,930 -branches coming out of it, because based on the outcome of this - -30 -00:01:28,930 --> 00:01:32,376 -predicate we will go one way or the other. In fact normally - -31 -00:01:32,376 --> 00:01:36,000 -what we do, we will label this edges accordingly. So for example, - -32 -00:01:36,000 --> 00:01:38,580 -here we will say that this is the label to be executed - -33 -00:01:38,580 --> 00:01:41,470 -when the predicate is true. And this is the label that is executed - -34 -00:01:41,470 --> 00:01:44,120 -when the predicate is false. Now, at this point, similar - -35 -00:01:44,120 --> 00:01:47,080 -thing, statement five which corresponds with this one, we have another - -36 -00:01:47,080 --> 00:01:49,650 -if statement and if that statement is true, then we get - -37 -00:01:49,650 --> 00:01:51,830 -to this point and if it's false, we get to this - -38 -00:01:51,830 --> 00:01:54,370 -point. So as you can see, this graph represents my - -39 -00:01:54,370 --> 00:01:57,040 -code, in a much more intuitive way, because I can see - -40 -00:01:57,040 --> 00:02:00,650 -right away where the control flows, while I execute the code. - -41 -00:02:00,650 --> 00:02:01,730 -So we're going to use this - -42 -00:02:01,730 --> 00:02:04,430 -representation to introduce further coverage criteria. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/12 - Branch Coverage - lang_en_vs3.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/12 - Branch Coverage - lang_en_vs3.srt deleted file mode 100644 index a1dedad..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/12 - Branch Coverage - lang_en_vs3.srt +++ /dev/null @@ -1,267 +0,0 @@ -1 -00:00:00,400 --> 00:00:03,780 -So now that we know what the CFG is, let's see how we can get into a count, - -2 -00:00:03,780 --> 00:00:07,860 -that else part, that is missing in our example code by introducing a new - -3 -00:00:07,860 --> 00:00:10,480 -kind of coverage. Branch coverage. As usual, - -4 -00:00:10,480 --> 00:00:13,550 -I'm going to describe branch coverage in terms of test requirements and - -5 -00:00:13,550 --> 00:00:17,100 -coverage measure. So starting from test requirements. The test requirement for - -6 -00:00:17,100 --> 00:00:21,080 -branch coverage are the branches in the program. In other words, - -7 -00:00:21,080 --> 00:00:25,240 -the goal of branch coverage is to execute all of the branches in the program. - -8 -00:00:25,240 --> 00:00:28,370 -The coverage measure is defined accordingly as the number of branches - -9 -00:00:28,370 --> 00:00:33,150 -executed by my test cases over the total number of branches in the program. And - -10 -00:00:33,150 --> 00:00:37,020 -let me remind you that branches are the outgoing edges from a decision point. - -11 -00:00:37,020 --> 00:00:40,410 -Therefore, an if statement, a switch statement, a while statement. - -12 -00:00:40,410 --> 00:00:44,400 -Any note in the c of g that has got more than one outgoing edge. - -13 -00:00:44,400 --> 00:00:48,170 -Those edges are called branches. So let's look at that using our example. So - -14 -00:00:48,170 --> 00:00:51,690 -now we're looking back at our printSum example. And in addition to the code, - -15 -00:00:51,690 --> 00:00:54,440 -I also want to represent the CFG for the program. So - -16 -00:00:54,440 --> 00:00:58,590 -let's start by looking at how many branches we have in our code. Which means how - -17 -00:00:58,590 --> 00:01:02,070 -many test requirements we have. And in this case there are two decision points. - -18 -00:01:02,070 --> 00:01:04,940 -The first one that corresponds to the first if, and the second one that - -19 -00:01:04,940 --> 00:01:10,100 -corresponds to the second if. So we have one, two, three, and four branches. So - -20 -00:01:10,100 --> 00:01:13,880 -now, let's bring back our current set of test cases. We had two test cases. - -21 -00:01:13,880 --> 00:01:17,440 -The one's that, with which we achieved a 100% statement coverage. And - -22 -00:01:17,440 --> 00:01:21,140 -let's see what happens in terms of branch coverage when we run these test cases. - -23 -00:01:21,140 --> 00:01:24,110 -I start from the first one, when we execute it, we follow the code, - -24 -00:01:24,110 --> 00:01:27,890 -we get to this decision point because the predicate in the if statement is true. - -25 -00:01:27,890 --> 00:01:30,770 -We follow the true branch, therefore we get here and then, - -26 -00:01:30,770 --> 00:01:34,970 -we exit from the program. So, in this case, we covered one of the branches, - -27 -00:01:34,970 --> 00:01:40,710 -which means that we got to 25% coverage. Now when we run the second test case, - -28 -00:01:40,710 --> 00:01:44,160 -again we follow this path. We get to this, the first if and - -29 -00:01:44,160 --> 00:01:47,730 -in this case the predicate of the if is false. Therefore, we go this way. - -30 -00:01:47,730 --> 00:01:51,160 -We reach the second predicate, the second if. The result is true, so - -31 -00:01:51,160 --> 00:01:55,490 -we follow the true branch and therefore, we cover these additional two branches. - -32 -00:01:55,490 --> 00:01:59,900 -So at this point, we are at 75% branch coverage. So what - -33 -00:01:59,900 --> 00:02:04,070 -happens is that we're missing this branch. For now, the inputs that we consider, - -34 -00:02:04,070 --> 00:02:07,740 -this branch is executed. Therefore, we need to add an additional test case. And - -35 -00:02:07,740 --> 00:02:11,600 -that this case that we need, is one for which this predicate is false and this - -36 -00:02:11,600 --> 00:02:15,680 -predicate is false. The simplest possibility in this case is the test case for - -37 -00:02:15,680 --> 00:02:20,170 -which A is equal to 0 and B is equal to 0. If we execute this test case, - -38 -00:02:20,170 --> 00:02:24,250 -our execution again followed this path, follows the fourth branch here. - -39 -00:02:24,250 --> 00:02:28,090 -And in this case, because result is not less than zero either, will follow - -40 -00:02:28,090 --> 00:02:33,090 -this branch as well. And therefore, we will reach our 100% branch coverage. And - -41 -00:02:33,090 --> 00:02:35,820 -this covered the problem. Something that I would like to clarify before we - -42 -00:02:35,820 --> 00:02:40,920 -move to the next topic, is that 100% coverage does not provide any guarantee of - -43 -00:02:40,920 --> 00:02:44,530 -finding the problems in the code. All we saw so far is the fact that by - -44 -00:02:44,530 --> 00:02:48,580 -testing more thoroughly we have more chances of finding a problem in the code. - -45 -00:02:48,580 --> 00:02:51,680 -But it doesn't matter which kind of coverage we utilize, and how much - -46 -00:02:51,680 --> 00:02:55,440 -coverage we achieve. There's always a chance that we might miss something. And - -47 -00:02:55,440 --> 00:02:59,760 -I will get back to this later on in the lesson. I just mentioned the fact that - -48 -00:02:59,760 --> 00:03:03,910 -we tested more fully when we went from statement coverage to branch coverage. - -49 -00:03:03,910 --> 00:03:06,900 -What does that mean exactly? To explain that, I'm going to introduce the concept - -50 -00:03:06,900 --> 00:03:11,420 -of test criteria subsumption. One test criteria subsumes another criteria when - -51 -00:03:11,420 --> 00:03:16,760 -all the tests widths that satisfy that criteria will also satisfy the other one. - -52 -00:03:16,760 --> 00:03:18,740 -So let me show you that with statement and - -53 -00:03:18,740 --> 00:03:23,220 -branch coverage. If we identify a test width that achieves 100% branch coverage, - -54 -00:03:23,220 --> 00:03:27,780 -the same test width will also achieve, necessarily, 100% statement coverage. - -55 -00:03:27,780 --> 00:03:31,290 -That's what happened for our example, and also what happens in general, - -56 -00:03:31,290 --> 00:03:35,660 -because branch coverage is a stronger criteria than statement coverage. - -57 -00:03:35,660 --> 00:03:39,670 -There is no way to cover all the branches without covering all the statements. - -58 -00:03:39,670 --> 00:03:43,480 -It is not true that any test results satisfies statement coverage will also - -59 -00:03:43,480 --> 00:03:47,040 -satisfy branch coverage. And, in fact, we just saw a counter example. - -60 -00:03:47,040 --> 00:03:50,569 -When we look at the printSum code. We had a test where there was achieving 100% - -61 -00:03:50,569 --> 00:03:53,910 -statement coverage and was not achieving 100% branch coverage. Therefore, - -62 -00:03:53,910 --> 00:03:58,301 -in this case we have a substantial relation in this direction. Branch coverage, - -63 -00:03:58,301 --> 00:04:02,860 -subsumes statement coverage. What it also means is that normally, or in general, - -64 -00:04:02,860 --> 00:04:06,910 -it is more expensive to achieve branch coverage than achieve statement coverage, - -65 -00:04:06,910 --> 00:04:09,490 -because achieving branch coverage requires the generation of - -66 -00:04:09,490 --> 00:04:13,770 -a larger number of test cases. So what this relation means is that - -67 -00:04:13,770 --> 00:04:17,779 -branch coverage is stronger than statement coverage but also more expensive. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/13 - Condition Coverage - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/13 - Condition Coverage - lang_en_vs4.srt deleted file mode 100644 index bba72fc..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/13 - Condition Coverage - lang_en_vs4.srt +++ /dev/null @@ -1,191 +0,0 @@ -1 -00:00:00,470 --> 00:00:03,046 -What I'm going to do next is to introduce a few additional - -2 -00:00:03,046 --> 00:00:06,350 -coverage criteria using a slightly more complex example, but still a - -3 -00:00:06,350 --> 00:00:09,370 -pretty simple one. What I'm showing here is a program that - -4 -00:00:09,370 --> 00:00:12,145 -reads two real numbers, x and y. And then, if x - -5 -00:00:12,145 --> 00:00:15,148 -is equal to 0 or y is greater than 0, it - -6 -00:00:15,148 --> 00:00:19,360 -computes y as y divided by x. Otherwise, it computes x - -7 -00:00:19,360 --> 00:00:22,110 -as y plus 2, then it writes the value of x, - -8 -00:00:22,110 --> 00:00:25,730 -the value of y, and it terminates. Let's also introduce a CFG - -9 -00:00:25,730 --> 00:00:29,160 -for the program. As you can see, the CFG represents the statements - -10 -00:00:29,160 --> 00:00:31,900 -in the code and their control flow. And in this case, I made - -11 -00:00:31,900 --> 00:00:35,800 -explicit over the branches what are the conditions under which those branches - -12 -00:00:35,800 --> 00:00:38,530 -are taken to make it simpler to look at the example. Let's assume - -13 -00:00:38,530 --> 00:00:41,280 -that we have two tests for this code that are shown here. - -14 -00:00:41,280 --> 00:00:44,286 -For the first one, the inputs are 5 and 5. For the second - -15 -00:00:44,286 --> 00:00:47,610 -one, 5 and minus 5. If we consider branch coverage for this code - -16 -00:00:47,610 --> 00:00:50,990 -and we consider the two test cases, for the first one this condition - -17 -00:00:50,990 --> 00:00:53,440 -is true. Because x is not equal to 0 but y - -18 -00:00:53,440 --> 00:00:56,660 -is greater than 0. And therefore, we will follow this tree branch. - -19 -00:00:56,660 --> 00:00:59,640 -For the second one, the condition is false. Because x is not - -20 -00:00:59,640 --> 00:01:03,200 -equal to 0 and y is not greater than 0. Therefore, the - -21 -00:01:03,200 --> 00:01:05,990 -negation of it is true and we will follow this branch. In - -22 -00:01:05,990 --> 00:01:09,530 -other words, these two test cases achieve 100% branch coverage on this - -23 -00:01:09,530 --> 00:01:12,120 -code. If we look at the code though, we can see that - -24 -00:01:12,120 --> 00:01:15,879 -there is the possibility of making this code fail. Consider this statement, - -25 -00:01:15,879 --> 00:01:18,147 -if x is equal to 0, we could have a division - -26 -00:01:18,147 --> 00:01:21,710 -by 0. However, these two test cases, despite the fact that - -27 -00:01:21,710 --> 00:01:25,008 -they achieved 100% branch coverage, will not rebuild this problem. So - -28 -00:01:25,008 --> 00:01:27,564 -how can we be more thorough? I'll let you think about it - -29 -00:01:27,564 --> 00:01:29,964 -for a second, so think about how can we test more - -30 -00:01:29,964 --> 00:01:33,260 -thoroughly, in a more complete way, this code. So, in a way - -31 -00:01:33,260 --> 00:01:37,220 -that goes beyond branch coverage. And the answer is that we - -32 -00:01:37,220 --> 00:01:41,250 -can make each condition true and false. Instead of just considering the - -33 -00:01:41,250 --> 00:01:44,570 -whole predicate here. And that's exactly what is required by - -34 -00:01:44,570 --> 00:01:47,480 -the next criteria that we're going to consider which is condition - -35 -00:01:47,480 --> 00:01:49,880 -coverage. We're going to define it as usual in terms of - -36 -00:01:49,880 --> 00:01:53,070 -test requirements and coverage measure. In this case, the test - -37 -00:01:53,070 --> 00:01:57,540 -requirements for condition coverage are the individual conditions in the - -38 -00:01:57,540 --> 00:02:00,510 -program. So we want each condition in the program to - -39 -00:02:00,510 --> 00:02:03,900 -be both true and false first time execution. So the - -40 -00:02:03,900 --> 00:02:06,420 -way in which we can measure that is by measuring the - -41 -00:02:06,420 --> 00:02:09,150 -number of conditions that were both true and false - -42 -00:02:09,150 --> 00:02:11,580 -when we executed our tests over the total number - -43 -00:02:11,580 --> 00:02:14,180 -of conditions. And that gives us the percentage of - -44 -00:02:14,180 --> 00:02:17,270 -coverage that we achieved for condition coverage. Again, if you - -45 -00:02:17,270 --> 00:02:18,638 -want to look at this criteria in the form - -46 -00:02:18,638 --> 00:02:21,245 -of a question. The question would be, has each boolean - -47 -00:02:21,245 --> 00:02:25,480 -sub-expression, which means every condition in every predicate, evaluated - -48 -00:02:25,480 --> 00:02:28,010 -both to true and false when we run our tests. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/14 - Subsumption Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/14 - Subsumption Quiz - lang_en_vs4.srt deleted file mode 100644 index de78093..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/14 - Subsumption Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,31 +0,0 @@ -1 -00:00:00,080 --> 00:00:02,220 -So now that we introduced an additional criterion, - -2 -00:00:02,220 --> 00:00:04,290 -let's have a quiz to see whether everybody - -3 -00:00:04,290 --> 00:00:06,410 -understood the concept of subsumption. We know that - -4 -00:00:06,410 --> 00:00:09,490 -branch coverage subsumes statement coverage, but what about condition - -5 -00:00:09,490 --> 00:00:12,230 -coverage? Does it subsume branch coverage? Is it - -6 -00:00:12,230 --> 00:00:13,720 -the case that all of the test which - -7 -00:00:13,720 --> 00:00:16,980 -satisfy condition coverage will also satisfy branch coverage? - -8 -00:00:16,980 --> 00:00:19,160 -Think about it, and answer either yes or no. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/15 - Subsumption Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/15 - Subsumption Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 87140c9..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/15 - Subsumption Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,7 +0,0 @@ -1 -00:00:00,170 --> 00:00:02,475 -In this case, the answer is no, and I'm - -2 -00:00:02,475 --> 00:00:04,650 -going to show you an example of this in a minute. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/16 - Branch and Condition Coverage - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/16 - Branch and Condition Coverage - lang_en_vs4.srt deleted file mode 100644 index 143a534..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/16 - Branch and Condition Coverage - lang_en_vs4.srt +++ /dev/null @@ -1,143 +0,0 @@ -1 -00:00:00,052 --> 00:00:03,407 -Let's go back to our test criteria subsumption representation, where - -2 -00:00:03,407 --> 00:00:06,770 -we already had branch coverage and statement coverage. In this case, - -3 -00:00:06,770 --> 00:00:09,350 -if we want to add condition coverage to this page, we - -4 -00:00:09,350 --> 00:00:12,050 -have to put it here on this side, with no relationship - -5 -00:00:12,050 --> 00:00:15,830 -of subsumption with either branch coverage or statement coverage, which - -6 -00:00:15,830 --> 00:00:19,270 -means that the criteria are not comparable. So now let's consider - -7 -00:00:19,270 --> 00:00:22,108 -again our last example, and let's use a different test - -8 -00:00:22,108 --> 00:00:25,060 -with this time. We still have two tests, but the first - -9 -00:00:25,060 --> 00:00:27,575 -one has x is equal to 0 and y is equal to minus - -10 -00:00:27,575 --> 00:00:30,370 -5. And the second one, x is equal to 5 and y is - -11 -00:00:30,370 --> 00:00:33,350 -equal to 5 as well. Let's see what happens in terms of condition - -12 -00:00:33,350 --> 00:00:36,740 -coverage, when we run these tests. If we look at the first condition, x - -13 -00:00:36,740 --> 00:00:39,820 -is equal to 0. It is both true, for the first test case, - -14 -00:00:39,820 --> 00:00:43,240 -and false for the second one. As for the second condition, y greater - -15 -00:00:43,240 --> 00:00:45,800 -than 0, it is false for the first test case and true for - -16 -00:00:45,800 --> 00:00:47,770 -the second one. Therefore we've achieved a - -17 -00:00:47,770 --> 00:00:50,400 -100% condition coverage with these two tests. - -18 -00:00:50,400 --> 00:00:54,020 -But what about branch coverage? If we consider the whole predicate, - -19 -00:00:54,020 --> 00:00:56,620 -instead of just the individual conditions, let's see what happens for the - -20 -00:00:56,620 --> 00:00:59,090 -two test cases. If we look at the first one, because - -21 -00:00:59,090 --> 00:01:02,200 -x is equal to 0, the overall predicate is true. As for - -22 -00:01:02,200 --> 00:01:05,750 -the second one, because the second condition is true, the overall - -23 -00:01:05,750 --> 00:01:09,110 -predicate is true. In other words, despite the fact that we're exercising - -24 -00:01:09,110 --> 00:01:12,520 -all the possible values for the two conditions, the overall predicate is - -25 -00:01:12,520 --> 00:01:16,540 -always true. And therefore, we're covering only one of the two branches. - -26 -00:01:16,540 --> 00:01:19,591 -So our coverage will be 50%. This is the reason - -27 -00:01:19,591 --> 00:01:22,231 -why normally the two criteria that we just saw, decision - -28 -00:01:22,231 --> 00:01:26,205 -coverage, and condition coverage are considered together. And the resulting - -29 -00:01:26,205 --> 00:01:30,320 -criterion is called branch and condition coverage, or also decision - -30 -00:01:30,320 --> 00:01:33,030 -and condition coverage. At this point the test requirements and - -31 -00:01:33,030 --> 00:01:35,590 -the coverage measure should be pretty straight forward, because they're - -32 -00:01:35,590 --> 00:01:38,120 -just considering the two criteria together. As far as the - -33 -00:01:38,120 --> 00:01:39,960 -requirements are concerned, they include - -34 -00:01:39,960 --> 00:01:41,880 -all the branches and individual conditions - -35 -00:01:41,880 --> 00:01:44,580 -in the program. Where as the coverage measure is computed - -36 -00:01:44,580 --> 00:01:49,180 -considering both coverage measures, both branch coverage and condition coverage. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/17 - Subsumption Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/17 - Subsumption Quiz - lang_en_vs4.srt deleted file mode 100644 index 9d99495..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/17 - Subsumption Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,15 +0,0 @@ -1 -00:00:00,100 --> 00:00:02,810 -Let's have another quick quiz about Subsumption. If we - -2 -00:00:02,810 --> 00:00:06,720 -consider Branch and Condition Coverage, does it subsume Branch Coverage, - -3 -00:00:06,720 --> 00:00:09,512 -and therefore Statement Coverage? Or in other words, does branch - -4 -00:00:09,512 --> 00:00:12,750 -and condition coverage imply branch coverage? Answer yes, or no. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/18 - Subsumption Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/18 - Subsumption Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index c836f04..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/18 - Subsumption Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,19 +0,0 @@ -1 -00:00:00,150 --> 00:00:01,880 -In this case it should be clear that the - -2 -00:00:01,880 --> 00:00:05,640 -answer is yes, because branch and condition coverage actually includes - -3 -00:00:05,640 --> 00:00:08,580 -branch coverage. Therefore, by definition, any test rate that - -4 -00:00:08,580 --> 00:00:10,505 -satisfies branch and condition coverage - -5 -00:00:10,505 --> 00:00:12,664 -will necessarily satisfy branch coverage. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/19 - Test Criteria Subsumption - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/19 - Test Criteria Subsumption - lang_en_vs4.srt deleted file mode 100644 index f40ccdf..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/19 - Test Criteria Subsumption - lang_en_vs4.srt +++ /dev/null @@ -1,47 +0,0 @@ -1 -00:00:00,280 --> 00:00:03,400 -So we can now update our test criteria subsumption. We start - -2 -00:00:03,400 --> 00:00:06,700 -from this situation in which there are no relationship between condition - -3 -00:00:06,700 --> 00:00:09,650 -coverage, branch coverage and statement coverage. And when we add branch - -4 -00:00:09,650 --> 00:00:12,160 -and condition coverage, we can mark the fact that branch and - -5 -00:00:12,160 --> 00:00:14,470 -condition coverage subsumes branch coverage - -6 -00:00:14,470 --> 00:00:16,370 -and also subsumes condition coverage, for - -7 -00:00:16,370 --> 00:00:19,660 -the same reason. Therefore, this is a stronger criterion than both - -8 -00:00:19,660 --> 00:00:23,100 -branch coverage and condition coverage, and of course indirectly also soft - -9 -00:00:23,100 --> 00:00:25,330 -statement coverage. So once more, to make sure we are all - -10 -00:00:25,330 --> 00:00:29,255 -on the same page, if I develop a test rate that satisfies branch and condition - -11 -00:00:29,255 --> 00:00:31,380 -coverage, the same test will satisfy also - -12 -00:00:31,380 --> 00:00:35,210 -branch coverage, statement coverage and condition coverage, necessarily. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt deleted file mode 100644 index c032d40..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/2 - Overview - lang_en_vs4.srt +++ /dev/null @@ -1,167 +0,0 @@ -1 -00:00:00,420 --> 00:00:03,860 -In the last lesson, we talked about black-box testing or functional testing, - -2 -00:00:03,860 --> 00:00:06,110 -which is the kind of testing that you perform when you just - -3 -00:00:06,110 --> 00:00:09,860 -look at the description of the software. Today we're going to cover white-box - -4 -00:00:09,860 --> 00:00:12,490 -testing, which is the kind of testing that we perform when we - -5 -00:00:12,490 --> 00:00:15,800 -open up the box. We look inside the program, and we actually - -6 -00:00:15,800 --> 00:00:19,490 -test it based on its code. And there is one basic assumption - -7 -00:00:19,490 --> 00:00:22,320 -behind the idea of white-box testing, which is a very intuitive one, - -8 -00:00:22,320 --> 00:00:26,080 -and the assumption is that executing the faulty statement is a necessary - -9 -00:00:26,080 --> 00:00:29,380 -condition for revealing a fault. In other words, if there is - -10 -00:00:29,380 --> 00:00:31,690 -a bug in the program there is no way were going to be - -11 -00:00:31,690 --> 00:00:34,490 -able to find this bug or this fault, if we don't execute - -12 -00:00:34,490 --> 00:00:37,110 -the statement that contains it. Which makes a lot of sense. As - -13 -00:00:37,110 --> 00:00:40,240 -we did for black-box testing, we're going to start by summarizing what - -14 -00:00:40,240 --> 00:00:43,650 -are the main advantages of white-box testing. The main advantage is that - -15 -00:00:43,650 --> 00:00:48,050 -it's based on the code, and as such, the quality of white-box - -16 -00:00:48,050 --> 00:00:51,760 -testing can be measured objectively. And what I mean here by objectively - -17 -00:00:51,760 --> 00:00:54,390 -is that if you think about black-box testing In many - -18 -00:00:54,390 --> 00:00:57,630 -cases, there were subjective decisions, there were had to be made - -19 -00:00:57,630 --> 00:01:00,830 -in order to define tests in a black-box fashion. In the - -20 -00:01:00,830 --> 00:01:03,990 -case of white-box testing, because everything is based on the quota, - -21 -00:01:03,990 --> 00:01:07,965 -we don't have to make such subjective decisions. And similarly, because - -22 -00:01:07,965 --> 00:01:10,680 -white-box testing is based on the code, it can be measured - -23 -00:01:10,680 --> 00:01:14,250 -automatically. So we can build tools and actually there are tools, - -24 -00:01:14,250 --> 00:01:16,860 -and there's plenty of tools, that can be measured, the level - -25 -00:01:16,860 --> 00:01:19,670 -of white-box testing can be achieved in a fully automated way. - -26 -00:01:19,670 --> 00:01:21,620 -And we're going to see some of them in the course of the - -27 -00:01:21,620 --> 00:01:24,600 -class. Another advantage of white-box testing is that it can be - -28 -00:01:24,600 --> 00:01:27,700 -used to compare test suites. So if you have two alternative sets - -29 -00:01:27,700 --> 00:01:30,270 -of tests that you can use to assess the quality of - -30 -00:01:30,270 --> 00:01:34,360 -your software, white-box testing techniques can tell you which one of these - -31 -00:01:34,360 --> 00:01:37,580 -two test suites is likely to be more effective in testing - -32 -00:01:37,580 --> 00:01:42,350 -your code. And finally, white-box testing allows for covering the coded behavior - -33 -00:01:42,350 --> 00:01:44,470 -of the software. What that means is that if there is - -34 -00:01:44,470 --> 00:01:47,680 -some mistake in the code and is not obvious by looking at - -35 -00:01:47,680 --> 00:01:50,300 -the specification of the code, white box testing might be able - -36 -00:01:50,300 --> 00:01:53,330 -to catch it, because it tries to exercise the code. There's many - -37 -00:01:53,330 --> 00:01:56,430 -different kinds of white box testing, there are control flow based - -38 -00:01:56,430 --> 00:02:00,550 -techniques, data flow based techniques, and fault based techniques. And for each - -39 -00:02:00,550 --> 00:02:04,030 -one of these family of techniques there are many variations. So - -40 -00:02:04,030 --> 00:02:07,450 -this field is very, very broad. In this lesson we will talk - -41 -00:02:07,450 --> 00:02:09,110 -about white-box testing by focusing - -42 -00:02:09,110 --> 00:02:11,910 -mainly on control-flow based testing techniques. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/20 - B&C Coverage Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/20 - B&C Coverage Quiz - lang_en_vs4.srt deleted file mode 100644 index 6f5c9a1..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/20 - B&C Coverage Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,35 +0,0 @@ -1 -00:00:00,130 --> 00:00:02,969 -So let's have another quiz using the example that we just - -2 -00:00:02,969 --> 00:00:06,660 -used. This one is about achieving 100% branch and condition coverage. So - -3 -00:00:06,660 --> 00:00:08,850 -let me bring back the two tests cases that we just saw, - -4 -00:00:08,850 --> 00:00:11,420 -and as we saw a few minutes ago, these tests cases do - -5 -00:00:11,420 --> 00:00:15,540 -not achieve 100% branch and condition coverage despite the fact that they - -6 -00:00:15,540 --> 00:00:19,430 -achieve 100% condition coverage. So both conditions are both true and false, - -7 -00:00:19,430 --> 00:00:21,820 -for these test cases. So what I want you to do is - -8 -00:00:21,820 --> 00:00:25,630 -to add a test case, to achieve 100% branch and condition coverages. - -9 -00:00:25,630 --> 00:00:29,740 -So specify the test case by writing here the value for x, and the value for y. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/21 - B&C Coverage Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/21 - B&C Coverage Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 2b6fe4b..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/21 - B&C Coverage Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,67 +0,0 @@ -1 -00:00:00,190 --> 00:00:03,010 -Obviously there are many possible tests that we can use to - -2 -00:00:03,010 --> 00:00:05,910 -reach 100% branch and condition coverage. So I'm just going to show a - -3 -00:00:05,910 --> 00:00:08,740 -possible one, which is x equal to 3 and y is - -4 -00:00:08,740 --> 00:00:11,410 -equal to negative 2. If we specify this as case, you can - -5 -00:00:11,410 --> 00:00:14,900 -see that the overall condition is false, because neither x is - -6 -00:00:14,900 --> 00:00:17,990 -equal to 0 nor y is greater than 0. Therefore we will - -7 -00:00:17,990 --> 00:00:22,420 -follow the false, false branch, and achieve 100% branch and condition coverage - -8 -00:00:22,420 --> 00:00:25,400 -for this code. And we might require to be even more thorough, - -9 -00:00:25,400 --> 00:00:28,690 -that all the combinations of our conditions inside each - -10 -00:00:28,690 --> 00:00:31,790 -decision, inside each predicate, are tested. Which is what is - -11 -00:00:31,790 --> 00:00:36,010 -called, multiple condition coverage. But because of the way - -12 -00:00:36,010 --> 00:00:39,050 -this criterion is defined, it is combinatorial, becuse you have - -13 -00:00:39,050 --> 00:00:42,220 -to consider all the possible combinations of conditions. And - -14 -00:00:42,220 --> 00:00:45,210 -therefore it's extremely expensive, to the point of being impractical. - -15 -00:00:45,210 --> 00:00:47,610 -So instead of defining that criterion, we're going to find - -16 -00:00:47,610 --> 00:00:51,000 -another one which finds a good trade off between thoroughness - -17 -00:00:51,000 --> 00:00:52,815 -of the tests and their cost. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/22 - MC DC Coverage - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/22 - MC DC Coverage - lang_en_vs4.srt deleted file mode 100644 index 1a971f9..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/22 - MC DC Coverage - lang_en_vs4.srt +++ /dev/null @@ -1,307 +0,0 @@ -1 -00:00:00,120 --> 00:00:03,880 -And this criterion is called Modified Condition/Decision Coverage, - -2 -00:00:03,880 --> 00:00:07,750 -also called MC/DC. This criterion is very important because - -3 -00:00:07,750 --> 00:00:10,330 -it is often required for safety critical applications. - -4 -00:00:10,330 --> 00:00:13,920 -For example, the FAA, the Federal Aviation Administration, requires - -5 -00:00:13,920 --> 00:00:15,980 -for all the software that runs on commercial - -6 -00:00:15,980 --> 00:00:19,420 -airplanes to be tested according to the Modified Condition/Decision - -7 -00:00:19,420 --> 00:00:22,015 -Coverage. So what is the key idea behind the - -8 -00:00:22,015 --> 00:00:25,270 -MC/DC criterion? It is to test only the important - -9 -00:00:25,270 --> 00:00:28,710 -combinations of conditions instead of all of them, and limit - -10 -00:00:28,710 --> 00:00:32,189 -the testing cost by excluding the other combinations. And the way - -11 -00:00:32,189 --> 00:00:34,950 -in which it works is by extending branch and decision - -12 -00:00:34,950 --> 00:00:38,860 -coverage with the requirement that each condition should affect the decision - -13 -00:00:38,860 --> 00:00:41,940 -outcome independently. So let's see what this means with an - -14 -00:00:41,940 --> 00:00:45,030 -example that will also show you how you can reduce the - -15 -00:00:45,030 --> 00:00:46,940 -number of combinations in this way. I am going to - -16 -00:00:46,940 --> 00:00:50,460 -show you an example of how MC/DC works using this predicate - -17 -00:00:50,460 --> 00:00:53,870 -which consists of three conditions. a, b, and c, which are all - -18 -00:00:53,870 --> 00:00:57,070 -in and, so the overall predicate is a and b and c. - -19 -00:00:57,070 --> 00:00:58,980 -The first thing I'm going to do is to show you how - -20 -00:00:58,980 --> 00:01:03,370 -many test cases we will need to satisfy the multiple condition coverage - -21 -00:01:03,370 --> 00:01:06,440 -for this simple predicate. Which means, how many test cases we will - -22 -00:01:06,440 --> 00:01:09,780 -need to test all the possible combinations of true and false values - -23 -00:01:09,780 --> 00:01:12,740 -for these conditions. So I'm going to populate this table. And as you - -24 -00:01:12,740 --> 00:01:15,870 -can see, at the end we have eight test cases. Each test case - -25 -00:01:15,870 --> 00:01:18,650 -tests a different combination of values for a, b, and c. - -26 -00:01:18,650 --> 00:01:21,200 -I'm also showing, for each test case, the outcome of the overall - -27 -00:01:21,200 --> 00:01:23,930 -predicate. So, for example, if we look at the first one, the - -28 -00:01:23,930 --> 00:01:27,200 -first test case, will be such that a is true, b is - -29 -00:01:27,200 --> 00:01:30,210 -true, and c is true. And therefore, the overall outcome of - -30 -00:01:30,210 --> 00:01:34,360 -the predicate is true. Now lets consider the first condition, a. As - -31 -00:01:34,360 --> 00:01:36,300 -I said a minute ago, what we want to test are the - -32 -00:01:36,300 --> 00:01:39,160 -important combination. Which are the comibatinos - -33 -00:01:39,160 --> 00:01:41,930 -in which a single condition independently - -34 -00:01:41,930 --> 00:01:45,610 -effects the outcome of the overall predicate. So if we consider a - -35 -00:01:45,610 --> 00:01:48,490 -and we look at this possible set of this cases. Let's try to - -36 -00:01:48,490 --> 00:01:51,850 -find two test cases such that the only difference between the two - -37 -00:01:51,850 --> 00:01:54,960 -test cases is the value of a, and the overall outcome of the - -38 -00:01:54,960 --> 00:01:57,790 -predicate is different. If we look at the table, we can see - -39 -00:01:57,790 --> 00:02:00,920 -that this is true for test cases one and five. If we look - -40 -00:02:00,920 --> 00:02:04,030 -at these two cases, we can see that the overall of the predicate - -41 -00:02:04,030 --> 00:02:07,420 -in the two cases is true and false, and that the only difference - -42 -00:02:07,420 --> 00:02:10,720 -between the value of the conditions in the value of a. So - -43 -00:02:10,720 --> 00:02:13,990 -these test cases satisfy exactly what we wanted. There are two test - -44 -00:02:13,990 --> 00:02:18,270 -cases in which the value of a independently decides the overall value - -45 -00:02:18,270 --> 00:02:21,380 -of the predicate. What we do, therefore, is to add these first - -46 -00:02:21,380 --> 00:02:24,740 -two test cases to our set of tests down here. Now let's - -47 -00:02:24,740 --> 00:02:27,520 -focus on b and let's try to find two test cases such - -48 -00:02:27,520 --> 00:02:30,280 -that the value of b is the only value that changes between - -49 -00:02:30,280 --> 00:02:32,450 -the two test cases, but the overall value of the predicate is - -50 -00:02:32,450 --> 00:02:34,980 -different, the same thing we did for a. And in this case, - -51 -00:02:34,980 --> 00:02:37,610 -we can see that if we select test case number one, and test - -52 -00:02:37,610 --> 00:02:40,420 -case number three, we have exactly that situation. b is true in the - -53 -00:02:40,420 --> 00:02:44,250 -first case, false in the second one, a and c don't change, but - -54 -00:02:44,250 --> 00:02:46,830 -the overall value of the predicate changes. And now you can notice - -55 -00:02:46,830 --> 00:02:50,720 -something else. That even though we selected two test cases, tested two values, - -56 -00:02:50,720 --> 00:02:53,950 -one we already had. So, we only need three test cases overall to - -57 -00:02:53,950 --> 00:02:57,730 -test a and b according to MC/DC. Now, let's look at our last - -58 -00:02:57,730 --> 00:03:00,510 -condition, c. At this point, we know the game, so we - -59 -00:03:00,510 --> 00:03:02,780 -just have to look for two test cases that satisfy our - -60 -00:03:02,780 --> 00:03:06,380 -requirements. And in this case, one and two are suitable candidates. - -61 -00:03:06,380 --> 00:03:08,820 -And once more, because we already have one, we just have to - -62 -00:03:08,820 --> 00:03:11,070 -add two to our list. So as you can see from - -63 -00:03:11,070 --> 00:03:14,730 -this example, we went from having eight test cases needed to cover - -64 -00:03:14,730 --> 00:03:18,600 -all the possible combinations of conditions to only four test cases - -65 -00:03:18,600 --> 00:03:22,880 -to satisfy the MC/DC criteria. So let's see where MC/DC stands in - -66 -00:03:22,880 --> 00:03:25,510 -our substantion hierarchy. This is what we had so far - -67 -00:03:25,510 --> 00:03:28,430 -in the hierarchy and this is where the MC/DC criterion will - -68 -00:03:28,430 --> 00:03:31,930 -stand. MC/DC criterion is stronger than branch and condition coverage. - -69 -00:03:31,930 --> 00:03:35,210 -Why? Because it requires every single condition to be true and - -70 -00:03:35,210 --> 00:03:38,400 -false. And therefore, this section was a condition coverage criteria. - -71 -00:03:38,400 --> 00:03:41,610 -And it also requires every predicate to be true and false - -72 -00:03:41,610 --> 00:03:44,425 -and therefore, this section is branch coverage. And in addition, - -73 -00:03:44,425 --> 00:03:47,790 -it's got the additional requirements that the true and false values, - -74 -00:03:47,790 --> 00:03:50,330 -all the conditions have to also decide the overall - -75 -00:03:50,330 --> 00:03:53,190 -value of the predicate. So it's stronger. Which is more - -76 -00:03:53,190 --> 00:03:56,130 -thorough than branch and condition coverage and, as usual, - -77 -00:03:56,130 --> 00:03:59,380 -also stronger than branch coverage, statement coverage, and condition coverage. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/23 - Other Criteria - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/23 - Other Criteria - lang_en_vs4.srt deleted file mode 100644 index 27bf0fe..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/23 - Other Criteria - lang_en_vs4.srt +++ /dev/null @@ -1,275 +0,0 @@ -1 -00:00:00,120 --> 00:00:02,900 -As I mentioned at the beginning of the class, there are many, many, - -2 -00:00:02,900 --> 00:00:06,150 -many, white box criteria. And we're not going to have time to cover them - -3 -00:00:06,150 --> 00:00:08,000 -all. So what I like to do now is just to give you - -4 -00:00:08,000 --> 00:00:11,290 -the flavor, of some other criteria by just mentioning them, and saying a - -5 -00:00:11,290 --> 00:00:13,740 -couple of things, about the way they work. And the first one I - -6 -00:00:13,740 --> 00:00:15,370 -want to mention is path coverage. And - -7 -00:00:15,370 --> 00:00:17,800 -in path coverage, the test requirements are - -8 -00:00:17,800 --> 00:00:21,220 -all the paths in the program. So what that means is that to - -9 -00:00:21,220 --> 00:00:25,250 -satisfy this criteria, we need to generate enough test cases such that all - -10 -00:00:25,250 --> 00:00:27,230 -the paths in my program are covered. As you can - -11 -00:00:27,230 --> 00:00:31,970 -imagine this is incredibly expensive because any nontrivial program has got - -12 -00:00:31,970 --> 00:00:34,280 -a virtual infinite number of paths and therefore we will - -13 -00:00:34,280 --> 00:00:36,910 -need a virtual infinite number of test cases to satisfy the - -14 -00:00:36,910 --> 00:00:39,950 -path coverage criteria. Another family of criteria I want to mention - -15 -00:00:39,950 --> 00:00:43,970 -are data-flow coverage criteria and in data-flow coverage criteria, the focus - -16 -00:00:43,970 --> 00:00:47,480 -shifts from the coverage of individual elements in the code to - -17 -00:00:47,480 --> 00:00:50,320 -the coverage of pairs of elements. And in particular the coverage - -18 -00:00:50,320 --> 00:00:53,770 -of Statements, in which the content of some memory locations - -19 -00:00:53,770 --> 00:00:56,920 -modified, and statements in which the content of the same - -20 -00:00:56,920 --> 00:00:59,860 -memory location is used. So in this way, our test - -21 -00:00:59,860 --> 00:01:03,030 -will exercise the assignments of values to memory, and the - -22 -00:01:03,030 --> 00:01:06,020 -usage of those assignments,. Finally, I want to mention mutation - -23 -00:01:06,020 --> 00:01:09,150 -coverage. And this is a fairly different and new idea, - -24 -00:01:09,150 --> 00:01:11,910 -so the key concept in mutation coverage is that we - -25 -00:01:11,910 --> 00:01:16,156 -want to evaluate the goodness of our test by modifying the code. - -26 -00:01:16,156 --> 00:01:18,704 -For example here in this small I'm, I'm showing that I might - -27 -00:01:18,704 --> 00:01:21,820 -change it. An if statement from K greater than 9, to K - -28 -00:01:21,820 --> 00:01:24,310 -greater than or equal to 9. And the reason why I want to - -29 -00:01:24,310 --> 00:01:27,750 -do that, is that if I generate enough mutants, enough variation of - -30 -00:01:27,750 --> 00:01:30,340 -my program, then I can use them to assess how good are - -31 -00:01:30,340 --> 00:01:34,330 -my tests at distinguishing the original program and the mutants. And because - -32 -00:01:34,330 --> 00:01:37,430 -I'm changing the code based on the way I expect to introduce - -33 -00:01:37,430 --> 00:01:41,370 -errors in the code, the more my test can identify mutants, the better - -34 -00:01:41,370 --> 00:01:44,140 -they are at identifying real faults. This is a very high - -35 -00:01:44,140 --> 00:01:46,470 -level [UNKNOWN], but just to give you the intuition and the idea - -36 -00:01:46,470 --> 00:01:50,210 -behind these criteria, and I'm going to provide as usual, additional pointers - -37 -00:01:50,210 --> 00:01:52,930 -to go in more depth on these topics in the notes of - -38 -00:01:52,930 --> 00:01:55,170 -the class. So now, let's go back to our test criteria - -39 -00:01:55,170 --> 00:01:59,310 -subsumption hierarchy, and see how all of these criteria fit together in - -40 -00:01:59,310 --> 00:02:02,663 -this context. Let me start with multiple condition coverage. As we saw, - -41 -00:02:02,663 --> 00:02:06,740 -MC/DC is sort of as [UNKNOWN] way of doing multiple condition coverage - -42 -00:02:06,740 --> 00:02:09,229 -in the sense that it doesn't consider all the combinations, - -43 -00:02:09,229 --> 00:02:11,460 -but only the ones that are more likely to matter. - -44 -00:02:11,460 --> 00:02:14,930 -And as such, MC/DC exercises a subset of the elements - -45 -00:02:14,930 --> 00:02:16,220 -of the multiple condition coverage - -46 -00:02:16,220 --> 00:02:18,760 -exercises. And therefore, multiple condition coverage - -47 -00:02:18,760 --> 00:02:22,060 -is more thorough than MC/DC and subsumption, and it's also - -48 -00:02:22,060 --> 00:02:26,300 -as we saw incredibly more expensive. Path coverage subsumes branch coverage, - -49 -00:02:26,300 --> 00:02:28,090 -because if I cover all of the paths in the - -50 -00:02:28,090 --> 00:02:32,140 -code, I necessarily cover all the branches. However, it doesn't subsume - -51 -00:02:32,140 --> 00:02:34,130 -multiple condition coverage, MC/CD, or - -52 -00:02:34,130 --> 00:02:35,910 -branch and condition coverage. Because this - -53 -00:02:35,910 --> 00:02:38,590 -criteria, have additional requirements involving - -54 -00:02:38,590 --> 00:02:39,940 -the conditions of the predicate that - -55 -00:02:39,940 --> 00:02:42,730 -path coverage does not have. As for data-flow coverage criteria and - -56 -00:02:42,730 --> 00:02:45,010 -mutation coverage criteria, there really no - -57 -00:02:45,010 --> 00:02:46,860 -relation with the other criteria, because - -58 -00:02:46,860 --> 00:02:49,710 -they look at different aspects, of the code. So they're not - -59 -00:02:49,710 --> 00:02:52,220 -really comparable, and therefore we're just going to put them on the - -60 -00:02:52,220 --> 00:02:55,000 -side, without any relationship with the other ones. The reason why - -61 -00:02:55,000 --> 00:02:57,300 -I want to represent them all anyways is because I want to make - -62 -00:02:57,300 --> 00:02:59,950 -an important distinction between these different criteria. And - -63 -00:02:59,950 --> 00:03:02,690 -that's the distinction between practical criteria, which are - -64 -00:03:02,690 --> 00:03:05,490 -criteria that are actually not too expansive to - -65 -00:03:05,490 --> 00:03:08,790 -be used in real scenarios. And theoretical criteria, which - -66 -00:03:08,790 --> 00:03:11,590 -are criteria that are useful in theory from - -67 -00:03:11,590 --> 00:03:14,140 -the conceptual standpoint, but they're not really applicable - -68 -00:03:14,140 --> 00:03:16,540 -in practice because they're too expansive, because they - -69 -00:03:16,540 --> 00:03:19,460 -require basically too many test cases to be satisfied. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/24 - Review Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/24 - Review Quiz - lang_en_vs4.srt deleted file mode 100644 index 3f1d6ee..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/24 - Review Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,63 +0,0 @@ -1 -00:00:00,160 --> 00:00:04,260 -White box testing, in general, and coverage criteria in particular, involve some - -2 -00:00:04,260 --> 00:00:07,550 -subtle concepts, so before I conclude this lesson, I want to have a - -3 -00:00:07,550 --> 00:00:10,960 -few more quizzes to make sure that we all understand these concepts. - -4 -00:00:10,960 --> 00:00:13,610 -The first one involves a very simple piece of code, a straight line - -5 -00:00:13,610 --> 00:00:16,540 -of code, three statements, in which we simply read an integer and - -6 -00:00:16,540 --> 00:00:20,180 -then prints 10 divided by the value of that integer minus 3. Now, - -7 -00:00:20,180 --> 00:00:22,250 -let's imagine that we have a test where there consists of three - -8 -00:00:22,250 --> 00:00:25,250 -test cases for this code, and what I'm showing in the test cases - -9 -00:00:25,250 --> 00:00:28,580 -is the input and the expected output. So for the first one, - -10 -00:00:28,580 --> 00:00:31,280 -the input is 1, and I expect the output to be minus 5. - -11 -00:00:31,280 --> 00:00:34,480 -For the second one, the input is minus 1, I'm expecting to - -12 -00:00:34,480 --> 00:00:37,120 -have 2.5. And for the third one, the input is 0, and I'm - -13 -00:00:37,120 --> 00:00:42,440 -expecting to have minus 3.3333 as the result. Now the first question - -14 -00:00:42,440 --> 00:00:45,180 -I want to ask, is if we considered this test suite, and we - -15 -00:00:45,180 --> 00:00:48,050 -run it on the code, does it achieve path coverage? And remember - -16 -00:00:48,050 --> 00:00:50,960 -that path coverage is one of the strongest coverage criteria that we saw. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/25 - Review Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/25 - Review Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 19d1177..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/25 - Review Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,7 +0,0 @@ -1 -00:00:00,180 --> 00:00:04,040 -And the answer in this case is clearly yes. In fact, any input will really - -2 -00:00:04,040 --> 00:00:07,430 -achieve path coverage for this code because there is only one path in the code. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/26 - Review Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/26 - Review Quiz - lang_en_vs4.srt deleted file mode 100644 index a07e712..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/26 - Review Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,15 +0,0 @@ -1 -00:00:00,108 --> 00:00:03,890 -But now want to ask a different question which is, does this test reveal - -2 -00:00:03,890 --> 00:00:07,760 -the fault at line three? Clearly, here there is a possible problem, because - -3 -00:00:07,760 --> 00:00:10,440 -if we pass the right input, we can have a division by zero. - -4 -00:00:10,440 --> 00:00:13,760 -Is that revealed if we run this test within the code? Yes or no? diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/27 - Review Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/27 - Review Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 9945694..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/27 - Review Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,55 +0,0 @@ -1 -00:00:00,340 --> 00:00:03,581 -And the answer is no. So even path coverage's stronger criteria - -2 -00:00:03,581 --> 00:00:06,376 -that we saw is not enough to reveal this problem. And why - -3 -00:00:06,376 --> 00:00:08,526 -is that? So let me go back to a concept that - -4 -00:00:08,526 --> 00:00:12,491 -I mentioned in a previous lesson, the cost of exhaustive testing. Exhaustive - -5 -00:00:12,491 --> 00:00:15,236 -testing is really the only way in which we can exercise - -6 -00:00:15,236 --> 00:00:18,230 -all of the possible behaviors of a program. So we can say - -7 -00:00:18,230 --> 00:00:21,884 -even though it might sound like topology, that only exhaustive testing is - -8 -00:00:21,884 --> 00:00:25,710 -exhausted. All the coverage criteria that we saw are just process and - -9 -00:00:25,710 --> 00:00:30,900 -just approximation of a complete test. So test is a best effort kind - -10 -00:00:30,900 --> 00:00:33,180 -of activity. Coverage criteria help you assess - -11 -00:00:33,180 --> 00:00:35,430 -how well you tested but once more, - -12 -00:00:35,430 --> 00:00:40,060 -test can only reveal issues, can only reveal problems. You can never show the - -13 -00:00:40,060 --> 00:00:42,530 -absence of problems. And that's something is - -14 -00:00:42,530 --> 00:00:44,377 -important to remember when we do testing. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/28 - Review Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/28 - Review Quiz - lang_en_vs4.srt deleted file mode 100644 index d6dd40a..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/28 - Review Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,31 +0,0 @@ -1 -00:00:00,290 --> 00:00:03,220 -Now let's do another quiz considering a slightly different piece - -2 -00:00:03,220 --> 00:00:05,050 -of code. In this case we have five layers of - -3 -00:00:05,050 --> 00:00:08,684 -code. We have a variable i, a variable j. Variable - -4 -00:00:08,684 --> 00:00:11,620 -j is read from the input. And then based on - -5 -00:00:11,620 --> 00:00:14,500 -this predicate, we either print the value of i or - -6 -00:00:14,500 --> 00:00:16,850 -simply exit. And what I want to know from you is - -7 -00:00:16,850 --> 00:00:19,990 -whether you can create a test suite to achieve statement - -8 -00:00:19,990 --> 00:00:22,660 -coverage for this simple piece of code. Yes or no? diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/29 - Review Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/29 - Review Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 1e3228c..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/29 - Review Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,119 +0,0 @@ -1 -00:00:00,230 --> 00:00:02,520 -And the answer in this case is no. And - -2 -00:00:02,520 --> 00:00:05,480 -the reason for this is because this code is - -3 -00:00:05,480 --> 00:00:08,950 -unreachable. This is dead code because no matter the - -4 -00:00:08,950 --> 00:00:12,560 -value of j, this condition will always be false because - -5 -00:00:12,560 --> 00:00:14,770 -i will always be 0. And notice that this - -6 -00:00:14,770 --> 00:00:17,670 -is a small example, but another important truth is - -7 -00:00:17,670 --> 00:00:22,520 -that any non-trivial program contains dead or unreachable code, - -8 -00:00:22,520 --> 00:00:25,610 -code that no matter how well we test our system, - -9 -00:00:25,610 --> 00:00:28,450 -we will never be able to exercise. Why is that? Various - -10 -00:00:28,450 --> 00:00:31,030 -reasons. For example, defensive programming or, - -11 -00:00:31,030 --> 00:00:33,600 -for example, developing for future releases. - -12 -00:00:33,600 --> 00:00:36,320 -So there might be pieces of code that are added, but they're - -13 -00:00:36,320 --> 00:00:39,380 -still not activated. They're still not invoked by any other part of - -14 -00:00:39,380 --> 00:00:41,970 -the code because of modifications to the code. There are some - -15 -00:00:41,970 --> 00:00:44,480 -parts that were executed before and in the new version of the - -16 -00:00:44,480 --> 00:00:47,915 -program, they are no longer exercised. They are no longer executable. But - -17 -00:00:47,915 --> 00:00:50,760 -they still remain in the code base. And this is a very, - -18 -00:00:50,760 --> 00:00:54,560 -very normal situation for this reason and many more. And this is - -19 -00:00:54,560 --> 00:00:56,790 -an important concept because this affects - -20 -00:00:56,790 --> 00:00:58,760 -the very meaning of coverage measures. If - -21 -00:00:58,760 --> 00:01:01,225 -there is some unreachable code, we will never be able to reach - -22 -00:01:01,225 --> 00:01:04,709 -100% code coverage. And in fact, if you remember, at the beginning of - -23 -00:01:04,709 --> 00:01:07,380 -the lesson, we discussed this concept and asked you why you think - -24 -00:01:07,380 --> 00:01:10,470 -that, in the industry, the target for coverage is not 100% but less - -25 -00:01:10,470 --> 00:01:12,850 -that that. And that's the answer, to account for the fact that there - -26 -00:01:12,850 --> 00:01:15,890 -are infeasibility problems that I have to take into account when I test - -27 -00:01:15,890 --> 00:01:19,920 -the code, infeasible paths, unexecutable statements, conditions that can never - -28 -00:01:19,920 --> 00:01:22,780 -be true, and so on. So most criteria, if not - -29 -00:01:22,780 --> 00:01:24,620 -all, are affected, and we need to take that into - -30 -00:01:24,620 --> 00:01:27,270 -account when we try to achieve a given coverage target. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/3 - Coverage Criteria Intro - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/3 - Coverage Criteria Intro - lang_en_vs4.srt deleted file mode 100644 index b1a38a7..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/3 - Coverage Criteria Intro - lang_en_vs4.srt +++ /dev/null @@ -1,171 +0,0 @@ -1 -00:00:00,320 --> 00:00:04,059 -So let's start our lesson on white docs testing by considering again the - -2 -00:00:04,059 --> 00:00:07,470 -program PrintSum. If you remember, this is the same program that we used when - -3 -00:00:07,470 --> 00:00:10,300 -we were talking about black box testing. It's the program that takes two - -4 -00:00:10,300 --> 00:00:14,050 -integers, A and B, and produces, as a result, the sum of the two. - -5 -00:00:14,050 --> 00:00:16,570 -And when we were looking at this problem in the context of black - -6 -00:00:16,570 --> 00:00:17,760 -box testing, we did not look at - -7 -00:00:17,760 --> 00:00:19,700 -implementation. But that's exactly what we're going to - -8 -00:00:19,700 --> 00:00:22,090 -do now. So we're going to open the box and look at how the - -9 -00:00:22,090 --> 00:00:25,700 -code is implemented. And as you can see, the programmer was kind of creative. - -10 -00:00:25,700 --> 00:00:28,670 -Because instead of just adding the two numbers and printing them, he - -11 -00:00:28,670 --> 00:00:32,080 -or she also decided to print them in a specific color depending - -12 -00:00:32,080 --> 00:00:36,840 -on whether they were positive numbers or negative numbers. So positive results - -13 -00:00:36,840 --> 00:00:40,040 -are printed in red and negative results are printed in blue. And as - -14 -00:00:40,040 --> 00:00:41,970 -you can see by looking at the code we can see some - -15 -00:00:41,970 --> 00:00:45,150 -interesting cases that we might want to test. For instance you can - -16 -00:00:45,150 --> 00:00:48,190 -see that there are two decisions made here. So we might decide - -17 -00:00:48,190 --> 00:00:51,100 -that this is an interesting case and therefore we want to test it. - -18 -00:00:51,100 --> 00:00:53,770 -Similarly we might look at this other case and we might - -19 -00:00:53,770 --> 00:00:56,465 -also decide that this is another interesting case and therefore we - -20 -00:00:56,465 --> 00:00:58,970 -want to test this one as well. So let's discuss this - -21 -00:00:58,970 --> 00:01:01,480 -in a slightly more formal way by introducing the concept of - -22 -00:01:01,480 --> 00:01:05,110 -coverage criteria which are really the essence of why box testing. - -23 -00:01:05,110 --> 00:01:08,930 -First of all coverage criteria are defined in terms of test - -24 -00:01:08,930 --> 00:01:13,460 -requirements where test requirements are the elements, the entities in the - -25 -00:01:13,460 --> 00:01:16,250 -code that we need to exercise. That we need to execute - -26 -00:01:16,250 --> 00:01:18,690 -in order to satisfy the criteria. And we'll see plenty - -27 -00:01:18,690 --> 00:01:21,430 -of examples of that. And normally, when I apply a coverage - -28 -00:01:21,430 --> 00:01:24,810 -criterion, my result is a set of test specifications. And we - -29 -00:01:24,810 --> 00:01:26,540 -already saw test specifications. Those - -30 -00:01:26,540 --> 00:01:29,540 -are basically descriptions, specifications, of how - -31 -00:01:29,540 --> 00:01:32,786 -the tests should be in order to satisfy the requirements. And - -32 -00:01:32,786 --> 00:01:36,210 -they also result in actual test cases, which are instantiations of - -33 -00:01:36,210 --> 00:01:39,470 -the test specifications. And again this is exactly analogous to what - -34 -00:01:39,470 --> 00:01:41,830 -we saw when we were talking about the black box testing. - -35 -00:01:41,830 --> 00:01:43,960 -So let's see what this means by going back to our - -36 -00:01:43,960 --> 00:01:46,680 -example. A minute ago, we looked at the print sum code - -37 -00:01:46,680 --> 00:01:50,000 -and we identified two interesting cases for our code. And those - -38 -00:01:50,000 --> 00:01:53,430 -are exactly our test requirements. So we have a first test - -39 -00:01:53,430 --> 00:01:57,840 -requirement here, which is the execution of this particular statement and - -40 -00:01:57,840 --> 00:02:01,500 -a second requirement here and this one corresponds to the execution - -41 -00:02:01,500 --> 00:02:04,580 -of this other statement. So for this example there are two - -42 -00:02:04,580 --> 00:02:06,920 -things that we need to do in order to satisfy our - -43 -00:02:06,920 --> 00:02:10,100 -coverage requirements. Execute this statement and execute this statement. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/30 - Summary - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/30 - Summary - lang_en_vs4.srt deleted file mode 100644 index 264491d..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/30 - Summary - lang_en_vs4.srt +++ /dev/null @@ -1,127 +0,0 @@ -1 -00:00:00,190 --> 00:00:02,800 -Now let me conclude the lesson by summarizing a few important - -2 -00:00:02,800 --> 00:00:06,840 -aspects of white-box testing. The first important aspect is that white-box - -3 -00:00:06,840 --> 00:00:09,770 -testing works on a formal model. The code itself are models - -4 -00:00:09,770 --> 00:00:12,430 -derived from the code. So when we do white-box testing, we - -5 -00:00:12,430 --> 00:00:15,160 -don't need to make subjective decision, for example, on the level - -6 -00:00:15,160 --> 00:00:18,890 -of obstruction of our models. Normally, we simply represent what's there. - -7 -00:00:18,890 --> 00:00:22,010 -And so what we will obtain are objective, results and objective - -8 -00:00:22,010 --> 00:00:25,400 -measures. As I also said at the beginning, coverage criteria allows - -9 -00:00:25,400 --> 00:00:28,790 -us to compare different test suites, different sets of tests, - -10 -00:00:28,790 --> 00:00:31,780 -because I can measure the coverage achieved by one test suite - -11 -00:00:31,780 --> 00:00:34,280 -and by the other, and then decide which one to use - -12 -00:00:34,280 --> 00:00:37,400 -based on this measure. And again, remember, these measures aren't perfect, - -13 -00:00:37,400 --> 00:00:40,700 -but they at least give you an objective number, an objective - -14 -00:00:40,700 --> 00:00:44,470 -measure of the likely effectiveness of your tests. So even though - -15 -00:00:44,470 --> 00:00:48,130 -achieving 100% coverage does not mean that you identify all the - -16 -00:00:48,130 --> 00:00:50,710 -problems in the code for sure. If your level of coverage - -17 -00:00:50,710 --> 00:00:53,700 -is 10%, for example, for stemen coverage. That means that - -18 -00:00:53,700 --> 00:00:57,360 -you haven't exercised 90% of your code, and therefore the trouble - -19 -00:00:57,360 --> 00:01:00,565 -is in a piece of software that is inadequately tested - -20 -00:01:00,565 --> 00:01:03,600 -and likely to be of inadequate quality. We also saw that - -21 -00:01:03,600 --> 00:01:07,370 -there are two broad classes of coverage criteria, practical criteria - -22 -00:01:07,370 --> 00:01:10,430 -that we can actually use, and theoretical criteria that are interesting - -23 -00:01:10,430 --> 00:01:13,410 -from a conceptual standpoint, but that they are totally impractical. - -24 -00:01:13,410 --> 00:01:16,170 -They are too expensive to be used on real world software. - -25 -00:01:16,170 --> 00:01:18,350 -And finally, as we also said at the beginning, - -26 -00:01:18,350 --> 00:01:20,820 -one of the great things about white box testing and - -27 -00:01:20,820 --> 00:01:23,910 -coverage criteria, is that they are fully automatable. There are - -28 -00:01:23,910 --> 00:01:27,380 -tools that can take your code, instrument it automatically. And - -29 -00:01:27,380 --> 00:01:29,050 -when you run your test cases, they will tell - -30 -00:01:29,050 --> 00:01:31,330 -you, what is the level of coverage that you achieve - -31 -00:01:31,330 --> 00:01:34,410 -with your test at no cost for you. So there's - -32 -00:01:34,410 --> 00:01:36,763 -really no reason not to measure coverage of your code. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/4 - Coverage Criteria Intro Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/4 - Coverage Criteria Intro Quiz - lang_en_vs4.srt deleted file mode 100644 index 2adce93..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/4 - Coverage Criteria Intro Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,39 +0,0 @@ -1 -00:00:00,620 --> 00:00:03,310 -Now let's see if we are all on the same page - -2 -00:00:03,310 --> 00:00:07,180 -on the concept of testing specifications, and we're going to do that through - -3 -00:00:07,180 --> 00:00:09,280 -a quiz. What I would like for you to do is to - -4 -00:00:09,280 --> 00:00:12,830 -tell me, what are some possible test specifications that will satisfy the - -5 -00:00:12,830 --> 00:00:15,380 -requirements that we just saw? And I want you to express this - -6 -00:00:15,380 --> 00:00:17,270 -specification in terms of constraints on - -7 -00:00:17,270 --> 00:00:19,166 -the inputs. Which means, what constraints - -8 -00:00:19,166 --> 00:00:23,130 -should the input satisfy in order for this statement to be executed, - -9 -00:00:23,130 --> 00:00:26,210 -and this statement to be executed, and you can write your answer - -10 -00:00:26,210 --> 00:00:27,510 -here in these two slots. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/5 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/5 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 2a1ff16..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/5 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,47 +0,0 @@ -1 -00:00:00,080 --> 00:00:02,560 -To satisfy our first requirements, we need to find an - -2 -00:00:02,560 --> 00:00:06,300 -input that causes the execution of this statement. Because this statement - -3 -00:00:06,300 --> 00:00:09,080 -is executed only when result is greater than zero, our - -4 -00:00:09,080 --> 00:00:12,815 -test requirement is that a plus b must be greater than - -5 -00:00:12,815 --> 00:00:16,219 -0. When this is satisfied, this statement is executed therefore - -6 -00:00:16,219 --> 00:00:19,960 -any test case that implements this specification will cause the execution - -7 -00:00:19,960 --> 00:00:22,770 -of this statement. Similarly if we want to cover this statement - -8 -00:00:22,770 --> 00:00:25,220 -which is our second requirement we need to have a result - -9 -00:00:25,220 --> 00:00:27,370 -that is less than 0. Again, the result is equal - -10 -00:00:27,370 --> 00:00:29,490 -to a plus b, so all we need to do - -11 -00:00:29,490 --> 00:00:32,450 -is find the test case such that a plus b - -12 -00:00:32,450 --> 00:00:34,700 -is less than 0. So this is pretty straight forward. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/6 - Coverage Criteria Intro Quiz - lang_en_vs3.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/6 - Coverage Criteria Intro Quiz - lang_en_vs3.srt deleted file mode 100644 index 42b5720..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/6 - Coverage Criteria Intro Quiz - lang_en_vs3.srt +++ /dev/null @@ -1,63 +0,0 @@ -1 -00:00:00,220 --> 00:00:03,780 -So, now that we have our test specifications. Test specification number one that - -2 -00:00:03,780 --> 00:00:06,740 -says that a plus b must be greater than zero. And test specification - -3 -00:00:06,740 --> 00:00:09,990 -number two, for which a plus b must be less than zero. I'd - -4 -00:00:09,990 --> 00:00:12,580 -like to do another small quiz and ask you to write some test - -5 -00:00:12,580 --> 00:00:16,440 -cases that will implement these specifications. And I want you to write the - -6 -00:00:16,440 --> 00:00:19,780 -test cases in this format. I want you to specify for each one - -7 -00:00:19,780 --> 00:00:22,300 -of the test cases, what is the value of a that you need - -8 -00:00:22,300 --> 00:00:25,160 -to use. What is the value of b that you need to use. - -9 -00:00:25,160 --> 00:00:28,200 -And since a test case, I like to remind you, is - -10 -00:00:28,200 --> 00:00:30,960 -not just a set of inputs. But is the set of - -11 -00:00:30,960 --> 00:00:34,050 -inputs plus expected output? I also want you to specify, what - -12 -00:00:34,050 --> 00:00:37,040 -is the expected output for these test cases? And in particular, - -13 -00:00:37,040 --> 00:00:39,540 -since we have two characteristics of the output, one is the - -14 -00:00:39,540 --> 00:00:41,630 -color of the output and the other one is the actual - -15 -00:00:41,630 --> 00:00:44,040 -value. I want you to specify for each test case, what - -16 -00:00:44,040 --> 00:00:46,790 -is the expected output color and what is the expected value? diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/7 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/7 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt deleted file mode 100644 index 749a57e..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/7 - Coverage Criteria Intro Quiz Solution - lang_en_vs4.srt +++ /dev/null @@ -1,87 +0,0 @@ -1 -00:00:00,090 --> 00:00:02,360 -In this case like in many other cases, there's not just a - -2 -00:00:02,360 --> 00:00:05,550 -single right answer because you can build many test cases that will - -3 -00:00:05,550 --> 00:00:09,730 -satisfy this test specification. So for example, we could pick value 3 - -4 -00:00:09,730 --> 00:00:13,640 -for a and value 9 for b. Those satisfy the specification because a - -5 -00:00:13,640 --> 00:00:16,400 -plus b is equal to 12 and therefore is greater than 0, - -6 -00:00:16,400 --> 00:00:19,470 -and therefore this is a test case that implements this test specification. - -7 -00:00:19,470 --> 00:00:21,960 -And in terms of results, what we expect to see is in - -8 -00:00:21,960 --> 00:00:25,200 -the case of a result greater than 0, the caller should be red, - -9 -00:00:25,200 --> 00:00:28,109 -and the upper value should be 12. And obviously for this test - -10 -00:00:28,109 --> 00:00:30,971 -specification, we just need to pick two inputs such that the sum - -11 -00:00:30,971 --> 00:00:33,674 -of the two inputs is less than 0. So for example, we - -12 -00:00:33,674 --> 00:00:37,125 -could pick minus 5 and minus 8. The output color in this - -13 -00:00:37,125 --> 00:00:40,230 -case is going to be blue, and the output value is going to be - -14 -00:00:40,230 --> 00:00:43,780 -minus 13. So, what we just saw is basically how we can - -15 -00:00:43,780 --> 00:00:47,060 -go from a piece of code to a set of requirements, which - -16 -00:00:47,060 --> 00:00:50,260 -are the interesting aspects of the code that we want to exercise. How we - -17 -00:00:50,260 --> 00:00:52,800 -can satisfy the requirements by finding - -18 -00:00:52,800 --> 00:00:54,700 -the right test specifications, and then how - -19 -00:00:54,700 --> 00:00:58,450 -we can initiate the test specifications into actual test cases. And this is - -20 -00:00:58,450 --> 00:01:01,010 -what we will do in general when doing white books testing. And we'll - -21 -00:01:01,010 --> 00:01:02,520 -do things likely different way, depending on - -22 -00:01:02,520 --> 00:01:04,230 -the specific criteria that we are considering. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/8 - Statement Coverage - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/8 - Statement Coverage - lang_en_vs4.srt deleted file mode 100644 index 23cfb0a..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/8 - Statement Coverage - lang_en_vs4.srt +++ /dev/null @@ -1,291 +0,0 @@ -1 -00:00:00,270 --> 00:00:03,080 -Now that we saw this overview of white-box testing, I'd like - -2 -00:00:03,080 --> 00:00:06,270 -to start talking about specific coverage criterion. And I'm going to start - -3 -00:00:06,270 --> 00:00:09,010 -with the first one, which is Statement Coverage. This criterion is - -4 -00:00:09,010 --> 00:00:12,355 -going to be characterized by two aspects, the first one is which - -5 -00:00:12,355 --> 00:00:15,580 -are the Test requirements for the criteria and the second one - -6 -00:00:15,580 --> 00:00:18,830 -is how we measure Coverage for that criteria. In the case - -7 -00:00:18,830 --> 00:00:22,200 -of statement coverage, these test requirements are all the statements in - -8 -00:00:22,200 --> 00:00:25,804 -the program. So this is the basic, the first the, the simplest - -9 -00:00:25,804 --> 00:00:29,860 -coverage criteria in the white-box arena. Let me remind you the assumption - -10 -00:00:29,860 --> 00:00:33,280 -that we made at the beginning. White-box testing is based on the assumption - -11 -00:00:33,280 --> 00:00:35,990 -that if there isn't a faulty element in the code, we need - -12 -00:00:35,990 --> 00:00:38,390 -to exercise it. We need to execute it, in order to find the - -13 -00:00:38,390 --> 00:00:41,260 -fault. And that's exactly what statement coverage does. If there is a - -14 -00:00:41,260 --> 00:00:44,450 -statement that is faulty in the code, we need to exercise it, in - -15 -00:00:44,450 --> 00:00:47,490 -order to find the fault. And therefore, a good measure of how well - -16 -00:00:47,490 --> 00:00:51,480 -we exercise the code, is the ratio of the number of executed statements. - -17 -00:00:51,480 --> 00:00:54,950 -So all the statements that my test cases executed, to the total - -18 -00:00:54,950 --> 00:00:58,570 -number of statements in the program. The higher this number, the better - -19 -00:00:58,570 --> 00:01:01,870 -I exercise my code. And we can also look at coverage criterion - -20 -00:01:01,870 --> 00:01:04,290 -in terms of questions. So what is the questions they were trying - -21 -00:01:04,290 --> 00:01:06,940 -to answer when we look at a specific set of test cases - -22 -00:01:06,940 --> 00:01:09,870 -and we assess the statement coverage that they achieved. And the question - -23 -00:01:09,870 --> 00:01:13,440 -is whether each statement in the program has been executed. So, statement - -24 -00:01:13,440 --> 00:01:16,920 -coverage is satisfied when all the statements in the program have been executed. - -25 -00:01:16,920 --> 00:01:19,640 -And we can satisfy to different degrees and the degrees to which it's - -26 -00:01:19,640 --> 00:01:23,320 -satisfied is measured by this value. So now let's go ahead and measure - -27 -00:01:23,320 --> 00:01:27,250 -statement coverage on our printSum example. What I'm going to show down here is - -28 -00:01:27,250 --> 00:01:30,500 -this progress bar in which we show the amount of coverage, the percentage of - -29 -00:01:30,500 --> 00:01:33,060 -coverage achieved. So what this means is that the, if I get to - -30 -00:01:33,060 --> 00:01:36,680 -this point I've covered 25% of the statements in the code. And my goal - -31 -00:01:36,680 --> 00:01:39,160 -is to get up here to cover all the statements in the code. - -32 -00:01:39,160 --> 00:01:42,000 -We have two test cases for this code. The first one that we just - -33 -00:01:42,000 --> 00:01:45,700 -saw, consists of the inputs a equal to 3 and b equal - -34 -00:01:45,700 --> 00:01:48,200 -to 9, and the second one has the inputs a is equal to - -35 -00:01:48,200 --> 00:01:50,840 -minus 5 and b is equal to minus 8. So now let's see - -36 -00:01:50,840 --> 00:01:53,450 -what happens when we run this test case. When we run this test - -37 -00:01:53,450 --> 00:01:56,920 -case, I'm going to show you by highlighting in the code the parts that - -38 -00:01:56,920 --> 00:02:00,430 -we cover when we start executing the code. We cover the first statement, - -39 -00:02:00,430 --> 00:02:01,910 -then we always execute the second - -40 -00:02:01,910 --> 00:02:04,180 -statement, which computes the result, we continue - -41 -00:02:04,180 --> 00:02:07,070 -the execution, we get to the if statement. If the result is greater - -42 -00:02:07,070 --> 00:02:10,038 -than zero, in this case our result is 12 because we - -43 -00:02:10,038 --> 00:02:12,360 -are working with the inputs 3 and 9, and therefore we - -44 -00:02:12,360 --> 00:02:15,710 -execute the true part of the if, we execute the statement. - -45 -00:02:15,710 --> 00:02:18,850 -And at this point, we just jump to the end. Because we - -46 -00:02:18,850 --> 00:02:21,530 -do not execute the else part of the statement, since we - -47 -00:02:21,530 --> 00:02:24,240 -have executed a true one, and therefore, we cover this final - -48 -00:02:24,240 --> 00:02:27,534 -statement. So at the end of the execution of this test - -49 -00:02:27,534 --> 00:02:32,313 -case, we cover one, two, three, four, five statement out of seven - -50 -00:02:32,313 --> 00:02:36,299 -which is roughly speaking 71%. So we can mark in here - -51 -00:02:36,299 --> 00:02:39,779 -that we more or less got to 71% of coverage for - -52 -00:02:39,779 --> 00:02:42,719 -this code. Now let's look at what happens when we execute - -53 -00:02:42,719 --> 00:02:45,700 -test case number two. In this case again, we execute the - -54 -00:02:45,700 --> 00:02:48,660 -first statement, the second statement, the third statement. In this case - -55 -00:02:48,660 --> 00:02:52,010 -though, the first statement, when it evaluates the value of result, - -56 -00:02:52,010 --> 00:02:54,250 -it sees that the result is not greater than zero because - -57 -00:02:54,250 --> 00:02:57,590 -our inputs are minus five and minus eight. Therefore, you will execute - -58 -00:02:57,590 --> 00:03:00,090 -line number five. And because the result is less than - -59 -00:03:00,090 --> 00:03:02,810 -zero, you will also execute line number six. So, at - -60 -00:03:02,810 --> 00:03:05,230 -this point, all of the statements in our code are - -61 -00:03:05,230 --> 00:03:09,360 -executed and therefore, we achieved a 100% statement coverage, which - -62 -00:03:09,360 --> 00:03:13,090 -was our goal. Before looking at other kinds of coverage, - -63 -00:03:13,090 --> 00:03:16,060 -let's see how our statement coverage is used in practice. - -64 -00:03:16,060 --> 00:03:19,170 -First of all, statement coverage is the most used kind - -65 -00:03:19,170 --> 00:03:22,830 -of coverage criterion in industry. Normally for company that uses - -66 -00:03:22,830 --> 00:03:27,200 -statement coverage, the typical coverage target is 80-90%, which - -67 -00:03:27,200 --> 00:03:29,690 -mean the outcome of the test should be such - -68 -00:03:29,690 --> 00:03:32,770 -that 80-90% of the statements are exercised at the - -69 -00:03:32,770 --> 00:03:34,880 -end of testing. So at this point, you might - -70 -00:03:34,880 --> 00:03:36,850 -be wondering, why don't we just shoot for 100%? - -71 -00:03:36,850 --> 00:03:38,580 -Why don't we try to cover all of the - -72 -00:03:38,580 --> 00:03:40,050 -code? We just saw that we could do it. - -73 -00:03:40,050 --> 00:03:41,730 -And so I'm going to ask you the same question. diff --git a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/9 - Statement Coverage Quiz - lang_en_vs4.srt b/usth/ICT2.7/P4L3 White-Box Testing Subtitles/9 - Statement Coverage Quiz - lang_en_vs4.srt deleted file mode 100644 index 893fe06..0000000 --- a/usth/ICT2.7/P4L3 White-Box Testing Subtitles/9 - Statement Coverage Quiz - lang_en_vs4.srt +++ /dev/null @@ -1,15 +0,0 @@ -1 -00:00:00,100 --> 00:00:01,540 -so, I would like to hear from you. Why - -2 -00:00:01,540 --> 00:00:03,850 -do you think that we don't aim normally at - -3 -00:00:03,850 --> 00:00:05,939 -100 % college but, slightly less than that and - -4 -00:00:05,939 --> 00:00:07,560 -I want you to put your answer right here. |