In this demo, we're going to do exactly what we
did just now in the lesson. We're going to use
the category partition method to go from a high-level
description of a piece of software of a program to
a set of test cases for that program. To
do that, we're going to use a simple tool. So
I'm going to show you here the tool that is
called a tsl generator right here. This tool is available
to you, so you can look in the class notes to
see information on how to download it. And together with the tool,
we are also going to provide a manual for the tool, and
a set of files that I'm going to use in this demo. So
you should be able to do exactly what I'm doing. So
again, all of those are available from the class notes. So specifically,
today we're going to write test cases for the grep program. So in
case you're familiar with the grep utility, this is a simplified version
of that utility. So basically the grep utility allows you
to search a file for the occurrences of a given
pattern. So you can invoke it, as it's shown here
in the synopsis, by executing grep, the pattern that you're
looking for, and the filename in which you want to
look for the pattern. And let me read the description
of the grep utility. The grep utility searches files for
a pattern and brings all lines that contain that pattern
on the standard output. A line that contains multiple occurrences of
the pattern is printed only once. The pattern is any sequence
of characters. To include a blank in the pattern, the entire
pattern must be enclosed in single quotes. To include a quote
sign in the pattern, the quote sign must be escaped, which
means that we have to put a slash in front of
the quotes sign. And in general, it is safest to enclose
the entire pattern in single quotes. So this is our high
level description for the program, for the softer system, that
we need to test. So now let me show you
what a possible set of categories and partitions could be
for this program. So what I have here is a
file, a textual file, which contains all the categories and
partitions for the elements that are relevant for my program.
In particular, when we look at the file, we can
see that the file can be characterized by its size.
And in this case, I've got two choices. The file can
be empty or not empty. The second characteristic of the file
that I'm considering is the number of occurrences of the pattern
in the file. And I'm considering that the pattern might not occur
in the file or it might occur once, or multiple times.
I'm not going to go through the rest of the file because we
already covered how to apply the category partition method in the
lesson. So if you had doubts about that, about the method and
how to apply, you might want to go back and watch again the
lesson. What I want to show you here is how you can go
from this information that you have here, that we have derived by
applying the, the first steps of the method, to a set of
test frames, and then, a set of test packs. So to do
that we're going to use the tool that I just mentioned. So let
me bring back my terminal. So first of all, let's see how
we can run the tool. So you have a manual that will explain
all the details on how to build the file that we're
going to feed the tool. So what is the format and so
on. Here I'm just going to see how I can run the
tool. So first of all, let me point out that this was
developed together by professors from the University of California Irvine and
Oregon State University. And as you can see, we can run
TSL generator and specify that we want to see the main
page. So in this case if we run it this, this way,
you'll have some basic information on how to run
the tool. And from the main page you can see
that you can specify the minus c flag and in
this case the TSL generator will report the number of
test frames generated without writing them to output. For
example, you might want to use this as we will do
to see how many tests that you will generate with
a current set of category partitions and choices. The minus
s option will bring the result of the TSL
generator on the standard output. And finally, you can
use minus o to specify an output file, where
to put the output of the program. So let's
at first run our TSL generator by specifying the
minus c option and by bypassing our current set
of category partitions and choices. Okay, so let me
remind you that what the, the tool will do
is what we will do manually. Otherwise, which is
to combine all these choices so as to have one
test case for each combination. So if we do
that, you can see that the tool tells us that
we will generate 7776 test frames in this case.
And this seems to be a little too much for
a program as small as the one that we are
testing. And assume for instance that we don't have the
resources to run this many test cases for, for
the grep program. In addition, consider that in this case,
we're computing all possible combinations of choices. And there's going to
be some combination that do not make sense as we
discussed in the lesson. So what we might want to
do in this case is to go back to our
spec and start adding constraints to eliminate this meaningless combination.
So I'm going to show you the result of doing that.
And I'm going to show you a few examples.
For example here, when the file is empty, I'm going to
define this property empty file. And how am I
going to use this property? Well for example here, it
doesn't make sense to consider the case in which
we have one or many occurrences of the pattern in
the file if the file is empty. Therefore I'm
going to tell the tool that it should consider this specific
choice only if the file is not empty, only if
empty file is not defined. And that will skip, for
example, all of the combinations in which the file is
empty. And I'm trying to generate the test case that has
one occurrence of the pattern in the file, which is
simply not possible. For another example, in case I have
an empty pattern, I define the property empty pattern. And
then I avoid the choices that involve the pattern in case
the pattern is empty. because, for example, I cannot
have quotes in a pattern that is empty. For example,
it doesn't make sense to have blanks. So, one or
more blanks if the pattern is empty. So I'm going to
specify again that this choice should be considered only if
we don't have an empty pattern. And so on and
so forth. So now after I edit these constraints, I
can go back and compute again the number of test
frames and therefore the test cases that will be
generated with these constraints. So let me go again
to my terminal. Okay, so now I'm going to run
my TSL generator again, and I'm going to run it on
the second version of this file. And you can
see that I reduced the, the number of test frames
from about 7800 to about 1700. So it's quite
a, quite a big reduction by eliminating all these combinations
that do not make sense. But let's assume again that
we want to reduce this further so that we don't want to
generate those many test frames and therefore test cases. So
what can we do? We go back to our spec. And
in this case, we start adding error constraints. So if
you remember what we said in the lesson, error constraints are
constraints that indicate a choice that has to do with an
erroneous behaviour. For example, an erroneous input provided to the problem.
So here for instance, we're indicating the presence
of incorrectly enclosing quotes as an error choice. Same
thing if there's no file corresponding to the
name that we provide to the tool, we say
that this corresponds to an error. So how
is the tool going to use this information? It uses
this information by producing only one combination that involves
error choices, instead of combining them with other choices.
So let's see what happens after we added this
error constraints. So we go back to our console
once more. And in this case, we want to run
the TSL generator with the version of the, of my
file that contains the area of constraints. And again,
I reduce quite a bit the number of test frames.
So now I have only 562 test frames that
will be generated by using the file that I provided.
So for the last time, let's assume that we really want
to cut down the number of test frames or the number of
test cases. So once more, we go back to our file, and
at this point what we can add is the final type of
constraints that we have, which are
single constraints. And single constraints are
basically indicated choices that we don't want to combine with other choices.
So they have the same effect of the error constraints, but they
have a different meaning, so they do not indicate choices that corresponds
to an error. In other words, I can use a
single constraints to identify choices that I want to test only once.
So for example in this case, I might decide that I
want to have only one test frame that tests my program
with a file being empty and I can do the
same for other choices. So basically I can continue adding this
single constraint until I get down to the number of test
frames and therefore the number of test cases that I want.
So now let's go back once more to our console. And so now if we run
using this file as input, you can see that we have 35 test frames generated. So
this is a fairly low number of test cases, so we might decide that we want to
go ahead and write these test frames to a file. So now let's open this file
that we just generated. And as you can see here, I have exactly 35 test frames,
as expected. Some of those correspond to the single and error cases. So in this
case, the only choice that I have indicated
is the one that corresponds to the single
or error constraint. What is for the other
ones? I actually have the whole test spec.
So let's pick one just to give you an example.
In this case, that's frame number 15 that will correspond to
test case number 15. And here you can see that
we have all the information. So this is a test specification.
All the information that we need to generate the corresponding
test. We know that we need a file that is not
empty. That we need to have one occurrence of the pattern
in the file. One occurrence of the pattern in one line.
The position of the pattern in the file can
be any position. The length of the pattern must
be more than one character. The pattern should not
be enclosed in quotes. There should be one white
space, one quote within the pattern, and finally the
file that would pass through the program should exist.
So the file should be present. So I can
easily transform all of this into an actual test case.
And notice that even though we're not, we're not
going to do it here. In cases like this, it might
even be possible to automatically generate the test cases
from the test specifications because, here for example, here it
should be relatively straight forward to parse these test
specifications and generate test cases accordingly. So, just to summarize,
what we have done is to go from one high-level
description of a program to a set of categories, partitions,
and choices for that program. Then we have combined them
in different ways, adding more and more constraints to reduce the
number of combinations until we ended up with the right number
of test cases, so the number of test cases that we
were fine generating. We generated the corresponding test specifications. And at
that point, we could just go ahead, generate the test case,
and test our application. So, and you can see how this
can result in a much more thorough testing of your application.
Because instead of reading this description and just trying to come up with test
cases for it, we can break down the process in steps that are easy to perform
individually. They can be automated as much
as possible. And they will end up with
a set of test cases that will test
all the interests and aspects of your application.