And before I start talking about specific testing techniques, there's
something else that I want to discuss, which is Testing
Granularity Levels. So let's consider a software system, a system
made out of components that interact with one another. So the
first level that we consider in testing is called Unit
Testing, which is the testing of the individual units or modules
in isolation. The next step, is to see there are
multiple modules and their interactions. And this is called Integration Testing.
So, integration testing is the testing of the interactions among
different modules. And it can be performed according to different
strategies. Depending on the order in which the modules are
integrated and on whether we integrate one module at a time
or multiple modules together, all at once. And in this
latter case, we call this kind of integration testing, the one that
integrates all the modules at once, Big Bang integration testing.
And after performing integration testing, the next step is to test
the complete system as a whole. And this level of
testing is normally called, System Testing. So system testing in the
testing of the complete system and it includes both functional and
non functional testing. We will discuss functional and non functional testing
in details in the next two lessons. But I just
want to give you an idea of what they are intuitively. Functional
tests are the test that aim to verify the functionality provided
by the system. For example if you consider the function double
value that we saw earlier in the lesson, a
functional test will try to assess that that function
is producing the right value given a specific input.
Conversely, no functional test are the one that target, as
surprisingly, no functional properties of the system. For example,
no functional test will include performance tests, load tests,
robustness tests. In general, no functional tests will try
to assess different qualities of the system, such as reliability,
maintainability, usability, so basically, all the ilities that you can
think about. In addition to these three basic testing levels, there
are two more levels that I want to consider and that
I want to discuss. And they both involve the whole system.
And the first one is Acceptance Testing which is the
validation of the software against the Customer requirements. So this is
the testing that makes sure that the system does what the
customer wants it to do. And the last type of testing
that I want to mention is Regression Testing. And regression testing
is the type of testing or retesting, that we perform every time
that we change our system. And we need to make sure
that the changes behave as intended and that the unchanged code is
not negatively affected by the modification, by these changes. In fact,
what can happen when you modify the code is that parts of
the code that are related to the changes, are actually affected
by the changes, and start misbehaving. And we call those regression errors.
And regression errors, are very common. For example, you're probably
familiar with the situation in which, one software update is
released, and just a few days later, another software update
is released. In many cases that happens because the first update
was containing regression errors. So the changes in the code
that broke some functionality, that resulted in failures on the user's
machine and in bug reports and therefore that caused further
maintenance, further bug fixes, and a release on a new version.
Something else I'd like to mention about regression testing, is that
regression testing is one of the main causes why software maintenance is
so expensive. And that's also why researchers have invested a great
deal of effort into refining regression
testing techniques that can make regression
testing more effective and more efficient. So let me leave you,
with a little piece of advice which is try to automate as
much as possible regression testing. For example use scripts, use tools, make
sure to save your harness, make sure to save your input, and
outputs for the test, because you want to be able
to rerun your test, at a push of a button as
much as possible every time you change your code, to
avoid the presence of regression errors in the code you release.