Software Engineering, Page 053

C. Types of testing

For our purposes here a component will refer to a coherent portion of the system which we plan to test independently. If it does not contain a (proper) sub-component (i.e. something smaller that will be tested), then it is refered to as a unit. A unit might be a class or a package containing a few classes that are tightly coupled, and that we prefer to test together. A thorough or testing plan will call for each of these units to be tested independently, and integrated into larger subsytems that will also be tested.

When testing a component A of the system, it is often necessary to simulate the behavior of another component B. This might be because A needs the functionality of B. If so, and if B has not been fully developed (maybe not even begun), then it becomes necessary to substitute a fake version of B called a test stub. The stub might have some of the functionality of the real deal, but is expected to fake at least a portion of it. So for example, if some method of B is required to do a complex computation and return a floating-point value, the stub might just return a random value instead.

On the other hand, if some not-ready-to-go component C is going to make extensive use of A, then it might be worthwhile to rapidly develop a test driver that simulates the behavior of C in connection with its usage of A. The idea is that if A faithfully services the driver, then we are more confident that it will later faithfully service C as well. Of course it is possible to build a fake component that functions both as a stud in some ways, and a driver in other ways. Bridge patterns can simplify the substituion of stubs and drivers for actual components.

The following is a list containing some types of tests that might be performed as part of an overall testing strategy (to be discussed later). Note that the terminology is not mutually exclusive, meaning for example that a certain unit test might be a blackbox (or whitebox) test.

A blackbox test tests a component in a functional way, in that it is only concerned with the behavior of the component, and makes no use of any specific knowledge of the internal design of the component.
A whitebox test is the opposite. An example is a state-based test where the objective is to drive the component into all possible (at some not too low level of detail) state.
A unit test naturally is just the testing of a unit, independent of other components.
An integration test is concerned with testing the interactions of a number of already tested components.
A double test (etcetera) is an integration test involving two (etcetera) already tested components.
A system test tests the system as a whole (naturally).
A structure test is a final integration test that focuses on testing the overall system architecture and the integration of the various subsystems.
A functional test tests some functional requirement(s) of the system.
A performance test tests some non-functional requirement(s) of the system.
A stress test is a performance test that studies the behavior of the system under extreme conditions involving high numbers of users (high traffic)
A volume test is similar, but is concerned with processing large amounts of data. (Although, I would personally still call this a stress test.)
A security test deliberately tries to violate the secutity features built into the system (naturally).
A timing test is a performance test that deliberately tries to prove that a non-functional requirement concerning timing is violated.
A recovery test attempts to prove that the system is not sufficiently robust in recovering from exceptions.
A pilot test or a field test is a trial by a number of end users of the system, generally done before any permanent installation or acceptance of the system, and intended for feedback to guide further refinements.
An alpha test is a pilot test, generally conducted in the development environment, to ferret out the more common faults still left in the system code.
A beta test is similar, but is generally done after an alpha test to discover and repair more subtle faults, and tends to be conducted outside the development environment by a larger group of end users.
An acceptance test is a test conducted by the customer, against some agreed upon criteria, that will decide whether the customer is ready to accept the system.