Overview of Software Testing

Software Testing is the process of executing a program or system with the intent of finding errors. Or, it involves any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. Software is not unlike other physical processes where inputs are received and outputs are produced. Where software differs is in the manner in which it fails. Most physical systems fail in a fixed (and reasonably small) set of ways. By contrast, software can fail in many bizarre ways. Detecting all of the different failure modes for software is generally infeasible.
Regardless of the limitations, testing is an integral part in software development. It is broadly deployed in every phase in the software development cycle. Typically, more than 50% percent of the development time is spent in testing. Testing is usually performed for the following purposes.

To improve quality
The minimum requirement of quality, means performing as required under specified circumstances.

For Verification & Validation (V&V)
• Verification: Confirms that products work properly reflect the requirements specified for them. In other words, verification ensures that "you built it right”
• Validation: Confirms that the product, as provided, will fulfill its intended use. In other words, validation ensures that "you built the right thing."
Verification Techniques
• Dynamic testing - Testing that involves the execution of a system or component. Basically, a number of test cases are chosen where each test case consists of test data. These input test cases are used to determine output test results. Dynamic testing can be further divided into three categories - functional testing, structural testing, and random testing.
• Functional testing - Testing that involves identifying and testing all the functions of the system as defined within the requirements. This form of testing is an example of black-box testing since it involves no knowledge of the implementation of the system.
• Structural testing - Testing that has full knowledge of the implementation of the system and is an example of white-box testing. It uses the information from the internal structure of a system to devise tests to check the operation of individual components. Functional and structural testing both chooses test cases that investigate a particular characteristic of the system.
• Random testing - Testing that freely chooses test cases among the set of all possible test cases. The use of randomly determined inputs can detect faults that go undetected by other systematic testing techniques. Exhaustive testing, where the input test cases consists of every possible set of input values, is a form of random testing. Although exhaustive testing performed at every stage in the life cycle results in a complete verification of the system, it is realistically impossible to accomplish. [Andriole86]
• Static testing - Testing that does not involve the operation of the system or component. Some of these techniques are performed manually while others are automated. Static testing can be further divided into 2 categories - techniques that analyze consistency and techniques that measure some program property.
• Consistency techniques - Techniques that are used to insure program properties such as correct syntax, correct parameter matching between procedures, correct typing, and correct requirements and specifications translation.
• Measurement techniques - Techniques that measure properties such as error proneness, understandability, and well-structuredness.
Validation Techniques
There are also numerous validation techniques, including formal methods, fault injection, and dependability analysis. Validation usually takes place at the end of the development cycle, and looks at the complete system as opposed to verification, which focuses on smaller sub-systems. Formal methods - Formal methods is not only a verification technique but also a validation technique. A formal method means the use of mathematical and logical techniques to express, investigate, and analyze the specification, design, documentation, and behavior of both hardware and software.
Fault injection - Fault injection is the intentional activation of faults by either hardware or software means to observe the system operation under fault conditions.
Hardware fault injection - Can also be called physical fault injection because we are actually injecting faults into the physical hardware.
Software fault injection - Errors are injected into the memory of the computer by software techniques. Software fault injection is basically a simulation of hardware fault injection.
Dependability analysis - Dependability analysis involves identifying hazards and then proposing methods that reduces the risk of the hazard occurring.
Hazard analysis - Involves using guidelines to identify hazards, their root causes, and possible countermeasures.
Risk analysis - Takes hazard analysis further by identifying the possible consequences of each hazard and their probability of occurring
For reliability estimation
Software reliability has important relations with many aspects of software, including the structure, and the amount of testing it has been subjected to. Based on an operational profile (an estimate of the relative frequency of use of various inputs to the program), testing can serve as a statistical sampling method to gain failure data for reliability estimation.