Best Practices in Automated Testing

Best Practices in Automated Testing

The Case for Automated Testing

Today, rigorous application testing is a critical part of virtually all software development projects. As more organizations develop mission – critical systems to support their business activities, the need is greatly increased for testing methods that support business objectives. It is necessary to ensure that these systems are reliable, built according to specification and have the ability to support business processes. Many internal and external factors are forcing organizations to ensure a high level of software quality and reliability.

Why Automate the Testing Process?

In the past, most software tests were performed using manual methods. This required a large staff of test personnel to perform expensive and time-consuming manual test procedures. Owing to the size and complexity of today’s advanced software applications, manual testing is no longer a viable option for most testing situations.

Using Testing Effectively: By definition, testing is a repetitive activity. The methods that are employed to carry out testing (manual or automated) remain repetitious throughout the development life cycle. Automation of testing processes allows machines to complete the tedious, repetitive work while human personnel perform other tasks. Automation eliminates the required “think time” or “read time” necessary for the manual interpretation of when or where to click the mouse. An automated test executes the next operation in the test hierarchy at machine speed, allowing test to be completed many times faster than the fastest individual. Automated test also perform load/stress testing very effectively.

Reducing Testing Costs

The cost of performing manual testing is prohibitive when compared to automated methods. The reason is that computers can execute instructions many times faster and with fewer errors than individuals. Many automated testing tools can replicate the activity of a large number of users (and their associated transactions) using a single computer. Therefore, load/stress testing using automated methods requires only a fraction of the computer hardware that would be necessary to complete a manual test.

Replicating testing across different platforms

Automation allows the testing organization to perform consistent and repeatable test. When applications need to be deployed across different hardware or software platforms, standard or benchmark tests can be created and repeated on target platforms to ensure that new platforms operate consistently.

Greater Application Coverage

The productivity gains delivered by automated testing allow and encourage organization to test more often and more completely. Greater application test coverage also reduces the risk if exposing users to malfunctioning or non-compliant software.

Results Reporting

Full-featured automated testing systems also produce convenient test reporting and analysis. These reports provide a standardized measure of test status and results, thus allowing more accurate interpretation of testing outcomes. Manual methods require the user to self-document test procedures and test results

Understanding the Testing Process

The introduction of automated testing into the business environment involves far more than buying and installing an automated testing tool.

Typical Testing Steps: Most software testing projects can be divided into general steps

Test Planning: This step determines like ‘which’ and ‘when’.
Test Design: This step determines how the tests should be built the level of quality.
Test Environment Preparation: Technical environment is established during this step.
Test Construction: At this step, test scripts are generated and test cases are developed.
Test Execution: This step is where the test scripts are executed according to the test plans.
Test evaluation: After the test is executed, the test results are compared to the expected results and evaluations can be made about the quality of an application.

Identifying Tests Requiring Automation

Most, but not all, types of tests can be automated. Certain types of tests like user comprehension tests test that run only once and tests that require constant human intervention are usually not worth the investment incurred to automate. The following are examples of criteria that can be used to identify tests that are prime candidates for automation.

High path frequency – Automated testing can be used to verify the performance of application paths that are used with a high degree of frequency when the software is running in full production. Examples include: creating customer records.

Critical Business Processes – Mission-critical processes are prime candidates for automated testing. Examples include: financial month-end closings, production planning, sales order entry and other core activities. Any application with a high –degree of risk associated with a failure is a good candidate for test automation.

Repetitive Testing – If a testing procedure can be reused many times, it is also a prime candidate for automation

Applications with a Long Life Span – If an application is planned to be in production for a long period of time, the greater the benefits are from automation.

Task Automation and Test Set-Up

In performing software testing, there are many tasks that need to be performed before or after the actual test. For example, if a test needs to be executed to create sales orders against current inventory, goods need to be in inventory. The tasks associated with placing items in inventory can be automated so that the test can run repeatedly. Additionally, highly repetitive tasks not associated with testing can be automated utilizing the same approach.

Who Should Be Testing?

There is no clear consensus in the testing community about which group within an organization should be responsible for performing the testing function. It depends on the situation prevailing in the organization.