Test cost estimation and nearshoring – improve test quality and lower costs

As the complexity of new software increases, testing is likewise becoming more important. At the same time, companies are experiencing time and cost pressures. Test cost estimation and nearshoring are two very different strategies which can be used to overcome these challenges. Both reduce costs while simultaneously raising quality.

Relationship between test and error costs

Fig. 1: Relationship between test and error costs

Awareness of the importance of testing has grown in recent years. This is due to a simple economic fact: errors that are discovered during testing are significantly more cost-effective to correct than errors that are not discovered until after productive operation has begun. Of course, faulty new products can also cause serious reputation damage. On the other hand, it is also possible to test too much such that the test costs outweigh the error costs. The more testing that is performed, the greater the cost for each additional error that is discovered, which also translates into higher test costs.

Well-founded test cost estimation - making good economic sense

Well-founded test cost estimation can help to avoid both problems (too little and/or too much testing). The goal is to set the test intensity (test cost) in an economically sensible manner, thereby minimising the sum of the test costs and error costs. There are many different methods available for estimation purposes. They range from very costly approaches such as test point analysis (TPA) and function point analysis (FPA) to very elementary calculations. For example, the test cost is commonly estimated to equal 40% of the development cost. The more complex methods entail relatively high expenditure and are still subjective in the end as they also use subjective conversion factors. A more suitable approach is based on an existing body of experience and provides a realistic cost estimate using base quantities that are easy to determine in conjunction with simple, comprehensible formulas.

Realistic test cost estimation - thanks to task differentiation

In order to obtain a realistic estimate, it is necessary to divide the testing into five areas: management, preparation, design, execution and conclusion. For each of these activities, the cost is first estimated individually. Empirical values are used here; they are multiplied by indicators reflecting the size of the project. For example, there is the following formula for test design: (Number of test cases per requirement x Number of requirements) x Design time for a single test case. In this instance, the number of requirements is the indicator for the size of the project. The two other variables are based on experience gathered in past projects. If you are just starting out with this test cost estimation method, you can have testers with many years of practical experience estimate the empirical values. The real values can then be assessed for each new project. This makes the estimation process more and more accurate as additional projects are completed. In the test design example, the number of created test cases, the total time for development and assessment of the test cases, and the number of requirements underlying the test cases must be systematically determined for the projects. Once these values are collected, a realistic estimate of test design expenditure becomes possible after only a few projects, using the formula given above.

Cost-effective and efficient regression tests - with the right nearshoring partner

A large financial institution would like to subcontract regression tests and automated tests to a nearshoring partner. The company has a well-established test process. All of the participants have a consistent understanding of the roles and process steps. However, the process is not very formalised. In the starting phase, an experienced senior tester is acquainted with the large company’s business and test teams. The nearshoring partner handles organisation of the stay at the customer’s site. The large company ensures that the senior tester has a chance to talk to all relevant parties in order to be able to optimally gather and/or pass along the expertise. This process of gathering information includes integration of the senior tester into the existing test team and participation in the test execution. Thanks to the senior tester’s good integration and extensive experience, the process of gathering information on-site is successfully completed in only three weeks.

Effective quality assurance in nearshoring projects - with virtual classroom and periodic checks

Next, the nearshoring partner works with the senior tester to build the test team in Eastern Europe. At the same time, access to the test infrastructure is arranged. The testers use customer notebooks to remotely access the infrastructure at the customer’s office. Currently, up to three testers are available so that demand peaks can be managed as well. There is intensive communication with the teams (different development teams), the IT and development managers as well as the representatives of the large company’s business end. Virtual status meetings are held each week. Telephone calls are also made practically every day. For quality assurance purposes, quality checks are made every half-year. These checks are discussed with the customer and help to ensure sustainable service that meets the requirements. This example shows just how routine the subcontracting of test activities to nearshoring partners has now become and how little expense it generates. This expense delivers benefits in the form of lower costs – by about 50% – and better scalability of the test team. Moreover, higher quality is obtained since adequate resources are available for testing even if the deadline is very tight. The quality is also boosted because external test teams have clear starting criteria for their work. This leads to fewer misunderstandings between developers and testers, thereby reducing the risk of undetected errors. Like test cost estimation, nearshoring is a strategy with an attractive cost/benefit profile. Both reflect the multiple possibilities available today to meet seemingly contradictory requirements: improving the quality of the tests while simultaneously lowering the cost of testing.