In practice, functional test automation can deliver a number of benefits to an organisation that is prepared to invest. The time to test is decreased, the quality and accuracy of test execution increases and when the functional test automation is frequently executed, costs can decrease as well compared to manual functional testing.
Functional Test Automation can sometimes fail to deliver and causes frustration amongst project management who have invested in a long term outcome. Often the underlying problem is the automated functional test analysts spending more time maintaining their test automation than it would take to execute manually.
Take a test pack with 100 regressions testcases in it. Manual execution at the rate of 16 per day would take 6 days. When these tests are automated, if 50% of these fail because the application has been updated, then 50 scripts corrected at the rate of 4 per day would take 12.5 days to resolve.
While the problem may appear to reside with the functional automated test engineers who have not created robust test automation, generally the problem lies with management who have failed to invest correctly to achieve the desired outcome.
What does your organisation want to achieve from Functional Test Automation?
There can be many reasons to invest in functional test automation, the common ones are:
- Improve testing efficiency
- Cost Savings
- Improve Quality
- Decrease Time to Market
- Risk Reduction
What Automation Do You Get?
| Business or Test Analyst Testing | Technical Level Testing | Developer Level Testing | |
|---|---|---|---|
| Technical Resource | Largely unskilled in Test Automation | Technical Tester / Automation Engineer | Developer Level Technician |
| Dedicated Resources | No | Yes | Yes |
| Resource Costs | Limited investment in time and resources | Significant investment in time and resources | Very large investment in time and resources |
| Internal / External Resources | Customer self-sufficient with internal resources | Blend between investment in a partner and self sufficiency | Investment in a partner |
| Test Automation Tool | Visual Scripting | Uses technical automation tool | Uses technical automation tool integrated with IDE (Visual Studio, Eclipse, IntelliJ etc) |
| Test Tool Costs | Medium | High | Low |
| Outcome | One time use scripts, good for creating test data or one time tests. | Regression test up to date with current sprint-3 | Automation Test Pack with many scripts, Up to date for Current Sprint / Sprint -1 |
Functional test Automation can often be deemed to fail when investment made does not match the outcome desired, for instance, trying to achieve Technical testing outcomes with Business or Test Analysts who are more used to manual testing.
By Derek Pool