Functional Test Automation: Why companies often feel let down by the outcome of their investment

Posted on

October 4, 2023

Posted in

In practice, functional test automation can deliver a number of benefits to an organisation that is prepared to invest. The time to test is decreased, the quality and accuracy of test execution increases and when the functional test automation is frequently executed, costs can decrease as well compared to manual functional testing.

Functional Test Automation can sometimes fail to deliver and causes frustration amongst project management who have invested in a long term outcome. Often the underlying problem is the automated functional test analysts spending more time maintaining their test automation than it would take to execute manually.

Take a test pack with 100 regressions testcases in it. Manual execution at the rate of 16 per day would take 6 days. When these tests are automated, if 50% of these fail because the application has been updated, then 50 scripts corrected at the rate of 4 per day would take 12.5 days to resolve.

While the problem may appear to reside with the functional automated test engineers who have not created robust test automation, generally the problem lies with management who have failed to invest correctly to achieve the desired outcome.

What does your organisation want to achieve from Functional Test Automation?

There can be many reasons to invest in functional test automation, the common ones are:

  • Improve testing efficiency
  • Cost Savings
  • Improve Quality
  • Decrease Time to Market
  • Risk Reduction

What Automation Do You Get?

Business or Test Analyst TestingTechnical Level TestingDeveloper Level Testing
Technical ResourceLargely unskilled in Test AutomationTechnical Tester / Automation Engineer
Developer Level Technician
Dedicated ResourcesNoYesYes
Resource CostsLimited investment in time and resources
Significant investment in time and resources
Very large investment in time and resources
Internal / External ResourcesCustomer self-sufficient with internal resourcesBlend between investment in a partner and self sufficiency
Investment in a partner
Test Automation ToolVisual ScriptingUses technical automation tool
Uses technical automation tool integrated with IDE (Visual Studio, Eclipse, IntelliJ etc)
Test Tool CostsMediumHighLow
OutcomeOne time use scripts, good for creating test data or one time tests.Regression test up to date with current sprint-3Automation Test Pack with many scripts, Up to date for Current Sprint / Sprint -1

Functional test Automation can often be deemed to fail when investment made does not match the outcome desired, for instance, trying to achieve Technical testing outcomes with Business or Test Analysts who are more used to manual testing.

More posts by Derek Pool