Automation helps to lower costs and reduce release time. However, one has to be very judicious while deciding how much to “spend” on automation while trying to save on time. The cost on automation is decided on two main factors:…
Automation helps to lower costs and reduce release time. However, one has to be very judicious while deciding how much to “spend” on automation while trying to save on time. The cost on automation is decided on two main factors:
- When to Automate and
- How much to Automate
When to Automate:
Let us first understand “when” we should decide or start to automate. “When” could be interpreted as; (i) when in the development cycle should we automate and
(ii) In which kind of projects we should go for automation.
The former could be answered simply as “As Early as Possible”. Once the Requirements are frozen, mockups are done, Automation can start.
For the later question, below I try to list down some pointers on the nature of projects where automation could provide substantial gains:
- Working in Iterations: When the product is taking form in iterations; or you are following Agile or Scrum methodologies, it is wisest to automate regression tests, so that at the end of next iteration you only have to run automation to confirm that the new code has not broken anything. It is best if these regression tests are integrated with your CI to enable Continuous Delivery.
- Support on various platforms and/or configurations: If your product supports multiple platforms, operating systems, or you want to test that the product works with the latest release of dependent products (say database support; libraries), then this also becomes a good candidate for automation. For instance you want to certify your portal works on all the major browsers, then without Automation, it is a herculean effort while with Automation, if done correctly, it is as easy as clicking a button.
- Providing multilingual support: If your product is going to be released in several languages, Instead of re-running the tests manually in all languages, automate the test suite and only change the input file as per language. Of course there is much more to this and it is a topic in itself.
How much to Automate:
Reaching 100% Automation is the state of ‘Nirvana’. But do is it really required? I think not because the sheer effort and time it might take far out weigh the gains. So one must be judicious in identifying what should be automated. Some of the key points one must keep in mind are:
- Tests that take a long time to do manually are a good candidate for automation. This gives the biggest bang for the buck.
- Tests that are fairly complicated (require complex calculations or business rules) also should be automated to remove human inefficiencies.
- Common pieces of code that are used by other parts of the application should be tested frequently and become a candidate for automation. They should be part of your Regression Tests.
- Then, identify key functionalities that are imperative for User Experience and a rogue defect could potentially result in a lost customer.
- Mundane UI testing that validates field types, max length, allowed characters etc., is a low hanging fruit but often missed. This can be easily automated and must be automated.
Automation is like a third eye to testing, but without any artificial intelligence. If you would test the same thing manually, you would think and look around for even what is not written in the test case. This gap will always remain between automation and manual testing. However Automation helps you in increased coverage which is difficult and time/resource intensive if done manually.
Good luck with your Test Automation endeavors!