Automating application testing helps a business in numerous ways. It brings about cost reduction, eliminates unnecessary efforts and reduces time to market. All the above contribute to increased profits and greater productivity.
However, before you emabark on this journey, please be aware that Test Automation comes with its own set of challenges.
Lets take a closer look at what NOT to do.
1. Focusing Only On UI Testing
The first, and a most common reason why test automation can fail is focusing only on UI (User Interface) testing. Although UI testing is a significant part of test automation, it most certainly is not the only part.
UI tests are generally slow and flaky if not written properly. Executing them takes time. In order to reduce this time and test faster, use headless browsers and parallelize tests. The following pyramid shows the best practices used when deciding on what type of tests to write and their ratio. Most of the functionality should be covered with Unit Tests that run blazingly fast. Then writing automated tests for testing your API interfaces. They are fast too. Further, your API interfaces can be mocked during development that enables the team to practice TDD/BDD.
Bqurious Platform allows you to easily create automated API and component tests with an intuitive interface.
2. Failure to test from the End-User Perspective
The next important mistake that many test automation engineers make is to fail to understand the user’s perspective. Most test automation is written to check whether a particular feature works. Rarely do automated tests check for exceptional situations and errors.
For instance, while a test can check whether users can log in, very few can check what happens when the user types in a long or wrong username. Does the server crash? Is the user shown an appropriate error message? So in essence, testing for “Negative Scenarios” is an important aspect that should also be covered in automated tests.
3. Inability To Differentiate Between A Test Error And An App Error
GUI Automation tests are flaky and fail often; this is not surprising. What is surprising is that sometimes, it becomes difficult to identify it the failure is due to a test error or a genuine application error. Tests that do not provide sufficient information to clearly differentiate between a test and app error, tend to be ignored both by developers and QA.
Bqurious Platform helps you clearly distinguish between an application error and test error by way of screenshots and easy to understand messages.
4. Inability To Cope-Up With Changing Requirements
In today’s agile world, application development is a continuous process, and testing is a part of it. This naturally means that automation testing is most effective when carried out during development. However, such a process has an inherent problem in the form of a test lag. Automation test teams are typically a few sprints behind the development team. This leads to their not being able to cope with the changing requirements that software in development always goes through.
This problem is minimized with matured teams that use BDD/TDD based development and diligently write Unit Tests and Integration Tests. Bqurious’ visual collaborative development paradigm allows you to create automated tests on your mockups. In addition, Bq decouples UI from tests, and BqGenie automatically detects changes to the UI, making your tests extremely easy to maintain and cope-up with changing requirements.
5. Incomplete Process/Guidelines For Test Creation
Development processes have evolved over the years and developers typically have well defined best practices and coding guidelines. However, this level of maturity has not reached Test Processes yet, leading to lack of standardization and generally inefficient code. Inappropriate skillset and experience of test engineers further exacerbates this problem. Therefore, a set of well-defined process and guidelines for test automation will lead to a better test automation engagement.
6. Lack Of Adequate Infrastructure
With automation, the biggest bang for the buck is when tests are run in parallel across all the supported platforms. For example, consider a non-trivial web application. It needs to support most, if not all, the current browsers. The image below shows a list of most popular browsers.
Then there is the Operating System. Each browser may behave differently depending on the underlying OS. Image below shows the distribution of most popular OSs.
To further complicate the situation, not everyone keeps their OS or browsers updated. As a product owner in a startup, you need to reach out to the largest audience and your research tells you that your target audience uses the following:
- IE, Edge, Firefox and Chrome on Windows 10 and Windows 8
- Safari, Chrome and Firefox on MacOS
To ensure the best experience for your target audience, you need to test your cutting-edge application on all the above 11 combinations (4×2 + 3= 11). But a startup more often than not, does not have the wherewithal to setup all the required infrastructure. Even a large enterprise finds it difficult to maintain and manage such infrastructure. Moreover, manually testing on all this infra becomes almost impossible.
There are new age companies like Browserstack and Saucelabs that provide this infrastructure as a Service. So to summarize, the automated tests that your team creates should be written in such a way to leverage these IaaS providers. Bqurious provides integration with these third party providers out of the box. It also helps you execute your tests on multiple inhouse mobile devices and browsers at the click of the button. Take a look at the video below for Bqurious’ integration with Browserstack.
7. Tight Deadlines
Deadlines are a reality of any modern business. Depending on the automation tools and technologies being used, deadlines need to be made flexible. Test automation is not a magical process that can yield results overnight. Yet, many managers expect the team to come up with flawless test cases in record time.
Often, the problem lies not with the automation team but the tools that are being used. Most test automation tools must be evaluated from multiple aspects before settling on the right one. In case you have the required talent pool, your company can build its own test automation tools in-house. It often becomes a Build vs Buy decision.
8. Siloed Test Automation
Finally, perhaps the most important reason for test automation failure is siloed automation. In a majority of cases, the test engineers write and run the tests separately from the development team. The development team has little idea how those tests are being written but has to rely on them. Such a siloed testing approach breeds unnecessary confusion.
Testing an app, is not an isolated process. It needs to be carried out in tandem with development. Ideally, test automation should be a part of the CI/CD Pipeline. Without effective interaction with the development and other project teams, test automation cannot succeed.
Conclusion
Test automation is not tough, but it’s not something to be taken carelessly either. Dated practices, inadequate tools, and a lack of effective communication can lead to test automation failing miserably. The success depends on the planning and execution of the tests to a large extent.
Keeping the above points in mind can help test engineers develop suitably automated tests for new as well as existing applications. A lack of awareness about the common mistakes can lead to the entire test automation process becoming redundant.