Faster and Reliable Automated Tests

As functional testers, we prefer writing end-to-end tests, that test the complete workflow from user perspective. They not only take longer to run but are also flaky, prone to random failures. If one such tests fails in the middle, you need to run the entire test and often the entire suite, exponentially increasing your overall run time.

And this is a common problem. Enter Atomic Automated Tests. Atomicity is not only limited to the world of developers and Unit Tests. We can leverage the same principal for writing automated functional tests.

An atomic test is one that focuses on testing a single feature and which makes clear what it is that you are testing. If the test fails, then you know what needs to be fixed. Atomic tests form a single fundamental unit and should not do end-to-end automation. This means that your tests are highly focused on a single thing. So an atomic test has:

• only one or two assertions at most. Because we do not need many assertions to make sure that our state is correct.
• very few UI interactions, and they’re only on a maximum of two screens

Advantages of Atomic Automated Test

1. Atomic Tests Fail Fast and hence, you will get the feedback real quick.
2. Increase test stability by decreasing number of moving pieces or failure points.
3. Increased Test Coverage since one tests are independent and failure of one test does not block the others.
4. Easier to maintain since atomic tests serve one and only one purpose. They are smaller in size and consequently easier to maintain.
5. Faster Execution since tests are smaller, run independently and can be parallelized. In our experience, we have seen a suite consisting of 16 end-to-end tests taking about 20 minutes while a suite of 200 atomic tests taking less than 5 minutes when run in parallel. By the way, we have seen automated tests that take 90 minutes to execute.
6. Easy to integrate into CI/CD pipeline as execution is fast and robust.

Converting end-to-end tests to Atomic Tests

Now that we see the benefits, how do we convert our existing end-to-end tests into atomic ones? To be honest, there’s no magic wand, or even an easy answer for it. We suggest to take this decomposition process in stages.

Here is a simple scenario:

1. Open Amazon.com
2. Assert that the page opens
3. Search for an item
4. Assert that item is found
5. Add item to cart
6. Assert that item is added
7. Checkout
8. Assert that checkout is complete

Many automation engineers will assume that this scenario requires an end-to-end test because you need to complete Step 1 before Step 2. How can you get to the checkout process without having an item in the cart.

One key best practice for test automation is to have application under test in a known state before performing any UI interaction. That means populating data into the application before doing UI stuff.

Populating and manipulating test data for UI Automation

Injecting or populating data allows you to isolate and test individual steps. There are several mechanisms to inject test data, such as :

• Using REST APIs to inject populate required data
• Insert directly into the db
• Using Javascript to bring app to a known state
• Using cookies

Always clear any data thus created so that this data does not impact other tests.

Using an API or inserting data is extremely fast and can be performed in milliseconds as opposed to doing it in the UI which may take multiple seconds. Moreover, calling APIs makes your UI automation tests more robust and less flaky.

Challenges

You may have a situation where you can not inject test data or your application is not really automatable. In this scenario you should work with your developers to make the application testable. Quality of the application is entire team’s job, not just the testers.

Remember, not every application can be automated. If you think your application is not automation friendly, don’t do it.

How does Bqurious Test Automation Platform help?

BQ Test Automation Platform allows you to quickly create atomic automated tests. Moreover, injecting data into your automated UI tests is extremely easy. You can integrate your UI tests seamlessly with REST APIs, make database calls or set cookies without writing a single line of code. This doc explains how to integrate REST APIs with your automated UI tests.

The following video explains how to integrate REST APIs with your automated UI tests.

Also See Test Automation Best Practices , Atomic Tests