We recently ran a short survey on what people want in a Software Test Automation Platform. While most people had similar requirements, there were quite a few very interesting needs. Some of the most common requested features are: It should…
We recently ran a short survey on what people want in a Software Test Automation Platform. While most people had similar requirements, there were quite a few very interesting needs. Some of the most common requested features are:
- It should support multiple platforms – Windows, OSX & Mobile. It was also referred to as “Portability to different platforms and browsers”
- Number of scripting languages it supports – Java, Javascript, Python, Ruby, C#
- Integration with CI tools – Jenkins, Hudson, Bamboo
- Easy Parameterization of scripts
- Reasonable Reporting Capabilities
Some very interesting and pretty important requirements :
- Rapid Development – (Many tools available take excessive time to develop unless you are fortunate to have a QA person that is also a developer)
- End result that is READABLE – Window synchronization built in. IMHO a test script should appear similar to the English written below when complete……..(Obviously framework needs to be written correctly)login (name, password)
Select(Weekday,”Monday”)
Save.ClickYou would be surprised what some of the automation code looks like that companies rely on. Then the person leaves who created the scripts and no one else can maintain them. - A reliable method of accessing objects that do not contain standard attributes — QTP analog mode would be nice if it was “reliable” but we never found it to be 100%….
- Since global automation is the key in the future, the tool should support team-based global automation frame work.
- I would also like some sort of linkage with the defect tracking system so that I know which scripts to re-run when a defect is fixed.
- Also linkage to Business Requirements and test specs so I can quote both (or have pre-filled?) in my defect.
Then there were some more from people with a developer’s perspective:
- First off, honesty in the Marketing/Sales materials. No more of the “Codeless/Scriptless” hype.
- IDE that gives me the capability to build my code quickly (like Intellisense / auto-complete) and to hook my data inputs to it easily. As part of this it needs to provide very strong debugging capabilties (Watch/List variables at run-time, call stack tracing, Stepping, In-line command execution, etc.).
- Capability to create Driver Scripts (group and execute test case scripts) quickly. Have a Wizard that can help with this, or a template or functionality in tool to create this.
- Reporting functionality so that I can write my logs easily but also be able to pull from them stats on progress and centralize in one report.
One point I liked the most:
- Give me the ability to do my work. As part of this give me Help files that have both descriptive chapters with good solid full examples, and an API/Function reference that describes usage well and good example code of its use. The easier you make it for me to find my own answers/solutions the happier I’ll be. I’m not expecting an Automagical Silver Bullet, but a tool that is flexible and powerfull enough to let me do my job effectively and efficiently.
There’s a clear distinction between the needs of QA people with and without development background. QA with dev background look for tools that can help them write Automation Test Scripts in multiple languages, debug them easily and a framework/library that is extensible. On the other hand, QA with no programming background are keen on ease of use, readability, platform portability and easy reporting.
Tools/platforms currently available also address these two diverse set of users. On one side we have Opensource frameworks like hugely popular Selenium, that is a developer’s tool and on the other hand there is this Commercial offering from behemoth HP, called QTP, that tries to serve both categories.
Wait for more in my subsequent posts.