Netflix has more than 183 million subscribers around the world, on more than half a billion devices spanning over 1,900 different types. So how does it deliver the same rich experience on both high-end devices like Apple TV and PS4 as well as memory and processor-constrained smartphones?
The answer is with rigorous quality assurance testing, which puts a huge emphasis on automation. Since introducing its own test automation framework, which is responsible for running over 20,000 test cases on any given day, Netflix has seen execution times drop from 5 minutes to 56 seconds.
But Netflix isn’t alone when it comes to test automation. From global players including Facebook and Spotify to domestic leaders such as Australia Post, many organisations have embraced its extensive advantages.
The expectation from quality assurance (QA) is to ensure the systems Australia Post provides to its customers are functional, reliable and available…. As an organisation we recognise the need to increase test automation.
– Donna Shepherd, Head of Testing and Service Assurance, Australia Post.
With the evolution of software development lifecycles and the need to deliver quick, frequent solutions, alongside the time consuming and painstaking nature of traditional approaches, QA teams are increasingly exploring automation to not only streamline their own work, but also improve the end product.
What’s more, businesses can benefit from scalability, usability and extensibility alongside quicker release cycles and increased ROI.
The Benefits of Automation
Save time
What a QA can spend days doing, automation completes in a matter of minutes. Along with regression testing, this is also true of parallel device testing, as the same script can be run over and over again against multiple environments and devices.
Better quality
Assuming test scripts are written appropriately, automation offers better quality, consistency, efficiency and reliability because it performs testing without deviation or the possibility of human error.
Increase morale
Seeing as repetitive tasks can be completed automatically, QA teams are able to spend their time on more challenging, enjoyable and rewarding test activities.
Save money
Any money spent on automation tools will soon be recovered, as fewer resources are required for manual testing work. Less time spent on manual procedures ultimately ensures greater productivity too.
Better feedback
When creating new builds in Continuous Integration (CI) environments, the development team can get immediate feedback about their work. There’s no need to formally pass builds onto QA teams as the only method for issues to be detected.
Increase coverage
Lengthy tasks that are often avoided during manual testing can be run unattended on multiple devices, machines, or even in the cloud with different configurations, delving the depths of data tables, file contents and internal program states.
We believe intelligent automation will take cost benefits to even higher levels. We’re already seeing scenarios in which testing is being used to increase revenues.
– Prabakaran Karuppiah, Vice President Financial Services, Capgemini
The Value of Automation – Testing Vodafone Foundation’s DreamLab
In order to reduce manual testing requirements, identify potential issues quickly and gain more confidence in our work, the Vodafone Foundation DreamLab apps became a project where we could apply a vast range of automation.
At Transpire, we strongly believe in using testing tools best suited to the application’s technology stack. This has many benefits; sharing the test scripting tasks between developer and QA, encouraging team collaboration and clarity on complete test coverage, and efficient and regular test execution within a team member’s local environment.
Completely integrated with our Continuous Integration environment, we run various tests against backend APIs and the frontend iOS and Android apps – be it unit, integration, and user interface testing, providing instant feedback to the development team.
For DreamLab’s recent global rollout, we utilised test automation to great effect, achieving the following gains through comprehensive visual snapshot testing.
– 158 Days –
Manual Regression Testing Time Without Automation
– 20 Days –
Regression Testing Time With New Automation Strategy
– 84% –
Reduction in Effort per Regression Cycle
– 2.5x –
Increase in Device Coverage
– 87% –
Increase in Release Frequency
This is just one example of how automation can deliver tangible benefits. If you need help with your own quality assurance approach, don’t hesitate to contact the testing team at Transpire.
