Automated Testing Applied

During my time at Octobot, I’ve been part of different projects, and have documented use of automated testing. I was interested in seeing how much these practices could leverage our teams’ work. 

I want to showcase how automated testing has made our work more efficient and more diverse. Our client was a company that created an intelligent system to decrease energy consumption and operating costs of motors.

Before moving  to the application cases, let me explain briefly the tools and technologies we used to make these cases happen. The system’s requirements were written using Gherkin; then, we developed the automation software with Behave – a Python package that helped us with the natural language in which the requirements were written – we also had Gitlab CI/CD for Continuous Integration and Continuous Delivery, Docker to run tests on simulations of our active cases, Mypy for checking our code quality, and Radon for evaluating metrics.

Now, let’s get into the cases.

Motor Controllers Software

In this case, the main goal is optimization of the motors’ firmware developers, which allows prioritizing of other tasks central to the project. 

First step:measuring the number of engine requirements that our framework is capable of testing manually, as well as the number of requirements expected manually by our developers. We also measured the time for our testing framework to execute all the test cases versus the time it takes firmware developers to test the features of the engine.

When comparing the numbers, we found that applying automated testing software resulted in immense benefits for the team: one engineer, using our automated software, was able to test 450 cases in one day, which is the amount of tests the team of three engineers can run in 8 weeks. Using this framework allowed the client to save time and money, and increase the amount of functionalities tested. These saved resources can now be invested on developer work that adds more value to the platform

Hubs Testing Project

A Hub in this project is a computer that runs a software developed by the client, which gathers information from the different temperature and humidity sensors in a building. The Hub is always running and is constantly collecting data. For Gherkin, we wrote requirements to test in an automated way. There are different functions that can be performed through the Hub, such as finding devices that are connected to the network, or controlling the temperature of the equipment.

We were able to automate 175 requirements, achieving a 90% success rate in testing cases. This took approximately two hours of work for our automated framework – much less than it would take for an engineer to manually perform these tasks.

Cross-Wiring Tests

Our client is focused on energy optimization, so this is a key aspect for improvement. The cross-wiring test is to verify that the equipment won’t release hot air when set to cold, nor cold air when set to hot. This allows the company to verify that all units are running correctly. Manually, this requires a lot of time: each unit must be checked along with a second unit, since the most likely mistake is that they have been cross-wired with each other.

Running our automation framework, in this case, means a huge time saver because the amount of units in a building increases, and the time spent assessing each unit decreases. Before, a person would have to evaluate each unit and give directions to them manually whereas with the automated framework, all that work is done. 

We were able to automate 99% of the tests, and we believe it’s possible to improve and reach a 100% of automated tests. We achieved that by using Specification By Example, a method in which we wrote the requirements in language that anyone on the team could understand, and then connects to a Python program which interprets and executes orders automatically.

Final Conclusions

Automating tasks can help development teams increase productivity and decrease the likelihood of mistakes. It is a useful tool for many cases, as it saves time and money, which allows the team to relocate those resources and focus on tasks more relevant to the project’s goals. Be creative and go through your work’s repetitive tasks; there is probably a smarter way to tackle them!