Continuous Development is GREAT! But how do we test it?
The discussion on the advantages and disadvantages of Agile development, is something than can go on forever. When I was introduced to the Agile-method for the first time, I was told that the only way to perform Agile development was to follow the Scrum method to 100%.
I followed this advice and started to see that this was the way to go if you were aiming for a great failure with lots of delayed delivery as a final result.
With this said, I still believe that Agile development is the way to go. But you have to tweak it, so it fit into your organization.
Agile development, Scrum, in combination with fixed delivery dates is an equation that most data-driven companies face daily.
Almost every development company is structured with a product management team that plans and presents a roadmap with long-term plans. This structure and long-time planning aren’t going to disappear overnight, so instead of complaining on that the agile development can’t have fixe dates of delivery. We need to adapt the organization to handle these delivery dates.
One major problem with this combination is that the development is often mature for the Agile quick-changing method, however, test and delivery is not at the same pace.
In many organizations it takes several days, sometimes weeks, of code-freeze and testing to actual deliver an update/release to production.
When the test and delivery is complete, the development already has continued on to the next update/release. This contributes to the fact that it is difficult to use the test-data to see if an update/release is successful. As the development for the next release already begun or, at worst, is ready for a new code-freeze period when the user data comes in.
All organizations strive to have a continuous delivery and to achieve this you have to include testing and release management.
There is no blueprint-solution on how such work-flow should look like, but there are methods that are more and less proven to reach an efficient continuous delivery work-flow.
A/B Testing
A/B testing is used to ensure that you are building a product that will add value to the user. This method needs to be done continuously to ensure that your development is in line with what the user is looking for and will bring more value to your product.
The method is completely data driven, which means that it is easy to see a defined result.
The method is based on two different variations (hypotheses) of the same product and these are given to two different user groups. The result is then measured on successful user interaction.
When the tests are done you will receive a result on what which hypotheses that has the highest score and also results for where the user gets stuck and did not finish the UX workflow.
A / B testing is usually divided into 5 parts,
- Research (Data from Analytical Tools such as Google Analytics, user tests or data from previous tests)
- Hypothesis (How do we think the user will act)
- Variation (Creating different variations of the product)
- Testing
- Analysis of the test results
Important to consider when going with A/B testing.
Accuracy in determining the conditions and develop the hypothesis for what we want to get from the A/ B test. The greater amount of data collected, the higher the credibility of the result.
Continuous Testing
Continuous testing is something that all organizations that strive for an effective development environment should consider. Continuous Testing is all about minimizing the manual tests, and instead continuous test the code with automated tests. This is done by using test scripts that runs through the code and test different parts of the product.
Continuoustesting is very effective method to find bugs at an early state and in the best of world’s already when they are being introduced.
The benefits are many with Continuous testing, but there are also some disadvantages and things that you need to take into account when introducing it.
It takes time to keep a test environment healthy and up and running. At the same time, it takes time to implement new automated testcases to cover new functionality.
To have the full benefits of the automated tests, it is important to keep automated tests up to around 80% of all tests. This to ensure that the time required for maintenance and new implementation of the automatic tests will be smaller than the time spent on manual test.
What should I choose?
Optimizing and streamlining a development team with new test cases is not about choose one of the methods mentioned in the previous paragraph.
To achieve an efficient development with high speed and continuous releases and also get user feedback, it is necessary to find a balance between the different test models in order to achieve as high efficiency as possible.
There is a big step to get to a fully functionally and efficiency continuous development workflow. But never less, it is important to take the first step and start thinking and set up a goal architecture, describing how the team’s efficiency should look, tomorrow, next week, next quarter …
There are many different variants of data-driven tests (A/B tests) and automatic tests (continuous testing). If you use these two in combination, you cover a big amount of test cases and also this will be a first step towards a more efficient development environment.
Never less, it is necessary to keep in mind that it will take time to reach the goal architecture that you have set up. Start by adding one automated test in each sprint, and you will soon be there.
Daniel Eriksson is a Project Manager and Video Specialist Consultant at Eyevinn Technology. Eyevinn Technology is the leading independent consultant firm specializing in video technology and media distribution.
