As Marc Andreessen famously predicted six years ago, “Software is eating the world”. However, only a handful of user-friendly and reliable software can claim this prize. The rest just goes through the cycle of frustrating managers, confusing users and eventually being forgotten by history. Fifteen years ago it was barely enough simply to “log in and check that everything worked”. But the quality standards and especially end-user expectations have been evolving a lot since then.
Today, the truth is that you cannot guarantee software quality if you don’t use automatic unit testing.
The best way to use it properly is to follow a methodology called Test-Driven Development (TDD). This methodology doesn’t only dictate that you should write unit tests that cover most of the code you produce. It points out that you should write the tests first and the code later. This may seem counter-intuitive and the honest truth is that it requires a mental change that can be pretty costly on the first tries. But as a colleague and mentor once told me, “if you’re not wasting money on testing, then you’re probably wasting it somewhere else.”
SHIFTING FROM TECHNOLOGY TO OUTCOMES
Being good at what you do requires hard skills. Being excellent requires the right mindset. So what mindset do you need if you want to make the most of the Test-Driven Development way?
IT ALL STARTS WITH STRAWBERRIES
As an introduction, let’s explain the Test-Driven Development methodology with the help of a real life example. Imagine that you are the proud owner and CEO of Company A, a fictional hot newcomer in the food processing industry. Your analysts have determined that building an efficient strawberry-sorting mechanism is critical for the success of your next line of desserts. There are two different ways of approaching the problem.
1. The old-fashioned way
Your competitor is the renowned Company B: it is very likely to choose to put engineering first. Their lead architects took a few weeks to agree on a conveyor belts technology, a specific brand of cameras to measure and compare sizes and also on the kind of mechanical arms you would need to shuffle strawberries around. Their teams gave estimates based on what they knew of these various techniques. Then Company B has budgeted the whole project according to their input.
2. The modern way
As an agile company, you have decided to take advantage of this new test-driven concept that everyone is talking about. But in order to do this, you need to operate a shift that can be pretty difficult for an organization yet so rewarding in the long run: shifting from technology to outcomes. On the first weeks of a new project, you need to detox from the very addictive question of “how it can be done”. You should instead stay focused on “what exactly needs to be done — in great details”.
BUILDING A MACHINE TO TEST MACHINES
Your first mission is to design a test-suite that precisely models all the possible outcomes of a strawberry-sorting machine. But rather than starting to hack away, your developers will sit down and translate the business need of “building a machine that sorts strawberries by size” into a series of assertions that they feed into a testing machine:
• When inputting an empty tray, the machine should output the same empty tray.
• When inputting a single strawberry, the machine should output the same strawberry.
• When inputting a tray containing an apple, a football, and a Russian MIG jet fighter, the machine should stop with an error message stating that these are not strawberries
• Finally, when inputting a tray of strawberries, the machine should output the same strawberries, sorted by size
This may seem redundant to start with creating a machine that tests the machine. Yet, this first deliverable has the advantage of exposing the technical assumptions made by your developers, in a way that is perfectly readable (and adjustable) by your business team. From day one, your teams are already communicating in a common language. Also, you have a system able to determine that what is developed is on par with what is expected.
RED, GREEN, REFACTOR
Now that you have defined the expected outcomes independently of the technology used, you can start implementing the Test-Driven Development mantra: “Red, Green, Refactor”.
First, you put a tray on an unmoving conveyor belt and run your test-suite. All lights should go red: the machine doesn’t output anything.
Then, you start the conveyor belt. The first two tests are green, but the last two remaining are red. Your machine shuffles stuff around but doesn’t actually sort them.
Then you could add a person that stops the machine if the input isn’t entirely composed of strawberries. Three down, one to go.
Finally, you instruct the operator to stop the belt, sort the strawberries by hand, and start the conveyor again. All lights turn to green and you have the first iteration of your machine.
Now, is it scalable? No, it’s not (yet). But it took you only two hours and it will guarantee you a faster go-to-market, outrunning your competitor by months. Congratulations!
Your next step is now to start thinking about refactoring, which means, in this case, introducing computer vision and mechanical implements to automate each step of the process.
If you want to go further on automation, feel free to get in touch with our Data Scientists on comet.
A METHODOLOGY AGILE BY NATURE
You may have noticed that the simple practice of designing the tests first already changes your organization in a subtle way. Let’s dive into the natural side effects of this methodology.
All of this sounds ridiculous when you are talking about such a simple process. But bear in mind that this method scales very well to software components with dozens (if not hundreds) of complex requirements. By focusing on a rigorous analysis of your needs first, then on an iterative, somewhat naive implementation, you have completed a whole cycle from specification to implementation. And most importantly, you now have a working machine instead of a pile of pending projects. Plus, your business team has tried the machine out with different sets of input and are satisfied with the output (although the operator is quite noisy and complains a lot).
In short, your company is already more agile!
The best way to visualize the impact of Test-Driven Development is to zoom out to the organizational level:
1. With the old-fashioned approach
Company B has a team of conveyor belt specialists, one of computer vision gurus, etc… piloted by a team of business analysts that can only estimate the tech/business alignment when a big chunk of the project is delivered.
2. With our modern approach
You, a.k.a Company A, have multidisciplinary teams (business AND technical). Each team is dedicated to a particular set of outcomes: one is focused on “distinguishing strawberries from other stuff”, another one on “calculating strawberry size” and another one on “physically sorting the strawberries”.
The first effect is that your delivery cycles are considerably shorter. A couple of weeks in, you are already learning the ropes of strawberry-sorting in a real world scenario. Moreover, back and forth between development, quality assurance (QA) and business have already started. Even when a requirement changes, only the relevant outcome is impacted, not the rest of the teams as they are not blocked in their work.
MORE RELIABLE PLANNING
Because your big project has now become a series of small projects, your budgets and timelines are also much closer to reality. We all know that it is easier to plan and estimate a 2-week iteration than a 3-month project. Company B has a highly speculative release date set a long way in the future, but your release date is every other Tuesday. A bad estimate only costs you a 2-week sprint and doesn’t threaten the whole delivery.
Similarly, the hunt for bugs is somewhat simpler in a Test-Driven Development environment, especially if you couple this methodology with Continuous Integration. Every new code commit leads to a build, which leads to a cycle of testing. When a bug is introduced, you can often pinpoint the exact line of code that introduced it — and remedy it without breaking your planning.
Now, compare this to the situation of Company B. Each day invested in the project entrenches them further in the choices and assumptions they made on day one. A day when they had (figuratively) never seen a strawberry in their life. They are gambling the success of their project on the technical choices they made back then, and should one of them proves to be unfeasible, or unrealistically expensive, the whole thing could come crashing at their feet when it is too late to pivot. Even worse, a bug or design error from the early days of the project often brings the project to a halt, potentially costing their teams weeks of extremely stressful rewriting.
BOLDER WHILE STRESS-FREE
Another positive effect is that each mini project now has well-defined and easily measurable outcomes. You can take risks on component B and completely refactor it with a new technology, without fearing potential side-effects. A testing suite can be used as a rigorous interface contract, guaranteeing that component B will accept the inputs from component A and deliver an output compatible with component C. As long as a new version fulfills this contract, then compatibility should be preserved, even if the changes under the hood are deep and world-changing.
This, in turn, leads to a well-known side-effect of Test-Driven Development: the overall stress levels of your technical teams are greatly reduced. Without automated testing, developers can often get jittery when the time comes to refactor a whole chunk of code or experiment with new technology. Manual testing is notoriously unreliable as it requires a combination of business knowledge and technical know-how that is rarely achievable by one human. When the deadlines get tight, the effect is two-sided. Your great engineers become nervous and risk-averse because they cannot guarantee the quality of their deliveries. But the less rigorous ones will find loopholes in the process, “testing only what works” and kicking critical bugs and bad design down the road for someone else to take care of.
Test-Driven Development takes care of these aspects by giving the latter ones access and visibility over their own shortcomings. The methodology also guarantees the most audacious ones that their bold moves haven’t broken the machine. It also provides everyone with the certainty that what once worked has not been broken, removing the need to re-test long forgotten business needs after shipping every new functionality.
TEST-DRIVEN DEVELOPMENT WILL BENEFIT YOUR NON-TECHNICAL COLLEAGUES TOO
What Test-Driven Development brings technical teams in terms of visibility and peace of mind is also generously redistributed to the managing layer. The output from their engineers is less opaque and reporting is straightforward. The latter provides valuable insight into the creative process since testing focuses precisely on measurable metrics.
It should also be noted that, when comparing conventional methodologies with test-first strategies, a 2005 Canadian study found that not only did the Test-Driven Development teams wrote more unit tests, they were also more productive overall.
To sum it up, you now have an unbroken line of communication from business to tech, shorter iteration cycles, fluid and realistic planning, and happy productive developers. Your strawberry-sorting machine is sure to be a hit and to leave Company B far behind. And so will be your next projects. While you progress with Test-Driven Development and start “getting it right”, it will help you ship better products consistently and minimize bad surprises.
And this fantastic methodology, ladies and gentlemen, is how to make sure that it is YOUR software that eats the world. ?