Have you ever wondered how to achieve a faster time to market (TTM) for your product teams? Do you have a straightforward task, a personal project, or a team project, that is taking too long?
Wouldn’t it feel amazing to get your product out the door and start gathering feedback so you can adapt?
Today we are going to talk about one culprit that prevents a fast time to market: Solving problems you don’t have. When you are working on problems you don't have, you are wasting time because "You aren't gonna need it!" (YAGNI)
This article is part of a series on the rules of scalability for product development. It focuses on reducing the equation by not overengineering the solution. The principles discussed originated from Abbott and Fisher’s book Scalability Rules. In this article we will discuss overengineering; what it is, why it is bad, and how to identify and prevent it.
What is Overengineering?
Overengineering is solving problems you don't have.
We will look at two sides of overengineering: Exceeding useful requirements, and spending too much effort to get a job done.
Exceeding useful requirements
Exceeding useful requirements is when we go beyond the bounds of when something can be used.
To illustrate the principle, consider a few examples.
Imagine designing a heating and air conditioning unit for a normal home. What would happen if we designed and built it to work in plus or minus 300 degree weather? How about a timecard system designed to handle 1 trillion unique people? What about a keyboard designed to handle 1 billion keypresses?
From an engineering perspective, these problems might be a joyful journey of discovery. Yet, from a product perspective, the outcome might not have value related to the required effort.
The extreme temperatures mentioned above are not common on Earth. Unless building in space, it is likely wasted time.
One trillion unique people, is more than 140 times greater than earth's population in 2020. Changing Earth's population that much is not a likely event.
One billion keypresses would last 500-1,000 years of normal use. I'm sure by then we'll have a better way to interface with computers.
“Wait a minute!” you might say. “There was a Kickstarter campaign that raised over $200,000 USD for a 1 billion keypress keyboard.”
Very true. The Keystone Keyboard claims to allow for 1 billion keypresses. This is a different case, allow me to explain why.
In this case, the Keystone Keyboard design uses magnetics and Hall effect sensors to measure the depth of a key press. In turn, this data augments traditional on/off data by including depth and speed of the keypress.
Developers targeting this keyboard can use the richer data to enhance the user experience. With richer data, music synthesizers can have a wider range of input or video games can have more precise control. Potential uses expand from there.
With magnetics and Hall effect sensors, the Keystone Keyboard achieves the design goal of more granular input. The keyboard also has a longer than normal life expectancy because of the use of magnetics compared to a mechanical keyboard.
In this case, the keyboard was not overengineered.
Too much effort to get a job done
As described by Abbott and Fisher in Scalability Rules, the second side of this equation relates to:
- Making something work harder to get a job done than is necessary,
- Making a user work harder to get a job done than is necessary,
- Or making an engineer work harder to understand something than is necessary.
Make something work harder than necessary
Abbott and Fisher share an example about working harder than necessary as it relates to a grocery store (page 4). In their words:
“Imagine that you ask your significant other to go to the grocery store. When he agrees, you tell him to pick up one of everything at the store, and then to pause and call you when he gets to the checkout line. Once he calls, you tell him the handful of items that you would like from the many baskets of items he has collected. And he can throw everything else on the floor.
Sounds crazy right? Well, I have seen this example all too often in databases where a query "selects all" from some tables and the application only uses part of the result. It is the same idea. Don't make the database, one of the costliest resources, do more work than necessary!
Make a user work harder than necessary
Let's consider the case of making a user work harder than necessary to get a job done. I’ll share a personal example I find myself in that deals with almond milk poured on cereal--a simple task.
When we are well stocked, the process includes:
- Open the fridge
- Get the milk
- Pour the milk
- Put the milk away
- Close the fridge
- Enjoy breakfast
On those unfortunate days when we are out of almond milk, the process goes from a simple 3-5 second process to a long, arduous 5-10 minute process. Extra steps include:
- Find the almonds
- Find the blender
- Clean the blender before starting
- Wait for the blender to blend
- Find a jar to store the milk
- Clean the blender
- Put everything away
The difference in time is a staggering 60-120 times longer, for the same result! (Although my wife might argue with my conclusions.) I, for one, am thankful for almond milk in a box, or kids that make the almond milk for me. I don’t even want to think about the extra effort if we used cow’s milk!
In software product deployment look for tasks that seem to take too long, or that take a little time but are done repeatedly.
Let’s not work harder than necessary. Let's simplify the workflow and focus on getting stuff done!
Make an engineer work harder than necessary to understand something
Software engineers must often continue a project someone else started. Excellent code is easy to understand and contribute to. More difficult code, unfortunately, takes more time to understand event though it does the same thing.
For example, consider the "Hello World" program. A “Hello World” tutorial is often used to introduce developers to a programming language. The program is usually a single line of code that prints “Hello World” to the screen. Here is what it looks like in Javascript:
console.log(“Hello world”);
For an engineer, easy to understand. By contrast, on GitHub, there is a 146 line file in Java that also prints “Hello world.” The output is the same, but understanding the code takes much more effort.
Yes, sometimes people do this for fun, which is fine on their own time. From a product perspective, let's focus on what the business needs.
Why is Overengineering 'Bad'?
Let’s make sure we have a clear understanding of Good vs. Bad. In the context, of this article, good relates to a fast time to market and keeping our costs in check. Bad is the opposite: longer to market and more costly.
Why overengineering is bad comes down to time. As Ashley Zahabian said,
You can’t buy time, you can’t grow time, you can’t ask for more time, you can only lose time. … We give ourselves too much time to accomplish what needs to be done. It makes us waste time, we abuse it. … Time is the most valuable resource we have.”
Time spent on the right things helps business.
Time spent on the wrong things hurts business.
Whenever we spend time on something, we trade our time from someplace else. Always. There is always a trade. The trade even extends to our personal lives.
So stop wasting time on over-engineering. You aren't gonna need it! (YAGNI)
How do I identify and avoid over-engineering?
The important thing to identify over-engineering is to ask questions and talk about it. Following are some areas to look at as you investigate. Preventing over-engineering is a matter of identifying it then finding solutions to reverse the status quo.
Projects take longer than expected
If projects, features, or any request of a development team is taking longer than expected, look into it. Find out why. It may be over-engineering, it may be something else, but a good leader needs to know why.
Undefined limits to a project
When a project is poorly defined, an engineer may be doing their best and also be doing more than is necessary. Vague requirements lead to engineers adding additional unconfirmed requirements.
For example, consider the instruction to keep a house at a regulated temperature. An engineer might assume bounds between -300 and 300 Fahrenheit to play it safe.”
Make the requirements more clear. For example, "keep a house at a regulated temperature between 60 and 80 degrees when the outside temperature is between -20 and 120 degrees."
Clarity of the limits makes sure everyone is on the same page of what is being built and when it should work.
Effort is on lower priority items
When you see effort on something that doesn’t seem important, there might be a problem.
Caleb Mbakwe shares a related story:
I know a developer who used one month to set up a deployment pipeline for a software product we were working on. In all fairness, what he tried to set up would have been very good to have but I implored him to focus on the main product first and then we can optimize the deployment process later but, well, he didn’t listen. Eventually, we missed our deadline and the developer, well, he had to be let off the team. ... If we had focused on solving the problem and building the product, maybe, just maybe, we would have met our already tight deadline…
Make sure you are working on the most important problems. Once you know what they are, focus on them, rally the team around them, solve them, and go to the next one. One step at a time, one problem at a time, we refine the product until it shines for our customers.
Lack of Visibility to Time Drivers
Reporting on effort is important for improving and understanding ineffective use of time.
In a traditional Scrum process for building products, we use story points to estimate the size of a unit of work. Along with story point estimates, we should be recording the actual effort, in hours. If we do this, we can tell when a unit of work is taking longer than expected, and overengineering could be a reason.
Here is another experiment to try. Ask your development team how much time they actually spend writing code per day. Surveys here and here show that engineers working 7-9 hour days reported actual time writing code between 2-6 hours on average.
One responder reported: “4 is pretty much average, 5 is a pretty nice day, 6 or more feels like god-mode!”
What about your team? If you don’t know these numbers for your team, how can you improve them?
Discussions about “needing a feature to close a deal”
Someone from sales might contact the development team and request a feature to close a deal. Without qualifying the request, this approach could lead to a bloated product.
The Scrum Framework has opportunities for handling feature requests built into it. Adding the request to the backlog and grooming the backlog would put the request where it should be. The Product Owner should vet such requests to make sure they have value to other customers.
Summary
In this article, we’ve talked about overengineering; what it is, why it is bad, and how to identify and prevent it. The crux of overengineering is solving problems you don’t have.
When you are working on problems you don't have, you are wasting time because "You aren't gonna need it!" (YAGNI)
You can achieve faster time to market by focusing on the problems you do have!
Now go make a difference.