One of the questions we are often asked is when do I scale my company? When setting out to build a product, there is a lot to learn and figure out. Scale is one of those questions. The purpose of this article is to talk about scale design principles.
When I get excited about something, I usually jump right in -- both feet, in the deep end. Some might think that is too scary. I just figure there is a lot I don't know and the best way to learn is to start learning.
With this approach, I learn a ton.
Yes, I make mistakes. But I learn things I never knew I never knew, and I learn fast.
A trap I have fallen into more than once is to read something inspiring and decide to follow the advice "right now". Unfortunately, all advice isn't meant to be followed "right now."
There is a time and season for everything! The Bible even says so! Sometimes, it isn't time to make the change yet. The product, team, or customers might not be ready. You might not be ready.
To avoid this trap, I’ve learned to be a little more patient. Keep reading, keep learning, but also learn when to apply what I've learned. Learn the deeper meaning behind the lesson, what it should affect, why it is valuable, and when it should be applied.
Today, I'll discuss some problems with scaling at the wrong time, and finding the right time. Then I'll discuss the DID (Design Implement Deploy) process for keeping costs down as you scale your product.
This article is part of a series on the rules of scalability for product development. It focuses on Rule 2 - Designing Scale Into the Solution from the book Scalability Rules.
Problems with scaling at the wrong time
Scaling is a challenge! Companies need to scale gradually, as the user-base grows. Scale too soon, and it'll cost too much. Scale too late, and the user experience is hit, inhibiting the adoption of your product. Either situation can threaten a company.
At AKF Partners, we are often asked if a company is scaled appropriately or prepared to scale. Our technical due diligence and workshops dive into a company's ability to scale. If you have the same question, or need help reach out to us.
Problems scaling too early
In a study from a few years ago, the researchers estimated that 70% of tech startups failed because they scaled too soon. There are many ways to scale too early. Here are some signs to look out for.
One simple way to describe the problem of scaling too early is that it spends money too fast. It ends up costing more to provide your product than you earn.
Suppose today, with a simple architecture, you were able to handle demand. Now compare this ideal cost to a system handling 10x, 20x, or 100x our current demand. The first would cost less and serve the need. The second would cost more while we pay for idle servers and still serve the same need. We want to reduce the use of resources not in use.
Another side-effect of a high burn rate, is that it affects a company's ability to respond to customers. Quality usually goes down as focus shifts to meeting demand.
As money gets tighter, executives tend to tighten and narrow their focus. This means there is less chance to experiment and tune and make sure the product is the best it can be. There are less funds available for quality assurance. This further limits the likelihood of building a viable product.
So don't scale too early!
Problems scaling too late
The problem with scaling too late relates primarily to the user experience.
Initially, there is interest and excitement in your product. People tell their friends. More users come. Revenue starts to grow. And you happily dream of making it big and think, "This is it" as you watch your creation grow.
But, if you aren’t ready to scale when the need hits, you might disappoint customers with your service. This leads to two problems.
- Excited customers who told friends about your product might feel embarrassed and stop telling others. They also might use it less feeling a little betrayed. You'll have to work extra to bring them back.
- First impressions matter! The person visiting the site for the first time might never come back. At best, you'll have to overcome the initial negative experience and bias towards your product before product adoption will occur.
Either one of these conditions affects the "usefulness" and "ease of use" of your product. In turn, this affects fanout which inhibits your company's ability to grow. (See the Drivers for Virtual Growth in Abbot, Fisher, and Lyytinen's Book "The Power of Customer Misbehavior").
So, let's get it right and keep costs down, while also maintaining readiness to scale.
When is the right time to scale?
There are two things to consider when planning to scale: First, look at your overall product development cycle and second, user growth.
Consider your product development cycle
First, let's look at the product development cycle. Sean Ellis, Founder and CEO of Qualaroo, identified four stages in SaaS products. The stages are: pre-startup, startup, growth, and maturity.
Before diving into scalability questions, you must be out of the pre-startup stage. You need to have identified an idea and tested it with customers. It's best to have a buyable minimally viable product (MVP) and real customers paying you.
If you are not at this point, keep working with your customers to make sure you are solving a real problem, but don't worry about scaling just yet.
Also, remember, this applies to any work within your company. Even a feature enhancement should be validated with paying customers before scaling. One thing to watch out for is if you have built a "quick feature" to test interest, and then notice there is minimal interest. Should you "double down" and "improve" the feature? If it is lacking what people are asking for, perhaps. But if no one is using it, that is an indication that your effort might be better placed in another area.
Once you have paying customers, you’ve got something to work with. Now is the time to consider the growth of your users.
Consider the growth in users
At this point, consider factors that affect user's interaction with your product. Ask questions like:
- How many unique users do I have?
- How are new users brought in?
- What are they doing with the product?
- What are they paying for?
- Why are they paying for it?
- How will their behavior change in the next 3 months, 6 months, or 1 year?
- What is driving their interaction with my product?
These questions tease out the value of the product and highlight what you should scale.
How much should I scale?
Ideally, what you want is JIT (just-in-time) scalability. The idea originates from JIT manufacturing, and relates to reducing delivery time. JIT scalability is the ability to scale up or down when needed, as needed.
For the best value, your resources should be well used, but not overloaded. You want to minimize idle resources, especially ones you are paying for.
The problem is that you don’t know the day scale goes up. You can see trends, and guess, but getting it perfect is hard. And again, it can be troublesome if you are too early or too late.
As this article is part of the series on reducing the equation, the goal here is to simplify the deployment. We don't want to over-engineer or solve a problem we don't have. In this case, we don’t want to overdo the deployment, and solve the problem (scale) that we don’t have today.
DID process for scale
At AKF Partners, we help companies design for scale. We have an approach to provide JIT scalability called the DID (Design Implement Deploy) Process.
The DID process originates in Rule 2 of Abbott and Fisher’s book on Scalability Rules.
The principle is to design for 20x capacity, implement for 3x capacity, and deploy for about 1.5x capacity. Below is a table for guidance.
Design
To start with, we assume that whiteboarding a design and the consequent discussion are significantly less expensive than building a solution in code.
We want to design far enough in advance of our need to give clarity of what to build when use of our product rises.
Scalability in the design phase should focus on the big problems.
Some areas to consider include:
- Bottlenecks
- Customer uses and geography
- The organization of services (AKF scale cube will help)
- Monolith vs microservices
When your design is ready, you end up with a plan that you can give your engineering team to build when the time is right. In the design phase you should consider 20x to an infinite amount more than your current design can handle.
In the design phase, your overall costs are low. You will have higher intellectual costs as you leverage scaling experts. But your engineering and asset costs will be low since nothing is built.
Implement
When your product approaches the need to serve larger volumes of traffic, it is time to build the solution. This is where your team writes the code and prepares for deployment. Your design likely needs adjustment, so you'll still use experts, but this stage is about building.
Start early enough that you'll be able to finish before the demand arrives. Iterate to make sure you have a viable solution. Test it.
You'll want to implement a solution that is a little beyond what you'll need, but not too far ahead. We recommend an implemented solution between 3x and 20x your current demand.
The cost of implementation is a little higher as engineers build the solution. Because we already have the design, the intellectual cost has gone down. The cost of assets goes up because of the need to test the implementation and validate the design.
Hopefully, we are scaling as revenue is going up. Scaling at the right time, in sync with revenue, will offset some of the costs to scale, keeping our costs low.
Deploy
When users descend upon our service, it is time to deploy our product. To handle the increased load, we start spinning up servers.
Ideally this happens as close to the day the load increases. In practice, we suggest keeping a deployment 1.5x to 3x above your expected capacity. This keeps a cap on the amount paid for resources that aren’t in use while allowing for room to grow without changing the deployment.
The cost for deployment is still under control. The physical assets do cost more, but they scale with increased use. The intellectual costs are lower for our experts and engineers as they do less work on the project.
An advantage of this approach is it lends itself to purchasing equipment just ahead of our needs. We should be able to model the growth and anticipate when to purchase new equipment so we are prepared to scale.
Infrastructure as a service
Today, many cloud providers offer infrastructure-as-a-service (IaaS) environments. These services allow just-in-time scaling by adjusting hardware in a matter of minutes. Automatic load balancing and serverless technology are particularly beneficial.
Automatic load balancing delivers traffic to different computers to effectively handle requests. When there are more requests, the service starts new servers automatically. When there are fewer requests, the services shuts down servers.
Serverless technology starts from the idea that a server ONLY needs to be running while it is doing work. These services provide ways to only use a server when needed, and only charge when the server is in use. Examples include AWS Lambda functions, Azure Functions, and Google Cloud Functions.
Leveraging these services leads to an ideal deployment keeping costs inline with resource use. The cost of the assets is directly related to how much the asset is used. Since the number of assets automatically grows or shrinks on demand, our product is scaled correctly.
This doesn't mean you should move everything to the cloud right now. The transition needs to be thought through. Just be aware that these services exist. Use them as benefits you and your company. If it makes sense, make the move.
Following the DID process helps keep costs in control. This chart illustrates the management of the total costs over the project.
Conclusion
When building a new product, it is important to scale at the right time. Wait until you have a product built that you know is solving a problem and customers are willing to pay for it.
When it comes time, design how to scale early, and think big, try to figure out where the bottlenecks will appear.
Design to at least 20x your current load. As growth comes, implement a solution to handle 3x to 20x to help manage costs. Once it comes time to deploy, try to keep each asset as busy as you can without overloading it. Look for a deployment around 1.5x to 3x your current demand.
This approach helps teams scale just-in-time. It keeps costs down while allowing for increased use of the product.