When adopting serverless computing for your applications, one of the common challenges our clients face is increased latency, especially during the startup process, which is often called a cold start. Cold starts are more prominent when your code payload is too large or when other factors contribute to delayed function execution. In a prior post we talked about selectively using serverless for the appropriate use cases. While many of the cloud providers have introduced “keep warm” features, they don’t always avoid cold starts or long running warm up times. The user of serverless compute, if not carefully architected, can result in an impact to your software product’s customers. In this blog post, we’ll explore strategies to avoid slow serverless startups, highlight common causes of latency, and discuss the trade-offs and benefits of using serverless in the public cloud.
Avoiding Slow Serverless Startups with Large Code Payloads
One of the primary causes of latency in serverless functions is the size of the code payload. Larger codebases take longer to load into memory, which can significantly increase cold start times. When serverless functions aren’t actively running, the environment that handles your function is paused, and when it is invoked again, it needs to be reinitialized—this is where cold starts occur. Large payloads exacerbate this issue by making the initialization process slower.
Strategies to Reduce Code Payload:
1. Minimize Dependencies: Many developers include unnecessary libraries or modules that inflate the code size. Tools like Webpack or Rollup can help bundle and tree-shake your code to include only the necessary functions and dependencies. For example, if you’re using AWS Lambda, ensure that your package only contains the essential code.
2. Use Lambda Layers: AWS Lambda supports layers, which allow you to package libraries, frameworks, or other dependencies separately. This reduces the size of the core function, speeding up the cold start. Azure Functions offers a similar capability with Azure Functions Proxies, where certain logic is abstracted away.
3. Lazy Loading of Dependencies: Load libraries or modules only when they’re needed during function execution. This approach can reduce cold start time by deferring heavy computations and imports until they’re absolutely required.
Other Causes of Serverless Latency
While large code payloads are a common issue, they are far from the only reason why latency occurs in serverless architectures. Here are other causes to consider:
1. Cold Starts: In serverless environments, cold starts occur when the infrastructure spins up from a halted state. This is especially noticeable when functions haven’t been used for a while. Cold starts are more frequent in languages with heavy runtimes like Java or .NET compared to lighter ones like Node.js or Python. Serverless platforms like AWS Lambda and Azure Functions try to mitigate this by keeping “warm” instances for frequently accessed functions, but cold starts can still impact response times.
2. Network Latency: Functions that rely heavily on external services, such as APIs or databases, may experience higher latency. For example, a function invoking an API in a different region could introduce round-trip delays. To reduce this, it’s important to co-locate your serverless functions and dependent resources within the same region to avoid cross-region latency.
3. I/O and Database Operations: Serverless functions often rely on databases, and poorly optimized queries or high-latency database connections can slow down your function’s response time. Using tools like DynamoDB, Firebase, or Aurora Serverless can help, as these are designed to handle serverless traffic with elasticity, but careful database management and indexing are essential.
4. Concurrency and Throttling: Serverless platforms have concurrency limits. For example, AWS Lambda has a default concurrency limit of 1,000 simultaneous executions per region. If your system exceeds this limit, subsequent requests can queue up, adding latency as requests wait for an available execution slot. You can adjust concurrency settings based on your needs, but hitting these limits can cause performance bottlenecks.
Technologies that Promote Elasticity
One of the biggest advantages of serverless computing is its elasticity—the ability to automatically scale based on demand. Some technologies stand out for their ability to manage elasticity and handle spikes in traffic:
1. AWS Lambda: AWS Lambda automatically scales based on incoming requests and adjusts the number of function instances running at any time. However, for critical functions, using Provisioned Concurrency can ensure that pre-warmed instances are always available, drastically reducing cold start times.
2. Google Cloud Functions: Google Cloud Functions scales automatically without any manual intervention, making it ideal for elastic workloads. It works well with Google Cloud Pub/Sub for event-driven architectures, allowing automatic scalability based on incoming events.
3. Azure Functions: Azure Functions offers premium plans that eliminate cold starts and provide automatic scaling. The premium plan also includes features like virtual network integration and unlimited function duration, which allow greater control over execution environments.
Drawbacks of Serverless in Public Cloud
Despite the many advantages of serverless computing—such as scalability, reduced operational overhead, and cost savings—there are a few drawbacks to be aware of when using serverless in public cloud environments:
1. Cold Starts: As mentioned earlier, cold starts can introduce unpredictable latency, especially for high-throughput or latency-sensitive applications. While provisioned concurrency and other optimizations help mitigate this, the problem still persists in certain use cases.
2. Vendor Lock-in: Using serverless functions often ties you to the specific APIs and tooling of a cloud provider, such as AWS Lambda or Google Cloud Functions. Moving away from a serverless provider can involve substantial rewriting of code and architecture, leading to long-term vendor lock-in.
3. Cost Overhead for Frequent Functions: Serverless architectures work best for spiky workloads where traffic is unpredictable. However, if a function is invoked continuously or has long-running processes, it may be more cost-effective to use traditional server-based or containerized approaches like Kubernetes or any of the managed Kubernetes cloud services. Serverless can become expensive if not carefully monitored and optimized.
4. Limited Execution Time: Platforms like AWS Lambda cap the maximum execution time at 15 minutes, which can be a limiting factor for tasks that require longer processing times. Azure Functions and Google Cloud Functions have similar restrictions. For long-running tasks, serverless might not be the best option.
How AKF Can Help
Serverless computing provides an incredible platform for elasticity and scalability but comes with challenges like slow cold starts and latency caused by large code payloads. Reducing code size, minimizing dependencies, and using smart strategies like Lambda Layers can reduce cold start time. Additionally, network latency, concurrency limits, and database performance can all contribute to latency in serverless environments, making it essential to architect systems with best practices in mind.
While serverless architectures are powerful for specific use cases, engineers need to be aware of the limitations and drawbacks, including cold starts, cost implications for heavy workloads, and potential vendor lock-in. When used correctly, serverless architectures can provide a cost-effective, scalable, and resilient foundation for modern applications.
We've helped hundreds of clients around the globe scale their architecture and avoid pitfalls that can have a negative impact on the software product. Contact Us For more insights on scaling architectures and making the most of cloud technologies, check out AKF Partners’ recommendations.