Nimbly Logo
Is Serverless really the way to go?

Is Serverless really the way to go?

Chris Ballment, March 14, 2023.

This blog will dive into serverless technology, and compare it to traditional server-based applications. We will look at the benefits and challenges of serverless technology, and compare the cost of running a serverless application with a traditional server-based application. In future blog posts, we will explore specific tooling, processes, and accelerators to speed up your time to customer value and reduce development effort.

So, what is serverless?


Serverless technology is a relatively new approach to application development that has gained a lot of popularity in recent years. It refers to the design and deployment of applications and services that do not require the traditional approach of deploying a server to run the application.

In a traditional server-based architecture, developers create applications that run on a server. This server is responsible for handling all the application logic and processing, and it is the developer's responsibility to manage the server infrastructure. This can be a significant burden, requiring expertise in system administration and infrastructure management, and it can also be expensive to maintain.

In contrast, serverless technology abstracts away the server infrastructure and allows developers to focus on writing code that runs in response to specific events or triggers. In a serverless architecture, a cloud provider is responsible for managing the underlying infrastructure, including server management, scaling, and maintenance. The cloud provider will only charge developers for the actual time and resources used to execute the code, rather than requiring them to pay for a fixed amount of server capacity.

One of the key benefits of serverless technology is that it enables developers to focus on writing code rather than managing server infrastructure. This can significantly reduce the time and resources required to develop and deploy applications, allowing developers to be more productive and deliver more value to their customers.

Another benefit of serverless technology is that it enables developers to create highly scalable and resilient applications. Serverless architectures can automatically scale up or down based on demand, which makes it easy to handle sudden spikes in traffic or usage. Additionally, since the cloud provider is responsible for managing the infrastructure, it can often provide high levels of reliability and availability.

Applications for APIs


Traditional APIs are typically built using a server-based architecture. This means that developers must manage the infrastructure required to run the API, including servers, networking, and storage. The developer must also ensure that the API can handle the expected load and traffic, which can be difficult to predict and plan for. Traditional APIs can be expensive to build and maintain, as the infrastructure required to run them must be purchased or leased, and often requires a team of system administrators to manage.

Serverless APIs, on the other hand, do not require any infrastructure management. They are built using a cloud-based service, such as AWS Lambda or Azure Functions, which takes care of the infrastructure required to run the API. Developers can simply upload their code to the cloud service and specify the events or triggers that will invoke the API. The cloud service automatically scales the infrastructure to meet the demand, so developers do not need to worry about scaling up or down.

In terms of performance, serverless APIs can be just as fast as traditional APIs. Since the cloud service automatically scales the infrastructure to meet demand, serverless APIs can handle sudden spikes in traffic without any additional configuration or management.

Challenges with Serverless


In this section, we will explore some of the challenges that developers face when building serverless applications.

Cold start latency

The largest and most commonly experienced challenge faced with the adoption of serverless technology is the time it takes to provision a new instance of the function or its cold start latency. This typically happens when the function has been invoked after a period of inactivity (typically between 5-7 minutes). The cold start latency is impacted by various factors, including the language of the function, instance sizing (CPU-intensive activites), and package sizing (the size and number of dependencies).

AWS recently released a feature called AWS SnapStart [1] (for Java language functions only at this point). This feature initialises the execution environment and takes a snapshot, which it uses on every subsequent execution rather than re-initialising it. This performs the heavy lifting once, but does have some considerations - particularly around initialisation activities like database connection pooling. This has been shown to reduce the cold start latency by up to 90%.

There are other ways that you can avoid cold starts:

  • Set up a CloudWatch event rule to call the lambda function at a pre-defined interval, e.g. every minute, to keep the function warm and prevent cold starts. This would result in 1,440 additional calls per day ($0.01 monthly in calls of a 128mb Lambda function), and any engineering time to set it up.
  • Create the function with Provisioned capacity [2]. This adds between $1.50 and $7 per month to keep a single instance of a function warm, with no engineering cost.

Debugging serverless functions

Debugging and testing serverless functions can be challenging, as the functions are typically invoked by events or triggers, rather than being called directly. This means that it can be difficult to test the function locally, as it is not possible to simulate the event or trigger that would normally invoke the function.

This can be mitigated with a robust logging strategy however the debugging and testing will typically take longer than a server application. Logging can be used to record the input and output of the function, which can be used to debug any issues that occur. It can also be used to record any errors that occur, which can be used to identify and fix any bugs in the code. In between, a developer must commit and deploy the code to their cloud platform so without robust CI/CD processes, this can become time-consuming and onerous.

Statelessness

While it's possible that a traditional microservice to be either stateful (persisting data between calls) or stateless, you should assume that your serverless function environment only exists for one invocation. This means that any required data must be retrieved, and stored externally (in a database or other service). It is generally best practice in microservice development to persist data externally to be more fault tolerant. For example, consider keeping state of a shopping cart within a stateful service. If during the course of a user's session the service becomes degraded and fails, without external persistence, the instance created to replace the failed instance would have no knowledge of the prior state of the shopping cart.

This lack of persistence further impacts other non-data components. One important example is connection pooling for databases. In a traditional microservice, a number of open connections to a database are maintained in memory, which can be reused for subsequent requests. In a serverless environment, the function is invoked and executed in a new instance of the execution environment, requiring a new connection be started at each cold start.When designing a microservice that connects to an AWS RDS database, Amazon RDS Proxy [3] provides connection pooling with no code changes for serverless functions.

In both architectures with high loads, it is possible for the number of connections to the database to exceed the maximum number allowed, or for the database to become overloaded. In this case, it is recommended to consider a decoupled architecture [4] if there is high volumes expected.

Vendor lock-in

Serverless functions are typically hosted on a cloud platform, such as AWS Lambda or Azure Functions. This means that the application is tied to the cloud platform, and cannot be easily migrated to another cloud platform. This can be mitigated by using a serverless framework, such as Serverless Framework [5], which allows developers to write their code in a language of their choice, and deploy it to any cloud platform that supports the framework. This means that the application can be easily migrated to another cloud platform, or even to a traditional server-based architecture.

Cost comparisons


In this section, we will compare the cost of running a serverless application with a traditional server-based application. The assumptions that have been used are:

  • A largely idle development and testing environment.
  • A production environment servicing various volumes of requests per second (rps) as stated in the below scenarios.
  • Each microservice hosted on traditional architecture is a t4g.micro (2 vCPUs and 1GB of memory), servicing at peak 5,000 requests per second.
  • Traditional architectures utilise a network load balancer for routing.
  • Both architectures will have an identical volume, and consequently cost, of API Gateway requests.
  • The AWS region "ap-southeast-2" (Sydney, Australia) was used to estimate cost.
  • All costs are in USD per month.

Single microservice

This comparison is for a single microservice, which is invoked by an API Gateway endpoint. The microservice is a simple function that returns a JSON response.

Production load Traditional $ Serverless $ Savings $ Savings % Comments
At rest $26.14 $0.00 $26.14 100%
1 rps $81.77 $3.68 $78.09 95% 2.6m requests per month
10 rps $169.47 $96.06 $73.41 43% 26m requests per month
50 rps $533.47 $481.08 $52.39 9% 150m requests per month

Five microservices

This comparison is a more likely analogue for a real application, containing five microservices invoked by an API Gateway endpoint.

Production load Traditional $ Serverless $ Savings $ Savings % Comments
At rest $130.70 $0.00 $130.70 100%
5 rps
(1 rps each)		 $438.09 $48.42 $389.67 89% 13m requests per month
50 rps
(10 rps each)		 $852.25 $485.98 $366.27 43% 150m requests per month
250 rps
(50 rps each)		 $2,691.85 $2,430.70 $261.15 10% 650m requests per month

Conclusion


Overall, serverless technology offers a promising approach to application development, but developers should be aware of the potential challenges and plan accordingly. Additionally, in most start-up or scale-up organisations, there are significant cost and time benefits to adopting serverless versus traditional architectures.

Are you looking to advance your journey on serverless and need a boost? Reach out to Nimbly..

References