AWS Lambda concurrency is crucial for application performance and stability. You may notice lambda concurrency once it is too late, in the middle of a production incident.
For example, you may not do any concurrency config, and suddenly, your requests and processing do not work because AWS throttles new lambda executions. Or you may do some concurrency configuration, and you still get throttles. Finally, you may tie things down so that it cannot catch up with the request throughput, and your requests may wait in a queue for a while.
Sounds familiar? Let's discover various lambda concurrency configurations so you can feel confident about the topic.
AWS Lambda: concurrency types
Here is a general list of possible concurrency configurations that are available when we talk about lambdas in AWS:
Unreserved concurrency
Reserved concurrency
Provisioned concurrency
For SQS events - maximum concurrency
For Kinesis events - the parallelization factor
AWS Lambda: unreserved concurrency
Every AWS account has a so-called lambda concurrency pool. Its size is 1000 by default, and you can reach out to AWS Support and ask them to increase the limit. Whenever your Lambda is started, it takes one slot from the pool. Once it finishes, it returns a slot to the pool. As a result, you can have 1000 concurrent lambda executions at a time. If you need to start 1001st Lambda when you already have 1000 Lambda running, you won't be able to, and your Lambda will be throttled. So, if your pool is empty, you can only start a new lambda execution once other lambdas complete their execution.
AWS Lambda: reserved concurrency
You can configure a lambda with a reserved concurrency. With reserved concurrency, Lambda will get a piece of the pool. As a result, if you configured your Lambda to have 5 reserved concurrency executions, your unreserved concurrency pool will be smaller by 5 and become a 995 value pool. You can have fewer lambda executions at a time, but you cannot have more. So if your Lambda is 5 reserved concurrency, you have 5 instances already running and won't be able to start a new lambda execution. The pros of this approach are that even if your unreserved pool is empty, you can still run reserved concurrency lambdas if they have reserved concurrency pool capacity.
AWS Lambda: provisioned concurrency
With this concurrency configuration, you can have several instances in a hot start state (read more about AWS Lambda instances and start types). As a result, you will have some number of lambda instances ready to handle requests that come in - without waiting for additional init time (cold start time). Once you have more lambda executions than you configured in your provisioned concurrency configuration, you will get new instances cold started using unreserved pool executions, or using the reserved concurrency if your Lambda has it.
AWS will charge you for the running instances, like if it is a VM running, so be mindful of it. Lambda provisioned concurrency does not come for free.
To configure provisioned concurrency for a lambda, you have to publish a version or create an alias and then reference the Lambda using it. Provisioned concurrency does not work with $LATEST. This is an AWS requirement, so I will not go too deep into how to configure provisioned concurrency for a lambda.
AWS Lambda: SQS trigger max concurrency
When Lambda has an SQS trigger configured, AWS has a polling process running in the background. It constantly reads from the queue, and when new messages come in, it starts a new lambda execution based on how it is configured (batch size, batching window).
It is possible to configure how many lambda executions can be started by this polling process.
For example, you configure max concurrency to be 5 and not batching. Once you receive 7 messages, the polling process will start 5 executions. Two messages will be waiting for some of the executions to complete.
The executions here are taken from the unreserved concurrency pool, or from the reserved concurrency if your Lambda has it. So, if that pool is empty, new executions will be throttled.
Another thing to note is that SQS trigger invocation is synchronous (read more about AWS Lambda Invocation Types).
Combine SQS trigger max concurrency with reserved concurrency
You can use both reserved concurrency executions and SQS trigger max concurrency. This way, your Lambda can always start executions for critical processing tasks. It is better to use the same number for both concurrency configurations.
AWS Lambda: Kinesis events parallelization factor
AWS Lambda's Parallelization Factor for Amazon Kinesis allows you to increase the speed at which your function processes records by specifying the number of concurrent batches that Lambda polls from a single shard. This feature helps with faster stream processing without the need to over-scale the number of shards, while still ensuring the order of records processed. By setting the Parallelization Factor, you can have multiple Lambda invocations processing data from the same Kinesis shard concurrently, enhancing scalability and throughput. The default factor is 1, but it can be increased up to 10, and Lambda still processes records with the same partition key in order.
Let's do the math real quick. The maximum number of concurrent executions for a Kinesis trigger is the number of shards multiplied by the parallelization factor. For example, a stream with 4 shards and a parallelization factor of 10 can run up to 40 lambda executions at the same time. Like with SQS, these executions are taken from the unreserved concurrency pool, or from the reserved concurrency if your lambda has it.
AWS Lambda: which configuration to use
Now, let's go back to the problems from the beginning of this post and see which configuration helps with each of them.
No concurrency config, and lambdas get throttled
All lambdas without reserved concurrency share the same unreserved pool. A traffic spike in one lambda can take the whole pool, and then every other lambda in the account gets throttled. Give your critical lambdas reserved concurrency, so they always have their own capacity.
AWS always keeps at least 100 executions in the unreserved pool, so you cannot reserve all of it.
Concurrency is configured, and you still get throttles
One common reason is SQS max concurrency set higher than reserved concurrency. The polling process starts more executions than the lambda is allowed to run, and the extra ones get throttled. That is why I keep both numbers the same.
Throttled messages go back to the queue, and after too many attempts, they can end up in the dead-letter queue.
Requests wait in a queue for a while
Here, you tied things down too much for your throughput. Messages are not lost, but they wait until running executions finish. Check how old the oldest message is (ApproximateAgeOfOldestMessage for SQS, IteratorAge for Kinesis), and raise the max concurrency, the reserved concurrency, or the parallelization factor.
The API is slow on the first requests
This is usually cold starts. Provisioned concurrency keeps instances ready for you, but you pay for them for as long as they run, even when no requests come in.
AWS Lambda: concurrency summary
Here is a short recap of all the configurations:
Unreserved concurrency - the shared pool of the account, 1000 by default.
Reserved concurrency - a part of the pool only for one lambda, which is also its upper limit.
Provisioned concurrency - instances initialized ahead of time to avoid cold starts. You pay for them.
SQS maximum concurrency - how many executions the SQS polling process can start.
Kinesis parallelization factor - how many batches from one shard are processed at the same time, from 1 to 10.
Final words
That concludes the third topic of the series on AWS Lambdas. With these configurations in mind, you can plan your lambda concurrency before throttles show up in production. Please reach out if you have questions or think some information needs to be updated and corrected. Thank you so much for your attention!