AWS Lambda, for those not familiar with it, is an event driven serverless (what is serverless?) platform for compute resources. Rather than paying for an EC2 instance or running docker instances on ECS that runs executable code, Lambda offers an alternative that is for many applications much less expensive. For instance, if you need to routinely resize photos that are provided to you from various distributed sources, you create an S3 bucket for images to be loaded to, which can act as a trigger for a Lambda job to automatically resize them and output the resized photos into a different bucket.
- What is AWS lambda used for?
- Which language is best for AWS lambda?
- How to use AWS lambda
- Lambda/Serverless Frameworks
- Lambda Gotchas
- Setting up EBS Snapshotting Using Lambda and Terraform
What is AWS lambda used for?
There are a plethora of useful functions you can leverage AWS Lambda for. You can use it as a backend service for mobile applications that need to call databases, post data, update files. Lambda can also be used on a cron to run an EBS snapshot backup routinely as well as snapshotting RDS and replicating snapshots to another region to facilitate disaster recovery. Lambda can also be built to run repeated jobs such as resizing images, reacting to cloudwatch metrics and actioning accordingly (rebooting systems with pegged CPU, etc), or hitting an analytics API and loading the parsed data into a data warehouse.
Which language is best for AWS lambda?
First and foremost the languages supported in AWS Lambda at the time of this writing (last updated 11/16/21) include
To keep up to speed with supported languages for Lambda you can refer to their FAQ page located here.
How to use AWS lambda
Methods for invoking Lambas currently include:
- Amazon Alexa
- Amazon Managed Streaming for Apache Kafka
- Self-managed Apache Kafka
- Amazon API Gateway
- AWS CloudFormation
- Amazon CloudFront (Lambda@Edge)
- Amazon EventBridge (CloudWatch Events)
- Amazon CloudWatch Logs
- AWS CodeCommit
- AWS CodePipeline
- Amazon Cognito
- AWS Config
- Amazon Connect
- Amazon DynamoDB
- Amazon Elastic File System & Elastic Load Balancing (Application Load Balancer)
- AWS IoT & IoT Events
- Amazon Kinesis & Kinesis Data Firehose
- Amazon Lex
- Amazon MQ
- Amazon Simple Email Service
- Amazon Simple Notification Service
- Amazon Simple Queue Service
- Amazon Simple Storage Service (Amazon S3)
- Amazon Simple Storage Service Batch
- Secrets Manager
- AWS X-Ray
A full list of AWS services that can act as Lambda triggers can be found here.
Lambda/Serverless Frameworks
Over the past few years a number of frameworks have emerged for building and maintaining serverless resources in AWS. I will break these down into broadly 2 categories, the first is a more traditional approach to infrastructure as code in which you define each of the pieces needed for your lambda operation and use the framework to generate CloudFormation templates and apply them. The second category is frameworks specifically built to update and maintain your Lambda and serverless resources such as API Gateway and Dynamo DBs throughout their full life cycle.
Infrastructure as Code Frameworks:
Serverless Frameworks:
Your mileage may vary with each of these choices. Tools like Claudia, Node Lambda, and Serverless are built on node and will require npm and node locally to run. Tools such as Zappa and Chalice are Python based. Each of these frameworks supports more than their native language in Lambda, however depending on your workflow, development environment, and any CI processes you have this may sway your decision.
Lambda Gotchas
As with any cloud service offering there are configuration maximums and limitations to be aware of. It is best to be aware of these limitations and maximums ahead of time so there are no surprises down the road as demand increases for your application and your scaling needs change. The information outlined here is based off of current limitations, please ensure you are using the most up to date information by checking here.
Service Limits:
- Invocations Per Second: 1,000 (this limit can be increased by requesting a limit increase from AWS support, please note this can take more than a day)
- Concurrent Executions: 1000 per region
- Default Max Execution Duration: 300 seconds (NOTE: this can be increased to 15 minutes)
- Memory Allocation: Min:128mb Max:10GB (as of December 1, 2020)
- Deploy Package Size(compressed): 50MB
- Total Size of All Deployment Packages Per Region:75GB
Some other things worth mentioning are that if you are running a cron based Lambda function for things such as snapshotting EBS volumes it is important to monitor these jobs, as the size of the backup set grows it will take longer to execute the lambda function. You can setup monitoring on an individualized basis in CloudWatch. While these individualized monitors are a fantastic idea it is also worth setting up some generalized blanket monitoring such as lambda invocation rate per minute and Lambda error rate. This way you are alerted when you are approaching your service limits for AWS Lambda invocations and can request service increases, or be aware if there is a spike in Lambda error rates.
Setting up EBS Snapshotting Using Lambda and Terraform
Rather than endlessly rambling about all the cool features and things you can do with AWS I wanted to create an actual get your hands dirty dive into Lambda. For this lab we'll setup a Lambda running on cron trigger to snapshot our EBS volumes and another lambda to delete snapshots that are more than 90 days old. For this example we will set this up using terraform. To follow along visit the Github project here, follow the instructions in the readme and profit! The code for the snapshot and snapshot management is originally from this repo.
Final Thoughts
Hopefully this post as has been helpful in seeing ways you can use Lambda and what tools and frameworks are available to use with your AWS Lambda projects. Thank you for taking the time to read this post, I look forward to creating future blogs around serverless resources and other infrastructure as code topics.