Deploying a Python Lambda with dependencies with AWS CDK
Context
I recently had to deploy a simple Lambda function, and I figured I’d take the opportunity to test AWS CDK.
I thought it’s be a quick project, but that just shows how naïve I am. Long story short, the AWS docs fall far short and even Stack Overflow doesn’t have a complete answer to this problem so I thought I’d document my solution.
In this tutorial, we will:
- set up the CDK environment and extra library we need to deploy our Lambda function
- define a simple Lambda + API Gateway stack
- organize our applicative code, complete with external library
- bundle and deploy the Lambda function
The final project structure is available in this GitHub repo.
Prerequisites
You will need to install Docker and the AWS CDK.
Set up the CDK environment
The first step is to create the CDK environment. Create an empty project folder, and initiate a new CDK project:
$ cdk init --language python
This initiates the folder like so:
<project-name>
├── .gitignore
├── .venv
└── ...
├──README.md
├── app.py
├── cdk.json
├── requirements-dev.txt
├── requirements.txt
├── source.bat
├── tests
│ ├── __init__.py
│ └── unit
│ ├── __init__.py
│ └── test_<project-name>_stack.py
└── <project-name>
├── __init__.py
└── <project-name>_stack.py
Note: the AWS CDK chooses crappy default names for the generated files, you can go ahead and rename them.
The AWS CDK generates a new virtual environment and requirement file. However, you’ll need an additional library to bundle your Python dependencies. Go ahead and add aws-cdk.aws-lambda-python-alpha to your requirements.txt file.
requirements.txt:
aws-cdk-lib==2.66.1
constructs>=10.0.0,<11.0.0
aws-cdk.aws-lambda-python-alpha
Now you can activate the virtual environment and install the dependencies:
$ source .venv/bin/activate
$ pip install -r requirements.txt
$ pip install -r requirements-dev.txt # if you want to write tests for your infra
At this stage, your CDK environment is set up. If you run cdk synth you should not see any error messages (but nothing useful will happen either).
Set up your Lambda code
The next step is to create a folder for the code that will go in your Lambda function and its dependencies. Go ahead and create a src folder (you can call it what you want, obviously, just change the rest of your code accordingly).
$ mkdir src
$ cd src
This is the root folder for your application code. You can now initialize a new project using poetry or pipenv (I only tested poetry but from the docs, pipenv should also work).
⚠️ If you’ve been following along, you’re still in the CDK virtual environment. Don’t forget to run deactivate before creating your new environment.
$ poetry init
$ poetry add aws-lambda-typing
In this example, the primary role of aws-lambda-typing is to demonstrate how to add an external library. It is also a nifty tool to navigate the ugly JSON payloads your Lambda functions receive as input.
For the purposes of this tutorial, we’ll be writing a Lambda that just sends back what it’s been called with.
src/echo/main.py:
from aws_lambda_typing import context as context_, events
def handler(event: events.APIGatewayProxyEventV1, context: context_.Context):
method = event['requestContext']['httpMethod']
path = event['requestContext']['path']
print(f"{method} {path}")
echo = ""
if method == 'GET':
query_params = event['queryStringParameters']
echo = "&".join(f"{k}={v}" for k,v in query_params.items())
elif method == 'POST':
echo = str(event['body'])
return {
'statusCode': 200,
'headers': { "Content-Type": "text/plain" },
'body': echo + " right back at you"
}
Configuring the stack
We can now configure the infra that we want to deploy. Here we’ll keep to a simple stack comprised of our Lambda function and an API Gateway.
The meat of the configuration is in the infra/stack.py file, which is how I renamed the original file generated by the cdk init command:
from aws_cdk import (
aws_apigateway as apigateway,
aws_lambda as lambda_,
aws_lambda_python_alpha as lambda_python,
Stack
)
from constructs import Construct
class EchoService(Construct):
def __init__(self, scope: Construct, id: str):
super().__init__(scope, id)
handler = lambda_python.PythonFunction(
self,
"echo_lambda",
entry="src", # this should point to the root level of your applicative code
runtime=lambda_.Runtime.PYTHON_3_9,
index="echo/main.py", # relative path to your main Python file, from `entry`
handler="handler" # which function to call in the main Python file
)
api = apigateway.RestApi(
self,
"echo_lambda_apigw"
)
api_lambda_integration = apigateway.LambdaIntegration(
handler,
request_templates={"application/json": '{ "statusCode": "200" }'}
)
api.root.add_method("GET", api_lambda_integration)
api.root.add_method("POST", api_lambda_integration)
class EchoServiceStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
EchoService(self, "echoService")
The main point to note is the use of the PythonFunction class (which comes from aws-cdk.aws-lambda-python-alpha). In particular, the entry parameter is where the magic happens, although the official doc doesn’t really highlight this.
This parameter should point to the root of your applicative code, where the requirements file (Pipfile, poetry.lock or requirements.txt). Everything in this directory will be bundled in the Lambda code, unless you use the bundling option (with bundling=python.BundlingOptions(asset_excludes=["list", "of", "assets"])).
⚠️ As the name implies, aws-cdk.aws-lambda-python-alpha is an alpha version, use at your own risks.
Deploy
☝️ Don’t forget to (re)activate the CDK virtual environment before running the following commands.
Now that everything is set up, it’s time to deploy the infrastructure: run cdk synth to build the CloudFormation files and bundle your project assets, and cdk deploy to deploy to AWS and build the infrastructure.
cdk synth will download a Docker image and install your project dependencies, which means you need to have Docker installed.
The output of cdk deploy will contain the URL of the new API Gateway.
Outputs:
EchoServiceStack.echoServiceecholambdaapigwEndpoint3B71407F = https://<api-gateway-ID>.execute-api.eu-central-1.amazonaws.com/prod/
Your Lambda is now live!
$ curl -X GET https://<api-gateway-ID>.execute-api.eu-central-1.amazonaws.com/prod/?foo=bar
foo=bar right back at you
If you connect to the AWS console, you can check that your code was bundled and deployed with its dependencies.
