Minimizing the size of lambdas by avoiding having boto3 and its stubs in the deployment package

If you are having problems with AWS lambdas exceeding the 250MB limit, here is a tip that might help.

First of all, it is good to know that, unless you need a specific version, you do not need to include boto3 in your deployments because it is already provided with the runtime environment.

When you know that boto3 uses ~1.3Mb and botocore, which comes with it, ~65Mb, it is major.

However, there are situations where boto3 is a dependency of one of your dependencies, and so it comes back into the picture. This could be the case if you are using mypy and boto stubs, and you need to explicitly type an object from boto, like in a function signature.

from mypy_boto3_dynamodb.service_resource import DynamoDBServiceResource

def do_stuff(dynamodb: DynamoDBServiceResource) -> None:
    ...

Since boto3-stubs does not come in the Lambda runtime environment, this code crashes if deployed without the said library. So you have to include it. But, since this library depends on boto3, you will end up with the whole boto3 in your deployment.

By modifying the code, there is a way around the problem.

try:
    from mypy_boto3_dynamodb.service_resource import DynamoDBServiceResource
except ModuleNotFoundError:
    DynamoDBServiceResource = "mypy_boto3_dynamodb.service_resource.DynamoDBServiceResource"  # type: ignore

def do_stuff(dynamodb: DynamoDBServiceResource) -> None:
    ...

Locally / in your development environment, boto3-stubs is installed and mypy will correctly identify DynamoDBServiceResource. In production, without the stubs, the variable becomes a string instead. As type annotations are ignored at runtime, it doesn’t change anything.

Note: the assignment on line 4 must be annotated with # type: ignore because mypy will correctly identify the type change of the DynamoDBServiceResource variable, which is a typing error.

With this modification, you can move the boto3-stubs library back to your development dependencies and regain ~66MB of space.

Edit 2022-08-30: If you have production dependencies that depend on boto3, and if you are using serverless, there is another solution that will work for you: Minimize the size of lambdas by deleting boto3 from the deployment package.

Edit 2022-11-16: Here is another way to write the code so that it works with stubs locally and ignores it in production.

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from mypy_boto3_dynamodb.service_resource import DynamoDBServiceResource

def do_stuff(dynamodb: "DynamoDBServiceResource") -> None:
    ...