1

I am havin an issue with importing a common function util file in my AWS lambda. It is a python file and the folder structure looks something like this

(functions folder)
   common_util.py
   (lambda 1 folder)
      lambda1
   (lambda 2 folder)
      lambda2

I need to access the common_util from both these lambdas. When I run my CDK project locally this is easy i use something like .. on the import statement to tell the file it is one directory up

from ..common_util import (...)

When I deploy to AWS as a lambda (I package all of the above) I need to specify the import without the .. because this is the root folder of the lambda

from common_util import(...)

I need an import statement or a solution that will work for both my CDK project and the lambda.

here is the CDK where the lambda is created

const noteIntegrationLambda = new Function(this as any,"my-lambda",
      {
        functionName:
          "my-lambda",
        runtime: StackConfiguration.PYTHON_VERSION,
        handler:
          "my_lambda.execute",
        timeout: Duration.seconds(15),
        code: Code.fromAsset("functions/"),
        role,
        layers: [dependencyLayer],
        environment: env,

        },
      }
    );
AnonymousAlias
  • 1,149
  • 2
  • 27
  • 68
  • Try to separate `common_utils.py` to different folder(ex. utils) inside functions folder and then `from functions_folder.utils.common_util import(...)` – Andrey Sherman Aug 12 '20 at 15:05
  • That won't work though because locally I would need to say `import functions.function_folder.common_util()` or `..functions_folder.common_util()` Whereas from the lambda i would be just saying `import functions_folder.common_util` – AnonymousAlias Aug 12 '20 at 15:08

2 Answers2

1

Lambda layers provide an ideal mechanism to include in solving this problem. As mentioned in https://medium.com/@manojf/sharing-code-among-lambdas-using-lambda-layers-ca097c8cd500,

Lambda layers allow us to share code among lambda functions. We just have to upload the layer once and reference it in any lambda function.

So consider deploying your common code via a layer. That said, to structure your code, I recommend you create a common package that you install locally using pip install, as outlined at Python how to share package between multiple projects. Then you put that package into a layer that both of your lambdas reference. That completely solves the problem of how to structure code when your local file structure is different than the lambda file structure.

Also consider these resources:

As a layer example, suppose you wanted to include a "common_utils" library for your lambdas to reference. To make a layer, you would need to create a directory structure that contains that code, then zip the entire directory. It may be as follows:

/python
    /common_utils
        __init__.py
        common_util.py
        ...

When zipped, the zip file must have the "python" folder and inside of that you put your code. If you do this and also install your common code as a package, you can import it in your local code and in your lambdas using the same import.

What I do is use pip install to install to a certain file location--the location that I then zip into a layer. For example, if I wanted to make a layer for the pymysql library I might do

pip install --target=c:\myLayers\python pymysql

That will install the library files into the location I specified, which makes it easy to know what to zip up (just create a zip that includes the "python" directory).

Shawn
  • 8,374
  • 5
  • 37
  • 60
  • I find the layers thing quite confusing to be honest, i have updated my question with the code where i create my lambda, is there anyway you could show me a sample of how that layer is implemented? – AnonymousAlias Aug 12 '20 at 15:38
  • I added an example, and https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path also has an example – Shawn Aug 12 '20 at 15:55
  • but this is me manually zipping a file, i want to do this through the CDK ? – AnonymousAlias Aug 12 '20 at 15:57
  • This is just an example of the layer structure, but you can still use the CDK. See https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-lambda.LayerVersion.html – Shawn Aug 12 '20 at 16:03
1

I know this question is old, but I ran into a similar issue. My solution was to detect if the current environment is local or lambda using the os package, and then import differently based on the environment (local or cloud). Will leave here as a reference.

if os.environ.get("AWS_EXECUTION_ENV") is not None:
    # For use in lambda function
    from package_a import class_a
else:
    # For local use
    from ...package_a import class_a

Credits to: How to check if Python app is running within AWS lambda function?

Juan Park
  • 21
  • 2