1

Abstract

To use openCV on AWS Lambda, I complied a zip file with the openCV package and a python program and save it in AWS S3. After that, I execute it by indicating the URL of the program on S3. However, I got the error below.

Error Message

Error

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'cv2.cv2'

whole log

START RequestId: 58869885-9abd-4b18-8393-6d023d268637 Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'cv2.cv2'
END RequestId: 58869885-9abd-4b18-8393-6d023d268637
REPORT RequestId: 58869885-9abd-4b18-8393-6d023d268637  Duration: 1.59 ms   Billed Duration: 2 ms   Memory Size: 256 MB Max Memory Used: 66 MB  Init Duration: 294.98 ms

※ There is a python file called lambda_function.py in the zip file saved in S3. import cv2 is executed in the python file.

What I tried

I thought there is a trouble in the version of opencv or compatibility with other packages. I changed two points below, but it didn't work.

  • from import opencv-python -t . to pip install opencv-python==3.4.5.20 -t .
  • I created a virtual environment with python3 venv (vertual env name), and executed pip install command.

※ I can import cv2 and execute lambda_function.py in my computer.

zip file contents in S3

lambda-package.zip

  • lambda_function.py
  • bin
  • cv2
  • include
  • lib
  • numpy
  • numpy-1.20.1.dist-info
  • opencv_python-3.4.5.20.dist-info
  • pyvenv.cfg

my environment

  • venv Python：3.7.6
  • Lambda Python：3.7
  • OpenCV：3.4.5.20
  • Lambda Timeout：1min
  • Lambda Memory：256MB

Thank you for your kindness.

tm_mm
  • 11
  • 2
  • Have you checked this [answer](https://stackoverflow.com/questions/64016819/cant-use-opencv-python-in-aws-lambda/64019186#64019186)? – Marcin Feb 18 '21 at 11:42
  • Also since you have a lot of dependecies, maybe container lambda with 10GB of space would be easier to use and setup. – Marcin Feb 18 '21 at 11:44
  • @Marcin Thanks Marcin. I read the answer you sent. It needs the knowledge of Docker, doesn't it? I'm not familiar with it. Is there another way to solve this problem, or do I have to learn Docker? Thanks again about your advise of memory. I changed it as you said. – tm_mm Feb 18 '21 at 14:10
  • There is not much to learn, and if you want to work lambda in long-term, basics of docker are very useful for installing many other dependencies, not only opencv. – Marcin Feb 18 '21 at 22:27

2 Answers2

0

This Stack Overflow question may have the answers you need. It seems that the the person asking the question is having a similar problem to you.

One answer suggests this:

The problem is that your local numpy and pandas are compiled for the local machine's architecture. Since AWS Lambda uses custom Linux, they are probably not compatible. So if you want to use them, you have two choices: Compile dependencies on EC2 instance which uses the same Amazon Linux version as AWS Lambda and create a deployment package or use one of the precompiled packages from https://github.com/Miserlou/lambda-packages/tree/master/lambda_packages

What worked for the OP was to zip numpy and scipy precompiled packages from here.

Maybe this info will help.

Daniel
  • 275
  • 1
  • 9
  • Thanks for your quick answer, Daniel. I used the precompiled packages from the github link you sent, but it doesn't still work. Here's what I did. I unzipped python3.6-OpenCV-3.3.0.tar.gz I got from the github, and I added the main python file in the same directory. After that, I compiled them in a zip file, so there are lambda_function.py and cv2 (folder) in the zip file. Do you have some ideas what was wrong with that? – tm_mm Feb 18 '21 at 14:05
  • @tm_mm Try downloading the GitHub tar.gz file using `curl -O -L https://github.com/Miserlou/lambda-packages/blob/master/lambda_packages/OpenCV/python2.7-OpenCV-3.1.0.tar.gz`. Once downloaded, see if you can import the OpenCV module inside a Python script. – Daniel Feb 18 '21 at 14:24
  • I'm sorry, I cannot fix it. I have four questions. First, I cannot unzip the file I downloaded as you said. Second, OpenCV of your link is for python2.7. I'm thinking about using python3, which one should I choose. Third, what exactly do I have to compile in the zip file for S3 ? Finally, do I have to use docker or some virtual environment like venv? I'm sorry for asking you questions again and again. – tm_mm Feb 19 '21 at 10:17
  • @tm_mm Sorry, I copied the wrong link. I meant to put `curl -O -L https://github.com/Miserlou/lambda-packages/blob/master/lambda_packages/OpenCV/python3.6-OpenCV-3.3.0.tar.gz`. I'm not sure why you cannot unzip the file but as something else you could try, read the README.md file on the GitHub repository, and follow what it says. It mentions using something called Zappa, so check it out. I am not an expert on all this, I just found that StackOverflow question and thought the answers may help, but I am happy to help as much as I can. – Daniel Feb 19 '21 at 14:04
  • Okey, I see. Thank you very much for your help. I'll give it a try. – tm_mm Feb 19 '21 at 14:50
0

You could try installing cv2 on a machine running Amazon Linux 2 and then uploading that with your code. I would recommend creating a lamba layer with cv2 though, just so you don't have to include cv2 in your code directory.

Ozymandias
  • 839
  • 1
  • 8
  • 13