Pandas in AWS lambda gives numpy error

Kingz picture Kingz · May 9, 2017 · Viewed 20.6k times · Source

I've been trying to run my code in AWS Lambda which imports pandas. So here is what I've done. I have a python file which contains a simple code as follows(This file has the lambda handler)

import json
print('Loading function')
import pandas as pd
def lambda_handler(event, context):
    return "Welcome to Pandas usage in AWS Lambda"
  1. I have zipped this python file along with numpy, pandas and pytz libraries as a deployment package (Did all these in Amazon EC2 linux machine)
  2. Then uploaded the package into S3
  3. Created a lambda function(runtime=python3.6) and uploaded the deployment package from S3

But when I test the lambda function in AWS Lambda, I get the below error:

Unable to import module 'lambda_function': Missing required dependencies ['numpy']

I already have numpy in the zipped package but still I get this error. I tried to follow the hints given at Pandas & AWS Lambda but no luck.

Did anyone ran into the same issue. Would appreciate any hint or suggestions to solve this problem.

Thanks

Answer

pbegle picture pbegle · Aug 22, 2017

EDIT: I figured out finally how to run pandas & numpy in a AWS Lambda python 3.6 runtime environment.

I have uploaded my deployment package to the following repo:

git clone https://github.com/pbegle/aws-lambda-py3.6-pandas-numpy.git

Simply add your lambda_function.py to the zip file by running:

zip -ur lambda.zip lambda_function.py

Upload to S3 and source to lambda.

ORIGINAL:

The only way I have gotten Pandas to work in a lambda function is by compiling the pandas (and numpy) libraries in an AWS Linux EC2 instance following the steps from this blog post and then using the python 2.7 runtime for my lambda function.