[ERROR] Runtime.ImportModuleError: Unable to import module

[debojit777]

sam issue

Hello SAM - LAMBDA users,

On running :

sam local start-api --env-vars test/sample/integration_test/environment.json

and

curl http://127.0.0.1:3000/sample

Getting below error

START RequestId: 156a4eec-0ec1-182f-d06e-04bfce652b1f Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'list_handler': No module named 'src'
END RequestId: 156a4eec-0ec1-182f-d06e-04bfce652b1f
REPORT RequestId: 156a4eec-0ec1-182f-d06e-04bfce652b1f  Init Duration: 537.49 ms    Duration: 4.27 ms   Billed Duration: 100 ms Memory Size: 1024 MB    Max Memory Used: 32 MB  
Lambda returned empty body!
Invalid API Gateway Response Keys: {'errorType', 'errorMessage'} in {'errorType': 'Runtime.ImportModuleError', 'errorMessage': "Unable to import module 'list_handler': No module named 'src'"}
<class 'samcli.local.apigw.local_apigw_service.LambdaResponseParseException'>

This is my project structure

.
├── README.md
├── api-event.json
├── environment.json
├── events
│   └── event.json
├── src
│   └── sample
│       ├── handler
│       │   ├── create_handler.py
│       │   ├── get_handler.py
│       │   ├── list_handler.py
│       │   ├── requirements.txt
│       └── helper
│           ├── dynamodb_helper.py
│           └── logging_helper.py
├── template.yaml
└── test
    └── sample
        ├── integration_test
        │   ├── api_tests.py
        │   └── environment.json
        ├── test_helper.py
        └── unit_test

This is my template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sample-sam-app

  SAM Template for sample-sam-app

Globals:
  Function:
    Timeout: 20
    MemorySize: 1024
    Environment:
      Variables:
        TABLE_NAME: !Ref SampleTable
        APPLICATION_NAME: "SAMPLE"

Resources:
  CreateFunction:
    Type: AWS::Serverless::Function 
    Properties:
      CodeUri: src/sample/handler
      Handler: create_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref SampleTable
      Events:
        Create:
          Type: Api 
          Properties:
            Path: /sample
            Method: post

  GetFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/sample/handler
      Handler: get_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref SampleTable
      Events:
        Get:
          Type: Api 
          Properties:
            Path: /sample/{data_id}
            Method: get

  ListFunction:
    Type: AWS::Serverless::Function 
    Properties:
      CodeUri: src/sample/handler
      Handler: list_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref SampleTable
      Events:
        Get:
          Type: Api 
          Properties:
            Path: /sample
            Method: get

  SampleTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: sample
      AttributeDefinitions:
        - AttributeName: user_id
          AttributeType: S
        - AttributeName: data_id
          AttributeType: S
      KeySchema:
        - AttributeName: user_id
          KeyType: HASH
        - AttributeName: data_id
          KeyType: RANGE
      BillingMode: PAY_PER_REQUEST 

Outputs:
    CreateFunction:
      Description: "Create Lambda Function ARN"
      Value: !GetAtt CreateFunction.Arn

    GetFunction:
      Description: "Get Lambda Function ARN"
      Value: !GetAtt GetFunction.Arn

    ListFunction:
      Description: "List Lambda Function ARN"
      Value: !GetAtt ListFunction.Arn

Also my docker doesnt show sam-app

docker ps

CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                    NAMES
2e45a561d2ab        amazon/dynamodb-local   "java -jar DynamoDBL…"   19 minutes ago      Up 19 minutes       0.0.0.0:8000->8000/tcp   romantic_mahavira

1. OS: MAC
2. sam --version: SAM CLI, version 0.48.0

[jfuss]

SAM CLI will mount your CodeUri into the container. So the file structure on your host will not match what is in the container. In your case: files within src/sample/handler will live within /var/task on the container, which means src is not a valid module in lambda. Which gives you the error you are seeing. You will need to reconfigure your imports and your CodeUri to allow importing in Lambda's filesystems not your host.

[sriram-mv]

Closing this issue, since @jfuss has responded with the next steps.

[debojit777]

Fixed by :

• restructuring the project as below

├── README.md
├── __init__.py
├── create_handler.py
├── delete_handler.py
├── dynamodb_helper.py
├── environment.json
├── events
│   ├── create_event.json
│   └── get_event.json
├── get_handler.py
├── list_handler.py
├── logging_helper.py
├── packaged.yaml
├── requirements.txt
├── template.yaml
└── update_handler.py

• modifying the template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sample-app

  SAM Template for  service

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 20
    MemorySize: 1024
    Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
      Variables:
        TABLE_NAME: ""
        DDB_ENDPOINT_OVERRIDE: ""
        INTEGRATION_TESTS_ENDPOINT: ""
        APPLICATION_NAME: ""

Resources:
  CreateQuestionFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: .
      Handler: create_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref QuestionsTable
      Events:
        CreateQuestion:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /sample
            Method: post

  GetQuestionFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: .
      Handler: get_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref QuestionsTable
      Events:
        GetQuestion:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /sample/{sample_id}
            Method: get

  ListQuestionFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: .
      Handler: list_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref QuestionsTable
      Events:
        GetQuestions:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /sample
            Method: get

  UpdateQuestionFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: .
      Handler: update_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref QuestionsTable
      Events:
        UpdateQuestion:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /sample/{sample_id}
            Method: post

  DeleteQuestionFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: .
      Handler: delete_handler.lambda_handler
      Runtime: python3.7
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref QuestionsTable
      Events:
        DeleteQuestion:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /sample/{sample_id}
            Method: delete

  QuestionsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: sample
      AttributeDefinitions:
        - AttributeName: user_id
          AttributeType: S
        - AttributeName: _id
          AttributeType: S
      KeySchema:
        - AttributeName: user_id
          KeyType: HASH
        - AttributeName: topic_id
          KeyType: RANGE
      BillingMode: PAY_PER_REQUEST 

Outputs:
    CreateQuestionFunction:
      Description: "CreateQuestion Lambda Function ARN"
      Value: !GetAtt CreateQuestionFunction.Arn

    GetQuestionFunction:
      Description: "GetQuestion Lambda Function ARN"
      Value: !GetAtt GetQuestionFunction.Arn

    ListQuestionFunction:
      Description: "GetQuestions Lambda Function ARN"
      Value: !GetAtt ListQuestionFunction.Arn

    UpdateQuestionFunction:
      Description: "UpdateQuestion Lambda Function ARN"
      Value: !GetAtt UpdateQuestionFunction.Arn

    DeleteQuestionFunction:
      Description: "DeleteQuestion Lambda Function ARN"
      Value: !GetAtt DeleteQuestionFunction.Arn

• also keep in mind to make all environment variables empty in the template.yaml needed for local testing

don't like the fix for :

• not being able to create packages.
• making environment variables (needed for local testing) empty in template.yaml feels hacky

[rr83019]

+1 for not being able to create packages.

[jsundy]

you don't need to add all your lambdas into one directory. See aws-samples/aws-serverless-samfarm#5 (comment)

[austinben]

As mentioned in a issue I submitted to aws/aws-cli#6615, I am struggling to configure the docker build that occurs when running the aws stepfunction start-execution command, which I believe invokes a docker build in the other terminal running sam local start-lambda as per https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-lambda.html .

I am not sure if the restructuring of the AWS SAM template will solve the issue of needing to install many dependencies in the container being used. This was complicated enough when doing local testing using sam build and sam local invoke, so I fear it may not work when trying to run a step function composed of multiple lambdas locally using these tools.

If anyone could provide insight into how to set up local testing of stepfunctions with multiple lambda functions with various modules and dependencies? The current structure that is used for local functional testing with sam build sam local invoke is as follows:

├── index.py
├──modules1
│   ├── some_module.py
│   └── other_module.py
├──modules2
│   ├── some_module2.py
│   └── other_module2.py
├── requirements.txt
├── template.yaml
├── Dockerfile.yaml

template.yaml :

Parameters:
  EnvironmentName:
    Type: String
    Description: Environment name to prefix on resource names in this stack
  Memory:
    Type: Number
    Description: Memory (MB) allocated the lambda. This directly affects the CPU allocated
    Default: 512
  Timeout:
    Type: Number
    Description: Timeout (seconds)
    Default: 300

Resources:
  MyLambdaFunction:
    Type: AWS::Serverless::Function
    Metadata:
      DockerTag: python3.8-v1
      DockerContext: ../../../
      Dockerfile: ./lambdas/path/to/Dockerfile
    Properties:
      Description: My Lambda Function
      PackageType: Image
      Environment:
        Variables:
          AWS_ENVIRONMENT_NAME:
            Ref: EnvironmentName
      MemorySize:
        Ref: Memory
      Timeout:
        Ref: Timeout

Dockerfile :

FROM amazon/aws-lambda-python:3.8

COPY lambdas/path/to/index.py lambdas/path/to/requirements.txt ./
ADD ./modules/somemodules ./somemodules
ADD ./modules/somemodules2 ./somemodules2
RUN python3.8 -m pip install -r requirements.txt -t .

CMD ["index.lambda_handler"]

This works correctly just using sam build and sam local invoke -e for functional testing, (and for debugging in VSCode after creating a launch.json.

I have multiple of these Lambda functions, each composing a portion of a step function defined in a .json template, along with everything else in a cloudformation .json template.

Is it possible to use the aws stepfunctions local tool with multiple lambdas defined in this way? Currently, it seems to just build the image from some default settings, and I cant seem to configure it as I have with the individual tests.

If this is too off-topic for this issue I am happy to open a new issue, I was directed here as the build process seems to be from the SAM CLI but the stepfunctions portion of the workflow if from the AWS CLI, please advise! Thanks :)

[itaybaror]

I now realize that there is a conflict between my utils in my other layer and the utils in this repository, any suggestions for a fix?