Pass and use input (parameters) to a lambda task from a step function

Question

I have a simple step function launching a lambda and I am looking for a way to pass parameters (event / context) to each of several consequent tasks. My step function looks like this:

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Task",
      "Parameters": {
        "TableName": "table_example"
      },
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "End": true
    }
  }
}

In the lambda written with Python I am using a simple handler which is:

def lambda_handler(event, context):
    #...

The event and context look like this (checking the logs):

START RequestId: f58140b8-9f04-47d7-9285-510b0357b4c2 Version: $LATEST

I cannot find a way to pass parameters to this lambda and to use them in the script. Essentially, what I am trying to do is to run the same lambda passing a few different values as a parameter.

Could anyone please point me in the right direction?

The line from the log you posted is just the "standard mark" when a Lambda starts, not the `event` or `context` objects. Your step function looks fine. Check the content of the `event` parameter (you can just `print(event)` and check the logs), you should see the `TableName` present there. — Milan Cermak, Jan 25 '19 at 13:57

score 11 · Answer 1 · edited Jul 27 '20 at 09:51

Based on what you said: "looking for a a way to pass parameters (event / context) to each of several consequent tasks" I assumed that you want to pass non-static values to lambdas.

There are two ways to pass arguments through state machine. Via InputPath and Parameters. For differences please look here.

If you do not have any static values that you want to pass to lambda, I would do the following. Passed all parameters to step function in json format.

Input JSON for state machine

{
    "foo": 123,
    "bar": ["a", "b", "c"],
    "car": {
        "cdr": true
    }
    "TableName": "table_example"
}

In step function you are passing entire JSON explicitly to lambda using "InputPath": "$", except for a first step where it is passed implicitly. For more about $ path syntax please look here. You also need to take care of task result, with one of multiple approaches using ResultPath. For most of cases the safest solution is to keep task result in special variable "ResultPath": "$.taskresult"

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloWorld",
  "States": {
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "Next": "HelloWorld2"
    },
    "HelloWorld2": {
      "Type": "Task",
      "InputPath": "$",
      "ResultPath": "$.taskresult"
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync_2",
      "End": true
    }
  }
}

Which in lambda became event variable and can be access as python dictionary

def lambda_handler(event, context):
    table_example = event["TableName"]
    a = event["bar"][0]
    cdr_value = event["car"]["cdr"]
    # taskresult will not exist as event key 
    # only on lambda triggered by first state
    # in the rest of subsequent states
    # it will hold a task result of last executed state
    taskresult = event["taskresult"]

With this approach you can use multiple step functions and different lambdas and still keep both of them clean and small by moving all the logic in lambdas. Also it is easier to debug because all events variables will be the same in all lambdas, so via simple print(event) you can see all parameters needed for entire state machine and what possibly went wrong.

Hi thanks for the explanation, I still a bit confused, this is my question: https://stackoverflow.com/questions/66463859/error-when-invoking-lambda-from-stepfunction-with-paypload Could you take a look please? Many thanks. — wawawa, Mar 03 '21 at 19:55
The confusing part is that how we pass the JSON payload to stepfunction? it's not showing anywhere in the state machine definition... — wawawa, Mar 03 '21 at 19:57
Yeah, he really neglected to mention it, but the JSON input to helloworld2 needs to be output from helloworld1. — Chris Ivan, Mar 15 '21 at 03:07

Farid Nouri Neshat · Answer 2 · 2020-01-09T07:55:51.827

I came across this, apparently, when Resource is set to lambda ARN(for example "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync"), you can't use the Parameters to specify the input and instead the state of the step function is passed(possibly the input to your state function, if there's no step before it).

To pass the function input via Parameters you can specify Resource as "arn:aws:states:::lambda:invoke" and the provide your FunctionName in the Parameters section:

{
    "StartAt": "HelloWorld",
    "States": {
        "HelloWorld": {
            "Type": "Task",
            "Resource": "arn:aws:states:::lambda:invoke",
            "Parameters": {
                "FunctionName": "YOUR_FUNCTION_NAME",
                "Payload": {
                    "SOMEPAYLOAD": "YOUR PAYLOAD"
                }
            },
            "End": true
        }
    }
}

You can find the documentation for invoking Lambda functions in here: https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html

You can also potentially use inputPath, or also have elements from your step function state function: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html

Even when this solution comes from the documentation, I have tested it and doesn't work. Or at least the console doesn't show me the parameters were passed in, and the output of the step comes with additional wrapper of the response data. — yucer, Oct 01 '21 at 11:48
I found the opposite to be true: `resource` MUST be set to the lambda arn in order to use `parameters` for other key-value pairs. — Excel Developers, Jul 14 '22 at 10:33

yucer · Answer 3 · 2021-10-01T12:15:04.167

For some reason, specifying directly the lambda function ARN in Resource doesn't work.

The following workaround is purely with ASL definition, you just create a Pass step before with the parameters, and its output will be used as input of the next step (your step HelloWorld with the lambda call):

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloParams",
  "States": {
    "HelloParams": {
      "Type": "Pass",
      "Parameters": {
        "TableName": "table_example"
      },
      "Next": "HelloWorld"
    },
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "End": true
    }
  }
}

there is a workaround in another response that tells you to make the previous step using a lambda function, but it is not needed for simple cases. Context values can also be mapped, for example the current timestamp:

"HelloParams": {
  "Type": "Pass",
  "Parameters": {
    "TableName": "table_example",
    "Now": "$$.State.EnteredTime"
  },
  "Next": "HelloWorld"
},

Also, InputPath and ResultPath can be used to prevent overwriting the values from previous steps. For example:

{
  "Comment": "A Hello World example of the Amazon States Language using an AWS Lambda function",
  "StartAt": "HelloParams",
  "States": {
    "HelloParams": {
      "Type": "Pass",
      "Parameters": {
        "TableName": "table_example"
      },
      "ResultPath": "$.hello_prms",
      "Next": "HelloWorld"
    },
    "HelloWorld": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:ap-southeast-2:XXXXXXX:function:fields_sync",
      "InputPath": "$.hello_prms",
      "ResultPath": "$.hello_result",
      "End": true
    }
  }
}

that would save the parameters in hello_prms (so that you can reuse them in other steps) and save the result of the execution in hello_result without values from previous steps (in case you add them).

score -1 · Answer 4 · answered Jan 07 '20 at 10:01

-1

Like Milan mentioned in his comment, you can pass on data to a Lambda function from a Step Function State.

In the Lambda function you'll need to read the event contents.

import json

def lambda_handler(event, context):
    TableName = event['TableName']

answered Jan 07 '20 at 10:01

Bart Schuijt

591
1
10
14

Hey, it seems like you downvoted my answer. I would appreciate if you could comment on which part is wrong or how it's not useful. With the step function that poster has shown no parameters are passed to lambda, so I was answering how to actually pass parameters to a lambda in a step function. – Farid Nouri Neshat Jan 08 '20 at 08:42
1

Hi Farid, sure, OP's step function logic is correct, it's a matter of reading the `event` var in the lambda_handler. Your answer did not work for me; was not able to invoke Lambda's this way (the exact error message has escaped me). – Bart Schuijt Jan 08 '20 at 20:22

Pass and use input (parameters) to a lambda task from a step function

4 Answers4