There are multiple times, an application needs to schedule some task to be done in the future. If you are using AWS as a cloud provider, you have a few options to schedule tasks to be done in the future.

Crontab or ‘at’ Command

The first option is using an EC2 server that has some crontab installed or running an ‘at’ command with the desired date. With a crontab, you can run .sh files and execute any command that you may need for your application periodically or even schedule a command execution in the future, using the ‘at’ command.

This jobs will run not only once, but each time.

In this image, we have a cron scheduled for a specific day in the year and then we have a line for a .sh file to check if the actual year is the year we want the job to run. So this will technically run only once on the specific date that we specify.

If we analyze just that, this is a great option. But what happens when we start thinking of having a cluster of servers and more complex infrastructure? Also if we need to schedule tasks from inside an application code when something happens, using crontab or ‘at’ command may be really difficult and messy.

In a few cases, this solution may still work but in other cases, this idea is no longer a correct solution.

DynamoDB TTL

Another possible solution is using DynamoDB to store rows in a table that has a TTL expiration time. And then we can configure a lambda function execution for when a row expires.

This gives us a great solution but the problem comes when we read the DynamoDB TTL documentation. If we read closely, we will find out that the lambda execution of an expiring row in a DynamoDB table, may execute up to 48 hours after the expiration time.

As we said, we want seconds precision for the scheduled task, so this idea is no longer a correct solution either.

AWS Step Functions

For those that don’t know what AWS Step Functions is, it’s a service that allows users to create a custom workflow with multiple steps, conditionals, inputs and outputs.

With AWS Step Function, we have the ability to create a Step Function that receives multiple parameters as input and then executes as we configure it.

Wait block A really important building block that we can use of AWS Step Function, is the Wait block, that given a number of seconds, this block will block the execution and wait until the amount of time indicated passes. There is only one limitation, the amount of seconds must be less than one year (31,536,000 seconds).

Lambda block After that period the execution continues. And the other important building block that is really important for this, is the AWS Lambda execution block. This block uses a lambda function, and an input to execute some code.

Using these two blocks, we will build our Step Function that waits for the needed time and then execute some code.

Step by step

1- Create Step Function

To start we need to create a Step Function, to do so, we will search on the AWS services search bar for Step Function, and open the service. Then on the left side, we have a sidebar that we need to open and then click on State machines. Once we are on the State machines page, we need to create a new one.

Then we will choose the default options on the create panel.

Once we click Next, we will have the Design Workflow tool.

2- Design our workflow

Our State Machine Workflow will be really simple because we will only have two states apart from the start and end states.

It will look like this:

If you want you can start building it on the visual workflow editor but in case you want, you can copy the following JSON definition, put it in a .json file and import it. The only thing you need to do is replace the AWS Lambda function ARN where the LAMBDA FUNCTION ARN appears on the JSON definition with your lambda function arn.

{
  "Comment": "A description of my state machine",
  "StartAt": "Wait",
  "States": {
    "Wait": {
      "Type": "Wait",
      "Next": "Lambda Invoke",
      "SecondsPath": "$.waitSeconds"
    },
    "Lambda Invoke": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "OutputPath": "$.Payload",
      "Parameters": {
        "Payload.$": "$",
        "FunctionName": "<LAMBDA FUNCTION ARN>:$LATEST"
      },
      "Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException"
          ],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2
        }
      ],
      "End": true
    }
  }
}

3- Understand our Workflow

If we take a look on the JSON definition, we will see two specific parts that are really important for the Workflow execution.

First the “$.waitSeconds” parameter that is a reference for the amount of seconds that will wailt the execution. This parameter will be passed by our code when we start the execution.

And then ”$” on the “Payload.$” parameter of the lambda function, this means that every parameter that we pass to the State Machine execution (including waitSeconds) will be sent to the Lambda function.

4- Lambda function

Once we have our State Machine, we need to set up our Lambda function to execute after the waiting period.

Here i will give an example of a Lambda function that executes after a State Machine waiting period.

const aws = require('aws-sdk');

exports.handler = (event, context, callback) => {
    // Here we have our parameters on the event object
    //For example: event.waitSeconds.
};

On this AWS Lambda function you can run the code that you need to be scheduled.

5- Run our State Machine

To run our State Machine we will use the AWS SDK, so you can use any language where the AWS SDK is available. Here i will post a Node.js example:

const input = {
  // This is the amount of seconds
  // that the execution will be scheduled
  waitSeconds: seconds,

  // Here you can pass other params
  // that will be received by the Lambda function
};

const params = {
  stateMachineArn: process.env.STATE_MACHINE_ARN,
  input: JSON.stringify(input),
};

const stepFunction = new AWS.StepFunctions();
stepFunction
  .startExecution(params)
  .promise()
  .then(() => {
    console.log(`Scheduled event to run in ${seconds} seconds`);
  })
  .catch((err) => {
    console.error(err);
  });

6- Configure IAM Permissions

For all of this infrastructure to work, we need to set up some IAM permissions.

First we need a user to execute the code of step 5, that user must have the “states:StartExecution” action on the State machine.

And in second place, the State Machine role, must have the “lambda:InvokeFunction” action on the Lambda function that we just created.

Conclusion

And that’s it! With this you can schedule lambda function executions with different amount of seconds and to be precisely executed. Just check out the AWS Step Functions and Lambda pricing to understand better how much this system could cost you when using it with more traffic.

Hope this is an useful resource for anyone building this kind of system. I’m open to answer questions.