Running Cron Jobs with AWS

Cron jobs are scheduled commands that run periodically in the background that can be used to execute your programs automatically at set intervals. This may be to scrape data, update a database or perform cleanup operations.

Cron jobs can be easily set up your Linux system by adding a new cron config into the crontab file. Of course, this would only work while your device is running. To have your cron job working 24/7, you could move it to a Linode Linux instance, although this would be expensive, especially if your program only runs for a short time.

A better solution is to turn your program into a serverless function that lives on the cloud. These functions can be triggered remotely or through scheduled within your cloud service, and you only pay for the total running time used. For a short program executed every hour, this will close to free.

1. Open Lambda page

Once you have created an AWS account and signed in, search through the Compute section to find Lambda. Here we can create serverless Lambda functions that are reusable functions that live in the clouds and can be accessed remotely.

2. Create a Lambda Function

We can create a new Lambda function and upload a zip file containing the code.

Note: With compiled languages like Go, we upload the executable instead of the code, compiled for Linux (by running set GOOS=linux).

In runtime settings, ensure the handler exactly matches the name of your target file to be executed (e.g. main).

You may need to modify your code to use the handler function from the AWS Lambda library. Similarly to the role of a 'main' function, the handler function is invoked when your Lambda function is triggered. Handler functions vary depending on your chosen language. For further guidance, check our the AWS Lambda function docs.

3. Set up a cron job with EventBridge

Finally, we can create a trigger event to start the execution of the Lambda function at scheduled times. This can be done with AWS EventBridge (previously CloudWatch).

Navigate to the EventBridge page in AWS and go to Events > Rules > Create Rule. Here we can create a name for our rule. At the bottom, we select schedule. We can define when our cron job runs using either:

1. a chron expression; or
2. a rate expression

Cron expressions can be much more specific whereas rate expressions simply specify the rate at which to execute the job, starting from the moment it's set. Cron expressions can include exact times for the job to run and can specify irregular intervals, which can be useful.

Cron Expressions

A cron expression is split into 7 values.

<minute> <hour> <day-of-month> <month> <day-of-week> <year>

In each of these positions, we can specify a specific integer value. We can also enter asterisk (*) to represent all, e.g. * in the hour field means run every hour. A question mark can represent any in <day-of-month> and <day-of-week> to accept any value. A comma can be used to add multiple numbers to a single field.

Run every second starting from now.
cron(* * * * ? *)

Run every hour on the hour (e.g. 21:00, 22:00, 23:00, 00:00...).
cron(0 * * * ? *)

Run every hour at 10 minutes past the hour (e.g. 21:10, 22:10, 23:10, 00:10...).
cron(10 * * * ? *)

Run every 30 minutes on the hour and at 30 past (e.g. 21:00, 21:30, 22:00, 22:30...).
cron(0,30 * * * ? *)

Run every 15 minutes on the hour and at 15, 30 and 45 past (e.g. 21:00, 21:15, 22:30, 22:45...).
cron(0,15,30,45 * * * ? *)

Run every 5 minutes, when the minutes are divisible by 5 (e.g. 21:05, 21:10, 22:15, 22:20...).
cron(*/5 * * * ? *)

Run every day at 1:30.
cron(30 1 * * ? *)

Run every second on the 3rd of February.
cron(* * 3 2 ? *)