0

After searching hi and lo for the best way to run a cron job when deploying an app using on Amazon Elastic Beanstalk with auto-scaling enabled, I came across a few solutions.

Problem: the problem with running cron on beanstalk (or auto-scaling cloud) is that every instance will have it's own cron daemon and the script will run many copies simultaneously (on each EC2 instance) when it auto-scales.

Solutions offered:

  1. Use a worker instance + SQS
    • Problem: it's too complicated and I don't want to be billed for an additional instance for one simple cron job like sending an alert to the user.
  2. Use cronlock
    • Problem: requires you to install a central redis sever.

My own solution:

My site already has a Mysql server (Amazon RDS) and so I was thinking that maybe this can work?

  1. I can create a table with just one field last_run.
  2. Create the following script and add it to cron daemon instead:
    1. LOCK TABLES crond WRITE
    2. Read the value in crond.last_run
    3. If time() - last_run > 5 (i.e. 5 seconds have elapsed) then I update the time in the last_run field to current time.
    4. If time() - last_run < 5 then I exit immediately saying, cron was executed less than 5 seconds ago (i.e. some other instance got the LOCK first)
    5. UNLOCK TABLES.
    6. Spawn the actual cron job script(s) if last_run > 5 (i.e. #3 is true)

Questions:

  • Will this be sufficient to ensure that only one copy of cron runs at a time?
  • Are there any easier alternatives to what I'm trying to achieve?

Your insights will be helpful before I design and test this thing.

supersan
  • 5,671
  • 3
  • 45
  • 64
  • Have you thought about using a scheduled Lambda function instead of cron? That would certainly be easier than your proposed solution, and much cheaper than a dedicated cron instance. – Mark B Nov 14 '15 at 21:33
  • hi, no i didn't know about it. Thanks for the tip, i will look for more info about it – supersan Nov 15 '15 at 06:07

1 Answers1

1

Elastic Benastalk contains a mechanism just for that. It's called leader_only and ensures that your cron will run on a single instance rather than all of them. See documentation here.

Here's a stackoverflow thread that tackles just that: AWS Elastic Beanstalk, running a cronjob

If that doesn't work for you, give Cronally a chance.

Community
  • 1
  • 1
Tal
  • 7,827
  • 6
  • 38
  • 61
  • 1
    Hi, yes I read about it but the problem is the `leader` can be killed after auto-scaling scales down. So it is possible that the cron job will stop working if it is set to leader only :/ (it's mentioned in the update to first answer) – supersan Nov 14 '15 at 18:42
  • You are correct. In that case, I guess Cronally is the only "simple" option: http://blog.turret.io/run-cron-jobs-on-aws-with-cronally/ – Tal Nov 15 '15 at 10:33