Database

Celery Periodic Tasks: From Installation to Infinity

This is a quick example of how I got Periodic Tasks to work using Celery without Django.


Filed under:

This is a quick example of how I got Periodic Tasks to work using Celery without Django. There are lots of examples out there for creating tasks and loose documentation on how to start Celery and Celery Beat, but most of them involve Django. I just wanted to run a simple example and I spent way too long trying to fill in the gaps to get even this simple task to run periodically. So hopefully this quick example will help somebody else out there save some time.

Pre-requisites

Installing Celery

sudo pip install Celery
You'll want to choose your broker; RabbitMQ is what comes recommended by default.

Create a celeryconfig.py

Once you install your broker of choice you can setup the configuration options in celeryconfig.py. In my case I was using MySQL. Here is an example of what my file looked like.

# Change this to your settings
BROKER_URL = 'sqla+mysql://user: password@localhost/database'
CELERY_RESULT_BACKEND = "database"
CELERY_RESULT_DBURI = 'mysql://user: password@localhost/database'

Create some tasks

Create your tasks (generally they will be saved into a file called tasks.py). This is what my tasks.py looks like:

from celery import Celery

celery = Celery('tasks')
celery.config_from_object('celeryconfig')

@celery.task
def add(x, y):
return x + y

Update your celeryconfig.py

Now in order for Celery to know how often you want to run your periodic tasks, update your celeryconfig.py to add the following.

from celery.schedules import crontab

CELERYBEAT_SCHEDULE = {
'every-minute': {
'task': 'tasks.add',
'schedule': crontab(minute='*/1'),
'args': (1,2),
},
}

Start Celery

Now we just need to start a Celery worker with the --beat flag and our task will run every minute. This part was not so obvious, but the --beat flag needs to appear after worker, otherwise nothing will happen.

$ celery -A tasks worker --loglevel=info --beat

Wait a minute...

You should see Celery output a bunch of info as tasks are ran. Among them you should see a line that looks like this:

[2012-12-07 13:26:00,343: INFO/MainProcess] Task tasks.add[44477955-906c-4827-bda2-f6f19822c652] succeeded in 0.00760388374329s: 3

Similar posts

Get notified on new marketing insights

Be the first to know about new B2B SaaS Marketing insights to build or refine your marketing function with the tools and knowledge of today’s industry.