Celery requires a broker to handle message-passing. We use RabbitMQ because it’s easy to setup and it is well supported.
Install rabbitmq using the following command
sudo apt-get install rabbitmq-server
Once the installation is complete, create user, add a virtual host and set permissions.
sudo rabbitmqctl add_user myuser mypassword sudo rabbitmqctl add_vhost myvhost sudo rabbitmqctl set_user_tags myuser mytag sudo rabbitmqctl set_permissions -p myvhost myuser ".*" ".*" ".*"
To start the server:
We can install celery with pip:
pip install celery
In your Django settings.py file, your broker URL would then look something like
BROKER_URL = 'amqp://myuser:mypassword@localhost:5672/myvhost'
Now start the celery worker
celery -A your_app worker -l info
This command start a Celery worker to run any tasks defined in your django app.
Supervisor is a Python program that allows you to control and keep running any unix processes. It can also restart crashed processes. We use it to make sure Celery workers are always running.
First, Install supervisor
sudo apt-get install supervisor
Create your_proj.conf file in your supervisor conf.d (/etc/supervisor/conf.d/your_proj.conf):
[program:your_proj_celery] command=/home/your_user/your_proj/.venv/bin/celery --app=your_proj.celery:app worker -l info directory=/home/your_user/your_proj numprocs=1 stdout_logfile=/home/your_user/your_proj/logs/celery-worker.log stderr_logfile=/home/your_user/your_proj/logs/low-worker.log autostart=true autorestart=true startsecs=10
Once our configuration file is created and saved, we can inform Supervisor of our new program through the supervisorctl command. First we tell Supervisor to look for any new or changed program configurations in the /etc/supervisor/conf.d directory with:
sudo supervisorctl reread
Followed by telling it to enact any changes with:
sudo supervisorctl update
Once our programs are running, there will undoubtedly be a time when we want to stop, restart, or see their status.
sudo supervisorctl status
For restart your celery instance:
sudo supervisorctl restart your_proj_celery