芹菜——需要优先运行的任务

In my website users can UPDATE they profile (manual) every time he want, or automatic once a day.

在我的网站上，用户可以随时更新他们的个人资料(手册)，或者每天自动更新一次。

This task is being distributed with celery now.

这个任务现在正用芹菜分发。

But i have a "problem" :

但我有个“问题”:

Every day, in automatic update, a job put ALL users (+-6k users) on queue:

每天，在自动更新中，一个任务将所有用户(+-6k用户)放在队列中:

from celery import group
from tasks import *
import datetime
from lastActivityDate.models import UserActivity

today   = datetime.datetime.today()
one_day = datetime.timedelta(days=5)
today -= one_day

print datetime.datetime.today()

user_list = UserActivity.objects.filter(last_activity_date__gte=today)
g = group(update_user_profile.s(i.user.auth.username) for i in user_list)

print datetime.datetime.today()
print g(user_list.count()).get()

If someone try to do the manual update, they will enter on te queue and last forever to be executed.

如果有人尝试手动更新，他们将进入te队列并一直执行下去。

Is there a way to set this manual task to run in a piority way? Or make a dedicated for each separated queue: manual and automatic?

是否有一种方法可以将这个手动任务设置为以优先级方式运行?或者为每一个分开的队列做一个专用的:手动的和自动的?

1 个解决方案

#1

Celery does not support task priority. (v3.0)

芹菜不支持任务优先级。(v3.0)

http://docs.celeryproject.org/en/master/faq.html#does-celery-support-task-priorities

http://docs.celeryproject.org/en/master/faq.html does-celery-support-task-priorities

You may solve this problem by routing tasks.

您可以通过路由任务来解决这个问题。

http://docs.celeryproject.org/en/latest/userguide/routing.html

Prepare default and priority_high Queue.

准备默认和priority_high队列。

CELERY_DEFAULT_QUEUE = 'default'
CELERY_QUEUES = (
    Queue('default'),
    Queue('priority_high'),
)

Run two daemon.

两个守护进程运行。

user@x:/$ celery worker -Q priority_high
user@y:/$ celery worker -Q default,priority_high

And route task.

和路线的任务。

your_task.apply_async(args=['...'], queue='priority_high')

#1