Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do we plan to have function to delete the queue. #54

Open
xmywang opened this issue Aug 15, 2014 · 2 comments
Open

Do we plan to have function to delete the queue. #54

xmywang opened this issue Aug 15, 2014 · 2 comments

Comments

@xmywang
Copy link

xmywang commented Aug 15, 2014

Hi!, We need dynamic to create the queue and delete the queue. Are we going to support this function? If not, any instruction that we can do it by our own lua script?

More precisely, we are trying to solve the problem that some jobs in the queue taking very long time and some jobs in the queue taking short time.
Here is the situation:
Our jobs are grouped in certain way that we do not want each group of jobs interacts each other. The group is dynamically created and can be gone forever. Currently, we are trying to dynamically associate each group to its own queue. However, the number of groups is not fixed. So if the groups are created and then gone forever, we have to delete the those queues to avoid leaking.

Thanks a lot!

@xmywang xmywang closed this as completed Aug 15, 2014
@xmywang xmywang reopened this Aug 15, 2014
@dlecocq
Copy link
Contributor

dlecocq commented Aug 18, 2014

This should be possible by 1) ensuring that all the jobs in that queue are completed or canceled and then 2) calling queue.forget on the queue to remove it from the list of known queues. This will avoid all leakage.

@xmywang
Copy link
Author

xmywang commented Aug 18, 2014

For our case, the deletion of queue is triggered by the user from our UI. We need cancel all jobs in the queue asap. So we write a small lua script:
In queue.lua, add this function:
{code}
-- List jobs and their depends ids
function QlessQueue.jobs(now, name, count)

local queue = Qless.queue(name)
-- These are the ids that we're going to return. We'll begin with any jobs
-- that have lost their locks
local jids = queue.work.peek(count)
table.extend(jids, queue.locks.peek(now, 0, count - #jids))
table.extend(jids, queue.scheduled.peek(now, 0, count - #jids))
table.extend(jids, queue.recurring.peek(now, 0, count - #jids))

local dependents = {}
for _, jid in ipairs(jids) do
dependents[jid] = redis.call(
'smembers', QlessJob.ns .. jid .. '-dependents') or {}
end
table.extend(jids, dependents)
return jids
end
{code}

In api.lua, we write this

{code}
QlessAPI['job.batch_cancel'] = function(now, queue, count)
local jids = QlessQueue.jobs(now, queue, count)
Qless.cancel(unpack(jids))
return #jids
end
{code}

In this way, our application can repeat calling job.batch_cancel to cancel all jobs in the queue. Then we can safely delete the queue.

Let's know if this is OK or not.

Thanks a lot!

Yan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants