0
0
Fork 0

Clean up reblog tracking keys, related improvements (#5428)

* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
This commit is contained in:
aschmitz 2017-10-17 04:45:06 -05:00 committed by Eugen Rochko
parent a2b600428c
commit 554c2fd8af
4 changed files with 119 additions and 34 deletions

View file

@ -5,18 +5,36 @@ class Scheduler::FeedCleanupScheduler
include Sidekiq::Worker
def perform
reblogged_id_sets = {}
feedmanager = FeedManager.instance
redis.pipelined do
inactive_users.each do |account_id|
redis.del(FeedManager.instance.key(:home, account_id))
redis.del(FeedManager.instance.key(:home, account_id, 'reblogs'))
inactive_user_ids.each do |account_id|
redis.del(feedmanager.key(:home, account_id))
reblog_key = feedmanager.key(:home, account_id, 'reblogs')
# We collect a future for this: we don't block while getting
# it, but we can iterate over it later.
reblogged_id_sets[account_id] = redis.zrange(reblog_key, 0, -1)
redis.del(reblog_key)
end
end
# Remove all of the reblog tracking keys we just removed the
# references to.
redis.pipelined do
reblogged_id_sets.each do |account_id, future|
future.value.each do |reblogged_id|
reblog_set_key = feedmanager.key(:home, account_id, "reblogs:#{reblogged_id}")
redis.del(reblog_set_key)
end
end
end
end
private
def inactive_users
@inactive_users ||= User.confirmed.inactive.pluck(:account_id)
def inactive_user_ids
@inactive_user_ids ||= User.confirmed.inactive.pluck(:account_id)
end
def redis