Unfloyded/Legamunt
0
0
Fork 0
Code Activity

Add a spam check (#11217)

Browse source 
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
This commit is contained in:
Eugen Rochko 2019-07-13 16:45:50 +02:00 • committed by GitHub
parent 402302776c
commit 6ff67be0f6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 377 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

13
app/lib/activitypub/activity/create.rb
View file

@ -41,6 +41,7 @@ class ActivityPub::Activity::Create < ActivityPub::Activity
resolve_thread(@status)
fetch_replies(@status)
check_for_spam
distribute(@status)
forward_for_reply if @status.distributable?
end
@ -406,6 +407,18 @@ class ActivityPub::Activity::Create < ActivityPub::Activity
Account.local.where(username: local_usernames).exists?
end
def check_for_spam
spam_check = SpamCheck.new(@status)
return if spam_check.skip?
if spam_check.spam?
spam_check.flag!
else
spam_check.remember!
end
end
def forward_for_reply
return unless @json['signature'].present? && reply_to_local?
ActivityPub::RawDistributionWorker.perform_async(Oj.dump(@json), replied_to_status.account_id, [@account.preferred_inbox_url])