Unfloyded/Legamunt
0
0
Fork 0
Code Activity

Add a spam check (#11217)

Browse source 
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
This commit is contained in:
Eugen Rochko 2019-07-13 16:45:50 +02:00 • committed by GitHub
parent 402302776c
commit 6ff67be0f6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 377 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

1
db/schema.rb
View file

@ -148,6 +148,7 @@ ActiveRecord::Schema.define(version: 2019_07_06_233204) do
t.string "also_known_as", array: true
t.datetime "silenced_at"
t.datetime "suspended_at"
t.integer "trust_level"
t.index "(((setweight(to_tsvector('simple'::regconfig, (display_name)::text), 'A'::\"char\") || setweight(to_tsvector('simple'::regconfig, (username)::text), 'B'::\"char\")) || setweight(to_tsvector('simple'::regconfig, (COALESCE(domain, ''::character varying))::text), 'C'::\"char\")))", name: "search_index", using: :gin
t.index "lower((username)::text), lower((domain)::text)", name: "index_accounts_on_username_and_domain_lower", unique: true
t.index ["moved_to_account_id"], name: "index_accounts_on_moved_to_account_id"