Revamp post filtering system (#18058)
* Add model for custom filter keywords * Use CustomFilterKeyword internally Does not change the API * Fix /filters/edit and /filters/new * Add migration tests * Remove whole_word column from custom_filters (covered by custom_filter_keywords) * Redesign /filters Instead of a list, present a card that displays more information and handles multiple keywords per filter. * Redesign /filters/new and /filters/edit to add and remove keywords This adds a new gem dependency: cocoon, as well as a npm dependency: cocoon-js-vanilla. Those are used to easily populate and remove form fields from the user interface when manipulating multiple keyword filters at once. * Add /api/v2/filters to edit filter with multiple keywords Entities: - `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context` `keywords` - `FilterKeyword`: `id`, `keyword`, `whole_word` API endpoits: - `GET /api/v2/filters` to list filters (including keywords) - `POST /api/v2/filters` to create a new filter `keywords_attributes` can also be passed to create keywords in one request - `GET /api/v2/filters/:id` to read a particular filter - `PUT /api/v2/filters/:id` to update a new filter `keywords_attributes` can also be passed to edit, delete or add keywords in one request - `DELETE /api/v2/filters/:id` to delete a particular filter - `GET /api/v2/filters/:id/keywords` to list keywords for a filter - `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a filter - `GET /api/v2/filter_keywords/:id` to read a particular keyword - `PUT /api/v2/filter_keywords/:id` to edit a particular keyword - `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword * Change from `irreversible` boolean to `action` enum * Remove irrelevent `irreversible_must_be_within_context` check * Fix /filters/new and /filters/edit with update for filter_action * Fix Rubocop/Codeclimate complaining about task names * Refactor FeedManager#phrase_filtered? This moves regexp building and filter caching to the `CustomFilter` class. This does not change the functional behavior yet, but this changes how the cache is built, doing per-custom_filter regexps so that filters can be matched independently, while still offering caching. * Perform server-side filtering and output result in REST API * Fix numerous filters_changed events being sent when editing multiple keywords at once * Add some tests * Use the new API in the WebUI - use client-side logic for filters we have fetched rules for. This is so that filter changes can be retroactively applied without reloading the UI. - use server-side logic for filters we haven't fetched rules for yet (e.g. network error, or initial timeline loading) * Minor optimizations and refactoring * Perform server-side filtering on the streaming server * Change the wording of filter action labels * Fix issues pointed out by linter * Change design of “Show anyway” link in accordence to review comments * Drop “irreversible” filtering behavior * Move /api/v2/filter_keywords to /api/v1/filters/keywords * Rename `filter_results` attribute to `filtered` * Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer * Fix systemChannelId value in streaming server * Simplify code by removing client-side filtering code The simplifcation comes at a cost though: filters aren't retroactively applied anymore.
This commit is contained in:
parent
5823ae70c4
commit
02851848e9
60 changed files with 1292 additions and 250 deletions
|
@ -352,7 +352,6 @@ class FeedManager
|
|||
def filter_from_home?(status, receiver_id, crutches)
|
||||
return false if receiver_id == status.account_id
|
||||
return true if status.reply? && (status.in_reply_to_id.nil? || status.in_reply_to_account_id.nil?)
|
||||
return true if phrase_filtered?(status, receiver_id, :home)
|
||||
|
||||
check_for_blocks = crutches[:active_mentions][status.id] || []
|
||||
check_for_blocks.concat([status.account_id])
|
||||
|
@ -388,7 +387,6 @@ class FeedManager
|
|||
# @return [Boolean]
|
||||
def filter_from_mentions?(status, receiver_id)
|
||||
return true if receiver_id == status.account_id
|
||||
return true if phrase_filtered?(status, receiver_id, :notifications)
|
||||
|
||||
# This filter is called from NotifyService, but already after the sender of
|
||||
# the notification has been checked for mute/block. Therefore, it's not
|
||||
|
@ -418,34 +416,6 @@ class FeedManager
|
|||
false
|
||||
end
|
||||
|
||||
# Check if the status hits a phrase filter
|
||||
# @param [Status] status
|
||||
# @param [Integer] receiver_id
|
||||
# @param [Symbol] context
|
||||
# @return [Boolean]
|
||||
def phrase_filtered?(status, receiver_id, context)
|
||||
active_filters = Rails.cache.fetch("filters:#{receiver_id}") { CustomFilter.where(account_id: receiver_id).active_irreversible.to_a }.to_a
|
||||
|
||||
active_filters.select! { |filter| filter.context.include?(context.to_s) && !filter.expired? }
|
||||
|
||||
active_filters.map! do |filter|
|
||||
if filter.whole_word
|
||||
sb = /\A[[:word:]]/.match?(filter.phrase) ? '\b' : ''
|
||||
eb = /[[:word:]]\z/.match?(filter.phrase) ? '\b' : ''
|
||||
|
||||
/(?mix:#{sb}#{Regexp.escape(filter.phrase)}#{eb})/
|
||||
else
|
||||
/#{Regexp.escape(filter.phrase)}/i
|
||||
end
|
||||
end
|
||||
|
||||
return false if active_filters.empty?
|
||||
|
||||
combined_regex = Regexp.union(active_filters)
|
||||
|
||||
combined_regex.match?(status.proper.searchable_text)
|
||||
end
|
||||
|
||||
# Adds a status to an account's feed, returning true if a status was
|
||||
# added, and false if it was not added to the feed. Note that this is
|
||||
# an internal helper: callers must call trim or push updates if
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue