* Fix Sidekiq warnings about JSON serialization
This occurs on every symbol argument we pass, and every symbol key in hashes,
because Sidekiq expects strings instead.
See https://github.com/mperham/sidekiq/pull/5071
We do not need to change how workers parse their arguments because this has
not changed and we were already converting to symbols adequately or using
`with_indifferent_access`.
* Set Sidekiq to raise on unsafe arguments in test mode
In order to more easily catch issues that would produce warnings in production
code.
Conflicts:
- `config/environments/production.rb`:
Upstream changed a header but we had different default headers.
Applied the same change, and also dropped HSTS headers redundant with
Rails'.
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
Upstream refactored how `Create` activities are handled and how values are
extracted from `Create`d objects. This conflicted with how glitch-soc
supported the `directMessage` flag to explicitly distinguish between
limited and direct messages.
Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Upstream largely refactored that file and changed some of the logic.
This conflicted with glitch-soc's handling of the direct timeline and
the options to allow replies and boosts in public feeds.
Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored to move mention-related ActivityPub deliveries to
`ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
to not send local-only toots to remote mentioned users.
Took upstream's version, as the check is not needed anymore, since it is
performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
Upstream added support for `update` toot events, while glitch-soc had
support for an extra timeline support, `direct`.
Ported upstream changes and extended them to the `direct` timeline.
Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
Added code to handle the `directMessage` flag and take it into account
to compute visibility.
- `app/lib/feed_manager.rb`:
Extended upstream's support of `update` toot events to glitch-soc's
`direct` timeline.
* Add support for editing for published statuses
* Fix references to stripped-out code
* Various fixes and improvements
* Further fixes and improvements
* Fix updates being potentially sent to unauthorized recipients
* Various fixes and improvements
* Fix wrong words in test
* Fix notifying accounts that were tagged but were not in the audience
* Fix mistake
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream introduced support for private pinned toots, but glitch-soc's query
was a bit different as it filtered out local-only toots.
Used upstream's query, while adding local-only filtering back.
- `app/controllers/activitypub/collections_controller.rb`:
Same thing with regards to local-only posts.
- `app/validators/status_pin_validator.rb`:
Not a real conflict, but the line below was different in glitch-soc due to
the configurable pinned toots limit.
Conflicts:
- `.env.production.sample`:
Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
Minor conflict due to glitch-soc's extra “enable_keybase” setting.
Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
Minor conflict due to glitch-soc's extra “enable_keybase” setting.
Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
Minor conflict due to an optimization that wasn't shared between
the two versions. Copied upstream's version.
Conflicts:
- `app/views/admin/tags/index.html.haml`:
Removed upstream while it had changes in glitch-soc to accomodate for the
theming system.
Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
* Add trending links
* Add overriding specific links trendability
* Add link type to preview cards and only trend articles
Change trends review notifications from being sent every 5 minutes to being sent every 2 hours
Change threshold from 5 unique accounts to 15 unique accounts
* Fix tests
* Add tests
* Fix some link previews being incorrectly generated from different prior links
PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.
This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
* Add support for fetching Create and Announce activities by URI
This should improve compatibility with ZAP and offer a way to fetch boosts,
which is currently not possible.
* Add tests
* Add followed_by? to account_interactions
* Add RemoveFromFollowersService
* Fix AccountBatch to use RemoveFromFollowersService
* Add remove from followers API
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
Conflicts:
- `streaming/index.js`:
Filtering code for streaming notifications has been refactored upstream, but
glitch-soc had similar code for local-only toots in the same places.
Ported upstream changes, but did not refactor local-only filtering.
Eliminate need to have custom notifications filtering logic in the
streaming API code by publishing notifications into a separate stream
and then simply using the multi-stream capability to subscribe to that
stream when necessary
Fix issue #16603 undefined method `serialize_payload' for Unsuspend Account Service error.
It seems that this service forgot to `include Payloadable` so that `serialize_payload` could not be found in this service.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream-updated dependency (redis) textually too
close to glitch-soc-only dependecy.
Updated redis gem like upstream did.
* Add account statuses cleanup policy model
* Record last inspected toot to delete to speed up successive calls to statuses_to_delete
* Add service to cleanup a given account's statuses within a budget
* Add worker to go through account policies and delete old toots
* Fix last inspected status id logic
All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.
* Add tests
* Refactor scheduler and add tests
* Add user interface
* Add support for discriminating based on boosts/favs
* Add UI support for min_reblogs and min_favs, rework UI
* Address first round of review comments
* Replace Snowflake#id_at_start with with_random parameter
* Add tests
* Add tests for StatusesCleanupController
* Rework settings page
* Adjust load-avoiding mechanisms
* Please CodeClimate
* Add account_notes relationship
* Add tests
* Fix owned account notes not being deleted when an account is deleted
* Add post-migration to clean up orphaned account notes
* Fix Delete and Create-related locks expiring too fast
Fixes#16238
By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.
This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.
* Fix other RedisLock autorelease durations
Fixes#15645
- things that only perform a few simple database queries (e.g. finding and
saving a record) have been left unchanged, so they'll still use the default
10s duration
- things that perform significantly more complex database queries have been
changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
timeout
- `app/views/statuses/_simple_status.html.haml`:
Small markup change in glitch-soc, on a line that has been modified by
upstream. Ported upstream changes.
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending
Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots
Remove references to hashtags in profile directory from the code
and the admin UI
* Add tests
* Ensure deleted statuses are marked as such
* Save some redis memory by not storing URIs in delete_upon_arrival values
* Avoid possible race condition when processing incoming Deletes
* Avoid potential duplicate Delete forwards
* Lower lock durations to reduce issues in case of hard crash of the Rails process
* Check for `lock.aquired?` and improve comment
* Refactor RedisLock usage in app/lib/activitypub
* Fix using incorrect or non-existent sender for relaying Deletes
Conflicts:
- `README.md`:
Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
Not really a conflict, upstream change (removing the spam checker) too close
to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
Same.
- `app/services/remove_status_service.rb`:
Same.
- `app/views/admin/settings/edit.html.haml`:
Same.
- `config/settings.yml`:
Same.
- `config/environments/production.rb`:
Not a real conflict, upstream added a default HTTP header, but we have
extra headers in glitch-soc.
Added the header.
* Update devise-two-factor to unreleased fork for Rails 6 support
Update tests to match new `rotp` version.
* Update nsa gem to unreleased fork for Rails 6 support
* Update rails to 6.1.3 and rails-i18n to 6.0
* Update to unreleased fork of pluck_each for Ruby 6 support
* Run "rails app:update"
* Add missing ActiveStorage config file
* Use config.ssl_options instead of removed ApplicationController#force_ssl
Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.
* Fix nonce directives by removing Rails 5 specific monkey-patching
* Fix fixture_file_upload deprecation warning
* Fix yield-based test failing with Rails 6
* Use Rails 6's index_with when possible
* Use ActiveRecord::Cache::Store#delete_multi from Rails 6
This will yield better performances when deleting an account
* Disable Rails 6.1's automatic preload link headers
Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.
In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.
* Switch to Rails 6.0 default config
* Switch to Rails 6.1 default config
* Do not include autoload paths in the load path
Conflicts:
- `app/validators/status_length_validator.rb`:
Upstream changes too close to glitch-soc MAX_CHARS changes, but not a real
conflict.
Applied upstream changes.
- `package.json`:
glitch-soc-only dependency textually too close to a dependency updated
upstream, not a real conflict.
Applied upstream changes.
* Update twitter-text from 1.14 to 3.1.0
* Disable emoji parsing
* Properly depend on twitter-text for url detection
* Fix some URLs being wrongly detected client-side
* Add test for server-side validation of non-autolinkable URLs
* Fix server-side status length counting
* Change ResolveAccountService's handling of skip_webfinger
Change it so it never makes any webfinger query, as the name would imply.
* Add tests
* Change FollowService to not take an URI for target_account
* Restore domain-block check in FollowService
* Fix tests
Conflicts:
- `app/javascript/styles/mastodon/modal.scss`:
For some reason we changed the file loading path in glitch-soc,
but now upstream has completely changed how the logo is loaded.
Applied upstream changes.
Conflicts:
- `app/models/public_feed.rb`:
Upstream refactored a bit, glitch-soc had specific code for local-only
statuses.
Updated glitch-soc's specific code accordingly.
* disable NewCops
* update TargetRubyVersion
* Fix Lint/MissingSuper for ActiveModelSerializers::Model
* Fix Lint/MissingSuper for feed
* Fix Lint/FloatComparison
* Do not use instance variables
Conflicts:
- `config/webpack/configuration.js`:
Upstream updated the `js-yaml` dependency, which changed how to call it.
Those changes conflicted because that code is pretty different in glitch-soc
which has to deal with its more complex theming system.
Proceeded to the same compatibility changes in glitch-soc's code.
- `package.json` and `yarn.lock`:
Not really a conflict, just glitch-soc-specific dependencies textually too
close to some dependencies updated upstream.
* Fix being able to import more than allowed number of follows
Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.
This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).
* Catch validation errors to not re-queue failed follows
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
As a regression from the recent optimizations, mentions were left untouched
until `account.destroy`, which would then delete them individually,
and executing queries to find and delete associated notifications, resulting
in a massive slowdown.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/lib/feed_manager.rb`:
Not a real conflict, glitch-soc-only DM-related method
too close to changed upstream stuff.
Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
Additional logic in glitch-soc to clear DMs from timelines.
Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
Additional code in glitch-soc to clear DM timelines.
Ported upstream changes.
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix deleting polls not deleting notifications
* Fix fav notification deletion when deleting a toot
* Refactor DeleteAccountService spec
* Add DeleteAccountService tests for other associations and notifications
* Add favourite handling spec in status removal
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix account deletion workers being queued multiple times for a single account
* Fix poll votes being unnecessarily instantiated on poll deletion
* Fix favourites being unnecessarily instantiated on status deletion
* Remove inaccurate comments
* Delete polls instead of destroying them
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix ResolveAccountService accepting mismatching acct: URI
* Set attributes that should be updated regardless of suspension
* Fix key fetching
* Automatically merge remote accounts with duplicate `uri`
* Add tests
* Add "tootctl accounts fix-duplicates"
Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Improve searching for private toots from URL
Most of the time, when sharing toots, people use the toot URL rather than
the toot URI, which makes sense since it is the user-facing URL.
In Mastodon's case, the URL and URI are different, and Mastodon does not
have an index on URL, which means searching a private toot by URL is done
with a slow query that will only succeed for very recent toots.
This change gets rid of the slow query, and attempts to guess the URI from
URL instead, as Mastodon's are predictable.
* Add tests
* Only return status with guessed uri if url matches
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/form/admin_settings.rb`:
New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
New setting added upstream. Ported it.
* Add indication to admin UI of whether a report has been forwarded
* Rework how forwarded status is displayed
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/services/remove_status_service.rb`:
Conflict caused by us having a distinc Direct timeline.
Ported upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
Conflict between glitch-soc's variable character limit and upstream
refactoring that part of the code.
Ported upstream changes.
Extract logic for determining ActivityPub inboxes to send deletes
to to its own class and explicitly include the person the status
replied to (even if not mentioned), people who favourited it, and
people who replied to it (though that one is still not recursive)
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
Upstream added the `bug` label to bug reports.
Did the same.
- `app/services/fan_out_on_write_service.rb`:
Upstream put DMs back into timelines, glitch-soc was already doing it.
Ignored upstream changes.
* Fix webfinger redirect handling in ResolveAccountService
ResolveAccountService#process_webfinger! handled a one-step webfinger
redirection, but only accepting the result if it matched the exact URI passed
as input, defeating the point of a redirection check.
Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`,
updating the resulting `acct:` URI with the result of the first webfinger
query.
* Add tests
Nginx can be configured to bypass proxy cache when a special header
is in the request. If the response is cacheable, it will replace
the cache for that request. Proxy caching of media files is
desirable when using object storage as a way of minimizing bandwidth
costs, but has the drawback of leaving deleted media files for
a configured amount of cache time. A cache buster can make those
media files immediately unavailable. This especially makes sense
when suspending and unsuspending an account.
Most of the time, when sharing toots, people use the toot URL rather than
the toot URI, which makes sense since it is the user-facing URL.
In Mastodon's case, the URL and URI are different, and Mastodon does not
have an index on URL, which means searching a private toot by URL is done
with a slow query that will only succeed for very recent toots.
This change gets rid of the slow query, and attempts to guess the URI from
URL instead, as Mastodon's are predictable.
* Fix crash in SuspendAccountWorker
`follows` is an array thanks to `to_a`
* Fix code style issue
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
Conflict due to upstream changing suspension logic while
glitch-soc has an extra option to hide followers count.
Ported upstream changes.
e.g. if someone on an instance that previously had followers gets mentioned
in a private toot, before this PR, they would not receive a
Collection-Synchronization header and may show the toot to the former followers
in addition to the mentioned person.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, we deleted it to not be flooded by Depandabot.
Kept deleted.
- `Gemfile.lock`:
Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
Upstream updated the serializer to support timed mutes, while
glitch-soc added a custom API ages ago to get information that
is already available elsewhere.
Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
🤷
- `app/models/user.rb`:
New user setting added upstream while we have glitch-soc-specific
user settings. Added upstream's user setting.
- `config/settings.yml`:
Upstream added a new user setting close to a user setting we had
changed the defaults for. Added the new upstream setting.
- `package.json`:
Upstream dependency updated “too close” to a glitch-soc-specific
dependency. No real conflict. Updated the dependency.
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.
Fix#14091
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
Upstream changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/models/user.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc not preventing moved accounts from logging
in.
Ported upstream changes while keeping the ability for moved accounts to log
in.
- `app/policies/status_policy.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's ability to hide followers count.
Ported upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `package.json`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb
All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
Conflicts:
- `config/webpack/shared.js`:
Upstream has changed how Tesseract.js gets included and dropped a dependency.
The conflict is caused by glitch-soc having different code due to its
theming system.
Ported upstream changes.
- `lib/mastodon/version.rb`:
Upstream refactor/code style change in a place we replaced upstream's
repo URL with ours.
Ported upstram changes, keeping our repo URL.
- `yarn.lock`:
Upstream dropped dependencies, one of which was textually too close to
a glitch-soc-specific dependency. Not a real conflict.
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream change too close to a glitch-soc change related to
instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Minor conflict due to glitch-soc's handling of Direct Messages,
merged upstream changes.
- `yarn.lock`:
Not really a conflict, caused by glitch-soc-only dependencies
being textually too close to updated upstream dependencies.
Merged upstream changes.
DISTINCT clause removes duplicated records according to all the selected
attributes. In reality, it can remove duplicated records only looking at
statuses.id, but the clause confuses the query planner and yields
insufficient performance.
The behavior is also problematic if the scope produced by HashQueryService
is used to query columns without id (using pluck method, for example). The
scope is expected to contain unique statuses, but the uniquness will be
evaluated with some arbitrary columns other than id.
GROUP BY clause resolves those problem by explicitly specifying the
column to take into account for the record distinction.
A workaround for the problem of DISTINCT clause in
Api::V1::Timelines::TagController is no longer necessary and removed.
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
Not a “real” conflict, but change too close to a change we made to
fix the vanilla WebUI locally pushing authored local-only toots in the
public TL view.
Conflicts:
- `.env.production.sample`:
Upstream deleted it, I decided to keep it.
- `package.json`:
Upstream updated a dependency textually too close to wavesurfer.js
which has been deleted from upstream but is kept in glitch-soc for now.
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
Conflict due to glitch-soc having to take care of local-only
pinned toots in that controller.
Took upstream's changes and restored the local-only special
handling.
- `app/controllers/auth/sessions_controller.rb`:
Minor conflicts due to the theming system, applied upstream
changes, adapted the following two files for glitch-soc's
theming system:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`
- `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
Minor conflict due to glitch-soc having to handle local-only
toots specially. Applied upstream changes and restored
the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
Minor conflict due to the theming system.
- `package.json`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
- `yarn.lock`:
Upstream dependency updated, too close to a glitch-soc-only
dependency in the file.
* Split media cleanup from reject-media domain blocks to its own service
* Slightly improve ClearDomainMediaService error handling
* Lower DomainClearMediaWorker to lowest-priority queue
* Do not catch ActiveRecord::RecordNotFound in domain block workers
* Fix DomainBlockWorker spec labels
* Add some specs
* Change domain blocks to immediately mark accounts as suspended
Rather than doing so sequentially, account after account, while cleaning
their data. This doesn't change much about the time the block takes to
complete, but it immediately prevents interaction with the blocked domain,
while up to now, it would only be guaranteed when the process ends.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, just a glitch-soc-only dependency too close to a
dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
Not a real conflict, just a change too close to glitch-soc-changed code
for optionally showing boosts in public timelines.
Applied upstream changes.
- `app/views/layouts/application.html.haml`:
Upstream a new, static CSS file, conflict due to glitch-soc's theming
system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
both files are very different. Removed 'unsafe-inline' as well.
Conflicts:
- `app/controllers/statuses_controller.rb`:
Upstream disabled the embed controller for reblogs.
Not a real conflict, but glitch-soc has an extra line to deal
with its theming system.
Ported upstream changes.
- `app/javascript/packs/public.js`:
Upstream made changes to get rid of most inline CSS, this changes
javascript for public pages, which in glitch are split between
different files. Ported those changes.
- `app/models/status.rb`:
Upstream changed the block check in `Status#permitted_for` to
include domain-block checks. Not a real conflict with glitch-soc,
but our scope is slightly different, as our scope for
unauthenticated access do not include instance-local toots.
Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
Not a real conflict, upstream added a new field to the instance
serializer, the conflict is one line above since we added more of
that.
Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
Upstream got rid of most inline CSS and moved hidden elements
to data attributes in the process, in fields were we have
different values.
Ported upstream changes while keeping our glitch-specific
values.
- `app/views/statuses/_simple_status.html.haml`:
Upstream got rid of inline CSS on an HAML line we treat
differently, stripping empty text nodes.
Ported upstream changes to the style attribute, keeping
the empty text node stripping behavior.
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
Upstream bumped poll option character limit, but we already had
a higher one, kept ours.
- `app/validators/poll_validator.rb`:
Upstream bumped poll option character limit, but we already had
a higher one, kept ours.
- `config/initializers/content_security_policy.rb`:
Upstream added a rule, the way we compute ours is different, but
that added rule has been ported.
- `package.json`:
No real conflict, dependency update. Performed the same update.
- `yarn.lock`:
No real conflict, dependency update. Performed the same update.
Mastodon enforces the “sensitive” flag on media attachments whenever a toot
is posted with a Content Warning. However, it does so *after* potentially
converting the Content Warning to toot text (when there is no toot text),
which leads to inconsistent and surprising behavior for API clients.
This commit fixes this inconsistency.
Conflicts:
- `README.md`:
Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
Updating rails-ujs, no real conflict, but a comment to close to changed
code. Various glitch-soc-only files have been updated to match those changes,
though.
- `package.json`:
No real conflict, just an additional dependency in glitch-soc that was too
close to something updated upstream. Took upstream's changes.
Only look up private toots from database if the request failed because of 401,
403 or 404 errors, as those may indicate a private toot, rather than something
that isn't a toot or cannot be processed.
* Change ActivityPub follower/following collections to not link first page
* Add support for hiding followers and following of remote users
* Switch to using a single `hide_collections` column
* Address code style remarks
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
Conflict due to upstream adding a new parameter (with_rate_limit),
too close to glitch-soc's own additional parameter (content_type).
Added upstream's parameter.
- `app/services/post_status_service.rb`:
Conflict due to upstream adding a new parameter (rate_limit),
too close to glitch-soc's own additional parameter (content_type).
Added upstream's parameter.
- `app/views/settings/preferences/appearance/show.html.haml`:
Conflict due to us not exposing theme settings here (as we have
a different flavour/skin menu).
Took upstream change, while still not exposing theme settings.
- `config/webpack/shared.js`:
Coding style fixes for a part we have rewritten.
Discarded upstream changes.
Fixes#13136
When a user's canonical acct domain is different from its id's domain
(WEB_DOMAIN ≠ LOCAL_DOMAIN), two webfinger queries are required to find the
canonical domain from the URI. However, we skip webfinger queries when
updating only the key of a remote user, which led to the creation of a
duplicate account, using the URI's domain instead of the canonical acct: one.
Conflicts:
- `Gemfile.lock`:
No real conflict, glitch-soc-only dependency (redcarpet) too close to an
upstream one (rdf-normalize)
- `README.md`:
we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
Upstream added code to keep bookmarked statuses, we were already doing so
with slightly different code. Discarded upstream's changes.
- `package.json`:
No real conflict, glitch-soc-only dependency (favico.js) too close to
an upstream one
* Fix invalid votes from the API being accepted
Fixes#12556
- Ensure `choice` is an integer instead of silently converting to 0
- Ensure `choice` corresponds to an actual choice of the poll
* Please CodeClimate
Conflicts:
- `app/controllers/application_controller.rb`:
Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
Conflict due to theming system.
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.
Fix#7812, fix#12246
* Make BackupService resilient to read timeouts
If an attachment read times out, assume that the resources is
inaccessible and continue the backup without it. This fixes#12280.
* Both errors on one line
Conflicts:
- `config/locales/en.yml`
No real conflict, upstream added a translatable string “too close” to
one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
No real conflict, additional dependency in glitch-soc
* Remove “protocol” argument and return value, as only ActivityPub is supported
* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService
* Fix tests
Conflicts:
- package.json
Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
Not really a conflict, caused by an additional dependency in glitch-soc.
* Add follow_request notification type
The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.
Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.
Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.
* Show follow request link in single-column if there are pending requests, even if account isn't locked
* Push follow requests from notifications to the follow_requests list
* Offer to accept or reject follow request from the notification
* Redesign follow request notification
If a poll contains empty options (which is apparently possible on Pleroma),
it is created without them. However, the poll update code doesn't filter
empty options, and thus:
1. Clear known votes, as it assumes the set of options has changed
2. Errors out because it tries adding empty options, which fails validation
This commit fixes that by filtering them out the same way they are filtered
out at poll creation time.
When suspending or silencing a domain, one probably wants the silence or
suspend parts to take effect as soon as possible. Deleting media files can
take a while (possibly days), so perform silencing as soon as possible, and
clean up media afterwards.
* add youtube oembed endpoint
* add check for oembed endpoint
* change unless for a more readable if
* clear blank lines
* endpoint via https
* Fix string literal in condition
* use cache for endpoints
* use cache for endpoints
* clean up and adding check
* clean up and remove redundant return
* add html check
* add false to return
* use double quotes
* use double quotes
* Clean up
* Add backend support for bookmarks
Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.
* Add spec for bookmark endpoints
* Add front-end support for bookmarks
* Introduce OAuth scopes for bookmarks
* Add bookmarks to archive takeout
* Fix migration
* Coding style fixes
* Fix rebase issue
* Update bookmarked_statuses to latest UI changes
* Update bookmark actions to properly reflect status changes in state
* Add bookmarks item to single-column layout
* Make active bookmarks red
* Fix already-known remote private toots not being searchable by URL
Whenever an URL search fails, and an account is provided, try getting a
private status matching that URL from database.
* Fix searching by `id` field instead of `uri`
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`
conflict because of the poll option limit being different than upstream's
* Fix being able to follow oneself by moving to an account that was following the old one
* Add specs
* Add spec to catch MoveWorker issue with local followers following both accounts
* Fix move worker breaking when a local account follows both source and target accounts
* Fix migration from remote to local account not sending Undo Follow
* Fix show_reblogs not being preserved for moved account's followers
* Add ability to add oneself to lists
* Change search results to include oneself when searching through followers
* Mark follow relation as optional in ListAccount
Conflicts:
- README.md
- app/helpers/statuses_helper.rb
Upstream moved account helpers to their own file, we had extra
helpers there, moved too.
- app/lib/sanitize_config.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- config/locales/simple_form.en.yml
- spec/lib/sanitize_config_spec.rb
Conflicts:
- app/models/media_attachment.rb
Upstream raised max image size from 8MB to 10MB while our limit is
configurable. Raised the default to 10MB.
* Add test to handle suspended and missing users in BootstrapTimelineService
* Fix BootstrapTimelineService crashing when bootstrapped accounts are invalid
* Add voters count to polls
* Add ActivityPub serialization and parsing of voters count
* Add support for voters count in WebUI
* Move incrementation of voters count out of redis lock
* Reword “voters” to “people”
* Change silenced accounts to require approval on follow
* Also require approval for follows by people explicitly muted by target accounts
* Do not auto-accept silenced or muted accounts when switching from locked to unlocked
* Add `follow_requests_count` to verify_credentials
* Show “Follow requests” menu item if needed even if account is locked
* Add tests
* Correctly reflect that follow requests weren't auto-accepted when local account is silenced
* Accept follow requests from user-muted accounts to avoid leaking mutes
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
Conflict because we changed the number of default results to be
configurable
- app/lib/settings/scoped_settings.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
Addition of a new “noindex” site-wide setting,
conflict due to our change of the two other site-wide settings
(default flavour and skin instead of theme)
Conflicts:
- app/views/directories/index.html.haml
Upstream has redesigned the profile directory, and we
had a glitch-soc-specific change to hide follower counts.
Ported that change to the new design.
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb
All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
* Add more accurate hashtag search
Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly
* Fix search analyzer dropping non-ascii characters
- Restrict followers counts to local users to minimize local advantage
- Fix emoji shortcodes causing error in search
- Fix search syntax parse errors not being caught
* Add more accurate account search
When ElasticSearch is available, a more accurate search is implemented:
- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)
The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.
* Support single-letter usernames
* Fix tests
* Fix not picking up account updates
* Add weights and normalization for scores, skip zero terms queries
* Use local counts for accounts index, adjust search parameters
* Fix mistakes
* Using updated_at of accounts is inadequate for remote accounts
* Add database columns for adding notes to domain blocks/restrctions
* Add admin UI to set private and public comments when blocking a domain
* Add text for private and public comments on domain blocks
* Show domain block comments in admin UI
* Add comments to the domain block undo page
* Make UnblockDomainService more robust regarding upgraded domain blocks
* Allow editing domain blocks
* Rename button from “undo domain block” to “view domain block” in account admin UI
* Change test to unsilence silenced users from upgraded blocks
* Fix BackupService crashing when an attachment is missing
For various reasons such as admin error or out-of-sync media and
database backups, it might be possible for local attachments to be lost.
This commit allows the BackupService to continue its work even if some media
file is missing.
* Change error message
* Add a spam check
* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance
* Add more tests
* Add exemption when the message is a reply to something that mentions the sender
* Use Nilsimsa Compare Value instead of Levenshtein distance
* Use MD5 for messages shorter than 10 characters
* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account
* Expire spam check data after 3 months
* Add support for local statuses, reduce expiration to 1 week, always create a report
* Add content warnings to the spam check and exempt empty statuses
* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check
* Add all matched statuses into automatic report
* Fix BackupService crashing when an attachment is missing
For various reasons such as admin error or out-of-sync media and
database backups, it might be possible for local attachments to be lost.
This commit allows the BackupService to continue its work even if some media
file is missing.
* Change error message
* Remove Salmon and PubSubHubbub endpoints
* Add error when trying to follow OStatus accounts
* Fix new accounts not being created in ResolveAccountService
Conflicts:
- app/models/media_attachment.rb
Upstream added audio attachment support
- app/serializers/initial_state_serializer.rb
Upstream added audio attachment support and how mimetypes are returned
- app/serializers/rest/instance_serializer.rb
Upstream added a few fields
- config/application.rb
Upstream added a different paperclip transcoder
* Change domain blocks to automatically support subdomains
If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)
* Match subdomains of existing accounts when blocking/unblocking domains
* Improve code style
* Add audio uploads
Fix#4827
Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts
them to OGG. Media attachments get a new `audio` type. In the UI,
audio uploads are displayed identically to video uploads.
* Improve code style
* Record account suspend/silence time and keep track of domain blocks
* Also unblock users who were suspended/silenced before dates were recorded
* Add tests
* Keep track of suspending date for users suspended through the CLI
* Show accurate number of accounts that would be affected by unsuspending an instance
* Change migration to set silenced_at and suspended_at
* Revert "Also unblock users who were suspended/silenced before dates were recorded"
This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.
* Switch from using suspended and silenced to suspended_at and silenced_at
* Add post-deployment migration script to remove `suspended` and `silenced` columns
* Use Account#silence! and Account#suspend! instead of updating the underlying property
* Add silenced_at and suspended_at migration to post-migration
* Change account fabricator to translate suspended and silenced attributes
* Minor fixes
* Make unblocking domains always retroactive
Conflicts:
- app/javascript/mastodon/features/compose/components/compose_form.js
Upstream cleaned up a bit, including on lines in which
we replaced the hardcoded 500 character limit with a maxChar
constant. Applied the changes while keeping maxChar instead of 500.
- app/javascript/packs/public.js
Moved upstream's new animated avatar hover handling in
app/javascript/core/public.js
- app/javascript/styles/fonts/montserrat.scss
Upstream fixed local font name, applied those changes.
- app/javascript/styles/fonts/roboto.scss
Upstream fixed local font name, applied those changes.
- lib/mastodon/version.rb
Upstream made repo URL configurable, did the same, but
default to glitch-soc
The @property attribute in HTML is a space-separated list of values.
This change normalizes whitespace and finds the desired value in
the list instead of requiring an exact single-value match.
More details:
https://www.ctrl.blog/entry/rdfa-socialmedia-metadata.html
* Refactor imports
* Export show_reblogs when exporting list of followed users
* Add support for importing show_reblogs with following collection
* Fix tests
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"
This reverts commit 120544067f.
* Revert "Hide blocking accounts from blocked users (#10442)"
This reverts commit 62bafa20a1.
* Improve blocked view of profiles
- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
Conflicts:
- app/controllers/following_accounts_controller.rb
Conflicts were due to glitch-soc's theming system.
- app/javascript/packs/public.js
Some code has been change upstream, but it has been
moved to app/javascript/core/settings.js in glitch-soc.
Applied the changes there.
* Revert "Add indication that you have been blocked in web UI (#10420)"
This reverts commit bd02ec6daa.
* Revert "Add `blocked_by` relationship to the REST API (#10373)"
This reverts commit 9745de883b.
* Hide blocking accounts from search results
* Filter blocking accouts from account followers
* Filter blocking accouts from account's following accounts
* Filter blocking accounts from “reblogged by” and “favourited by” lists
* Remove blocking account from URL search
* Return 410 on trying to fetch user data from a user who blocked us
* Return 410 in /api/v1/account/statuses for suspended or blocking accounts
* Fix status filtering when performing URL search
* Restore some React improvements
Restore some cleanup from bd02ec6daa
* Refactor by adding `without_blocking` scope
Conflicts:
- app/controllers/admin/settings_controller.rb
- app/models/form/admin_settings.rb
Conflicts caused by upstream refactoring, while we have
flavours and skins, with the flavour_and_skin pseudo-setting.
Conflicts:
- config/locales/en.yml
Conflict caused by the glitch-soc-specific “flavour” string being too close
to the newly introduced “identity_proofs” string. Just included both.
Conflicts:
- app/controllers/accounts_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/statuses_controller.rb
All conflicts caused by the additional `use_pack` used for glitch-soc's theming
system.
Conflicts:
- app/services/remove_status_service.rb
Conflict because we still handle the direct timeline.
Took upstream changes, with that one extra function call.
- config/locales/ca.yml
Conflict because theme names were updated.
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
- config/locales/oc.yml
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
- config/locales/pl.yml
Decided to *keep* the theme names even if they are useless
to avoid future conflicts.
Conflicts:
- app/controllers/settings/follower_domains_controller.rb
Removed upstream. Did the same here. Maybe we should not have?
- config/locales/en.yml
Upstream removed the “Authorized followers” page and associated
translations. This is too close in the file to our glitch-soc-specific
“flavour” string. No actual conflict.
- config/locales/ja.yml
Same as above.
- config/locales/pl.yml
Same as above.
- config/navigation.rb
No real conflict. New route added too close to the glitch-soc-specific
“flavours” one.
- config/webpack/configuration.js
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
- config/webpack/loaders/babel.js
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
The contents of this file have been moved to package.json.
- config/webpack/shared.js
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
- config/webpacker.yml
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
- jest.config.js
The contents of this file have been moved to package.json.
- package.json
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
- yarn.lock
Upstream refactored the webpack(er) configuration quite a bit.
Tried to keep up.
* Fix poll update handler calling method was that was not available
Fix regression from #10209
* Refactor VoteService
* Refactor ActivityPub::DistributePollUpdateWorker and optimize it
* Fix typo
* Fix typo
