Since #14145, the `set_type_and_extension` has been moved from
`before_post_process` to `before_file_post_process`, but while the former
runs before all validations performed by Paperclip, the latter is dependent
on the order validations and hooks are defined.
In our case, this meant video files could be checked against the generic 10MB
limit, causing validation failures, which, internally, make Paperclip skip
post-processing, and thus, transcoding of the video file.
The actual validation would then happen after the type is correctly set, so
the large file would pass validation, but without being transcoded first.
This commit moves the hook definition so that it is run before checking for
the file size.
* Add UserNote model
* Add UI for user notes
* Put comment in relationships entity
* Add API to create user notes
* Copy user notes to new account when receiving a Move activity
* Address some of the review remarks
* Replace modal by inline edition
* Please CodeClimate
* Button design changes
* Change design again
* Cancel note edition when pressing Escape
* Fixes
* Tweak design again
* Move “Add note” item, and allow users to add notes to themselves
* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
- Fix audio attachments not being represented in OpenGraph tags
- Fix audio being represented as "1 image" in OpenGraph descriptions
- Fix video metadata being overwritten by paperclip-av-transcoder
- Fix embedded player not using Mastodon's UI
- Fix audio/video progress bars not moving smoothly
- Fix audio/video buffered bars not displaying correctly
* Fix admin page crashing when trying to block an invalid domain name
Fixes#13880
* Fix trailing and leading spaces not being properly stripped for domain blocks
* Improve RSS entries for statuses
- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests
* Remove title from OEmbed serialization
Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
When deduplicating account usernames for OAuthable users, the routine did check if any account was known with that username, including remote accounts. This caused some unnecessary deduplication, and usernames ending with unexpected trailing _1.
This fixes#13580
Follow-up to #13452, fixing broken `uri.nil?` test.
Also remove the separate check for `uri` presence, as that would result
in a “Please review 2 errors below” while only one would be listed.
* Fix “Email changed” notification sometimes having wrong e-mail
Fixes#6778
The root of the issue is that `send_devise_notification` was called before
the changes were properly commited to the database, causing the mailer to
pick previous values if running too early.
Devise's documentation provides guidance on how to handle that[1][2], however,
I have found it to not be working, as the following happens, in that order:
- `send_devise_notification` is called for the `email_changed` notification.
In that case, `changed?` is false and `saved_changes?` is true, so
if we use the former, we have the same issue.
- the `after_commit` hook is called
- `send_devise_notification` is called for the `confirmation_instructions`
notification.
In that case, `changed?` is still false, and `saved_changes?` still true,
so if we use the latter, that second notification email is simply not
going to be sent (as we would be queuing the notification *after*
executing the after_commit hook).
This is because it may be called from either an `after_update` or
`after_commit` hook, the difference not being a call to `save` but the
transaction actually being committed to the database. This may arguably
be a bug in Devise, or Devise's notification.
The proposed workaround is inspired by Devise's documentation but checks
whether a transaction is open to make the call whether to immediately
send the notification or defer it to the `after_commit` hook.
[1]: https://www.rubydoc.info/github/plataformatec/devise/Devise%2FModels%2FAuthenticatable:send_devise_notification
[2]: 406915cb78/lib/devise/models/authenticatable.rb (L133-L194)
* Fix cases when sending notifications without changing the model
* Defer sending if and only if in transaction including current record
* Fix PostgreSQL load when linking in announcements
Fixes#13245 by caching status lookups
Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.
Since it's only used by announcements so far, there won't
be much statuses to cache.
* Perform status lookup when saving announcements, not when rendering them
* Change EntityCache#status to fetch URLs instead of looking into the database
* Move announcement link lookup to publishing worker
* Address issues pointed out during review
* Change ActivityPub follower/following collections to not link first page
* Add support for hiding followers and following of remote users
* Switch to using a single `hide_collections` column
* Address code style remarks
Change `all_day` to be a visual client-side cue only
Publish immediately if `scheduled_at` is in the past
Add `published_at` and `updated_at` to announcements JSON
While making browser requests in the other sessions after a password
change or reset does not allow you to be logged in and correctly
invalidates the session making the request, sessions have API tokens
associated with them, which can still be used until that session
is invalidated.
This is a security issue for accounts that were already compromised
some other way because it makes it harder to throw out the hijacker.
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
Allow browsing and filtering all relationships instead of just
followers, unify the codebase with the user-facing relationship
manager, add ability to see who the user invited
* Fix unused role routes being generated
* Remove unused JavaScript code
* Refactor filters code to be DRYer
* Fix `.count == 0` comparisons to `.empty?` in views
* Fix filters in views
Fixes#12795
It was already possible to create domain blocks for TLDs, but those
weren't enforced, nor editable. This commit changes it so that they
are enforced and editable.
This changes the REST API to return unicode domains in the `acct`
attribute instead of punycode, and to render unicode instead of
punycode on public HTML pages as well.
Fix#7812, fix#12246
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"
This reverts commit dff46b260b.
* Fix statuses being rejected because of invalid hashtag names
* Add spec for invalid hashtag names in statuses
* Add test for featured tags controller
* Show badge on group actor in WebUI
* Do not notify in case of by following group actor
* If you mention group actor, also mention group actor followers
* Relax characters that can be used in username (same as Application)
* Revert "Relax characters that can be used in username (same as Application)"
This reverts commit 7e10a137b8.
* Delete display_name method
* Add follow_request notification type
The notification type already existed in the backend but was never pushed
to the front-end. This also means translation strings were also available
for the backend, from the notification mailer.
Unlike other notification types, these are off by default, to match what
I remember of Gargron's view on the topic: that follow requests should not
clutter notifications and should instead be reviewed at the user's own
leisure in the dedicated column.
Since follow requests have their own column, I've deemed it unnecessary to
add a specific tab for them in the notification quick filter.
* Show follow request link in single-column if there are pending requests, even if account isn't locked
* Push follow requests from notifications to the follow_requests list
* Offer to accept or reject follow request from the notification
* Redesign follow request notification
Before this patch, if remote poll options have leading or trailing spaces,
the information stored locally won't match them, causing federated voting to
fail.
* Add backend support for bookmarks
Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.
* Add spec for bookmark endpoints
* Add front-end support for bookmarks
* Introduce OAuth scopes for bookmarks
* Add bookmarks to archive takeout
* Fix migration
* Coding style fixes
* Fix rebase issue
* Update bookmarked_statuses to latest UI changes
* Update bookmark actions to properly reflect status changes in state
* Add bookmarks item to single-column layout
* Make active bookmarks red
* Add ability to add oneself to lists
* Change search results to include oneself when searching through followers
* Mark follow relation as optional in ListAccount
Change the behaviour of remotable concern. Previously, it would skip
downloading an attachment if the stored remote URL is identical to
the new one. Now it would not be skipped if the attachment is not
actually currently stored by Paperclip.
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive
Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
* Change animated GIF detection to not shell out to ImageMagick
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Change video encoding parameters to limit to 10800 video frames
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Limit GIF image size further
Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>
* Always strip metadata from video files
* Fix code style issues
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
* Add voters count to polls
* Add ActivityPub serialization and parsing of voters count
* Add support for voters count in WebUI
* Move incrementation of voters count out of redis lock
* Reword “voters” to “people”
* Addition of update activity distribution by alias, minor correction
* Distribute Update activity after adding alias
* Add uniqueness verification to alias uri
* accept acct starting with @
* fix double-quoted to single-quoted
* Add search and sort functions to hashtag admin UI
* Move scope processing from tags_controller to tag_filter
* Fix based on method naming conventions
* Fixed not to get 500 errors for invalid requests
* Add soft delete for statuses to allow them to appear instant
* Allow reporting soft-deleted statuses and show them in the admin UI
* Change index for getting an account's statuses
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
* Add automatic blocklist display in /about/blocks
Inspired by https://github.com/Gargron/mastodon.social-misc
* Add admin option to set who can see instance blocks
* Normalize locales files
* Rename “Sandbox” to “Silence” for consistency
* Disable /about/blocks when in whitelist mode
* Optionally display rationale for domain blocks
* Only display domain blocks that have user-facing limitations, and order them
* Redesign table of blocked domains to better handle long domain names and rationales
* Change domain blocks ordering now that rationales aren't displayed right away
* Only show explanation for block severities actually in use
* Reword instance block explanations and add disclaimer for public fetch mode
* Add more accurate hashtag search
Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly
* Fix search analyzer dropping non-ascii characters
* Add more accurate account search
When ElasticSearch is available, a more accurate search is implemented:
- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)
The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.
* Support single-letter usernames
* Fix tests
* Fix not picking up account updates
* Add weights and normalization for scores, skip zero terms queries
* Use local counts for accounts index, adjust search parameters
* Fix mistakes
* Using updated_at of accounts is inadequate for remote accounts
Fix percentages in usage breakdown having too many digits
Change trending hashtags to only ask for review if a hashtag enters
the top 3 position in the set, since it's the only items shown in
the default web UI
* Add database columns for adding notes to domain blocks/restrctions
* Add admin UI to set private and public comments when blocking a domain
* Add text for private and public comments on domain blocks
* Show domain block comments in admin UI
* Add comments to the domain block undo page
* Make UnblockDomainService more robust regarding upgraded domain blocks
* Allow editing domain blocks
* Rename button from “undo domain block” to “view domain block” in account admin UI
* Change test to unsilence silenced users from upgraded blocks
* Fix account tags not being saved correctly
Regression from f371b32
Fix Tag#discoverable not returning tags where listable is nil instead of true
Add notice when saving hashtags in admin UI
Change public hashtag and directory pages to return 404 for forbidden tags
* Remove unused locale string
Fix#271
Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.
When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.
The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.
The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
* Change hashtag search to only return results that have trended in the past
A way to eliminate typos and other one-off "junk" results
* Fix excluding exact matches that don't have a score
* Fix tests
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.
Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.
After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.
Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
* Add support for an instance actor
* Skip username validation for local Application accounts
* Add migration script to create instance actor
* Make Codeclimate happy
* Switch to id -99 for instance actor
* Remove unused `icon` and `image` attributes from instance actor
* Use if/elsif/else instead of return + ternary operator
* Add instance actor to fresh installs
* Use instance actor as instance representative
Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.
* Seed database in test environment
* Fix single-user mode
* Fix tests
* Fix specs to accomodate for an extra `Account`
* Auto-reject follows on instance actor
Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.
* Fix webfinger lookup and serialization for instance actor
* Rename instance actor
* Make it clear in the HTML view that the instance actor should not be blocked
* Raise cache time for instance actor as there's no dynamic content
* Re-use /about/more with a flash message for instance actor profile
* Add a spam check
* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance
* Add more tests
* Add exemption when the message is a reply to something that mentions the sender
* Use Nilsimsa Compare Value instead of Levenshtein distance
* Use MD5 for messages shorter than 10 characters
* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account
* Expire spam check data after 3 months
* Add support for local statuses, reduce expiration to 1 week, always create a report
* Add content warnings to the spam check and exempt empty statuses
* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check
* Add all matched statuses into automatic report
* Remove Salmon and PubSubHubbub endpoints
* Add error when trying to follow OStatus accounts
* Fix new accounts not being created in ResolveAccountService
* Change domain blocks to automatically support subdomains
If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)
* Match subdomains of existing accounts when blocking/unblocking domains
* Improve code style
* Add audio uploads
Fix#4827
Accept uploads of OGG, WAV, FLAC, OPUS and MP3 files, and converts
them to OGG. Media attachments get a new `audio` type. In the UI,
audio uploads are displayed identically to video uploads.
* Improve code style
* Add responsive panels to the single-column layout
* Fixes
* Fix not being able to save the preference
* Fix code style issues
* Set max-height on the compose textarea and add a link to relationship manager
* Add `from_account` to notifications API
this adds the ability to filter notifications by the account they
originated from
* passing a non-existent user should cause none to be returned
* Fix codeclimate warnings
* fix more codeclimate warnings
* make requested changes:
* use account id instead of user@domain
* name the param `account_id` instead of `from_account`
* Don't use `return` in a lambda
* Change note_length validator from 160 to 500
* Change input maxlength from 160 to 500
* update bio test from 160 to 500
* Multiply a string 30 times instead of 10
* Record account suspend/silence time and keep track of domain blocks
* Also unblock users who were suspended/silenced before dates were recorded
* Add tests
* Keep track of suspending date for users suspended through the CLI
* Show accurate number of accounts that would be affected by unsuspending an instance
* Change migration to set silenced_at and suspended_at
* Revert "Also unblock users who were suspended/silenced before dates were recorded"
This reverts commit a015c65d2d.
* Switch from using suspended and silenced to suspended_at and silenced_at
* Add post-deployment migration script to remove `suspended` and `silenced` columns
* Use Account#silence! and Account#suspend! instead of updating the underlying property
* Add silenced_at and suspended_at migration to post-migration
* Change account fabricator to translate suspended and silenced attributes
* Minor fixes
* Make unblocking domains always retroactive
* When trying to block an already-blocked domain, provide a link to the block
* Fix styling for links in flash messages
* Allow blocks to be upgraded but not downgraded
* Check that an invite link is valid before bypassing approval mode
Fixes#10656
* Add tests
* Only consider valid invite links in registration controller
* fixup
* Add blurhash
* Use fallback color for spoiler when blurhash missing
* Federate the blurhash and accept it as long as it's at most 5x5
* Display unknown media attachments as blurhash placeholders
* Improve style of embed actions and spoiler button
* Change blurhash resolution from 3x3 to 4x4
* Improve dependency definitions
* Fix code style issues
* Add "why do you want to join" field to invite requests
Fix#10512
* Remove unused translations
* Fix broken registrations when no invite request text is submitted
* Refactor imports
* Export show_reblogs when exporting list of followed users
* Add support for importing show_reblogs with following collection
* Fix tests
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"
This reverts commit 120544067f.
* Revert "Hide blocking accounts from blocked users (#10442)"
This reverts commit 62bafa20a1.
* Improve blocked view of profiles
- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
* Revert "Add indication that you have been blocked in web UI (#10420)"
This reverts commit bd02ec6daa.
* Revert "Add `blocked_by` relationship to the REST API (#10373)"
This reverts commit 9745de883b.
* Hide blocking accounts from search results
* Filter blocking accouts from account followers
* Filter blocking accouts from account's following accounts
* Filter blocking accounts from “reblogged by” and “favourited by” lists
* Remove blocking account from URL search
* Return 410 on trying to fetch user data from a user who blocked us
* Return 410 in /api/v1/account/statuses for suspended or blocking accounts
* Fix status filtering when performing URL search
* Restore some React improvements
Restore some cleanup from bd02ec6daa
* Refactor by adding `without_blocking` scope
* Check if image value is nil? before creating an image
Check if uploaded images aren't nil before creating SiteUpload models
for them.
* Validate presence of file in SiteUpload
* Fix file presence validation
* Fabricate SiteUpload#file
* Add link to Creative Commons license
Red crosses implied that it was bad/unexpected that certain features
were not enabled. In reality, they are options, so showing a green
or grey power-off icon is more appropriate.
Add status of timeline preview as well
Fix sample accounts changing too frequently due to wrong query
Sample accounts are intended to be sorted by popularity
* Backend changes for custom emoji support in poll options
* Serialize poll emojis in REST API
* Render custom emojis in poll options
* Render custom emoji in poll options on public pages
* create account_identity_proofs table
* add endpoint for keybase to check local proofs
* add async task to update validity and liveness of proofs from keybase
* first pass keybase proof CRUD
* second pass keybase proof creation
* clean up proof list and add badges
* add avatar url to keybase api
* Always highlight the “Identity Proofs” navigation item when interacting with proofs.
* Update translations.
* Add profile URL.
* Reorder proofs.
* Add proofs to bio.
* Update settings/identity_proofs front-end.
* Use `link_to`.
* Only encode query params if they exist.
URLs without params had a trailing `?`.
* Only show live proofs.
* change valid to active in proof list and update liveness before displaying
* minor fixes
* add keybase config at well-known path
* extremely naive feature flagging off the identity proof UI
* fixes for rubocop
* make identity proofs page resilient to potential keybase issues
* normalize i18n
* tweaks for brakeman
* remove two unused translations
* cleanup and add more localizations
* make keybase_contacts an admin setting
* fix ExternalProofService my_domain
* use Addressable::URI in identity proofs
* use active model serializer for keybase proof config
* more cleanup of keybase proof config
* rename proof is_valid and is_live to proof_valid and proof_live
* cleanup
* assorted tweaks for more robust communication with keybase
* Clean up
* Small fixes
* Display verified identity identically to verified links
* Clean up unused CSS
* Add caching for Keybase avatar URLs
* Remove keybase_contacts setting
* Add UI for enabling/disabling poll notifications
* Add poll notifications to the (advanced) quick filter bar
* Update poll notification message
“Your poll has ended” → “A poll you have voted in has ended”
* Clear up associated notifications when a poll is deleted
* Fix poll update handler calling method was that was not available
Fix regression from #10209
* Refactor VoteService
* Refactor ActivityPub::DistributePollUpdateWorker and optimize it
* Fix typo
* Fix typo
* Squish username before validation (#10101)
Fix#10101
* Move before_validation hook to a private method
Also add Unicode wite-spaces to the spec to support the use of squish
over strip.
* Process incoming poll tallies update
* Send Update on poll vote
* Do not send Updates for a poll more often than once every 3 minutes
* Include voters in people to notify of results update
* Schedule closing poll worker on poll creation
* Add new notification type for ending polls
* Add front-end support for ended poll notifications
* Fix UpdatePollSerializer
* Fix Updates not being triggered by local votes
* Fix tests failure
* Fix web push notifications for closing polls
* Minor cleanup
* Notify voters of both remote and local polls when those close
* Fix delivery of poll updates to mentioned accounts and voters
* Fix ActivityPub poll results being serialized even with hide_totals
* Fix poll refresh button having a different font size
* Display poll in OpenGraph description
* Fix NoMethodError when serializing votes
Regression from #10158
* Fix polls on public pages being broken for non-logged-in users
* Do not show time remaining if poll has no expiration date
* Add polls
Fix#1629
* Add tests
* Fixes
* Change API for creating polls
* Use name instead of content for votes
* Remove poll validation for remote polls
* Add polls to public pages
* When updating the poll, update options just in case they were changed
* Fix public pages showing both poll and other media
* Fetch up to 5 replies when discovering a new remote status
This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.
* Add some tests for ActivityPub::FetchRepliesWorker
* Add specs for ActivityPub::FetchRepliesService
* Serialize up to 5 public self-replies for ActivityPub notes
* Add specs for ActivityPub::NoteSerializer
* Move exponential backoff logic to a worker concern
* Fetch first page of paginated collections when fetching thread replies
* Add specs for paginated collections in replies
* Move Note replies serialization to a first CollectionPage
The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.
* Use pluck(:uri) instead of map(&:uri) to improve performances
* Fix fetching replies when they are in a CollectionPage
* Add type, limit, offset, min_id, max_id, account_id to search API
Fix#8939
* Make the offset work on accounts and hashtags search as well
* Assure brakeman we are not doing mass assignment here
* Do not allow paginating unless a type is chosen
* Fix search query and index id field on statuses instead of created_at
* Add hashtag filter to profiles
GET /@:username/tagged/:hashtag
GET /api/v1/accounts/:id/statuses?tagged=:hashtag
* Display featured hashtags on public profile
* Use separate model for featured tags
* Update featured hashtag counters on-write
* Limit featured tags to 10
* Add Tombstone model to remember object deletion
* Do not recreate a status if it has been deleted
* Record Tombstone for remote deleted items
Also, only record deleted items from same-host actors
* Clear an user's tombstones when their key change
* Use Contact User as Relay, Report, Subscribe.
* Use Account.representative to fetch contact user.
* Use find_local.
* No reason to use Account.representative in subscribe_service.
* Don't required representative!
* Fallback is included in Account.representative method.
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
* Add local followers page to admin account UI
For moderation, I often find myself wondering who, locally, is following
a remote user. Currently, to see this, I have to go back to the web UI,
paste in their full handle, click their profile, and go to the
"Followers" tab (plus, this information is incidental, and if mastodon
ever decides to resolve all of the follower information, there will be
no place local followers are shown). This PR adds a new page which is
accessible via the "following" count on the admin's account view
page, which shows the local followers. (It has filter parameters for
account location to indicate that only local followers are shown, and
leave room for expansion if mastodon ever decides to store the entire
remote follow list).
* Normalize en.yml
* Add REST API for creating an account
The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.
The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.
The method is rate-limited by IP to 5 requests per 30 minutes.
* Redirect users back to app from confirmation if they were created with an app
* Add tests
* Return 403 on the method if registrations are not open
* Require agreement param to be true in the API when creating an account
* Add moderation warnings
Replace individual routes for disabling, silencing, and suspending
a user, as well as the report update route, with a unified account
action controller that allows you to select an action (none,
disable, silence, suspend) as well as whether it should generate an
e-mail notification with optional custom text. That notification,
with the optional custom text, is saved as a warning.
Additionally, there are warning presets you can configure to save
time when performing the above.
* Use Account#local_username_and_domain
* Add instance-wide setting to disable profile directory
Fixes#9496
When the profile directory is disabled:
- The “discoverable” setting is hidden from users
- The “profile directory” link is not shown on public pages
- /explore returns 404
* Move Setting.profile_directory check to a before_action filter
* Make custom emoji domains case sensitive #9351
* Fixup style in downcase_domain to comply with codeclimate.
* switch if! to unless
* Don't use transactions, operate in batches.
Also revert spurious schema change.
* Add setting to not aggregate reblogs
Fixes#9222
* Handle cases where user is nil in add_to_home and add_to_list
* Add hint for setting_aggregate_reblogs option
* Reword setting_aggregate_reblogs label
* Fix missing variable in directory page title
* Order hashtags by number of people instead of alphabetically
* Add icon to OpenGraph preview of directory page
* Prevent line breaks in hashtags and ensure lowercase in the table
* Use `update_column` instead of `update_attribute` in callback
`update_attribute` would normally cause callbacks to be called.
Called from a callback, it seems to stop further callbacks from executing.
`update_column` does the same work, but without calling callbacks or
preventing other callbacks from executing.
* Fix deadlocks by moving account stats update outside transaction
- Some associations were missing from the clean-up
- Some attributes were not reset on suspension
- Skip federation and streaming deletes when purging a dead domain
- Move account association definitions to concern
* fix: change Identity's id column to a bigint
This appears to be the last model created using a 5.0 migration,
where column types defaulted to `integer` rather than `bigint`.
This migration changes the column type to match that of all of the
other ID columns.
* Change user_id column in identities to bigint and fix down-migration
Fixes#9309
This only allows hyphens in the middle of a username, much like dots,
although I don't have a compelling reason to do so other than keeping
the changes minimal.
* Improve overview of accounts in admin UI
- Display suspended status, role, last activity and IP prominently
- Default to showing local accounts
- Default to not showing suspended accounts
* Remove unused strings
* Fix tests
* Allow filtering accounts by IP mask
* Ensure replied-to is a status not a boost
* Consider case of not a reply
* Add test case for replying to boost
* Move reblog-reply resolution to model
* Remove unnecessary comment
* Nascent tag menu on frontend
* Hook up frontend to search
* Tag intersection backend first pass
* Update yarnlock
* WIP
* Fix for tags not searching correctly
* Make radio buttons function
* Simplify radio buttons with modeOption
* Better naming
* Rearrange options
* Add all/any/none functionality on backend
* Small PR cleanup
* Move to service from scope
* Small cleanup, add proper service tests
* Don't use send with user input :D
* Set appropriate column header
* Handle auto updating timeline
* Fix up toggle function
* Use tag value correctly
* A bit more correct to use 'self' rather than 'all' in status scope
* Fix some style issues
* Fix more code style issues
* Style select dropdown more better
* Only use to_id'ed value to ensure no SQL injection
* Revamp frontend to allow for multiple selects
* Update backend / col header to account for more flexible tagging
* Update brakeman ignore
* Codeclimate suggestions
* Fix presenter tag_url
* Implement initial PR feedback
* Handle additional tag streaming
* CodeClimate tweak
* Reset status cache when status_stat or media_attachment updates
Fix#8711
Media attachments are generally immutable, but admins can update
the sensitive flag, and this would ensure the change is visible
instantly. Same for updates to status stats. That is a regression
from #8185, because even the correct updated_at fetched from a join
doesn't seem to invalidate the cache.
* Remove join from Status#cache_ids since it has no effect
* Add silent column to mentions
* Save silent mentions in ActivityPub Create handler and optimize it
Move networking calls out of the database transaction
* Add "limited" visibility level masked as "private" in the API
Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic
* Ensure silent column is non-null, add spec
* Ensure filters don't check silent mentions for blocks/mutes
As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering
* Clean up code
* Use Status#active_mentions to limit returned mentions
* Fix code style issues
* Use Status#active_mentions in Notification
And remove stream_entry eager-loading from Notification
* Add conversations API
* Add web UI for conversations
* Add test for conversations API
* Add tests for ConversationAccount
* Improve web UI
* Rename ConversationAccount to AccountConversation
* Remove conversations on block and mute
* Change last_status_id to be a denormalization of status_ids
* Add optimistic locking
* Allow min_id pagination in Feed#get
* Add min_id pagination to home and list timeline APIs
* Add min_id pagination to account statuses, public and tag APIs
* Remove unused stub in reports API
* Use min_id pagination in notifications, favourites, and fix order
* Fix HomeFeed#from_database not using paginate_by_id
* Verify link ownership with rel="me"
* Add explanation about verification to UI
* Perform link verifications
* Add click-to-copy widget for verification HTML
* Redesign edit profile page
* Redesign forms
* Improve responsive design of settings pages
* Restore landing page sign-up form
* Fix typo
* Support <link> tags, add spec
* Fix links not being verified on first discovery and passive updates
* Allow moderators to disable/enable login
* Instead of rejecting login, show forbidden error when login disabled
Avoid confusion because when login is rejected, the message is that
the account is not activated, which is wrong.
* Fix tests
* Add default_settings class method to ScopedSettings
ScopedSettings was extended to use value of unscoped setting instead of
only using defaults set in config/settings.yml for selected settings.
This adds possibility for admins to set default values of users' settings,
for example default theme (as requested in #7092).
* Add ability to change an instance default theme
Closes#7092
* Add remote interaction dialog for toots
* Change AuthorizeFollow into AuthorizeInteraction, support statuses
* Update brakeman.ignore
* Adjust how interaction buttons are display on public pages
* Fix tests
Initially I thought there might be valid reasons for remote users to
have a different, unpredicted username format. However, I now realize
such a difference would be unusable and unexpected within Mastodon.
Fix#8058
To improve the way super tall or super ride images are treated, the
numbers remain the same, 1280x1280 and 400x400, but if an image
is less in one dimension than the other, the other can become larger
Thanks to @WAHa_06x36@mastodon.social for the tip
DELETE /api/v1/suggestions/:account_id
When blocking, remove suggestion from both sides. Muting not affected,
since muting is supposed to be invisible to the target.
This replace calls of String#match? with rails Regex#match?
This follows the same idea used to keep Rails 5.2 compatible with Ruby
2.2.2 in https://github.com/rails/rails/pull/32973
* Re-add follow recommendations API
GET /api/v1/suggestions
Removed in 8efa081f21 due to Neo4J
dependency. The algorithm uses triadic closures, takes into account
suspensions, blocks, mutes, domain blocks, excludes locked and moved
accounts, and prefers more recently updated accounts.
* Track interactions with people you don't follow
Replying to, favouriting and reblogging someone you're not following
will make them show up in follow recommendations. The interactions
have different weights:
- Replying is 1
- Favouriting is 10 (decidedly positive interaction, but private)
- Reblogging is 20
Following them, muting or blocking will remove them from the list,
obviously.
* Remove triadic closures, ensure potential friendships are trimmed
* Add keyword filtering
GET|POST /api/v1/filters
GET|PUT|DELETE /api/v1/filters/:id
- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration
* Make sure expired filters don't get applied client-side
* Add missing API methods
* Remove "regex filter" from column settings
* Add tests
* Add test for FeedManager
* Add CustomFilter test
* Add UI for managing filters
* Add streaming API event to allow syncing filters
* Fix tests
While this isn't exactly *wrong*, files uploaded with a “.jpe” extension will
keep that extension, which will often cause them to be served with an
incorrect mimetype.
Old statuses and statuses from Pawoo, which runs a modified version of
Mastodon, may not have been marked sensitive even if spoiler text is
present.
Such statuses are still not marked sensitve if they are local or
arrived before version upgrade. Marking recently fetched remote status
sensitive contradicts the behavior.
Considering what people expected when they authored such statuses, this
change removes the sensitivity enforcement.
Do not touch statuses_count on accounts table when mass-destroying
statuses to reduce load when removing accounts, same for
reblogs_count and favourites_count
Do not count statuses with direct visibility in statuses_count
Fix#828
* optimize direct timeline
* fix typo in class name
* change filter condition for direct timeline
* fix codestyle issue
* revoke index_accounts_not_silenced because direct timeline does not use it.
* revoke index_accounts_not_silenced because direct timeline does not use it.
* fix rspec test condition.
* fix rspec test condition.
* fix rspec test condition.
* revoke adding column and partial index
* (direct timeline) move merging logic to model
* fix pagination parameter
* add method arguments that switches return array of status or cache_ids
* fix order by
* returns ActiveRecord.Relation in default behavor
* fix codestyle issue
* Track trending tags
- Half-life of 1 day
- Historical usage in daily buckets (last 7 days stored)
- GET /api/v1/trends
Fix#271
* Add trends to web UI
* Don't render compose form on search route, adjust search results header
* Disqualify tag from trends if it's in disallowed hashtags setting
* Count distinct accounts using tag, ignore silenced accounts
> Good lord what is happening in there
Previously the contents of the Web Push API payloads closely resembled the structure of JavaScript's [Notification](https://developer.mozilla.org/en-US/docs/Web/API/Notification). But now that the API is open to non-browser apps, and given that there is no required coupling between contents of the payload and a Notification object, here is how I changed the payload:
```json
{
"access_token": "...",
"preferred_locale": "en",
"notification_id": "12345",
"notification_type": "follow",
"title": "So and so followed you",
"body": "This is my bio",
"icon": "https://example.com/avatar.png"
}
```
The title, body and icon attributes are included as a fallback so you can construct a minimal notification if you cannot perform a network request to the API to get more data.
* Add preference to hide following/followers lists
- Public pages
- ActivityPub collections (does not return pages but does give total)
- REST API (unless it's your own) (does not federate)
Fix#6901
* Add preference
* Add delegation
* Fix issue
* Fix issue
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
Same URI passed between follow request and follow, since they are
the same thing in ActivityPub. Local URIs are generated during
creation using UUIDs and are passed to serializers.
* Revert "Fixes/do not override timestamps (#7331)"
This reverts commit 581a5c9d29.
* Document Snowflake ID corner-case a bit more
Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.
Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.
* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
* Add entity cache
Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.
Reduce duplicate text extractions
* Fix code style issue
The goal here isn't to prevent these hashtags from existing, but just to strongly curtail their usage; The hashtags may still exist in the database via federated status, or from being created prior to this feature.
* Update annotate to version 2.7.3
* Update aws-sdk-s3 to version 1.9.2
* Update browser to version 2.5.3
* Update capistrano to version 3.10.2
* Update domain_name to version 0.5.20180417
* Update http to version 3.2.0
* Update lograge to version 0.10.0
* Update oj to version 3.5.1
* Update parallel_tests to version 2.21.3
* Update puma to version 3.11.4
* Update rubocop to version 0.55.0
* Update scss_lint to version 0.57.0
* Update simplecov to version 0.16.1
* Update tty-command to version 0.8.0
* Update tty-prompt to version 0.16.0
* Update pkg-config to version 1.3.0
* Update fog-local to version 0.5.0
* Update fog-openstack to version 0.1.25
* Update devise-two-factor to version 3.0.3
* bundle update
* Force convert to JPG for preview card thumbnails to avoid animations
Fix#7093
* Conditionally convert to JPG only if original is GIF
Coalesce and strip on all formats to ensure no animated APNGs
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL
* Changes to match other timelines in 2.0
* Add bio fields
- Fix#3211
- Fix#232
- Fix#121
* Display bio fields in web UI
* Fix output of links and missing fields
* Federate bio fields over ActivityPub as PropertyValue
* Improve how the fields are stored, add to Edit profile form
* Add rel=me to links in fields
Fix#121
This also limits the statuses returned by API, but pagination is not
implemented in Web API yet. I still expect it brings user experience
better than making a user wait to fetch all ancestor statuses and flooding
the column with them.
* Further improvements to Reports UI
- Clean up notes display
- Clean up add new note form
- Simplify controller
- Allow reopening a report with a note
- Show created at date for reports
- Fix report details table formatting
* Show history of report using Admin::ActionLog beneath the report
* Fix incorrect log message when reopening a report
* Implement fetching of all ActionLog items that could be related to the report
* Ensure adding a report_note updates the report's updated_at
* Limit Report History to actions that happened between the report being created and the report being resolved
* Fix linting issues
* Improve report history builder
Thanks @gargron for the improvements
* Admin: Show unconfirmed email address on account page
* Admin: Allow staff to change user email addresses
* ActionLog: On change_email, log current email address and new unconfirmed email address
- Explicitly specify video codec.
When ffmpeg isn't compiled with libx264 but openh264, mpeg4 is selected as video codec.
- Swap avarage bitrate and max bitrate.
ancestor_statuses and descendant_statuses used to include the root status
itself, but the behavior is confusing because the root status is not
an ancestor nor descendant.
* Implement Assignment of Reports (#6967)
* Change translation of admin.report.comment.label to "Report Comment" for clarity
As we'll soon add the ability for reports to have comments on them, this clarification makes sense.
* Implement notes for Reports
This enables moderators to leave comments about a report whilst they work on it
* Fix display of report moderation notes
* Allow reports to be reopened / marked as unresolved
* Redirect to reports listing upon resolution of report
* Implement "resolve with note" functionality
* Add inverse relationship for report notes
* Remove additional database querying when loading report notes
* Fix tests for reports
* Fix localisations for report notes / reports
Comparison was downcasing only one side, therefore if previously
existing account had a non-lowercase spelling, it would be ignored
when checking for duplicates.
New rake task `mastodon:maintenance:find_duplicate_usernames` will
help find constraint violations that might have occured from the
presence of this bug.
Bump version to 2.3.3
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
Usernames with dots in them do not work with routes, because the dot usually separates the desired page format (e.g. json). I don't want to mess with changing route constraints for this patch release.
* Permit dots in usernames with conditions
- Dot cannot be the start or end of username
- a.lice and al.ice are considered the same during sign-up
* Fix regex mixin flags
* update to new version of devise_pam_authenticatable2
* fix behaviour if suffix is nil, fix environment loading, fix user email creation
* code cleanup/fix linter warning
This will prevent, for example, `rake mastodon:redownload_avatars` from crashing when an instance is no longer responding to connection attempts, instead silently continuing as expected.
* Rescue when there's no extension in the remotable
Sometimes the remotable is pointing to a directory with no file
extension. Maybe it should not be expecting to identify based on
extensions to begin with, but since it's the case, it should be ready
for it.
* Fix codeclimate issue
* Check if filename is nil instead of rescueing exception
Suggestion made in the PR
* Avoid concatenation issue if filename is nil
If filename is nil, extname was undefined
* Invert condition
Address PR comments
* Fix prev/next links on public profile page
* Don't make pagination urls if no available statuses
* Fix empty check method
* Put left chevron before prev page link
* Add scope for pagination "starting at" a given id
* Status pagination try 2:
s/prev/older and s/next/newer
"older" on left, "newer" on right
Use new scope for "newer" link
Extract magic 20 page size to constant
Remove max_id from feed pagination as it's not respected
* Reinstate max_id for accounts atom stream
* normalize
* Ensure the app does not even start if OTP_SECRET is not set
* Remove PAPERCLIP_SECRET (it's not used by anything, actually)
Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them
* Add focus param to media API, center thumbnails on focus point
* Add UI for setting a focal point
* Improve focal point icon on upload item
* Use focal point in upload preview
* Add focalPoint property to ActivityPub
* Don't show focal point button for non-image attachments
* Fix#201: Account archive download
* Export actor and private key in the archive
* Optimize BackupService
- Add conversation to cached associations of status, because
somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
(Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
(Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
(Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
GIF-to-MP4 media are all named "media.mp4" for some reason
* Keep uniquely generated filename in Paperclip::GifTranscoder
* Ensure dumped files do not overwrite each other by maintaing directory partitions
* Give tar archives a good name
* Add scheduler to remove week-old backups
* Fix code style issue
* Fix avatar and header issues by using custom geometry detector
Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.
One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.
Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.
* Only invoke convert if dimension or format needs to be changed
Also don't apply "-quality 80" option which is probably the reason
for slight color differences between original and remote image
(because it would apply it twice, once on original instance, and
again on the receiving instance)
* Add full-text search for authorized statuses
- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index
Fix#5880Fix#4293Fix#1152
* Add commented out docker-compose configuration for ES container
* Optimize index import, filter search results
* Add basic normalization to the index
* Add better stemming and normalization to the index
* Skip webfinger request if search query includes both @ and a space
* Fix code style
* Visually separate search result sections
* Fix code style issues
* Add option to show only local toots in timeline preview
Right know, toots from all the known fediverse are shown in the main
page of an instance. That however doesn't reflect the instance itself.
With this option the admin may choose to display only local toots so
that users checking the instance get a better idea of internal
conversations.
* Fix issues pointed by codeclimate and eslint
* Add default message for community timeline
* Update pl.yml
UserTrackingConcern is circumvented by SessionsController#create
because it calls warden, which calls the User#update_tracked_fields!
method directly. Move returning user logic to that method.
A change introduced in #6125 prevents
`Devise::Models::Confirmable#confirm` from being called for existing
users, which in turn leads to `email` not being set to
`unconfirmed_email`, breaking email updates. This also adds a test
that would've caught this issue.
* Add confirmation step for email changes
This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.
Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.
Fixes#3871
* Add review fixes
* Add GET /api/v1/instance/peers API to reveal known domains
* Add GET /api/v1/instance/activity API
* Make new APIs disableable, exclude private statuses from activity stats
* Fix code style issue
* Fix week timestamps
* Add list of lists component to web UI
* Add list adding
* Add list removing
* List editor modal
* Add API account search limited by following=true relation
* Rework list editor modal
* Remove mandatory pagination of GET /api/v1/lists/:id/accounts
* Adjust search input placeholder
* Fix rspec (#5890)
* i18n: (zh-CN) Add missing translations for #5811 (#5891)
* i18n: (zh-CN) yarn manage:translations -- zh-CN
* i18n: (zh-CN) Add missing translations for #5811
* Fix some issues
- Display loading/missing state for list timelines
- Order lists alphabetically in overview
- Fix async list editor reset
- Redirect to /lists after deleting unpinned list
- Redirect to / after pinning a list
* Remove dead list columns when a list is deleted or fetch returns 404
* Avoid sending explicit Undo->Announce when original deleted
* Do not forward a reply back to the server that sent it
* Deduplicate inboxes of rebloggers' followers for delete forwarding
* Adjust test
* Fix wrong class, bad SQL, wrong variable, outdated comment
* Allow hiding of reblogs from followed users
This adds a new entry to the account menu to allow users to hide
future reblogs from a user (and then if they've done that, to show
future reblogs instead).
This does not remove or add historical reblogs from/to the user's
timeline; it only affects new statuses.
The API for this operates by sending a "reblogs" key to the follow
endpoint. If this is sent when starting a new follow, it will be
respected from the beginning of the follow relationship (even if
the follow request must be approved by the followee). If this is
sent when a follow relationship already exists, it will simply
update the existing follow relationship. As with the notification
muting, this will now return an object ({reblogs: [true|false]}) or
false for each follow relationship when requesting relationship
information for an account. This should cause few issues due to an
object being truthy in many languages, but some modifications may
need to be made in pickier languages.
Database changes: adds a show_reblogs column (default true,
non-nullable) to the follows and follow_requests tables. Because
these are non-nullable, we use the existing MigrationHelpers to
perform this change without locking those tables, although the
tables are likely to be small anyway.
Tests included.
See also <https://github.com/glitch-soc/mastodon/pull/212>.
* Rubocop fixes
* Code review changes
* Test fixes
This patchset closes#648 and resolves#3271.
* Rubocop fix
* Revert reblogs defaulting in argument, fix tests
It turns out we needed this for the same reason we needed it in muting:
if nil gets passed in somehow (most usually by an API client not passing
any value), we need to detect and handle it.
We could specify a default in the parameter and then also catch nil, but
there's no great reason to duplicate the default value.
* Add consumable invites
* Add UI for generating invite codes
* Add tests
* Display max uses and expiration in invites table, delete invite
* Remove unused column and redundant validator
- Default follows not used, probably bad idea
- InviteCodeValidator is redundant because RegistrationsController
checks invite code validity
* Add admin setting to disable invites
* Add admin UI for invites, configurable role for invite creation
- Admin UI that lists everyone's invites, always available
- Admin setting min_invite_role to control who can invite people
- Non-admin invite UI only visible if users are allowed to
* Do not remove invites from database, expire them instantly
