1
0
puyopuyotetr.is/app
David Yip 4a64181461 Allow keywords to match either substrings or whole words.
Word-boundary matching only works as intended in English and languages
that use similar word-breaking characters; it doesn't work so well in
(say) Japanese, Chinese, or Thai.  It's unacceptable to have a feature
that doesn't work as intended for some languages.  (Moreso especially
considering that it's likely that the largest contingent on the Mastodon
bit of the fediverse speaks Japanese.)

There are rules specified in Unicode TR29[1] for word-breaking across
all languages supported by Unicode, but the rules deliberately do not
cover all cases.  In fact, TR29 states

    For example, reliable detection of word boundaries in languages such
    as Thai, Lao, Chinese, or Japanese requires the use of dictionary
    lookup, analogous to English hyphenation.

So we aren't going to be able to make word detection work with regexes
within Mastodon (or glitchsoc).  However, for a first pass (even if it's
kind of punting) we can allow the user to choose whether they want word
or substring detection and warn about the limitations of this
implementation in, say, docs.

[1]: https://unicode.org/reports/tr29/
     https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-21 14:54:36 -05:00
..
controllers Spike out index and new views for keyword mutes controller. 2017-10-21 14:54:36 -05:00
helpers Set up /settings/keyword_mutes. #164. 2017-10-21 14:54:36 -05:00
javascript Compose buttons bar redesign + generalize dropdown (#194) 2017-10-21 20:24:53 +02:00
lib Rework KeywordMute interface to use a matcher object; spec out matcher. #164. 2017-10-21 14:54:36 -05:00
mailers Supply @instance variable in password reset instructions / password change mailer (#5215) 2017-10-04 13:25:24 +02:00
models Allow keywords to match either substrings or whole words. 2017-10-21 14:54:36 -05:00
policies try to tighten up local only toot stuff, like... properly (#163) 2017-10-09 09:56:17 -04:00
presenters Merge commit 'b9f59ebcc68e9da0a7158741a1a2ef3564e1321e' into merging-upstream 2017-09-28 09:18:35 +02:00
serializers Merge remote-tracking branch 'upstream/master' into gs-master 2017-10-16 09:23:59 -05:00
services Merge tag 'v2.0.0' into gs-master 2017-10-18 11:52:04 -05:00
validators Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
views Spike out index and new views for keyword mutes controller. 2017-10-21 14:54:36 -05:00
workers Clean up reblog tracking keys, related improvements (#5428) 2017-10-17 11:45:06 +02:00