92 commits

Author	SHA1	Message	Date
Claire	379115e601	Add SELF_DESTRUCT env variable to process self-destructions in the background (#26439)	2023-10-23 15:46:21 +00:00
Claire	4e55001e65	Add a short-lived lock to trend refresh scheduler (#27253)	2023-10-03 12:21:32 +02:00
Claire	5e56f3db65	Change some worker lock TTLs (#27246)	2023-10-03 10:09:00 +02:00
Claire	16681e0f20	Add admin notifications for new Mastodon versions (#26582)	2023-09-01 17:47:07 +02:00
Claire	5c0a9aac3b	Revert to using primary database in IndexingScheduler (#26754)	2023-09-01 15:09:44 +02:00
Eugen Rochko	e263db276f	Change indexing jobs to use database replica (#26692)	2023-08-27 22:38:01 +02:00
jsgoldstein	30c191aaa0	Add new public status index (#26344) ... Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com>	2023-08-24 16:40:04 +02:00
Eugen Rochko	a0fad5c8bb	Change indexing frequency from 5 minutes to 1 minute, add locks to schedulers (#26304)	2023-08-03 11:04:05 +02:00
Claire	b4e739ff0f	Change interaction modal in web UI (#26075) ... Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>	2023-07-27 16:11:17 +02:00
Claire	144a406d33	Clean up unused application records (#24871)	2023-07-21 13:13:16 +02:00
Emelia Smith	3a91603b15	Prevent UserCleanupScheduler from overwhelming streaming (#25519)	2023-06-20 18:04:35 +02:00
Claire	c3c7d59072	Fix infinite loop in AccountsStatusesCleanupScheduler (#24840)	2023-05-04 10:13:00 +02:00
Claire	8b636a29c6	Improve automatic post cleanup worker performances (#24785)	2023-05-03 10:31:40 +02:00
Claire	32a030dd74	Rewrite import feature (#21054)	2023-05-02 12:08:48 +02:00
Claire	5dc3173ef8	Fix AccountsStatusesCleanupScheduler not spreading deletes across accounts correctly (#24607)	2023-04-23 22:25:40 +02:00
Claire	501d6197c4	Change automatic post deletion thresholds and load detection (#24614)	2023-04-21 18:14:19 +02:00
Vyr Cossont	d5ad8b6422	Fix Redis client and type errors introduced in #24285 (#24342)	2023-04-01 08:28:35 +02:00
Vyr Cossont	7646ad8a2b	IndexingScheduler: fetch and import in batches (#24285) ... Co-authored-by: Claire <claire.github-309c@sitedethib.com>	2023-03-31 14:38:47 +02:00
Jean byroot Boussier	922837dc96	Upgrade to latest redis-rb 4.x and fix deprecations (#23616) ... Co-authored-by: Jean Boussier <jean.boussier@gmail.com>	2023-03-04 16:38:28 +01:00
Claire	20b80c62ff	Change auto-deletion throttling constants to better scale with server size (#23320)	2023-02-23 16:52:40 +01:00
Nick Schonning	717683d1c3	Autofix Rubocop remaining Layout rules (#23679)	2023-02-20 06:58:28 +01:00
Nick Schonning	21bf326356	Autofix Rubocop Rails/Pluck (#23730)	2023-02-20 02:28:40 +01:00
Nick Schonning	2177daeae9	Autofix Rubocop Style/RedundantBegin (#23703)	2023-02-19 07:09:40 +09:00
Claire	9edefc779f	Fix UserCleanupScheduler crash when an unconfirmed account has a moderation note (#23318) ... * Fix `UserCleanupScheduler` crash when an unconfirmed account has a moderation note * Add tests	2023-02-07 01:14:44 +01:00
luzpaz	596923da4a	Fix typos in source documentation (#21046) ... Fixed 2 source comment/documentation typos	2022-12-15 15:57:26 +01:00
Claire	5333447be0	Change account deletion requests to spread out over time (#20222)	2022-11-09 14:08:19 +01:00
Eugen Rochko	d0c9ac3919	Fix indexing scheduler trying to index when Elasticsearch is disabled (#19805) ... Fix #19646	2022-11-05 22:31:52 +01:00
Claire	03d9618595	Fix UserCleanupScheduler crash when an unconfirmed account has a moderation note (#19629) ... Fixes #19109	2022-11-01 12:59:23 +01:00
Eugen Rochko	bf0ab3e0fa	Fix vacuum scheduler missing lock, locks never expiring (#19458) ... Remove vacuuming of orphaned preview cards	2022-10-26 12:10:48 +02:00
Eugen Rochko	5c9abdeff1	Add retention policy for cached content and media (#19232)	2022-09-27 03:08:19 +02:00
Eugen Rochko	0b3e4fd5de	Remove digest e-mails (#17985) ... * Remove digest e-mails * Remove digest-related code	2022-08-25 23:38:22 +02:00
Jeremy Kescher	25e076505e	ip_cleanup_scheduler: Make IP and session retention configurable (#18757)	2022-07-07 03:14:28 +02:00
Eugen Rochko	679b7158e3	Change search indexing to use batches to minimize resource usage (#18451)	2022-05-18 23:29:14 +02:00
Eugen Rochko	f6d35ed57d	Remove IP matching from e-mail domain blocks (#18190) ... Clear out e-mail domain blocks created from automatically resolved DNS records	2022-04-29 23:27:03 +02:00
Eugen Rochko	3917353645	Fix single Redis connection being used across all threads (#18135) ... * Fix single Redis connection being used across all Sidekiq threads * Fix tests	2022-04-28 17:47:34 +02:00
Jeong Arm	3349cbeb3e	Fix clean_discarded_statuses (#17822)	2022-03-19 09:24:08 +01:00
Eugen Rochko	50ea54b3ed	Change authorized applications page (#17656) ... * Change authorized applications page * Hide revoke button for superapps and suspended accounts * Clean up db/schema.rb	2022-03-01 16:48:58 +01:00
Claire	3d60708508	Fix crash in EmailDomainBlockRefreshScheduler (#17649)	2022-02-26 16:28:28 +01:00
Eugen Rochko	27965ce5ed	Add trending statuses (#17431) ... * Add trending statuses * Fix dangling items with stale scores in localized sets * Various fixes and improvements - Change approve_all/reject_all to approve_accounts/reject_accounts - Change Trends::Query methods to not mutate the original query - Change Trends::Query#skip to offset - Change follow recommendations to be refreshed in a transaction * Add tests for trending statuses filtering behaviour * Fix not applying filtering scope in controller	2022-02-25 00:34:14 +01:00
Eugen Rochko	a29a982eaa	Change e-mail domain blocks to block IPs dynamically (#17635) ... * Change e-mail domain blocks to block IPs dynamically * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>	2022-02-24 17:28:23 +01:00
Eugen Rochko	35850f8195	Fix localization of cold-start follow recommendations (#17479)	2022-02-08 01:53:49 +01:00
Claire	52c1b86964	Fix Ruby 2.5 incompatibility (#17465)	2022-02-07 19:57:06 +01:00
Claire	03d59340da	Fix Sidekiq warnings about JSON serialization (#17381) ... * Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.	2022-01-28 00:43:56 +01:00
Eugen Rochko	14f436c457	Add notifications for statuses deleted by moderators (#17204)	2022-01-17 09:41:33 +01:00
Eugen Rochko	8e84ebf0cb	Remove IP tracking columns from users table (#16409)	2022-01-16 13:23:50 +01:00
Takeshi Umeda	bda8e4f815	Fix follow recommendation biased towards older accounts (#17126)	2021-12-13 23:21:14 +01:00
Eugen Rochko	6e50134a42	Add trending links (#16917) ... * Add trending links * Add overriding specific links trendability * Add link type to preview cards and only trend articles Change trends review notifications from being sent every 5 minutes to being sent every 2 hours Change threshold from 5 unique accounts to 15 unique accounts * Fix tests	2021-11-25 13:07:38 +01:00
Claire	4ac78e2a06	Add feature to automatically delete old toots (#16529) ... * Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate	2021-08-09 23:11:50 +02:00
Eugen Rochko	d174d12c83	Add authentication history (#16408)	2021-06-21 17:07:30 +02:00
Claire	d9ae3db8d5	Improve performance of follow recommendation scheduler (#16159) ... Express follow_recommendations in terms of account_summaries rather than accounts, integrate filters that are unconditionally used, and materialize the resulting view. This should result in the bulk of the computation being performed only once instead of **once per recommendation language**.	2021-05-05 22:04:52 +02:00