osr-plastic/osr-discourse-src
Archived
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
This repository has been archived on 2023-03-18. You can view files and clone it, but cannot push or open issues or pull requests.
a03ae9b323
osr-discourse-src/lib/tasks
History
Martin Brennan faf5b4d3e9
PERF: Speed up secure media and ACL sync rake tasks (#16849) 
Incorporates learnings from /t/64227:

* Changes the code to set access control posts in the rake
  task to be an efficient UPDATE SQL query.
  The original version was timing out with 312017 post uploads,
  the new query took ~3s to run.
* Changes the code to mark uploads as secure/not secure in
  the rake task to be an efficient UPDATE SQL query rather than
  using UploadSecurity. This took a very long time previously,
  and now takes only a few seconds.
* Spread out ACL syncing for uploads into jobs with batches of
  100 uploads at a time, so they can be parallelized instead
  of having to wait ~1.25 seconds for each ACL to be changed
  in S3 serially.

One issue that still remains is post rebaking. Doing this serially
is painfully slow. We have a way to do this in sidekiq via PeriodicalUpdates
but this is limited by max_old_rebakes_per_15_minutes. It would
be better to fan this rebaking out into jobs like we did for the
ACL sync, but that should be done in another PR.
2022-05-23 13:14:11 +10:00
..
add_topic_to_quotes.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
admin.rake DEV: Use strings for :user_email job type argument 2022-02-04 18:28:18 +00:00
annotate.rake DEV: Add annotate rake tasks, and enforce via GitHub actions 2021-07-06 10:11:06 +01:00
api.rake FEATURE: Hash API keys in the database (#8438) 2019-12-12 11:45:00 +00:00
assets.rake DEV: Fix production sourcemaps with Ember CLI (#16707) 2022-05-11 10:23:32 +01:00
auto_annotate_models.rake DEV: Allow annotations to work in symlinked plugins, add binstub (#13635) 2021-07-05 15:43:10 +01:00
autospec.rake DEV: Add API docs for uploads and API doc watcher (#15387) 2021-12-23 08:40:15 +10:00
avatars.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
backfill.thor DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
categories.rake FIX: rake categories:move_topics shouldn't move category description topic (#14797) 2021-11-03 11:27:51 +08:00
cdn.rake DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
db.rake FIX: Show error message if extensions cannot be created (#16719) 2022-05-11 15:46:49 +03:00
destroy.rake REFACTOR: Cleanup rake tasks based on feedback 2019-07-18 19:15:01 -06:00
dev.rake DEV: move discourse_dev gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
docker.rake DEV: Add SKIP_DB_CREATE option to docker:test (#16225) 2022-03-21 13:21:06 +01:00
emails.rake DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
emoji.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
export.rake FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
groups.rake Make rubocop happy. 2019-05-13 09:55:44 +08:00
i18n.rake DEV: Update i18n:check rake task to detect invalid Markdown links (#13728) 2021-07-14 13:26:12 +02:00
images.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
import.rake FEATURE: Add last visit indication to topic view page. (#13471) 2021-07-05 14:17:31 +08:00
incoming_emails.rake FEATURE: Remove attachments and truncate raw field for incoming emails (#8253) 2019-10-30 16:54:35 +11:00
integration.rake DEV: Remove hints of jshint (#14828) 2021-11-08 09:12:37 +08:00
javascript.rake FIX: uses tippy for popover (#15409) 2022-05-02 17:10:26 +02:00
log.rake FIX: whoops, linting 2020-09-03 12:10:11 +01:00
maxminddb.rake FIX: maxminddb:get task no longer working 2019-10-04 09:47:33 +10:00
plugin.rake DEV: Improve plugin:versions task (#16391) 2022-04-06 18:29:39 +02:00
populate.rake DEV: Add ability to populate private messages (#16705) 2022-05-12 14:29:49 -04:00
populate.thor DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
posts.rake FIX: LocalJumpError in rake tasks (#14874) 2021-11-10 16:53:55 +01:00
qunit.rake DEV: Allow running theme-qunit tests via testem (#16540) 2022-04-22 15:04:01 +01:00
redis.rake FIX: add support for pipelined and multi redis commands (#16682) 2022-05-10 08:19:02 +10:00
release_note.rake DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
rspec.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
s3.rake DEV: Ensure Ember CLI sourcemaps are uploaded to S3 (#15893) 2022-02-10 19:00:47 +00:00
scheduler.rake FIX: DistributedMutex (#7953) 2019-08-01 09:12:05 +01:00
search.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
site_settings.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
site.rake DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
smoke_test.rake DEV: Minor linting fixes (#16359) 2022-04-02 21:15:38 +02:00
svg_icons.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00
tags.rake FEATURE: Support append when bulk tagging via rake (#9978) 2020-06-04 09:33:48 -04:00
themes.rake DEV: Only raise rake themes:update errors when flag provided (#16254) 2022-03-22 17:02:14 +00:00
topics.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
turbo.rake DEV: Add a verbose option to ./bin/turbo_rspec 2019-06-27 15:49:21 +01:00
typepad.thor FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
uploads.rake PERF: Speed up secure media and ACL sync rake tasks (#16849) 2022-05-23 13:14:11 +10:00
users.rake DEV: enable frozen string literal on all files 2019-05-13 09:31:32 +08:00