This repository has been archived on 2023-03-18. You can view files and clone it, but cannot push or open issues or pull requests.
osr-discourse-src/spec/requests
Martin Brennan d295a16dab
FEATURE: Uppy direct S3 multipart uploads in composer (#14051)
This pull request introduces the endpoints required, and the JavaScript functionality in the `ComposerUppyUpload` mixin, for direct S3 multipart uploads. There are four new endpoints in the uploads controller:

* `create-multipart.json` - Creates the multipart upload in S3 along with an `ExternalUploadStub` record, storing information about the file in the same way as `generate-presigned-put.json` does for regular direct S3 uploads
* `batch-presign-multipart-parts.json` - Takes a list of part numbers and the unique identifier for an `ExternalUploadStub` record, and generates the presigned URLs for those parts if the multipart upload still exists and if the user has permission to access that upload
* `complete-multipart.json` - Completes the multipart upload in S3. Needs the full list of part numbers and their associated ETags which are returned when the part is uploaded to the presigned URL above. Only works if the user has permission to access the associated `ExternalUploadStub` record and the multipart upload still exists.

  After we confirm the upload is complete in S3, we go through the regular `UploadCreator` flow, the same as `complete-external-upload.json`, and promote the temporary upload S3 into a full `Upload` record, moving it to its final destination.
* `abort-multipart.json` - Aborts the multipart upload on S3 and destroys the `ExternalUploadStub` record if the user has permission to access that upload.

Also added are a few new columns to `ExternalUploadStub`:

* multipart - Whether or not this is a multipart upload
* external_upload_identifier - The "upload ID" for an S3 multipart upload
* filesize - The size of the file when the `create-multipart.json` or `generate-presigned-put.json` is called. This is used for validation.

When the user completes a direct S3 upload, either regular or multipart, we take the `filesize` that was captured when the `ExternalUploadStub` was first created and compare it with the final `Content-Length` size of the file where it is stored in S3. Then, if the two do not match, we throw an error, delete the file on S3, and ban the user from uploading files for N (default 5) minutes. This would only happen if the user uploads a different file than what they first specified, or in the case of multipart uploads uploaded larger chunks than needed. This is done to prevent abuse of S3 storage by bad actors.

Also included in this PR is an update to vendor/uppy.js. This has been built locally from the latest uppy source at d613b849a6. This must be done so that I can get my multipart upload changes into Discourse. When the Uppy team cuts a proper release, we can bump the package.json versions instead.
2021-08-25 08:46:54 +10:00
..
admin FIX: Revoking admin or moderator status doesn't require refresh to delete/anonymize/merge user (#14073) 2021-08-19 09:57:16 +08:00
api DEV: Fix some openapi spec issues (#14037) 2021-08-13 04:22:15 -06:00
about_controller_spec.rb FIX: Correct user profile URLs in /about crawler view 2020-07-14 16:09:27 +01:00
application_controller_spec.rb PERF: Eager load Theme associations in Stylesheet Manager. 2021-06-21 11:06:58 +08:00
associate_accounts_controller_spec.rb DEV: Improve robustness of associate_accounts_controller 2021-08-10 15:07:40 +01:00
badges_controller_spec.rb FEATURE: add noindex header to badges, groups, and /my pages (#9736) 2020-05-11 15:05:42 +10:00
bookmarks_controller_spec.rb FIX: When user has already hit bookmark limit, do not error for clear_reminder! or other updates (#12658) 2021-04-09 13:06:35 +10:00
bootstrap_controller_spec.rb FIX: allows authentication data to be present in bootstrap (#13885) 2021-07-29 15:01:11 +02:00
categories_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
clicks_controller_spec.rb DEV: Fix failling test. 2019-05-07 11:19:13 +03:00
composer_messages_controller_spec.rb DEV: Use response.parsed_body in specs (#9615) 2020-05-07 17:04:12 +02:00
csp_reports_controller_spec.rb DEV: Only include "report-sample" CSP directive when reporting is enabled (#9337) 2020-04-02 11:16:38 -04:00
directory_columns_controller_spec.rb DEV: Plugin API to add directory columns (#13440) 2021-06-22 13:00:04 -05:00
directory_items_controller_spec.rb FIX: Include user_field_ids in pagination URL for directory items (#13569) 2021-06-29 14:43:38 -05:00
do_not_disturb_controller_spec.rb DEV: Replace 'processed' column on notifications with new table (#11864) 2021-01-27 10:29:24 -06:00
draft_controller_spec.rb FIX: saving drafts unconditionally increases sequence 2020-05-12 16:55:42 +10:00
drafts_controller_spec.rb DEV: Use response.parsed_body in specs (#9615) 2020-05-07 17:04:12 +02:00
email_controller_spec.rb FIX: Show Uncategorized when unsubscribing (#13832) 2021-07-26 12:19:30 +10:00
embed_controller_spec.rb FIX: Better and more secure validation of periods for TopicQuery 2021-07-23 14:24:44 -04:00
exceptions_controller_spec.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
export_csv_controller_spec.rb DEV: Switch to new ExportUserArchive job 2020-08-28 11:46:53 -07:00
extra_locales_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
finish_installation_controller_spec.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
forums_controller_spec.rb FEATURE: Allow a cluster_name to be configured and used for /srv/status (#12365) 2021-03-15 15:41:59 +11:00
groups_controller_spec.rb REVERT "FIX: do not show private group flair on user avatars" (#13991) 2021-08-10 17:25:11 +05:30
hashtags_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
inline_onebox_controller_spec.rb DEV: Use response.parsed_body in specs (#9615) 2020-05-07 17:04:12 +02:00
invites_controller_spec.rb FIX: Let staged users choose their username (#13678) 2021-07-12 07:57:38 +10:00
list_controller_spec.rb DEV: Return 400 instead of 500 for invalid top period (#13828) 2021-07-23 14:58:10 -06:00
metadata_controller_spec.rb PERF: cache all metadata for 60 seconds 2020-07-01 12:58:02 +10:00
notifications_controller_spec.rb FIX: Typo in NotificationsController#index not caught by tests. 2020-07-22 09:22:26 +08:00
offline_controller_spec.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
omniauth_callbacks_controller_spec.rb FEATURE: Allow linking an existing account during external-auth signup 2021-08-10 15:07:40 +01:00
onebox_controller_spec.rb FEATURE: Onebox local categories (#11311) 2020-11-25 10:53:05 +11:00
permalinks_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
post_action_users_controller_spec.rb DEV: Cleanup ignored user logic (#11107) 2020-11-03 12:38:54 +00:00
post_actions_controller_spec.rb DEV: Use response.parsed_body in specs (#9615) 2020-05-07 17:04:12 +02:00
post_readers_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
posts_controller_spec.rb FIX: Deprecated method should still behave the same. (#14067) 2021-08-19 09:58:26 +08:00
published_pages_controller_spec.rb FIX: Do not enable published page if secure media enabled (#11131) 2020-11-06 10:33:19 +10:00
push_notification_controller_spec.rb DEV: Prefabrication (test optimization) (#7414) 2019-05-07 13:12:20 +10:00
qunit_controller_spec.rb FIX: Theme tests should work in production (#13333) 2021-06-08 22:03:59 +03:00
reviewable_claimed_topics_controller_spec.rb FIX: Make reviewable claiming work with deleted topics (#9040) 2020-02-25 15:49:23 +02:00
reviewables_controller_spec.rb FEATURE: Blocking is optional when deleting a user from the review queue. (#13375) 2021-06-15 12:35:45 -03:00
robots_txt_controller_spec.rb FEATURE: explicitly ban outlier traffic sources in robots.txt (#11553) 2020-12-23 08:51:14 +11:00
safe_mode_controller_spec.rb Code review comments. 2021-06-21 11:06:58 +08:00
search_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
session_controller_spec.rb UX: suspend forever time period messages (#13776) 2021-07-20 14:42:08 +04:00
similar_topics_controller_spec.rb FIX: reindex_search job should work on model with no search data (#11819) 2021-01-25 11:23:36 +01:00
site_controller_spec.rb DEV: Include login_required attribute in basic info endpoint (#14064) 2021-08-17 14:05:51 -04:00
static_controller_spec.rb DEV: apply allow origin response header for CDN requests. (#11893) 2021-01-29 07:44:49 +05:30
steps_controller_spec.rb DEV: use #frozen_string_literal: true on all spec 2019-04-30 10:27:42 +10:00
stylesheets_controller_spec.rb PERF: Eager load Theme associations in Stylesheet Manager. 2021-06-21 11:06:58 +08:00
svg_sprite_controller_spec.rb FIX: Use absolute URL when redirecting SVG sprite path. 2021-06-30 11:25:05 +08:00
tag_groups_controller_spec.rb DEV: Improve tag groups test (#12848) 2021-04-27 14:05:45 +03:00
tags_controller_spec.rb FEATURE: New and Unread messages for user personal messages. (#13603) 2021-08-02 12:41:41 +08:00
theme_javascripts_controller_spec.rb DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
topics_controller_spec.rb FEATURE: revert disallowing putting URLs in titles for TL0 users (#13970) 2021-08-06 20:07:42 +04:00
uploads_controller_multisite_spec.rb DEV: Isolate multisite specs (#13634) 2021-07-07 18:57:42 +02:00
uploads_controller_spec.rb FEATURE: Uppy direct S3 multipart uploads in composer (#14051) 2021-08-25 08:46:54 +10:00
user_actions_controller_spec.rb FEATURE - SiteSetting to disable user option to hide their profiles and presences (#10885) 2020-10-09 17:18:44 -04:00
user_api_keys_controller_spec.rb DEV: Move UserApiKey scopes to dedicated table (#10704) 2020-09-29 10:57:48 +01:00
user_avatars_controller_spec.rb DEV: Remove the remaining Travis code (#13255) 2021-06-02 20:29:47 +02:00
user_badges_controller_spec.rb FIX: simplify and improve choosing favorite badges (#13743) 2021-07-16 11:13:00 +08:00
users_controller_spec.rb FEATURE: Make allow_uploaded_avatars accept TL (#14091) 2021-08-24 10:46:28 +03:00
users_email_controller_spec.rb No need to disable rate limiter after running tests (#13093) 2021-05-19 16:04:35 +04:00
webhooks_controller_spec.rb Revert "Revert "Merge branch 'master' of https://github.com/discourse/discourse"" 2020-05-23 00:56:13 -04:00
wizard_controller_spec.rb DEV: Use response.parsed_body in specs (#9615) 2020-05-07 17:04:12 +02:00