This repository has been archived on 2023-03-18. You can view files and clone it, but cannot push or open issues or pull requests.
osr-discourse-src/app/assets/javascripts
Martin Brennan d295a16dab
FEATURE: Uppy direct S3 multipart uploads in composer (#14051)
This pull request introduces the endpoints required, and the JavaScript functionality in the `ComposerUppyUpload` mixin, for direct S3 multipart uploads. There are four new endpoints in the uploads controller:

* `create-multipart.json` - Creates the multipart upload in S3 along with an `ExternalUploadStub` record, storing information about the file in the same way as `generate-presigned-put.json` does for regular direct S3 uploads
* `batch-presign-multipart-parts.json` - Takes a list of part numbers and the unique identifier for an `ExternalUploadStub` record, and generates the presigned URLs for those parts if the multipart upload still exists and if the user has permission to access that upload
* `complete-multipart.json` - Completes the multipart upload in S3. Needs the full list of part numbers and their associated ETags which are returned when the part is uploaded to the presigned URL above. Only works if the user has permission to access the associated `ExternalUploadStub` record and the multipart upload still exists.

  After we confirm the upload is complete in S3, we go through the regular `UploadCreator` flow, the same as `complete-external-upload.json`, and promote the temporary upload S3 into a full `Upload` record, moving it to its final destination.
* `abort-multipart.json` - Aborts the multipart upload on S3 and destroys the `ExternalUploadStub` record if the user has permission to access that upload.

Also added are a few new columns to `ExternalUploadStub`:

* multipart - Whether or not this is a multipart upload
* external_upload_identifier - The "upload ID" for an S3 multipart upload
* filesize - The size of the file when the `create-multipart.json` or `generate-presigned-put.json` is called. This is used for validation.

When the user completes a direct S3 upload, either regular or multipart, we take the `filesize` that was captured when the `ExternalUploadStub` was first created and compare it with the final `Content-Length` size of the file where it is stored in S3. Then, if the two do not match, we throw an error, delete the file on S3, and ban the user from uploading files for N (default 5) minutes. This would only happen if the user uploads a different file than what they first specified, or in the case of multipart uploads uploaded larger chunks than needed. This is done to prevent abuse of S3 storage by bad actors.

Also included in this PR is an update to vendor/uppy.js. This has been built locally from the latest uppy source at d613b849a6. This must be done so that I can get my multipart upload changes into Discourse. When the Uppy team cuts a proper release, we can bump the package.json versions instead.
2021-08-25 08:46:54 +10:00
..
admin DEV: updates chart.js to 3.5.1 (#14107) 2021-08-23 13:49:49 +02:00
confirm-new-email DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
discourse FEATURE: Uppy direct S3 multipart uploads in composer (#14051) 2021-08-25 08:46:54 +10:00
discourse-common DEV: Prevent npm usage (#13945) 2021-08-04 22:04:58 +02:00
discourse-hbr DEV: Prevent npm usage (#13945) 2021-08-04 22:04:58 +02:00
discourse-widget-hbs DEV: Prevent npm usage (#13945) 2021-08-04 22:04:58 +02:00
docs DEV: enforces eslint’s curly rule to the codebase (#10720) 2020-09-22 16:28:28 +02:00
ember-addons DEV: Remove ember-addons (#9559) 2020-04-28 10:14:49 -04:00
locales DEV: Correct typos and spelling mistakes (#12812) 2021-05-21 11:43:47 +10:00
pretty-text SECURITY: Sanitize d-popover attributes (#13958) 2021-08-05 16:39:17 +03:00
select-kit FIX: sk3 wizard regressions (#14120) 2021-08-23 19:57:42 +02:00
truth-helpers DEV: Prevent npm usage (#13945) 2021-08-04 22:04:58 +02:00
wizard DEV: Use key over the deprecated keyCode (#13795) 2021-08-09 11:41:36 +02:00
.npmrc DEV: Prevent npm usage (#13945) 2021-08-04 22:04:58 +02:00
activate-account.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
admin.js.erb FIX: Load .js files from plugins in qunit testing env (#11304) 2020-12-03 10:25:42 -06:00
app-boot.js REFACTOR: Sync up master with changes for Ember-CLI (#11671) 2021-01-12 10:13:21 -05:00
application.js FEATURE: Improve group email settings UI (#13083) 2021-05-28 09:28:18 +10:00
auto-redirect.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
browser-detect.js Improve support for old browsers (#9515) 2020-04-29 21:40:21 +03:00
browser-update.js FIX: browser-update should work with old browsers (#12436) 2021-03-18 19:09:01 +02:00
discourse-loader.js DEV: Don't use the Ember global (#13535) 2021-06-29 19:54:18 +02:00
discourse-shims.js FEATURE: Uppy direct S3 multipart uploads in composer (#14051) 2021-08-25 08:46:54 +10:00
embed-application.js DEV: prettier 2.2.1 (#11862) 2021-01-27 12:39:20 +01:00
ember_include.js.erb move template compiler into vendor for now 2015-11-25 15:54:54 +11:00
ember_jquery.js Revert "Revert "DEV: Import MessageBus from message-bus-client instead of globals"" 2020-05-28 08:42:36 -04:00
env.js FIX: attempts to fix an error with EXTEND_PROTOTYPES (#7067) 2019-02-26 11:04:05 +01:00
google-tag-manager.js FEATURE: Implement nonces for Google Tag Manager integration (#12531) 2021-03-26 11:19:31 -04:00
google-universal-analytics-v3.js FEATURE: Add Google Universal Analytics v4 as an option (#11123) 2020-11-06 14:15:36 -06:00
google-universal-analytics-v4.js FEATURE: Add Google Universal Analytics v4 as an option (#11123) 2020-11-06 14:15:36 -06:00
handlebars-shim.js FIX: It seems sometimes shims are evaluated by older JS engines (#11813) 2021-01-22 10:41:01 -05:00
main_include_admin.js REFACTOR: Support bundling our admin section as an ember addon 2020-09-22 15:14:29 -04:00
markdown-it-bundle.js REFACTOR: Do not rely on module order for discourse-markdown loading 2020-09-18 13:31:55 -04:00
onpopstate-handler.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
package.json FIX: Make truth-helpers work in Ember CLI (#13044) 2021-05-12 17:52:36 +02:00
polyfills.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
pretty-text-bundle.js FIX: pretty text allow list (#10977) 2020-10-28 13:22:06 +11:00
print-page.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
service-worker.js.erb DEV: debug: false for service workers to stop log spam via workbox (#12093) 2021-02-16 14:01:19 +10:00
set-prototype-polyfill.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
start-discourse.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
template_include.js DEV: Support for import Handlebars from 'handlebars'; (#9600) 2020-04-30 16:41:02 -04:00
test-shims.js DEV: Migrate to Ember CLI (#11932) 2021-02-03 14:22:20 -05:00
vendor.js DEV: Add uppy.js to build and project (#13645) 2021-07-07 10:39:33 +10:00
widget-runtime.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
wizard-application.js DEV: implements some of ember-truth-helpers (#12667) 2021-04-12 11:40:00 +02:00
wizard-shims.js FIX: adds shims for popper in wizard (#10658) 2020-09-14 12:33:14 -04:00
wizard-start.js DEV: apply new coding standards (#10592) 2020-09-04 13:42:47 +02:00
wizard-vendor.js DEV: Remove ember_jquery in most situations (#13237) 2021-06-01 15:32:51 -04:00
yarn.lock DEV: Update pretender and fake-xml-http-request (#13937) 2021-08-05 08:23:01 +10:00