Uploading file to openstack object storage from JavaScript

Question

I have a openstack object storage container to which I'm trying to upload files directly from browser.

As per the documentation here, I can upload the file using a PUT request and I'm doing this using Angularjs provided $http.put method as shown below.

$http.put(temporaryUploadUrl,
    formData,
    {
        headers: {
            'Content-Type': undefined
        }
    }
).then(function (res) {
    console.log("Success");
});

The file uploads successfully and it has no problems in authentication and gives me a 201 Created response. However the file is now containing junk lines on the top and bottom of it because its a multipart request sent using FormData().

Sample file content before upload:

Some sample text 
here is more text 
here is some other text

File content after downloadiong back from openstack container :

------WebKitFormBoundaryTEeQZVW5hNSWtIqS
Content-Disposition: form-data; name="file"; filename="c.txt"
Content-Type: text/plain
Some sample text 
here is more text 
here is some other text
------WebKitFormBoundaryTEeQZVW5hNSWtIqS--

I tried the FileReader to read the selected file as a binary string and wrote the content to the request body instead of FormData and the request which works fine for text files but not the binary files like XLSX or PDF The data is entirely corrupted this way.

Answer 1

EDIT:

The following answer is now considered a less performing workaround As it will encode the entire file to base64 multipart form data. I would suggest go ahead with @georgeawg's Answer if you are not Looking for a formData + POST solution

Openstack also provides a different approach using FormData for uploading one or more files in a single go as mentioned in this documentation. Funny this was never visible in google search.

Here is a brief of it.

First you need to generate a signature similar to tempUrl signature using the following python procedure.

import hmac
from hashlib import sha1
from time import time
path = '/v1/my_account/container/object_prefix'
redirect = 'https://myserver.com/some-page'
max_file_size = 104857600
max_file_count = 1
expires = 1503124957
key = 'mySecretKey'
hmac_body = '%s\n%s\n%s\n%s\n%s' % (path, redirect,
max_file_size, max_file_count, expires)
signature = hmac.new(key, hmac_body, sha1).hexdigest()

Then in your javascript call post to the container like this.

var formData = new FormData();
formData.append("max_file_size", '104857600');
formData.append("max_file_count", '1');
formData.append("expires", '1503124957');
formData.append("signature", signature);
formData.append("redirect", redirect);
formData.append("file",fileObject);
$http.post(
    "https://www.example.com/v1/my_account/container/object_prefix",
    formData,
    {
        headers: {'Content-Type': undefined},
        transformRequest: angular.identity
    }
).then(function (res) {
    console.log(response);
});

Points to note.

• The formData in POST request should contain only these parameters.
• The file entry in the formData should be the last one.(Not sure why it doesnt work the other way around).
• The formData content like path with prefix, epoch time, max file size, max file count and the redirection urls should be the same as the one which were used to generate the signature. Otherwise you will get a 401 Unauthorized.

Answer 2

I tried the FileReader to read the selected file as a binary string and wrote the content to the request body instead of FormData and the request which works fine for text files but not the binary files like XLSX or PDF The data is entirely corrupted this way.

The default operation for the $http service is to use Content-Type: application/json and to transform objects to JSON strings. For files from a FileList, the defaults need to be overridden:

var config = { headers: {'Content-Type': undefined} };

$http.put(url, fileList[0], config)
  .then(function(response) {
     console.log("Success");
}).catch(function(response) {
     console.log("Error: ", response.status);
     throw response;
});

By setting Content-Type: undefined, the XHR send method will automatically set the content type header appropriately.

Be aware that the base64 encoding of 'Content-Type': multipart/form-data adds 33% extra overhead. It is more efficient to send Blobs and File objects directly.

Sending binary data as binary strings, will corrupt the data because the XHR API converts strings from DOMSTRING (UTF-16) to UTF-8. Avoid binary strings as they are non-standard and obsolete.