RESTful way to upload file along with some data in django

Question

I am creating a webservice with django using django rest framework. Users are able to upload some images and videos. Uploading media is a two step action, first user uploads the file and receives an ID then in a separate request uses that ID to refer to the media (for example (s)he can use it as profile picture or use it in a chat message).

I need to know who is uploading the media for both HMAC authentication middleware and setting owner of media in database. All other requests are in JSON format and include a username field that it used by HMAC middleware to retrieve the secret shared key.

It first came to my mind that media upload api may look like this:

{
  "username":"mjafar",
  "datetime":"2015-05-08 19:05",
  "media_type":"photo",
  "media_data": /* base64 encoded image file */
}

But i thought that base64 encoding may have significant overhead for larger files like videos; or there may be some restrictions on size of data that can be parsed in json or be created in user side. (This webservice is supposed to communicate with a Android/iOS app, they have limited memory)! Is this a good solution? Are my concerns real problems or i shouldn't worry? Better solutions?

Answer 1

You could separate the two. Meta data at one interface with a URL pointing to the actual file. Depending on how you store the actual file you could then reference the file directly via URL at a later point.

You could then have the POST API directly accept the file and simply return the JSON meta data

{
  "username":"mjafar", // inferred from self.request.user
  "datetime":"2015-05-08 19:05", // timestamp on server
  "media_type":"photo", // inferred from header content-type?
  // auto-generated hashed location for file
  "url": "/files/1dde/2ecf/4075/f61b/5a9c/1cec/53e0/ca9b/4b58/c153/09da/f4c1/9e09/4126/271f/fb4e/foo.jpg"
}

Creating such an interface using DRF would be more along the lines of implementing rest_framework.views.APIView

Here's what I'm doing for one of my sites:

class UploadedFile(models.Model):
    creator = models.ForeignKey(auth_models.User,blank=True)
    creation_datetime = models.DateTimeField(blank=True,null=True)
    title = models.CharField(max_length=100)
    file = models.FileField(max_length=200, upload_to=FileSubpath)
    sha256 = models.CharField(max_length=64,db_index=True)
    def save(self,*args,**kw_args):
        if not self.creation_datetime:
            self.creation_datetime = UTC_Now()
        super(UploadedFile,self).save(*args,**kw_args)

serializer:

class UploadedFileSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = UploadedFile
        fields = ('url', 'creator','creation_datetime','title','file')

And the view to use this:

from rest_framework.views import APIView
from qc_srvr import serializers,models
from rest_framework.response import Response
from rest_framework import status
from rest_framework import parsers
from rest_framework import renderers
import django.contrib.auth.models as auth_models
import hashlib


class UploadFile(APIView):
    '''A page for uploading files.'''

    throttle_classes = ()
    permission_classes = ()
    parser_classes = (parsers.FormParser, parsers.JSONParser,)
    renderer_classes = (renderers.JSONRenderer,)
    serializer_class = serializers.UploadedFileSerializer

    def calc_sha256(self,afile):
        hasher = hashlib.sha256()
        blocksize=65536
        hasher.update('af1f9847d67300b996edce88889e358ab81f658ff71d2a2e60046c2976eeebdb') # salt
        buf = afile.read(blocksize)
        while len(buf) > 0:
            hasher.update(buf)
            buf = afile.read(blocksize)
        return hasher.hexdigest()


    def post(self, request):
        if not request.user.is_authenticated():
            return Response('User is not authenticated.', status=status.HTTP_401_UNAUTHORIZED)

        uploaded_file = request.FILES.get('file',None)
        if not uploaded_file:
            return Response('No upload file was specified.', status=status.HTTP_400_BAD_REQUEST)


        # calculate sha
        sha256 = self.calc_sha256(uploaded_file)  

        # does the file already exist?
        existing_files = models.UploadedFile.objects.filter(sha256=sha256)
        if len(existing_files):
            serializer = self.serializer_class(instance=existing_files[0],context={'request':request})
        else:
            instance = models.UploadedFile.objects.create(
                creator = request.user,
                title= uploaded_file.name,
                file = uploaded_file,
                sha256 = sha256)
            serializer = self.serializer_class(instance=instance,context={'request':request})
        #import rpdb2; rpdb2.start_embedded_debugger('foo')
        #serializer.is_valid()
        return Response(serializer.data)

FYI, this is a bit of security-through-obscurity since all the uploaded files are retrievable if you have the URL to the file.

I'm still using DRF 2.4.4, so this may not work for you on 3+. I haven't upgraded due to the dropped nested-serializers support.