Read a base 64 encoded image from memory using OpenCv python library

Question

I'm working on an app that to do some facial recognition from a webcam stream. I get base64 encoded data uri's of the canvas and want to use it to do something like this:

cv2.imshow('image',img)

The data URI looks something like this:

 data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7

So, for clarity I've shown what the image looks like so the base64 string is not broken.

<img src="data:image/gif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7">

The official doc says, that imread accepts a file path as the argument. From this SO answer, if I do something like:

 import base64
 imgdata = base64.b64decode(imgstring) #I use imgdata as this variable itself in references below
 filename = 'some_image.jpg'
 with open(filename, 'wb') as f:
    f.write(imgdata)

The above code snippet works and the image file gets generated properly. However I don't think so many File IO operations are feasible considering I'd be doing this for every frame of the stream. I want to be able to read the image into the memory directly creating the img object.

I have tried two solutions that seem to be working for some people.

1. Using PIL reference:

   pilImage = Image.open(StringIO(imgdata))
   npImage = np.array(pilImage)
   matImage = cv.fromarray(npImage)
   

   I get cv not defined as I have openCV3 installed which is available to me as cv2 module. I tried img = cv2.imdecode(npImage,0), this returns nothing.

2. Getting the bytes from decoded string and converting it into an numpy array of sorts

   file_bytes = numpy.asarray(bytearray(imgdata), dtype=numpy.uint8)
   img = cv2.imdecode(file_bytes, 0) #Here as well I get returned nothing
   

The documentation doesn't really mention what the imdecode function returns. However, from the errors that I encountered, I guess it is expecting a numpy array or a scalar as the first argument. How do I get a handle on that image in memory so that I can do cv2.imshow('image',img) and all kinds of cool stuff thereafter.

I hope I was able to make myself clear.

Answer 1

This is my solution for python 3.7 and without using PIL

import base64

def readb64(uri):
   encoded_data = uri.split(',')[1]
   nparr = np.fromstring(base64.b64decode(encoded_data), np.uint8)
   img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
   return img

i hope that this solutions works for all

Answer 2

(Edit: Updated for python 3) This worked for me on python 3, and doesn't require PIL/pillow or any other dependencies (except cv2):

import cv2
import numpy as np
import base64

def data_uri_to_cv2_img(uri):
    encoded_data = uri.split(',')[1]
    nparr = np.frombuffer(base64.b64decode(encoded_data), np.uint8)
    # old (python 2 version):
    # nparr = np.fromstring(encoded_data.decode('base64'), np.uint8)

    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    return img

data_uri = "data:image/jpeg;base64,/9j/4AAQ..."
img = data_uri_to_cv2_img(data_uri)
cv2.imshow("Image", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Answer 3

You can just use both cv2 and pillow like this:

import base64
from PIL import Image
import cv2
from StringIO import StringIO
import numpy as np

def readb64(base64_string):
    sbuf = StringIO()
    sbuf.write(base64.b64decode(base64_string))
    pimg = Image.open(sbuf)
    return cv2.cvtColor(np.array(pimg), cv2.COLOR_RGB2BGR)

cvimg = readb64('R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7')
cv2.imshow(cvimg)

Answer 4

I found this simple solution.

import cv2
import numpy as np
import base64
image = ""  # raw data with base64 encoding
decoded_data = base64.b64decode(image)
np_data = np.fromstring(decoded_data,np.uint8)
img = cv2.imdecode(np_data,cv2.IMREAD_UNCHANGED)
cv2.imshow("test", img)
cv2.waitKey(0)

Source : https://gist.github.com/HoweChen/7cdd09b08147133d8e1fbe9b52c24768