Gmail API decoding messages in Javascript

Question

I am having serious problems decoding the message body of the emails I get using the Gmail API. I want to grab the message content and put the content in a div. I am using a base64 decoder, which I know won't decode emails encoded differently, but I am not sure how to check an email to decide which decoder to use -- emails that say they are utf-8 encoded are successfully decoded by the base64 decoder, but not be a utf-8 decoder.

I've been researching email decoding for several days now, and I've learned that I am a little out of my league here. I haven't done much work with coding around email before. Here is the code I am using to get the emails:

gapi.client.load('gmail', 'v1', function() {
var request = gapi.client.gmail.users.messages.list({
  labelIds: ['INBOX']
});
request.execute(function(resp) {
  document.getElementById('email-announcement').innerHTML = '<i>Hello! I am reading your <b>inbox</b> emails.</i><br><br>------<br>';
  var content = document.getElementById("message-list");
  if (resp.messages == null) {
    content.innerHTML = "<b>Your inbox is empty.</b>";
  } else {
    var encodings = 0;
    content.innerHTML = "";
    angular.forEach(resp.messages, function(message) {
      var email = gapi.client.gmail.users.messages.get({
      'id': message.id
      });
      email.execute(function(stuff) {
        if (stuff.payload == null) {
          console.log("Payload null: " + message.id);
        }
        var header = "";
        var sender = "";
        angular.forEach(stuff.payload.headers, function(item) {
          if (item.name == "Subject") {
            header = item.value;
          }
          if (item.name == "From") {
            sender = item.value;
          }
        })
        try {
          var contents = "";
          if (stuff.payload.parts == null) {
            contents = base64.decode(stuff.payload.body.data);
          } else {
            contents = base64.decode(stuff.payload.parts[0].body.data);
          }
          content.innerHTML += '<b>Subject: ' + header + '</b><br>';
          content.innerHTML += '<b>From: ' + sender + '</b><br>';
          content.innerHTML += contents + "<br><br>";
        } catch (err) {
          console.log("Encoding error: " + encodings++);
        }
      })
    })
  }
 });
});

I was performing some checks and debugging, so there's leftover console.log's and some other things that are only there for testing. Still, you can see here what I am trying to do.

What is the best way to decode the emails I pull from the Gmail API? Should I try to put the emails into <script>'s with charset and type attributes matching the encoding content of the email? I believe I remember charset only works with a src attribute, which I wouldn't have here. Any suggestions?

score 25 · Accepted Answer · edited Oct 07 '21 at 05:49

For a prototype app I'm writing, the following code is working for me:

var base64 = require('js-base64').Base64;
// js-base64 is working fine for me.

var bodyData = message.payload.body.data;
// Simplified code: you'd need to check for multipart.

base64.decode(bodyData.replace(/-/g, '+').replace(/_/g, '/'));
// If you're going to use a different library other than js-base64,
// you may need to replace some characters before passing it to the decoder.

Caution: these points are not explicitly documented and could be wrong:

The users.messages: get API returns "parsed body content" by default. This data seems to be always encoded in UTF-8 and Base64, regardless of the Content-Type and Content-Transfer-Encoding header.

For example, my code had no problem parsing an email with these headers: Content-Type: text/plain; charset=ISO-2022-JP, Content-Transfer-Encoding: 7bit.
The mapping table of the Base64 encoding varies among various implementations. Gmail API uses - and _ as the last two characters of the table, as defined by RFC 4648's "URL and Filename safe Alphabet"¹.

Check if your Base64 library is using a different mapping table. If so, replace those characters with the ones your library accepts before passing the body to the decoder.

¹ There is one supportive line in the documentation: the "raw" format returns "body content as a base64url encoded string". (Thanks Eric!)

overall very good and helpful, thanks! note that regarding point #2 it is mentioned in the users.messages.get API docs: https://developers.google.com/gmail/api/v1/reference/users/messages/get (see the format=RAW docs). But perhaps it should be expanded further... — Eric D, Sep 30 '14 at 16:32
I have checked so many options before I found your working solution, thanks! — walla, Nov 18 '17 at 20:48

score 5 · Answer 2 · edited May 23 '17 at 12:25

5

Use atob to decode the messages in JavaScript (see ref). For accessing your message payload, you can write a function:

var extractField = function(json, fieldName) {
  return json.payload.headers.filter(function(header) {
    return header.name === fieldName;
  })[0].value;
};
var date = extractField(response, "Date");
var subject = extractField(response, "Subject");

referenced from my previous SO Question and

var part = message.parts.filter(function(part) {
  return part.mimeType == 'text/html';
});
var html = atob(part.body.data);

If the above does not decode 100% properly, the comments by @cgenco on this answer below may apply to you. In that case, do

var html = atob(part.body.data.replace(/-/g, '+').replace(/_/g, '/'));

edited May 23 '17 at 12:25

Community

1
1

answered Feb 20 '15 at 05:22

FullStack

5,902
4
43
77

1

It looks like the `.replace(/-/g, '+').replace(/_/g, '/')` still has to be done for it to decode correctly. – cgenco Mar 02 '15 at 03:45
@cgenco, thanks for sharing. Can you explain why it is insufficient without your `replace` functions? I will be happy to revise my answer. – FullStack Mar 02 '15 at 09:44
per @ento's answer, Base64 encoding [varies across implementations](https://en.wikipedia.org/wiki/Base64#Implementations_and_history), and google happens to use the encoding that uses `-` instead of `+` and `_` instead of `/`. – cgenco Mar 22 '15 at 16:52

score 2 · Answer 3 · answered Dec 31 '17 at 19:51

Here is the solution: Gmail API - "Users.messages: get" method has in response message.payload.body.data parted base64 data, it's separated by "-" symbol. It's not entire base64 encoded text, it's parts of base64 text. You have to try to decode every single part of this or make one mono string by unite and replace "-" symbol. After this you can easily decode it to human text. You can manually check every part here https://www.base64decode.org

score 2 · Answer 4 · edited Apr 29 '20 at 08:04

I was also annoyed by this point. I discovered a solution through looking at an extension for VSCode. The solution is really simple:

const body = response.data.payload.body; // the base64 encoded body of a message
 body = Buffer.alloc(
        body.data.length,
        body.data,
        "base64"
      ).toString();  // the decoded message

It worked for me as I was using gmail.users.messages.get() call of Gmail API.

score 1 · Answer 5 · answered Feb 19 '18 at 05:51

1

Please use websafe decoder for decoding gmail emails and attachments. I got blank pages when I used just base64decoder, had to use this: https://www.npmjs.com/package/urlsafe-base64

answered Feb 19 '18 at 05:51

Vandana Chadha

55
7

score 1 · Answer 6 · answered Oct 18 '20 at 23:00

Thank @ento 's answer. I explain more why you need to replace '-' and '_' character to '+' and '/' before decode.

Wiki Base64 Variants summary table shows:

RFC 4648 section 4: base64 (standard): use '+' and '/'
RFC 4648 section 5: base64url (URL-safe and filename-safe standard): use '-' and '_'

In short, Gmail API use base64url (urlsafe) format('-' and '_'), But JavaScript atob function or other JavaScript libraries use base64 (standard) format('+' and '/').

For Gmail API, the document says body use base64url format, see below links:

For Web atob/btoa standards, see below links:

score 0 · Answer 7 · answered Mar 31 '19 at 06:14

0

I can easily decode using another tool at https://simplycalc.com/base64-decode.php

In JS: https://www.npmjs.com/package/base64url

In Python 3:

import base64
base64.urlsafe_b64decode(coded_string)

answered Mar 31 '19 at 06:14

anhtran

2,006
4
27
53

Gmail API decoding messages in Javascript

7 Answers7

Linked