23

Is there a way to do protocol buffers in JavaScript?

Why for .js?

If you think about sciencey requirements for a moment, situations pop up where you might want to send a large block of data to the client. With CRUD-style it doesn't really matter so much what you use. With sciencey stuff it does matter (at least I think it does).

tradeoffs:

  • protobuffs balances compactness, serialize and deserialize speeds well.

  • text based protocols (xml / json) have a larger message size... but with javascript I wonder which is more effective.

reference:

Additional references provided by community (see below for more context):

Community
  • 1
  • 1
sgtz
  • 8,849
  • 9
  • 51
  • 91
  • ty. Added this as a reference. – sgtz Aug 16 '11 at 06:25
  • 2
    I came across another Protocol Buffers for JavaScript project. It seems to be active and relies on Google's `closure` library: http://code.google.com/p/protobuf-plugin-closure/ – alavrik Aug 16 '11 at 15:06
  • ty. There's some interest out there for this at least. I wonder sometimes why people don't combine forces more. – sgtz Aug 16 '11 at 15:15
  • @AndrewBarber upon review, perhaps your objection was that the question opened in a chatty way. Since the question was first written, the JS community has evolved considerably. However, first-class protobuffer support is still needed in JavaScript. I think the question is still relevant. – sgtz Nov 21 '13 at 05:33

3 Answers3

14

Google makes heavy use of Protocol Buffers in JS (GMail, etc.) through their Closure Library, generating JS code with a (unfortunately non-open-sourced) modified protoc (it would probably have to be ported to a protoc extension before being open-sourced).

Apache Wave (whose client webapp is built with GWT) also uses Protocol Buffers for its communications with the server, generating Java code by reflecting on the Java classes produced by protoc (this is the PST, aka protobuf-stringtemplate, subproject).
Previously, Wave was using protostuff (and I don't know why they switched to their own solution, I suspect PST is derived from what the original Google Wave was using, and protostuff was only an intermediate step during the move to open-source).

As a side note, I started exploring using Protocol Buffers on the browser side a while ago: http://blog.ltgt.net/exploring-using-protobuf-in-the-browser/ & http://blog.ltgt.net/using-protobuf-client-side-with-gwt with some almost-working code at http://code.google.com/p/protobuf-gwt/ that you might want to resurrect.

Finally, there's work underway to make GWT RequestFactory proxies compatible with the server-side Java classes generated by protoc (and you could use a protoc extension or a similar approach to Wave's PST to generate your RequestFactory proxies). It should already be possible, provided you use builders all the way on the server-side (which is not quite how the Protocol Buffers Java API was designed).

Thomas Broyer
  • 64,353
  • 7
  • 91
  • 164
  • I started thinking about the cross browser issues for a JavaScript version of protobufs. Did you many issues with this in your testing? – sgtz Aug 19 '11 at 11:03
  • We actually settled on a different architecture and aren't using Protocol Buffers at all, so there haven't been much testing per se. But most if not all of the cross-browser issues (related to encoding/decoding the messages to JSON and transporting your protobuf/json-encoded message (XMLHttpRequest)) would be taken care of by either Closure or GWT (or the JS lib of your choice). – Thomas Broyer Aug 20 '11 at 01:26
  • sorry... switched from your informative answer to dcode's one. Looks like the state of js proto is changing. – sgtz Mar 09 '13 at 06:23
8

Historically javascript made working with binary a pain, which probably in part explains a relative lack of tooling - but with javascript typed arrays it could well be a lot easier now. I kinda agree that if you have to get the same volume of data (via some format), using less bandwidth is a plus - but before embarking on anything you'd need to check that bandwidth / processing was an actual bottleneck (and if bandwidth: have you tried gzip/deflate first).

I'm a fan of protobuf - and I'd happily see stronger browser-side tooling for it, but json is so ubiquitous that you'd need a compelling reason to challenge the status-quo. Also; think "jsonp".

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • 1
    took a look at "jsonp". Nice technique. Thanks for the reminder on gzip... it seems a pity to introduce extra representations + steps. – sgtz Aug 16 '11 at 06:38
  • @sgtz my point re jsonp is that jsonp solves a specific issue re same-origin; I'm not sure how you could address the same same-origin issue with raw data. – Marc Gravell Aug 16 '11 at 06:54
7

I have been looking for protobuf for javascript. There is a project here: https://github.com/dcodeIO/ProtoBuf.js

Community
  • 1
  • 1
agarcian
  • 3,909
  • 3
  • 33
  • 55
  • thanks. I'll keep an eye on this project. In future I'd like to support something like this. Fingers crossed that a capable library will emerge. – sgtz Apr 15 '12 at 07:27
  • Nothing [changed](http://code.google.com/p/protobuf-js/source/list) since the 0.0.0 release in 2008. Might still be okay, of course. Did you experiment with this? – Arjan Aug 23 '12 at 08:15
  • Sorry, haven't worked on this on a while. Cannot answer much. From what I remember it did what it advertised. I don't have this in production so I cannot vouch for it with certainty. – agarcian Aug 23 '12 at 16:25
  • This is the best js protobuf library I have found. Currently using it and it is working well. – sethro Oct 14 '13 at 16:01
  • Seems it's active recently (2014). With 11 releases, 21 open vs 127 closed issues, wiki seems not rich enough. – Evi Song Aug 15 '14 at 05:15