Protobuf3: How to describe map of repeated string?

Question

The Official documentation about map type says:

map<key_type, value_type> map_field = N;

...where the key_type can be any integral or string type (so, any scalar type except for floating point types and bytes). The value_type can be any type.

I want to define a map<string, repeated string> field, but it seems illegal on my libprotoc 3.0.0, which complains Expected ">". So I wonder if there is any way to put repeated string into map.

A Possible workaround could be:

message ListOfString {
    repeated string value = 1;
}

// Then define:
map<string, ListOfString> mapToRepeatedString = 1;

But ListOfString here looks redundant.

Just checking the basics :) Have you set `syntax = "proto3";`? — Mark, Aug 22 '16 at 20:08

score 27 · Accepted Answer · edited Apr 03 '19 at 08:14

I had the same need, and got the same error. I do not believe this is possible. Here is the relevant BNF definitions from the language specification.

https://developers.google.com/protocol-buffers/docs/reference/proto3-spec

messageType = [ "." ] { ident "." } messageName
mapField = "map" "<" keyType "," type ">" mapName "=" fieldNumber [ "["fieldOptions "]" ] ";"
type = "double" | "float" | "int32" | "int64" | "uint32" | "uint64"
  | "sint32" | "sint64" | "fixed32" | "fixed64" | "sfixed32" | "sfixed64"
  | "bool" | "string" | "bytes" | messageType | enumType
messageName = ident
ident = letter { letter | decimalDigit | "_" }
field = [ "repeated" ] type fieldName "=" fieldNumber [ "[" fieldOptions "]" ] ";"

"repeated" keyword only appears in the field definition. The map definition requires a "type", which does not include the repeated keyword.

That means there are a few options.

You could create a wrapper around the repeated value as you indicated.

There is the older way people defined define maps, which is more burdensome but is equivalent. This is the backwards compatible example from the language guide. https://developers.google.com/protocol-buffers/docs/proto3#maps

    message MapFieldEntry {
      key_type key = 1;
      repeated value_type value = 2;
    }
    repeated MapFieldEntry map_field = N;

You would need to convert the data to a map yourself, but this should be fairly trivial in most languages. In Java:

    List<MapFieldEntry> map_field = // Existing list from protobuf.
    Map<key_type, List<value_type>> = map_field.stream()
        .collect(Collectors.toMap(kv -> kv.key, kv -> kv.value));

Use google.protobuf.ListValue https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#listvalue This is an untyped list collection from their well known types.

I'm curious why Protobuf has both 'ListValue' and 'repeated'. Do they not serve the same purpose? The immediate documentation on 'ListValue' is somewhat sparse. — EdwinW, Aug 01 '19 at 18:18
@EdwinW `ListValue` and `repeated` are different. `ListValue` is an untyped list -- its contents could be anything. `repeated` takes a type and indicates that it is explicitly a list of that type. — Charles, Jun 20 '22 at 23:14

Viraths · Answer 2 · 2019-11-19T23:23:50.467

14

I think it should be as follows.

message ListOfString {
   repeated string what_ever_name = 1;
}

// Then define:
map<string, ListOfString> what_ever_name = 1;

Remember what_ever_name should be same in both places.

edited Nov 19 '19 at 23:23

answered Sep 23 '19 at 01:01

Viraths

840
1
13
23

3

Why does the field name of `what_ever_name` need to be the same in both places? Isn't it a completely separate type? – Big Money May 21 '21 at 17:39
3

@BigMoney it doesn't have to be. I think it's just how this example is laid out. – Bryan McGrane May 28 '21 at 20:28

Protobuf3: How to describe map of repeated string?

2 Answers2

Linked

Related