3

I don't understand what does it mean to register a class for kryo serialization. To give some background the link here says that

Kryo does not support all Serializable types and requires you to register the classes you’ll use in the program in advance for best performance.

Again, I don't understand what is the meaning of registering a class? Isn't the serialization code static in the sense that you apply the same logic to serialize all kinds of objects. How could mentioning in advanced the classes that will be serialized be helpful later then?

Thanks!

hesk
  • 317
  • 3
  • 11
  • 1
    I've given an example here https://stackoverflow.com/questions/32667068/save-spark-dataframe-into-elasticsearch-can-t-handle-type-exception – eliasah Jun 19 '17 at 14:54

1 Answers1

4

The Kyro docs mention that registering in advance is not necessary but increases performance as the first time a class is encountered the name of the class does not need to be serialized; the is generated during registration can be used instead, which also results in fewer bytes used.

Excerpt:

When Kryo writes out an instance of an object, first it may need to write out something that identifies the object's class. By default, the fully qualified class name is written, then the bytes for the object. Subsequent appearances of that object type within the same object graph are written using a variable length int. Writing the class name is somewhat inefficient, so classes can be registered beforehand:

DK_
  • 2,648
  • 2
  • 21
  • 20
  • Do we need to register all classes(i.e also primitives) or just the classes we create (e.g class Person(name: String)? @DK_ , currently I am using scala – supernatural Nov 20 '19 at 10:26
  • @phineas AFAIK you need to register all the classes you will serialise (excluding primitives). There are various utility libraries that have Serialisers for you. See [Twitter chill](https://github.com/twitter/chill) and [kryo-serializers](https://github.com/magro/kryo-serializers) – DK_ Nov 22 '19 at 08:18