Customizable serializers #2514

Horusiath · 2017-02-11T15:13:04Z

Motivation

Some of the users lack an ability to customize used serializers i.e. adding custom converters to json.net serializer. Another time some of them may want to harden serializer constraints on themselves in order to achieve better performance. This is especially a case with new Hyperion serializer, which uses things like preserving object references and version tolerance, which both increase the serialization time and a payload size, causing our default hyperion settings to be actually always a worst case scenario for that library.

Description

This PR introduces an ability to add custom options to serializers through HOCON config. It can be used as a general purpose mechanism. From now, users may define their own serializer configuration within akka.actor.serialization-settings i.e.:

akka.actor {
  serializers {
    hyperion = "Akka.Serialization.HyperionSerializer, Akka.Serialization.Hyperion"
  }
  serialization-bindings {
    "System.Object" = hyperion
  }
  serialization-settings {
    hyperion {
      version-tolerance = false
      preserve-object-references = false
    }
  }
}

This way a Config object holding an akka.actor.serialization-settings.hyperion section will be passed as a second parameter to constructor of a serializer aliased as "hyperion" in akka.actor.serializers. From there, a serializer class is free to interpret it. This behavior is done at Serialization class level, so it's applied automatically to any serializer, that wants to use it.

This PR introduces this behavior on 2 serializers:

NewtonSoftJsonSerializer may optionally take a config with a list of custom converter types to be added.
HyperionSerializer may optionally take a config with 3 settings:
- perserve-object-references used to track references (not only values) in serialized object graph, including cyclic references.
- version-tolerance which can be turned off to reduce a payload manifest size.
- known-types-provider which takes a type name for a class implementing IKnownTypesProvider - an interface that will be used at start of an actor system to provide a list of types, that are assumed to be known by all communicating sides using Hyperion serializer. This also can be used to optimize Hyperion usage.

This new feature has its separate test to check if HOCON configs are bound to right serializers when serialization map is being created.

rogeralsing · 2017-02-11T16:01:35Z

Some background here, and I've verified this with your current codebase too.
Version tolerance is not implemented, there is some initial step towards it by storing type information in the payload, but nothing more. there is nothing on the deserializing side to make something out of this information.

By exposing the version-tolerance property, it might make people believe that there is tolerance support.
I've seen some people talking about using Wire/Hyperion for persistence, which I've explicitly warned about in the readme.

Aaronontheweb

Love the idea and the implementation thus far, but needs some spec coverage, additional HOCON settings for JSON.NET, and some XML-DOC comments on any public members.

Aaronontheweb · 2017-02-15T15:54:12Z

src/contrib/serializers/Akka.Serialization.Hyperion/HyperionSerializer.cs

+        }
+    }
+
+    public sealed class HyperionSerializerSettings


Needs XML-DOC comments per @sean-gilliam's initiative

Aaronontheweb · 2017-02-15T15:55:45Z

src/core/Akka/Serialization/NewtonSoftJsonSerializer.cs

@@ -19,6 +21,39 @@

 namespace Akka.Serialization
 {
+    internal sealed class NewtonSoftJsonSerializerSettings


again with the XML-DOC comments. Should also be made public since this file is going to be moved out of Akka.NET core in the near future

Aaronontheweb · 2017-02-15T15:55:58Z

src/core/Akka/Serialization/NewtonSoftJsonSerializer.cs

+            _serializer = JsonSerializer.Create(_settings);
+        }
+
+        public NewtonSoftJsonSerializer(ExtendedActorSystem system, Config config)


Aaronontheweb · 2017-02-16T01:02:37Z

src/contrib/serializers/Akka.Serialization.Hyperion/HyperionSerializer.cs

+        {
+            if (config == null) throw new ArgumentNullException(nameof(config), "HyperionSerializerSettings require a config, default path: `akka.serializers.hyperion`");
+
+            var type = Type.GetType(config.GetString("known-types-provider"), true);


Need some specs to verify that all of these settings can be loaded by the Hyperion serializer from a valid HOCON file.

Aaronontheweb · 2017-02-16T01:03:05Z

src/contrib/serializers/Akka.Serialization.Hyperion/reference.conf

+		hyperion {
+			# If true, hyperion serializer will preserve references between objects. 
+			# This is necessary to support things such as cyclic dependencies between objects.
+			preserve-object-references = true


Need specs to validate the defaults (protects against regressions where the defaults get changed by accident)

Aaronontheweb · 2017-02-16T01:03:55Z

src/core/Akka/Serialization/NewtonSoftJsonSerializer.cs

+        public NewtonSoftJsonSerializer(ExtendedActorSystem system, Config config)
+            : base(system)
+        {
+            var serializerSettings = NewtonSoftJsonSerializerSettings.Create(config);


Need specs to validate that these settings can be correctly loaded from config

Also: what are the default JSON.NET HOCON settings? Shouldn't these be expressed somewhere also?

Horusiath · 2017-02-16T21:10:09Z

I've added config specs + 2 extra settings for NewtonsoftJsonSerializer (in case if it will be used as secondary serializer) + xml docs for settings.

Aaronontheweb

Looks good to me. Resubmitted the perf test, but otherwise I think this is ready for merge. Lays the ground work for making serializers customizable and extensible in the future.

Aaronontheweb · 2017-02-18T21:51:50Z

src/contrib/serializers/Akka.Serialization.Hyperion.Tests/HyperionConfigTests.cs

+    public class HyperionConfigTests
+    {
+        [Fact]
+        public void Hyperion_serializer_should_have_correct_defaults()


Thank you @Horusiath. Looks great.

Horusiath added akka-actor serialization labels Feb 11, 2017

Horusiath changed the title ~~Customizable hyperion serializer~~ Customizable serializers Feb 11, 2017

Horusiath mentioned this pull request Feb 11, 2017

Full support for version tolerance akkadotnet/Hyperion#31

Open

Horusiath force-pushed the customizable-hyperion-serializer branch 3 times, most recently from 3e39d19 to a06434f Compare February 15, 2017 15:50

Aaronontheweb requested changes Feb 16, 2017

View reviewed changes

Aaronontheweb approved these changes Feb 18, 2017

View reviewed changes

Horusiath added 4 commits February 19, 2017 09:12

added settings to hyperion serializer

7997e8d

making configurable settings a general purpose

f76bb0f

approvals API

3494ebd

added comments and config specs for json.net & hyperion

6721657

Horusiath force-pushed the customizable-hyperion-serializer branch from f230ff3 to 6721657 Compare February 19, 2017 08:13

Horusiath added addressed remarks reviewed positively labels Feb 19, 2017

Aaronontheweb merged commit c8181f0 into akkadotnet:dev Feb 20, 2017

Aaronontheweb modified the milestone: 1.1.4 Feb 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Customizable serializers #2514

Customizable serializers #2514

Uh oh!

Horusiath commented Feb 11, 2017 •

edited

Loading

Uh oh!

rogeralsing commented Feb 11, 2017

Uh oh!

Aaronontheweb left a comment

Uh oh!

Aaronontheweb Feb 15, 2017

Uh oh!

Aaronontheweb Feb 15, 2017

Uh oh!

Aaronontheweb Feb 15, 2017

Uh oh!

Aaronontheweb Feb 16, 2017

Uh oh!

Aaronontheweb Feb 16, 2017

Uh oh!

Aaronontheweb Feb 16, 2017

Uh oh!

Aaronontheweb Feb 16, 2017

Uh oh!

Horusiath commented Feb 16, 2017

Uh oh!

Aaronontheweb left a comment

Uh oh!

Aaronontheweb Feb 18, 2017

Uh oh!

Uh oh!

Customizable serializers #2514

Customizable serializers #2514

Uh oh!

Conversation

Horusiath commented Feb 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Description

Uh oh!

rogeralsing commented Feb 11, 2017

Uh oh!

Aaronontheweb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Horusiath commented Feb 16, 2017

Uh oh!

Aaronontheweb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Horusiath commented Feb 11, 2017 •

edited

Loading