Skip to content

Avoid intermediate JSON objects when serializing and deserializing #662

@osa1

Description

@osa1

Currently methods like mergeFromProto3Json that parse a message encoded as JSON take Dart representation of the JSON message as argument. This means to parse a JSON message we first parse the JSON to a Dart object representing the JSON, then convert it to a message.

We could avoid the intermediate step of parsing JSON to Dart maps and lists by using a "pull based" JSON parser like jsontool.

I don't know if it's common to pass JSON objects around within the same Dart app (without serializing JSON objects) and update messages with them, but if this is common we could keep the current methods as well.


A concern when targeting JS is that we use JS engine's JSON.parse to parse JSON to JS representation of the JSON, and then convert it to Dart representation. So conversion of JSON serialization to JS objects is done by the engine and given how common JSON is I suspect this code is highly optimized. So on the JS we will be comparing:

  1. Parsing JSON string to JSON (done by the JS engine, probably highly optimized) + converting JSON object to a Dart object (done by Dart code, in core) + updating message fields using the Dart (in protobuf library)
  2. Updating message fields using the pull-based parser (entirely in protobuf library)

It is possible that parsing JSON string in Dart is so much slower than parsing JSON string to JSON object at the engine level, the pull-based approach will be slower on the browser. Also, it seems like when targeting JS, generating of Dart objects from a JSON object is done lazily (_convertJsonToDartLazy in the VM code linked above), so perhaps the GC pressure is also not high in (1).

With AOT and JIT, the only help we get from the VM is Double_parse for parsing doubles. Rest of the JSON parsing seems to be in pure Dart, and as far as I can see we don't have the lazy conversion that we have in JS version, so I suspect using a pull-based parser will probably a win when targeting JIT and AOT.

We could use conditional compilation and fall back to the current implementation if this only performs well in AOT/JIT and not in JS.


Similarly for serializing as JSON we could use jsontool's JsonStringWriter and avoid intermediate JSON objects.

We already have benchmarks for parsing and generating JSON, so evaluation shouldn't be too difficult.

Metadata

Metadata

Assignees

No one assigned

    Labels

    perfRelated to runtime performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions