Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

OOI, in a 'modern' app, what would you be using this kind of serialisation and deserialisation for? Some kind of complex RPC?


There are 3 kinds of scenarios are serialization bounded: 1. RPC for large microservices application, which are latency&throughput sensitive. The domain model can be huge, serialization will be the bottleneck. Some domain object graph may contains thousands of objects, serializing using jdk will take seconds, and the serialization result may be 50~100kb. Take Order as an example, imagine you need to process an order which has 500 items.

2. Data transfer for bigdata distributed systems: bigdata systems will handle much data, the data needs be transfered between workers. The serialization can be the bottleneck too. Spark RDD,Flink DataStream all have this bottleneck. The use bianry format such as arrow/tungsten format to reduce serialization overhead. But it's limited, sql oritiened, and can't express complex logic such as graph/event/domain.

3. Task scheduling: Image you have a mpp distributed systems, you need to schedule thounds of tasks in a process every sub-seconds. Serialization will be the bottleneck too.


Yes and: Kyro was motivated by building multi player games.


Yes, Game is another scenario, it's very latency sensitive. Fury is very suitable too. Actually the java implememtation has been featured by some game developers. And there has always been a demand within the community for FURY to support C#: https://github.com/alipay/fury/issues/686 . I don't have experience for c#, so it's not supported. Hope c# can be supported with the help of the community


Given it's a binary serialization framework, it should not be too difficult, because the domain is well-explored and numerous libraries exist in C# which address same goals that Fury does.

More popular/newer examples are https://github.com/Cysharp/MemoryPack (which is similar to Fury with its own spec, C#-code first schema), https://github.com/MessagePack-CSharp/MessagePack-CSharp or even gRPC / Protobuf tooling https://github.com/grpc/grpc-dotnet


Glad to see those great libraries. Can't wait to make fury support c#. One of the biggest difference for fury is that if support circular/shared reference and polymorphism. It's different from many other libraries.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: