use references to represent string/bytes proto3 field types

[pushkarmoi]

Hi folks,

prost currently uses owned objects is String and Vec<u8> in code generated structs for proto3 string and bytes fields.

This can add considerable performance overhead for -

1. Serialization: We currently store utf8 in custom in memory data structures. To construct a serializable prost struct, we need to deep copy the data to create owned String objects. Allowing us to pass &str or Cow<&str> (like https://github.com/tafia/quick-protobuf) will help a lot. The deep copying is a major consumer of cpu cycles as per flamegraph

2. Deserialization: We anticipate there's a similar issue there with prost deep copying utf8 out of underlying bytes to create the String struct fields. We benchmarked quick-protobuf and it outperformed prost in this aspect - i believe it doesn't need to deep copy the actual string bytes.

Understand this can be incompatible api wise so an opt in config might be best to support it.

Thanks!

[caspermeijn]

Hi @pushkarmoi

I understand your concerns. I have a plan to make the types used in generated fields more configurable. Cow<&str> could be one of the new types. However, I currently have not the bandwidth to work on that.

If you are willing to contribute (time or money), please contact me to figure out the details.

Thanks, Casper