-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Avoid double serialize when proxying values (#117)
Proxied objects end up being serialized twice: first when the object is serialized with Colmena to check the size of the serialized object and then again by ProxyStore when the object is proxied. ProxyStore supports passing custom serializer/deserializer functions so this commit adds a shim serializer and a deserializer wrapper to minimize serialization overheads when an object gets proxied. Note that there is still one extra step when using Colmena's pickle serialization method and ProxyStore. SerializationMethod.serialize will pickle the object producing a byte string then convert those bytes to a hex string then the shims convert that hex string back to bytes. I.e., there's this intermediate string representation that's created then discarded in this path. I don't really see a way of avoiding this because its an artifact of the different serialization types between Colmena and ProxyStore (str vs bytes).
- Loading branch information
Showing
1 changed file
with
76 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters