compiling to a shared library

[pohutukawa]

I want/need to use schnorrkel signing in another project. However the project is non Rust (Python), so I would like to be able to just compile it to a shared library, which I then can (hopefully) wrap using either Python's built-in ctypes or CFFI.

However, I haven't found a dynlib or cdylib directive anywhere that would result in it. Someone more familiar with Rust has told me to add the following to the Cargo.toml file to accomplish this. Though, for future code distribution reasons I'm hesistant to the need of patching code after cloning it from the repo to generate a Python package.

[lib]
crate-type = ["cdylib", "rlib"]

Would it be possible to empower the code in this schnorrkel repo to allow for building a shared library out of the box?

[burdges]

We'll want this ourselves probably, and maybe someone at Parity already did roughly this, like as part of schnorrkel-js, but regardless I'm probably not the best person for this task. I'm happy with a pull request for whatever the appropriate change should be. :)

cc @kianenigma

[burdges]

I maybe miss-interpreted this as being about js, for which you likely want https://github.com/paritytech/schnorrkel-js/ but actually you just need some build types turned on? Is adding that bit to Cargo.toml all I do? Or do I need to any tests or CI stuff?

[kianenigma]

At least the path that we take on shcnorrkel-js was (obviously) to compile the main lib to Wasm and write a wrapper for it in Javascript. Not sure if that will be of any help when it comes to using it in python (might be worth googling though: how to execute wasm blob from python). Otherwise, this is an interesting issue that can be valuable for schnorrkel, yet it does not have much to do with the javascript related work.

Update: related? https://github.com/wasmerio/python-ext-wasm

[burdges]

I rushed my last comment. I meant: Are you asking for C bindings? I'm not exactly sure how rust hooks to js and if that requires some C-like bindings on the rust side. If so, maybe those could be abstracted from the schnorrkel-js code?

[pohutukawa]

OK, some further feedback from our side. I've got quite some experience in wrapping (mainly C) libraries to Python. There once you've got the shared lib, things become quite easy. However, this is a different beast, and it's not C but rather Rust.

Due to this, there will be some enhanced need for wrapping things, just as @burdges has stated, e.g. in the schnorrkel-js case.

Now, I can see two approaches in principle, that will both require in certain areas similar, but in other respect different approaches. A libschnorrkel can be wrapped and made accessible via these methods:

• Native library wrapping, using an FFI approach. This would require (for the Python case) wrapping the internal functionality so that it is exposed and symbols externally accessible in an unmangled way, then compiling things to a shared library. This could be assisted e.g. in Rust already by using stuff like PyO3, or then accomplished later in Python through the likes of ctypes or CFFI. Obviously there are also other alternatives like SWIG or others, which I'm not going more deeply into here (which I'm also not really considering for now).
• WASM library execution. This would require similar effort on the wrapping side to expose the schnorrkel functionality to a library. It should be very similar in principle to the schnorrkel-js approach. However on execution then it would require a fully fledged WASM runtime (in whichever language, e.g. Python). However, such a runtime will weigh in severely in additional 'weight' (code deployment and overhead), which is quite cumbersome, unless one is already in need of doing 'WASMy stuff'.

We're right now looking into doing another Rust 'project' (I believe it's called a 'crate' in Rust lingo), which will depend on the schnorrkel crate. In this one then we're wrapping the functionality (similarly to and inspired by schnorrkel-js) to procedural wrappers (unmangled and exported), which then will be compiled to a shared library that is basically C compatible. We'll look at wrapping that one then to be called into by Python.

I suppose an approach like this may be also a simple way to go for other high level languages. So I'm open to suggestions for improvement or further usage downwind. For simple use, the WASM approach is a bit 'heavy' in many cases that are more generic.

[xlc]

One quick attempt for python binding: https://github.com/holygits/schnorpy

[pohutukawa]

We've made a first cut of a PyO3 wrapped Python library. Packaging/building is still the most sore point. And not the entire schnorrkel functionality is wrapped. But feel free to have a look:
https://gitlab.com/kauriid/schnorrpy/