Support for Windows

[mat-hek]

Even though Bundlex provides some support for Windows, the last time we made it work was several years ago and it was a tough experience. It's problematic, as many plugins depend on Bundlex, and the only sensible way to make them run on Windows is to use WSL, which may be unacceptable for some users. Thus, proper support for Windows in Bundlex would be very welcome. The needed features are:

• Compiling C code to a DLL
• Dynamically linking dependencies installed in the system

To provide the support we have on Linux/OS X, we'd as well need:

• Compiling C++ code to a DLL
• Statically linking Bundlex dependencies
• Precompilation of system dependencies

I created a simple project that contains just simple C code and no dependencies: https://github.com/membraneframework-labs/bundlex_simple_example - making that work on Windows could be a good starter.

To provide these features, I'd recommend having a look at Zig toolchain. It's an easy-to-use toolchain (compared to Visual Studio, for example) that handles the compilation of both Zig and C. There is also Zigler, which supports Windows. I'm unsure about C++ though - however, we don't need that for the start, as only a few plugins use it.

Related discussion:

• Cannot run on livebook windows #775
• Unable to compile for MSVC 2022 bundlex#95
• Possible to compile using mingw? bundlex#82
• Thread on Membrane Discord

As for now, we don't plan to work on this, but we can provide help & assistance to anyone willing to contribute.

[harrisi]

:)

The solution ended up being pretty straightforward - went with the zig toolchain route. Getting there took kind longer than I had hoped, but half of that was just fighting with the terminal. There are only a few changes to be made to Bundlex to get here, so I'll open a PR this weekend hopefully.

Two things of note, maybe: (1) zig can compile C++ as well, as far as I understand, and (2) I have no idea if it'll work for more complex projects. I was going to try getting it set up for the simple pipeline demo, but I don't have time right now.

The only other changes were mostly just small, typical Windows vs *nix things like .dll vs .so, \r\n, command flags.. not a ton.

[mat-hek]

Awesome! Can't wait for the PR ;)

half of that was just fighting with the terminal

just Windows things 🫠

zig can compile C++ as well, as far as I understand

that's good to hear

I have no idea if it'll work for more complex projects

We try to keep the C/C++ code simple, so the projects generally aren't much more complex ;)

I was going to try getting it set up for the simple pipeline demo

For that to work, we need to link MAD and PortAudio libraries. I'd start with MAD (https://github.com/membraneframework/membrane_mp3_mad_plugin), it should be simpler.

[harrisi]

Took a little longer than I had hoped - jury duty is more draining than I realized.

I just opened membraneframework/bundlex#132, which is actually just for doing Windows builds on Windows machines. There are still some things I want/need to add, but it's enough to support Windows builds in some capacity.

The zig toolchain stuff is still something that required a few changes in various places due to how the custom compiler is set up (defaulting mostly to the Unix toolchain, which is sometimes awkward to use as a base on Windows), so I'm going to have a separate PR for that soon (hopefully jury duty is over soon).

I haven't tested with MAD or PortAudio yet. Not sure what all I need to do yet.

[mat-hek]

Thanks for the PR!

defaulting mostly to the Unix toolchain, which is sometimes awkward to use as a base on Windows

It's very much possible that the Unix toolchain contains generic code that is not Unix-specific, so feel free to extract/rename these generic parts or just use them and we'll refactor later ;)

hopefully jury duty is over soon

🤞 for that

[harrisi]

I feel mostly okay with where the current PR for MSVC support is, but I'm still struggling with cross platform building in general (not just Bundlex, it's generally troublesome).

So far I've, by a long shot, had the most luck experimenting with the zig toolchain. I was wondering if there's any interest in having that be a dedicated toolchain option for Bundlex. Currently the options needed for it are pretty verbose and redundant, and could be simplified.

I also wonder if there's any overlap with what Burrito has done that Bundlex could utilize.

[mat-hek]

I'm still struggling with cross platform building in general (not just Bundlex, it's generally troublesome).

Yeah, that's why I think it's simpler to use a Docker/CI/VM to avoid cross-platform building

I was wondering if there's any interest in having that be a dedicated toolchain option for Bundlex

Definitely!

I also wonder if there's any overlap with what Burrito has done that Bundlex could utilize.

Not sure what it does there, but it seems it's not perfect either. I've recently been struggling to build a project with elixir-image on Ubuntu, even without cross-compilation. Gave up on it for now.

[harrisi]

Yeah, that's why I think it's simpler to use a Docker/CI/VM to avoid cross-platform building

I generally agree, I think. I suppose getting Windows builds working natively was maybe a useful step for that. I still like the idea of not having to maintain several different images for builds if possible, but I don't know if that's actually a problem people care about.

Not sure what it does there, but it seems it's not perfect either. I've recently been struggling to build a project with elixir-image on Ubuntu, even without cross-compilation. Gave up on it for now.

There are definitely some issues with Burrito, but when it works it works pretty well. I haven't dived too deep into it, but I think it's zig build setup is pretty close to what Bundlex's would be.

Anyway, I'll look more into the zig and custom toolchain stuff when I have time, but for now I've spent more time than I wanted on the Windows stuff. Gotta get back to actually getting my stuff working :)

Still, feel free to ping me if any new Windows or cross-platform issues come up.

[mat-hek]

Sure, thanks again for the contribution!