Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MacOS Sonoma (Mac M2) issue uploading firmware with WCH-Link (+workaround) #44

Open
scomx opened this issue Jan 24, 2024 · 26 comments
Open
Assignees
Labels
bug Something isn't working package uploader

Comments

@scomx
Copy link

scomx commented Jan 24, 2024

Hi,

After installing for VSCode as per instructions I was able to build but not flash firmware (noneos-sdk) because of libusb not being found. This despite having both Homebrew and MacPorts instances of libusb installed on my Mac M2.

Configuring upload protocol...
AVAILABLE: isp, minichlink, wch-link
CURRENT: upload_protocol = wch-link
Uploading .pio/build/ch32v307_evt/firmware.elf

dyld[10087]: Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: <6AC9F4F5-91C8-347E-9C66-6C2E77C81C80> /Users/metx/.platformio/packages/tool-openocd-riscv-wch/bin/openocd
  Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/lib/libusb-1.0.0.dylib' (no such file), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)

Copying libusb-1.0.0.dylib (either Homebrew or MacPort) to /usr/lib/ generated the above error plus the following:

(mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))

Workaround was to download pre-built binaries from https://sourceforge.net/projects/libusb/ and copy libusb-1.0.0.dylib (macos11_6/lib/ directory) to /usr/lib.

Clunky but at least now I can flash and debug. My questions, why x86_64? I assume this is a WCH/OpenOCD thing?

Also, is it possible to add additional search paths somewhere in the project config so that DYLD can find the appropriate libusb? Currently it is looking for it in all the wrong places.

Anyway, great to see PlatformIO support being added for these chips, well done!

@maxgerhardt
Copy link
Member

For the binaries, I'm using what WCH provides as precompiled (http://www.mounriver.com/download). Sources are hard to find and sometimes outdated (although they are technically obliged to release them through the GPL.).

But yes, other PlatformIO packages bundle the necesary dylib files in the lib/ folder. The OpenOCD package used here should do the same. Let's see if I can't update it..

@maxgerhardt
Copy link
Member

maxgerhardt commented Jan 24, 2024

Yes, none of the Mac packages (neither darwin x86_64 nor arm64) has a lib/ folder with the needed .dylib file per https://api.registry.platformio.org/v3/packages/community-ch32v/tool/tool-openocd-riscv-wch. I should correct that but I have to reupload the packages for all OSes again.

@maxgerhardt maxgerhardt self-assigned this Feb 2, 2024
@maxgerhardt maxgerhardt added bug Something isn't working package uploader labels Feb 2, 2024
@domhel
Copy link

domhel commented Feb 3, 2024

Workaround was to download pre-built binaries from https://sourceforge.net/projects/libusb/ and copy libusb-1.0.0.dylib (macos11_6/lib/ directory) to /usr/lib.

Thanks for sharing. Had to copy it to /usr/local/lib and set permissions with sudo xattr -rd com.apple.quarantine /usr/local/lib/libusb-1.0.0.dylib, then it worked.

@maxgerhardt
Copy link
Member

I've published a package update to first of all update to the latest OpenOCD version (might resolve CH32X035 issues) and second of all the Mac x64 and ARM packages now have the libusb dylib prepackaged in them. It now awaits moderation by the PlatformIO team.

@francismariano
Copy link

Hi folks.

I have the same issue here. @maxgerhardt any update with the moderation of PIO team?

Thanks.

@maxgerhardt
Copy link
Member

maxgerhardt commented Feb 12, 2024

Still no update. Depending on whether you're on Windows, Linux or Mac I suggest injecting the right package as workaround for now, by adding to the platformio.ini

; Use only one.
; for Windows
platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git
; for Linux
platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#linux
; for MacOS x64
platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_x64
; for MacOS ARM (M1 etc.)
platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_arm

@maxgerhardt
Copy link
Member

maxgerhardt commented Feb 13, 2024

I have now added this workaround by default on the platform side, so that the above is not necessary anymore: 188074b

Uploading to a CH32X035 works for me that way in Windows.

Any feedback regarding Mac would be appriecated to.

You can update your platform using CLI -> pio pkg update -g -p ch32v

@scomx
Copy link
Author

scomx commented Feb 14, 2024

Update went fine, however, flashing problem remains:

dyld[54142]: Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: <70E70692-EF1D-3B00-8443-F53525ACCE20> /Users/metx/.platformio/packages/tool-openocd-riscv-wch/bin/openocd
  Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)
*** [upload] Error -6

@francismariano
Copy link

francismariano commented Feb 14, 2024

The same here. I created o symbolic link in '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' but with no success

dyld[63161]: Library not loaded: /usr/local/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: <70E70692-EF1D-3B00-8443-F53525ACCE20> /Users/francis/.platformio/packages/tool-openocd-riscv-wch/bin/openocd
  Reason: tried: '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/usr/local/opt/libusb/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache), '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (no such file), '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)

(mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))

@francismariano
Copy link

I also tried to add platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_arm

Uploading .pio/build/genericCH32V003F4U6/firmware.elf
sh: openocd: command not found
*** [upload] Error 127

@maxgerhardt
Copy link
Member

Great, then the two MacOS packages I've submitted are a catastrophic failure. Cursed be this OS.

@Jason2866 sorry to tag you directly, could this issue of the openocd binary not finding the libusb-1.0.0.dylib file although it's in the same folder because of the RPath of the Mach-O binary? Per [https://stackoverflow.com/questions/33991581/install-name-tool-to-update-a-executable-to-search-for-dylib-in-mac-os-x] this might be resolved by executing install_name_tool -add_rpath "@loader_path/" openocd, but that tool is Mac only; Could you try this on your machine?

@Jason2866
Copy link

Jason2866 commented Feb 14, 2024

@maxgerhardt Built openocd for espressif32 myself on my M1. Did not run in this issue.
My guess there is something wrong or missing in bootstrap / configure.
Maybe the GH Action (Apple ARM runner) workflow to build for espressif32 gives you an idea
https://github.com/Jason2866/openocd-esp32/blob/m1/.github/workflows/build.yml

@francismariano
Copy link

@maxgerhardt after updating the framework with pio pkg update -g -p ch32v the tool manager is installing the darwin_x64 but my platform is darwin_arm

Resolving genericCH32V003F4U6 dependencies...
Tool Manager: Installing git+https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_x64
git version 2.39.3 (Apple Git-145)

@maxgerhardt
Copy link
Member

@francismariano what's the output of

pio system info

in the PIO core cli? PlatformIO may still, for some reason, decide to install the x64 version on your ARM machine. I think Jason even into that.

@francismariano
Copy link

francismariano commented Feb 14, 2024

--------------------------  ----------------------------------------------
PlatformIO Core             6.1.13
Python                      3.11.7-final.0
System Type                 darwin_x86_64
Platform                    macOS-10.16
File System Encoding        utf-8
Locale Encoding             UTF-8
PlatformIO Core Directory   /Users/francis/.platformio
PlatformIO Core Executable  /Users/francis/.platformio/penv/bin/platformio
Python Executable           /Users/francis/.platformio/penv/bin/python
Global Libraries            0
Development Platforms       1
Tools & Toolchains          7
--------------------------  ----------------------------------------------

But my mac has an Apple M2 chip

@maxgerhardt
Copy link
Member

Then see platformio/platformio-vscode-ide#3949. But this shouldn't matter too much, then it should just use the x64 packages; But those don't find their dylib as well.

@francismariano
Copy link

Ok. Another thing.

When I force platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_arm and running directly the openocd I get the following message:

% ./openocd       
dyld[70281]: Library not loaded: /opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: <6BBF79F9-C1A5-38BD-85E2-6A75FB8D03BD> /Users/francis/.platformio/packages/tool-openocd-riscv-wch/bin/openocd
  Reason: tried: '/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (code signature in <F82823ED-3641-3C62-B22D-3E917A1DCB81> '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (code signature in <F82823ED-3641-3C62-B22D-3E917A1DCB81> '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache), '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (code signature in <F82823ED-3641-3C62-B22D-3E917A1DCB81> '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (no such file), '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' (code signature in <F82823ED-3641-3C62-B22D-3E917A1DCB81> '/opt/homebrew/Cellar/libusb/1.0.26/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)
zsh: abort      ./openocd

@maxgerhardt
Copy link
Member

It doesn't even attempt to look in the current directory for the file; I don't entirely get that and I still think this is an RPath issue in the openocd Macho-Binary. Since MounRiver builds these binaries (without releasing source code although this is GPL covered, hm hm), I can't really influence the build process, only try to post-path their binaries.. But, that would require me to have a Mac.

Can you check with the now cloned binary whether this command helps at all?

install_name_tool -add_rpath "@loader_path/" openocd

@francismariano
Copy link

francismariano commented Feb 14, 2024

I ran the command above

 % ./openocd
zsh: killed     ./openocd

I dunno if the opencd was executed successfully. I tried run directly into vscode (with upload action) but the opencd is not executed.

Uploading .pio/build/genericCH32V003F4U6/firmware.elf
sh: openocd: command not found
*** [upload] Error 127

PS. Even before to execute the command above, the opencd was not executed into vscode when the platform-package is forced inside platformio.ini file.

@maxgerhardt
Copy link
Member

maxgerhardt commented Feb 14, 2024

Sigh. This is going to prove difficult.

I might just add to the readme that libusb has to be manually installed for this to work, if I can't patch the binary to properly use the included libusb dylib. This is what https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch/tree/darwin_arm says in the README after all too.

Just to confirm, openocd does work correctly if you now rm -rf ~/.platformio/packages/tool-openocd-riscv-wch* again and brew install libusb for the x64 variant? arch -x86_64 brew install libusb

@francismariano
Copy link

Sorry, I cant install x64 variant because the rosetta is not install for now. I will install rosetta and I let you know.

Do you know the reason the opencd is not found when platform-package is forced into platformio.ini??

@francismariano
Copy link

francismariano commented Feb 14, 2024

Just out of curiosity, is not possible to install the open-ocd via homebrew and binding it to vscode??

@maxgerhardt
Copy link
Member

PlatformIO actively discourages that, all tools it needs should be packaged as PlatformIO packages (~/.platformio/packages), nothing should be assumed for the host system.

Also, the openocd installed via homebrew will absolutely not work, as it specifically needs to be the WCH variant. That is not added in upstream OpenOCD at all.

@Jason2866
Copy link

Jason2866 commented Feb 14, 2024

Great, then the two MacOS packages I've submitted are a catastrophic failure. Cursed be this OS.

@Jason2866 sorry to tag you directly, could this issue of the openocd binary not finding the libusb-1.0.0.dylib file although it's in the same folder because of the RPath of the Mach-O binary? Per [https://stackoverflow.com/questions/33991581/install-name-tool-to-update-a-executable-to-search-for-dylib-in-mac-os-x] this might be resolved by executing install_name_tool -add_rpath "@loader_path/" openocd, but that tool is Mac only; Could you try this on your machine?

Not at home so I can't test the next days. After some googling i do understand the problem, but probably hard to solve without trying things.
Earle is doing this for his OpenOCD version for Pico. Maybe his GH Action to build helps you
https://github.com/earlephilhower/pico-quick-toolchain/blob/master/.github/workflows/build-macos-arm.yml
and
https://github.com/earlephilhower/pico-quick-toolchain/blob/master/macos-relative-dylibs.sh

@francismariano
Copy link

Hello everyone. Good news here.

First, I comment the line platform_packages = tool-openocd-riscv-wch@https://github.com/Community-PIO-CH32V/tool-openocd-riscv-wch.git#darwin_arm to download the default packages.

So I downloaded MRS_Toolchain_MAC_V191 in http://www.mounriver.com/download page.

That file has openocd to arm64 and x86_654 architecture.

image

otool -L openocd                                                                
openocd:
	/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib (compatibility version 4.0.0, current version 4.0.0)
	/opt/homebrew/opt/hidapi/lib/libhidapi.0.dylib (compatibility version 0.0.0, current version 0.13.1)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

The opencd executable points to libusb and libhidapi installed by homebrew.

When I tried to execute the opencd I get error of code signature and invalid IDs:

./openocd 
dyld[90625]: Library not loaded: /opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib
  Referenced from: <6BBF79F9-C1A5-38BD-85E2-6A75FB8D03BD> /Users/francis/.platformio/packages/openocd_arm64/bin/openocd
  Reason: tried: '/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (code signature in <8DD5BDC1-E490-325E-957C-BDD4B75715C8> '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (no such file), '/opt/homebrew/opt/libusb/lib/libusb-1.0.0.dylib' (code signature in <8DD5BDC1-E490-325E-957C-BDD4B75715C8> '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache), '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' (code signature in <8DD5BDC1-E490-325E-957C-BDD4B75715C8> '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' (no such file), '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' (code signature in <8DD5BDC1-E490-325E-957C-BDD4B75715C8> '/opt/homebrew/Cellar/libusb/1.0.27/lib/libusb-1.0.0.dylib' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/usr/lib/libusb-1.0.0.dylib' (no such file, not in dyld cache)
zsh: abort      ./openocd

So I executed the following command in openocd

codesign --force -s - openocd and it executed normally now.

So I replaced the openocd in tool-openocd-riscv-wch/bin and the upload performed with success.

@ufanders
Copy link

Following up to @francismariano 's comment, I also got further in my attempt to work on my Mac M3 but I had to do the following:

  • codesign --force -s - openocd, in /Users/<me>/.platformio/packages/tool-openocd-riscv-wch/bin
  • brew install hidapi

I then started running into this error:

Configuring upload protocol...
AVAILABLE: isp, minichlink, wch-link, wlink
CURRENT: upload_protocol = wch-link
Uploading .pio/build/genericCH32V003F4P6/firmware.elf
Open On-Chip Debugger 0.11.0+dev-02415-gfad123a16-dirty (2023-06-12-19:28)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
debug_level: 1

Warn : Transport "sdi" was already selected
Ready for Remote Connections
Error: open failed

*** [upload] Error 1

I got past this by doing 2 things and I'm not sure which method fixed it:

  1. I held down the ModeS button on the Link-E as I inserted it into my USB hub. Only the red LED came on as opposed to the red and blue LEDs.

  2. Using the Windows tool WCH-LinkUtility in my Windows 11 VM, I set the active WCH-Link Mode to WCH-LinkRV.

Screenshot 2024-08-17 at 11 47 47 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working package uploader
Projects
None yet
Development

No branches or pull requests

6 participants