How to propagate -ldflags and cross-compile using go build CGO_ENABLED

josephroberts · March 15, 2017, 12:52pm

I am trying to cross-compile a Golang application for a Raspberry Pi with CGO_ENABLED=1.

During compilation I get the error:

arm-linux-gnueabihf/bin/ld: warning: libudev.so.1, needed by /usr/lib/arm-linux-gnueabihf/libusb-1.0.so, not found (try using -rpath or -rpath-link)

I have tried multiple variations of the command below with no luck:

CGO_ENABLED=1 GOARCH=arm GOARM=7 PKG_CONFIG_LIBDIR=/usr/lib/arm-linux-gnueabihf/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ go build -x -ldflags '-extld=arm-linux-gnueabihf-gcc -rpath=/usr/lib/arm-linux-gnueabihf -L/usr/lib/arm-linux-gnueabihf -ludev'

When I run the build with the -x flag I get the verbose output, and it kinda looks like its failing whilst building the src/net package:

cd /usr/local/go/src/net
/usr/local/go/pkg/tool/linux_amd64/cgo -dynpackage net -dynimport $WORK/net/_obj/_cgo_.o -dynout $WORK/net/_obj/_cgo_import.go
arm-linux-gnueabihf-gcc -I . -fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o $WORK/net/_obj/_all.o $WORK/net/_obj/_cgo_export.o $WORK/net/_obj/cgo_linux.cgo2.o $WORK/net/_obj/cgo_resnew.cgo2.o $WORK/net/_obj/cgo_socknew.cgo2.o $WORK/net/_obj/cgo_unix.cgo2.o -g -O2 -Wl,-r -nostdlib -Wl,--build-id=none
/usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/net.a -trimpath $WORK -p net -buildid 9e58c94d1ddeba63666a35ecee9409056baf5d3c -D _/usr/local/go/src/net -I $WORK -pack ./addrselect.go ./conf.go ./dial.go ./dnsclient.go ./dnsclient_unix.go ./dnsconfig_unix.go ./dnsmsg.go ./fd_mutex.go ./fd_poll_runtime.go ./fd_posix.go ./fd_unix.go ./file.go ./file_unix.go ./hook.go ./hook_cloexec.go ./hook_unix.go ./hosts.go ./interface.go ./interface_linux.go ./ip.go ./iprawsock.go ./iprawsock_posix.go ./ipsock.go ./ipsock_posix.go ./lookup.go ./lookup_unix.go ./mac.go ./net.go ./nss.go ./parse.go ./pipe.go ./port.go ./port_unix.go ./sendfile_linux.go ./sock_cloexec.go ./sock_linux.go ./sock_posix.go ./sockopt_linux.go ./sockopt_posix.go ./sockoptip_linux.go ./sockoptip_posix.go ./tcpsock.go ./tcpsock_posix.go ./tcpsockopt_posix.go ./tcpsockopt_unix.go ./udpsock.go ./udpsock_posix.go ./unixsock.go ./unixsock_posix.go ./writev_unix.go $WORK/net/_obj/_cgo_gotypes.go $WORK/net/_obj/cgo_linux.cgo1.go $WORK/net/_obj/cgo_resnew.cgo1.go $WORK/net/_obj/cgo_socknew.cgo1.go $WORK/net/_obj/cgo_unix.cgo1.go $WORK/net/_obj/_cgo_import.go
# github.com/resin-io/edge-node-manager/vendor/github.com/kylelemons/gousb/usb
/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian-x64/bin/../lib/gcc/arm-linux-gnueabihf/4.8.3/../../../../arm-linux-gnueabihf/bin/ld: warning: libudev.so.1, needed by /usr/lib/arm-linux-gnueabihf/libusb-1.0.so, not found (try using -rpath or -rpath-link)
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_filter_add_match_subsystem_devtype@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_enable_receiving@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_enumerate_scan_devices@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_new@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_device_get_devnode@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_enumerate_new@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_list_entry_get_name@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_enumerate_add_match_subsystem@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_device_get_sysname@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_enumerate_get_list_entry@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_device_new_from_syspath@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_new_from_netlink@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_receive_device@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_device_unref@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_enumerate_unref@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_get_fd@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_list_entry_get_next@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_monitor_unref@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `clock_gettime@GLIBC_2.17'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_unref@LIBUDEV_183'
/usr/lib/arm-linux-gnueabihf/libusb-1.0.so: undefined reference to `udev_device_get_action@LIBUDEV_183'
collect2: error: ld returned 1 exit status

The strange thing is that the compiler can find /usr/lib/arm-linux-gnueabihf/libusb-1.0.so and if I do ls -al in that directory libudev.so is also present. Both files linking back to /lib/arm-linux-gnueabihf/

lrwxrwxrwx 1 root root     41 Jan  7 04:14 libudev.so -> /lib/arm-linux-gnueabihf/libudev.so.1.5.0
-rw-r--r-- 1 root root 105464 Jun 16  2014 libusb-1.0.a
lrwxrwxrwx 1 root root     44 Jun 16  2014 libusb-1.0.so -> /lib/arm-linux-gnueabihf/libusb-1.0.so.0.1.0
drwxr-xr-x 2 root root   4096 Mar 15 14:48 pkgconfig

/usr/lib/arm-linux-gnueabihf/pkgconfig contains:

-rw-r--r-- 1 root root  513 Jan  7 04:13 libudev.pc
-rw-r--r-- 1 root root  332 Jun 16  2014 libusb-1.0.pc

Finally, running file shows both libusb and libudev to be the correct architecture:

/lib/arm-linux-gnueabihf/libusb-1.0.so.0.1.0: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=7d7b487c91042c1f254b39516b4aeabedfb828fc, stripped
/lib/arm-linux-gnueabihf/libudev.so.1.5.0: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=334f4f0d8b1222a324ccd0827c1613fdd1e2ec15, stripped

Any ideas? Thanks in advance.

josephroberts · March 15, 2017, 1:14pm

I thought I had fixed this by changing the PKG_CONFIG_LIBDIR line from

PKG_CONFIG_PATH=/usr/lib/arm-linux-gnueabihf/pkgconfig

to

PKG_CONFIG_LIBDIR=/usr/lib/arm-linux-gnueabihf/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig

EDIT: I spoke too soon It worked because I removed the github.com/kylelemons/gousb dependency, once I put that back in I am back to the same isue

dfc · March 15, 2017, 7:37pm

It’s no secret that I don’t think much of cross compilation with cgo enabled.

Given you already have the pi and there are binary releases of Go available for arm, why not compile locally? It’ll take you a few minutes as opposed to a few seconds on an Intel machine. But I’ll wager that is less time than you’ve spent trying to make cgo enabled cross complication work.

josephroberts · March 16, 2017, 9:42am

OK, thanks for the feedback. This is part of an automated deployment pipeline so I was trying to avoid building on a Pi.

Gonna get a pi build server hooked up with Travis CI and S3 now

Maxim · March 20, 2017, 9:11am

I would suggest using xgo (https://github.com/karalabe/xgo).

I successfully used it to build arm binaries from code depending on the sqlite driver, which requires CGO.

Maxim

system · June 18, 2017, 9:12am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.