"go test" with CGO on macOS and DYLD_LIBRARY_PATH

I have been trying to get one of our company’s project that leverages CGO and builds on Linux to build on macOS. To that end, I created a little CGO Hello World test project.

I can get my test project to build and run executables and it respects my environment setting for DYLD_LIBRARY_PATH just fine (since macOS System Integrity Protection will allow DYLD_LIBRARY_PATH to be inherited by non-system executables – i.e.: executables that exist in non-system locations).

HOWEVER, when I run “go test” it fails to find my native library on DYLD_LIBRARY_PATH. I tried setting the environment variable CGO_LDFLAGS and that changes the output that I get, but it still ultimately complains about not finding the native library.

Here is the output without setting CGO_LDFLAGS:

% go test
# localhost/cgo2.test
/Users/barry/dev/go/pkg/tool/darwin_amd64/link: running clang failed: exit status 1
ld: library not found for -lhello
clang: error: linker command failed with exit code 1 (use -v to see invocation)

FAIL	localhost/cgo2 [build failed]

Here is the output after setting it:

% echo "CGO_LDFLAGS = $CGO_LDFLAGS"; go test
CGO_LDFLAGS = -L/Users/barry/testdir/
dyld[68952]: Library not loaded: libhello.so
  Referenced from: <0F30CBC8-09BB-38FF-A929-66A67266A540> /private/var/folders/fl/yvgrj4p53w7blwlr0zwtf_5h0000gn/T/go-build1534663025/b001/cgo2.test
  Reason: tried: 'libhello.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhello.so' (no such file), 'libhello.so' (no such file), '/usr/local/lib/libhello.so' (no such file), '/usr/lib/libhello.so' (no such file, not in dyld cache), '/Users/barry/testdir/chello/libhello.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/barry/testdir/chello/libhello.so' (no such file), '/Users/barry/testdir/chello/libhello.so' (no such file), '/usr/local/lib/libhello.so' (no such file), '/usr/lib/libhello.so' (no such file, not in dyld cache)
signal: abort trap
FAIL	localhost/cgo2	0.104s

My educated guess is that “go test” is spawning a child process using the default system shell and that shell will not inherit the DYLD_LIBRARY_PATH.

Anybody else building/testing Go language applications using CGO on macOS and figured out how to get it to link to dynamic libraries in non-standard locations?

I would really rather not put all my development builds of my native libraries in /usr/local/lib to get this working.

NOTE: I made sure to install the Go executables in my home directory rather than /usr/local/ to make sure macOS did not see “go” as a system executable.

Follow-up: I tried setting the TMPDIR environment variable so the text executable was not in the macOS /private directory. I set it to my $HOME/temp – the hope here was to disable SIP (System Integrity Protection) by converting it to a “user executable” rather than a “system executable”, but it would seem that this does not work either. The console still shows the following with “System Integrity Protection” as “enabled”.

Translated Report (Full Report Below)

Process:               cgo2.test [75437]
Path:                  /Users/USER/*/cgo2.test
Identifier:            cgo2.test
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        go [75383]
Responsible:           Terminal [422]
User ID:               501

Date/Time:             2023-07-11 11:05:17.2703 -0700
OS Version:            macOS 13.4 (22F66)
Report Version:        12
Bridge OS Version:     7.5 (20P5058)
Anonymous UUID:        4E0686DB-AE71-A92E-37DB-85FA5B2631D2

Sleep/Wake UUID:       F52AC1CC-E55A-410F-86D2-47B4CA8BE1A8

Time Awake Since Boot: 240000 seconds
Time Since Wake:       6575 seconds

System Integrity Protection: enabled

Crashed Thread:        0

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace DYLD, Code 1 Library missing
Library not loaded: libhello.so
Referenced from: <59F3BBCE-CE7F-3677-82DB-06D6882D729F> /Users/USER/*/cgo2.test
Reason: tried: 'libhello.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhello.so' (no such file), 'libhello.so' (no such file), '/usr/local/lib/libhello.so' (no such file), '/usr/lib/libhello.so' (no such file, not in dyld cache), '/Users/barry/dev/senzing/cgo2/libhello.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/barry/dev/senzing/cgo2/libhello.so' (no such file), '/Users/barry/dev/senzing/cgo2/libhello.so' (no such file), '/usr/local/lib/libhello.so' (no such file), '/usr/lib/libhello.so' (no such file, not in dyld cache)
(terminated at launch; ignore backtrace)

Thread 0 Crashed:
0   dyld                          	    0x7ff8065e6c42 __abort_with_payload + 10
1   dyld                          	    0x7ff806600fd7 abort_with_payload_wrapper_internal + 82
2   dyld                          	    0x7ff806601009 abort_with_payload + 9
3   dyld                          	    0x7ff8065858f0 dyld4::halt(char const*) + 375
4   dyld                          	    0x7ff806582b71 dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*) + 4526
5   dyld                          	    0x7ff8065813bd start + 1805

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000002000209  rbx: 0x0000000000000000  rcx: 0x00007ff7bfefdfc8  rdx: 0x00007ff7bfefe430
  rdi: 0x0000000000000006  rsi: 0x0000000000000001  rbp: 0x00007ff7bfefe010  rsp: 0x00007ff7bfefdfc8
   r8: 0x00007ff7bfefe030   r9: 0x0000000000000000  r10: 0x0000000000000054  r11: 0x0000000000000246
  r12: 0x0000000000000054  r13: 0x00007ff7bfefe430  r14: 0x0000000000000001  r15: 0x0000000000000006
  rip: 0x00007ff8065e6c42  rfl: 0x0000000000000246  cr2: 0x000000010021a000
Logical CPU:     0
Error Code:      0x02000209 
Trap Number:     133

I solved this… the TMPDIR was not the answer. The answer was using go test -exec [my-script] to execute the test using a script that could first setup the DYLD_LIBRARY_PATH. This is a similar solution to what I had to do with Java/Junit/Maven which also forks a process to run the tests.

It boils down to CGO_LDFLAGS will get you linking and the -exec xprog option to go test will let you create a script to work around System Integrity Protection and set your DYLD_LIBRARY_PATH.