when I do a go run main.go it runs - on my MBP -M1
when I do a go build . it builds,
when I run try and build it with the below exports or execute the Makefile it fails - error below
<ps I’ve tried this same command/steps on a amizonlinux based EC2 instance (x86-64) on AWS, as I intend to deploy to Lambda>
BINARY_NAME=main
export GOOS=linux
export GOARCH=amd64
export CGO_ENABLED=0
.DEFAULT_GOAL := deploy
deploy:
go build -o ${BINARY_NAME} .
zip -r function.zip main
aws lambda update-function-code --function-name "S3JSONDecomposer-Golang" --zip-file fileb://function.zip --region="af-south-1" | jq .
run:
go run ${BINARY_NAME}.go
I assume you use confluent-kaftka-go. This library is based on the C library librdkafka, therefore you need CGO for compiling.
Cross-compiling is effortless for pure Go code only. I would perhaps use a VM on the Mac (Docker Desktop, Podman Desktop, OrbStack, Colima,…) to set up a Linux build environment.
i’ve tried (with Python) docker compiling and sadly always battle, for Python I ended building this EC2 images, that has the target OS/Architecture…
As mentioned, tried this compile on the EC2 instance also and ended with the same error, will try again quickly, never know
setting CGO_ENABLED=1 (in the MAC, lets go try on my EC2 instance)
go build -o main .
# runtime/cgo
linux_syscall.c:67:13: error: call to undeclared function 'setresgid'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
linux_syscall.c:67:13: note: did you mean 'setregid'?
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:593:6: note: 'setregid' declared here
linux_syscall.c:73:13: error: call to undeclared function 'setresuid'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
linux_syscall.c:73:13: note: did you mean 'setreuid'?
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:595:6: note: 'setreuid' declared here
make: *** [deploy] Error 1
with my code copied onto the EC2 instance, based on Amazon Linux… x86-64… I’m back at
this is with CGO=0 … as I’d assume it’s the same architecture as my target, and it’s x86-64 so i’m guessing it’s the same architecture as the source.
if I set CGO=1 then it complains for lack of C compiler. have to see what c compiler to install, never done that before.
make deploy
go build -o main .
# jsondecom
./main.go:130:14: undefined: kafka.ConfigMap
./main.go:142:25: undefined: kafka.NewProducer
make: *** [Makefile:11: deploy] Error 1
eish…
it seems the required glib libs that i installed on my EC2 image to get it compiled is not on the lambda OS by default, strange through, lack of knowledge, but i thought this would be a complete/self contained binary…
/var/task/main: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/main)
2023/07/02 07:47:37 exit status 1
/var/task/main: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.25' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/main)
/var/task/main: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/main)
2023/07/02 07:47:37 exit status 1
Only wir pure Go. Any included C code has C dependencies.
I don’t know anything about LambdaOS, but I found this Stackoverflow thread that indicates that you might need to set LD_LIBRARY_PATH and probably also supply the required glibc somehow.
probably have to modify this, you can set environment variables…
but to fit in with the concept you probably don’t want to install OS level packages, as the idea is you provide your code and it’s dynamically pushed into a docker container and executed.
Hi there
By having build my binary on a amazonlinux based EC2 I accomplish the very same,
I’ve also tried building inside a container and I run into the same problems at the moment.
the base container does not include the libraries, as it seems even if i follow the static link method, there is some components that’s still dynamically linked which is not in the target container used by Lambda when it initialise the lambda function,
I’m a bit fighting with a problem, with not enough knowledge/experience on either Lambda or Golang atm.
G
Get what you thinking, not sure… I know you give it a zip file, that contains the main, not sure if that zip can contain requirements and if they can be copied around.
for me… it also does not make sense… the exe is suppose to be all encompassing/self contained, otherwise if you copy it from one host to the next… you end a deployment nightmare, the entire thing that the single golang binary is trying to fix.
I saw in one of the videos/notes that you can build the docker image that is used to deploy your app, wondering if that won’t work, in that case instead of lambda pushing my binary into a image and invoking it, it deploys my provided image (which is then everything required), but if i think about that, then I’m heading to why not just do this on our own K8S / EKS cluster, for me at that stage the benefit of the Lambda deployment is watered down/reduced.
bit stuck, looking at the instructions, I’ve build a Dockerfile from the instructions…
of course when I then run and then invoke the docker container I get the known problems.
So back to the link, it talks about compiling those libra files by using that .configure… WELL where is that .configure from, and where is the source files that it will compile from?
it then copies them into {$PWD}/lib/, but which is this, inside the container.
at times wish that people that write these how to fix problems step down and work on concept that the person executing them might not have same base knowledge they do.
of course once i get this local run/invoke working will have to look at how to deploy using AWS ECR… saw a note somewhere re it.
Ye golang it seems is not Lambda friendly.
G
Consider performing a static link of the Go and C. This article explain in great detail the process and caveats of doing such a thing. Matt Turner - Statically Linking Go in 2022
Good luck
