Hi hi all
I wrote a Python function that consumes a gzip’d file from AWS S3, and posts the lines/records onto a confluent Kafka topic, it runs at just over 8000 records/lines per second.
I then duplicated the functionality in Golang… had to build the golang version into a docker container to get it to work due to the usage of c modules by the confluent package.
strange thing, it only runs at 5128 constantly/second…
same lambda memory allocations, same basic design/steps etc.
Any chance anyone is willing to take a peak and make suggestions how to get the golang faster… I expected it faster, actually allot…
G