Hi!
I have been playing a bit with Apache Cassandra, and wanted to play with its Map-Reduce companion Spark.
I have been unable to find a client library for Go. Do you know any before I start writing one?
/Klaus
Spark uses serialized Java objects to communicate between client and master server and between nodes and java byte code as format for executable jobs. Creating a Go client would be quite difficult.
I too am looking to start using spark in some ways at my dayjob. FYI, we are invested in go as our primary backend programming language (new tools or services, migrating older services, data pipelining and engineering, etc…).
For me, I did not even consider looking at go for interacting with Spark, i brought the decision to the team as either python or scala… if that is any indication about go and spark. As a point of reference and another indication about go + spark, I am also looking at using our hadoop infrastructure too and feel fine about go with hadoop (ultimately it depends on my needs).
I have since looked for go spark clients or support or any hint that a company has open-sourced a package (specifically Databricks or Hortonworks) and I’ve come up empty. So at least for me, when it comes to production ready data, I’d be okay using go with Hadoop (our Spark instance is built against Hadoop) as an isolated thing or possibly for pre-processing that I could run spark against… but i would not feel comfortable at all using any currently existing go package for spark.
jbowles - Thanks. I have come to the same conclusion so far. I have not had any opportunity to look deeper into the possibilities, so I have not pursued it further.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.