Problems deploying Go+WASM app to Zeit/Now

carlca · June 28, 2018, 9:28pm

If you haven’t already seen it, this post by Brian Ketelsen is really worth reading https://brianketelsen.com/web-assembly-and-go-a-look-to-the-future/. I’ve managed to build the MarkdownVecty app linked to from the article https://github.com/bketelsen/wasmplay/tree/master/markdownvecty and it is really exciting to have a genuine Golang/WASM app running in the browser.

I then wondered if it would be possible to deploy said app to the cloud via Zeit/Now. It’s perfectly possible to deploy simple Golang apps which server simple output, as this page by @lukemorton illustrates… https://www.lukemorton.co.uk/thoughts/2017-01-15-deploying-go-on-zeit-now.

Could someone with experience in Golang + Zeit/Now (not necessarily WASM, I don’t think that’s where my problems lie) have a look at my posts in the Zeit/Now channel on Slack. I feel that I’m getting close to a solution, but that solution is tantalisingly out of reach at the moment!

This is a link to my message in Slack https://zeit-community.slack.com/archives/C1TMPLCKS/p1530218186000512

Thanks a lot for any help anyone can give me!

CurtGreen · June 28, 2018, 10:56pm

I can’t see your slack message because I’m not on that channel, but are you deploying a go server to serve the wasm page or just trying to use now to host the static html and binary output of compiling wasm?

carlca · June 29, 2018, 12:11am

Thanks for the reply, Curtis.

Yes, I am deploying a Go server to serve the WASM app. Note that I am not trying to build the WASM code in Docker. I built the example.wasm locally and I am trying to load the compiled app into the Docker image. This was my first attempt at a Dockerfile…

FROM golang:alpine
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm
ADD . /go/src/github.com/carlca/server
RUN go install github.com/carlca/server
CMD ["/go/bin/server"]
EXPOSE 3000

This gave me errors about dependencies…

now --public                                                                                                                                +869 1:05 ❰─┘
> Deploying ~/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty under carl.caulkett@gmail.com
> https://markdownvecty-frnkxqbkwp.now.sh [in clipboard] (bru1) [6s]
> Building…
> ▲ docker build
Sending build context to Docker daemon 3.891 MBkB
> Step 1/7 : FROM golang:alpine
>  ---> c4b5d89b27f4
> Step 2/7 : ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html
>  ---> 8d798d950a48
> Step 3/7 : ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm
>  ---> 7d91edb3d290
> Step 4/7 : ADD . /go/src/github.com/carlca/server
>  ---> 0ab68faa3748
> Step 5/7 : RUN go install github.com/carlca/server
>  ---> Running in d6dfda8510d2
> src/github.com/carlca/server/markdown.go:4:2: cannot find package "github.com/gowasm/vecty" in any of:
> 	/usr/local/go/src/github.com/gowasm/vecty (from $GOROOT)
> 	/go/src/github.com/gowasm/vecty (from $GOPATH)
> src/github.com/carlca/server/markdown.go:5:2: cannot find package "github.com/gowasm/vecty/elem" in any of:
> 	/usr/local/go/src/github.com/gowasm/vecty/elem (from $GOROOT)
> 	/go/src/github.com/gowasm/vecty/elem (from $GOPATH)
> src/github.com/carlca/server/markdown.go:6:2: cannot find package "github.com/gowasm/vecty/event" in any of:
> 	/usr/local/go/src/github.com/gowasm/vecty/event (from $GOROOT)
> 	/go/src/github.com/gowasm/vecty/event (from $GOPATH)
> src/github.com/carlca/server/markdown.go:7:2: cannot find package "github.com/microcosm-cc/bluemonday" in any of:
> 	/usr/local/go/src/github.com/microcosm-cc/bluemonday (from $GOROOT)
> 	/go/src/github.com/microcosm-cc/bluemonday (from $GOPATH)
> src/github.com/carlca/server/markdown.go:8:2: cannot find package "github.com/russross/blackfriday" in any of:
> 	/usr/local/go/src/github.com/russross/blackfriday (from $GOROOT)
> 	/go/src/github.com/russross/blackfriday (from $GOPATH)
> The command '/bin/sh -c go install github.com/carlca/server' returned a non-zero code: 1
> Error! Build failed

I then tried to address those dependencies with these changes to the Dockerfile…

FROM golang:alpine
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm
ADD . /go/src/github.com/carlca/server
ADD . /go/src/github.com/gowasm/vecty
ADD . /go/src/github.com/gowasm/vecty/elem
ADD . /go/src/github.com/gowasm/vecty/event
ADD . /go/src/github.com/microcosm-cc/bluemonday
ADD . /go/src/github.com/russross/blackfriday
RUN go install github.com/carlca/server
CMD ["/go/bin/server"]
EXPOSE 3000

This gave me some different error which I don’t recognise or understand…

now --public                                                                                                                           1 ⏎  +871 1:09 ❰─┘
> Deploying ~/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty under carl.caulkett@gmail.com
> https://markdownvecty-kmivimddsf.now.sh [in clipboard] (bru1) [4s]
> Building…
> ▲ docker build
Sending build context to Docker daemon 3.892 MBkB
> Step 1/12 : FROM golang:alpine
>  ---> c4b5d89b27f4
> Step 2/12 : ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html
>  ---> cbfb203e3c9e
> Step 3/12 : ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm
>  ---> 94564b6e1973
> Step 4/12 : ADD . /go/src/github.com/carlca/server
>  ---> a24ecf2985cc
> Step 5/12 : ADD . /go/src/github.com/gowasm/vecty
>  ---> e022088f661a
> Step 6/12 : ADD . /go/src/github.com/gowasm/vecty/elem
>  ---> 62dc45676328
> Step 7/12 : ADD . /go/src/github.com/gowasm/vecty/event
>  ---> 27a04c2fd1f4
> Step 8/12 : ADD . /go/src/github.com/microcosm-cc/bluemonday
>  ---> cee6e0d9a801
> Step 9/12 : ADD . /go/src/github.com/russross/blackfriday
>  ---> de56d64728ba
> Step 10/12 : RUN go install github.com/carlca/server
>  ---> Running in 1b6ae9f47c77
> src/github.com/carlca/server/markdown.go:4:2: import "github.com/gowasm/vecty" is a program, not an importable package
> src/github.com/carlca/server/markdown.go:5:2: import "github.com/gowasm/vecty/elem" is a program, not an importable package
> src/github.com/carlca/server/markdown.go:6:2: import "github.com/gowasm/vecty/event" is a program, not an importable package
> src/github.com/carlca/server/markdown.go:7:2: import "github.com/microcosm-cc/bluemonday" is a program, not an importable package
> src/github.com/carlca/server/markdown.go:8:2: import "github.com/russross/blackfriday" is a program, not an importable package
> The command '/bin/sh -c go install github.com/carlca/server' returned a non-zero code: 1
> Error! Build failed

The server part is a simple Golang app…

package main

import (
	"log"
	"net/http"
)

func wasmHandler(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/wasm")
	http.ServeFile(w, r, "example.wasm")
}
func main() {
	mux := http.NewServeMux()
	mux.Handle("/", http.FileServer(http.Dir(".")))
	mux.HandleFunc("/example.wasm", wasmHandler)
	log.Fatal(http.ListenAndServe(":3000", mux))
}

CurtGreen · June 29, 2018, 12:24am

I’m thinking that you need to install those dependencies by either using go get on them for the image, or by running go install independently on the files you are moving over. Because that stuff gets added to go/bin if I’m not mistaken, and that is actually what the compiler uses when importing packages.

carlca · June 29, 2018, 12:26am

Thanks again Curt! To be honest, it’s getting late for me, so I’ll look at trying out what you suggest in the morning!

CurtGreen · June 29, 2018, 2:39am

Further looking revealed this discusssion where benweissmann shows how you can use the go dep tool to accomplish this: https://github.com/golang/dep/issues/1374

carlca · June 29, 2018, 8:04am

I think that I would rather avoid go dep if at all possible. The way things are going, I would prefer to use vgo in some way, if I have to use a dependency tool at all.

Also, on reflection, I don’t understand why I was getting those dependency errors at all. The server.go code does not reference the wasm stuff at a source level at all. Why would running RUN go install github.com/carlca/server involve those dependencies? Surely all we’re doing here is to get server.go to serve a pre-compiled entity, namely example.wasm…

CurtGreen · June 29, 2018, 8:29am

I didn’t even catch that myself. Makes this extra weird doesn’t it?
Wait the error says its in the markdown.go file in that server package which according to the line numbers isn’t the server code.

carlca · June 29, 2018, 9:17am

It’s very strange. As far as I can see from the initial Dockerfile, there’s no reason why the Docker Go compiler should even be attempting to build markdown.go at all…

FROM golang:alpine
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html
ADD . /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm
ADD . /go/src/github.com/carlca/server
RUN go install github.com/carlca/server
CMD ["/go/bin/server"]
EXPOSE 3000

NobbZ · June 29, 2018, 9:31am

Your dockerfile feels wrong.

You are creating 3 directories /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/index.html, /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/example.wasm and /go/src/github.com/carlca/server, all containing everything from your docker context.

I am sure this is not what you want.

In an earlier iteration you even pulled your code into the places where go expected to find referenced dependencies.

A more proper Dockerfile is probably this one:

FROM golang:alpine AS build
ADD wasm/first/wasmplay/markdownvecty/ /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/
ADD server /go/src/github.com/carlca/server

RUN go get github.com/carlca/server && go install github.com/carlca/server

FROM alpine:latest
COPY --from=build /go/bin/server /go/bin/server

EXPOSE 3000

CMD ["/go/bin/server"]

This will result in a much smaller image.

It does expect though, that you have your docker context rooted at $GOPATH/src/github.com/carlca, which probably again is not what you want, but your only option as you have currently structured your app into two separate repositories.

carlca · June 29, 2018, 9:45am

It comes as no surprise that my Dockerfile feels wrong! There are certainly a few concepts relating to Dockerfiles that I am missing at the moment. One of which is: Which `$GOPATH setting does Docker use, my local $GOPATH or something else?

You mention “that you have your docker context rooted at $GOPATH/src/github.com/carlca”. How would I change that if I wanted? Especially as my local $GOPATH is /Users/carlca/Code/go…

carlca · June 29, 2018, 9:47am

Another question. I’ve tried looking at the official Docker documentation but did get on too well with it. Are there any other places I can find out quickly and easily how Dockerfiles work?

NobbZ · June 29, 2018, 9:47am

The container runs in isolation, thats what docker is about, so it uses its own $GOPATH.

The default golang:* containers have ENV GOPATH=/go in their defining Dockerfile.

The “Docker workspace” is the folder you specify when doing a docker build <path>.

NobbZ · June 29, 2018, 9:49am

There are a plentora of blog posts in the wild of varying level and quality.

Personally I learnt most stuff from the Dockerfile reference and Best practices for writing Dockerfiles.

carlca · June 29, 2018, 9:49am

I’ll be back later. I’ll try out your amended Dockerfile and see how I get on. Thanks a lot for your help so far!

carlca · June 29, 2018, 9:50am

Cool! I’ll check them out. Cheers!

carlca · July 2, 2018, 8:20pm

Hello again, I tried your changes but they failed with the following message…

┌─( ~/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty )────────────────────────────────────────────────────────────────────( @Carls-Mac-Pro )─┐
└─❱❱❱ now                                                                                                                                        +919 15:47 ❰─┘
> Deploying ~/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty under carl.caulkett@gmail.com
> Your deployment's code and logs will be publicly accessible because you are subscribed to the OSS plan.

> NOTE: You can use `now --public` or upgrade your plan (https://zeit.co/account/plan) to skip this prompt
> Synced 1 file (940B) [840ms]
> https://markdownvecty-dygwgkhowq.now.sh [in clipboard] (bru1) [4s]
> Building…
> ▲ docker build
Sending build context to Docker daemon 3.892 MBkB
> Step 1/8 : FROM golang:alpine AS build
>  ---> 22e024490b47
> Step 2/8 : ADD wasm/first/wasmplay/markdownvecty/ /Users/carlca/code/go/src/github.com/carlca/wasm/first/wasmplay/markdownvecty/
> ADD failed: stat /var/lib/docker/tmp/docker-builder086580357/wasm/first/wasmplay/markdownvecty: no such file or directory
> Error! Build failed

You mention that I currently have my app structured into 2 separate repositories. I am by no means bound to such an approach. Any suggestions as to how to improve this would be more than welcome!

NobbZ · July 2, 2018, 8:40pm

I do not know anything about your now command. So before doing anything else, make sure a docker build runs properly on your system, then you can debug the actual deploy.

And as I said earlier, your docker context needs to be at ~/code/go/src/github.com/carlca, not somewhere deep down below. But as I do not know now, I can’t tell for sure if thats the issue.

Therefore, try building your image from exactly that path. Make sure to have the Dockerfile at the correct location or to use -f switch to specify its location.

But to be honest, I’m not even sure if the boundary is well choosen. Do you really want to split the application over 2 git repositories?

Also I stronlgy suggest doing a proper docker tutorial/walkthrough that doesn’t touch any go specifics (best it doesn’t do anything specific, except for a bit package management and echoing some static textfile or such).

Then do the go stuff without docker involved. If you have the hardware to spare, try even deploying the binary to a second computer and play around with it there.

After both of those worked, then you can try to combine them and deploy to a docker container. And if that worked locally, you can try it with the specifics added on top of the process by your provider.

carlca · July 2, 2018, 8:42pm

Sounds like good advice, thanks very much! I’m deep into some Logic Pro technical stuff at the moment so I’ll study your words tomorrow.

system · September 30, 2018, 8:42pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.