Caddy 0.7.6 released

Releases
#1

Caddy 0.7.6 was released today, thanks to the work of literally a dozen contributors from all over the world.

Caddy is a general-purpose, configurable HTTP/2 web server. It’s very easy to use and I hope you’ll give it a try when you have a chance.

Also, there’s three new add-ons: cms, jsonp, and search. The cms directive is a web-based admin panel for your Hugo site, but it can also power other static site generators. The jsonp directive wraps JSON responses in JSONP, and search activates a self-contained site search engine that you can customize, opening a HTML and JSON search endpoint at /search. Very cool stuff.

Next, we’ll focus more on refining what we already have, fixing bugs, and general work on the underlying code base to prepare it for a stable 1.0.

11 Likes
#2

Matt, I’m getting an error when viewing the site over https (http works fine). Error in Google Chrome:

This webpage is not available

ERR_CONNECTION_CLOSED

The webpage at https://caddyserver.com/blog/caddy-0_7_6-released might be temporarily down or it may have moved permanently to a new web address.

I noticed this a while back too, but never got around to creating an issue on github.

EDIT: https is working fine in Firefox and IE. Not sure what the problem could be with Chrome.

#3

Thanks for letting me know.

I’m glad you’re getting that error, too, since only I have ever seen it and only from one network: my university campus. No matter the client (phone or computer, Chrome/FF/Safari, etc.) I still see that error from campus. If I’m not using that network, I don’t get that error. But fortunately, on campus I can load it over plain HTTP; try that for now.

Any ideas on how to debug this? This one has puzzled me for some time…

#4

Hmm… I’m on my home network. I’ll try on my school network tomorrow and see if it works.

Not even sure where I would start debugging this. I tried using Google’s developer tools, but there’s nothing in there that seemed useful. I guess Wireshark is the next best thing. If anyone else has any suggestions I’m open to them!

#5

Do you use a proxy of any sort at home? Or special router software or any monitoring software or anything of the sort? (Sorry to pry, just trying to figure out if this is a proxy-related issue.)

#6

No proxy software and no monitoring software outside of antivirus, but I would think that would stop Firefox and IE from working too which isn’t the case. I tried out https://caddyserver.com my Mac using Chrome and it works fine. So my only way to reproduce this is with Google Chrome on my Windows 8 machine. Everything else works as normal.

#7

This is bugging me so I did a little digging. By default, Caddy enables these 10 cipher suites (they come with the Go std lib):

  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  • TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  • TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA
  • TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  • TLS_RSA_WITH_3DES_EDE_CBC_SHA

But over HTTPS, Caddy serves HTTP/2 by default. HTTP/2 blacklists almost 300 cipher suites for security reasons. 8 of the above cipher suites are on the list:

  • TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  • TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  • TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA
  • TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
  • TLS_RSA_WITH_3DES_EDE_CBC_SHA

So only two suites remain, except for the bold one which somehow slips through according to SSL Labs:

  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b) ECDH 256 bits (eq. 3072 bits RSA) FS 128
  • TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a) ECDH 256 bits (eq. 3072 bits RSA) FS 256
  • TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009) ECDH 256 bits (eq. 3072 bits RSA) FS 128

(I do not think that third one showing up here is cause for alarm, although I’m not sure why it’s there at this point.)

I strongly suspect that if a connection fails to the Caddy site over HTTPS, it is because some relevant software only supports insecure cipher suites. This is unfortunate, but am not sure how to fix right now without compromising the security and stability of HTTP/2 or your websites.

#8

If it wasn’t HTTP/2 you could just capture the traffic with Wireshark and compare. We need better HTTP/2 tooling…

#9

True, http2 tooling is still scant, but do you think this error occurs at the TLS layer or the HTTP layer? If it occurs at the transport layer then maybe Wireshark could still be useful.

#10

You’re right I do think it’s the transport layer and If you compare 2 captures (from a location that works, and a location that doesn’t) and everything looks normal then you can at-least eliminate the whole layer.

#11

@alexmullins @netixen I’ve confirmed with Wireshark that my client is in fact sending 21 cipher suites to the server in the Client Hello:

client_hello_campus.png1316×1000 176 KB

The server responds with a FIN packet and outputs this to the log:

2015/10/12 18:30:51 http: TLS handshake error from 128.187.97.18:39703: tls: no cipher suite supported by both client and server
2015/10/12 18:30:51 http: TLS handshake error from 128.187.97.18:39713: tls: no cipher suite supported by both client and server
2015/10/12 18:30:51 http: TLS handshake error from 128.187.97.18:39728: tls: no cipher suite supported by both client and server
2015/10/12 18:30:51 http: TLS handshake error from 128.187.97.18:39746: tls: no cipher suite supported by both client and server

This reeks of software meddling in between. The client definitely accepts the cipher suites the server requires (at least in my case).

#12

Ok, I’ll check Wireshark too and see what the results are on my end. @matt , this problem happens to you with any client while on your school network, correct? On my end I could only replicate this problem on my Windows 8.1 desktop running Google Chrome’s latest release (45.0.2454.101m); both Firefox and IE work. From your SS above I see that your client sent TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b) and Caddy (Go stdlib) has that as one of its supported ciphers, so you’d expect the TLS connection to succeed.

#13

When I connected using Chrome it sent 14 cipher suites and had TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b) as the top one, so it shouldn’t have had a problem connecting using that cipher suite.

The protocol progression was:

caddy2.PNG1177×500 25.6 KB

TLSv1.2 - Client Hello
TLSv1.2 - Server Hello
TLSv1.2 - Certificate
TLSv1.2 - Server Key Exchange
TLSv1.2 - Server Hello Done
TLSv1.2 - Alert (Level: Fatal, Description: Decode Error)
TLSv1.2 - Alert (Level: Fatal,Description:Decode Error) Client Hello, Alert (Level: Fatal, Description: Unexpected Message)
TLSv1.1 - Client Hello
TLSv1.1 - Encrypted Alert

You can see my client is responding with fatal alerts (Decode Error and Unexpected Message). Also, it looks like Chrome is trying to reconnect using TLSv1.1, but that will not work with http2 as TLS1.2+ is required.

EDIT: It’s weird though, b/c for me this ONLY happens using Chrome. I’m starting to think that we are experiencing two separate issues. I could be wrong though.

1 Like
#14

Wow, thanks for digging into this! That is a strange sequence of packets. Why would it have a decode error and downgrade to TLS 1.1? I’m using the exact same version of Chrome right now and it’s working fine. But I’m on OS X 10.10…

I’m beginning to believe with you that these are separate issues. I’m ~95% certain that there’s a proxy on my school’s campus running old software that is incompatible with the modern requirements. But I can’t debug it without access to it (right?). In your case it’s just your Chrome browser that has issues. Very odd.

I wonder if it’s an obscure bug specific to Chrome on Windows 8.1, maybe something with an underlying net library or a C binding that Chrome on Windows is using??

When I use the same version of Chrome on my Mac, I get these results in Wireshark – the highlighted line is where it appears yours encountered a decoding error:

Screen Shot 2015-10-12 at 10.30.21 PM.png1634×694 149 KB

So if these two experiments are analogous, we could assume it failed at the Client Key Exchange et. al. where it does most of the handshake work.

1 Like
#15

The proxy is a blackbox but you can still reverse engineer what it’s doing by inspecting the front and back of the box :-P.

Try inspecting wireshark data from your computer and from the server, for the same problematic connection attempt. Look for suspicious differences.

#16

Yea I have the same version on my Mac too and it works fine. I’ll keep looking into it later this week and report back. Gotta study for midterms now :cry:

#17

Sounds good – I’m in the same boat :slight_smile: Also, Analytics tells me that ~12% of Windows visitors to the site are on 8.1, and 63% of Windows visitors are on Chrome. I just had a couple people on Twitter with Win8.1 verify that they were able to access it as well on Chrome. So… I’m stumped for now. :person_frowning:

@nictuku Good ideas, that’ll take a lot of time. But it may come to that.

#18

Ahaahh! I have figured out what was going wrong on my end. After a few more Google searches and looking on the Chrome bug tracker I narrowed the problem down to my antivirus software (ofc!). I use Avast antivirus on my Windows 8.1 machine and it has a web shield that basically man-in-the-middles any https connection to check if the content contains malware (you can read more: http://security.stackexchange.com/a/73481). So I disabled that feature, connected to https://caddyserver.com and it worked! And thinking back, I ran into another problem with Avast preventing any Go programs I compiled from running. I had to turn off a ‘feature’ called DeepScan which scans unknown running files and stops anything ‘suspicious’. Now, I’d like to know why Avast is preventing the https connection to caddyserver.com and not the http one… More rabbit holes to chase!

2 Likes
[SOLVED] Slow `go test` with Avast on Windows 8.1
#19

Great find! That makes sense. I wonder if Avast suffers from similar neglect as my university’s campus proxy.

Also, I just saw this today: Chrome Canary has a HTTPS debugging tool. I’m excited to try this. (Edit: Just tried it. No help whatsoever. Oh well.)

Thanks for your help!

1 Like
#20

Or… stop using antivirus. They are a scam, essentially.

1 Like