CloudFront error when serving over HTTPS using SNI
Amazon recently rolled out a new feature on CloudFront that supports custom SSL certificates at no charge using SNI (Server Name Indication).
I got my distribution set up with a free Class 1 certificate from StartSSL and everything was working when I was noticing that the site would go down a short time after it's deployed. Running SSL Checker returns that my certificate is working properly:
But then I would hit this error page when trying to access the site via HTTPS (it would work for the first request then go down in subsequent attempts to connect).
Here's a verbose output when accessing with ssl (succeeds on index):
$ curl -I -v -ssl https://wikichen.is
* Adding handle: conn: 0x7f9f82804000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f9f82804000) send_pipe: 1, recv_pipe: 0
* About to connect() to wikichen.is port 443 (#0)
* Trying 54.230.141.222...
* Connected to wikichen.is (54.230.141.222) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_RC4_128_MD5
* Server certificate: www.wikichen.is (6w984WNu7vM5OrdU)
* Server certificate: StartCom Class 1 Primary Intermediate Server CA
* Server certificate: StartCom Certification Authority
> HEAD / HTTP/1.1
> User-Agent: curl/7.30.0
> Host: wikichen.is
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Content-Length: 1153
Content-Length: 1153
< Connection: keep-alive
Connection: keep-alive
< Date: Sun, 09 Mar 2014 16:09:54 GMT
Date: Sun, 09 Mar 2014 16:09:54 GMT
< Cache-Control: max-age=120
Cache-Control: max-age=120
< Content-Encoding: gzip
Content-Encoding: gzip
< Last-Modified: Wed, 05 Mar 2014 20:40:48 GMT
Last-Modified: Wed, 05 Mar 2014 20:40:48 GMT
< ETag: "34685bc45353d1030d3a515ddba78f3e"
ETag: "34685bc45353d1030d3a515ddba78f3e"
* Server AmazonS3 is not blacklisted
< Server: AmazonS3
Server: AmazonS3
< Age: 4244
Age: 4244
< X-Cache: Hit from cloudfront
X-Cache: Hit from cloudfront
< Via: 1.1 4f672256eaca5524999342dc8678cdd2.cloudfront.net (CloudFront)
Via: 1.1 4f672256eaca5524999342dc8678cdd2.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: h4TEULH44TCi7m2lL42A8lO-5-Gmx8iY2M2C1AOmRlK543zFN6jCtQ==
X-Amz-Cf-Id: h4TEULH44TCi7m2lL42A8lO-5-Gmx8iY2M2C1AOmRlK543zFN6jCtQ==
<
* Connection #0 to host wikichen.is left intact
Then fails on other pages:
$ curl -i -v https://wikichen.is/writing/index.html
* Adding handle: conn: 0x7fa153804000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fa153804000) send_pipe: 1, recv_pipe: 0
* About to connect() to wikichen.is port 443 (#0)
* Trying 54.230.140.160...
* Connected to wikichen.is (54.230.140.160) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_RC4_128_MD5
* Server certificate: www.wikichen.is (6w984WNu7vM5OrdU)
* Server certificate: StartCom Class 1 Primary Intermediate Server CA
* Server certificate: StartCom Certification Authority
> GET /writing/index.html HTTP/1.1
> User-Agent: curl/7.30.0
> Host: wikichen.is
> Accept: */*
>
< HTTP/1.1 502 Bad Gateway
HTTP/1.1 502 Bad Gateway
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 472
Content-Length: 472
< Connection: keep-alive
Connection: keep-alive
* Server CloudFront is not blacklisted
< Server: CloudFront
Server: CloudFront
< Date: Sun, 09 Mar 2014 17:54:41 GMT
Date: Sun, 09 Mar 2014 17:54:41 GMT
< Age: 6
Age: 6
< X-Cache: Error from cloudfront
X-Cache: Error from cloudfront
< Via: 1.1 9096435f28f91f92bacdf76122de09ee.cloudfront.net (CloudFront)
Via: 1.1 9096435f28f91f92bacdf76122de09ee.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: iAUOQbP8O4A0pI9KGvVz0VgBT1TW-j0yVDa7vdSvIAuxnKOyQghtnw==
X-Amz-Cf-Id: iAUOQbP8O4A0pI9KGvVz0VgBT1TW-j0yVDa7vdSvIAuxnKOyQghtnw==
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD><BODY>
<H1>ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
</BODY></HTML>
<BR clear="all">
<HR noshade size="1px">
<ADDRESS>
Generated by cloudfront (CloudFront)
</ADDRESS>
* Connection #0 to host wikichen.is left intact
</BODY></HTML>%
Would love some pointers as to where to start troubleshooting.
Answer
A kind rep by the name of Alastair@AWS from the AWS CloudFront forums solved this for me:
I have identified your CloudFront distribution and the S3 bucket acting as the origin for this distribution.
I can re-create and explain the intermittent '502 Bad Gateway' response you are receiving.
This response is returned by CloudFront when you attempt to access a URL using the HTTPS protocol that is not currently cached by CloudFront. The reason for this error is CloudFront is attempting to contact your origin using the HTTPS protocol, and this is failing.
The reason for this failure is you have configured your origin as an S3 bucket, but you are using the "Custom Origin" type and directing to the S3 website URL for this bucket. If you attempt to hit your S3 website URL using HTTPS, you will note this does not work. S3 website hosting only supports serving content using the HTTP protocol (http://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteEndpoints.html#WebsiteRestEndpointDiff).
Now, the intermittent page load behavior you are seeing is due to CloudFront returning the pages it currently has in its cache. You should be able to re-create this scenario as follows:
- Hit a page on your site using HTTPS. You should get a '502 Bad Gateway' error back.
- Hit the same page using HTTP. You should see the page.
- Hit the page again using HTTPS. You should now get the expected result, as CF has served the content from its cache rather than attempting to contact your origin.
To resolve this issue, please try the following:
- Open the CloudFront Management Console and open your distribution.
- Navigate to the Origins tab, select your origin and click "Edit"
- Modify the "Origin Protocol Policy" to "HTTP Only".
- Save the changes and wait about 15 minutes for the change to take effect.
- Test
My expectation is this will force CloudFront to contact your origin using HTTP only. I have tested this in my environment with an S3 Website hosted bucket and I can successfully load content via both HTTP and HTTPS.