How do I set up global load balancing using Digital Ocean DNS and Nginx?

Question

UPDATE: See the answer I've provided below for the solution I eventually got set up on AWS.

I'm currently experimenting with methods to implement a global load-balancing layer for my app servers on Digital Ocean and there's a few pieces I've yet to put together.

The Goal

Offer highly-available service to my users by routing all connections to the closest 'cluster' of servers in SFO, NYC, LON, and eventually Singapore.

Additionally, I would eventually like to automate the maintenance of this by writing a daemon that can monitor, scale, and heal any of the servers on the system. Or I'll combine various services to achieve the same automation goals. First I need to figure out how to do it manually.

The Stack

1. Ubuntu 14.04
2. Nginx 1.4.6
3. node.js
4. MongoDB from Compose.io (formerly MongoHQ)

Global Domain Breakdown

Once I rig everything up, my domain would look something like this:

**GLOBAL**
global-balancing-1.myapp.com
global-balancing-2.myapp.com
global-balancing-3.myapp.com

**NYC**
nyc-load-balancing-1.myapp.com
nyc-load-balancing-2.myapp.com
nyc-load-balancing-3.myapp.com

nyc-app-1.myapp.com
nyc-app-2.myapp.com
nyc-app-3.myapp.com

nyc-api-1.myapp.com
nyc-api-2.myapp.com
nyc-api-3.myapp.com

**SFO**
sfo-load-balancing-1.myapp.com
sfo-load-balancing-2.myapp.com
sfo-load-balancing-3.myapp.com

sfo-app-1.myapp.com
sfo-app-2.myapp.com
sfo-app-3.myapp.com

sfo-api-1.myapp.com
sfo-api-2.myapp.com
sfo-api-3.myapp.com

**LON**
lon-load-balancing-1.myapp.com
lon-load-balancing-2.myapp.com
lon-load-balancing-3.myapp.com

lon-app-1.myapp.com
lon-app-2.myapp.com
lon-app-3.myapp.com

lon-api-1.myapp.com
lon-api-2.myapp.com
lon-api-3.myapp.com

And then if there's any strain on any given layer, in any given region, I can just spin up a new droplet to help out: nyc-app-4.myapp.com, lon-load-balancing-5.myapp.com, etc…

Current Working Methodology

• A (minimum) trio of global-balancing servers receive all traffic. These servers are "DNS Round-Robin" balanced as illustrated in this (frankly confusing) article: How To Configure DNS Round-Robin Load Balancing.

• Using the Nginx GeoIP Module and MaxMind GeoIP Data the origin of any given request is determined down to the $geoip_city_continent_code.

• The global-balancing layer then routes the request to the least connected server on the load-balancing layer of the appropriate cluster: nyc-load-balancing-1, sfo-load-balancing-3, lon-load-balancing-2, etc.. This layer is also a (minimum) trio of droplets.

• The regional load-balancing layer then routes the request to the least connected server in the app or api layer: nyc-app-2, sfo-api-1, lon-api-3, etc…

The details of the Nginx kung fu can be found in this tutorial: Villiage Idiot: Setting up Nginx with GSLB/Reverse Proxy on AWS. More general info about Nginx load-balancing is available here and here.

Questions

Where do I put the global-balancing servers?

It strikes me as odd that I would put them either all in one place, or spread that layer out around the globe either. Say, for instance, I put them all in NYC. Then someone from France hits my domain. The request would go from France, to NYC, and then be routed back to LON. Or if I put one of each in SFO, NYC, and LON then isn't it still possible that a user from Toronto (Parkdale, represent) could send a request that ends up going to LON only to be routed back to NYC?

Do subsequent requests get routed to the same IP?

As in, if a user from Toronto sends a request that the global-balancing layer determines should be going to NYC, does the next request from that origin go directly to NYC, or is it still luck of the draw that it will hit the nearest global-balancing server (NYC in this case).

What about sessions?

I've configured Nginx to use the ip_hash; directive so it will direct the user to the same app or api endpoint (a node process, in my case) but how will global balancing affect this, if at all?

Any DNS Examples?

I'm not exactly a DNS expert (I'm currently trying to figure out why my CNAME records aren't resolving) but I'm a quick study when provided with a solid example. Has anyone gone through this process before and can provide a sample of what the DNS records look like for a successful setup?

What about SSL/TLS?

Would I need a certificate for every server, or just for the three global-balancing servers since that's the only public-facing gateway?

If you read this whole thing then reward yourself with a cupcake. Thanks in advance for any help.

Answer 1

The Goal: Offer highly-available service to my users by routing all connections to the closest 'cluster' of servers in SFO, NYC, LON, and eventually Singapore.

The global-balancing layer then routes the request to theleast connected server...

If I'm reading your configuration correctly, you're actually proxying from your global balancers to the balancers at each region. This does not meet your goal of routing users to the nearest region.

There are three ways that I know of to get what you're looking for:

1. 30x Redirect
   Your global balancers receive the HTTP request and then redirect it to a server group in or near the region it thinks the request is coming from, based on IP address. This sounds like what you were trying to set up. This method has side effects for some applications, and also increases the time it takes for a user to get data since you're adding a ton of overhead. This only makes sense if the resources you're redirecting to are very large, and the local regional cluster will be able to serve much more efficiently.

2. Anycast (taking advantage of BGP routing)
   This is what the big players like Akamai use for their CDN. Basically, there are multiple servers out on the internet with the exact same routable IP address. Suppose I have servers in several regions, and they have the IP address of 192.0.2.1. If I'm in the US and try to connect to 192.0.2.1, and someone is in Europe that tries to connect to 192.0.2.1, it's likely that we'll be routed to the nearest server. This uses the internet's own routing to find the best path (based on network conditions) for the traffic. Unfortunately, you can't just use this method. You need your own AS number, and physical hardware. If you find a VPS provider that lets you have a chunk of their Anycast block, let me know!

3. Geo-DNS
   There are some DNS providers that provide a service often marketed as "Geo-DNS". They have a bunch of DNS servers hosted on anycast addresses which can route traffic to your nearest servers. If a client queries a European DNS server, it should return the address for your European region servers, vs. some in other regions. There are many variations on the Geo DNS services. Others simply maintain a geo-IP database and return the server for the region they think is closer, just like the redirect method but for DNS before the HTTP request is ever made. This is usually the good option, for price and ease of use.

Do subsequent requests get routed to the same IP?

Many load balancers have a "stickiness" option that says requests from the same network address should be routed to the same end server (provided that end server is still up and running).

What about sessions?

This is exactly why you would want that stickiness. When it comes to session data, you are going to have to find a way to keep all your servers up-to-date. Realistically, this isn't always guaranteed. How you handle it depends on your application. Can you keep a Redis instance or whatever out there for all your servers to reliably hit from around the world? Do you really need that session data in every region? Or can you have your main application servers dealing with session data in one location?

Any DNS Examples?

Post separate questions for these. Everyone's "successful setup" looks differently.

What about SSL/TLS?

If you're proxying data, only your global balancers need to handle HTTPS. If you're redirecting, then all the servers need to handle it.