Haproxy in front of varnish or the other way round?

Question

I can imagine two setups:

Load-balance then cache

                          +-- Cache server #1 (varnish) -- App server #1
                         /
Load Balancer (haproxy)-+---- Cache server #2 (varnish) -- App server #2
                         \
                          +-- Cache server #3 (varnish) -- App server #3

Cache then load-balance

                                                       +-- App server #1
                                                      /
Cache Server (varnish) --- Load Balancer (haproxy) --+---- App server #2
                                                      \
                                                       +-- App server #3

The problem with the first setup is that there are multiple caches, which wastes a lot of memory and makes invalidating cache more complicated.

The problem with the second setup is that there might be a performance hit and two single points of failure (varnish and haproxy) instead of just one (haproxy)?

I'm tempted to go with the second setup because both haproxy and varnish are supposed to be fast and stable: what's your opinion?

Answer 1

I built a similar setup a few years back for a busy web application (only I did it with Squid instead of Varnish), and it worked out well.

I would recommend using your first setup (HAProxy -> Varnish) with two modifications:

1. Add a secondary HAProxy server using keepalived and a shared virtual IP
2. Use the balance uri load balancing algorithm to optimize cache hits

Pros:

• Peace of mind with HAProxy (x2) and Varnish (x3) redundancy
• Better hit rate efficiency on Varnish with HAProxy URI load balancing option
• Better performance from the cache servers as they don't need to keep as much in memory
• Invalidating cache is easier since the same URI will go to the same server every time

Cons:

• URI balancing works well, but if a cache server goes down, your backend servers will get hit as the other cache server(s) that pick up the slack from the updated URI balancing hash will need to re-retrieve the cached data. Maybe not a big con, but I did have to keep that in mind for my system.