Kubernetes vs. CloudFoundry
The next version of CloudFoundry / Diego will offer native support for Docker containers which will be orchestrated across multible hosts [link]. This sounds very similar to Kubernetes.
Of course, the problem Kubernetes is trying to solve is more a generic, where CloudFoundry is more focussed on app development. However, for me it sounds both are heading into a similar direction and CloudFoundry is adding a lot more features on top of the plain orchestration.
So I'm wondering about use-cases where Kubernetes would add more value than CloudFoundry?
Answer
As both a CloudFoundry (past) and Kubernetes (present) commiter, I'm probably uniquely qualified to answer this one.
PaaS-like
I like to call CloudFoundry an "Application PaaS" and Kubernetes a "Container PaaS", but the distinction is fairly subtle and fluid, given that both projects change over time to compete in the same markets.
The distinction between the two is that CF has a staging layer that takes a (12-factor) user app (e.g. jar or gem) and a Heroku-style buildpack (e.g. Java+Tomcat or Ruby) and produces a droplet (analogous to a Docker image). CF doesn't expose the containerization interface to the user, but Kubernetes does.
Audience
CloudFoundry's primary audience is enterprise application devs who want to deploy 12-factor stateless apps using Heroku-style buildpacks.
Kubernetes' audience is a little broader, including both stateless application and stateful service developers who provide their own containers.
This distinction could change in the future:
- CloudFoundry could start to accept docker images (Lattice accepts Docker images).
- Kubernetes could add an image generation layer (OpenShift does something like this).
Feature Comparison
As both projects mature and compete, their similarities and differences will change. So take the following feature comparison with a grain of salt.
Both CF and K8s share many similar features, like containerization, namespacing, authentication,
Kubernetes competitive advantages:
- Group and scale pods of containers that share a networking stack, rather than just scaling independently
- Bring your own container
- Stateful persistance layer
- Larger, more active OSS community
- More extensible architecture with replacable components and 3rd party plugins
- Free web GUI
CloudFoundry competitive advantages:
- Mature authentication, user grouping, and multi-tenancy support [x]
- Bring your own app
- Included load balancer
- Deployed, scaled, and kept alive by BOSH [x]
- Robust logging and metrics aggregation [x]
- Enterprise web GUI [x]
[x] These features are not part of Diego or included in Lattice.
Deployment
One of CloudFoundry's competitive advantages is that it has a mature deployment engine, BOSH, which enables features like scaling, resurrection and monitoring of core CF components. BOSH also supports many IaaS layers with a pluggable cloud provider abstraction. Unfortunately, BOSH's learning curve and deployment configuration management are nightmarish. (As a BOSH committer, I think I can say this with accuracy.)
Kubernetes' deployment abstraction is still in its infancy. Multiple target environments are available in the core repo, but they're not all working, well tested, or supported by the primary developers. This is mostly a maturity thing. One might expect this to improve over time and increase in abstraction. For example, Kubernetes on DCOS allows deploying Kubernetes to an existing DCOS cluster with a single command.
Historical Context
Diego is a rewrite of CF's Droplet Execution Agent. It was originally developed before Kubernetes was announced and has taken on more feature scope as the competitive landscape has evolved. Its original goal was to generate droplets (user application + CF buildpack) and run them in Warden (renamed Garden when rewritten in Go) containers. Since its inception it's also been repackaged as Lattice, which is somewhat of a CloudFoundry-lite (although that name was taken by an existing project). For that reason, Lattice is somewhat toy-like, in that it has deliberately reduced user audience and scope, explicitly missing features that would make it "enterprise-ready". Features that CF already provides. This is partly because Lattice is used to test the core components, without some of the overhead from the more complex CF, but you can also use Lattice in internal high-trust environments where security and multi-tenancy aren't as much of a concern.
It's also worth mentioning that CloudFoundry and Warden (its container engine) predate Docker as well, by a couple years.
Kubernetes on the other hand, is a relatively new project that was developed by Google based on years of container usage with BORG and Omega. Kubernetes could be thought of as 3rd generation container orchestration at Google, the same way Diego is 3rd generation container orchestration at Pivotal/VMware (v1 written at VMware; v2 at VMware with Pivotal Labs help; v3 at Pivotal after it took over the project).