How to run Hadoop on a Mesos cluster?
I am trying to set up a Apache Mesos Cluster and run Hadoop-Job on it. The documentation here is not at my level, so that i am not able to understand and maybe someone here can explain me:
First should I set up a working Hadoop Cluster? Or first set up a Mesos cluster? where do I point the slaves? in Hadoop-slaves file or registered Mesos slaves should only be used?
Answer
The goal of Mesos is to run an abstraction for your cluster, where Hadoop would just be 1 service among others. In order for this to work, you need to first setup your Mesos cluster as the primary component, and then you can start adding services like Hadoop to this cluster using the Mesos abstraction.
There is an excellent tutorial from Mesosphere that you should take a look at, and it explains in details how to run Hadoop on top of Mesos, so this would be a good place to start.
Alternatively, this company recently started a serviced for Elastic Mesos, similar in nature to Amazon's Elastic MapReduce. So if you want to get started quickly with Hadoop on Mesos without having to go through the pain of configuring everything, this is a good place to start.