What is Google's Dremel? How is it different from Mapreduce?

Yktula picture Yktula · Jul 7, 2011 · Viewed 34.7k times · Source

Google's Dremel is described here. What's the difference between Dremel and Mapreduce?

Answer

Michael Manoochehri picture Michael Manoochehri · Jan 27, 2012

Dremel and MapReduce are not directly comparable, but rather they are complementary technologies.

MapReduce is not specifically designed for analyzing data - rather it's a software framework that allows a collection of nodes to tackle distributed computational problems for large datasets.

Dremel is a data analysis tool designed to quickly run queries on massive, structured datasets (such as log or event files). It supports a SQL-like syntax, but apart from table appends, it is read-only. It doesn't support update or create functions, nor does it feature table indexes. Data is organized in a "columnar" format, which contributes to very fast query speed. Google's BigQuery product is an implementation of Dremel accessible via RESTful API.

Hadoop (an open source implementation of MapReduce) in conjunction with the "Hive" data warehouse software, also allows data analysis for massive datasets using a SQL-style syntax. Hive essentially turns queries into MapReduce functions. In contrast to using a ColumIO format, Hive attempts to make queries quick by using techniques such as table indexing.