Hadoop and MySQL Integration

mysql integration hadoop
Jonar picture Jonar · Jan 26, 2011 · Viewed 11.5k times · Source

We would like to implement Hadoop on our system to improve its performance.

The process works like this: Hadoop will gather data from MySQL database then process it. The output will then be exported back to MySQL database.

Is this a good implementation? Will this improve our system's overall performance? What are the requirements and has this been done before? A good tutorial would really help.

Thanks

Answer

Joe Stein picture Joe Stein · Jan 26, 2011

Sqoop is a tool designed to import data from relational databases into Hadoop

https://github.com/cloudera/sqoop/wiki/

and a video about it http://www.cloudera.com/blog/2009/12/hadoop-world-sqoop-database-import-for-hadoop/

Related questions

How do I import an SQL file using the command line in MySQL?

I have a .sql file with an export from phpMyAdmin. I want to import it into a different server using the command line. I have a Windows Server 2008 R2 installation. I placed the .sql file on the C drive, and …

Read More
How can I prevent SQL injection in PHP?

If user input is inserted without modification into an SQL query, then the application becomes vulnerable to SQL injection, like in the following example: $unsafe_variable = $_POST['user_input']; mysql_query("INSERT INTO `table` (`column`) VALUES ('$unsafe_variable')"); That's …

Read More
MySQL error code: 1175 during UPDATE in MySQL Workbench

I'm trying to update the column visited to give it the value 1. I use MySQL workbench, and I'm writing the statement in the SQL editor from inside the workbench. I'm writing the following command: UPDATE tablename SET columnname=1; It gives …

Read More