sqoop and password encryption using password-file option

sqoop
Roman Cwalina picture Roman Cwalina · May 28, 2014 · Viewed 7k times · Source

I'm using sqoop-1.4.3-cdh4.6.0.jar and i'm wondering if the --password-file option is avialable in that version. If yes, can someone give me an example of how the encryption process would be invoked? provide a command example , i can see that the patch has been introduced with cdh4.6.0 but not sure if sqoop 1.4.3 can use it.

Answer

abeaamase picture abeaamase · Mar 17, 2015

Chances are it's not in CDH 4.6.0 given https://issues.apache.org/jira/browse/SQOOP-1223. I'd upgrade to Sqoop 1.4.5 to get this functionality.

Here's a quick Sqoop command example:

sqoop import \
    -Dorg.apache.sqoop.credentials.loader.class=org.apache.sqoop.util.password.CryptoFileLoader \
    -Dorg.apache.sqoop.credentials.loader.crypto.passphrase=sqoop2 \
    --connect jdbc:mysql://example.com/sqoop \
    --username sqoop \
    --password-file file:///tmp/pass.enc \
    --table tbl

The important parameters to note are:

  1. org.apache.sqoop.credentials.loader.class - Credentials loader
  2. org.apache.sqoop.credentials.loader.crypto.alg – The Algorithm used to decrypt the file (default is AES/ECB/PKCS5Padding).
  3. org.apache.sqoop.credentials.loader.crypto.salt – The salt used to derive a key with the passphrase (default is SALT).
  4. org.apache.sqoop.credentials.loader.crypto.iterations – Number of PBKDF2 iterations (default is 10000).
  5. org.apache.sqoop.credentials.loader.crypto.salt.key.len – Derived key length (default is 128).
  6. org.apache.sqoop.credentials.loader.crypto.passphrase Passphrase used to derive key.

There are several other options available to you as well more.

Related questions

How to copy data from one HDFS to another HDFS?

I have two HDFS setup and want to copy (not migrate or move) some tables from HDFS1 to HDFS2. How to copy data from one HDFS to another HDFS? Is it possible via Sqoop or other command line?

Read More
Using Sqoop to import data from MySQL to Hive

I am using Sqoop (version 1.4.4) to import data from MySQL to Hive. The data will be a subset of one of tables, i.e. few columns from a table. Is it necessary to create table in Hive before hand. Or …

Read More
what are the following commands in sqoop?

Can anyone tell me what is the use of --split-by and boundary query in sqoop? sqoop import --connect jdbc:mysql://localhost/my --username user --password 1234 --query 'select * from table where id=5 AND $CONDITIONS' --split-by table.id --target-dir /dir

Read More