Could not find uri with key dfs.encryption.key.provider.uri to create a keyProvider in HDFS encryption for CDH 5.4

Question

CDH Version: CDH5.4.5

Issue: When HDFS Encryption is enabled using KMS available in Hadoop CDH 5.4 , getting error while putting file into encryption zone.

Steps:

Steps for Encryption of Hadoop as follows:

1. Creating a key [SUCCESS]

   [tester@master ~]$ hadoop key create 'TDEHDP' 
   -provider kms://[email protected]/key_generator/kms -size 128
   tde group has been successfully created with options 
   Options{cipher='AES/CTR/NoPadding', bitLength=128, description='null', attributes=null}.
   KMSClientProvider[https://10.1.118.1/key_generator/kms/v1/] has been updated.
   

2.Creating a directory [SUCCESS]

[tester@master ~]$ hdfs dfs -mkdir /user/tester/vs_key_testdir

3. Adding Encryption Zone [SUCCESS]

   [tester@master ~]$ hdfs crypto -createZone -keyName 'TDEHDP' 
   -path /user/tester/vs_key_testdir
   Added encryption zone /user/tester/vs_key_testdir
   

4. Copying File to encryption Zone [ERROR]

   [tdetester@master ~]$ hdfs dfs -copyFromLocal test.txt /user/tester/vs_key_testdir
   

15/09/04 06:06:33 ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !! copyFromLocal: No KeyProvider is configured, cannot access an encrypted file 15/09/04 06:06:33 ERROR hdfs.DFSClient: Failed to close inode 20823 org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): No lease on /user/tester/vs_key_testdir/test.txt.COPYING (inode 20823): File does not exist. Holder DFSClient_NONMAPREDUCE_1061684229_1 does not have any open files.

Any idea/suggestion will be helpful.

Answer 1

This issue was crossposted here: https://community.cloudera.com/t5/Storage-Random-Access-HDFS/Could-not-find-uri-with-key-dfs-encryption-key-provider-uri-to/td-p/31637

Main conclusion: It is a non-issue

Here is the answer that was provided by the support staff:

CDH's base release versions are just that: base. The fix for the harmless log print due to HDFS-7931 is present in all CDH5 releases since CDH 5.4.1.

If you see that error in context of having configured a KMS, then its a worthy one to consider. If you do not use KMS or EZs, then the error may be ignored. Alternatively upgrade to the latest CDH5 (5.4.x or 5.5.x) releases to receive a bug fix that makes the error only appear when in the context of a KMS being configured over an encrypted path.

Per your log snippet, I don't see a problem (the canary does not appear to be failing?). If you're trying to report a failure, please send us more characteristics of the failure, as HDFS-7931 is a minor issue with an unnecessary log print.