Hive connector to HDP 2.6 not working

KNIME Extensions
,
1

I am using KNIME 3.4.1 and trying to connect it to HDP 2.6.1 using this KNIME blog post.

The kerberos ticket is present as shown in the attached screenshot, also, the krb5.conf file is present under KNIME jre

[libdefaults]
  renew_lifetime = 7d
  forwardable = true
  default_realm = GLOBAL.SCD.COM
  ticket_lifetime = 10h
  dns_lookup_realm = false
  dns_lookup_kdc = true
  default_ccache_name = /tmp/krb5cc_%{uid}
  allow_weak_crypto = yes

[logging]
  default = FILE:/var/log/krb5kdc.log
  admin_server = FILE:/var/log/kadmind.log
  kdc = FILE:/var/log/krb5kdc.log

[realms]
  GLOBAL.SCD.COM = {
    default_domain = sss.se.com
  }

  SE = {
    default_domain = sss.se.com
  }

I do have a valid kerberos ticket as shown below :

E:\Omkar\Development\Software\Analysis\KNIME>klist

Credentials cache: C:\Users\ojoqcu\krb5cc_ojoqcu

Default principal: ojoqcu@GLOBAL.SCD.COM, 1 entry found.

[1]  Service Principal:  krbtgt/GLOBAL.SCD.COM@GLOBAL.SCD.COM
     Valid starting:     Oct 06,  2017 10:49:39
     Expires:            Oct 06,  2017 20:49:39

E:\Omkar\Development\Software\Analysis\KNIME>knime.exe

E:\Omkar\Development\Software\Analysis\KNIME>

I have added the Hortonworks Hive JDBC jar, yet, the driver doesn't show up in the config.

The attached screenshot shows the config. of the Hive connector, upon execution, I get the following error :

ERROR Hive Connector       0:1        Exception creating Kerberos based jdbc connection. Error: null
ERROR Hive Connector       0:1        Execute failed: Could not create connection to database: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
2

Added latest connector config.

3

Hi Kayluga,

sorry for the trouble this is giving you. Setting up Kerberos authentication can sometimes be tricky.

 

The error message indicates the JDBC driver was not able to find your TGT.

Your krb5.conf on a Windows system should not contain the following line:

default_ccache_name = /tmp/krb5cc_%{uid}

 

I have added the Hortonworks Hive JDBC jar, yet, the driver doesn't show up in the config.

From the screenshots I can see that you are using transportMode=http and Kerberos authentication. There is currently an open issue with using transportMode=http and Kerberos authentication that prevents KNIME AP from connecting.

Usually we recommend all users to use the proprietary (Simba-based) Hive JDBC drivers from the Hortonworks website. But this driver does not support your specific combination of settings, because it can only do user/password authentication with transportMode=http.

The built-in Hive JDBC driver that comes with KNIME AP has a bug that also prevents your specific combination of settings from working. While technically the driver you have loaded (hive standalone JDBC v2.1), does not have that bug anymore, it is currently not possible to use these drivers from KNIME in that way.

Long story short: An update to the built-in Hive JDBC drivers is high up on our list of priorities. Once this is done, you can use one of our nightly builds. We are very sorry for the inconvenience.

The only suggestion I can currently make to you, is to either change the Hive transportMode to binary (Hiveserver2 setting) or to configure Hiveserver2 for user/password authentication.

 

Best regards,

Björn

4

Hi Kayluga,

the nightly KNIME store build now contains an updated Hive JDBC driver that will work with transportMode=http and Kerberos authentication:

https://www.knime.com/form/nightly-build

(you may have to add the KNIME store nightly build update site, the URL is on the nightly build update site)

 

The updated JDBC driver will be officially released beginning of December as part of KNIME AP 3.5.

 

Best,

Björn

 

 

5

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.