Examples of connecting to kerberos hive in JDBC

We had a need to authenticate user requests against AD in a kerberos enabled cluster, and allow “local” hive sessions to use only a keytab. Below are the examples of each.

First, we show how to connect over a binary TCP transport without knox. Notice the lack of a username and password in the connection string, using only the keytab…

import java.sql.*;
import org.apache.hadoop.security.UserGroupInformation;

public class hive2 {
  public static void main (String args[]) {
    try {
      org.apache.hadoop.conf.Configuration conf = new     org.apache.hadoop.conf.Configuration();
      conf.set("hadoop.security.authentication", "Kerberos");
      UserGroupInformation.setConfiguration(conf);
      UserGroupInformation.loginUserFromKeytab("hive/[email protected]", "/etc/security/keytabs/hive.service.keytab");
      Class.forName("org.apache.hive.jdbc.HiveDriver");
      System.out.println("getting connection");
      Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:10000/;principal=hive/[email protected]");
      System.out.println("got connection");
      con.close();
    }
    catch (Exception e) {
      e.printStackTrace();
    }
  }
}

..and then with http, again, using only the keytab…

import java.sql.*;
import org.apache.hadoop.security.UserGroupInformation;

public class hive2 {
  public static void main (String args[]) {
    try {
      org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
      conf.set("hadoop.security.authentication", "Kerberos");
      UserGroupInformation.setConfiguration(conf);
      UserGroupInformation.loginUserFromKeytab("hive/[email protected]", "/etc/security/keytabs/hive.service.keytab");
      Class.forName("org.apache.hive.jdbc.HiveDriver");
      System.out.println("getting connection");
      Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:10001/;principal=hive/[email protected];transportMode=http;httpPath=cliservice");
      System.out.println("got connection");
      con.close();
    }
    catch (Exception e) {
      e.printStackTrace();
    }
  }
}

…and with a simple user authentication against knox (notice the lack of a keytab and principal in the URL, but the addition of the username and password)…

import java.sql.*;

public class hive2 {
  public static void main (String args[]) {
    try {
      Class.forName("org.apache.hive.jdbc.HiveDriver");
      System.out.println("getting connection");
      Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:8443/;ssl=true;transportMode=http;httpPath=gateway/default/hive","showard","********");
      System.out.println("got connection");
      con.close();
    }
    catch (Exception e) {
      e.printStackTrace();
    }
  }
}

To connect with beeline, you must first kinit the hive service keytab (located under /etc/security/keytabs), then you can connect.

[[email protected] ~]# kinit -kt /etc/security/keytabs/hive.service.keytab hive/[email protected]
[[email protected] ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hive/[email protected]

Valid starting     Expires            Service principal
06/25/15 23:21:13  06/26/15 09:21:13  krbtgt/[email protected]
        renew until 07/02/15 23:21:13
[[email protected] ~]# beeline -u "jdbc:hive2://ambari2012:10001/;principal=hive/[email protected]D2012.LOCAL;transportMode=http;httpPath=cliservice"

It was surprisingly difficult to get simple examples of each, so hopefully this helps someone.

7 comments for “Examples of connecting to kerberos hive in JDBC

  1. August 5, 2015 at 6:47 PM

    Hi Lester,

    This will work with any account that has permissions to use the database. Simply create a keytab with your principal to which you do have access (using ktutil on linux), and you should be good to go. The hadoop code does assume the standard username/[email protected], so as long as that exists in your KDC (or its trusted partner such as Active Directory), you should be OK. Note the requirement of the host name in the principal. This is actually required in the org.apache.hive.service.auth.KerberosSaslHelper class

    public static TTransport getKerberosTransport(String principal, String host, TTransport underlyingTransport, Map saslProps, boolean assumeSubject)
        throws SaslException
      {
        try
        {
          String[] names = principal.split("[/@]");
          if (names.length != 3) {
            throw new IllegalArgumentException("Kerberos principal should have 3 parts: " + principal);
          }
    

    Having noted that, we used this when we *did* have access to the hive keytab. I actually think it is easier to configure LDAP authentication (even without knox), and simply instruct hive where to lookup the username/password when connecting.

  2. Blake
    September 17, 2015 at 10:51 AM

    Thanks, I’ve been having trouble connecting to Hive in a kerberized cluster for weeks and this helped me finally get it working.

  3. hari
    December 3, 2015 at 3:57 PM

    Hi,

    I have the keytab, krb5.conf on my mac and I am still getting the below issue, can some help be provided,please

    java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
    at java.lang.Thread.run(Thread.java:745)
    Caused by: java.lang.IllegalArgumentException: Can’t get Kerberos realm
    at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:261)
    at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:297)
    at com.capitalone.eCukeHive.AppTest.main(AppTest.java:29)

  4. Saucey
    December 10, 2015 at 1:55 PM

    Hi,

    Thanks for all the info. This is really helpful. I have quick question,

    In my org, hiveserver2 is set up with kerberos and using the java jdbc template you have provided. However, while running the template, on cmd, I am being asked to enter the kerberos password. Is this the exepcted behavior

  5. Anonymous
    May 23, 2018 at 2:42 AM

    thank you so much

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.