We had a need to authenticate user requests against AD in a kerberos enabled cluster, and allow “local” hive sessions to use only a keytab. Below are the examples of each.
First, we show how to connect over a binary TCP transport without knox. Notice the lack of a username and password in the connection string, using only the keytab…
import java.sql.*;
import org.apache.hadoop.security.UserGroupInformation;
public class hive2 {
public static void main (String args[]) {
try {
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hive/[email protected]", "/etc/security/keytabs/hive.service.keytab");
Class.forName("org.apache.hive.jdbc.HiveDriver");
System.out.println("getting connection");
Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:10000/;principal=hive/[email protected]");
System.out.println("got connection");
con.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
..and then with http, again, using only the keytab…
import java.sql.*;
import org.apache.hadoop.security.UserGroupInformation;
public class hive2 {
public static void main (String args[]) {
try {
org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab("hive/[email protected]", "/etc/security/keytabs/hive.service.keytab");
Class.forName("org.apache.hive.jdbc.HiveDriver");
System.out.println("getting connection");
Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:10001/;principal=hive/[email protected];transportMode=http;httpPath=cliservice");
System.out.println("got connection");
con.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
…and with a simple user authentication against knox (notice the lack of a keytab and principal in the URL, but the addition of the username and password)…
import java.sql.*;
public class hive2 {
public static void main (String args[]) {
try {
Class.forName("org.apache.hive.jdbc.HiveDriver");
System.out.println("getting connection");
Connection con = DriverManager.getConnection("jdbc:hive2://ambari2012:8443/;ssl=true;transportMode=http;httpPath=gateway/default/hive","showard","********");
System.out.println("got connection");
con.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
}
To connect with beeline, you must first kinit the hive service keytab (located under /etc/security/keytabs), then you can connect.
[root@ambari2012 ~]# kinit -kt /etc/security/keytabs/hive.service.keytab hive/[email protected] [root@cmhlpdlkedat01 ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: hive/[email protected] Valid starting Expires Service principal 06/25/15 23:21:13 06/26/15 09:21:13 krbtgt/[email protected] renew until 07/02/15 23:21:13 [root@ambari2012 ~]# beeline -u "jdbc:hive2://ambari2012:10001/;principal=hive/[email protected];transportMode=http;httpPath=cliservice"
It was surprisingly difficult to get simple examples of each, so hopefully this helps someone.
Hi Lester,
This will work with any account that has permissions to use the database. Simply create a keytab with your principal to which you do have access (using ktutil on linux), and you should be good to go. The hadoop code does assume the standard username/hostname@REALM, so as long as that exists in your KDC (or its trusted partner such as Active Directory), you should be OK. Note the requirement of the host name in the principal. This is actually required in the org.apache.hive.service.auth.KerberosSaslHelper class
Having noted that, we used this when we *did* have access to the hive keytab. I actually think it is easier to configure LDAP authentication (even without knox), and simply instruct hive where to lookup the username/password when connecting.
Thanks, I’ve been having trouble connecting to Hive in a kerberized cluster for weeks and this helped me finally get it working.
Hi,
I have the keytab, krb5.conf on my mac and I am still getting the below issue, can some help be provided,please
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Can’t get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:261)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:297)
at com.capitalone.eCukeHive.AppTest.main(AppTest.java:29)
Hi,
Thanks for all the info. This is really helpful. I have quick question,
In my org, hiveserver2 is set up with kerberos and using the java jdbc template you have provided. However, while running the template, on cmd, I am being asked to enter the kerberos password. Is this the exepcted behavior
thank you so much