Troubleshooting oozie browser connection to secure cluster – Part 1

This is really a generic listing of using the spnego open source software to access a network service that requires authentication. In our case, it was useful to troubleshoot our inability to connect to the oozie web UI. We still haven’t resolved that issue, but using what is below proved to us that the oozie configuration was OK. This passes an Active Directory username and password, used to fetch a ticket from AD, and then successfully connects to oozie programmatically.

However, when we request the oozie web UI URL, in all browsers we tried, we see a 403 (“GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag)”). We have configured each for what is supposed to be required. This includes the following for firefox:

* network.negotiate-auth.delegation-uris
* network.negotiate-auth.trusted-uris
* network.auth.use-sspi (tried both true and false)

Using wireshark, we can see a kerberos ticket (prefix of “YII”) passed to oozie, but it still returns a 401 response code, and ultimately a 403.

When we arrive at a resolution, a part 2 of this post will be provided.

—————————————————————-

Download the spnego jar from the following URL…

http://sourceforge.net/projects/spnego/files/

Place the jar in the same directory as the files you need to create, which are shown below.

spnego.java

import net.sourceforge.spnego.*;
import java.net.*;
import java.io.*;
import java.util.*;

public class spnego {
  public static void main(final String[] args) throws Exception {
    System.setProperty("java.security.krb5.conf", "krb5.conf");
    System.setProperty("java.security.auth.login.config", "login.conf");

    SpnegoHttpURLConnection con = null;

    try {
      con = new SpnegoHttpURLConnection("spnego-client", "your_domain_username", "your_domain_password");
      con.connect(new URL("http://your_fqdn_oozie_host:11000/oozie/"));
      InputStream is = con.getInputStream();
      Scanner s = new Scanner(is).useDelimiter("\\A");
      System.out.println(s.hasNext() ? s.next() : "");
    }
    finally {
      if (con != null) {
        con.disconnect();
      }
    }
  }
}

krb5.conf

[libdefaults]
  renew_lifetime = 7d
  forwardable = true
  default_realm = YOUR_DOMAIN
  ticket_lifetime = 24h
  dns_lookup_realm = false
  dns_lookup_kdc = false

[domain_realm]

  your_domain = YOUR_DOMAIN

[realms]
  YOUR_DOMAIN = {
    admin_server = your_domain_server
    kdc = your_domain_server
  }

login.conf

spnego-client {
  com.sun.security.auth.module.Krb5LoginModule required;
};

set your CLASSPATH to the current directory and the spnego jar you downloaded
compile the class above – javac spnego.java
run the class above at the command line – java spnego

You should not see any exceptions, and should see the main oozie page text returned on your console. If you do see an error such as 401 or 403, it means the problem is not exclusively with the browser.