Querying Hadoop from Tomcat

Below is a very simple example for how to print in Tomcat the contents of a file stored in HDFS.

I am not entirely sure where this would be useful, for the following reasons:

* Unless you have a list of the output files from a set of map reduce jobs, and can parse the output in a JSP, the raw files may not be relevant
* HUE (the Hadoop UI from Cloudera) already has a file browser component

However, the exercise was very useful in understanding the Hadoop filesystem API, and how it can be used in a web front end.

Use it as a learning tool.

<%@page import="java.io.*, java.net.*, java.util.*,org.apache.hadoop.conf.*,org.apache.hadoop.fs.*;"%>
<%
  try {
    Configuration conf = new Configuration();
    FileSystem fs = FileSystem.get(new URI("hdfs://expressdb1:9000"), conf);
    Path file = new Path(request.getParameter("fname"));
    FSDataInputStream getIt = fs.open(file);
    BufferedReader d = new BufferedReader(new InputStreamReader(getIt));
    String s = "";
    while ((s = d.readLine()) != null) {
      out.println(s + "
");
    }
    d.close();
    fs.close();
  }
  catch (Exception e) {
    e.printStackTrace();
  }
%>

If you wanted to take this to the next level, you could list every file in HDFS, and construct a hyperlink with the file name. The page would either:

present a list of hyperlinks to each HDFS file if the fname query string parameter wasn’t present

-or-

fetch the file in the fname parameter if the fname query string parameter was present

What is below does exactly that.

<%@page import="java.io.*, java.net.*, java.util.*,org.apache.hadoop.conf.*,org.apache.hadoop.fs.*;"%>
<%
  try {
    Configuration conf = new Configuration();
    FileSystem fs = FileSystem.get(new URI("hdfs://expressdb1:9000"), conf);
    if (request.getParameter("fname") == null) {
      RemoteIterator files = fs.listFiles(new Path("/"),true);
      while (files.hasNext()) {
        LocatedFileStatus lfs = (LocatedFileStatus)files.next();
        out.println("" + lfs.getPath() + "
");
      }
    }
    else {
      Path file = new Path(request.getParameter("fname"));
      FSDataInputStream fileIn = fs.open(file);
      BufferedReader d = new BufferedReader(new InputStreamReader(fileIn));
      String s = "";
      while ((s = d.readLine()) != null) {
        out.println(s + "
");
      }
      d.close();
    }
    fs.close();
  }
  catch (Exception e) {
    e.printStackTrace();
  }
%>