With data in the hiveserver2.log file, this awk scriptlet prints the timestamp, SQL, and seconds to run.\u00a0There is an issue where the parser thread hands off to the executor, and you can\u2019t always tie the two together.\u00a0 However, at a minimum, this tells you the SQL and execution time. One limitation is that it won’t show queries which aren’t completed for whatever reason.<\/p>\n
I may see if I can add a plugin to write this to a file in a more usable format.<\/p>\n
\r\nBEGIN {\r\n while (++i <= 100) {\r\n pad = pad\"*\"\r\n }\r\n}\r\n{\r\n if ($0 ~ \"Driver.java:execute.*Starting command\") {\r\n r[$5] = $2\r\n split($0,t,\":\")\r\n n = 6\r\n sql = \"\"\r\n while (++n <= length(t)) {\r\n sql = sql\"\"t[n]\r\n }\r\n\r\n getline\r\n while (1) {\r\n if ($0 !~ \"HiveServer2\") {\r\n sql = sql\"\"$0\r\n }\r\n else {\r\n break\r\n }\r\n getline;\r\n }\r\n s[$5] = sql\r\n }\r\n else if ($0 ~ \"PerfLogEnd.*PERFLOG method=Driver.run\") {\r\n split($13,end,\"=\")\r\n printf(\"%s\\n %-20s %s %.2f\\n\",pad,r[$5],s[$5],end[2]\/1000)\r\n }\r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"With data in the hiveserver2.log file, this awk scriptlet prints the timestamp, SQL, and seconds to run.\u00a0There is an issue where the parser thread hands off to the executor, and you can\u2019t always tie the two together.\u00a0 However, at a…<\/p>\n