{"id":752,"date":"2010-07-17T15:45:26","date_gmt":"2010-07-17T20:45:26","guid":{"rendered":"http:\/\/appcrawler.com\/wordpress\/?p=752"},"modified":"2011-07-06T09:33:17","modified_gmt":"2011-07-06T14:33:17","slug":"parsing-hanganalyze-output-for-blocking-sessions","status":"publish","type":"post","link":"http:\/\/appcrawler.com\/wordpress\/2010\/07\/17\/parsing-hanganalyze-output-for-blocking-sessions\/","title":{"rendered":"Parsing hanganalyze output for blocking sessions"},"content":{"rendered":"<p>The hanganalyze command was introduced in Oracle 8i.  It is very useful, especially to mere mortals like us.  It is formatted in a human readable format.  In contrast, a systemstate dump normally contains a lot of memory addresses and offsets, which is useful to Oracle support, but not so much to us.<\/p>\n<p>hanganalyze is *mandatory* to be generated in a cluster issue.  It can be obtained by issuing the following from any node in the cluster which you can access.<\/p>\n<pre lang=\"sql\" line=\"1\">\r\nsqlplus \/ as sysbda\r\noradebug setmypid\r\noradebug \u2013g all hanganalyze 3\r\noradebug tracefile_name\r\nexit\r\n<\/pre>\n<p>Go to $ORACLE_BASE\/udump and review the output by using the following python script:<\/p>\n<pre lang=\"python\" line=\"1\">\r\n#!\/home\/oracle\/local\/bin\/python\r\n\r\nimport re, string, fileinput, sys\r\n\r\nbefore = []\r\n\r\n#------------------------------------------------------------------------------------\r\n\r\nclass blocker:\r\n  def __init__(self, value):\r\n    self.blockedIds = []\r\n    self.blockerId = value\r\n  def addBlocked(self,value):\r\n    self.blockedIds.append(value)\r\n  def getBlockerId(self):\r\n    return self.blockerId\r\n\r\n#------------------------------------------------------------------------------------\r\n\r\ndef parseFile(f, a):\r\n  START_STATS=False\r\n  FOUND_STATS=False\r\n  for line in fileinput.input(f):\r\n    if START_STATS == False:\r\n      if re.match('^State of nodes', line):\r\n        START_STATS = True\r\n    elif FOUND_STATS == False:\r\n      if re.match('^[[0-9]', line):\r\n        FOUND_STATS = True\r\n        a.append(line)\r\n    elif FOUND_STATS == True:\r\n      if re.match('^session', line):\r\n        break\r\n      else:\r\n        a.append(line)\r\n  fileinput.close()\r\n\r\n#------------------------------------------------------------------------------------\r\n\r\nparseFile(sys.argv[1], before)\r\n\r\nblockers = []\r\narr = -1\r\nfound = False\r\nHANG = False\r\nfor e in before:\r\n  found = False\r\n  j = 0\r\n  BLOCKED=string.split(e,\"\/\")[0]\r\n  INST=int(string.split(e,\"\/\")[1]) + 1\r\n  SID=string.split(e,\"\/\")[2]\r\n  SERIAL=string.split(e,\"\/\")[3]\r\n  OSPID=string.split(e,\"\/\")[5]\r\n  STATE=string.split(e,\"\/\")[6]\r\n  BLOCKER=string.split(e,\"\/\")[9]\r\n  if string.find(STATE,\"HANG\") > -1:\r\n    HANG = True\r\n  if string.find(BLOCKER,\"[\") != -1:\r\n    for i in range(len(blockers)):\r\n      if blockers[i].getBlockerId() == BLOCKER:\r\n        arr = i\r\n        found = True\r\n        break\r\n      else:\r\n        found = False\r\n    if found != True:\r\n      arr = arr + 1\r\n      a = blocker(BLOCKER)\r\n      a.addBlocked(\"SID = \" + str(SID) + \" SERIAL# = \" + str(SERIAL) + \" on INST \" + str(INST) + \" (OSPID = \" + str(OSPID) + \")\")\r\n      blockers.append(a)\r\n    else:\r\n      a = blockers[arr]\r\n      a.addBlocked(\"SID = \" + str(SID) + \" SERIAL# = \" + str(SERIAL) + \" on INST \" + str(INST) + \" (OSPID = \" + str(OSPID) + \")\")\r\n\r\nif HANG == True:\r\n  print \"------------------------------------------------------------------------\"\r\n  print \"\\n\\nW A R N I N G ! ! !  Possible True hang found\\n\\n\"\r\n  print \"Search for the case sensitive word HANG in \" + sys.argv[1] + \"\\n\\n\"\r\n\r\n\r\nprint \"------------------------------------------------------------------------\"\r\nprint \"BLOCKING SESSION ANALYSIS:\"\r\nprint \"------------------------------------------------------------------------\"\r\n\r\nfor e3 in blockers:\r\n  for e in before:\r\n    BLOCKED=string.split(e,\"\/\")[0]\r\n    if BLOCKED == e3.blockerId:\r\n      INST=int(string.split(e,\"\/\")[1]) + 1\r\n      SID=string.split(e,\"\/\")[2]\r\n      SERIAL=string.split(e,\"\/\")[3]\r\n      OSPID=string.split(e,\"\/\")[5]  \r\n      BLOCKER=string.split(e,\"\/\")[9]\r\n      if string.find(BLOCKER,\"[\") != -1:\r\n        ALSO_BLOCKED = True\r\n      else:\r\n        ALSO_BLOCKED = False\r\n      break\r\n  print \"------------------------------------------------------------------------\"\r\n  if ALSO_BLOCKED:\r\n    print \"SID = \" + str(SID) + \" SERIAL# = \" + str(SERIAL) + \" on INST \" + str(INST) + \" (OSPID = \" + str(OSPID) + \") - Also blocked\"\r\n  else:\r\n    print \"SID = \" + str(SID) + \" SERIAL# = \" + str(SERIAL) + \" on INST \" + str(INST) + \" (OSPID = \" + str(OSPID) + \")\"\r\n  print \"...is blocking...\"\r\n  for e4 in e3.blockedIds:\r\n    print \"\\t\" + e4\r\n<\/pre>\n<p>This will generate output similar to the following:<\/p>\n<pre lang=\"text\" line=\"1\">\r\nC:\\>parseHang.py wcprod1_diag_23622.trc\r\n------------------------------------------------------------------------\r\nBLOCKING SESSION ANALYSIS:\r\n------------------------------------------------------------------------\r\n------------------------------------------------------------------------\r\nSID = 1533 SERIAL# = 35901 on INST 1 (OSPID = 28371)\r\n...is blocking...\r\n        SID = 1524 SERIAL# = 252 on INST 1 (OSPID = 28407)\r\n------------------------------------------------------------------------\r\nSID = 1524 SERIAL# = 252 on INST 1 (OSPID = 28407) - Also blocked\r\n...is blocking...\r\n        SID = 1504 SERIAL# = 33 on INST 2 (OSPID = )\r\n        SID = 1507 SERIAL# = 81 on INST 2 (OSPID = )\r\n        SID = 1511 SERIAL# = 37 on INST 2 (OSPID = )\r\n        SID = 1516 SERIAL# = 44536 on INST 2 (OSPID = )\r\n        SID = 1517 SERIAL# = 9722 on INST 2 (OSPID = )\r\n        SID = 1525 SERIAL# = 32 on INST 2 (OSPID = )\r\n        SID = 1532 SERIAL# = 372 on INST 2 (OSPID = )\r\n        SID = 1534 SERIAL# = 45 on INST 2 (OSPID = )\r\n        SID = 1535 SERIAL# = 61522 on INST 2 (OSPID = )\r\n        SID = 1539 SERIAL# = 59 on INST 2 (OSPID = )\r\n        SID = 1543 SERIAL# = 8 on INST 2 (OSPID = )\r\n        SID = 1547 SERIAL# = 37 on INST 2 (OSPID = )\r\n        SID = 1555 SERIAL# = 45 on INST 2 (OSPID = )\r\n        SID = 1556 SERIAL# = 34 on INST 2 (OSPID = )\r\n        SID = 1557 SERIAL# = 37 on INST 2 (OSPID = )\r\n        SID = 1562 SERIAL# = 20988 on INST 2 (OSPID = )\r\n        SID = 1581 SERIAL# = 35597 on INST 2 (OSPID = )\r\n        SID = 1584 SERIAL# = 63056 on INST 2 (OSPID = )\r\n        SID = 1589 SERIAL# = 47551 on INST 2 (OSPID = )\r\n        SID = 1614 SERIAL# = 60597 on INST 2 (OSPID = )\r\n\r\nC:\\>\r\n<\/pre>\n<p>You may be able to identify the session that is causing the issue using output similar to what is above.  It may be a background process on one of the instances, in which case you will have to terminate that instance.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The hanganalyze command was introduced in Oracle 8i. It is very useful, especially to mere mortals like us. It is formatted in a human readable format. In contrast, a systemstate dump normally contains a lot of memory addresses and offsets,&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/appcrawler.com\/wordpress\/2010\/07\/17\/parsing-hanganalyze-output-for-blocking-sessions\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[19,24,22,26],"tags":[],"_links":{"self":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/752"}],"collection":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/comments?post=752"}],"version-history":[{"count":12,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/752\/revisions"}],"predecessor-version":[{"id":917,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/752\/revisions\/917"}],"wp:attachment":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/media?parent=752"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/categories?post=752"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/tags?post=752"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}