Python script to print blocking threads that are not blocked themselves

There are a lot of GUI tools to process the call stacks for a “thread dump” taken on a running JVM. This is just a command line tool to do it, which is normally where I start troubleshooting when there is an issue.

In this case, these are simply “parent blockers”, for lack of a better word. This simply means those threads that are blocking another thread, when they themselves are not blocked. When a thread isn’t blocked, but is blocking another thread (or many threads), you need to look closely at the blockers call stack. This will enable you to determine why it has a lock on a synchronized block code, and what is preventing it from exiting the monitor. It could be a database lock, hung NFS filesystem, etc. The call stack will usually show you where the issue lies.

import sys

locked = dict()
stacks = dict()
addresses = dict()
done = dict()

def loadStackTraces():
  thisStack = list()
  next = True
  for line in open(sys.argv[1],"r"):
    if next == True:
      thisThread = line.split()[0]
      next = False
    if len(line) == 1: #only has newline character, so line is functionally empty
      stacks[thisThread] = thisStack
      thisStack = list()
      next = True
    else:
      thisStack.append(line)
    if line.find("waiting to lock") > -1:
      addresses[thisThread] = line.split()[4]
    elif line.find("locked") > -1:
      locked[thisThread] = line.split()[2]

loadStackTraces()
for i in addresses:
  for j in locked:
    if locked[j] == addresses[i]:
      if j not in addresses:
        if j not in done:
          print '\n-----------------------------------------------------------------------------------------\n'
          print j, " is a blocker, but is not blocked itself.  Investigate the thread stack below:"
          print '\n-----------------------------------------------------------------------------------------\n'
          k = 0
          for line in stacks[j]:
            if k > 9:
              break
            print line.strip()
            k = k + 1
        done[j] = 1

The following example run shows the first ten lines of a problematic thread…

-bash-4.1$ python threadTree.py dump.txt

-----------------------------------------------------------------------------------------

"http-172.28.8.60-10543-5"  is a blocker, but is not blocked itself.  Investigate the thread stack below:

-----------------------------------------------------------------------------------------

"http-172.28.8.60-10543-5" daemon prio=10 tid=0x00007feff8005800 nid=0x5cad in Object.wait() [0x00007ff029ee6000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0000000534ae34d8> (a atg.service.lockmanager.ClientLockEntry)
at atg.service.lockmanager.ClientLockEntry.acquireWriteLock(ClientLockEntry.java:421)
- locked <0x0000000534ae34d8> (a atg.service.lockmanager.ClientLockEntry)
at atg.service.lockmanager.ClientLockManager.acquireWriteLock(ClientLockManager.java:1167)
at atg.service.lockmanager.ClientLockManager.acquireWriteLock(ClientLockManager.java:1263)
at atg.commerce.gifts.GiftlistManager.removeItemFromGiftlist(GiftlistManager.java:1042)
at com.express.order.purchase.ExpressCartFormHandler.postAddItemToOrder(ExpressCartFormHandler.java:3756)
-bash-4.1$ exit

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.