{"id":3127,"date":"2013-07-24T13:18:45","date_gmt":"2013-07-24T18:18:45","guid":{"rendered":"http:\/\/appcrawler.com\/wordpress\/?p=3127"},"modified":"2013-07-24T13:18:45","modified_gmt":"2013-07-24T18:18:45","slug":"high-latch-free-waits-on-oracle-xa-transactions","status":"publish","type":"post","link":"http:\/\/appcrawler.com\/wordpress\/2013\/07\/24\/high-latch-free-waits-on-oracle-xa-transactions\/","title":{"rendered":"High latch free waits on Oracle XA transactions"},"content":{"rendered":"<p>We experienced large waits as well as the associated CPU spikes, when using distributed XA transactions in an ATG\/JBOSS application server to Oracle database environment.<\/p>\n<p>Oracle &#8220;support&#8221; was, per usual, of absolutely no help is diagnosing the situation.  I honestly don&#8217;t know where they get their pool of support &#8220;analysts&#8221;.  I guess they analyze how to quickly get something off their plate, collect your support payment and&#8230;ahhhh, never mind.<\/p>\n<p>What is below is a subset of the class that creates the XA objects using the Oracle classes, and then simply does a select from an inventory table using a variable value for the primary key being fetched.<\/p>\n<pre lang=\"java\">\r\n      XAConnection XACLocal = OXADSLocal.getXAConnection();\r\n      Connection ConnLocal = XACLocal.getConnection();\r\n      XAResource XARLocal = XACLocal.getXAResource();\r\n\r\n      byte [] gid = new byte[64];\r\n      gid[0] = (byte) 1;\r\n\r\n      byte [] bqidLocal = new byte[64];\r\n      bqidLocal[0] = (byte) 1;\r\n      int x = testXaLatch.r.nextInt();\r\n      Xid XidLocal = new OracleXid(x, gid, bqidLocal);\r\n\r\n      \/\/What is below will result in a latch free event, mostly due to the \"global tx hash mapping\" latch\r\n      XARLocal.start(XidLocal, XAResource.TMNOFLAGS);\r\n      PreparedStatement pst = ConnLocal.prepareStatement(\"select * from dcs_inventory where inventory_id = ?\");\r\n      String y = Integer.toString(testXaLatch.r.nextInt(INVENTORY_IDS.size()));\r\n      pst.setString(1,y);\r\n      ResultSet rst  = pst.executeQuery();\r\n      while (rst.next()) {\r\n        \/\/nothing\r\n      }\r\n      XARLocal.end(XidLocal, XAResource.TMSUCCESS);\r\n<\/pre>\n<p>We saw far fewer of these events when we ran what is functionally the same code, but invoked the PL\/SQL API as shown below&#8230;<\/p>\n<pre lang=\"java\">\r\nimport java.sql.*;\r\nimport javax.sql.*;\r\nimport java.util.*;\r\nimport java.math.*;\r\nimport javax.transaction.xa.*;\r\nimport oracle.jdbc.xa.OracleXid;\r\nimport oracle.jdbc.xa.client.*;\r\n\r\npublic class testXADB implements Runnable {\r\n\r\n  Thread t;\r\n  static Random r;\r\n  static ArrayList<String> INVENTORY_IDS;\r\n\r\n  public static void main(String args[]) {\r\n    try {\r\n      Class.forName(\"oracle.jdbc.driver.OracleDriver\");\r\n      Connection conn = DriverManager.getConnection(\"jdbc:oracle:thin:***************************\");\r\n      Statement stm = conn.createStatement();\r\n      stm.setFetchSize(10000);\r\n      r = new Random();\r\n      ResultSet rstInventoryIds = stm.executeQuery(\"select inventory_id from dcs_inventory\");\r\n      INVENTORY_IDS = new ArrayList<String>();\r\n      while (rstInventoryIds.next()) {\r\n        INVENTORY_IDS.add(rstInventoryIds.getString(1));\r\n      }\r\n      rstInventoryIds.close();\r\n      System.out.println(new java.util.Date());\r\n      for (int i = 1; i <= 200; i++) {\r\n        testXADB r1 = new testXADB();\r\n      }\r\n    }\r\n    catch(Exception e) {\r\n      e.printStackTrace();\r\n    }\r\n  }\r\n\r\n  testXADB() {\r\n    try {\r\n      t = new Thread(this);\r\n      t.start();\r\n    }\r\n    catch (Exception e) {\r\n      e.printStackTrace();\r\n    }\r\n  }\r\n\r\n  public void run() {\r\n    try {\r\n\r\n      Connection conn = DriverManager.getConnection(\"jdbc:oracle:thin:***********************************************\");\r\n      conn.setAutoCommit(false);\r\n      CallableStatement pst = conn.prepareCall(\"declare \" +\r\n                                                 \"l number;\" +\r\n                                                 \"xid number := ?;\" +\r\n                                               \"begin \" +\r\n                                                 \"l := dbms_xa.xa_start(dbms_xa_xid(xid,hextoraw('1'),hextoraw('1')), dbms_xa.tmnoflags);\" +\r\n                                                 \"for r in (select * from atg_core_prod.dcs_inventory where inventory_id = ?) loop \" +\r\n                                                 \"  null;\" +\r\n                                                 \"end loop;\" +\r\n                                                 \"l := dbms_xa.xa_end(dbms_xa_xid(xid), dbms_xa.tmsuccess);\" +\r\n                                               \"end;\");\r\n      long START_TIME = System.currentTimeMillis();\r\n\r\n      while (System.currentTimeMillis() - START_TIME < (20 * 1000 * 60)) {\r\n        int x = testXADB.r.nextInt();\r\n        pst.setInt(1,x);\r\n        String y = INVENTORY_IDS.get(testXADB.r.nextInt(INVENTORY_IDS.size()));\r\n        pst.setString(2,y);\r\n        pst.execute();\r\n      }\r\n    }\r\n    catch(Exception e) {\r\n      e.printStackTrace();\r\n    }\r\n  }\r\n}\r\n<\/pre>\n<p>Our initial thought was that the performance discrepancy may be due to the fact the code is executing directly in the database, but that didn't make sense as the latch appears to be taken for the duration of time between the start and end procedures.  In other words, you could sleep for a year between start and end, and the latch wouldn't be held for that duration (thank goodness).  Conceptually, this means the software used to call the procedure shouldn't matter.<\/p>\n<p>For those interested, we did find the following is the tree taken in a distributed transaction from straight Java JDBC code.  We found this by decompiling the classes in the ojdbc6.jar file.<\/p>\n<p>client driver calls java_xa database package via a CallableStatement in the client side driver<br \/>\nthe java_xa PL\/SQL API calls a java class stored in the database<br \/>\nThe java class in the database calls the native library for handling XA in the database<\/p>\n<p>Our assumption is that the native code is the same code invoked by dbms_xa (wrapped code).<\/p>\n<p>More to come on this, these are just initial notes.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We experienced large waits as well as the associated CPU spikes, when using distributed XA transactions in an ATG\/JBOSS application server to Oracle database environment. Oracle &#8220;support&#8221; was, per usual, of absolutely no help is diagnosing the situation. I honestly&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/appcrawler.com\/wordpress\/2013\/07\/24\/high-latch-free-waits-on-oracle-xa-transactions\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[19,24,25,22],"tags":[],"_links":{"self":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3127"}],"collection":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/comments?post=3127"}],"version-history":[{"count":14,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3127\/revisions"}],"predecessor-version":[{"id":4450,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/3127\/revisions\/4450"}],"wp:attachment":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/media?parent=3127"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/categories?post=3127"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/tags?post=3127"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}