I like simple, command line test cases. Lather, rinse, repeat (do any shampoo bottles actually have that anymore 🙂 ?) I wanted to ensure I could prove that ingests to hadoop actually didn’t send everything through the name node, which…
One way to tell if a thread pool is hung
A hung ThreadPoolExecutor in java can be manifested in various ways. At least one, that I wanted to post for my own notes, is below. A thread dump may show something like what is below: “pool-9-thread-5” prio=6 tid=0x01b84400 nid=0x16e0 waiting…
awk binary string to ascii string converter
There are probably better ways to this with lshift and rshift functions built into awk, but this is functional. { while (++i
Apache Hive inserts
This is a simple example for getting started with loading data into a hadoop environment front ended by hive. Notice how much longer simple operations take than they do in a RDBMS. This is attributable to the fact hadoop has…
Can you run java hprof and a custom agentlib together?
We run newrelic, which is an Advanced Performance Management (APM) software stack that will profile your java application from the browser all the way to the database. It is really good software, affordable, and highly recommended. I also wanted to…
Linux threads and CPU affinity
Linux provides native threading capabilities. This allows a program to run with multiple processes all accessing the same shared data, without the headache of Interprocess Communication (IPC) calls. Essentially, a single process makes calls to the system pthread library to…