stage is the worst term for this. Seriously. stage, to me at least, implies a temporary location. I come from a database background, so a staging table is thought of as the location where data is stored prior to being…
Month: April 2015
Are files in HDFS immutable?
Call me cynical, I just am a bit of a doubting Thomas. Using our previous write test code, we simply run the exact same test, only we do it twice. [root@cmhlpdlkedat15 ~]# hadoop HDFSWriteTest foobar.txt hdfs://cmhlpdlkedat14.expressco.com:8020 [root@cmhlpdlkedat15 ~]# hdfs dfs…
Getting additional data on a slow hive reducer
We have a query that has been running for almost two hours on the last of 1,009 reducers in the first reduce phase. We wanted to gain additional insight into what it was doing, so we first find the node…
Dump heap programatically
This is one way you can dump the heap of a running JVM from within a page. This is not something you would place in a running production system, as you don’t want random GET’s to keep dumping the heap…