Are files in HDFS immutable?

Call me cynical, I just am a bit of a doubting Thomas. Using our previous write test code, we simply run the exact same test, only we do it twice. [root@cmhlpdlkedat15 ~]# hadoop HDFSWriteTest foobar.txt hdfs://cmhlpdlkedat14.expressco.com:8020 [root@cmhlpdlkedat15 ~]# hdfs dfs…

Drop all tables in a schema

This assumes the tables have no integrity constraints related to a table in another schema. This just loops until all tables are dropped. If you are the nervous type, add an exception handler for the integrity constraint exception, and exit…

Hive transactions

Below is just a simple example of hive transactions. These are very useful on slowly changing type 1 dimension tables for which you do not wish to retain history, but only the most recent value of the row. The table…

JDBC batch re-processing

I have always wanted to duplicate how GoldenGate handles batch failures. In GoldenGate, you can set the parameter “batchsql BATCHTRANSOPS 2000″, for example, to send 2000 statements at a time to the database for execution. In general, this is far…