Archive for January, 2011

Gzip in python

Wednesday, January 26th, 2011

We have a large text file containing 325 million integer values that we need to load into a database. The file uncompressed is about 4.5GB. Even though disk is cheap, we didn’t have enough readily available to quickly decompress the file. As such, I used the python gzip module to decompress the source file... »

Posted in Database, Development, Oracle, Python | No Comments »

Python multiple producer/consumer queue

Wednesday, January 19th, 2011

Below is a simple skeleton for a python queue that is both fed by and consumed by multiple different threads. We run five threads, each of which populate a queue with random integers. Lastly, we sleep a couple of seconds between each put, to simulate an ongoing workload. Our popper thread, which reads items... »

Posted in Development, Python | No Comments »

Handling constraints when truncating a table

Wednesday, January 19th, 2011

Oracle doesn’t supply a TRUNCATE TABLE TABLE_NAME CASCADE CONSTRAINTS syntax structure as it does when dropping a table. The anonymous block below will truncate all tables in the current schema and handle the constraints violations (foreign keys) that may be violated based on the order in which the tables are truncated. DANGER! Please ensure... »

Posted in Database, Oracle | No Comments »

Dumping Oracle DDL part two

Saturday, January 8th, 2011

Below is a python script that strip the double quotes from the output of dbms_metadata.get_ddl, as well as the storage clauses. The initial clause is often far too high, so this allows you to extract the DDL and run it wherever you like. You can call it as follows: 1 ./getDdl.py case_insensitive_user_for_which_to_get_tables_and_indexes 1 2... »

Posted in Database, Development, Oracle, Python | No Comments »

History of computers

Saturday, January 1st, 2011

I found this interesting link while searching for something else. It is a fairly complete history of computer technology from the time before Christ to 2003. Enjoy. »

Posted in Architecture | No Comments »