{"id":6723,"date":"2018-06-23T08:17:21","date_gmt":"2018-06-23T13:17:21","guid":{"rendered":"http:\/\/appcrawler.com\/wordpress\/?p=6723"},"modified":"2019-01-15T14:33:17","modified_gmt":"2019-01-15T19:33:17","slug":"spark-in-python","status":"publish","type":"post","link":"http:\/\/appcrawler.com\/wordpress\/2018\/06\/23\/spark-in-python\/","title":{"rendered":"Spark in python"},"content":{"rendered":"<p>Just a simple example to get started.<\/p>\n<pre>\r\n#.\/pyspark --jars \/tmp\/mysql-connector-java-5.1.45-bin.jar\r\nWelcome to\r\n      ____              __\r\n     \/ __\/__  ___ _____\/ \/__\r\n    _\\ \\\/ _ \\\/ _ `\/ __\/  '_\/\r\n   \/__ \/ .__\/\\_,_\/_\/ \/_\/\\_\\   version 1.5.2\r\n      \/_\/\r\n\r\nUsing Python version 2.6.6 (r266:84292, Jan 22 2014 09:42:36)\r\nSparkContext available as sc, HiveContext available as sqlContext.\r\n>>> dataframe_mysql = sqlContext.read.format(\"jdbc\").options( url=\"jdbc:mysql:\/\/cmhlcarchapp01:3306\/inventory\",driver = \"com.mysql.jdbc.Driver\",dbtable = \"inventory\",user=\"root\", password=\"welcome\").load()\r\n>>> dataframe_mysql.show()\r\n+---+--------+--------+----------+--------+\r\n|sku|location|in_stock|in_transit|reserved|\r\n+---+--------+--------+----------+--------+\r\n|  1|     955|      10|         1|       1|\r\n|  1|    2009|      10|         1|       1|\r\n|  1|     173|      10|         1|       1|\r\n|  2|     955|       1|         0|       0|\r\n|  2|     955|       1|         0|       0|\r\n|  2|     955|       1|         0|       0|\r\n|  2|     955|       1|         0|       0|\r\n|  3|     173|      10|         5|       3|\r\n|  3|     173|      10|         5|       3|\r\n|  3|     173|      10|         5|       3|\r\n|  3|     173|      10|         5|       3|\r\n|  1|     955|       1|         1|       1|\r\n|  1|     955|     100|         1|       1|\r\n+---+--------+--------+----------+--------+\r\n\r\n>>>\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Just a simple example to get started. #.\/pyspark &#8211;jars \/tmp\/mysql-connector-java-5.1.45-bin.jar Welcome to ____ __ \/ __\/__ ___ _____\/ \/__ _\\ \\\/ _ \\\/ _ `\/ __\/ &#8216;_\/ \/__ \/ .__\/\\_,_\/_\/ \/_\/\\_\\ version 1.5.2 \/_\/ Using Python version 2.6.6 (r266:84292, Jan&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/appcrawler.com\/wordpress\/2018\/06\/23\/spark-in-python\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[104],"tags":[],"_links":{"self":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6723"}],"collection":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/comments?post=6723"}],"version-history":[{"count":2,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6723\/revisions"}],"predecessor-version":[{"id":6732,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/posts\/6723\/revisions\/6732"}],"wp:attachment":[{"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/media?parent=6723"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/categories?post=6723"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/appcrawler.com\/wordpress\/wp-json\/wp\/v2\/tags?post=6723"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}