Author: Steve

Adding a partition in Kafka

The impact is pretty clear. Below we load ten events using a given key. We add a partition, then load ten more events. Notice the partition differs after adding a partition, even though the key is the same.

Python KafkaProducer and lost events

While testing the python kafka package and its associated KafkaProducer class, I found that events were “lost”; i.e., they were never successfully persisted in the destination topic. Various blogs suggest setting a sleep(2) call, or whatever, which seemed odd. I…

Uploading a CSV to a REST web service

import org.apache.http.client.methods.*; import org.apache.http.client.*; import org.apache.http.impl.client.*; import org.apache.http.*; import org.apache.http.entity.mime.*; import org.apache.http.entity.*; import java.io.*; public class API_H01 { public static void main (String args[]) { try { HttpClient client = HttpClientBuilder.create().build(); File f = new File(“/opt/tomcat8/hierarchy.csv”); MultipartEntityBuilder builder = MultipartEntityBuilder.create();…

Jython to call REST service and write output to a file

from org.apache.http.client.methods import * from org.apache.http.impl.client import * from org.apache.http.client import * from org.apache.http import * from org.json import * from java.io import * from java.util import * from java.lang import * client = HttpClientBuilder.create().build(); url = “https://host/api/invoices”; rq =…

Example of a structure in C

This is just a placeholder for how these work. Largely, this is the predecessor of object oriented programming in that a structure has properties that can be assigned. #include #include struct student {   int studentId;   char studentName[40];  …

Robotic Process Automation

McKinsey Consulting defines Robotic Process Automation, or “RPA”, as follows: “RPA takes the robot out of the human. The average knowledge worker employed on a back-office process has a lot of repetitive, routine tasks that are dreary and uninteresting. RPA…

anon memory segments on Linux

[adm-showard@cmhlcarchapp01 ~]$ cat mem.c #include #include #include int main() { struct rusage r_usage; void *m = malloc(4194304000L); memset(m,0,4194304000L); sleep (60); } [adm-showard@cmhlcarchapp01 ~]$ gcc -o mem mem.c [adm-showard@cmhlcarchapp01 ~]$ ./mem & [1] 25748 [adm-showard@cmhlcarchapp01 ~]$ pmap -d 25748 25748: ./mem…