The impact is pretty clear. Below we load ten events using a given key. We add a partition, then load ten more events. Notice the partition differs after adding a partition, even though the key is the same.
Author: Steve
Consume tweets through Kafka and push to Elastic
Python KafkaProducer and lost events
While testing the python kafka package and its associated KafkaProducer class, I found that events were “lost”; i.e., they were never successfully persisted in the destination topic. Various blogs suggest setting a sleep(2) call, or whatever, which seemed odd. I…
Help, I locked myself out of my own bucket!
I found myself in the position in the title of this post. To skip to the root cause, I had configured a VPC endpoint and configured the bucket policy to allow access from only the associated VPC. That makes sense.…
Query SQL Server error log from SQL
use master go declare @table table(logdate datetime,processinfo varchar(max),text varchar(max)) insert into @table exec xp_readerrorlog select * from @table where processinfo like ‘spid%’ order by 1
Uploading a CSV to a REST web service
import org.apache.http.client.methods.*; import org.apache.http.client.*; import org.apache.http.impl.client.*; import org.apache.http.*; import org.apache.http.entity.mime.*; import org.apache.http.entity.*; import java.io.*; public class API_H01 { public static void main (String args[]) { try { HttpClient client = HttpClientBuilder.create().build(); File f = new File(“/opt/tomcat8/hierarchy.csv”); MultipartEntityBuilder builder = MultipartEntityBuilder.create();…
Jython to call REST service and write output to a file
from org.apache.http.client.methods import * from org.apache.http.impl.client import * from org.apache.http.client import * from org.apache.http import * from org.json import * from java.io import * from java.util import * from java.lang import * client = HttpClientBuilder.create().build(); url = “https://host/api/invoices”; rq =…
Example of a structure in C
This is just a placeholder for how these work. Largely, this is the predecessor of object oriented programming in that a structure has properties that can be assigned. #include #include struct student { int studentId; char studentName[40]; …
Robotic Process Automation
McKinsey Consulting defines Robotic Process Automation, or “RPA”, as follows: “RPA takes the robot out of the human. The average knowledge worker employed on a back-office process has a lot of repetitive, routine tasks that are dreary and uninteresting. RPA…
anon memory segments on Linux
[adm-showard@cmhlcarchapp01 ~]$ cat mem.c #include #include #include int main() { struct rusage r_usage; void *m = malloc(4194304000L); memset(m,0,4194304000L); sleep (60); } [adm-showard@cmhlcarchapp01 ~]$ gcc -o mem mem.c [adm-showard@cmhlcarchapp01 ~]$ ./mem & [1] 25748 [adm-showard@cmhlcarchapp01 ~]$ pmap -d 25748 25748: ./mem…