host=cmhlpecomecm* EOMReservationService AND (started OR ended) | eval tmp=split(_raw,” “) | eval thread=mvindex(tmp,5) | transaction thread startswith=”started” endswith=”ended” | timechart span=1h avg(duration) median(duration) The query above results in the following output on the Events tab of the UI… …and the…
Category: Splunk
Splunk query to extract JSP from stack trace in error log
We used what is below to parse through stack traces that had been logged in JBOSS. We were looking for the page most frequently impacted by a persistent transaction blocking issue. host=cmhlpecomecm* (ORA-00060 OR ORA-02049) AND org.apache.jsp | rex field=_raw…
Splunk – Histogram/distribution of JBOSS session length
As you can see below, 95% of our sessions are less than five minutes in length. This is very useful in sizing the session timeout variable in your context.xml file. host=cmhlpecomweb* sourcetype=access* | rex mode=sed field=_raw “s/\t/~/g” | eval tmp=split(_raw,”~”)…
Splunk calculating JBOSS session time statistics
We used this to produce a distribution of how long our sessions last. We print the median, average, and maximum session length based on the first and last occurrence of a given JSESSIONID. We also filter out those sessions less…
Splunk – Query to print distribution of requests by device type
We had a need to understand what type of devices our customers were using; specifically, Apple or Android. We came up with what is below… host=cmhlpecomweb* sourcetype=access* (“iphone” OR “ipad” OR “android”) | rex mode=sed field=_raw “s/\t/~/g” | eval tmp=split(_raw,”~”)…
Splunk regular expression count by day
This one tripped me up. The rex command is not a filter, it merely extracts the value where it exists. As such, you can’t simply say… rex field “(?mysearch)” | timechart span=1d count as total …as this will result in…
Interesting way to get domain associated with Amazon IP
While looking for timeouts in splunk for an unrelated reason… I noticed the Amazon IP’s above. We normally don’t know the service they represent. I connected to the IP in my browser on port 443, and noticed that of course…
Splunk query to group Apache sessions by minute
The 15th field of our Apache log is the application server session ID. We truncate the _time field to minute, and get a distinct count of sessions in each minute. The number is 14 below because the source array is…
Splunk query to group URI request by first three IP address octets
We needed this to understand the source of a large influx of requests for a given URI pattern. import splunklib.client as client import splunklib.results as results service = client.connect(host=”*******”,port=”8089″,username=”showard”,password=”************”) job = “”” search host=\”cmhlpecomweb*\” sourcetype=access_combined karlie-kloss | eval temp=split(_raw,\”\t\”) |…
Splunk query for custom Apache access log format
We have a kludgy access log format. It certainly isn’t standard. At any rate, the out of the box transforms.conf can’t handle it. Rather than change that, I elected to split the lines on the fly; not as fast, but…