Friday, November 02, 2012

Datadog and JMXTrans file output

You can use JMXTrans to poll JMX counters from your JVM and write them to a file using the KeyOutWriter.  

It will create files with the format:  <metric> <value> <timestamp>.

If you want to send these to Datadog for graphing and analysis, you can use the Dogstream facility to parse the Jmxtrans output files.  By default it execpts lines like:

     <metric> <timestamp> <value>

which isn't quite what JMXTrans outputs.   To get around this create this custom parser:
import calendar
from datetime import datetime
def parse_jmxtrans(logger, line):
        metric, value, timestamp = line.rstrip("\n").split("\t")
        date = datetime.fromtimestamp(float(timestamp[:10]))
        date = calendar.timegm(date.timetuple())
        value = float(value.strip())
        return (metric, date, value, {})

and configure it like this in datadog.conf:
       dogstreams: /tmp/jmx_key_out.txt:parsers:parse_jmxtrans

where you replace /tmp/jmx_key_out.txt with the path you configured in JMXTrans.