Importing old data into munin

This python script tries to figure out the original rrdtool create parameters that were used to create a given rrd. It's very basic (handles only RRA:Average afaik) so don't expect magic from it. It expects rrdtool to be your path.

Sample invocation:

lmwangi@jaghpu:~/rrd$ python rrdinfo-parser.py -f all.rrd

rrdtool create all.rrd --start 978300900 --step 300 \

DS:a:COUNTER:600:U:U \

DS:c:DERIVE:600:U:U \

DS:b:GAUGE:600:U:U \

DS:d:ABSOLUTE:600:U:U \

RRA:AVERAGE:0.5:1:10 \

lmwangi@jaghpu:~/rrd$ python rrdinfo-parser.py -f test.rrd

rrdtool create test.rrd --start 920804400 --step 300 \

DS:speed:COUNTER:600:U:U \

RRA:AVERAGE:0.5:1:24 \

RRA:AVERAGE:0.5:6:10 \

So say we have a Munin derived rrd that we need to import old data into. First we run the script to extract the schema, then we edit the --start parameter to an epoch timestamp that predates your data and finally recreate the rrd.

rrdtool create service-logs-total-g.rrd --start 1108598400 --step 300 \                                                   
       DS:42:GAUGE:600:U:U \
       RRA:AVERAGE:0.5:1:576 \
       RRA:MIN:0.5:1:576 \
       RRA:MAX:0.5:1:576 \
       RRA:AVERAGE:0.5:6:432 \
       RRA:MIN:0.5:6:432 \
       RRA:MAX:0.5:6:432 \
       RRA:AVERAGE:0.5:24:540 \
       RRA:MIN:0.5:24:540 \
       RRA:MAX:0.5:24:540 \
       RRA:AVERAGE:0.5:288:450

Then we import the processed data (in my case, log file summaries) which looks like:

head 2010.data
1262296800:4241:221:173:276
1262297400:3920:197:155:231
1262298000:4171:184:226:208
1262298600:3700:197:159:244
1262299200:3350:195:166:227

where the fields are ts:total:errors:unknowns:etc.

Using a simple awk script we extract field one and two (timestamp and total) and import them into the rrd and when we are done, overwrite the original rrd.

FILE=service-logs-total-g.rrd
for record in $(cat 2010.data|awk -F: '{print $1":"$2}');
do 
 rrdupdate $FILE $record;
done

A simple munin graphing script that uses mk-heartbeat to monitor replication lag between two MySQL servers. Place it in your /etc/munin/plugins/ directory and configure your environment.

/etc/munin/plugin-conf.d//mysql

[mysql_*]

env.mysqlconnection DBI:mysql:mysql

env.mysqluser Reporter

env.mysqlpassword Blah

/etc/munin/plugins/mysql_replication_lag

#!/bin/sh
# 
# Plugin to monitor the mysql replication lag
#
# Parameters:
#       
#       config   (required)
#       autoconf (optional - used by munin-config)
#
# $Log$
# Revision 1: Tue Feb 15 17:04:59 MUT 2011
# Initial writeup and testing
#
#
#
# Magic markers (Used by munin-config and some installation scripts.
# Optional):
#
#%# family=auto
#%# capabilities=autoconf

if [ "$1" = "autoconf" ]; then
       echo yes
       exit 0
fi

if [ "$1" = "config" ]; then

       echo 'graph_title Replication Lag'
       echo 'graph_args --base 1000 -l 0'
       echo 'graph_vlabel Replication lag in seconds'
       echo 'graph_category mysql'
       echo 'graph_info This graph monitors the replication lag using mk-heartbeat.'
       echo 'lag.label Lag in seconds'
       echo 'lag.warning :300'
       echo 'lag.warning :3600'
       echo 'lag.info The number of currently open files.'
       exit 0
fi
echo -n "lag.value "
mysql -u ${mysqluser} -p${mysqlpassword} -B -N -e "select now()-ts as replication_lag from maatkit.heartbeat;"