Importing old data into munin

This python script tries to figure out the original rrdtool create parameters that were used to create a given rrd. It's very basic (handles only RRA:Average afaik) so don't expect magic from it. It expects rrdtool to be your path.

Sample invocation:

lmwangi@jaghpu:~/rrd$ python rrdinfo-parser.py -f all.rrd

rrdtool create all.rrd --start 978300900 --step 300 \

DS:a:COUNTER:600:U:U \

DS:c:DERIVE:600:U:U \

DS:b:GAUGE:600:U:U \

DS:d:ABSOLUTE:600:U:U \

RRA:AVERAGE:0.5:1:10 \

lmwangi@jaghpu:~/rrd$ python rrdinfo-parser.py -f test.rrd

rrdtool create test.rrd --start 920804400 --step 300 \

DS:speed:COUNTER:600:U:U \

RRA:AVERAGE:0.5:1:24 \

RRA:AVERAGE:0.5:6:10 \

So say we have a Munin derived rrd that we need to import old data into. First we run the script to extract the schema, then we edit the --start parameter to an epoch timestamp that predates your data and finally recreate the rrd.

rrdtool create service-logs-total-g.rrd --start 1108598400 --step 300 \                                                   
       DS:42:GAUGE:600:U:U \
       RRA:AVERAGE:0.5:1:576 \
       RRA:MIN:0.5:1:576 \
       RRA:MAX:0.5:1:576 \
       RRA:AVERAGE:0.5:6:432 \
       RRA:MIN:0.5:6:432 \
       RRA:MAX:0.5:6:432 \
       RRA:AVERAGE:0.5:24:540 \
       RRA:MIN:0.5:24:540 \
       RRA:MAX:0.5:24:540 \
       RRA:AVERAGE:0.5:288:450

Then we import the processed data (in my case, log file summaries) which looks like:

head 2010.data
1262296800:4241:221:173:276
1262297400:3920:197:155:231
1262298000:4171:184:226:208
1262298600:3700:197:159:244
1262299200:3350:195:166:227

where the fields are ts:total:errors:unknowns:etc.

Using a simple awk script we extract field one and two (timestamp and total) and import them into the rrd and when we are done, overwrite the original rrd.

FILE=service-logs-total-g.rrd
for record in $(cat 2010.data|awk -F: '{print $1":"$2}');
do 
 rrdupdate $FILE $record;
done