, , , , ,

Scripting data collection

In part 1 I identified a number of SNMP values to obtain from the Fortigate Firewall, mine being an FG100D with version 5.0 firmware. This article presents a simple collection BASH shell script that will retrieve the System Processor metrics from the firewall and upload them to a RabbitMQ server using the Rabbitmq-c AMQP library, specifically the “amqp_sendstring” example program.

To get this example working you need a RabbitMQ server to send the data to and you need to download the rabbitmq-c library available on Github. other than that its a simple clean shell script to dump the SNMP data to a file, parse it and collect the name and value parameters and then send them to the MQ server.

If you don’t have an MQ server, you would simply omit the “/opt/amqp/amqp_sendstring” line and place your own output mechanism, this could be a file based app or even call this script via Nagios and output a nagios formatted string with the correct return code (usuall “OK”).

The Script

Normally this shell script is run every minute in CRON so data capture is continuous.

# Get system processor counters from SNMP MIB in Fortigate and upload to RabbitMQ server.
# formats data in "collectd" format for import into graphite/whisper database.
# (C) Sid Young 2014
# Free for educational use.
TIMESTAMP=`date '+%s'`
snmpwalk -v2c -m ALL -c public enterprises.fortinet.fnFortiGateMib.fgSystem| grep Processor 1>${FGSYS} 2>/dev/null
cat $FGSYS | cut -d ":" -f3-|grep -v STRING > $FGDATA
while read LINE
   PARAM=`echo $LINE|sed 's/^ *//'|cut -d '=' -f1`
   VALUE=`echo $LINE|cut -d ':' -f2-|sed 's/^ *//'|cut -d' ' -f1`
   MSG="qmon.${HOST}.system.${PARAM} ${VALUE} ${TIMESTAMP}"
   /opt/amqp/amqp_sendstring ${MQHOST} 5672 ${EXCHANGE} ${ROUTINGKEY} "`echo ${MSG}`" > /dev/null
done < ${FGDATA}
# Cleanup
rm -f $FGSYS
rm -f $FGDATA

 What happens now?

In my monitoring system the RabbitMQ server will place the data into the queue which has the exchange “metrics” and routing key “metrics” bound to it. Then an application called “carbon” retrieves the data from queue as it arrives and stores it in a database called “whisper”, there is a retention period defined so the database is a fixed size. I normally store 60 days of data.

At the front end a program called “Grafana” is configured to display a number of panels, and the data in the carbon database is bound to the graph. The result is a very cool dashboard of Fortigate Data.



Some other articles I wrote on Dashboards:

Some earlier SNMP articles