We have started to build a Performance System Monitoring solution at work using Message Queuing to transport the JSON data from client systems (servers, applications and containers) to bespoke and off the shelf monitoring an trend analysis software. In the middle is a RabbitMQ Message Queuing solution so that the metrics being generated can be routed to the apps that need the data. We need it to scale to 1000’s of containers and servers (and application metrics) so a conventional system wont cut it and we only want the clients to generate data once per interval but have the single piece of data route to all the concerned end points without the client needing to be reconfigured.

The solution for us is to generate JSON formatted metric data and then have the Message Queuing Server route the data. One of the processes I’m writing is a disk logging app in C++ that logs different kinds of metrics… so I decided to use RapidJSON to process the data as it claimed to be fast and was a header only installation, so no pre-compiled programs or modules.

The main issue with RapidJSON is their is virtually no documentation so I started to build this page more for my record of how to do some of the common tasks.

Compiling an App

My first processing application is given a file and processes it, the file contains JSON data, so to build the application the following commands can be used (this assumes the rapidjson header files directory is under the current directory):

g++ -g -Wall -c processor.cpp
g++ -g -Wall processor.o -o processor

The following sample code prints the JSON data type for the top level tags encountered.The file is read from the command line.

The following is the sample JSON code:

{"Protocol":{"Version":"1.0","System":"SERVER","SubSystem":"METRICS"},"HostInfo":{"HostName":"sid.conetix.com.au","HostIp":""},"MsgDetail":{"Class":"MEMORY","Date":"2014-01-10","Time":"13:25:01","MsgText":"mem-used=946124 |mem-free=102452 |mem-total=1048576","MsgStatus":"0"}}


#include "rapidjson/document.h"
#include "rapidjson/rapidjson.h"
#include "rapidjson/filestream.h"

#include <iostream>
#include <string>

using namespace rapidjson;
using namespace std;

int main(int argc, char**argv)
static const char* kTypeNames[] = { "Null", "False", "True", "Object", "Array", "String", "Number" };

	FILE * fp = fopen(argv[1],"r");
	FileStream is(fp);
	Document doc;

		cout << "Parsing error" << endl;
 		for(Value::ConstMemberIterator itr = doc.MemberBegin(); itr != doc.MemberEnd(); ++itr)
		printf("Type of member %s is %s\n", itr->name.GetString(), kTypeNames[itr->value.GetType()]);
	return 0;


./dump-top-level-tags /opt/mqget/sample-1.json 

Type of member Protocol is Object
Type of member HostInfo is Object
Type of member MsgDetail is Object

… more to come….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s