Introduction

We have started to build a Performance System Monitoring solution at work using Message Queuing to transport the JSON data from client systems (servers, applications and containers) to bespoke and off the shelf monitoring an trend analysis software. In the middle is a RabbitMQ Message Queuing solution so that the metrics being generated can be routed to the apps that need the data. We need it to scale to 1000’s of containers and servers (and application metrics) so a conventional system wont cut it and we only want the clients to generate data once per interval but have the single piece of data route to all the concerned end points without the client needing to be reconfigured.

A Possible Solution

The solution for us is to generate JSON formatted metric data and then have the Message Queuing Server route the data. One of the processes I’m writing is a disk logging app in C++ that logs different kinds of metrics… so I decided to use RapidJSON to process the data as it claimed to be fast and was a header only installation, so no pre-built compiled programs or library modules need to be built.

Issues!

The main issue with RapidJSON is their is virtually no documentation so I started to build this page more for my record of how to do some of the common tasks. there is

Compiling an App

My first processing application is given a file and processes it, the file contains JSON data, so to build the application the following commands can be used (this assures the RapidJSON header files directory is under the current directory):

g++ -g -Wall -c processor.cpp
g++ -g -Wall processor.o -o processor

The following sample code prints the JSON data type for the top level tags encountered.The file is read from the command line.

Code Samples

The following is the sample JSON code:

{"Protocol":{"Version":"1.0","System":"SERVER","SubSystem":"METRICS"},"HostInfo":{"HostName":"server.domain.com","HostIp":"192.168.66.179"},"MsgDetail":{"Class":"MEMORY","Date":"2014-01-10","Time":"13:25:01","MsgText":"mem-used=946124 |mem-free=102452 |mem-total=1048576","MsgStatus":"0"}}

Code

#include "rapidjson/document.h"
#include "rapidjson/rapidjson.h"
#include "rapidjson/filestream.h"

#include <iostream>
#include <string>

using namespace rapidjson;
using namespace std;

int main(int argc, char**argv)
{
static const char* kTypeNames[] = { "Null", "False", "True", "Object", "Array", "String", "Number" };

	FILE * fp = fopen(argv[1],"r");
	FileStream is(fp);
	Document doc;

	if(doc.ParseStream<0>(is).HasParseError()==true)
	{
		cout << "Parsing error" << endl;
 	}
 	else
 	{
 		for(Value::ConstMemberIterator itr = doc.MemberBegin(); itr != doc.MemberEnd(); ++itr)
		printf("Type of member %s is %s\n", itr->name.GetString(), kTypeNames[itr->value.GetType()]);
	}
	return 0;
}

Output

./dump-top-level-tags /opt/mqget/sample-1.json 

Type of member Protocol is Object
Type of member HostInfo is Object
Type of member MsgDetail is Object

… more to come….

Advertisements