Sitecore logging integration with Graylog

With Sitecore infrastructure which expands beyond the basic 1 CM and 1 CD, having a centralized logging mechanism is crucial in order to understand what happens in the servers in a consolidated view.

Some of the popular APM out there in the market offers an automatic  records of exception/errors which occurred in the application, this proves really useful when investigating a problem down to the exact line of code which raised the exception. Some types of issues requires you to dig through the Sitecore logs in order to gain an understanding what happened in the server itself , for example when investigating publishing issues, security audits, index crawling, exm logs and fxm logs in the delivery servers.

For those types issues we typically look at different types of Sitecore logs in each servers in order to find out what exactly happened, if we have multiple servers in the picture we would need to go into each servers or pull down the log files from each servers and analyze the log files. This is where centralized logging for Sitecore logs will come in handy as we would have a centralized view of the log files which span multiple servers and we can perform a search query against the log information.

A couple of centralized Log provider in the market are:

  • https://azure.microsoft.com/en-us/services/application-insights/
  • https://www.splunk.com/
  • https://www.elastic.co/products/elasticsearch
  • https://www.graylog.org/

I stumble across Graylog in a recent project where we currently pushing windows event logs to it but not the Sitecore log information yet. So I played around with it a bit to get a feel *rolling sleeves*.

There’s a couple of steps that we need to figure out in order to integrate the Sitecore logs to Graylog

  1. Create a custom log appender
  2. Install Graylog server
  3. Configure Graylog server
  4. Send Sitecore log information to Graylog server

Create a custom log appender

Sitecore uses log4net as the logging framework which is extensible, by default it uses the LogFileAppender class which outputs the log information in text files.

As I want to send the Sitecore log information to Graylog server through http protocol, I would need to create a custom log appender. And as sending log information from Sitecore to Graylog server needs to be in a certain format, GELF format to be precise, we would need to format the log format to match GELF specification in order for Graylog server to understand and able to parse it.

I found the gelf4net library which was mentioned in the Graylog documentation which already did all the heavy lifting of formatting the data. When I installed the nuget package and configured the log4net section in Sitecore according to this library documentation it doesn’t work though.

One gotcha that I found is that when we want to create a custom log appender in Sitecore , we need to reference the AppenderSkeleton class in the Sitecore.Kernel assembly – previously I added log4net nuget package (comes with gelf4net as a dependency) and was hoping that would work, instead it failed miserably 🙁

In the end I created my own appender class which replicate gelf4net appender implementation. https://github.com/reyrahadian/sitecore-gelf-logappender/blob/master/ScGraylog/Appender/GelfHttpAppender.cs 

Install Graylog server

Having read through the Graylog documentation, the easiest and quickest way to setup a Graylog server for my POC is to download the VM. It’s all preconfigured, we just need to load the .voa file using VMware player or VirtualBox to get it up and running.

reference: http://docs.graylog.org/en/latest/pages/getting_started.html

Configure Graylog server

The next things that we need to do after we have our Graylog server up and running is to configure the input source. There’s multiple options that Graylog provides: http, tcp, udp or file dumps.

HTTP input source fits what I need for my POC, so I created a new HTTP input source and have it running.

Send Sitecore log information to Graylog server

Here’s where we put things together. With our custom log appender ready, now we only need to send the Sitecore log information to our Graylog server by using the following log4net configuration

Check if things works as expected

If everything works as expected then you should see some log information coming from Sitecore

 

source code: https://github.com/reyrahadian/sitecore-gelf-logappender

Leave a Reply

Your email address will not be published. Required fields are marked *