Sitecore: Logging with EKL (ElasticSearch+Logstash+Kibana)

Why not to use standard files?

Usually when you are working with Sitecore you will have standard file based logs. For development this is more that enough but when it comes to other environments it could be very nice to have all logs sentralized. Moreover need to be sure that logs are not lost in cases when something went wrong and it is easy to find information in it.
So my goal is to create solution base on EKL (ElasticSearch+Logstash+Kibana) which will get log data from Sitecore and provide possibilities to search and work with them. ()


Kibana Query interface
Kibana chart options

First try

First of all I've tried to use file based input plugin from Logstash and it wasn't really successful :) It works, but there where some troubles. The plugin locks all files to check updates & Sitecore is doing the same, so this will lead to potential issues. So I look ed through other logstash input plugins and stopped at rabbitmq

RabbitMQ is a messages broker that works on AMQP.
What is AMQP?
AMQP (Advanced Message Queuing Protocol) is a networking protocol that enables conforming client applications to communicate with conforming messaging middleware brokers.
Brokers and Their RoleMessaging brokers receive messages from publishers (applications that publish them, also known as producers) and route them to consumers (applications that process them).
Next step was to integrate Sitecore with RabbitMQ but I will describe it in separate post.

3 comments :

  1. Did you ever get this to work? We are looking to do something similar.

    ReplyDelete
    Replies
    1. Yep, not in production btw, but tests went well. I've investigated several variant, and decided to use RabbitMQ (see separate post). However you could use log4net over TCP or even try to make it working with rolling files (you should choose MinimalLocking appender)

      Delete
    2. Thanks. Glad to hear it went well.

      Delete