Popular logging frameworks such as Log4j, Logback, and SLF4J, etc. provide logging functionality for the individual microservice application. However, when a group of run together to provide complete business functionality, it becomes really challenging to TRACE a request across all the services, especially in case of failures.

Hence it is highly recommended to have a centralized logging solution in place, to have all the log messages stored in a central location rather than on local machine/container of each microservice. This eliminates dependency on the local disk space (volumes) and can HELP retain the logs for a long time for analysis in the future.

The Elasticsearch, Logstash, and Kibana tools, collectively known as the ELK stack, provide an end-to-end logging solution in the distributed application; providing a centralized logging solution. ELK stack is one of the most commonly used architectures for custom logging management in cloud-based Microservices applications.

Elasticsearch is a NoSQL database used to store the logs as documents. Logstash is a log pipeline tool that accepts logs as input from various micro service applications, executes transformations if required and stores data into the target (Elasticsearch database).

Kibana is a UI that works on top of Elasticsearch, providing a VISUAL representation of the logs and ability to search them as required.  All three tools are typically installed on a single server, known as the ELK server.

In a centralized logging approach, applications should follow a standard for log messages. Each log message having a context, message, and correlation ID. The context information is ideally can be the IP address, user information, process details, timestamp, etc. The message is a simple text description of the SCENARIO. The correlation ID is dynamically generated and is common across all that used for end-to-end TRACKING of a request/task.