{"id":107,"date":"2023-04-04T09:05:00","date_gmt":"2023-04-04T09:05:00","guid":{"rendered":"https:\/\/cloudtechner.com\/blog\/?p=107"},"modified":"2024-06-13T11:06:45","modified_gmt":"2024-06-13T11:06:45","slug":"log-management-and-distributed-tracing-using-grafana-loki-and-tempo","status":"publish","type":"post","link":"https:\/\/cloudtechner.com\/blog\/log-management-and-distributed-tracing-using-grafana-loki-and-tempo\/","title":{"rendered":"Log Management and Distributed Tracing using Grafana Loki and Tempo"},"content":{"rendered":"\n

Author: Navneet Singh Guraya<\/a>, Associate Engineer – CloudDevOps<\/p>\n\n\n

Grafana<\/a>, <\/span>Log Management<\/a>, <\/span>Loki<\/a>, <\/span>Monitoring<\/a>, <\/span>Observability<\/a>, <\/span>Step-by-Step Guide<\/a>, <\/span>Tempo<\/a><\/div>\n\n\n
\n\n\n\n

Monitoring is a crucial part of any IT systems. Monitoring is the task of assessing the health of a system by collecting and analysing aggregate data from different IT systems based on a predefined set of metrics and logs. In general, monitoring measures the health of the application, such as creating a rule that alerts when the app is nearing 80% CPU usage, helping prevent downtime. in IT systems Monitoring is not new, organisations have been using monitoring tools to monitor their systems since long. However, monitoring has its limitations. For monitoring to work, we need to know what metrics and logs to track. If we have not predicted a problem, it can miss key production failures and other issues.<\/p>\n\n\n\n

Due to great benefits of micro-services based applications, organisations are now depending more and more on distributed architectures to provide application services. As systems and applications are becoming more complex, organisation are realising implementing just Monitoring tools is not sufficient for them. They need more than a monitoring tool which has the ability to understand a complex system\u2019s internal state based on external outputs. In short they need monitoring systems which can work on Pro-active approach not the reactive one which traditional monitoring tools provide and thats where Observability <\/strong>comes into picture. Observability tools go deeper than monitoring internal states and troubleshooting problems. These platforms help teams solve problems faster, which in turn, optimizes pipelines and gives more time for core business operations and innovation.<\/p>\n\n\n\n

With observability, collecting the measurements of logs, metrics, and distributed traces are the three key pillars to achieving success.<\/p>\n\n\n\n

    \n
  1. Logs<\/strong>: These are structured or unstructured text records of discreet events that occurred at a specific time.<\/li>\n\n\n\n
  2. Metrics:<\/strong> These are the values represented as counts or measures that are often calculated or aggregated over a period of time. Metrics can originate from a variety of sources, including infrastructure, hosts, services, cloud platforms, and external sources.<\/li>\n\n\n\n
  3. Distributed tracing:<\/strong> This displays activity of a transaction or request as it flows through applications and shows how services connect, including code-level details.<\/li>\n<\/ol>\n\n\n\n

    Logs and Distributed tracing are important aspect of any Observability <\/strong>platform. This blog explains instructions to setup Log management and distributed tracing using popular Open source observability platform Grafana and Grafana plugins Grafana LOKI and Grafana Tempo<\/p>\n\n\n\n

    Before we move into setup, let us look at the basics of Grafana, Grafana LOKI and Grafana Tempo<\/p>\n\n\n\n

    Grafana<\/h3>\n\n\n\n

    Grafana is a popular open-source interactive visualisation platform which allows users to see their data via charts and graphs that are unified into one dashboard for easier interpretation and understanding. We can also query and set alerts based on the metrics and thresholds. Grafana supports traditional server environments, Kubernetes Clusters and various Cloud Services. Being an open-source tool Grafana is Cloud Agnostics tool. When it comes to modern micro-services applications, Grafana integrates very well and can visualize deep level information of each micro-service.<\/p>\n\n\n\n

    Grafana LOKI<\/h3>\n\n\n\n

    Loki is a horizontally scalable, highly available, multi-tenant log aggregation solution. It\u2019s designed to be both affordable and simple to use. Rather than indexing the contents of the logs, it uses a set of labels for each log stream.<\/p>\n\n\n\n

    Grafana Loki was inspired by Prometheus\u2019 architecture, which uses labels to index data. This allows Loki to store indexes in less amount of space. Furthermore, Loki\u2019s design is fully compatible with Prometheus, allowing developers to apply the same label criteria across the two platforms.<\/p>\n\n\n\n

    Grafana Tempo<\/h3>\n\n\n\n

    Grafana Tempo is an open source, easy-to-use, and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki. Tempo can ingest common open source tracing protocols, including Jaeger, Zipkin, and OpenTelemetry<\/p>\n\n\n\n

    Grafana LOKI and Grafana Tempo Architecture<\/h3>\n\n\n\n

    Below architecture explains how Grafana collects and monitor logs and tracing using Grafana LOKI and Grafana Tempo Extensions.<\/p>\n\n\n\n