Tracing in Effect
Although logs and metrics are useful to understand the behavior of individual services, they are not enough to provide a complete overview of the lifetime of a request in a distributed system.
In a distributed system, a request can span multiple services and each service can make multiple requests to other services to fulfill the request. In such a scenario, we need to have a way to track the lifetime of a request across multiple services to diagnose what services are the bottlenecks and where the request is spending most of its time.
A span represents a single unit of work or operation within a request. It provides a detailed view of what happened during the execution of that specific operation.
Each span typically contains the following information:
Span Component | Description |
---|---|
Name | Describes the specific operation being tracked. |
Timing Data | Timestamps indicating when the operation started and its duration. |
Log Messages | Structured logs capturing important events during the operation. |
Attributes | Metadata providing additional context about the operation. |
Spans are key building blocks in tracing, helping you visualize and understand the flow of requests through various services.
A trace records the paths taken by requests (made by an application or end-user) as they propagate through multi-service architectures, like microservice and serverless applications.
Without tracing, it is challenging to pinpoint the cause of performance problems in a distributed system.
A trace is made of one or more spans. The first span represents the root span. Each root span represents a request from start to finish. The spans underneath the parent provide a more in-depth context of what occurs during a request (or what steps make up a request).
Many Observability back-ends visualize traces as waterfall diagrams that may look something like this:
Waterfall diagrams show the parent-child relationship between a root span and its child spans. When a span encapsulates another span, this also represents a nested relationship.
You can add tracing to an effect by creating a span using the Effect.withSpan
API. This helps you track specific operations within the effect.
Example (Adding a Span to an Effect)
Instrumenting an effect with a span does not change its type. If you start with an Effect<A, E, R>
, the result remains an Effect<A, E, R>
.
To print spans for debugging or analysis, you’ll need to install the required tracing tools. Here’s how to set them up for your project.
Choose your package manager and install the necessary libraries:
Once the dependencies are installed, you can set up span printing using OpenTelemetry. Here’s an example showing how to print a span for an effect.
Example (Setting Up and Printing a Span)
The output provides detailed information about the span:
Field | Description |
---|---|
traceId | A unique identifier for the entire trace, helping trace requests or operations as they move through an application. |
parentId | Identifies the parent span of the current span, marked as undefined in the output when there is no parent span, making it a root span. |
name | Describes the name of the span, indicating the operation being tracked (e.g., “myspan”). |
id | A unique identifier for the current span, distinguishing it from other spans within a trace. |
timestamp | A timestamp representing when the span started, measured in microseconds since the Unix epoch. |
duration | Specifies the duration of the span, representing the time taken to complete the operation (e.g., 2895.769 microseconds). |
attributes | Spans may contain attributes, which are key-value pairs providing additional context or information about the operation. In this output, it’s an empty object, indicating no specific attributes in this span. |
status | The status field provides information about the span’s status. In this case, it has a code of 1, which typically indicates an OK status (whereas a code of 2 signifies an ERROR status) |
events | Spans can include events, which are records of specific moments during the span’s lifecycle. In this output, it’s an empty array, suggesting no specific events recorded. |
links | Links can be used to associate this span with other spans in different traces. In the output, it’s an empty array, indicating no specific links for this span. |
Here’s how a span looks when the effect encounters an error:
Example (Span for an Effect that Fails)
In this example, the span’s status code is 2
, indicating an error. The message in the status provides more details about the failure.
You can provide extra information to a span by utilizing the Effect.annotateCurrentSpan
function.
This function allows you to attach key-value pairs, offering more context about the execution of the span.
Example (Annotating a Span)
In the context of tracing, logs are converted into “Span Events.” These events offer structured insights into your application’s activities and provide a timeline of when specific operations occurred.
Each span can include events, which capture specific moments during the execution of a span. In this example, a log message "Hello"
is recorded as an event within the span. Key details of the event include:
Field | Description |
---|---|
name | The name of the event, which corresponds to the logged message (e.g., 'Hello' ). |
attributes | Key-value pairs that provide additional context about the event, such as fiberId and log level. |
time | The timestamp of when the event occurred, shown in a high-precision format. |
droppedAttributesCount | Indicates how many attributes were discarded, if any. In this case, no attributes were dropped. |
Spans can be nested to represent a hierarchy of operations. This allows you to track how different parts of your application relate to one another during execution. The following example demonstrates how to create and manage nested spans.
Example (Nesting Spans in a Trace)
The parent-child relationship is evident in the span output, where the parentId
of the child
span matches the id
of the parent
span. This structure helps track how operations are related within a single trace.
In this tutorial, we’ll guide you through simulating and visualizing traces using a sample instrumented Node.js application. We will use Docker, Prometheus, Grafana, and Tempo to create, collect, and visualize traces.
Let’s understand the tools we’ll be using in simple terms:
-
Docker: Docker allows us to run applications in containers. Think of a container as a lightweight and isolated environment where your application can run consistently, regardless of the host system. It’s a bit like a virtual machine but more efficient.
-
Prometheus: Prometheus is a monitoring and alerting toolkit. It collects metrics and data about your applications and stores them for further analysis. This helps in identifying performance issues and understanding the behavior of your applications.
-
Grafana: Grafana is a visualization and analytics platform. It helps in creating beautiful and interactive dashboards to visualize your application’s data. You can use it to graphically represent metrics collected by Prometheus.
-
Tempo: Tempo is a distributed tracing system that allows you to trace the journey of a request as it flows through your application. It provides insights into how requests are processed and helps in debugging and optimizing your applications.
To get Docker, follow these steps:
-
Visit the Docker website at https://www.docker.com/.
-
Download Docker Desktop for your operating system (Windows or macOS) and install it.
-
After installation, open Docker Desktop, and it will run in the background.
Now, let’s simulate traces using a sample Node.js application. We’ll provide you with the code and guide you on setting up the necessary components.
-
Download Docker Files. Download the required Docker files: docker.zip
-
Set Up docker. Unzip the downloaded file, navigate to the
/docker/local
directory in your terminal or command prompt and run the following command to start the necessary services: -
Simulate Traces. Run the following example code in your Node.js environment. This code simulates a set of tasks and generates traces.
Before proceeding, you’ll need to install additional libraries in addition to the latest version of
effect
. Here are the required libraries: -
Visualize Traces. Now, open your web browser and go to
http://localhost:3000/explore
. You will see a generatedTrace ID
on the web page. Click on it to see the details of the trace.