Sampling
Latency Sampler
This action is an Endpoint Action that samples traces based on span attributes values.
This feature is in beta. It may be subject to changes and improvements based on user feedback.
Considerations
Before enabling latency sampler, please note the following:
- Supports only traces.
- Multiple
endpoint_filterscan be configured within the same action. - Adding this action causes a 30-second delay in sending the data.
- Traces with durations exceeding 30 seconds might not be sampled correctly.
Use Cases
Cost Reduction- In large-scale systems, collecting every trace can be expensive. Latency policies allows you to focus on the most critical traces, reducing the overall volume of data sent for analysis. This helps you optimize your costs.
- Removing short traces allows you to concentrate on the more important and interesting long traces, optimizing your system for efficiency without being overwhelmed by the volume of trace data.
Configuration Options
actionName
actionName
actionName
string : Allows you to attach a meaningful name to the action for convenience.- This field is
- Odigos does not use or assume any meaning from this field
notes
notes
notes
string : Allows you to attach notes regarding the action for convenience.- This field is optional
- Odigos does not use or assume any meaning from this field
disabled
disabled
disabled
boolean : Allows you to temporarily disable the action, but keep it saved for future use.- This field is optional, and defaults to
false
signals *
signals *
signals
string[] : An array with the signals that the processor will act on.- This field is required
- Supported values:
TRACES
endpoints_filters *
endpoints_filters *
endpoints_filters
object[] : An array of objects representing the filters for the service and http route.- This field is required
minimum_latency_threshold *
minimum_latency_threshold *
minimum_latency_threshold
string : Specifies the minimum latency in milliseconds; traces with latency below this threshold are ignored.- This field is required
service_name *
service_name *
service_name
string : The rule applies to a specific service name. Only traces originating from this service’s root span will be considered.- This field is required
http_route *
http_route *
http_route
string : The specific HTTP route prefix to match for sampling. Only traces with routes beginning with this prefix will be considered. For instance, configuring /buy will also match /buy/product.- This field is required
fallback_sampling_ratio *
fallback_sampling_ratio *
fallback_sampling_ratio
string : Specifies the percentage of traces that meet the service/http_route filter but fall below the threshold that you still want to retain. For example, if a rule is set for service A and http_route B with a minimum latency threshold of 1 second, you might still want to keep some traces below this threshold. Setting the ratio to 20% ensures that 20% of these traces will be retained.- This field is required
The duration is determined by looking at the earliest start time and latest end time of a trace. Configuring
minimum_latency_threshold for a specified service and http_route will sample any request with a duration exceeding this threshold for that particular service and http_route combination.Otherwise, the trace will be dropped. It is recommended to still keep a portion of these traces using the fallback_sampling_ratio setting. This allows you to retain a specified percentage of traces that fall below the threshold.Basic Example
The following example demonstrates how to sample all traces with a latency greater than 1000ms for thefrontend service and /buy http route. Additionally, it retains 20% of the traces that fall below the 1000ms threshold.
1
Create a YAML file with the following content:
latency-sampler.yaml
2
Apply the action to the cluster: