Skip to main content
This feature is in beta. It may be subject to changes and improvements based on user feedback.

Considerations

Before enabling latency sampler, please note the following:
  • Supports only traces.
  • Multiple endpoint_filters can be configured within the same action.
  • Adding this action causes a 30-second delay in sending the data.
  • Traces with durations exceeding 30 seconds might not be sampled correctly.

Use Cases

Cost Reduction
  • In large-scale systems, collecting every trace can be expensive. Latency policies allows you to focus on the most critical traces, reducing the overall volume of data sent for analysis. This helps you optimize your costs.
System Optimization
  • Removing short traces allows you to concentrate on the more important and interesting long traces, optimizing your system for efficiency without being overwhelmed by the volume of trace data.

Configuration Options

The LatencySampler action is configured using the odigos.io/v1alpha1.Action CRD with the samplers configuration section.
actionName string : Allows you to attach a meaningful name to the action for convenience.
  • This field is optional
  • Odigos does not use or assume any meaning from this field
notes string : Allows you to attach notes regarding the action for convenience.
  • This field is optional
  • Odigos does not use or assume any meaning from this field
disabled boolean : Allows you to temporarily disable the action, but keep it saved for future use.
  • This field is optional, and defaults to false
signals string[] : An array with the signals that the action will operate on.
  • This field is required
  • Supported values: TRACES
samplers object : Configuration for sampling actions.
  • This field is required for this action type
latencySampler object : Configuration for the LatencySampler.
  • This field is required for this action type
endpoints_filters object[] : An array of objects representing the filters for the service and http route.
  • This field is required
minimum_latency_threshold string : Specifies the minimum latency in milliseconds; traces with latency below this threshold are ignored.
  • This field is required
service_name string : The rule applies to a specific service name. Only traces originating from this service’s root span will be considered.
  • This field is required
http_route string : The specific HTTP route prefix to match for sampling. Only traces with routes beginning with this prefix will be considered. For instance, configuring /buy will also match /buy/product.
  • This field is required
fallback_sampling_ratio string : Specifies the percentage of traces that meet the service/http_route filter but fall below the threshold that you still want to retain. For example, if a rule is set for service A and http_route B with a minimum latency threshold of 1 second, you might still want to keep some traces below this threshold. Setting the ratio to 20% ensures that 20% of these traces will be retained.
  • This field is required
The duration is determined by looking at the earliest start time and latest end time of a trace. Configuring minimum_latency_threshold for a specified service and http_route will sample any request with a duration exceeding this threshold for that particular service and http_route combination.Otherwise, the trace will be dropped. It is recommended to still keep a portion of these traces using the fallback_sampling_ratio setting. This allows you to retain a specified percentage of traces that fall below the threshold.

Basic Example

The following example demonstrates how to sample all traces with a latency greater than 1000ms for the frontend service and /buy http route. Additionally, it retains 20% of the traces that fall below the 1000ms threshold using the new Action CRD.
1

Create a YAML file with the following content:
latency-sampler.yaml
apiVersion: odigos.io/v1alpha1
kind: Action
metadata:
  name: example-latencysampler
  namespace: odigos-system
spec:
  actionName: "example-latencysampler-action"
  signals:
    - TRACES
  samplers:
    latencySampler:
      endpoints_filters:
        - service_name: "frontend"
          http_route: "/buy"
          fallback_sampling_ratio: 20
          minimum_latency_threshold: 1000
2

Apply the action to the cluster:
kubectl apply -f latency-sampler.yaml
I