Span Attribute Sampler
This action is a Service Action that samples traces based on the presence or value of specific attributes within a trace.
This feature is in beta. It may be subject to changes and improvements based on user feedback.
Considerations
Before enabling the Service Name Sampler, please consider the following:
- Only supports traces.
- Sampling is trace-wide: either all spans in a trace are kept, or all are dropped.
- The sampler introduces a delay of up to 30 seconds before traces are exported.
- Traces with durations longer than 30 seconds might not be sampled correctly.
- Using the
json_condition
will cause a higher resource utilization!
Overview
The Span Attribute Sampler is an action designed to sample traces from a specified service based on conditions applied to span attributes. It supports several types of attribute evaluations:
- String Conditions: Check for existence, equality, inequality, substring containment, or regex matching.
- Number Conditions: Compare numeric attribute values using equality, inequality, or threshold comparisons.
- Boolean Conditions: Evaluate the presence of a boolean attribute or check for a specific boolean value.
- JSON Conditions: Validate JSON structure or evaluate nested JSON content using JSONPath expressions.
If any span from the specified service meets the condition, the trace is sampled. Otherwise, the trace is sampled based on the configured fallback sampling ratio.
Using JSON Conditions will cause spikes in performance and resources utilization.
Use Cases
Targeted Debugging
Sample only traces where a specific error flag is set. For example, sample traces from a checkout service where error = true
.
Feature Rollout Monitoring
Retain traces for users or sessions flagged via JSON content. For instance, sample traces for an API service where the JSON payload indicates "user.role": "admin"
.
Route-Specific Analytics
Track performance or usage of specific endpoints by matching against HTTP target attributes.
Production Environment Sampling
Collect traces only from the production environment.
Latest Version Monitoring
Sample only traces from the latest relesed version, while still capturing a representative subset of older versions.
Configuration Options
actionName
actionName
actionName string
: Allows you to attach a meaningful name to the action for convenience.
- This field is
- Odigos does not use or assume any meaning from this field
notes
notes
notes string
: Allows you to attach notes regarding the action for convenience.
- This field is optional
- Odigos does not use or assume any meaning from this field
disabled
disabled
disabled boolean
: Allows you to temporarily disable the action, but keep it saved for future use.
- This field is optional, and defaults to
false
signals *
signals *
signals string[]
: An array with the signals that the processor will act on.
- This field is required
- Supported values:
TRACES
attribute_filters *
attribute_filters *
attribute_filters object[]
: An array of objects representing the filters for the service and attribute filters.
- This field is required
service_name *
service_name *
service_name string
: Specifies the service name to search within the trace (Across all available spans).
- This field is required
attribute_key *
attribute_key *
attribute_key string
: Specifies the attribute key inside of the span.
- This field is required
condition *
condition *
condition object
: An object representing the filters for span attributes filters.
- This field is required
string_condition
string_condition
operation
operation
operation string
: Specifies the operation to run against the attribute.
accepted values for the boolean operation:
- exists : Checks that the attribute is present (and not an empty string).
- equals : String equality comparison with expected_value.
- not_equals : String inequality comparison with expected_value.
- contains : Checks if the attribute contains expected_value as a substring.
- not_contains : Checks if the attribute does not contain expected_value.
- regex : Interprets expected_value as a regular expression (RE2 syntax) and checks for a match within the attribute.
expected_value
expected_value
expected_value string
: The values to test the attribute against.
- Required when operation is set to one of
[equals, not_equals, contains, not_contains, regex]
.
number_condition
number_condition
operation
operation
operation string
: Specifies the operation to run against the attribute.
accepted values for the number condition:
- exists : Checks that the numeric attribute is present (non-null).
- equals : Checks if the attribute equals expected_value.
- not_equals : Checks if the attribute does not equal expected_value.
- greater_than : Checks if attribute > expected_value.
- less_than : Checks if attribute < expected_value.
- greater_than_or_equal : Checks if attribute >= expected_value.
- less_than_or_equal : Checks if attribute <= expected_value.
expected_value
expected_value
expected_value float
: The values to test the attribute against.
- Required when operation is set to one of
[equals, not_equals, greater_than, less_than, greater_than_or_equal, less_than_or_equal]
.
boolean_condition
boolean_condition
operation
operation
operation string
: Specifies the operation to run against the attribute.
accepted values for the boolean operation:
- exists : Checks that the boolean attribute is present.
- equals : Checks if the attribute exactly matches expected_value.
expected_value
expected_value
expected_value boolean
: The values to test the attribute against.
- Required when operation is set to
equals
.
json_condition
json_condition
operation
operation
operation string
: Specifies the operation to run against the attribute.
accepted values for the boolean operation:
- exists : Checks that the attribute is a non-empty string.
- equals : Compares the full JSON string to expected_value.
- not_equals : Verifies the full JSON string is not equal to expected_value.
- is_valid_json : Ensures the string parses as valid JSON.
- is_invalid_json : Ensures the string is not valid JSON.
- jsonpath_exists : Asserts that the specified json_path resolves to a non-empty result.
- key_equals : Compares the value at json_path to expected_value.
- key_not_equals : Verifies the value at json_path is not equal to expected_value.
expected_value
expected_value
expected_value string
: The values to test the attribute against.
- Required when operation is set to one of
[equals, not_equals, jsonpath_exists, key_equals, key_not_equals]
.
json_path
json_path
json_path string
: It is a JSONPath expression used to navigate the JSON structure.
- Required when operation is set to one of
[jsonpath_exists, key_equals, key_not_equals]
.
fallback_sampling_ratio *
fallback_sampling_ratio *
fallback_sampling_ratio float
: Specifies the percentage of traces that don’t meet the service name filter and that you still like to retain.
- This field is required
If any span from the specified service satisfies the attribute condition, the trace is sampled. If no match is found, the trace is sampled based on the fallback ratio.
Examples
Below are several YAML examples that demonstrate different use cases.
Match Error in Checkout Service
Sample traces where the error
attribute (a boolean) equals true
for the checkout-service
.
JSON Nested Attribute Match
For JSON evaluation, the JSON attribute is provided as a string. In this example the rule checks that the JSONPath expression returns a value and that the nested key equals the expected value.
Match HTTP Target Endpoint
Sample traces where the HTTP target endpoint (a string) contains /api/products.
Sample Prod Traces
For example, sample all traces from the staging environment by matching on the env attribute.
Regex Condition (No Match)
The following example shows a regex condition for a version string. If the version does not match the expected pattern, the rule does not match the span. In this case, if the attribute exists but fails the regex test, the trace will not be sampled by this rule (and the fallback ratio applies).
Final Notes
- Multiple Filters: You can define multiple attribute filters across different services and attribute types. These filters are OR-combined; if any filter matches, the trace is sampled.
- Fallback Sampling: When no span satisfies the condition, the trace is sampled based on the
fallback_sampling_ratio
. This allows you to collect a percentage of non-matching traces for broad observability without overwhelming storage costs. - JSON Evaluation: For JSON conditions, the sampler always uses the provided json_path to navigate the JSON structure. Ensure that the attribute value is a JSON-encoded string.
By following these guidelines and examples, you can configure the Span Attribute Sampler for production environments with clarity and precision.