Configurable Components
- Cluster Gateway Collector - a collector that runs as a k8s Deployment. It receives the OpenTelemetry data from the Node Collectors, processes it, and exports it to the configured destinations.
- Node Data Collection Collector - a collector that runs as a k8s DaemonSet. It collects / scrape the OpenTelemetry data from the applications running on the nodes and forwards it to the Cluster Gateway Collector.
Configuring the Pipeline
Odigos offers two main options for configuring the pipeline:
When configuring the pipeline, you must use only one of the two methods — either profiles, or directly editing the odigos-configuration ConfigMap.Using both methods will result in configurations overwriting each other, potentially leading to unexpected behavior.
1. Using Sizing Configuration
To set the sizing config, you need to use the Odigos CLI Command for .
This simplifies the setup process and ensures optimized settings for typical use cases.
Available Sizing config size_s, size_m, size_l are pre-defined configurations designed to simplify pipeline configurations. Each profile specifies the following parameters:
Cluster Gateway Collector:
| Size | Minimum Replicas | HPA Maximum Replicas | Request CPU (m) | Limit CPU (m) | Request Memory (Mi) | Limit Memory (Mi) |
|---|
size_s | 1 | 5 | 300m | 300m | 300Mi | 300Mi |
size_m | 2 | 8 | 1000m | 1000m | 600Mi | 600Mi |
size_l | 3 | 12 | 1250m | 1250m | 850Mi | 850Mi |
size_xl | 5 | 15 | 1500m | 1500m | 2000Mi | 2000Mi |
Node Data Collection Collector:
| Size | Request Memory (Mi) | Limit Memory (Mi) | Request CPU (m) | Limit CPU (m) |
|---|
size_s | 150Mi | 300Mi | 150m | 300m |
size_m | 250Mi | 500Mi | 250m | 500m |
size_l | 500Mi | 750Mi | 500m | 750m |
size_xl | 1024Mi | 1024Mi | 1000m | 1000m |
2. Advanced Pipeline Configuration
For advanced control over the pipeline, you can override the default sizing and fine-tune resource and scaling parameters for the collectorGateway and collectorNode. The way you apply these advanced settings depends on how you installed Odigos:
-
If you installed Odigos using the CLI:
Edit the
odigos-configuration ConfigMap directly in your cluster. This allows you to set advanced options for both collectorGateway and collectorNode (such as custom CPU/memory requests, limits, and replica counts).
-
If you installed Odigos using Helm:
Make your changes in the
values.yaml file or by using the --set flag during installation or upgrade. Do not edit the odigos-configuration ConfigMap directly, as Helm manages this resource and will overwrite manual changes.
This method gives you full control over the pipeline configuration, letting you override any default or profile-based settings to match your specific requirements.
Below is an example of how to configure the collectorGateway and collectorNode parameters:
# HYBRID CONFIGURATION APPROACH:
# By default, all values below are automatically set based on the ResourceSizePreset values.yaml attribute describe above.
# Uncomment and modify any value to override the automatic sizing for that specific parameter.
#
# IMPORTANT CONSTRAINTS:
# 1. minReplicas must be less or equal to maxReplicas
# 2. If you set limitMemoryMiB without requestMemoryMiB, request will equal limit
# 3. If you set requestMemoryMiB without limitMemoryMiB, limit will equal request
# 4. Same logic applies to CPU settings
collectorGateway:
# the memory request for the cluster gateway collector deployment.
# it will be embedded in the deployment as a resource request
# of the form "memory: <value>Mi".
# default value is 500Mi
# If you set only requestMemoryMiB, the limitMemoryMiB will be set to the same value.
# requestMemoryMiB: 625
# the memory limit for the cluster gateway collector deployment.
# it will be embedded in the deployment as a resource limit
# of the form "memory: <value>Mi".
# default value is 625Mi
# If you set only limitMemoryMiB, the requestMemoryMiB will be set to the same value.
# limitMemoryMiB: 625
# the CPU request for the cluster gateway collector deployment.
# it will be embedded in the deployment as a resource request
# of the form "cpu: <value>m".
# default value is 500m
# If you set only requestCPUm, the limitCPUm will be set to the same value.
# requestCPUm: 500
# the CPU limit for the cluster gateway collector deployment.
# it will be embedded in the deployment as a resource limit
# of the form "cpu: <value>m".
# default value is 1000m
# If you set only limitCPUm, the requestCPUm will be set to the same value.
# limitCPUm: 1000
# The number of replicas for the cluster gateway collector deployment.
# Also uses in MinReplicas the HPA config.
# minReplicas: 1
# The maxReplicas in the HPA config.
# maxReplicas: 10
# sets the "limit_mib" parameter in the memory limiter configuration for the collector gateway.
# it is the hard limit after which a force garbage collection will be performed.
# if not set, it will be 50Mi below the memory limit.
# memoryLimiterLimitMiB: 575
# sets the "spike_limit_mib" parameter in the memory limiter configuration for the collector gateway.
# note that this is not the processor soft limit, but the diff in MiB between the hard limit and the soft limit.
# if not specified, this value will be set to 20% of the hard limit (so the soft limit will be 80% of the hard limit).
# memoryLimiterSpikeLimitMiB: 110
# the GOMEMLIMIT environment variable value for the collector gateway deployment.
# this is when go runtime will start garbage collection.
# if not specified, it will be set to 80% of the hard limit of the memory limiter.
# goMemLimitMiB: 460
collectorNode:
# RequestMemoryMiB is the memory request for the node collector daemonset.
# it will be embedded in the daemonset as a resource request of the form "memory: <value>Mi"
# default value is 250Mi
# If you set only requestMemoryMiB, the limitMemoryMiB will be set to the same value.
# requestMemoryMiB: 250
# LimitMemoryMiB is the memory limit for the node collector daemonset.
# it will be embedded in the daemonset as a resource limit of the form "memory: <value>Mi"
# default value is 500Mi
# If you set only limitMemoryMiB, the requestMemoryMiB will be set to the same value.
# limitMemoryMiB: 500
# the CPU request for the node collector daemonset.
# it will be embedded in the daemonset as a resource request
# of the form "cpu: <value>m".
# default value is 250m
# If you set only requestCPUm, the limitCPUm will be set to the same value.
# requestCPUm: 250
# the CPU limit for the node collector daemonset.
# it will be embedded in the daemonset as a resource limit
# of the form "cpu: <value>m".
# default value is 500m
# If you set only limitCPUm, the requestCPUm will be set to the same value.
# limitCPUm: 500
# this parameter sets the "limit_mib" parameter in the memory limiter configuration for the node collector.
# it is the hard limit after which a force garbage collection will be performed.
# if not set, it will be 50Mi below the memory limit.
# memoryLimiterLimitMiB: 450
# this parameter sets the "spike_limit_mib" parameter in the memory limiter configuration for the node collector.
# note that this is not the processor soft limit, but the diff in Mib between the hard limit and the soft limit.
# if not set, this will be set to 20% of the hard limit (so the soft limit will be 80% of the hard limit).
# memoryLimiterSpikeLimitMiB: 90
# the GOMEMLIMIT environment variable value for the node collector daemonset.
# this is when go runtime will start garbage collection.
# if not specified, it will be set to 80% of the hard limit of the memory limiter.
# goMemLimitMiB: 360
Getting Help
If you have any issues, or require our assistance, please open an issue in GitHub, or reach out to us in Odigos Slack
