Configurable Components
- Cluster Gateway Collector - a collector that runs as a k8s Deployment. It receives the OpenTelemetry data from the Node Collectors, processes it, and exports it to the configured destinations.
- Node Data Collection Collector - a collector that runs as a k8s DaemonSet. It collects / scrape the OpenTelemetry data from the applications running on the nodes and forwards it to the Cluster Gateway Collector.
Configuring the Pipeline
Odigos offers two main options for configuring the pipeline:
When configuring the pipeline, you must use only one of the two methods — either profiles, or directly editing the odigos-config ConfigMap.
Using both methods will result in configurations overwriting each other, potentially leading to unexpected behavior.
1. Using Profiles
To use profiles, you need to use the Odigos CLI Command for Profiles.
This simplifies the setup process and ensures optimized settings for typical use cases.
Sizing Profiles size_s, size_m, size_l are pre-defined configurations designed to simplify pipeline configurations. Each profile specifies the following parameters:
Cluster Gateway Collector:
| Profile | Minimum Replicas | HPA Maximum Replicas | Request CPU (m) | Limit CPU (m) | Request Memory (Mi) | Limit Memory (Mi) |
|---|
size_s | 1 | 5 | 150m | 300m | 300Mi | 300Mi |
size_m | 2 | 8 | 500m | 1000m | 500Mi | 600Mi |
size_l | 3 | 12 | 750m | 1250m | 750Mi | 850Mi |
Node Data Collection Collector:
| Profile | Request Memory (Mi) | Limit Memory (Mi) | Request CPU (m) | Limit CPU (m) |
|---|
size_s | 150Mi | 300Mi | 150m | 300m |
size_m | 250Mi | 500Mi | 250m | 500m |
size_l | 500Mi | 750Mi | 500m | 750m |
2. Directly Editing the odigos-config ConfigMap
For more advanced control over the pipeline, you can directly edit the odigos-config ConfigMap. This method allows you to configure additional collectorGateway settings.
Editing the ConfigMap allows you to fine-tune every aspect of the pipeline to meet custom requirements. Below is an example of how to configure the collectorGateway parameters:
collectorGateway:
# MinReplicas is the number of replicas for the cluster gateway collector deployment.
# It also sets the minReplicas for the HPA to this value.
minReplicas: 1
# MaxReplicas sets the maxReplicas for the HPA to this value.
maxReplicas: 8
# RequestMemoryMiB is the memory request for the cluster gateway collector deployment.
# This value will be embedded in the deployment as a resource request of the form "memory: <value>Mi".
requestMemoryMiB: 500
# LimitMemoryMiB is the memory limit for the cluster gateway collector deployment.
# This value will be embedded in the deployment as a resource limit of the form "memory: <value>Mi".
limitMemoryMiB: 625
# RequestCPUm is the CPU request for the cluster gateway collector deployment.
# This value will be embedded in the deployment as a resource request of the form "cpu: <value>m".
requestCPUm: 500
# LimitCPUm is the CPU limit for the cluster gateway collector deployment.
# This value will be embedded in the deployment as a resource limit of the form "cpu: <value>m".
limitCPUm: 1000
# MemoryLimiterLimitMiB sets the "limit_mib" parameter in the memory limiter configuration for the collector gateway.
# This is the hard limit after which a force garbage collection will be performed.
# Default: 50Mi below the memory request.
memoryLimiterLimitMiB:
# MemoryLimiterSpikeLimitMiB sets the "spike_limit_mib" parameter in the memory limiter configuration.
# This is the diff in MiB between the hard limit and the soft limit.
# Default: 20% of the hard limit (soft limit will be 80% of the hard limit).
memoryLimiterSpikeLimitMiB:
# GoMemLimitMib sets the GOMEMLIMIT environment variable value for the collector gateway deployment.
# This is when the Go runtime will start garbage collection.
# Default: 80% of the hard limit of the memory limiter.
goMemLimitMiB:
collectorNode:
# RequestMemoryMiB is the memory request for the node collector daemonset.
# it will be embedded in the daemonset as a resource request of the form "memory: <value>Mi"
# default value is 250Mi
requestMemoryMiB: 250
# LimitMemoryMiB is the memory limit for the node collector daemonset.
# it will be embedded in the daemonset as a resource limit of the form "memory: <value>Mi"
# default value is 2x the memory request.
limitMemoryMiB: 500
# RequestCPUm is the CPU request for the node collector daemonset.
# it will be embedded in the daemonset as a resource request of the form "cpu: <value>m"
# default value is 250m
requestCPUm: 250
# LimitCPUm is the CPU limit for the node collector daemonset.
# it will be embedded in the daemonset as a resource limit of the form "cpu: <value>m"
# default value is 500m
limitCPUm: 500
# this parameter sets the "limit_mib" parameter in the memory limiter configuration for the node collector.
# it is the hard limit after which a force garbage collection will be performed.
# if not set, it will be 50Mi below the memory request.
memoryLimiterLimitMiB:
# this parameter sets the "spike_limit_mib" parameter in the memory limiter configuration for the node collector.
# note that this is not the processor soft limit, but the diff in Mib between the hard limit and the soft limit.
# if not set, this will be set to 20% of the hard limit (so the soft limit will be 80% of the hard limit).
memoryLimiterSpikeLimitMiB:
# the GOMEMLIMIT environment variable value for the node collector daemonset.
# this is when go runtime will start garbage collection.
# if not specified, it will be set to 80% of the hard limit of the memory limiter.
goMemLimitMiB:
Getting Help
If you have any issues, or require our assistance, please open an issue in GitHub, or reach out to us in Odigos Slack
