Odigos sets up an observability pipeline in your Kubernetes cluster to collect, process and export OpenTelemetry data.

User can configure the following components:

  • Cluster Gateway Collector - a collector that runs as a k8s Deployment. It receives the OpenTelemetry data from the Node Collectors, processes it, and exports it to the configured destinations.
  • Node Data Collection Collector - a collector that runs as a k8s DaemonSet. It collects / scrape the OpenTelemetry data from the applications running on the nodes and forwards it to the Cluster Gateway Collector.

​
Configuring the Pipeline

Odigos offers two main options for configuring the pipeline:

When configuring the pipeline, you must use only one of the two methods—either profiles or directly editing the odigos-config ConfigMap. Using both methods will result in configurations overwriting each other, potentially leading to unexpected behavior.

​
1. Using Profiles

Sizing Profiles size_s, size_m, size_l are pre-defined configurations designed to simplify pipeline configurations. Each profile specifies the following parameters:

Cluster Gateway Collector:

ProfileMinimum ReplicasHPA Maximum ReplicasRequest CPU (m)Limit CPU (m)Request Memory (Mi)Limit Memory (Mi)
size_s15150m300m300Mi300Mi
size_m28500m1000m500Mi600Mi
size_l312750m1250m750Mi850Mi

Node Data Collection Collector:

ProfileRequest Memory (Mi)Limit Memory (Mi)Request CPU (m)Limit CPU (m)
size_s150Mi300Mi150m300m
size_m250Mi500Mi250m500m
size_l500Mi750Mi500m750m

To use profiles, you need to use the Odigos CLI Command for Profiles.
This simplifies the setup process and ensures optimized settings for typical use cases.

​
2. Directly Editing the odigos-config ConfigMap

For more advanced control over the pipeline, you can directly edit the odigos-config ConfigMap. This method allows you to configure additional collectorGateway settings.

Below is an example of how to configure the collectorGateway parameters:

collectorGateway:
  # MinReplicas is the number of replicas for the cluster gateway collector deployment.
  # It also sets the minReplicas for the HPA to this value.
  minReplicas: 1

  # MaxReplicas sets the maxReplicas for the HPA to this value.
  maxReplicas: 8

  # RequestMemoryMiB is the memory request for the cluster gateway collector deployment.
  # This value will be embedded in the deployment as a resource request of the form "memory: <value>Mi".
  requestMemoryMiB: 500

  # LimitMemoryMiB is the memory limit for the cluster gateway collector deployment.
  # This value will be embedded in the deployment as a resource limit of the form "memory: <value>Mi".
  limitMemoryMiB: 625

  # RequestCPUm is the CPU request for the cluster gateway collector deployment.
  # This value will be embedded in the deployment as a resource request of the form "cpu: <value>m".
  requestCPUm: 500

  # LimitCPUm is the CPU limit for the cluster gateway collector deployment.
  # This value will be embedded in the deployment as a resource limit of the form "cpu: <value>m".
  limitCPUm: 1000

  # MemoryLimiterLimitMiB sets the "limit_mib" parameter in the memory limiter configuration for the collector gateway.
  # This is the hard limit after which a force garbage collection will be performed.
  # Default: 50Mi below the memory request.
  memoryLimiterLimitMiB: 

  # MemoryLimiterSpikeLimitMiB sets the "spike_limit_mib" parameter in the memory limiter configuration.
  # This is the diff in MiB between the hard limit and the soft limit.
  # Default: 20% of the hard limit (soft limit will be 80% of the hard limit).
  memoryLimiterSpikeLimitMiB: 

  # GoMemLimitMib sets the GOMEMLIMIT environment variable value for the collector gateway deployment.
  # This is when the Go runtime will start garbage collection.
  # Default: 80% of the hard limit of the memory limiter.
  goMemLimitMiB:

collectorNode:
  # RequestMemoryMiB is the memory request for the node collector daemonset.
  # it will be embedded in the daemonset as a resource request of the form "memory: <value>Mi"
  # default value is 250Mi
  requestMemoryMiB: 250

  # LimitMemoryMiB is the memory limit for the node collector daemonset.
  # it will be embedded in the daemonset as a resource limit of the form "memory: <value>Mi"
  # default value is 2x the memory request.
  limitMemoryMiB: 500

  # RequestCPUm is the CPU request for the node collector daemonset.
  # it will be embedded in the daemonset as a resource request of the form "cpu: <value>m"
  # default value is 250m
  requestCPUm: 250

  # LimitCPUm is the CPU limit for the node collector daemonset.
  # it will be embedded in the daemonset as a resource limit of the form "cpu: <value>m"
  # default value is 500m
  limitCPUm: 500

  # this parameter sets the "limit_mib" parameter in the memory limiter configuration for the node collector.
  # it is the hard limit after which a force garbage collection will be performed.
  # if not set, it will be 50Mi below the memory request.
  memoryLimiterLimitMiB: 

  # this parameter sets the "spike_limit_mib" parameter in the memory limiter configuration for the node collector.
  # note that this is not the processor soft limit, but the diff in Mib between the hard limit and the soft limit.
  # if not set, this will be set to 20% of the hard limit (so the soft limit will be 80% of the hard limit).
  memoryLimiterSpikeLimitMiB: 

  # the GOMEMLIMIT environment variable value for the node collector daemonset.
  # this is when go runtime will start garbage collection.
  # if not specified, it will be set to 80% of the hard limit of the memory limiter.
  goMemLimitMiB:

Editing the ConfigMap allows you to fine-tune every aspect of the pipeline to meet custom requirements.
If you need any help fine-tuning the configurations, you can reach out to the Odigos team for support.

OverviewIntroduction