Skip to main content

​
Getting Started

firehose Odigos sends logs only through the existing OpenTelemetry CloudWatch Logs exporter: Odigos gateway → CloudWatch log group → subscription filter → Amazon Data Firehose → downstream storage. This destination requires an AWS account and a preconfigured subscription. It does not send OTLP to Firehose or call Firehose producer APIs directly. Metrics, traces, and profiles are not supported. A log’s trace and span IDs remain log fields; they are not exported spans. AWS setup
  1. Create a fixed CloudWatch log group in the same region as your Firehose stream.
  2. Create an active Firehose delivery stream with Direct PUT as its source and configure a downstream destination, such as Amazon S3. Enable CloudWatch Logs decompression and message extraction to obtain the structured JSON log messages as newline-delimited JSON. Do not enable additional Firehose compression on the already gzip-compressed CloudWatch subscription records unless your downstream reader explicitly supports it.
  3. Grant the CloudWatch subscription role firehose:PutRecord on that delivery stream. Trust logs.amazonaws.com and constrain aws:SourceArn to your account and source log group. Grant the separate Firehose service role access to downstream storage.
  4. Create a CloudWatch subscription filter with an empty filter pattern (all logs), the Firehose delivery stream ARN, and that subscription role ARN. Confirm the subscription is active before enabling the Odigos destination. Odigos does not create or validate this subscription.
  5. Set the source log group, log stream, and region in Odigos. The producer uses the AWS SDK credential chain. On EKS, configure IRSA or EKS Pod Identity for the gateway service account; on VMs, use an instance role or configured AWS credentials. The producer requires logs:PutLogEvents and logs:CreateLogStream on the source group. Precreating the log group avoids the need for logs:CreateLogGroup. An optional producer role ARN requires sts:AssumeRole and an appropriate trust policy; its external ID is separate from any Firehose credential.
Follow AWS’s CloudWatch-to-Firehose setup and Firehose decompression/message extraction. Record format and limits Each log is a JSON message containing body, severity_number, severity_text, attributes, resource, scope, and optional trace_id / span_id. Resource and scope attributes are preserved across multi-resource batches. This is the upstream CloudWatch structured log format, not OTLP JSON. Without Firehose message extraction, downstream data contains CloudWatch envelopes (logEvents[].message) and can include control messages. Configure downstream parsing for the chosen format. Message extraction also removes the CloudWatch event ID and timestamp; disable extraction and parse the envelope if you need those fields. AWS does not support this CloudWatch subscription path to an OpenSearch Service Firehose destination. The upstream exporter batches CloudWatch events, uses a bounded sending queue and retry backoff, and keeps raw_log disabled to preserve metadata. CloudWatch subscription records use gzip; Firehose handles decompression. Delivery is at least once, so downstream consumers must tolerate duplicates. CloudWatch rejects events older than 14 days or beyond the group’s retention window, or more than two hours in the future; the exporter warns about rejected events and does not replay the accepted subset. Oversized JSON messages are truncated at the upstream exporter’s default 256 KiB payload cap; truncated JSON may require special downstream handling. Keep ordinary log records below that cap. Verify downstream delivery Generate ordinary application logs with a unique marker, wait for Firehose’s configured buffering interval, then search/read the marker in the final downstream destination. For S3, inspect the delivered object and query it with a matching Athena JSON schema. Firehose ingestion metrics and a successful CloudWatch write alone do not prove downstream arrival. Check CloudWatch subscription delivery errors and Firehose delivery errors if the marker is absent.

​
Configuring Destination Fields

❌ Traces ❌ Metrics ✅ Logs ❌ Profiles
  • AWS_CLOUDWATCH_LOG_GROUP_NAME string : Source CloudWatch Log Group. Create a subscription filter from this fixed log group to an active Firehose delivery stream before sending logs. Odigos does not create the subscription or delivery stream.
    • This field is required
  • AWS_CLOUDWATCH_LOG_STREAM_NAME string : Source CloudWatch Log Stream. The CloudWatch log stream that Odigos writes to in the subscribed log group.
    • This field is required and defaults to odigos
  • AWS_CLOUDWATCH_REGION string : AWS Region. AWS region containing the CloudWatch log group and Firehose delivery stream. The gateway uses the AWS SDK credential chain, such as an IAM role for its service account.
    • This field is required
    • Example: us-east-1
  • FIREHOSE_ROLE_ARN string : Producer IAM Role ARN. Optional IAM role assumed by the gateway to write to CloudWatch Logs. This is separate from the CloudWatch subscription and Firehose delivery roles.
    • This field is optional
  • FIREHOSE_EXTERNAL_ID string : Producer Role External ID. External ID required by the producer role trust policy when assuming a cross-account role.
    • This field is optional

​
Adding Destination to Odigos

There are two primary methods for configuring destinations in Odigos:
Using the UI
1
Use the Odigos CLI to access the UI
2
Click on Add Destination, select Amazon Data Firehose (via CloudWatch Logs) and follow the on-screen instructions
Using Kubernetes manifests
1
Save the YAML below to a file (e.g. firehose.yaml)
2
Apply the YAML using kubectl