Make sure that you have the necessary AWS Credentials and permissions set up.
AWS credentials are retrieved from the default credential chain.
Setting up AWS credentials with AWS EKS (Elastic Kubernetes Service)
The following command will create a link between your EKS cluster and your IAM account, effectively allowing the EKS cluster to assume IAM roles and access AWS services.
Replace $EKS_CLUSTER_REGION with the region of your EKS cluster and $EKS_CLUSTER_NAME with the name of your EKS cluster (or set them as environment variables).
Replace otel-data with the name of your bucket if you used a different name.
The policy name is odigos-s3 in this example, you can use it or choose a different name. Take note of the policy ARN, you will need it in the next step.
Or use the aws management console to create the policy.
The last step is to configure this IAM policy to be assumed by the EKS cluster. Use the following cli command as template and adjust it according to your setup:
make sure to replace the --attach-policy-arn value with the ARN of the policy you created in the previous step.
The aws s3 exporter supports two formats: otlp_json and otlp_proto.
Each batch of logs / metrics / spans will be written as a single s3 object in the specified bucket.
The content of the object will be the serialized data in the OpenTelemetry Protocol format,
represented as either JSON or Protocol Buffers.
For example, the following JSON string represents a single span in otlp_json format, from Odigos demo application:
S3_BUCKETstring : Bucket Name. The name of the bucket where the data will be stored
This field is required
S3_REGIONstring : Bucket Region. The AWS region where the bucket is located
This field is required
S3_PARTITIONstring : Time granularity of S3 Bucket. Wether a new subdirectory should be created every minute or every hour
This field is required and defaults to minute
S3_MARSHALERstring : Marshaller (Data Format). The format in which the data will be encoded. It can be either otlp_json or otlp_proto. Default is otlp_json