PricingSign inRequest a Demo

Install BDOT Collector in AWS ECS EC2

Deploy BDOT Collector on AWS ECS EC2 for cost-effective, scalable collector deployment with full control over underlying infrastructure.

This guide walks you through deploying BDOT Collector on AWS ECS using EC2 launch type. EC2 provides full control over the underlying infrastructure and can be more cost-effective for consistent workloads.

Prerequisites

Before starting, ensure you have:

  • AWS CLI v2.x installed and configured with appropriate permissions

  • Valid AWS account with permissions to create ECS, VPC, and IAM resources

  • Bindplane Server running and accessible (self-hosted or cloud)

  • Collector secret key from your Bindplane Server

  • Basic understanding of AWS ECS, VPC, and container concepts

Quick Deployment with CloudFormation

For a quick deployment, you can use the provided CloudFormation template that creates all the necessary infrastructure automatically. This is the recommended approach for most users.

CloudFormation Template

The following CloudFormation template creates all the required AWS resources for BDOT Collector on ECS EC2:

AWSTemplateFormatVersion: '2010-09-09'
Description: 'BDOT Collector on AWS ECS EC2 with Auto Scaling and VPC'

Parameters:
  CollectorSecretKey:
    Type: String
    Description: BDOT Collector secret key from Bindplane Server
    NoEcho: true
  
  OpampEndpoint:
    Type: String
    Description: OpAMP endpoint URL
    Default: 'wss://app.bindplane.com/v1/opamp'
    AllowedPattern: '^(ws|wss)://.*'
  
  CollectorImage:
    Type: String
    Description: BDOT Collector Docker image
    Default: 'ghcr.io/observiq/bindplane-agent:1.84.0'
  
  Environment:
    Type: String
    Description: Environment name (used for resource naming)
    Default: prod
    AllowedValues: [dev, staging, prod]
  
  InstanceType:
    Type: String
    Description: EC2 instance type for ECS cluster
    Default: t3.medium
    AllowedValues: [t3.small, t3.medium, t3.large, t3.xlarge, m5.large, m5.xlarge]
  
  MinSize:
    Type: Number
    Description: Minimum number of EC2 instances
    Default: 1
    MinValue: 1
    MaxValue: 10
  
  MaxSize:
    Type: Number
    Description: Maximum number of EC2 instances
    Default: 5
    MinValue: 1
    MaxValue: 20
  
  DesiredCapacity:
    Type: Number
    Description: Desired number of EC2 instances
    Default: 2
    MinValue: 1
    MaxValue: 10

Resources:
  # VPC and Networking
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-vpc'

  # Internet Gateway
  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-igw'

  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  # Public Subnets
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [0, !GetAZs '']
      CidrBlock: 10.0.1.0/24
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-public-1a'

  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [1, !GetAZs '']
      CidrBlock: 10.0.2.0/24
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-public-1b'

  # Route Tables
  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-public-rt'

  DefaultPublicRoute:
    Type: AWS::EC2::Route
    DependsOn: InternetGatewayAttachment
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  PublicSubnet1RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnet1

  PublicSubnet2RouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnet2

  # Security Groups
  CollectorSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Security group for BDOT Collector
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 4317
          ToPort: 4317
          CidrIp: 0.0.0.0/0
          Description: OTLP gRPC
        - IpProtocol: tcp
          FromPort: 4318
          ToPort: 4318
          CidrIp: 0.0.0.0/0
          Description: OTLP HTTP
        - IpProtocol: tcp
          FromPort: 13133
          ToPort: 13133
          CidrIp: 0.0.0.0/0
          Description: Health check
        - IpProtocol: tcp
          FromPort: 55679
          ToPort: 55679
          CidrIp: 0.0.0.0/0
          Description: ZPages debugging
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
          Description: SSH access
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-sg'

  # IAM Roles
  TaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-execution-role'

  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-task-role'

  ECSInstanceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: ec2.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-instance-role'

  ECSInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Roles:
        - !Ref ECSInstanceRole

  # CloudWatch Log Group
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub '/ecs/${Environment}-bindplane-collector'
      RetentionInDays: 30

  # ECS Cluster
  ECSCluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Sub '${Environment}-bindplane-collector-cluster'

  # Launch Template
  LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    DependsOn: ECSInstanceProfile
    Properties:
      LaunchTemplateName: !Sub '${Environment}-bindplane-collector-lt'
      LaunchTemplateData:
        ImageId: !Sub '{{resolve:ssm:/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id}}'
        InstanceType: !Ref InstanceType
        IamInstanceProfile:
          Arn: !GetAtt ECSInstanceProfile.Arn
        SecurityGroupIds:
          - !Ref CollectorSecurityGroup
        UserData:
          Fn::Base64: !Sub |
            #!/bin/bash
            echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
            echo ECS_ENABLE_TASK_ENI=true >> /etc/ecs/ecs.config
        BlockDeviceMappings:
          - DeviceName: /dev/xvda
            Ebs:
              VolumeSize: 30
              VolumeType: gp3

  # Auto Scaling Group
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    DependsOn: LaunchTemplate
    Properties:
      AutoScalingGroupName: !Sub '${Environment}-bindplane-collector-asg'
      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplate
        Version: !GetAtt LaunchTemplate.LatestVersionNumber
      MinSize: !Ref MinSize
      MaxSize: !Ref MaxSize
      DesiredCapacity: !Ref DesiredCapacity
      VPCZoneIdentifier:
        - !Ref PublicSubnet1
        - !Ref PublicSubnet2
      HealthCheckType: EC2
      HealthCheckGracePeriod: 300
      Tags:
        - Key: Name
          Value: !Sub '${Environment}-bindplane-collector-instance'
          PropagateAtLaunch: true

  # ECS Task Definition
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Sub '${Environment}-bindplane-collector'
      NetworkMode: host
      RequiresCompatibilities:
        - EC2
      Cpu: 512
      Memory: 1024
      ExecutionRoleArn: !Ref TaskExecutionRole
      TaskRoleArn: !Ref TaskRole
      ContainerDefinitions:
        - Name: bdot-collector
          Image: !Ref CollectorImage
          PortMappings:
            - ContainerPort: 4317
              Protocol: tcp
            - ContainerPort: 4318
              Protocol: tcp
            - ContainerPort: 13133
              Protocol: tcp
            - ContainerPort: 55679
              Protocol: tcp
          Environment:
            - Name: OPAMP_ENDPOINT
              Value: !Ref OpampEndpoint
            - Name: OPAMP_SECRET_KEY
              Value: !Ref CollectorSecretKey
            - Name: OPAMP_LABELS
              Value: !Sub 'environment=${Environment},platform=aws-ecs-ec2'
            - Name: MANAGER_YAML_PATH
              Value: /etc/otel/storage/manager.yaml
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref LogGroup
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: ecs

  # ECS Service
  ECSService:
    Type: AWS::ECS::Service
    Properties:
      ServiceName: !Sub '${Environment}-bindplane-collector-service'
      Cluster: !Ref ECSCluster
      TaskDefinition: !Ref TaskDefinition
      DesiredCount: !Ref DesiredCapacity
      LaunchType: EC2

Outputs:
  VPCId:
    Description: VPC ID
    Value: !Ref VPC
    Export:
      Name: !Sub '${Environment}-bindplane-collector-vpc-id'

  ECSClusterName:
    Description: ECS Cluster Name
    Value: !Ref ECSCluster
    Export:
      Name: !Sub '${Environment}-bindplane-collector-cluster-name'

  ServiceName:
    Description: ECS Service Name
    Value: !Ref ECSService
    Export:
      Name: !Sub '${Environment}-bindplane-collector-service-name'

  TaskDefinitionArn:
    Description: ECS Task Definition ARN
    Value: !Ref TaskDefinition
    Export:
      Name: !Sub '${Environment}-bindplane-collector-task-definition-arn'

  AutoScalingGroupName:
    Description: Auto Scaling Group Name
    Value: !Ref AutoScalingGroup
    Export:
      Name: !Sub '${Environment}-bindplane-collector-asg-name'

Deploy with CloudFormation

Save the template above as bindplane-collector-ecs-ec2.yaml and deploy it:

aws cloudformation create-stack \
  --stack-name bindplane-collector-ecs-ec2 \
  --template-body file://bindplane-collector-ecs-ec2.yaml \
  --parameters \
    ParameterKey=CollectorSecretKey,ParameterValue=YOUR_SECRET_KEY \
    ParameterKey=OpampEndpoint,ParameterValue=wss://app.bindplane.com/v1/opamp \
    ParameterKey=CollectorImage,ParameterValue=ghcr.io/observiq/bindplane-agent:1.84.0 \
    ParameterKey=Environment,ParameterValue=prod \
    ParameterKey=InstanceType,ParameterValue=t3.medium \
    ParameterKey=MinSize,ParameterValue=1 \
    ParameterKey=MaxSize,ParameterValue=5 \
    ParameterKey=DesiredCapacity,ParameterValue=2 \
  --capabilities CAPABILITY_IAM

Architecture Overview

The deployment includes:

  • ECS EC2 Cluster: Managed EC2 instances running BDOT Collector

  • Auto Scaling Group: Manages EC2 instances for the cluster

  • VPC with Public Subnets: Network isolation with internet access

  • Host Networking: Containers use EC2 instance's network interface directly

  • Security Groups: Controlled access to collector ports

  • CloudWatch: Monitoring and logging

  • Auto Scaling: Configurable number of collector instances

Container Architecture

The ECS task runs a single container:

BDOT Collector (ghcr.io/observiq/bindplane-agent:1.84.0)

  • OpenTelemetry collector on ports 4317 (gRPC), 4318 (HTTP), 13133 (health), 55679 (ZPages)

  • Connects to Bindplane Server via OpAMP protocol

  • Automatically receives configurations from Bindplane Server

  • Persistent storage for manager.yaml configuration

Manual Deployment Steps

If you prefer to understand each component or need custom configurations, you can follow the manual deployment steps below.

Step 1: Set Up AWS Infrastructure

Important: Follow these steps in order, as later steps depend on resources created in earlier steps.

1.1 Create VPC and Networking

# Create VPC
VPC_ID=$(aws ec2 create-vpc \
  --cidr-block 10.0.0.0/16 \
  --tag-specifications 'ResourceType=vpc,Tags=[{Key=Name,Value=bindplane-collector-vpc}]' \
  --query 'Vpc.VpcId' --output text)

# Enable DNS hostnames
aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames

# Create Internet Gateway
IGW_ID=$(aws ec2 create-internet-gateway \
  --tag-specifications 'ResourceType=internet-gateway,Tags=[{Key=Name,Value=bindplane-collector-igw}]' \
  --query 'InternetGateway.InternetGatewayId' --output text)

# Attach Internet Gateway to VPC
aws ec2 attach-internet-gateway --vpc-id $VPC_ID --internet-gateway-id $IGW_ID

# Get Availability Zones
AZ1=$(aws ec2 describe-availability-zones --query 'AvailabilityZones[0].ZoneName' --output text)
AZ2=$(aws ec2 describe-availability-zones --query 'AvailabilityZones[1].ZoneName' --output text)

# Create Public Subnets
PUBLIC_SUBNET_1=$(aws ec2 create-subnet \
  --vpc-id $VPC_ID \
  --availability-zone $AZ1 \
  --cidr-block 10.0.1.0/24 \
  --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=bindplane-collector-public-1a}]' \
  --query 'Subnet.SubnetId' --output text)

PUBLIC_SUBNET_2=$(aws ec2 create-subnet \
  --vpc-id $VPC_ID \
  --availability-zone $AZ2 \
  --cidr-block 10.0.2.0/24 \
  --tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=bindplane-collector-public-1b}]' \
  --query 'Subnet.SubnetId' --output text)

# Enable auto-assign public IP
aws ec2 modify-subnet-attribute --subnet-id $PUBLIC_SUBNET_1 --map-public-ip-on-launch
aws ec2 modify-subnet-attribute --subnet-id $PUBLIC_SUBNET_2 --map-public-ip-on-launch

# Create Route Table
ROUTE_TABLE_ID=$(aws ec2 create-route-table \
  --vpc-id $VPC_ID \
  --tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=bindplane-collector-public-rt}]' \
  --query 'RouteTable.RouteTableId' --output text)

# Create default route
aws ec2 create-route \
  --route-table-id $ROUTE_TABLE_ID \
  --destination-cidr-block 0.0.0.0/0 \
  --gateway-id $IGW_ID

# Associate subnets with route table
aws ec2 associate-route-table --subnet-id $PUBLIC_SUBNET_1 --route-table-id $ROUTE_TABLE_ID
aws ec2 associate-route-table --subnet-id $PUBLIC_SUBNET_2 --route-table-id $ROUTE_TABLE_ID

1.2 Create Security Group

# Create Security Group
SECURITY_GROUP_ID=$(aws ec2 create-security-group \
  --group-name bindplane-collector-sg \
  --description "Security group for BDOT Collector" \
  --vpc-id $VPC_ID \
  --query 'GroupId' --output text)

# Allow OTLP gRPC (4317)
aws ec2 authorize-security-group-ingress \
  --group-id $SECURITY_GROUP_ID \
  --protocol tcp \
  --port 4317 \
  --cidr 0.0.0.0/0

# Allow OTLP HTTP (4318)
aws ec2 authorize-security-group-ingress \
  --group-id $SECURITY_GROUP_ID \
  --protocol tcp \
  --port 4318 \
  --cidr 0.0.0.0/0

# Allow Health Check (13133)
aws ec2 authorize-security-group-ingress \
  --group-id $SECURITY_GROUP_ID \
  --protocol tcp \
  --port 13133 \
  --cidr 0.0.0.0/0

# Allow ZPages (55679)
aws ec2 authorize-security-group-ingress \
  --group-id $SECURITY_GROUP_ID \
  --protocol tcp \
  --port 55679 \
  --cidr 0.0.0.0/0

# Allow SSH (22) - optional, for debugging
aws ec2 authorize-security-group-ingress \
  --group-id $SECURITY_GROUP_ID \
  --protocol tcp \
  --port 22 \
  --cidr 0.0.0.0/0

Step 2: Create ECS Resources

2.1 Create IAM Roles

# Create Task Execution Role
cat > task-execution-role-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

TASK_EXECUTION_ROLE_ARN=$(aws iam create-role \
  --role-name bindplane-collector-execution-role \
  --assume-role-policy-document file://task-execution-role-trust-policy.json \
  --query 'Role.Arn' --output text)

# Attach managed policy
aws iam attach-role-policy \
  --role-name bindplane-collector-execution-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

# Create Task Role
cat > task-role-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

TASK_ROLE_ARN=$(aws iam create-role \
  --role-name bindplane-collector-task-role \
  --assume-role-policy-document file://task-role-trust-policy.json \
  --query 'Role.Arn' --output text)

# Create ECS Instance Role
cat > ecs-instance-role-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

ECS_INSTANCE_ROLE_ARN=$(aws iam create-role \
  --role-name bindplane-collector-instance-role \
  --assume-role-policy-document file://ecs-instance-role-trust-policy.json \
  --query 'Role.Arn' --output text)

# Attach managed policy
aws iam attach-role-policy \
  --role-name bindplane-collector-instance-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role

# Create Instance Profile
INSTANCE_PROFILE_ARN=$(aws iam create-instance-profile \
  --instance-profile-name bindplane-collector-instance-profile \
  --query 'InstanceProfile.Arn' --output text)

aws iam add-role-to-instance-profile \
  --instance-profile-name bindplane-collector-instance-profile \
  --role-name bindplane-collector-instance-role

2.2 Create CloudWatch Log Group

# Create Log Group
aws logs create-log-group --log-group-name /ecs/bindplane-collector

2.3 Create ECS Cluster

# Create ECS Cluster
CLUSTER_NAME="bindplane-collector-cluster"
aws ecs create-cluster --cluster-name $CLUSTER_NAME

Step 3: Create Launch Template and Auto Scaling Group

3.1 Get Latest ECS-Optimized AMI

# Get latest ECS-optimized AMI ID
AMI_ID=$(aws ssm get-parameters \
  --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id \
  --query 'Parameters[0].Value' --output text)

3.2 Create Launch Template

# Create Launch Template
cat > launch-template-data.json << EOF
{
  "ImageId": "$AMI_ID",
  "InstanceType": "t3.medium",
  "IamInstanceProfile": {
    "Arn": "$INSTANCE_PROFILE_ARN"
  },
  "SecurityGroupIds": ["$SECURITY_GROUP_ID"],
  "UserData": "$(echo '#!/bin/bash\necho ECS_CLUSTER=bindplane-collector-cluster >> /etc/ecs/ecs.config\necho ECS_ENABLE_TASK_ENI=true >> /etc/ecs/ecs.config' | base64 -w 0)",
  "BlockDeviceMappings": [
    {
      "DeviceName": "/dev/xvda",
      "Ebs": {
        "VolumeSize": 30,
        "VolumeType": "gp3"
      }
    }
  ]
}
EOF

LAUNCH_TEMPLATE_ID=$(aws ec2 create-launch-template \
  --launch-template-name bindplane-collector-lt \
  --launch-template-data file://launch-template-data.json \
  --query 'LaunchTemplate.LaunchTemplateId' --output text)

3.3 Create Auto Scaling Group

# Create Auto Scaling Group
aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name bindplane-collector-asg \
  --launch-template LaunchTemplateId=$LAUNCH_TEMPLATE_ID,Version='$Latest' \
  --min-size 1 \
  --max-size 5 \
  --desired-capacity 2 \
  --vpc-zone-identifier "$PUBLIC_SUBNET_1,$PUBLIC_SUBNET_2" \
  --health-check-type EC2 \
  --health-check-grace-period 300

Step 4: Create Task Definition

# Create Task Definition
cat > task-definition.json << EOF
{
  "family": "bindplane-collector",
  "networkMode": "host",
  "requiresCompatibilities": ["EC2"],
  "cpu": "512",
  "memory": "1024",
  "executionRoleArn": "$TASK_EXECUTION_ROLE_ARN",
  "taskRoleArn": "$TASK_ROLE_ARN",
  "containerDefinitions": [
    {
      "name": "bdot-collector",
      "image": "ghcr.io/observiq/bindplane-agent:1.84.0",
      "portMappings": [
        {
          "containerPort": 4317,
          "protocol": "tcp"
        },
        {
          "containerPort": 4318,
          "protocol": "tcp"
        },
        {
          "containerPort": 13133,
          "protocol": "tcp"
        },
        {
          "containerPort": 55679,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "OPAMP_ENDPOINT",
          "value": "wss://app.bindplane.com/v1/opamp"
        },
        {
          "name": "OPAMP_SECRET_KEY",
          "value": "YOUR_SECRET_KEY"
        },
        {
          "name": "OPAMP_LABELS",
          "value": "environment=prod,platform=aws-ecs-ec2"
        },
        {
          "name": "MANAGER_YAML_PATH",
          "value": "/etc/otel/storage/manager.yaml"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/bindplane-collector",
          "awslogs-region": "$(aws configure get region)",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "healthCheck": {
        "command": [
          "CMD-SHELL",
          "curl -f http://localhost:13133/ || exit 1"
        ],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 60
      }
    }
  ]
}
EOF

TASK_DEFINITION_ARN=$(aws ecs register-task-definition \
  --cli-input-json file://task-definition.json \
  --query 'taskDefinition.taskDefinitionArn' --output text)

Step 5: Create ECS Service

# Create ECS Service
aws ecs create-service \
  --cluster $CLUSTER_NAME \
  --service-name bindplane-collector-service \
  --task-definition $TASK_DEFINITION_ARN \
  --desired-count 2 \
  --launch-type EC2

Configuration and Management

Connecting to Bindplane Server

  1. Get your collector secret key from your Bindplane Server:

    • Navigate to Agents → Install Agent

    • Choose Linux platform

    • Copy the secret-key

  2. Update the OpAMP endpoint in your task definition:

    • For Bindplane Cloud: wss://app.bindplane.com/v1/opamp

    • For self-hosted: ws://your-server:3001/v1/opamp (or wss:// with TLS)

  3. Update the task definition with your secret key:

    # Update the environment variable in task-definition.json
# Replace "YOUR_SECRET_KEY" with your actual secret key

Scaling Collectors

Manual Scaling

# Update desired count
aws ecs update-service \
  --cluster $CLUSTER_NAME \
  --service bindplane-collector-service \
  --desired-count 3

Auto Scaling with Application Auto Scaling

# Register scalable target
aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --resource-id service/$CLUSTER_NAME/bindplane-collector-service \
  --scalable-dimension ecs:service:DesiredCount \
  --min-capacity 1 \
  --max-capacity 10

# Create scaling policy
aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --resource-id service/$CLUSTER_NAME/bindplane-collector-service \
  --scalable-dimension ecs:service:DesiredCount \
  --policy-name collector-cpu-scaling \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 70.0,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ECSServiceAverageCPUUtilization"
    }
  }'

Monitoring and Logging

CloudWatch Logs

# View logs
aws logs tail /ecs/bindplane-collector --follow

# Filter logs by container
aws logs filter-log-events \
  --log-group-name /ecs/bindplane-collector \
  --log-stream-name-prefix ecs/bdot-collector

CloudWatch Metrics

The ECS service automatically sends metrics to CloudWatch:

  • CPU and Memory utilization

  • Task count and health

  • Network I/O

Health Checks

The collector includes health checks on port 13133:

  • Health endpoint: http://localhost:13133/

  • ZPages debugging: http://localhost:55679/

TLS Configuration

For Self-Hosted Bindplane with TLS

If your Bindplane Server uses TLS with a custom CA:

# Create a secret for the CA certificate
aws secretsmanager create-secret \
  --name collector-ca-cert \
  --secret-string file://ca.crt

# Update task role to access the secret
cat > task-role-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:*:*:secret:collector-ca-cert*"
    }
  ]
}
EOF

aws iam put-role-policy \
  --role-name collector-task-role \
  --policy-name SecretsManagerAccess \
  --policy-document file://task-role-policy.json

Then update your task definition to include:

{
  "name": "OPAMP_TLS_CA",
  "value": "/etc/otel/certs/ca.crt"
}

Troubleshooting

Common Issues

Collector Not Appearing in Bindplane

  1. Check OpAMP endpoint: Ensure the endpoint URL is correct

  2. Verify secret key: Make sure the secret key matches your Bindplane Server

  3. Check network connectivity: Ensure the collector can reach the Bindplane Server

  4. Review logs: Check CloudWatch logs for connection errors

# Check service status
aws ecs describe-services \
  --cluster $CLUSTER_NAME \
  --services bindplane-collector-service

# Check task logs
aws logs tail /ecs/bindplane-collector --follow

High CPU/Memory Usage

  1. Scale up resources: Increase CPU/memory in task definition

  2. Scale out: Add more collector instances

  3. Optimize configuration: Review collector configuration for efficiency

Best Practices

  1. Resource Sizing: Start with 512 CPU / 1024 Memory, adjust based on load

  2. Scaling: Use Application Auto Scaling for automatic scaling

  3. Monitoring: Set up CloudWatch alarms for key metrics

  4. Security: Use IAM roles with minimal required permissions

  5. Logging: Enable detailed logging for troubleshooting

  6. Updates: Regularly update collector image versions

Cleanup

To remove all resources created by this guide:

# Delete ECS Service
aws ecs update-service \
  --cluster $CLUSTER_NAME \
  --service bindplane-collector-service \
  --desired-count 0

aws ecs delete-service \
  --cluster $CLUSTER_NAME \
  --service bindplane-collector-service


# Delete Task Definition
aws ecs deregister-task-definition \
  --task-definition $TASK_DEFINITION_ARN

# Delete ECS Cluster
aws ecs delete-cluster --cluster $CLUSTER_NAME

# Delete CloudWatch Log Group
aws logs delete-log-group --log-group-name /ecs/bindplane-collector

# Delete IAM Roles
aws iam detach-role-policy \
  --role-name bindplane-collector-execution-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

aws iam delete-role --role-name bindplane-collector-execution-role
aws iam delete-role --role-name bindplane-collector-task-role

# Delete Security Group
aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID

# Delete Subnets
aws ec2 delete-subnet --subnet-id $PUBLIC_SUBNET_1
aws ec2 delete-subnet --subnet-id $PUBLIC_SUBNET_2

# Delete Route Table
aws ec2 delete-route-table --route-table-id $ROUTE_TABLE_ID

# Detach and Delete Internet Gateway
aws ec2 detach-internet-gateway --vpc-id $VPC_ID --internet-gateway-id $IGW_ID
aws ec2 delete-internet-gateway --internet-gateway-id $IGW_ID

# Delete VPC
aws ec2 delete-vpc --vpc-id $VPC_ID

Next Steps

After successfully deploying your BDOT Collector:

  1. Verify connection in your Bindplane Server UI

  2. Create configurations for your collectors

  3. Set up monitoring and alerting

  4. Configure auto-scaling based on your needs

  5. Review security settings and access controls

PreviousInstall BDOT Collector in AWS ECS FargateNextGoogle Marketplace

Last updated

Was this helpful?