Agents v. Gateways
Learn when and why to use gateways in your telemetry pipeline, and how to architect them for production environments.
A collector can run as either an agent or a gateway. An agent sits on the same host, container, or node as the workload it monitors. A gateway is a standalone collector that receives telemetry from agents, processes it, and forwards it to your destinations. Both use the same binary – the only difference is where they deploy and how they're configured.
For most production deployments, the agent-gateway pattern is the recommended architecture. This guide covers when gateways are the right choice, when you can skip them, how to size and scale them, and the architecture patterns that fit common deployment scenarios.
When to Use Gateways
Gateways solve three problems that come up in most production environments.
Credential isolation. Without gateways, every server that sends telemetry needs credentials to your destination, whether that's Google SecOps, Splunk, Datadog, or anything else. With gateways, only a small number of gateway nodes hold those credentials. If you have 1,000 servers, you may not want all 1,000 to need API keys for your SIEM.
Network traffic control. Gateways let you funnel outbound traffic through a controlled point in your network. Many environments restrict which machines can make outbound internet requests. Rather than opening direct egress for every host, route telemetry through gateways that sit in a DMZ or a network zone with permitted outbound access.
Offloading processing. Filtering, parsing, sampling, and enrichment all consume CPU and memory, and running that work on the same host as your application can starve your workload. Gateways are dedicated, well-provisioned machines built specifically for this job. Move heavy processing to gateways so your agents stay lightweight.
There are secondary benefits as well. Gateways batch telemetry into fewer, larger API requests, which can help stay within rate limits and reduce overhead on your destinations. They also give you a single, centralized place to change destinations, add processors, or adjust routing without touching every agent in your fleet.
When You Can Skip Gateways
Gateways add infrastructure, and infrastructure isn't free. There are cases where sending telemetry directly from agents to your destination is a reasonable choice:
Small or simple environments. If you have a handful of servers, minimal processing needs, and your agents can reach your destination directly, a gateway may add complexity without meaningful benefit.
Cloud-native sources already flowing to your destination. If you're already ingesting data into your destination through a managed service (for example, a SaaS integration or cloud-native pipeline) and don't need additional processing, filtering, or multi-destination routing, there may not be a reason to insert a gateway into that path.
Infrastructure cost outweighs the benefit. At scale, the cost of running gateway infrastructure can be significant. If your telemetry doesn't require processing, credential isolation, or network control, and your destination can handle direct ingest from your agents, the infrastructure savings may be worth the trade-off.
That said, the problems gateways solve (credential sprawl, egress costs, processing overhead) tend to emerge gradually and become harder to address retroactively. It's worth revisiting the decision as your environment scales.
Setting Up Gateways in Bindplane
Adding gateways to an existing Bindplane deployment requires three steps.
Install gateway collectors. Deploy one or more collectors on dedicated hosts to serve as gateways. Install the BDOT Collector the same way you would for an agent. See the Collector Install documentation for platform-specific steps.
Create a gateway configuration. Create a new configuration with a Bindplane Gateway source, which listens for OTLP traffic on ports 4317 (gRPC) and 4318 (HTTP) by default. Add your destinations and any processors you want to run centrally. This is where you move processing that was previously running on your agents. Roll out this configuration to your gateway collectors.
Point agents to the gateway. Update your agent configurations to replace their current destinations with a Bindplane Gateway destination that points to the gateway's address (or the load balancer in front of your gateway group).
Sizing and Scaling Considerations
For specific resource requirements and throughput tables for both agents and gateways, see the Sizing and Scaling documentation.
When planning your gateway fleet, size for both failure and maintenance. If one gateway goes down or is taken offline for an upgrade, the remaining gateways need enough capacity to handle the full load. Your network architecture also affects how many gateways you need. If your servers are spread across multiple cloud providers or data centers, you will need gateways in each environment. Deploy them close to the sources they serve to minimize latency and reduce egress costs.
If a gateway is struggling under load, adding another node is often more effective than increasing the instance size. Vertical scaling can hit diminishing returns where additional CPU and memory don't translate to higher throughput. That said, vertical scaling can still help when you're working with a fixed number of collectors.
Most gateway operations are stateless, which makes auto-scaling straightforward. Place your gateways behind a load balancer and scale the group based on CPU, memory, and network I/O. For details on health checks, internal collector metrics, and scaling configuration, see the High Availability documentation.
Architecture Patterns
A single gateway is the simplest setup. Agents export to one gateway, which handles processing and forwards telemetry to your destinations. This works for small environments but offers no redundancy.
For production, use two or more gateway nodes behind a load balancer. If a node fails, the load balancer routes traffic to the remaining nodes automatically.
As your environment grows, you may want to split telemetry across multiple gateway groups. Common approaches include segmenting by region or data center (to reduce egress costs and latency), by application or team (to simplify processing rules and enable chargeback), or by telemetry type (to tune scaling for each signal independently). These patterns can be combined. A large enterprise might have gateway groups segmented by region, with separate pipelines per telemetry type within each group.
Additional Considerations
Push vs. Pull Sources on Gateways
In addition to receiving forwarded telemetry from agents, gateways may also collect directly from push-based sources like syslog or firewall logs on the same nodes. However, adding pull-based sources to a gateway group requires more care. With a push-based source, the sender decides which gateway to hit. With a pull-based source, every gateway independently reaches out to collect data. If you add an API-based source to a group of five gateways, each gateway may pull from that API independently, which could result in duplicate data.
Before adding any pull-based source to a gateway cluster, verify that it supports being consumed by multiple collectors simultaneously without duplication.
Publicly Exposed Gateways
If you're collecting telemetry from endpoints that go on and off your corporate network, such as employee laptops, you may need gateways that are publicly accessible. This allows collectors to fail over to an external gateway when they're off-network rather than queuing data locally with no guarantee of delivery.
This introduces standard concerns around securing a public-facing service: authentication (the collector supports mTLS for certificate-based validation), DDoS protection, and appropriate load balancing. Every cloud provider offers tools for this, but specific recommendations will depend on your network and security requirements.
Further Reading
Last updated
Was this helpful?