Lookup Fields
Description
The Lookup Fields processor can be used to add matching telemetry fields from a CSV file.
Use
The Lookup Fields processor is used to dynamically add fields based on an existing telemetry value. For each unit of telemetry processed, this processor will grab the value of a field and perform a lookup using a CSV file. If that value exists, the processor will add all other fields associated with that CSV row.
The processor re-reads the CSV file every 60 seconds, allowing updates to be made without restarting the collector.
Supported Types
✓
✓
✓
v1.45.0\+
Configuration
CSV
The CSV file used to perform a lookup operation. The file path should include the .csv extension, and the file must be accessible on the collector host at the specified path.
Context
The context of the lookup operation. The source field must exist here in order to perform a lookup. If a lookup succeeds, all matching fields are also added to this location.
Field
The field to lookup in the specified context. A lookup operation is performed if the name and value of this field match a header and value in the CSV file. Enter only the field name (e.g., host.name or ip), not an OTTL path expression (e.g., resource["host.name"] or body["ip"]).
Lookup results will not appear in Live Preview. To verify the results, roll out the configuration and click the "View Recent Telemetry" button on the agent page, or inspect a downstream processor node. This limitation applies to all processors and connectors that depend on external files or resources.
Example Configurations
Adding Fields Based on Host
In this configuration, the processor will perform a lookup on the host.name value of all incoming metrics. If this field exists on the resource of a metric and the specified CSV file contains a matching value with that header, all other fields in that CSV row will be added to the metric. For example, in this particular configuration, if a metric has a value of MacBook-Pro-4.local for host.name, the corresponding values for region and env will automatically be added to the same context.
Web Interface
Example CSV
Enriching Logs Based on IP
In this configuration, the processor performs a lookup on the ip field in the log body. If a log has a body field named ip whose value matches a row in the CSV, all other columns from that row are added to the log body.
Web Interface
Example CSV
Last updated
Was this helpful?