Splunk Search API

Description

The Splunk Search API source uses the Splunk Search API to collect past events.

Supported Platforms

Platform	Metrics	Logs	Traces
Linux		✓
Windows		✓
macOS		✓

Prerequisites

• Splunk admin credentials

Use Case

Unlike other sources, the SSAPI source is not built to stream live data. Instead, it collects a finite set of event data and transfers it to a destination, preserving the timestamp from the original Splunk event.

Note: Once the source has started collecting events for a search, allow it to complete unless it is absolutely necessary to stop progress. If a search must be interrupted, run that search again to pick up where progress was interrupted. If you switch to a different search instead, the checkpoint for the search will be wiped from storage.

Configuration Table

Parameter	Type	Default	Description
hostname	string		Splunk search head hostname.
port	string	8089	Splunk instance endpoint port.
auth_mode	enum	"basic"	Authentication mode to use when connecting to the Splunk REST API. Valid values are "basic" and "token".
username	string		Username used to authenticate to the Splunk REST API.
password	string		Password used to authenticate to the Splunk REST API.
auth_token	string		Auth token used to authenticate to the Splunk REST API.
token_type	enum		Type of token used to authenticate to the Splunk REST API. Valid values are "Bearer" and "Splunk".
job_poll_interval	int	5	How many seconds to wait between polling for search job completion.
searches.query	string		Splunk search to run to retrieve the desired events. Queries must start with search and should not contain additional commands, nor any time fields (e.g. earliesttime).
searches.earliest_time	string		Earliest timestamp to collect logs (inclusive).
searches.latest_time	string		Latest timestamp to collect logs (inclusive).
searches.event_batch_size	int	100	Amount of events to query from Splunk for a single request.
enable_tls	bool	true	Whether or not to use TLS.
tls_certificate_path	string		Path to the TLS cert to use for TLS-required connections.
tls_private_key_path	string		Path to the TLS key to use for TLS-required connections.
enable_storage	bool	true	Whether or not to use a storage extension. Should be enabled in all environments.
storage_directory	string	$OIQ_OTEL_COLLECTOR_HOME/storage	The directory where the storage file will be created.

Configuration Instructions

1. Identify the Splunk index to migrate events from. Create a Splunk search to capture the events from that index. This will be the query you pass to the source.

• Example: search index=my_index

• Note: queries must begin with the explicit search command, and must not include additional commands, nor any time fields (e.g. earliesttime)

1. Determine the timeframe you want to migrate events from, and set the ‘Earliest Time’ and ‘Latest Time’ config fields accordingly.

• To migrate events from December 2024, EST (UTC-5):

  • Earliest Time: "2024-12-01T05:00"

  • Latest Time: "2025-01-01T04:59”

• Note: By default, GCL will not accept logs with a timestamp older than 30 days. Contact Google to modify this rule.

1. Repeat steps 1 & 2 for each index you wish to collect from

• This is not a requirement. The receiver can migrate multiple searches at once, but doing one at a time will allow for easier testing and debugging of the source.

1. Configure the rest of the source fields according to your Splunk environment.