Filelog

WARNING

This source offers a delete_after_read option that can be hazardous. When this option is combined with file globbing, it will delete every file that matches the globbing pattern. Use with caution and care.

---

Supported Platforms

Platform	Metrics	Logs	Traces
Linux		✓
Windows		✓
macOS		✓

Configuration Table

Parameter	Type	Default	Description
file_path*	strings		File or directory paths to tail for logs.
exclude_file_path	strings	""	File or directory paths to exclude.
log_type	string	"file"	A friendly name that will be added to each log entry as an attribute.
multiline_line_start_pattern	string		Regex pattern that matches the beginning of a log entry for handling multiline logs.
multiline_line_end_pattern	string		Regex pattern that matches the end of a log entry, useful for terminating parsing of multiline logs.
encoding	enum	utf-8	The encoding of the file being read. Valid values are nop, utf-8, utf-16le, utf-16be, ascii, and big5.
compression	enum	none	Indicate the compression format of input files. If set accordingly, files will be read using a reader that decompresses the file before scanning its content. Valid values are none or gzip. Requires start_at to be set to beginning. Ensure that your collector has permission to decompress the file.
include_file_name_attribute	bool	true	Whether to add the file name as the attribute log.file.name.
include_file_path_attribute	bool	false	Whether to add the file path as the attribute log.file.path.
include_file_name_resolved	bool	false	Whether to add the file name after symlinks resolution as the attribute log.file.name_resolved.
include_file_path_resolved	bool	false	Whether to add the file path after symlinks resolution as the attribute log.file.path_resolved.
delete_after_read	bool	false	Whether to delete the file(s) after reading. Only valid in combination start_at: beginning.
offset_storage_dir	string	$OIQ_OTEL_COLLECTOR_HOME/storage	The directory where the offset storage file will be created. It is okay if multiple receivers use the same directory. By default, the Bindplane Distro for OpenTelemetry Collector sets $OIQ_OTEL_COLLECTOR_HOME in its runtime.
poll_interval	int	200	The duration of time in milliseconds between filesystem polls.
max_concurrent_files	int	1024	The maximum number of log files from which logs will be read concurrently. If the number of files matched exceeds this number, then files will be processed in batches.
parse_to	string	body	The field that the log will be parsed to. Some exporters handle logs favorably when parsed to attributes over body and vice versa.
start_at	enum	end	Start reading the file from the 'beginning' or 'end'.

*required field

Parsing

For the latest version of the filelog source, parsing capabilities have been removed. To replace this functionality, add a processor to your pipeline.

To parse JSON data from a file, add the Parse JSON Processor.

To parse data from a file using regex, add the Parse with Regex Processor.