Cumulative To Delta

Cumulative to Delta

Converts cumulative sum, histogram, and exponential histogram metrics to delta aggregation temporality. The processor remembers the last value it saw for each metric series and emits the difference on each new data point. Only monotonic sums are converted. Non-monotonic sums, gauges, and metrics that are already delta pass through unchanged. Operates on metrics only.

Supported Telemetry Types

Platform	Metrics	Logs	Traces
Linux	✓
Windows	✓
macOS	✓
AIX	✓
Kubernetes Node	✓
Kubernetes Cluster	✓
Kubernetes Gateway	✓
OpenShift Node	✓
OpenShift Cluster	✓
OpenShift Gateway	✓

Where to Run This Processor

This processor is stateful. It computes each delta by subtracting the previous value it saw for a series, so it is only accurate when every data point for that series reaches the same collector instance. It produces its most useful output when the collector lifecycle is tied to the metric source. Place it as close to the edge as possible.

Placement	Recommendation	Why
Agent on the host or node emitting the metric	Recommended	One collector sees every point for each series. State is complete and restarts of the source and collector are visible to the processor.
Single gateway with a fixed set of sources	Acceptable	Works as long as each source always sends to this one gateway. Use auto or drop for Initial Value so a gateway restart does not emit one large delta per series.
Multiple gateways behind a load balancer	Avoid	A source's points are spread across gateways. Each gateway holds its own partial state and emits deltas computed from the wrong previous value. The result is silently inaccurate data, not an error.

If you must convert in a gateway tier, route each source to exactly one gateway and accept that a gateway restart resets state for every series it tracks. In most environments the simpler fix is to move the processor to the collector configuration and let gateways forward the already converted delta metrics.

Configuration

Include

Parameter	Type	Required	Default	Description
Metric Names	Strings	No	(empty)	Metric names to convert. If empty, all metrics are converted unless excluded. With regexp, each entry is a re2 pattern.
Match Type	Enum: strict, regexp	No	strict	How the metric names are matched. strict requires an exact metric-name match. regexp matches each entry as a regular expression.
Metric Types	Enums: sum, histogram, exponentialhistogram	No	(empty)	Metric types to convert. If empty, all supported types are converted.

Exclude

Parameter	Type	Required	Default	Description
Metric Names	Strings	No	(empty)	Metric names to leave as cumulative. With regexp, each entry is a re2 pattern.
Match Type	Enum: strict, regexp	No	strict	How the metric names are matched. strict requires an exact metric-name match. regexp matches each entry as a regular expression.
Metric Types	Enums: sum, histogram, exponentialhistogram	No	(empty)	Metric types to leave as cumulative.

Advanced

Parameter	Type	Required	Default	Description
Max Staleness	Duration	Yes	1h	Time after which a metric's previous state is forgotten if no new data point arrives. 0 keeps state indefinitely. Example: 1h.
Initial Value	Enum: auto, keep, drop	No	auto	How to handle the first data point of each series. auto drops it unless the start time indicates it is the true first point. keep always emits the first observed value as a delta. drop always discards it.

If a metric matches both Include and Exclude, Exclude takes precedence. If neither Include nor Exclude is set, every monotonic sum, histogram, and exponential histogram is converted.

Examples

Convert all cumulative metrics

With no Include or Exclude settings, every monotonic sum, histogram, and exponential histogram flowing through the pipeline is converted to delta.

apiVersion: bindplane.observiq.com/v1
kind: Processor
metadata:
  name: cumulative-to-delta
spec:
  type: cumulative_to_delta
  parameters: []

Convert only sums matching a name pattern

Convert every cumulative sum whose name starts with system.cpu. to delta. Histograms and metrics outside that prefix remain cumulative.

apiVersion: bindplane.observiq.com/v1
kind: Processor
metadata:
  name: cumulative-to-delta
spec:
  type: cumulative_to_delta
  parameters:
    - name: include_metrics
      value:
        - "^system\\.cpu\\..*$"
    - name: include_match_type
      value: regexp
    - name: include_metric_types
      value:
        - sum

Convert everything except specific metrics

Convert all supported metrics, but leave two named counters and all exponential histograms cumulative. Because the collector is co-located with the metric source, the first observed value is kept.

apiVersion: bindplane.observiq.com/v1
kind: Processor
metadata:
  name: cumulative-to-delta
spec:
  type: cumulative_to_delta
  parameters:
    - name: exclude_metrics
      value:
        - system.network.io
        - system.disk.operations
    - name: exclude_match_type
      value: strict
    - name: exclude_metric_types
      value:
        - exponentialhistogram
    - name: max_staleness
      value: 30m
    - name: initial_value
      value: keep

Configuration Tips

• When you add this processor and inspect it in the processor node view, the converted metrics appear to be deleted. This is expected. The cumulative series is replaced by a delta series, and the new metric is visible on the next downstream processor node.

• Choose Initial Value based on where the processor runs. auto is the safe default. It drops the first point of a series unless the start time shows the counter was just created, so a collector restart does not emit one large delta covering everything the counter accumulated before. Use keep on an agent or sidecar whose lifecycle is tied to the metric source, where the first observation really is the first value. Use drop in a gateway when you must guarantee no delta includes values observed before the collector started.

• Set Max Staleness comfortably above the reporting interval of your slowest metric. If a series goes quiet longer than Max Staleness, its state is forgotten and the next point is treated as a first point again. Setting it to 0 never expires state, which grows memory for every series ever seen.

• With strict match type each entry must be the full metric name. With regexp each entry is a re2 pattern. Anchor patterns (for example ^system\.cpu\.) to avoid unintended partial matches, and escape literal dots.

• Delta temporality is required or preferred by some destinations and rejected by others. Confirm what your destination expects before converting.

Troubleshooting

The first data point of each metric is missing

Symptoms: a converted metric has no value for its first reporting interval after the collector starts, or after it has been quiet for a while.

Solutions:

1. With Initial Value set to auto or drop, the first point of each series is discarded by design because there is no previous value to subtract from. Switch to keep if the collector is co-located with the metric source and the first observation is the true starting value.

2. If a metric disappears and reappears, check Max Staleness. A series that is idle for longer than Max Staleness loses its state and its next point is treated as a first point.

Deltas are wrong, negative, or spike unexpectedly

Symptoms: converted values do not match the change in the cumulative counter, or a single very large delta appears.

Solutions:

1. Confirm every data point for a series reaches the same collector. This is almost always a multi-gateway deployment. Move the processor to the agent, or pin each source to one gateway. See Where to Run This Processor.

2. A large spike immediately after a collector restart usually means Initial Value is keep. Switch to auto so the first observation after a restart is dropped.

3. A cumulative counter that resets on the source side (for example the process restarted) causes the processor to drop the data point where the decrease is detected rather than emit a negative delta. Deltas resume on the following point. A recurring gap of one point is expected whenever the source restarts.

A metric was not converted

Symptoms: a metric still arrives at the destination with cumulative temporality.

Solutions:

1. Only monotonic sums, histograms, and exponential histograms are converted. Gauges and non-monotonic sums are never converted and pass through unchanged.

2. If the metric matches an Exclude rule it is left cumulative even when it also matches Include. Exclude takes precedence.

3. If Include Metric Names is set, only listed metrics are converted. With strict match type the name must match exactly, including casing and the full dotted name. Switch to regexp if you need pattern matching.

4. If Include Metric Types is set, only the listed types are converted. Add the missing type or clear the list to convert all supported types.

The processor appears to delete metrics in the preview

Symptoms: the processor node view shows the converted metrics as removed.

Solutions:

1. This is how the conversion is displayed. The cumulative series is removed and a delta series is added. Select the next downstream processor node to see the converted metric.

Standalone Processor

apiVersion: bindplane.observiq.com/v1
kind: Processor
metadata:
  name: cumulative-to-delta
spec:
  type: cumulative_to_delta
  parameters:
    - name: include_metric_types
      value:
        - sum
    - name: max_staleness
      value: 1h
    - name: initial_value
      value: auto

Related Resources

• Cumulative to Delta Processor — OpenTelemetry Collector Contrib, the upstream processor this resource configures.

• Statefulness warning — OpenTelemetry Collector standard warnings

• re2 regular-expression syntax