Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.
Cortex XDR

Overview of data ingestion metrics

Manage and troubleshoot Cortex XDR data integrations.

Prerequisite

The data ingestion metrics are calculated in 5-minute aggregation periods and saved to the metrics_source dataset. These metrics measure the amount, size, and rate at which logs are ingested by a data source:

Metric
Description

total_size_bytes

Total size (in bytes) of the logs collected during the aggregation period.

total_size_rate

Average size (in bytes per second) of the logs collected during the aggregation period.

total_event_count

Total number of logs collected during the aggregation period

total_event_rate

Average number (in count per second) of logs collected during the aggregation period.

In the metrics_source dataset, the data ingestion metrics are saved alongside additional fields that describe the data source associated with the metrics. Only entries with ingestion metric values greater than zero are saved in the dataset. Entries with zero values are not saved in this dataset.

In addition, you can create your own custom logic for data ingestion health monitoring by setting up correlation rules that monitor the data ingestion metrics. For more information, see Creating correlation rules to monitor data ingestion health.

The following table describes all the fields in the metrics_source dataset:

Read more...
Field
Type
Description

total_size_bytes

Integer

Total size (in bytes) of the logs collected during the aggregation period.

total_size_rate

Integer

Average size (in bytes per second) of the logs collected during the aggregation period.

total_event_count

Integer

Total number of logs collected during the aggregation period

total_event_rate

Integer

Average number (in count per second) of logs collected during the aggregation period.

data_freshness_max_delay

Float

The highest delay bucket reached during the 5-minute aggregation period for logs from the data source (in seconds). This value represents the upper boundary of coarse predefined buckets (0, 30, 60, 300, 3600, 86400, 604800 seconds) and reflects end-to-end elapsed time (_insert_time − _time), including source device clock drift, buffering, network latency, and intermediate collection hops, not Cortex ingestion processing alone. It should not be interpreted as an exact delay measurement.

data_freshness_median

Float

Estimated median delay (p50) across logs collected during the aggregation period (in seconds), calculated from aggregated delay buckets. Measures end-to-end elapsed time (_insert_time − _time).

data_freshness_ninetieth_percentile

Float

Estimated 90th percentile delay (p90) across logs collected during the aggregation period (in seconds), calculated from aggregated delay buckets. For delays exceeding 1 hour, values are estimated across a wide bucket range (1h to 24h). Measures end-to-end elapsed time (_insert_time − _time).

last_seen

Datetime

Timestamp when the last real log was collected from the data source (grouped by _vendor, _product, _device_id, and _collector_type).

_vendor

String

Vendor of the observing data source.

_product

String

Product name of the observing data source.

_device_id

String

(For firewall devices) Device ID

_log_type

String

(For firewall devices) Log type

_collector_type

String

(Event Metadata) Type of collector that provided the log.

_collector_name

String

(Event Metadata) Name of the collector instance.

_collector_id

String

(Event Metadata) ID of the XDR Collector.

_collector_ip

String

(Event Metadata) IP address of the XDR Collector.

_reporting_device_name

String

(Event Metadata) Host name of the device where the log originated.

_reporting_device_ip

String

(Event Metadata) IP Address of the device where the log originated.

_final_reporting_device_name

String

(Event Metadata) Hostname of the device that the log was extracted from.

_final_reporting_device_ip

String

(Event Metadata) IP of the device that the log was extracted from.

_broker_device_name

String

(Event Metadata) Host name of the Broker VM.

_broker_device_ip

String

(Event Metadata) IP address of the Broker VM.

_broker_device_id

String

(Event Metadata) ID of the Broker VM.

_time

Datetime

Timestamp of the interval.

_insert_timestamp

Datetime

Recorded time of the entry.

Last updated

Was this helpful?