Optimize data management in Cortex XSIAM

Learn more about the differences between the Cortex XSIAM data management solutions.

The Cortex Data Lake tier is only available with an active Cortex XSIAM NG SIEM, Cortex XSIAM Enterprise, or Cortex XSIAM Premium license.

Federated Search is not enabled by default. To enable it in your tenant, contact your Customer Support Team.

In modern security operations, balancing the need for comprehensive visibility with the reality of high data volumes and ingestion costs is a constant challenge. Cortex XSIAM addresses this by offering flexible data management solutions designed to align with the specific security value, ingestion requirements, and compliance needs of different log types.

By categorizing data into Analytics or Data Lake tiers, organizations can ensure that high-value logs receive real-time AI/ML processing and detection while supplementary logs are stored cost-effectively. Furthermore, Cortex XSIAM provides Federated Search, a query mechanism designed to provide unified access to distributed data sources without requiring pre-ingestion or centralization. This capability enables you to query data in place, significantly reducing the complexity and operational costs associated with the ingestion process and long-term data retention.

Comparison of data management solutions

The following table breaks down the differences between the internal ingestion tiers and the federated query mechanism to help you determine the best approach for your data.

Point of Comparison	Analytics Tier (Regular cost ingestion)	Data Lake Tier (Cost-Optimized ingestion)	Federated Search (Query mechanism with no ingestion)
Primary Goal	Detection and Response	Compliance, Investigation, and Threat Hunting	Compliance and Historical Search
Best For	High-value security logs needed for real-time AI/ML detection.	High data volume, but low value in terms of real-time security, needed for compliance or as investigation supporting data	Massive historical archives or data you want to manage in your own cloud storage (AWS/GCP/Azure)
Data Location	Ingested and stored in Cortex XSIAM	Ingested and stored in Cortex XSIAM	Not Ingested. Stays in your Azure Blob/S3/GCS buckets.
AI/ML Analytics	Full AI/ML analytics support	None	None
Correlation Rules	Full support	Full Support (consumes Compute Units)	None (Search only)
Usage Model	Usage included (subject to limits)	Consumes Compute Units	Consumes Compute Units
Key Restriction	None	Cannot store PANW Firewall logs in Data Lake Tier No Cortex Data Model (XDM) data modeling No OOTB Analytics	Queries take longer to return results as dependent on external cloud Supports specific formats and structure Search only
Decision Criteria	1. Real time AI/ML detection 2. Heavy query and access usage	1. High data volume, but low value in terms of real-time security; you rarely find threats solely in these, but you need them for context. 2. Data is required primarily for compliance or hunting. You need to keep it for 12 months for auditors, but you only search it occasionally 3. Supports correlation rules. You can write XQL rules to trigger alerts on this data, even though it doesn't get the full AI/ML treatment.	1. No ingestion required, data remains in its original location 2. Strict data sovereignty or ownership 3. This tier is Search Only. You cannot run, for example, correlation rules, scheduled queries, widgets, and dashboards, on Federated Search data. It is strictly for ad-hoc investigations via the XQL query builder.
Examples	Any of the primary data sources like Firewall, Identity, Endpoint, Cloud audit log, and network logs (such as, Cloud Flow, WAF, Load Balancer, and NetFlow) for critical or sensitive environments.	Non-critical Application logs and high-volume Network logs (Cloud Flow, WAF, Load Balancer, raw NetFlow) from non-critical environments. Note If a data source is heavily used for real-time Detection and Response (whether out-of-the-box Cortex XSIAM detections or user-defined rules), we recommend ingesting it via the Analytics tier, which includes all real-time detection capabilities.	Given they follow the supported formats (CSV, Parquet, JSONL) and Hive structure: 1. Raw application logs 2. CDN Logs, such as Akamai and Cloudflare: High-volume edge traffic logs stored in S3/Blob storage. 3. Database Audit Logs, such as RDS and SQL Audit: Compliance-heavy logs that prove "who accessed what table" years ago. 4. Any structured data you export from other systems, such as HR data dumps and old SIEM archives, into a "Hive" folder structure, such as ds=2024-01-01/, in your cloud bucket. Cortex XSIAM can query these directly.