> For the complete documentation index, see [llms.txt](https://cortex-docs.paloaltonetworks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cortex-docs.paloaltonetworks.com/cortex-cloud-posture-management/cortex-cloud-xql/get-started-with-xql/datasets-and-presets.md). # Datasets and presets Every Cortex Query Language (XQL) dataset query begins by identifying a data source that the query will run against. Each data source has a unique name, and a series of fields. Your query specifies the data source, and then provides stages that identify fields of interest and perform operations against those fields. You can query against either datasets or Presets in a dataset query. XQL supports using different languages for dataset and field names. In addition, the dataset formats supported are dependent on the data retention offerings available in Cortex Cloud according to whether you want to query hot storage (default) or cold storage. For more information, see .
Datasets The standard, built-in data source that is available in every Cortex Cloud instance is the `xdr_data` dataset. This is a very large dataset with many available fields. For more information about this dataset, see [Cortex XQL Schema Reference](https://cortex-docs.paloaltonetworks.com/xql-schema-reference/). Cortex Query Language (XQL) supports using different languages for dataset and field names. In addition, the dataset formats supported are dependent on the data retention offerings available in Cortex Cloud according to whether you want to query hot storage (default) or cold storage. For more information, see [XQL Command Reference Guide](https://cortex-docs.paloaltonetworks.com/xql-command-reference-guide/). This dataset is comprised of both raw Endpoint Detection and Response (EDR) events reported by the Cortex Cloud agent, and of logs from different sources such as third-party logs. To help you investigate events more efficiently, Cortex Cloud also stitches these logs and events together into common schemas called stories. These stories are available using the Cortex Cloud Presets. ### Building queries in XQL When building queries in XQL, keep the following in mind about datasets: * Use the `dataset` keyword to specify a dataset on your query. * Create custom datasets using the `target` stage. * Dataset names can use uppercase characters, but in queries dataset names are always treated as if they are lowercase. In addition, dataset names are supported using different languages, numbers (**`0-9`**), and underscores (**`_`**). Yet, underscores cannot be the first character of the name. * Upon ingestion, all fields are retained even fields with a null value. You can also use XQL to query parsing rules for null values. * Schema changes to datasets may not be reflected in the autocomplete suggestions and definitions as you type in real time the XQL query and can appear with a slight delay. ### Available datasets | Data | Dataset | | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Active Directory via Cloud Identity Engine |

pan\_dss\_raw

Note

To set up this Cloud Identity Engine (previously called Directory Sync Service (DSS)) dataset, you need to set up a Cloud Identity Engine. Otherwise, you will not have a pan\_dss\_raw dataset. For more information, see Set up Cloud Identity Engine.

| | Asset groups | `asset_groups` | |


Provides metadata for asset groups. Use this dataset to retrieve the human-readable group name for use in queries, reports, and dashboards.

| | | Issues table in Cortex XDR |

issues

| | Amazon S3 | | | Authentication logs (subset of xdr\_data) |

Authentication logs, such as Okta: auth\_logs

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | AWS CloudTrail and Amazon CloudWatch | `__raw` | | Azure Event Hub |
  • All logs: MSFT\_Azure\_raw
  • Normalize and enrich audit logs: cloud\_audit\_logs
| | Azure Network Watcher |
  • All logs: MSFT\_Azure\_raw
  • Normalize and enrich flow logs: xdr\_dataset dataset with a preset called network\_story
| | BeyondTrust Privilege Management Cloud | `beyondtrust_privilege_management_raw` | | Box |

Events (admin\_logs)

  • box\_admin\_logs\_raw

Box Shield Alerts

  • box\_shield\_alerts\_raw

Users

  • box\_users\_raw

Groups

  • box\_groups\_raw
| | Checkpoint FW1/VPN1 | `__raw` | | Cisco ASA |

Cisco ASA firewalls or Cisco AnyConnect VPN

  • cisco\_asa\_raw
| | Collector status change audit for collection integrations, custom collectors, and marketplace collectors. | `collection_auditing` | | Corelight Zeek | `corelight_zeek_raw` | | Correlation rule executions | `correlations_auditing` | | Cortex Data Lakes | `xdr_data` | | Cortex XDR Collectors | `panw_xdrc_raw` | | Cortex XDR Host Firewall enforcement events | `host_firewall_events` | | CSV files in shared Windows directory | Custom datasets: Select from pre-existing user-created datasets or add a new dataset. | | Database data (MySQL, PostgreSQL, MSSQL, and Oracle) | `__raw` | | Data ingestion health metrics |

Datasets:

  • data\_ingestion\_health

IMPORTANT: This dataset will not be updated after June 2024. Use the health\_alerts dataset instead.

  • metrics\_source

Presets:

  • data\_ingestion\_metrics (this preset will be deprecated in the next release and replaced by metrics\_view).
  • metrics\_view
| | Dropbox |

Events

  • dropbox\_events\_raw

Member Devices

  • dropbox\_members\_devices\_raw

Users

  • dropbox\_users\_raw

Groups

  • dropbox\_groups\_raw
| | Elasticsearch Filebeat | `__raw` | | Elasticsearch Winlogbeat |

\\_\\_raw

If the vendor and product are not specified in the Winlogbeat profile’s configuration file, Cortex XSIAM creates a default dataset called microsoft\_windows\_raw.

| | Errors related to Parsing Rules and Data Model Rules | `parsing_rules_errors` | | Errors related to event forwarding | `event_forwarding_errors` | | Forcepoint DLP | `forcepoint_dlp_endpoint_raw` | | Fortinet Fortigate | `__raw` | | GlobalProtect access authentication logs |

xdr\_data

To ensure GlobalProtect access authentication logs are sent to Cortex AgentiX, verify that your PANW firewall’s Log Settings for GlobalProtect has the Cortex Data Lake checkbox selected.

| | Google Cloud Platform (GCP) logs |
  • All log types: google\_cloud\_logging\_raw

  • Normalize and enrich audit and flow logs: cloud\_audit\_logs

    • Audit logs: cloud\_audit\_logs
    • Network flow logs: xdr\_dataset dataset with a preset called network\_story
| | Google Kubernetes Engine (GKE) | `__raw` | | Google Workspace |
  • Google Chrome: google\_workspace\_chrome\_raw
  • Admin Console: google\_workspace\_admin\_console\_raw
  • Google Chat: google\_workspace\_chat\_raw
  • Enterprise Groups: google\_workspace\_enterprise\_groups\_raw
  • Login: google\_workspace\_login\_raw
  • Rules: google\_workspace\_rules\_raw
  • Google drive: google\_workspace\_drive\_raw
  • Token: google\_workspace\_token\_raw
  • User Accounts: google\_workspace\_user\_accounts\_raw
  • SAML: google\_workspace\_saml\_raw
  • Alerts: google\_workspace\_alerts\_raw
  • Emails: google\_gmail\_raw
| | Host Inventory and Vulnerability Assessment |

  • Datasets

    • host\_inventory
    • va\_cves
    • va\_endpoints

  • Presets

    • host\_inventory
    • host\_inventory\_accessibility
    • host\_inventory\_applications
    • host\_inventory\_auto\_runs
    • host\_inventory\_cpus
    • host\_inventory\_daemons
    • host\_inventory\_disks
    • host\_inventory\_drivers
    • host\_inventory\_endpoints
    • host\_inventory\_extensions
    • host\_inventory\_groups
    • host\_inventory\_kbs
    • host\_inventory\_mounts
    • host\_inventory\_services
    • host\_inventory\_shares
    • host\_inventory\_users
    • host\_inventory\_volumes
    • host\_inventory\_vss
| | Cases table in Cortex XDR | `cases` | | JSON or text logs from third-party source over HTTP | `__raw` | | Login logs (subset of xdr\_data) |

Login logs, such as WEC: login\_logs

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | Logs from third party source over FTP, FTPS, or SFTP | `__raw` | | Microsoft 365 (email) |
  • msft\_o365\_emails\_raw
  • msft\_o365\_users\_raw
  • msft\_o365\_groups\_raw
  • msft\_o365\_devices\_raw
  • msft\_o365\_mailboxes\_raw
  • msft\_o365\_rules\_raw
  • msft\_o365\_contacts\_raw
| | Microsoft Office 365 |

  • Microsoft Office 365 audit events from Management Activity API:

    • Azure AD Activity Logs: msft\_o365\_azure\_ad\_raw
    • Exchange Online: msft\_o365\_exchange\_online\_raw
    • Sharepoint Online: msft\_o365\_sharepoint\_online\_raw
    • DLP: msft\_o365\_dlp\_raw
    • General: msft\_o365\_general\_raw
  • Microsoft Office 365 emails via Microsoft’s Graph API: msft\_o365\_emails\_raw
  • Azure AD authentication events from Microsoft Graph API: msft\_azure\_ad\_raw
  • Azure AD audit events from Microsoft Graph API: msft\_azure\_ad\_audit\_raw
  • Alerts from Microsoft Graph Security API: msft\_graph\_security\_alerts\_raw
| | NetFlow |
  • ip\_flow\_ip\_flow\_raw (default)
  • When configured, uses the format \\_\\_raw
| | Network Share logs | `__raw` | | Okta | `okta_sso_raw` | | OneLogin |

Log collection

  • onelogin\_events\_raw

Directory

  • onelogin\_users\_raw
  • onelogin\_groups\_raw
  • onelogin\_apps\_raw
| | PANW EDR | `xdr_data` | | PANW IOT Security |

Alerts

  • panw\_iot\_security\_alerts\_raw

Devices

  • panw\_iot\_security\_devices\_raw
| | PANW NGFW |

panw\_ngfw\_\_raw

Supports the following logs.

  • Authentication Logs: panw\_ngfw\_auth\_raw

  • Configuration Logs: panw\_ngfw\_config\_raw

    Prisma Access firewalls do not send configuration logs to the Structured Log Storage (SLS).

  • File Data Logs: panw\_ngfw\_filedata\_raw
  • Global Protect Logs: panw\_ngfw\_globalprotect\_raw
  • Hipmatch Logs: panw\_ngfw\_hipmatch\_raw
  • System Logs: panw\_ngfw\_system\_raw
  • Threat Logs: panw\_ngfw\_threat\_raw
  • Traffic Logs: panw\_ngfw\_traffic\_raw
  • \*URL Logs: panw\_ngfw\_url\_raw
  • User ID Logs: panw\_ngfw\_userid\_raw
  • Tunnel Logs: panw\_ngfw\_tunnel\_raw
  • Configuration Logs: panw\_ngfw\_config\_raw

\*These datasets use the query field names as described in the Cortex XQL Schema Guide.

| | PingFederate | `ping_identity_pingfederate_raw` | | PingOne for Enterprise | `pingone_sso_raw` | | Playbook runs | `playbook_runs` | | Playbook tasks | `playbook_tasks` | | Prisma Browser | `panw_prisma_access_browser_raw` | | Prisma Cloud | `prisma_cloud_raw` | | Prisma Cloud Compute | `prisma_cloud_compute_raw` | | Proofpoint Targeted Attack Protection | `proofpoint_tap_raw` | | Scripts and commands metrics | `scripts_and_commands_metrics` | | SentinelOne DeepVisibility | `sentinelone_deep_visibility_raw` | | ServiceNow CMDB | A ServiceNow CMDB dataset is created for each table configured for data collection using the format `servicenow_cmdb__raw`. | | Salesforce.com |
  • salesforce\_connectedapplication\_raw
  • salesforce\_permissionset\_raw
  • salesforce\_profile\_raw
  • salesforce\_groupmember\_raw
  • salesforce\_group\_raw
  • salesforce\_user\_raw
  • salesforce\_userrole\_raw
  • salesforce\_document\_raw
  • salesforce\_contentfolder\_raw
  • salesforce\_attachment\_raw
  • salesforce\_contentdistribution\_raw
  • salesforce\_tenantsecuritylogin\_raw
  • salesforce\_useraccountteammember\_raw
  • salesforce\_tenantsecurityuserperm\_raw
  • salesforce\_account\_raw
  • salesforce\_audit\_raw
  • salesforce\_login\_raw
  • salesforce\_eventlogfile\_raw
| | Syslog/CEF | `__raw` | | USB devices connect and disconnect events reported by the agent |

xdr\_data

  • You can query in XQL for this data and build widgets based on the xdr\_data dataset or using the preset device\_control.
  • To view in an XQL query these events, the Device Configuration of the endpoint profile must be set to Block. Otherwise, the USB events are not captured. The events are also captured when a group of device types are blocked on the endpoints with a permanent or temporary exception in place. For more information, see \[Ingest Connect and Disconnect Events of USB Devices] in Device control.
| | VPN logs (subset of `xdr_data`) |

VPN logs, such as GlobalProtect: vpn\_logs

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | Windows Endpoints using Cortex XDR Forensics Add-on |
  • forensics\_amcache
  • forensics\_application\_resource\_usage
  • forensics\_arp\_cache
  • forensics\_background\_activity\_monitor
  • forensics\_chrome\_history
  • forensics\_cid\_size\_mru
  • forensics\_command\_history
  • forensics\_dns\_cache
  • forensics\_edge\_anaheim\_history
  • forensics\_edge\_spartan\_history
  • forensics\_event\_log
  • forensics\_file\_access
  • forensics\_file\_listing
  • forensics\_firefox\_history
  • forensics\_handles
  • forensics\_hosts\_file
  • forensics\_internet\_explorer\_history
  • forensics\_jumplist
  • forensics\_last\_visited\_pidl\_mru
  • forensics\_log\_me\_in
  • forensics\_net\_sessions
  • forensics\_network
  • forensics\_network\_connectivity\_usage
  • forensics\_network\_data\_usage
  • forensics\_open\_save\_pidl\_mru
  • forensics\_port\_listing
  • forensics\_prefetch
  • forensics\_process\_execution
  • forensics\_process\_listing
  • forensics\_psreadline
  • forensics\_recent\_files
  • forensics\_recentfilecache
  • forensics\_recycle\_bin
  • forensics\_registry
  • forensics\_remote\_access
  • forensics\_seven\_zip\_folder\_history
  • forensics\_shellbags
  • forensics\_shimcache
  • forensics\_team\_viewer
  • forensics\_typed\_paths
  • forensics\_typed\_urls
  • forensics\_user\_access\_logging
  • forensics\_user\_assist
  • forensics\_windows\_activities
  • forensics\_winrar\_arc\_history
  • forensics\_word\_wheel\_query
| | Windows event logs via Cortex XDR Windows agents | `microsoft_windows_raw` | | Windows Event Collector (WEC) |
  • xdr\_data
  • microsoft\_windows\_raw
| | Windows DHCP using Elasticsearch Filebeat | `microsoft_dhcp_raw` | | Windows DNS Debug using Elasticsearch Filebeat |

Raw Data

  • microsoft\_dns\_raw

Normalized Stories

  • xdr\_data with the preset called network\_story.
| | Workday | `workday_workday_raw` | | Zscaler Cloud Firewall |

ZIA

  • Firewall logs: zscaler\_nssfwlog\_raw
  • Web logs: zscalar\_nssweblog\_raw

ZPA

  • zscaler\_zpa\_raw
|
Presets Presets offer groupings of `xdr_data` fields that are useful for analyzing specific areas of network and endpoint activity. All of the fields available for a preset are also available on the larger `xdr_data` dataset, but by using the preset your query can run more efficiently. Presets are sorted at random by the first one million results found. Two of the available presets are stories. These contain information stitched together from Cortex Cloud agent events and log files to form a common schema. They are `authentication_story` and `network_story`. You use the `preset` keyword to specify a dataset in your query.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://cortex-docs.paloaltonetworks.com/cortex-cloud-posture-management/cortex-cloud-xql/get-started-with-xql/datasets-and-presets.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.