> For the complete documentation index, see [llms.txt](https://cortex-docs.paloaltonetworks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cortex-docs.paloaltonetworks.com/cortex-xdr-3.x/cortex-xdr-3.x-documentation/cortex-xdr-xql/get-started-with-xql/datasets-and-presets.md). # Datasets and presets Every Cortex Query Language (XQL) dataset query begins by identifying a data source that the query will run against. Each data source has a unique name, and a series of fields. Your query specifies the data source, and then provides stages that identify fields of interest and perform operations against those fields. You can query against either datasets or [Presets](#UUID-16537dbc-6a65-f773-0950-191198addae0_presets) in a dataset query. XQL supports using different languages for dataset and field names. In addition, the dataset formats supported are dependent on the data retention offerings available in Cortex XDR according to whether you want to query hot storage (default) or cold storage. For more information, see [XQL Language Structure](/cortex-xdr-3.x/cortex-xdr-3.x-documentation/cortex-xdr-xql/get-started-with-xql/xql-language-structure.md).
Datasets The standard, built-in data source that is available in every Cortex XDR instance is the `xdr_data` dataset. This is a very large dataset with many available fields. For more information about this dataset, see [Cortex XDR XQL Schema Reference](https://docs-cortex.paloaltonetworks.com/r/Cortex-XQL-Schema-Reference-Guide/Introduction). Cortex Query Language (XQL) supports using different languages for dataset and field names. In addition, the dataset formats supported are dependent on the data retention offerings available in Cortex XDR according to whether you want to query hot storage (default) or cold storage. For more information, see [XQL Language Structure](/cortex-xdr-3.x/cortex-xdr-3.x-documentation/cortex-xdr-xql/get-started-with-xql/xql-language-structure.md). This dataset is comprised of both raw Endpoint Detection and Response (EDR) events reported by the Cortex XDR agent, and of logs from different sources such as third-party logs. To help you investigate events more efficiently, Cortex XDR also stitches these logs and events together into common schemas called stories. These stories are available using the Cortex XDR [Presets](#UUID-16537dbc-6a65-f773-0950-191198addae0_presets). **Building queries in XQL** When building queries in XQL, keep the following in mind with respect to datasets: * Use the `dataset` keyword to specify a dataset on your query. * Create custom datasets using the [target](/cortex-xdr-3.x/cortex-xdr-3.x-documentation/cortex-xdr-xql/stages/target.md) stage. * Dataset names can use uppercase characters, but in queries dataset names are always treated as if they are lowercase. In addition, dataset names are supported using different languages, numbers (**`0-9`**), and underscores (**`_`**). Yet, underscores cannot be the first character of the name. * Upon ingestion, all fields are retained even fields with a null value. You can also use XQL to query parsing rules for null values. * Schema changes to datasets may not be reflected in the autocomplete suggestions and deļ¬nitions as you type in real time the XQL query and can appear with a slight delay. **Available datasets** Depending on your integrations, you can have the following datasets available for queries: | Data | Dataset | | --------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Active Directory via Cloud Identity Engine |

pan\_dss\_raw

Note

To set up this Cloud Identity Engine (previously called Directory Sync Service (DSS)) dataset, you need to set up a Cloud Identity Engine. Otherwise, you will not have a pan\_dss\_raw dataset. For more information, see Set up Cloud Identity Engine.

| | Alerts table in Cortex XDR |

alerts

Note

| | Amazon S3 | | | Authentication logs (subset of xdr\_data) |

Authentication logs, such as Okta: auth\_logs

Note

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | AWS CloudTrail and Amazon CloudWatch | `__raw` | | Azure Event Hub |
  • All logs: MSFT\_Azure\_raw
  • Normalize and enrich audit logs: cloud\_audit\_logs
| | Azure Network Watcher |
  • All logs: MSFT\_Azure\_raw
  • Normalize and enrich flow logs: xdr\_dataset dataset with a preset called network\_story
| | BeyondTrust Privilege Management Cloud | `beyondtrust_privilege_management_raw` | | Box |

Events (admin\_logs)

  • box\_admin\_logs\_raw

Box Shield Alerts

  • box\_shield\_alerts\_raw

Users

  • box\_users\_raw

Groups

  • box\_groups\_raw
| | Checkpoint FW1/VPN1 | `__raw` | | Cisco ASA |

Cisco ASA firewalls or Cisco AnyConnect VPN

  • cisco\_asa\_raw
| | Collector status change audit for collection integrations, custom collectors, and marketplace collectors. | `collection_auditing` | | Corelight Zeek | `corelight_zeek_raw` | | Correlation rule executions | `correlations_auditing` | | Cortex Data Lakes | `xdr_data` | | Cortex XDR Collectors | `panw_xdrc_raw` | | Cortex XDR Host Firewall enforcement events | `host_firewall_events` | | CSV files in shared Windows directory | Custom datasets: Select from pre-existing user-created datasets or add a new dataset. | | Database data (MySQL, PostgreSQL, MSSQL, and Oracle) | `__raw` | | Data ingestion health metrics |

Datasets:

  • metrics\_source
| | Dropbox |

Events

  • dropbox\_events\_raw

Member Devices

  • dropbox\_members\_devices\_raw

Users

  • dropbox\_users\_raw

Groups

  • dropbox\_groups\_raw
| | Elasticsearch Filebeat | `__raw` | | Elasticsearch Winlogbeat |

\\_\\_raw

Note

If the vendor and product are not specified in the Winlogbeat profileā€™s configuration file, Cortex XDR creates a default dataset called microsoft\_windows\_raw.

| | Errors related to Parsing Rules | `parsing_rules_errors` | | Forcepoint DLP | `forcepoint_dlp_endpoint_raw` | | Fortinet Fortigate | `__raw` | | GlobalProtect access authentication logs |

xdr\_data

Note

To ensure GlobalProtect access authentication logs are sent to Cortex XDR, verify that your PANW firewallā€™s Log Settings for GlobalProtect has the Cortex Data Lake checkbox selected.

| | Google Cloud Platform (GCP) logs |
  • All log types: google\_cloud\_logging\_raw

  • Normalize and enrich audit and flow logs: cloud\_audit\_logs

    • Audit logs: cloud\_audit\_logs
    • Network flow logs: xdr\_dataset dataset with a preset called network\_story
| | Google Kubernetes Engine (GKE) | `__raw` | | Google Workspace |
  • Google Chrome: google\_workspace\_chrome\_raw
  • Admin Console: google\_workspace\_admin\_console\_raw
  • Google Chat: google\_workspace\_chat\_raw
  • Enterprise Groups: google\_workspace\_enterprise\_groups\_raw
  • Login: google\_workspace\_login\_raw
  • Rules: google\_workspace\_rules\_raw
  • Google drive: google\_workspace\_drive\_raw
  • Token: google\_workspace\_token\_raw
  • User Accounts: google\_workspace\_user\_accounts\_raw
  • SAML: google\_workspace\_saml\_raw
  • Alerts: google\_workspace\_alerts\_raw
  • Emails: google\_gmail\_raw
| | Host Inventory and Vulnerability Assessment |

  • Datasets

    • host\_inventory
    • va\_cves
    • va\_endpoints

  • Presets

    • host\_inventory
    • host\_inventory\_accessibility
    • host\_inventory\_applications
    • host\_inventory\_auto\_runs
    • host\_inventory\_cpus
    • host\_inventory\_daemons
    • host\_inventory\_disks
    • host\_inventory\_drivers
    • host\_inventory\_endpoints
    • host\_inventory\_extensions
    • host\_inventory\_groups
    • host\_inventory\_kbs
    • host\_inventory\_mounts
    • host\_inventory\_services
    • host\_inventory\_shares
    • host\_inventory\_users
    • host\_inventory\_volumes
    • host\_inventory\_vss
| | Incidents table in Cortex XDR | `incidents` | | JSON or text logs from third-party source over HTTP | `__raw` | | Login logs (subset of xdr\_data) |

Login logs, such as WEC: login\_logs

Note

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | Logs from third party source over FTP, FTPS, or SFTP | `__raw` | | Microsoft 365 (email) |
  • msft\_o365\_emails\_raw
  • msft\_o365\_users\_raw
  • msft\_o365\_groups\_raw
  • msft\_o365\_devices\_raw
  • msft\_o365\_mailboxes\_raw
  • msft\_o365\_rules\_raw
  • msft\_o365\_contacts\_raw
| | Microsoft Office 365 |

  • Microsoft Office 365 audit events from Management Activity API:

    • Azure AD Activity Logs: msft\_o365\_azure\_ad\_raw
    • Exchange Online: msft\_o365\_exchange\_online\_raw
    • Sharepoint Online: msft\_o365\_sharepoint\_online\_raw
    • DLP: msft\_o365\_dlp\_raw
    • General: msft\_o365\_general\_raw
  • Microsoft Office 365 emails via Microsoftā€™s Graph API: msft\_o365\_emails\_raw
  • Azure AD authentication events from Microsoft Graph API: msft\_azure\_ad\_raw
  • Azure AD audit events from Microsoft Graph API: msft\_azure\_ad\_audit\_raw
  • Alerts from Microsoft Graph Security API: msft\_graph\_security\_alerts\_raw
| | NetFlow |
  • ip\_flow\_ip\_flow\_raw (default)
  • When configured, uses the format \\_\\_raw
| | Network Share logs | `__raw` | | Okta | `okta_sso_raw` | | OneLogin |

Log collection

  • onelogin\_events\_raw

Directory

  • onelogin\_users\_raw
  • onelogin\_groups\_raw
  • onelogin\_apps\_raw
| | PANW EDR | `xdr_data` | | PANW IOT Security |

Alerts

  • panw\_iot\_security\_alerts\_raw

Devices

  • panw\_iot\_security\_devices\_raw
| | PANW NGFW |

panw\_ngfw\_*\_raw

Supports the following logs.

\*These datasets use the query field names as described in the Cortex schema documentation.

| | PingFederate | `ping_identity_pingfederate_raw` | | PingOne for Enterprise | `pingone_sso_raw` | | Prisma Browser | `panw_prisma_access_browser_raw` | | Prisma Cloud | `prisma_cloud_raw` | | Prisma Cloud Compute | `prisma_cloud_compute_raw` | | Proofpoint Targeted Attack Protection | `proofpoint_tap_raw` | | ServiceNow CMDB | A ServiceNow CMDB dataset is created for each table configured for data collection using the format `servicenow_cmdb__raw`. | | Salesforce.com |
  • salesforce\_connectedapplication\_raw
  • salesforce\_permissionset\_raw
  • salesforce\_profile\_raw
  • salesforce\_groupmember\_raw
  • salesforce\_group\_raw
  • salesforce\_user\_raw
  • salesforce\_userrole\_raw
  • salesforce\_document\_raw
  • salesforce\_contentfolder\_raw
  • salesforce\_attachment\_raw
  • salesforce\_contentdistribution\_raw
  • salesforce\_tenantsecuritylogin\_raw
  • salesforce\_useraccountteammember\_raw
  • salesforce\_tenantsecurityuserperm\_raw
  • salesforce\_account\_raw
  • salesforce\_audit\_raw
  • salesforce\_login\_raw
  • salesforce\_eventlogfile\_raw
| | Syslog/CEF | `__raw` | | USB devices connect and disconnect events reported by the agent |

xdr\_data

Note

  • You can query in XQL for this data and build widgets based on the xdr\_data dataset or using the preset device\_control.
  • To view in an XQL query these events, the Device Configuration of the endpoint profile must be set to Block. Otherwise, the USB events are not captured. The events are also captured when a group of device types are blocked on the endpoints with a permanent or temporary exception in place. For more information, see \[Ingest Connect and Disconnect Events of USB Devices] in Device control.
| | VPN logs (subset of `xdr_data`) |

VPN logs, such as GlobalProtect: vpn\_logs

Note

The fields contained in this dataset are a subset of the fields in the xdr\_data dataset.

| | Windows Endpoints using Cortex XDR Forensics Add-on |
  • forensics\_amcache
  • forensics\_application\_resource\_usage
  • forensics\_arp\_cache
  • forensics\_background\_activity\_monitor
  • forensics\_chrome\_history
  • forensics\_cid\_size\_mru
  • forensics\_command\_history
  • forensics\_dns\_cache
  • forensics\_edge\_anaheim\_history
  • forensics\_edge\_spartan\_history
  • forensics\_event\_log
  • forensics\_file\_access
  • forensics\_file\_listing
  • forensics\_firefox\_history
  • forensics\_handles
  • forensics\_hosts\_file
  • forensics\_internet\_explorer\_history
  • forensics\_jumplist
  • forensics\_last\_visited\_pidl\_mru
  • forensics\_log\_me\_in
  • forensics\_net\_sessions
  • forensics\_network
  • forensics\_network\_connectivity\_usage
  • forensics\_network\_data\_usage
  • forensics\_open\_save\_pidl\_mru
  • forensics\_port\_listing
  • forensics\_prefetch
  • forensics\_process\_execution
  • forensics\_process\_listing
  • forensics\_psreadline
  • forensics\_recent\_files
  • forensics\_recentfilecache
  • forensics\_recycle\_bin
  • forensics\_registry
  • forensics\_remote\_access
  • forensics\_seven\_zip\_folder\_history
  • forensics\_shellbags
  • forensics\_shimcache
  • forensics\_team\_viewer
  • forensics\_typed\_paths
  • forensics\_typed\_urls
  • forensics\_user\_access\_logging
  • forensics\_user\_assist
  • forensics\_windows\_activities
  • forensics\_winrar\_arc\_history
  • forensics\_word\_wheel\_query
| | Windows event logs via Cortex XDR Windows agents | `microsoft_windows_raw` | | Windows Event Collector (WEC) |
  • xdr\_data
  • microsoft\_windows\_raw
| | Windows DHCP using Elasticsearch Filebeat | `microsoft_dhcp_raw` | | Windows DNS Debug using Elasticsearch Filebeat |

Raw Data

  • microsoft\_dns\_raw

Normalized Stories

  • xdr\_data with the preset called network\_story.
| | Workday | `workday_workday_raw` | | Zscaler Cloud Firewall |

ZIA

  • Firewall logs: zscaler\_nssfwlog\_raw
  • Web logs: zscalar\_nssweblog\_raw

ZPA

  • zscaler\_zpa\_raw
|
Presets Presets offer groupings of `xdr_data` fields that are useful for analyzing specific areas of network and endpoint activity. All of the fields available for a preset are also available on the larger `xdr_data` dataset, but by using the preset your query can run more efficiently. Presets are sorted at random by the first one million results found. Two of the available presets are stories. These contain information stitched together from Cortex XDR agent events and log files to form a common schema. They are `authentication_story` and `network_story`. You use the `preset` keyword to specify a dataset in your query.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://cortex-docs.paloaltonetworks.com/cortex-xdr-3.x/cortex-xdr-3.x-documentation/cortex-xdr-xql/get-started-with-xql/datasets-and-presets.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.