XQL Query best practices

Apply XQL query best practices in Cortex AgentiX.

Cortex AgentiX includes built-in mechanisms for mitigating long-running queries, such as default limits for the maximum number of allowed issues, and for the maximum number of returned rows. Only specified mapped datasets are searched when querying by the Cortex Data Model (XDM) to use system resources and time more efficiently. The following suggestions can help you to streamline your queries:

• Add a smaller limit to queries by using a limit stage.

  To help reduce the Cortex Query Language (XQL) response time, the default results for an XDM query or an XQL dataset query is limited to 1000, when no limit is explicitly stated in the query. This applies to basic queries with no stages except the fields stage. This default limit does not apply to widgets, Correlation Rules, public APIs, saved queries, or scheduled queries, where the limit is a maximum of 1,000,000 results. Queries based on legacy templates are limited to 10,000 results. Adding a smaller limit can greatly reduce the response time.

  Example 90.

  datamodel dataset = microsoft_windows_raw 
  | fields *host* 
  | limit 100

• Use a small time frame for queries by specifying the specific date and time in the Timeframe, such as selecting Relative time and defining Last 30 Minutes, instead of picking the nearest larger option available or defining an extended time period.

• Use filters that exclude data, along with other possible filters.

• Select the specific fields that you would like to see in the query results.

Expected results when querying fields

The following are returned when querying fields:

• If specific fields are stated in the fields stage, those exact fields will be returned.

• If no fields are stated in the query, the xdm_core fieldset will be returned.

• Unmapped fields are treated as NULL. An unmapped field is an xdm field that hasn't been mapped from the relevant datasets using a Data Model Rule.

• By default, the _time system field will be added to all data model queries. Yet, the _time system field will not be added to queries that contain the comp stage.

• For dataset queries, all current system fields will be returned, even if they are not stated in the query.

• For UNION between XDM and dataset, each part of the UNION will return its own fields.

• Each new column in the result set created by the alter stage will be added as the last column. You can specify a different column order by modifying the field order in the fields stage of the query.

• Each new column in the result set created by the comp stage will be added as the last column. Other fields that are not in the group by / calculated column will be removed from the result set, including the core fields and _time system field.

• When no limit is explicitly stated in a datamodel query, a maximum of 1000 results are returned (default). When this limit is applied to results using the limit stage, it will be indicated in the user interface.