Filter and transform data

Filter and transform data in Cortex XSOAR 8 SaaS playbooks.

In Cortex XSOAR, data is extracted and collected from various sources, such as playbook tasks, command results, and fetched incidents, and presented in JSON format. The data can be manipulated by using filters and transformers.

Filters

Filters enable you to extract relevant data which you can use elsewhere in Cortex XSOAR. For example, if an incident has several files with varying file types and extensions, you can filter the files by file extension or file type, and use the filtered files in a detonation playbook. You can filter as many objects as required. Cortex XSOAR automatically calculates the context root to which to filter. You can change the context root as necessary.

Caution

You can change the context data root to filter, but it is not recommended to select a different root, as it affects the filter results. The drop-down list displays the filter root for backward compatibility.

Transformers

Transformers modify or format data to make it suitable for further processing or presentation. For example, you can convert a date in non-Unix format to Unix format. Another example is applying the count transformer, which renders the number of elements. When you have more than one transformer, they apply in the order that they appear. You can reorder them using click-and-drag.

Add filters and transformers in a playbook task

1. Create or edit a playbook task.
2. In the field you want to add a filter or transformer (for example, inputs or outputs), click the curly brackets and then select Filters and Transformers.
3. In the Get field, type or select data you want to filter or transform. For example, EWS.Items.Name.
4. (Optional) To filter the data, do the following.

   1. In the Filter section, click Add filter.

      When adding a filter, the context root to filter is automatically populated.
   2. Select the data you want to filter.

      Note

      If you want to filter by a field name that contains a dot (.), you must escape it using a backslash (\ to prevent the system from interpreting it as a nested path. For example, to filter by the field @odata.type, use @odata\.type.
   3. Select the filter operators.
   4. Add the value.
   5. Click the checkbox to save the filter.
5. (Optional) To apply transformers to the field, click Add transformer.

   1. Click the transformer and select the relevant transformer.

      By default, the transformer is set to To upper case(String). Click it to pick a different transformer, for example to change the date format for when incidents occurred.
   2. Select the transformer operators.
   3. Click the tick box to save.
6. (Optional) To test the filter or transformation click Test and select the investigation or add it manually.

Example: Filter items with an EXE extension

In this example, we want to filter all EWS Item names that have the extension exe.

playbook-context.png

1. From the Filters & transformers window, in the Get field, type EWS.Items.Name to extract all Item names in EWS.

   The context root to filter is EWS,Items.

   

   filter-name.png
2. In the Filter section, click Add filter.
3. In the left-hand side, add Extension to the filter.
4. Select Equals (String) → ignore case.
5. In the right-hand side add exe.

   

   filter-exe.png
6. Click the tick box to save the filter.
7. Click Test.

   You should see Item names are filtered with the extension exe.

Example (advanced): Filter hostname for the last resolved time

In this example, we want to see the LastResolved time only from the demisto.com host name.

This is part of the data where we want to filter:

{
    "IP": [
       {
        "Address": "192.168.10.96",
        "AutoFocus": {
            "Resolutions": [
                {
                    "Hostname": "79463wwfqq,dattolocal.net",
                    "LastResolved": "2022-08-02 04:01:02"
                },
                {
                    "Hostname": "demisto.com",
                    "LastResolved": "2022-09-10 09:47:17"
                },
                {
                    "Hostname": "securesense.call4pchelp.com",
                    "LastResolved": "2022-04-22 11:49:06"
                }
            ]
        }
       },
       {
        "Address":"192.168.10.96",
        "AutoFocus": {
            "Resolutions":[
                {
                    "Hostname":"79463wwfqq,dattolocal.net",
                    "LastResolved":"2022-08-02 04:01:02"
                },
                {
                    "Hostname":"demisto.com",
                    "LastResolved":"2022-09-10 09:47:17"
                },
                {
                    "Hostname":"securesense.call4pchelp.com",
                    "LastResolved":"2022-04-22 11:49:06"
                }
            ]
        }
       }
    ]
}

1. From the Filters & transformers window, in the Get field, type IP.AutoFocus.Resolutions.LastResolve.

   

   playbook-filter-auto.png
2. In the Filter section, click Add filter.

   Cortex XSOAR automatically calculates that the context root to filter is IP.AutoFocus.Resolutions.

   

   playbook-filter-autores.png
3. In the left-hand side, add Hostname to the filter.
4. Select Equals (String) → Ends with
5. In the right-hand side add demisto.com.
6. Click the checkbox to save.

   

   playbook-filter-autofilter.png
7. Click Test.

   

   playbook-filter-autotest.png

Create custom filters and transformers

If you require a filter or transformer that is not provided out-of-the-box, you can create your own by creating a script and then adding to the operators window.

1. Select Scripts New Script.
2. Type a meaningful name for the script, and click Save.
3. To create a filter operator script, do the following:

   1. In the Tags field, add the filter tag.

      If you want a custom transformer that operates on an entire array rather than on each individual item, you need to add the entirelist tag.
   2. In the Arguments section, add the following arguments:

      Argument	Description
      left	Mark as mandatory. This argument defines the left-side value of the transformer operation. In this example, this is the value being checked if it falls within the range specified in the right-side value.
      right	Mark as mandatory. This argument defines the right-side value of the transformer operation. In this example, this is the range to check if the left-side value is in.

   3. Add the script syntax and save.
4. To create a transformer operator script do the following:

   1. In the Tags field, add the transformer tag.
   2. In the Arguments section, add the following arguments:

      Argument	Description
      value	Mark as mandatory. The value to transform. In this example, this is the UNIX epoch timestamp to convert to ISO format.

   3. Add the script syntax and save.
5. Go to the filters and transformers window and select the operator.