> For the complete documentation index, see [llms.txt](https://cortex-docs.paloaltonetworks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cortex-docs.paloaltonetworks.com/xsoar-6-administrator-guide/6.12/configure-cortex-xsoar/customize-and-configure-cortex-xsoar/machine-learning/additional-machine-learning-scripts.md). # Additional Machine Learning Scripts The following are additional machine learning scripts: * [DBotFindSimilarIncidents Script](#UUID-1d8abb0f-55dc-d975-9c09-6146f1016b91) * [DBotPredictURLPhishing Script](#UUID-159e4222-7aef-1055-b6b7-725415ec3a00) * [DBotUpdateLogoURLPhishing Script](#UUID-b03f2923-6f2f-1c76-04d5-61cdedec6d6f) ### **DBotFindSimilarIncidents Script** The following describes how to use the machine learning model in order to find incidents that are similar to the one you are investigating. This model does not need any training from the user. The **`DBotFindSimilarIncidents`** script finds past similar incidents based on an incident fields' similarity. It includes an option to also display indicator similarity. The model aims to detect similarity in a text or JSON file, even if the value is different. The script returns a summary of the query and all the past similar incidents and can be used to assist in the investigation of the current incident. You can run the script as part of an incident or inside the Playground (using the **`incidentId`** argument). The \*\*`DBotFindSimilarIncidents`\*\*script can run **`DBotFindSimilarIncidentsByIndicators`** as a subscript. **Execution of the Script** The execution of the script takes place in 4 steps: 1. **Scope of the research:** Fetch only part of the incidents from the instance. 2. **Similarity Metric:** Compute incident similarity based on similarity fields that you provide. 3. **Find similar incidents by indicators:** Compute incidents similarity based on shared indicators (optional). 4. **Output:** Display results according to your settings. **Scope of the Research** You have the ability to select a limited number of incidents on which to run the model. For this purpose, the you can select the following fields: * **fieldExactMatch:** List of incident fields that have to be equal to the current incident fields. This helps reduce the query size. For example, you can use the type of the field if you want to find similarities only on incidents from the same type as your current incident. * **fromDate:** The start date by which to filter incidents. * **toDate:** The end date by which to filter incidents. * **query:** Argument for any additional query. * **limit:** Maximum number of incidents to fetch and execute the model on. **Similarity Metric** This is the core of the script. The model now computes similarity on incidents defined in the Scope of the Research section. You can choose the fields manually or use all the incident fields from the incidents. Manually choosing the fields could lead to more accurate results. * **Manually:** Enter the list of fields on which to compute the similarity according to the type of the field: * **similarTextField:** Value should be textual and comma-separated. (In case of a mapped incident field, there is no need to specify \`incident\` at the beginning). For example, it can be a command line or a URL. * **similarCategoricalField:** Value should be categorical. In this case, we will have a similarity only if it’s an exact match. For example, it can be hostname or IP Address. * **similarJsonField:** Value should be a JSON. For example, it can be xdralerts or any custom grid field. * **Automatically:** * **useAllFields:** Whether to use a predefined set of fields and custom fields to compute similarity. If "True", it will ignore values in **`similarTextField`**, **`similarCategoricalField`**, **`similarJsonField`**. **Find Similar Incidents by Indicators** If the **`includeIndicatorsSimilarity`** argument is set to “True”, the script will call the **`DBotFindSimilarIncidentsByIndicators`** subscript. This subscript returns a list of incidents which share indicators with the current incident. Indicators have a score depending on their rarity, and very common indicators (which appear in a high number of incidents) are excluded. Each similar incident has a score between 0 and 1 depending on how many indicators they share and how rare the indicators are. **Output** The output has 3 parts: * Summary of the run. * Current incident (optional - only if the **`showCurrentIncident`** argument is True). * List of similar incidents. You can configure the output using the following arguments: * **fieldsToDisplay:** List of additional incident fields to display, but which are not taken into account when computing similarity. * **aggregateIncidentsDifferentDate:** Whether to aggregate the exact same rows in the output within different dates. * **showIncidentSimilarityForAllFields:** Similarity score is computed for each field and then aggregated in order to compute the final similarity score of the incidents. Whether to display the similarity score for each of the incident fields. If not, the script will display only the computed final similarity. * **minimumIncidentSimilarity:** Retain incidents with a similarity score that's higher than the **`minimumIncidentSimilarity`**. * **maxIncidentsToDisplay:** The maximum number of incidents to display. The rest of the incidents won’t be shown in the table but will be part of the content and context. **DBotFindSimilarIncidents Example** In this example, we execute the script on a Splunk incident based on a custom similarity using some incident labels and custom fields. Notice that we need to use the field machine name (lowercase, no spaces). **`` `!DBotFindSimilarIncidents similarTextField="incident.labels.host,incident.labels.threat_group,incident.labels.threat_source,details,srcs,incident.labels.threat_match_value,dsts" fieldsToDisplay="cmdline,tactic,technique,hostnames,ipaddress,parentcmdline,filepaths,severity" fieldExactMatch="type" useAllFields="False" showIncidentSimilarityForAllFields="True" showCurrentIncident="True"` ``** In the example, we are looking for similar incidents. The search scope are incidents with the same incident type, and the similarity based on the fields: incident.labels.host,incident.labels.threat\_group,incident.labels.threat\_source,details,srcs,incident.labels.threat\_match\_value,dsts. We will display the following fields in the results: cmdline,tactic,technique,hostnames,ipaddress,parentcmdline,filepaths,severity. The first entry is the summary. Here we see that the script fetched 1051 incidents with the criteria we defined using the **`fieldExactMatch`** argument. These are the candidates used to calculate the similarity. Within those incidents, the script found 3 incidents that have an overall similarity above the threshold of 0.5. The similarity score is computed from the **`details`** and **`srcs`** fields. Other fields provided to compute similarity from **`similarTextField`** cannot be found in the incidents (as the message indicates). ![ml-dbot-summary.png](/files/wT8wTG9zF3G7bRmDyzdi) Since **`showCurrentIncident`**="True”, the script shows the current incident under investigation (with the fields that werer defined). ![ml-dbot-current-incident.png](/files/bxqDjwMc4Y0KXZv6ynJP) The last entry to be returned is the similar incidents found. The first 2 incidents have a similarity score of 1 (exact matching for the **`details`** and **`srcs`** field). The last one has a similarity of 0.82 due to the small difference in the details field value. We can see that the similarity for this field is 0.63. ![ml-dbot-similar-incidents.png](/files/Q5NfbHo3qDJB8jQbVeGj) ### **DBotPredictURLPhishing Script** The following describes how to use the machine learning model for URL Phishing detection. The model is pre-trained and does not need any training from the user. The URL Phishing model ingests data such as the screenshot and HTML of a web page, URL syntax, and domain information, and predicts if the URL is a phishing attack. The verdict can be Malicious, Suspicious, or Benign. The command to use is **`DBotPredictURLPhishing`**. **Arguments** After running the **`DBotPredictURLPhishing`** command with your arguments, the model extracts URLs from the **`emailBody`**, **`emailBodyHTML`**, and **`urls`** arguments. Then the model selects and runs only on the **`maxNumberOfURL`** URLs. The selection of URLs is made in order to give priority to URLs from different domains and URLs for which the domain does not belong to our top 300k domains list (Majestic top domains). By default, if one of the selected URLs belongs to the first 300k Majestic top domains, the model won’t run. This behavior can be changed by setting the **`forceModel`** argument to True. **Inputs** The following are the list of inputs: * **urls:** Space-separated list of URLs. * **emailBody:** The plain text of the email body for which you want to get a prediction. * **emailHTML:** The HTML of the email for which you want to get a prediction. * **forceModel:** Whether to force the model to run if the URL belongs to the whitelist. If True, the model will run in every case. If False, the model will run only if the URL does not belong to the whitelist. * **resetModel:** Whether to reset the model to the model existing in Docker. * **maxNumberOfURL:** The maximum number of extracted URLs on which to run the model. **Outputs** After running the command, Cortex XSOAR returns the following: * **Phishing prediction summary for URLs:** Final verdict for each of the extracted URLs. ![ml-phishing-prediction.png](/files/T61g5QtDqrMqH4vkvVfv) * **Phishing prediction evidence | domain:** Explanation of the verdict for each of the URLs. ![ml-phishing-pred-evid.png](/files/UamRvKKtrxhsA7MnPLDb) In the Phishing prediction evidence section, the following information will appear: * **Domain:** Domain of the URL. * **Is there a login form:** Indicates if there is a login form in the HTML. Usually phishing attacks try to steal credentials from the victim and attackers using a login form to retrieve this information. * **New Domain (less than 6 months ago):** Indicates if the domain is younger than 6 months. New domains tend to be malicious. * **Search engine optimization:** Evaluates the SEO quality of the URL. Malicious domains tend to have a poor SEO. * **Suspicious use of company logo:** Checks if a logo (from our list of top most companies used for phishing) has been fraudulently used. Our predefined list of logos is: Paypal, iInstagram, Gmail, Outlook, Linkedin, Facebook, Ebay, amazon, Google, Microsoft. * **URL severity score:** Probability that the URL is malicious based only on the URL syntax. * A screenshot of the page is displayed with a matched logo (if available). * **DBotPredictURLPhishing.URL:** URL on which the model was run. * **DBotPredictURLPhishing.FinalVerdict:** Final verdict of the URL. **Troubleshooting** * **URL not correct:** If the URL is misspelled or does not exist, the model will display the corresponding error in the **Phishing prediction summary for URLs** section. * **URL blocked by firewall:** If the URL is blocked by your firewall, the model will display the corresponding error in the **Phishing prediction summary for URLs** section. * **Logo appears in legitimate URL:** If a logo from our predefined list of logos appears in a legitimate web page that does not belong to the top majestic domain, it will raise an alert. This is because it might not be a popular legitimate URL that was registered with Google. * **Skip phishing page registered under top Majestic domain:** It can happen that a page is registered under a domain that belongs to our top Majestic domain list. In that case, the URL will be predicted as Benign (but can be malicious). We use this skip whitelist because applying the model for many URLs might cause performance issues. For example, if a phishing page is registered under it will be skipped even if it is malicious. ### **DBotUpdateLogoURLPhishing Script** The following describes how to add, remove, or modify logos within the URL Phishing model. Phishing pages often use logos of well known companies to disguise themselves as legitimate websites. This model searches for these logos within the URL page to detect false usage of these logos. If a logo is found and the domain of the page is not associated with the logo, the model will trigger an alert. To better customize the model, you can add a logo that URL phishing attempt may use to target you. The predefined list of logos is: Paypal, Instagram, Gmail, Outlook, Linkedin, Facebook, Ebay, Amazon, Google, and Microsoft. This script provides you with the ability to add or remove custom logos and associate legitimate domains to the custom logos. You can also print all the logos that are currently loaded inside the model. We recommend not adding more than 5 custom logos to ensure good model performance. The command to use is **`DBotUpdateLogoURLPhishing`**. **Arguments** * **Action**: Action to execute on the model. This is a mandatory argument. Values can be: * AddLogo: Add a custom logo to the model. This action requires the logoImageId and logoName arguments. The new logo can be directly associated with domains using the associatedDomains argument. You can add domains at a later time using the ModifiedDomainForLogo argument. * RemoveLogo: Remove a custom logo from the model given the logo name. Only custom logos can be removed. This action requires the logoName argument. Logo names can be retrieved by displaying all the logos using the DisplayAllLogos argument. * ModifiedDomainForLogo: Modify the list of domains associated with a custom logo. This action requires the associatedDomains and logoName arguments. * DisplayAllLogos: Return a graphic with the names of all the logos used. For the custom logos, also returns the associated domains. * logoImageId: Entry ID in Cortex XSOAR of the uploaded logo. * logoName: Unique name of the logo. It is used to modify or remove a logo after it has been added. * associatedDomains: Comma-separated list of domains that are associated to the logo defined in the logoName argument. It will only be used if the action argument is AddLogo or ModifiedDomainForLogo. The format of the domains should be: **`[domain].[tld]`** For example: **`paloaltonetworks.com`**

Note

The domain name should not contain https, http, or www.

**Example** Display the Logos in the Model Let’s start by displaying all the logos in the model. There are no custom logos yet, so only the default logos in the original model appear. **`!DBotUpdateLogoURLPhishing action=DisplayAllLogos`** ![dbot-updatelogo-all.png](/files/FKGAeoo34M9DOnWUnb4L) Add a Custom Logo Let’s add a new logo and associate some domains to the logo. 1\. Drop the image into the Playground and upload it. ![dbot-updatelogo-upload-file.png](/files/buhxebfH2yJoXMKzJhFH) 2\. Get the Entry ID in Cortex XSOAR of the uploaded logo. ![dbot-updatelogo-entryid.png](/files/BdMf6I3EUAmm6uPMbthQ) 3\. Run the following command using the Entry ID. **`!DBotUpdateLogoURLPhishing logoImageId="4463@ec8a82a9-4c32-4ce8-85e3-6ec360fed132" logoName="palo_alto" action="AddLogo"`** We can associate domains to this logo by using the associatedDomains argument in this query, but let’s do it in a separate action. ![dbot-updatelogo-success.png](/files/oe3VpFD41NSkl4MA6SH5) If we display the logos inside the model now, we can see that our logo (Palo Alto) was added to the model. ![dbot-updatelog-all2.png](/files/B88GOILfcsjzaiJVZZoy) Associate Domains to Our Custom Logo We can associate some domains to our custom logo. Let’s associate paloaltonetworks.com to our logo, so in case the logo is found in a webpage from this domain it won’t be detected as malicious. To do this, run the following command: **`!DBotUpdateLogoURLPhishing logoName="palo_alto" associatedDomains="paloaltonetworks.com" action="ModifiedDomainForLogo"`** ![dbot-updatelogo-success.png](/files/oe3VpFD41NSkl4MA6SH5) Remove a Custom Logo In case we want to remove a custom logo from the model we can run the following command: **`!DBotUpdateLogoURLPhishing action=RemoveLogo logoName=palo_alto`** ![dbot-updatelogo-remove.png](/files/1pNtV8si0pmvL8nPx9TL) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://cortex-docs.paloaltonetworks.com/xsoar-6-administrator-guide/6.12/configure-cortex-xsoar/customize-and-configure-cortex-xsoar/machine-learning/additional-machine-learning-scripts.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.