> For the complete documentation index, see [llms.txt](https://cortex-docs.paloaltonetworks.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cortex-docs.paloaltonetworks.com/cortex-xsiam-developer-guide/cortex-xsiam-development-guide/integrations-and-scripts/advanced-topics/feed-integrations.md). # Feed Integrations Feed integrations allow fetching indicators from feeds, such as TAXII and Office 365. An example feed integration can be seen [here](https://github.com/demisto/content/tree/master/Packs/FeedOffice365/Integrations/FeedOffice365). While feed integrations are developed the same as other integrations, they include several extra configuration parameters and APIs. **Naming convention** Feed integration names (id, name and display fields) should end with the word `Feed`. This consistent naming convention ensures that users can easily understand what the integration is used for. **Required parameters** Every feed integration should have the following parameters in the integration YAML file: ```programlisting - display: Fetch indicators  name: feed  defaultvalue: true  type: 8  required: false - display: Indicator Reputation  name: feedReputation  defaultvalue: feedInstanceReputationNotSet  type: 18  required: false  options:  - None  - Good  - Suspicious  - Bad  additionalinfo: Indicators from this integration instance will be marked with this    reputation. - display: Source Reliability  name: feedReliability  defaultvalue: F - Reliability cannot be judged  type: 15  required: true  options:  - A - Completely reliable  - B - Usually reliable  - C - Fairly reliable  - D - Not usually reliable  - E - Unreliable  - F - Reliability cannot be judged  additionalinfo: Reliability of the source providing the intelligence data. - display: ""  name: feedExpirationPolicy  defaultvalue: indicatorType  type: 17  required: false  options:  - never  - interval  - indicatorType  - suddenDeath - display: ""  name: feedExpirationInterval  defaultvalue: "20160"  type: 1  required: false - display: Feed Fetch Interval  name: feedFetchInterval  defaultvalue: "240"  type: 19  required: false - display: Bypass exclusion list  name: feedBypassExclusionList  defaultvalue: ""  type: 8  required: false  additionalinfo: When selected, the exclusion list is ignored for indicators from    this feed. This means that if an indicator from this feed is on the exclusion    list, the indicator might still be added to the system. ``` The `defaultvalue` of the `feedReputation`, `feedReliability`, `feedExpirationPolicy`, and `feedFetchInterval` parameters should be set according to the qualities associated with the feed source for which you are developing a feed integration. **Incremental feeds** Incremental feeds pull only new or modified indicators that have been sent from the third party vendor. As the determination if the indicator is new or modified happens on the third-party vendor's side, and only indicators that are new or modified are sent to Cortex XSIAM, all indicators coming from these feeds are labeled new or modified. Examples of incremental feeds usually include feeds that fetch based on a time range. For example, a daily feed which provides new indicators for the last day or a feed which is immutable and provides indicators from a search date onwards. To indicate to Cortex XSIAM that a feed is incremental, add the configuration parameter: `feedIncremental`. If the user is not able to modify this setting, set the parameter to `hidden` with a `defaultValue` of `true`. For example: ```programlisting - additionalinfo: Incremental feeds pull only new or modified indicators that have been sent from the integration. The determination if the indicator is new or modified happens on the third-party vendor's side, so only indicators that are new or modified are sent to Cortex XSIAM. Therefore, all indicators coming from these feeds are labeled new or modified.  defaultvalue: 'true'  display: Incremental feed  hidden: true  name: feedIncremental  required: false  type: 8 ``` If the feed supports both incremental and non-incremental modes, provide the configuration parameter as non-hidden. Thus, a user will be able to modify this settings as they see fit. In the feed code inspect the `feedIncremental` parameter to perform the proper fetch logic. Code example of incremental feeds: * [AutoFocus Feed](https://github.com/demisto/content/blob/master/Packs/AutoFocus/Integrations/FeedAutofocus/FeedAutofocus.py) * [DHS Feed v2](https://github.com/demisto/content/blob/master/Packs/FeedDHS/Integrations/DHSFeedV2/DHSFeedV2.py) **Commands** Every feed integration has a minimum of three commands: * `test-module` - The command that is run when the **Test** button in the configuration panel of an integration is clicked. * `-get-indicators` - Where \ is replaced by the name of the Product or Vendor source providing the feed. For example, if you were developing a feed integration for Microsoft Intune, this command might be called msintune-get-indicators. This command should fetch a limited number of indicators from the feed source and display them in the War Room. * `fetch-indicators` - this command will initiate a request to the feed endpoint, format the data fetched from the endpoint to conform to Cortex XSIAM's expected input format, and create new indicators. If the integration instance is configured to fetch indicators, then this is the command that will be executed at the specified feed fetch Interval. **API command: demisto.createIndicators()** Use the `demisto.createIndicators()` function when the `fetch-indicators` command is executed. Here is an example from an existing feed integration: ```programlisting def main():    params = demisto.params()    client = Client(params.get('insecure'),                    params.get('proxy'))    command = demisto.command()    demisto.info(f'Command being called is {command}')    # Switch case    commands = {        'test-module': module_test_command,        'tor-get-indicators': get_indicators_command    }    try:        if demisto.command() == 'fetch-indicators':            indicators = fetch_indicators_command(client)            # we submit the indicators in batches            for b in batch(indicators, batch_size=2000):                demisto.createIndicators(b)        else:            readable_output, outputs, raw_response = commands[command](client, demisto.args())            return_outputs(readable_output, outputs, raw_response)    except Exception as e:        raise Exception(f'Error in {SOURCE_NAME} Integration [{e}]') ``` The `batch` function is imported from `CommonServerPython`. We see that indicators are returned from calling `fetch_indicators_command` and are passed to `demisto.createIndicators` in batches. **Indicator objects** Indicator Objects are passed to `demisto.createIndicators`. An example: ```programlisting {    "value": value,    "type": raw_json['type'],    "rawJSON": raw_json,    "fields": {'recordedfutureevidencedetails': lower_case_evidence_details_keys},    "score": score } ``` The object key and values: * `"value"` - required. The indicator value, e.g., `"8.8.8.8"`. * "`type"` - required. The indicator type (types as defined in Cortex XSIAM), e.g., `"IP"`. One can use the `FeedIndicatorType` class to populate this field. This class, which is imported from `CommonServerPython` has all of the indicator types that come out-of-the-box with Cortex XSIAM. ```programlisting class FeedIndicatorType(object):    """Type of Indicator (Reputations), used in TIP integrations"""    Account = "Account"    CVE = "CVE"    Domain = "Domain"    DomainGlob = "DomainGlob"    Email = "Email"    File = "File"    FQDN = "Domain"    MD5 = "File MD5"    SHA1 = "File SHA-1"    SHA256 = "File SHA-256"    Host = "Host"    IP = "IP"    CIDR = "CIDR"    IPv6 = "IPv6"    IPv6CIDR = "IPv6CIDR"    Registry = "Registry Key"    SSDeep = "ssdeep"    URL = "URL" ``` * `"rawJSON"` - required. This dictionary should contain the `"value"` and `"type"` fields as well as any other unmodified data returned from the feed source about an indicator. * `"fields"` - optional. A dictionary that maps values to existing indicator fields defined in Cortex XSIAM where the key is the `cliname` of an indicator field. To see the full list of existing fields: 1. In Cortex XSIAM, go to **Settings** → **Configurations** → **Object Setup** → **Indicators** and click on the **Fields** tab. 2. To inspect a specific field, **Edit** the field. Note that the field's `cliname` is listed as `machinename`. * `"relationships"` - optional. This list should contain a dictionary of values of the relationships. There are two ways to create relationships: * When creating an indicator. * Creating relationships by themselves without indicators associated with the relationships. For both ways, use `demisto.createIndicators`. To create a relationship: 1. Create an `EntityRelationship` object with the relationships data. If more than one relationship exists, create a list and append all of the `EntityRelationship` objects to it. The name of the relationships should be one of the existing [relationships](/cortex-xsiam-developer-guide/cortex-xsiam-development-guide/indicators/relationships.md): For more information, see [entity relationship](/cortex-xsiam-developer-guide/cortex-xsiam-development-guide/indicators/relationships.md). 2. Use the `to_indicator()` function of the object to convert each object (or a list of objects) to the required format. 3. If the relationships is part of an indicator, add the list to the relationship key of the indicator after running `to_indicator`. If the relationship is not attached to an indicator, create a dummy indicator with the value `$$DummyIndicator$$` and add the relationships key with the list after running `to_indicator`. {% hint style="info" %} ### Note In indicators of type "File", if you have multiple hash types for the same file (i.e., MD5, SHA256, etc.), you can use the corresponding `"fields"` to associate all hashes to the same object. The supported fields are: `md5`, `sha1`, `sha256`, `sha512`, `ssdeep`. You can use any of the aforementioned hash types as the indicator value for an indicator of type "File". {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://cortex-docs.paloaltonetworks.com/cortex-xsiam-developer-guide/cortex-xsiam-development-guide/integrations-and-scripts/advanced-topics/feed-integrations.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.