pre-commit

Run Demisto SDK checks and validations on changed Cortex content before committing.

The pre-commit command enhances the content development experience by running a variety of checks and linters. It utilizes the pre-commit framework for managing and maintaining multi-language pre-commit hooks. A .pre-commit-config-template.yaml file is used to configure the hooks if they are found in the content repository. Otherwise, a default is used.

Since content items run in containers with different Python versions and dependencies, this command matches content items with similar configurations before passing the generated temporary .pre-commit-config.yaml file.

Note

An internet connection is required to use pre-commit.

Arguments

-i, --input, --files	The paths to run pre-commit on. May pass multiple paths.
--staged-only	Whether to run only on staged files.
--committed-only	Whether to run on committed files only.
-g, --git-diff	Whether to use Git to determine which files to run on.
--prev-ver	The previous version to compare against. If not provided, the previous version will be determined using Git.
-a, --all-files	Whether to run on all files.
--mode	Special mode to run pre-commit with.
--skip	A list of pre-commit hooks to skip.
--validate/--no-validate	Whether to run demisto-sdk validate.
--format/--no-format	Whether to run demisto-sdk format.
--secrets/--no-secrets	Whether to run demisto-sdk secrets.
--show-diff-on-failure	Show diff on failure.
--dry-run	Whether to run the pre-commit hooks in dry-run mode, which only creates the configuration file.
--docker/--no-docker	Whether to run Docker-based hooks.
--image-ref	The Docker image reference to run Docker hooks with. Overrides the Docker image from the YAML file or the native image configuration.
--docker-image	Override the docker_image property in the template file. This is a comma-separated list of: from-yml, native:dev, native:ga, native:candidate.
--console-log-threshold	Minimum logging threshold for the console logger.
--file-log-threshold	Minimum logging threshold for the file logger.
--log-file-path	Path to save log files to.
--template-path	A custom path for the pre-defined pre-commit template. If not provided, the system uses the default template.

Examples

• demisto-sdk --pre-commit

  Runs pre-commit on all files collected by Git.
• demisto-sdk --pre-commit -i Packs/hello_world

  Runs pre-commit on all files in the hello_world pack.
• demisto-sdk --pre-commit --no-validate

  Runs pre-commit without the validate step.
• demisto-sdk --pre-commit --show-diff-on-failure

  Runs pre-commit and shows differences on failures.

Modes

You can set different arguments for different modes. For example, you might want to exclude some rules from the nightly build. Any key can be set this way.

You can use any mode you want and have multiple modes for each hook as in the following example.

Example

To set arguments for a mode:

  - id: sample-hook
    args:nightly: ['--exclude', '123']
    args:mode1: ['--test', '123']
    args: ['This is the default argument']
    otherkey:nightly: hello
    otherkey: world

To call the pre-commit command:

demisto-sdk pre-commit -a --mode nightly

Hooks

Hooks can be set in the .pre-commit-config_template.yaml file. The syntax is similar to the official pre-commit hooks, but allows more keys to be set.

Skip key

To skip a certain hook from running, add a skip key to the hook configuration:

- id: sample-hook
  skip: true

The skip hook can be used together with mode, to skip a hook when running in a specific mode.

Needs key

The needs keys allows you to define dependencies between hooks. If a hook with needs is skipped, hooks that depend on it are also skipped. In this example, both hooks are skipped:

- id: sample-hook
  skip: true
- id: needs-example
  needs: ["sample-hook"]

Parallel key

The parallel key indicates whether a hook should run in parallel. By default, hooks such as mypy, Ruff, and Docker produce multiple hooks which run in parallel. Default is True. To avoid running a specific hook in parallel, you can set it to false. When setting the parallel of a hook to false, a split hook will run sequentially.

- id: sample-hook
  parallel: false

External tools

• Ruff

  Runs Ruff, the extremely fast Python linter, using the Python version used in the container image set for the content item.

  This step uses the Ruff rules configured in the pyproject.toml at the root of the content repo. If pyproject.toml does not exist, it uses Ruff's default rules. The rules set under the official demisto/content repo pyproject.toml file were handpicked to ensure a high level of code quality, and prevent bugs often found in content.

  This step may modify files using Ruff's --fix flag. When files are modified, the step fails. We recommend committing them into the repo.
• Check JSON/YAML/AST(py)

  Validates that the content of the files is formatted correctly.
• Autopep8

  A linter that automatically formats Python code to conform to the PEP 8 Style Guide.
• Pycln

  Pycln is a formatter for finding and removing unused import statements.

Docker hooks

To run a command in a script's relevant container, you can set up a Docker hook.

A Docker hook must be under the local repo and its id must end with in-docker.

Example

  - id: simple-in-docker
    name: simple-in-docker
    description: my simple description
    entry: simplelinter
    args: ['run-all-checks']

The entry key should be the command that should be run (such as pylint, pytest).

env key

Some commands require environment variables to be set. To configure environment variables for Docker hooks you can set the env key. An example with pytest:

  - id: simple-in-docker
    env:
      DEMISTO_SDK__PYTEST__TESTS_PATH: Tests/scripts

copy_files key

Some hooks require several files to exist in the same directory as the code file in order to run properly. To configure this, set the copy_files key as follows:

- id: simple-in-docker
  copy_files:
    - Tests/scripts/conftest.py

config_file_arg key

Commands run in Docker often require a specific configuration file per integration or script. To configure this, you must set the config_file_arg key. The configuration file should be in the same directory as the code file.

Example with Ruff:

  - id: simple-in-docker
    config_file_arg:
      arg_name: '--config'
      file_name: 'ruff.toml'

SDK Commands

The following SDK commands are automatically run:

• validate
• format
• secrets