Unleash¶

The main command used in Gremlins is unleash, that unleashes the gremlins and starts a mutation test of your code. If unleash is too long to type for you, you can use its aliases run and r.

To execute a mutation testing run just type

gremlins unleash

If the module build requires tags

gremlins unleash --tags "tag1,tag2"

Flags¶

unleash supports several flags to fine tune its behaviour.

Arithmetic base¶

--arithmetic-base · Default: true

Enables/disables the ARITHMETIC BASE mutant type.

gremlins unleash --arithmetic-base=false

Conditionals-boundary¶

--conditionals-boundary · Default: true

Enables/disables the CONDITIONALS BOUNDARY mutant type.

gremlins unleash --conditionals_boundary=false

Conditionals negation¶

--conditionals-negation · Default: true

Enables/disables the CONDITIONALS NEGATION mutant type.

gremlins unleash --conditionals_negation=false

Cover packages¶

--coverpkg · Default: empty

Apply coverage analysis in each test to packages matching the patterns. The default is for each test to analyze only the package being tested.

gremlins unleash --coverpkg "./internal/...,./pkg/..."

Diff¶

--diff · Default: empty

Run tests only for mutants inside code changes between current state and git reference (branch or commit). The default is each mutant covered by tests.

Branch merge base¶

gremlins unleash --diff "origin/main"

Commit¶

gremlins unleash --diff "b62af323"

PR¶

gremlins unleash --diff "origin/$GITHUB_BASE_REF"

Use actions/checkout@v4 with fetch-depth: 0 to fetch all history.

Using¶

Dry run¶

--dry-run/-d · Default: false

Just performs the analysis but not the mutation testing.

gremlins unleash --dry-run

Increment decrement¶

--increment-decrement · Default: true

Enables/disables the INCREMENT DECREMENT mutant type.

gremlins unleash --increment-decrement=false

Integration mode¶

--integration/-i · Default: false

In normal mode, Gremlins executes only the tests of the packages where the mutant is found. This is done to optimize the performance, running less test cases for each mutation.

The drawback of this approach lies in the fact that if a mutation in a package influences the tests of another package, this is not caught by Gremlins. In general, this is an acceptable drawback because packages should rely on their own tests, not on the tests of other packages.

Nonetheless, there may be cases where you may want to run all the test suite for each mutation, for example if you are analysing integration or E2E tests. In this scenario, you can enable integration mode. However, you should be aware that integration mode is generally much slower, and you can also get slightly different results depending on your test suite.

gremlins unleash --integration

Invert assignments¶

--invert-assignments · Default: false

Enables/disables the INVERT ASSIGNMENTS mutant type.

gremlins unleash --invert-assignments

Invert bitwise¶

--invert-bitwise · Default: false

Enables/disables the INVERT BITWISE mutant type.

gremlins unleash --invert-bitwise

Invert bitwise assignments¶

--invert-bwassign · Default: false

Enables/disables the INVERT BWASSIGN mutant type.

gremlins unleash --invert-bwassign

Invert logical operators¶

--invert-logical · Default: false

Enables/disables the INVERT LOGICAL mutant type.

gremlins unleash --invert_logical

Invert loop control¶

--invert-loopctrl · Default: false

Enables/disables the INVERT LOOP mutant type.

gremlins unleash --invert-loopctrl

Invert negatives¶

--invert-negatives · Default: true

Enables/disables the INVERT NEGATIVES mutant type.

gremlins unleash --invert_negatives=false

Output¶

--output/-o · Default: empty

When set, Gremlins will write the give output file with machine readable results.

gremlins unleash --output=output.json

The output file in JSON format and has the following structure:

{
  "go_module": "github.com/go-gremlins/gremlins",
  "test_efficacy": 82.00,
  //(1)
  "mutations_coverage": 80.00,
  //(2)
  "mutants_total": 100,
  "mutants_killed": 82,
  "mutants_lived": 8,
  "mutants_not_viable": 2,
  //(3)
  "mutants_not_covered": 10,
  "elapsed_time": 123.456,
  //(4)
  "files": [
    {
      "file_name": "myFile.go",
      "mutations": [
        {
          "line": 10,
          "column": 8,
          "type": "CONDITIONALS_NEGATION",
          "status": "KILLED"
        }
      ]
    }
  ]
}

This is a percentage expressed as floating point number.
This is a percentage expressed as floating point number.
NOT VIABLE mutants are excluded from all the calculations.
The elapsed time is expressed in seconds, expressed as floating point number.

Warning

The JSON output file is not pretty printed; it is optimised for machine reading.

Remove self-assignments¶

--remove-self-assignments · Default: false

Enables/disables the REMOVE_SELF ASSIGNMENTS mutant type.

gremlins unleash --remove-self-assignments

Tags¶

--tags/-t · Default: empty

Sets the go command build tags.

gremlins unleash --tags "tag1,tag2"

Test CPU¶

--test-cpu · Default: 0

Tip

To understand better the use of these flag, check workers

This flag overrides the number of CPUs the Go test tool will utilize. By default, Gremlins doesn't set this value.

gremlins unleash --test-cpu=1

Threshold efficacy¶

--threshold-efficacy · Default: 0

When set, it makes Gremlins exit with an error (code 10) if the test efficacy threshold is not met. By default it is zero, which means Gremlins never exits with an error.

The test efficacy is calculated as KILLED / (KILLED + LIVED) and assesses how effective are the tests.

gremlins unleash --threshold-efficacy 80

Threshold mutant coverage¶

--threshold-mcover · Default: 0

When set, it makes Gremlins exit with an error (code 11) if the mutant coverage threshold is not met. By default it is zero, which means Gremlins never exits with an error.

The mutant coverage is calculated as (KILLED + LIVED) / (KILLED + LIVED + NOT_COVERED) and assesses how many mutants are covered by tests.

gremlins unleash --threshold-mcover 80

Timeout coefficient¶

--timeout-coefficient · Default: 0

Tip

To understand better the use of these flag, check workers

Gremlins determines the timeout for each Go test run by multiplying by a coefficient the time it took to perform the coverage run. It is possible to override this coefficient (0 means use the default).

gremlins unleash --timeout-coefficient=3

Workers¶

--workers · Default: 0

Tip

To understand better the use of these flag, check workers

Gremlins runs in parallel mode, which means that more than one test at a time will be performed, based on the number of CPU cores available.

By default, Gremlins will use all the available CPU cores of, and , in integration mode, it will use half of the available CPU cores.

The --workers flag allows to override the number of CPUs to use (0 means use the default).

gremlins unleash --workers=4