Environment and CLI

Tests built with Criterion expose by default various command line switchs and environment variables to alter their runtime behaviour.

Command line arguments

  • -h or --help: Show a help message with the available switches.
  • -q or --quiet: Disables all logging.
  • -v or --version: Prints the version of criterion that has been linked against.
  • -l or --list: Print all the tests in a list.
  • -f or --fail-fast: Exit after the first test failure.
  • --ascii: Don’t use fancy unicode symbols or colors in the output.
  • -jN or --jobs N: Use N parallel jobs to run the tests. 0 picks a number of jobs ideal for your hardware configuration.
  • --pattern [PATTERN]: Run tests whose string identifier matches the given shell wildcard pattern (see dedicated section below). (*nix only)
  • --no-early-exit: The test workers shall not prematurely exit when done and will properly return from the main, cleaning up their process space. This is useful when tracking memory leaks with valgrind --tool=memcheck.
  • -S or --short-filename: The filenames are displayed in their short form.
  • --always-succeed: The process shall exit with a status of 0.
  • --tap[=FILE]: Writes a TAP (Test Anything Protocol) report to FILE. No file or "-" means stderr and implies --quiet. This option is equivalent to --output=tap:FILE.
  • --xml[=FILE]: Writes JUnit4 XML report to FILE. No file or "-" means stderr and implies --quiet. This option is equivalent to --output=tap:FILE.
  • --json[=FILE]: Writes a JSON report to FILE. No file or "-" means stderr and implies --quiet. This option is equivalent to --output=tap:FILE.
  • --verbose[=level]: Makes the output verbose. When provided with an integer, sets the verbosity level to that integer.
  • -OPROVIDER:FILE or --output=PROVIDER:FILE: Write a test report to FILE using the output provider named by PROVIDER. If FILE is "-", it implies --quiet, and the report shall be written to stderr.

Shell Wildcard Pattern

Extglob patterns in criterion are matched against a test’s string identifier. This feature is only available on *nix systems where PCRE is provided.

In the table below, a pattern-list is a list of patterns separated by |. Any extglob pattern can be constructed by combining any of the following sub-patterns:

Pattern Meaning
* matches everything
? matches any character
[seq] matches any character in seq
[!seq] matches any character not in seq
?(pattern-list) Matches zero or one occurrence of the given patterns
*(pattern-list) Matches zero or more occurrences of the given patterns
+(pattern-list) Matches one or more occurrences of the given patterns
@(pattern-list) Matches one of the given patterns
!(pattern-list) Matches anything except one of the given patterns

A test string identifier is of the form suite-name/test-name, so a pattern of simple/* matches every tests in the simple suite, */passing matches all tests named passing regardless of the suite, and * matches every possible test.

Environment Variables

Environment variables are alternatives to command line switches when set to 1.

  • CRITERION_ALWAYS_SUCCEED: Same as --always-succeed.
  • CRITERION_NO_EARLY_EXIT: Same as --no-early-exit.
  • CRITERION_FAIL_FAST: Same as --fail-fast.
  • CRITERION_USE_ASCII: Same as --ascii.
  • CRITERION_JOBS: Same as --jobs. Sets the number of jobs to its value.
  • CRITERION_SHORT_FILENAME: Same as --short-filename.
  • CRITERION_VERBOSITY_LEVEL: Same as --verbose. Sets the verbosity level to its value.
  • CRITERION_TEST_PATTERN: Same as --pattern. Sets the test pattern to its value. (*nix only)
  • CRITERION_DISABLE_TIME_MEASUREMENTS: Disables any time measurements on the tests.
  • CRITERION_OUTPUTS: Can be set to a comma-separated list of PROVIDER:FILE entries. For instance, setting the variable to tap:foo.tap,xml:bar.xml has the same effect as specifying --tap=foo.tap and --xml=bar.xml at once.
  • CRITERION_ENABLE_TAP: (Deprecated, use CRITERION_OUTPUTS) Same as --tap.