Getting started

Adding tests

Adding tests is done using the Test macro:

Test(Suite, Name, ...) internal

Defines a new test.

Parameters
  • Suite: The name of the test suite containing this test.
  • Name: The name of the test.
  • ...: An optional sequence of designated initializer key/value pairs as described in the criterion_test_extra_data structure (see criterion/types.h). Example: .exit_code = 1

Example:

#include <criterion/criterion.h>

Test(suite_name, test_name) {
    // test contents
}

suite_name and test_name are the identifiers of the test suite and the test, respectively. These identifiers must follow the language identifier format.

Tests are automatically sorted by suite, then by name using the alphabetical order.

Asserting things

Assertions come in two kinds:

  • cr_assert* are assertions that are fatal to the current test if failed; in other words, if the condition evaluates to false, the test is marked as a failure and the execution of the function is aborted.
  • cr_expect* are, in the other hand, assertions that are not fatal to the test. Execution will continue even if the condition evaluates to false, but the test will be marked as a failure.

cr_assert() and cr_expect() are the most simple kinds of assertions criterion has to offer. They both take a mandatory condition as a first parameter, and an optional failure message:

#include <string.h>
#include <criterion/criterion.h>

Test(sample, test) {
    cr_expect(strlen("Test") == 4, "Expected \"Test\" to have a length of 4.");
    cr_expect(strlen("Hello") == 4, "This will always fail, why did I add this?");
    cr_assert(strlen("") == 0);
}

On top of those, more assertions are available for common operations. See Assertion reference for a complete list.

Configuring tests

Tests may receive optional configuration parameters to alter their behaviour or provide additional metadata.

Fixtures

Tests that need some setup and teardown can register functions that will run before and after the test function:

#include <stdio.h>
#include <criterion/criterion.h>

void setup(void) {
    puts("Runs before the test");
}

void teardown(void) {
    puts("Runs after the test");
}

Test(suite_name, test_name, .init = setup, .fini = teardown) {
    // test contents
}

If a setup crashes, you will get a warning message, and the test will be aborted and marked as a failure. Is a teardown crashes, you will get a warning message, and the test will keep its result.

Testing signals

If a test receives a signal, it will by default be marked as a failure. You can, however, expect a test to only pass if a special kind of signal is received:

#include <stddef.h>
#include <signal.h>
#include <criterion/criterion.h>

// This test will fail
Test(sample, failing) {
    int *ptr = NULL;
    *ptr = 42;
}

// This test will pass
Test(sample, passing, .signal = SIGSEGV) {
    int *ptr = NULL;
    *ptr = 42;
}

This feature will also work (to some extent) on Windows for the following signals on some exceptions:

Signal Triggered by
SIGSEGV STATUS_ACCESS_VIOLATION, STATUS_DATATYPE_MISALIGNMENT, STATUS_ARRAY_BOUNDS_EXCEEDED, STATUS_GUARD_PAGE_VIOLATION, STATUS_IN_PAGE_ERROR, STATUS_NO_MEMORY, STATUS_INVALID_DISPOSITION, STATUS_STACK_OVERFLOW
SIGILL STATUS_ILLEGAL_INSTRUCTION, STATUS_PRIVILEGED_INSTRUCTION, STATUS_NONCONTINUABLE_EXCEPTION
SIGINT STATUS_CONTROL_C_EXIT
SIGFPE STATUS_FLOAT_DENORMAL_OPERAND, STATUS_FLOAT_DIVIDE_BY_ZERO, STATUS_FLOAT_INEXACT_RESULT, STATUS_FLOAT_INVALID_OPERATION, STATUS_FLOAT_OVERFLOW, STATUS_FLOAT_STACK_CHECK, STATUS_FLOAT_UNDERFLOW, STATUS_INTEGER_DIVIDE_BY_ZERO, STATUS_INTEGER_OVERFLOW
SIGALRM STATUS_TIMEOUT

See the windows exception reference for more details on each exception.

Configuration reference

Here is an exhaustive list of all possible configuration parameters you can pass:

struct criterion_test_extra_data

Contains all the options that can be set for a test, through the Test and TestSuite macros, or other means.

Public Members

void (*init)(void)

The setup test fixture.

This function, if provided, will be executed during the initialization of the test.

void (*fini)(void)

The teardown test fixture.

This function, if provided, will be executed during the finalization of the test.

int signal

The expected signal to be raised by the test.

If the test does not raise the specified signal, then the test is marked as failed.

A value of 0 means that is it not expected for the test to raise any signal.

int exit_code

The expected exit status to be returned by the test.

By default, criterion exits the test process with a value of 0. If it is expected for the test to exit with a non-zero status, this option can be used.

bool disabled

If true, skips the test.

The test will still appear in the test list, but will be marked as skipped and will not be executed.

const char *description

The long description of a test.

If a description is provided, it will be printed in test reports, and logged if the runner runs in verbose mode.

double timeout

The timeout for the test, in seconds.

If the realtime execution of a test takes longer than the specified value, then the test is immediately aborted and reported as timing out.

A value of 0 is equivalent to +INFINITY and means that the test does not timeout.

It is unspecified behaviour for the value of timeout to be negative or NaN.

void *data

Extra user data.

This field is currently unused.

Setting up suite-wise configuration

Tests under the same suite can have a suite-wise configuration – this is done using the TestSuite macro:

TestSuite(Name, ...) internal

Explicitely defines a test suite and its options.

Parameters
  • Name: The name of the test suite.
  • ...: An optional sequence of designated initializer key/value pairs as described in the criterion_test_extra_data structure (see criterion/types.h). These options will provide the defaults for each test.

Example:

#include <criterion/criterion.h>

TestSuite(suite_name, [params...]);

Test(suite_name, test_1) {
}

Test(suite_name, test_2) {
}

Configuration parameters are the same as above, but applied to the suite itself.

Suite fixtures are run along with test fixtures.