Controlling test execution via the command line

The fcfTest library provides a flexible system for test filtering and logging configuration using command-line arguments. This allows you to run only the necessary test groups, exclude problematic areas, or change the output verbosity without recompiling the code.

Main Operating Modes

The fcf::NTest::cmdRun function determines the program's behavior based on the passed flags. The main control modes are:

--test-run - forces the execution of selected tests. Used if fcf::NTest::cmdRun is called with the fcf::NTest::CRM_EXECUTE or fcf::NTest::CRM_PARSE flag.
--test-list - displays a list of all registered tests in the hierarchy (Part -> Group -> Test).
--test-help - displays help information regarding available parameters.
--test-no-break - if this parameter is specified, the execution of the remaining tests is not interrupted in case of a test failure.

Filtering Parameters

Filtering works on the principle of inclusion or exclusion. Parameters can be used multiple times to combine conditions.

Parameter	Description
`--test-part NAME`	Run only tests belonging to the specified part.
`--test-group NAME`	Run only tests belonging to the specified group.
`--test-test NAME`	Run only a specific test by its name.
`--test-select [PART] [GROUP] [TEST]`	Runs only tests that satisfy the selector specified by the three parameters. If a parameter is an empty string or "*", it is assumed that the selector selects all elements from the group. Multiple values can be provided in a parameter, separated by the "\|" symbol. Example: `test --test-select "Library" "" "func1\|func2"`
`--test-ignore-part NAME`	Exclude the specified part from execution.
`--test-ignore-group NAME`	Exclude the specified group from execution.
`--test-ignore-test NAME`	Exclude a specific test from execution.
`--test-ignore-select PART GROUP TEST`	Exclude tests that satisfy the selector specified by the three parameters. If a parameter is an empty string or "*", it is assumed that the selector selects all elements from the group. Multiple values can be provided in a parameter, separated by the "\|" symbol.

Explanatory details:

The --test-part, --test-group, --test-test, --test-select commands are combined using the OR operation
The --test-ignore-part, --test-ignore-group, --test-ignore-test, --test-ignore-select commands are combined using the OR operation

Output Format

The framework supports an additional JUnit XML output format for integration with other automated testing systems.

Parameter Description

Parameter	Description
`--test-format FORMAT`	Specifies the format of terminal output. By default, the format parameter can take only two values: `default` and `junit`. However, if user code declares additional output formats, they can also be specified.

--test-format FORMAT

Specifies the format of terminal output.

By default, the format parameter can take only two values: default and junit. However, if user code declares additional output formats, they can also be specified.

File Output Management

The test execution results can be written to a separate file with the desired format.

Parameter Description

--test-file FILE The test execution result will be written to the file. The file format is determined by the --test-format parameter. Default is "default".

Parameter	Description
`--test-file FILE`	The test execution result will be written to the file. The file format is determined by the --test-format parameter. Default is "default".
`--test-file-FORMAT FILE`	Writes the test execution result to a file with the specified format. Example of writing in default format: `test --test-file-default report.txt` Example of writing in junit format: `test --test-file-junit report.xml`

--test-file-FORMAT FILE

Writes the test execution result to a file with the specified format.

Example of writing in default format: test --test-file-default report.txt

Example of writing in junit format: test --test-file-junit report.xml

Logging Configuration

The --test-log-level LEVEL parameter sets the logger's sensitivity threshold. Messages with a level lower than the specified one will be ignored.

Available levels (from lowest to highest priority):

off — logging is disabled.
ftl — Fatal (critical errors).
err — Error (errors).
wrn — Warning (warnings).
att — Attention (notices).
log — Log (standard level, default).
inf — Info (information).
dbg — Debug (debugging).
trc — Trace (tracing).
all — All levels.
def — Uses the application's default logging level (which is log by default).

Usage Examples

1. Running a specific test group with extended logging:

    ./my_tests --test-run --test-group Math --test-log-level dbg

2. Running all tests except for the "Integration" part and the "Legacy" group:

    ./my_tests --test-run --test-ignore-part Integration --test-ignore-group Legacy

3. Viewing the list of all available tests:

    ./my_tests --test-list

4. Running one specific test:

    ./my_tests --test-run --test-test VectorSizeCheck