-
Notifications
You must be signed in to change notification settings - Fork 4
unit.tutorial
While the macros and the basic structure of the test should seem familiar to everyone who ever wrote a text, the way it was launched may not.
Our test-library is meant to be used with our gdb-runner. This tool calls gdb, parses the output and executes a test with fitting plugins for breakpoints.
This is the most straight-forward way to execute a test on a remote device, such as a microcontroller. This library provides a plugin for the gdb-runner, named
metal-test-backend
; how to use it is described in detail in the gdb-runner documentation. A test launched with the runner will be called hosted
in this document.
For small tests, especially when integrated into a CI-System, the gdb-runner might be too slow and all the generated information not needed. To do this,
we also provide a standalone mode, with the bare minimum set of features. It does not generate any output, nor does it have any flow-control. It just will count
the errors and return a value != 0
in the main, so the build-system sees an indicated error. In this document this will be the called the standalone
mode.
You can add coverage support in the hosted environment by using the metal-newlib
plugin, so the compile gcov code can write into files.
metal.unit
provides two test levels, error
and warning
expressed in the code as
ASSERT
EXPECT
Both will be written into the console, but only errors will cause the test to fail, i.e. return a value != 0, that is.
All tests are implemented through macros, so parameter names and location can be obtained easily. That way the backend can provide detailed information of the code. The usual format for the macros looks like this:
METAL_``['`level`]``_``['test-name]``(``['arguments]``);
We will use a few of them in examples, though to see all, please refer to the reference.
As an example, we use the equality check:
METAL_ASSERT_EQUAL(x, y);
METAL_EXPECT_EQUAL(i, j);
In standalone mode, there will not be any output for this example. However if we use it the hosted
mode the gdb-runner will not only output the names of the
parameters and the location of the test, but also provide information about the actual value. Since the debugger is used for this, we can not only output types
with a defined serialization operator, but everything.
*The format of the output depends on the gdb version.] *
For the given example, the output could look like this.
test.cpp(10) assertion succeeded [equality]: x == y; [3 == 3]
test.cpp(11) expectation failed [equality]: i == j; [42 == -1]
Tests are static when they do not need to executed, which requires that they are constant-expressions. An example could look like this:
METAL_STATIC_ASSERT_EQUAL(sizeof(int), 4u);
This currently uses static_assert
in C++ and a simulation of this behaviour in C, which means that a test cannot be compiled when it fails.
This means that no static_warning
is provided. This behaviour may change in a future version.
Some tests provide an alternative called bitwise. This will take the corresponding operation and apply it for every single bit. It also provides bitwise output.
METAL_ASSERT_EQUAL_BITWISE(c, u);
This could produce output like this.
test.cpp(33) assertion succeeded [bitwise equality]: l == k; [0b11111111 == 0b11111111]
- The logical connection between the bits still differs; a not equal will fail if any of the bits is different, while an equal will fail if any of the bits are different. Please see the reference for details.*
Since the shift operator is used, the behaviour is implementation-defined for signed types.
For many tests this library also provide ranged tests. This makes test writing easier and cleans up the output.
This behaves differently between C++ and C - C++ expects two iterators, while C expects a pointer and size.
An example in C++ could look like this.
std::array<int, 3> arr1 = {-1,0,1};
std::array<short, 4> arr2 = {-1,0,1,2};
METAL_ASSERT_EQUAL_RANGED(arr1.begin(), arr1.end(), arr2.begin(), arr2.end());
Which would produce a similar output.
test.cpp(21) error entering ranged test [{[arr1.begin(), arr1.end()], [arr2.begin(), arr2.end()]}] with mismatch: 4 != 3
test.cpp(21) assertion succeeded [equality]: **range**[0]; [-1 == -1]
test.cpp(21) assertion succeeded [equality]: **range**[1]; [0 == 0]
test.cpp(21) assertion succeeded [equality]: **range**[2]; [1 == 1]
test.cpp(21) exiting ranged test: { executed : 3, warnings : 0, errors : 0}
In C an example could look like this.
int arr1[3] = {-1,0,1};
int arr1[4] = {-1,0,1};
METAL_EXPECT_EQUAL_RANGED(arr1, 3, arr2, 4);
Which would produce similar output.
This test start has a mismatch, which will produce an error. This will however not appear in the list of the erros of the ranged test.
A test can be written directly into the main
function, but we also provide a solution to seperate test cases using METAL_CALL.
In hosted mode these entering and leaving test cases will be documented. This allows grouping tests and controlling the test flow.
The same tests
Tests not written inside cases are called free tests.
void test_cases_1()
{
METAL_ASSERT_EQUAL(42, 42);
}
void test_cases_2()
{
METAL_ASSERT_NOT_EQUAL(42, 42);
}
int main(int argc, char *argv[])
{
METAL_EXPECT_EQUAL(42,0);
METAL_CALL(&test_cases_1, "Test Case 1");
METAL_CALL(&test_cases_2, "Test Case 2");
return METAL_REPORT();
}
Which would provide output like this.
starting test execution
cases.cpp(16) expectation failed [equality]: 42 == 0; [42 == 0]
cases.cpp(17) entering test case [Test Case 1]
cases.cpp(6) assertion succeeded [equality]: 42 == 42; [42 == 42]
leaving test case [Test Case 1]: { executed : 1, warnings : 0, errors : 0}
cases.cpp(18) entering test case [Test Case 2]
cases.cpp(11) assertion failed [equality]: 42 != 42; [42 != 42]
leaving test case [Test Case 1]: { executed : 1, warnings : 0, errors : 1}
free tests : { executed : 1, warnings : 1, errors : 0}
full test report: { executed : 2, warnings : 1, errors : 1}
Currently METAL_CALL only accepts function pointers with the signature void()
.
Critical tests are test, that will cause the test to chancel if failing. This of course only works in hosted mode, but the METAL_ERROR allows you to implement a custom handler for that.
void test_case()
{
METAL_CRITICAL(METAL_ASSERT(0));
}
int main(int argc, char * argv[])
{
METAL_CALL(func, "my test case");
METAL_EXPECT(0);
return METAL_REPORT();
}
Which might have this output.
starting test execution
cancel_case.cpp(11) entering test case [my test case]
cancel_case.cpp(6) critical assertion failed [expression]: 0
canceling test case [my test case]: { executed : 1, warnings : 0, errors : 1}
cancel_case.cpp(12) expectation failed [expression]: 0
free tests : { executed : 1, warnings : 1, errors : 0}
full test report: { executed : 2, warnings : 1, errors : 1}
A failing critical exception will not result in an error, but still stop execution.
There are three ways to include the macros for the test.
Header | Language | Comment |
---|---|---|
<metal/unit> |
C/C++ | Automatically selects the flavour |
<metal/unit.h> |
C | Macros in C style |
<metal/unit.hpp> |
C++ | Macros in C++ style |
<metal/unit.ipp> |
C/C++ | The function actually used for the breakpoint. It's included by the other headers unless METAL_NO_IMPLEMENT_INCLUDE is defined. |
- Overview
- Runner Introduction
- Runner Invocation
- Runner Plugins
- Runner Extender
- Runner FAQ
- Runner Reference
- Unit Introduction
- Unit Tutorial
- Unit FAQ
- Unit Reference
- Calltrace Introduction
- Calltrace Tutorial
- Calltrace FAQ
- Calltrace Plugin
- Calltrace Reference
- Serial Introduction
- Serial Tutorial
- Serial Invocation
- Serial Reference