111 lines
4.9 KiB
Plaintext
111 lines
4.9 KiB
Plaintext
iBoot has a system for building and running unit tests. Currently the system only allows
|
|
for tests to be run on an x86 host using the native libc, but it's intended that the
|
|
system will allow tests to run on x86 with iBoot's libc, and built for arm and run on
|
|
targets (either hosted or freestanding).
|
|
|
|
Overview
|
|
========
|
|
|
|
To add tests to a module, do the following:
|
|
1. Create a test driver .c file (details below).
|
|
2. Create a test makefile in the module's directory, normally called tests.mk. If the module
|
|
requires multiple test binaries (usually because of different compilation options), add
|
|
a makefile per binary, usually called tests-<flavor>.mk. Details on makefiles are below
|
|
3. Add the makefile to the TEST_MAKEFILES list in makefiles/tests.mk
|
|
|
|
Writing tests
|
|
=============
|
|
The only requirement for test binaries is that their main function returns 0 on success
|
|
and nonzero on failure. By default, tests should only print to stdio on failure. iBoot
|
|
includes a unit testing framework, but its use is optional is a different testing strategy
|
|
is needed (or for legacy tests that don't use the framework). Most unit tests should use
|
|
the testing framework, however.
|
|
|
|
Using the testing framework
|
|
===========================
|
|
|
|
To use the testing framework, add tests/unittest-main.o to the test's object file list
|
|
(normally TEST_SUPPORT_OBJS, not TEST_OBJS) and include <unittest.h> in your main test
|
|
file. unittest-main includes a main() function to run test cases, so only the test cases
|
|
themselves need to be defined when using the testing framework.
|
|
|
|
The unit testing frameworks works with test suites and test cases. A test suite includes
|
|
an optional setup function, a list of test cases, and an optional cleanup function.
|
|
The setup and cleanup function are respectively run before and after each test case.
|
|
Each test cases is simply a function that takes a uintptr_t parameter.
|
|
|
|
Use struct test_suite to define a test suite, and then use the TEST_SUITE macro to make
|
|
the test suite visble to the main() function. For example:
|
|
|
|
struct test_suite my_test_suite = {
|
|
.name = "mymodule",
|
|
.setup_function = my_setup_function // optional
|
|
.cleanup_function = my_cleanup_function // optional
|
|
.test_cases = {
|
|
{ "test1", run_test1, 0 },
|
|
{ "test2,0", run_test2, 0 }, // passes 0 in parameter
|
|
{ "test2,100", run_test2, 100 }, // passes 100 in parameter
|
|
TEST_CASE_LAST
|
|
}
|
|
};
|
|
|
|
Test cases are defined by their name, their function, and their argument (a uintptr_t).
|
|
Test case functions take a single uintptr_t parameter and return void.
|
|
|
|
A test succeeds if no assertions fail, no unexpected panics occur, expected panics
|
|
(if any) do occur, and TEST_FAIL is not called. Tests are immediately aborted on
|
|
assertion failures, panics (expected or unexpected), and calls to TEST_FAIL.
|
|
|
|
Assertions can be embedded inside the code under test using the typical iBoot ASSERT
|
|
and RELEASE_ASSERT macros. These assertions end up in shipping code, of course. For
|
|
assertions in the test code, the unittest framework provides a number of convenience
|
|
macros, including:
|
|
|
|
For integeger types: TEST_ASSERT_{EQ,NEQ,GT,LT,GTE,LTE}
|
|
For C strings: TEST_ASSERT_STR_EQ, TEST_ASSERT_STR_NEQ
|
|
For binary data: TEST_ASSERT_MEM_EQ, TEST_ASSERT_MEM_NEQ
|
|
For pointers: TEST_ASSERT_PTR_EQ, TEST_ASSERT_PTR_NEQ, TEST_ASSERT_NULL, TEST_ASSERT_NOT_NULL
|
|
|
|
These macros (with the exception of the MEM macros) support printing the expression and
|
|
value for the left hand and right hand side of the comparison on failure.
|
|
|
|
A general-purpose TEST_FAIL macro is also provided for things not covered by the assertion
|
|
macros above.
|
|
|
|
Writing test makefiles
|
|
======================
|
|
Test makefiles define the following variables:
|
|
|
|
TEST_NAME
|
|
The name of the test binary, must be unique
|
|
|
|
TEST_OBJS
|
|
The object files being tested
|
|
|
|
TEST_SUPPORT_OBJS (optional):
|
|
Object files used to run the tests. These are listed separately so that
|
|
their source files are not included in test coverage reporting
|
|
|
|
TEST_CFLAGS (optional):
|
|
CFLAGS to be used while building the test binary
|
|
|
|
TEST_LDFLAGS (optional):
|
|
LDFLAGS to be used when building the test binary
|
|
|
|
Test coverage
|
|
=============
|
|
The unit tests are automatically built with gcov support. This can be suppressed by passing
|
|
COVERAGE=NO on the make commandline.
|
|
make COVERAGE=NO tests
|
|
|
|
The best way to parse the coverage info generated by the tests is with the gcovr tool
|
|
(find it online until it gets included in iBoot source). Run the tool as follows:
|
|
mkdir -p build/coverage && ./gcovr -b -r . --html -o build/coverage/index.html --html-details
|
|
|
|
This will only give coverage statistics for the source files as built for unit testing, which
|
|
may be different from how they're built for real targets. To see how well the tests are
|
|
covering real world scenarios, it's possible to generate coverage info for real targets. This
|
|
will just emit info on which source lines emitted executable code; no profiling will actually
|
|
be performed on the targets. To generate this info, add COVERAGE=YES to the make commandline:
|
|
make COVERAGE=YES build
|