Testing

Since release 1.1.4 there is implemented testing subsystem that allows:

• perform manual tests (mtest module);
• perform automated unit testing (utest module);
• perform automated single-core performance testing (ptest module).

Manual tests are semi-automated or not automated at all. You may launch these tests to perform manual interaction with system or for retrieving some text data for further processing.

Unit tests are completely automated and check that the produced result is correct for the specific input data. By default unit tests are launched in separate processes and utilize all CPU cores for maximum performance. The status of each unit test is collected during execution and list of all failed tests are printed after completion.

Performance tests measure performance of some function/module under different conditions, gather final statistics and output them in a table format. These tests are very useful for measuring single-core performance of different modules and functions and performing code optimizations.

To build testing subsystem, issue the following commands:

make clean
make config TEST=1
make

After build, we can launch the test binary by issuing command:

.build/host/lsp-plugin-fw/lsp-plugins-test

This binary provides simple command-line interface, so here's the full usage:

USAGE: {utest|ptest|mtest} [args...] [test name...]
  First argument:
    utest                 Unit testing subsystem
    ptest                 Performance testing subsystem
    mtest                 Manual testing subsystem
  Additional arguments:
    -a, --args [args...]  Pass arguments to test
    -d, --debug           Disable time restrictions for unit tests
                          for debugging purposes
    -e, --execute         Launch tests specified after this switch
    -f, --fork            Fork child processes (opposite to --nofork)
    -h, --help            Display help
    -i, --ignore          Ignore tests specified after this switch
    -j, --jobs            Set number of job workers for unit tests
    -l, --list            List all available tests
    -mt, --mtrace         Enable mtrace log
    -nf, --nofork         Do not fork child processes (for better
                          debugging capabilities)
    -nt, --nomtrace       Disable mtrace log
    -o, --outfile file    Output performance test statistics to specified file
    -s, --silent          Do not output additional information from tests
    -t, --tracepath path  Override default trace path with specified value
    -v, --verbose         Output additional information from tests

Each test has fully-qualified name separated by dot symbols, tests from different test spaces (utest, ptest, mtest) may have similar fully-qualified names.

To obtain a list of all unit tests we can issue:

.build/host/lsp-plugin-fw/lsp-plugins-test utest --list

And then we can launch all complex number processing unit tests and additionally 'dsp.mix' unit test:

.build/host/lsp-plugin-fw/lsp-plugins-test utest dsp.complex.* dsp.pcomplex.* dsp.mix

If we don's specify any unit test name in argument, then all available unit tests will be launched.

To start debugging of some unit test, you need to pass additional arguments:

.build/host/lsp-plugin-fw/lsp-plugins-test/lsp-plugins-test utest --nofork --debug --verbose

Because unit tests are short-time fully-automated tests, they are parallelized and executed by default by number_of_cores*2 processes. To disable this, we specify option --nofork. Also, unit test execution time is limited by 5 seconds by default, so when debugging, our tests will be killed. That's why we specify --debug option to disable time control. Option --verbose allows to output additional information by the unit test that is turned off by default.

We also can use performance tests to obtain full performance profile of target machine. Because performance tests in most cases take much time for gathering statistics, the final statistics for each test can be saved in a separate file by specifying --outfile option:

.build/host/lsp-plugin-fw/lsp-plugins-test ptest -o performance-test.log

Manual tests are mostly designed for developers' purposes.