You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
108 lines
3.5 KiB
108 lines
3.5 KiB
Running the tests
|
|
-----------------
|
|
|
|
- All tests reside in the tests/ directory tree.
|
|
|
|
- Running the tests is as simple as:
|
|
$ ctest
|
|
|
|
- Alternatively, the test scripts can be run directly:
|
|
$ cd tests
|
|
$ ./run_format_tests.py
|
|
|
|
- To run the only the tests associated with language X ("cpp", "java", etc.):
|
|
$ cd tests
|
|
$ ./run_format_tests.py X
|
|
|
|
Adding new tests
|
|
----------------
|
|
|
|
- A test is defined by adding a line in one of the tests/*.test
|
|
files: you must pick the .test file for the proper programming
|
|
language, i.e. pick the X.test file which defines tests which use
|
|
inputs in language X.
|
|
|
|
- A test has a (unique) number (a.k.a. test name) -- it does not have
|
|
to be a sequential number, but it might be handy to treat it like
|
|
that.
|
|
|
|
- The X.test entry (line) has this format:
|
|
|
|
<test.name> <config.file> <input.file>
|
|
|
|
where the fields are separated by an arbitrary non-zero amount of
|
|
whitespace.
|
|
|
|
- As each test references a config file and an input file (the latter
|
|
written in programming language X), these are assumed to exist in
|
|
the paths tests/config/<config.file> and tests/<input.file>
|
|
respectively. It is _strongly_ advised to put input files in their
|
|
own subdirectory, so a 'C' source formatting test entry might look
|
|
like this (in 'c.test'):
|
|
|
|
99902 ger-full-cmt-reflow.cfg c/cmt-not-a-boxed-cmt.c
|
|
|
|
where 'ger-full-cmt-reflow.cfg' is stored at
|
|
tests/config/ger-full-cmt-reflow.cfg and 'c/cmt-not-a-boxed-cmt.c'
|
|
will be fetched from tests/c/cmt-not-a-boxed-cmt.c when you run
|
|
the tests.
|
|
|
|
|
|
Features
|
|
--------
|
|
|
|
- You can re-use config files and input files for multiple tests. So
|
|
one might observe test sets like these:
|
|
|
|
# clark's style - blank lines before and after flow control,
|
|
indented comments:
|
|
|
|
20100 clark.cfg c/i2c-core.c
|
|
20101 clark.cfg c/comment-indent.c
|
|
20102 clark.cfg c/indent.cpp
|
|
20103 clark.cfg c/output.cpp
|
|
|
|
and
|
|
|
|
21051 return-1.cfg c/nl_return_expr.c
|
|
21052 return-2.cfg c/nl_return_expr.c
|
|
21053 return-3.cfg c/nl_return_expr.c
|
|
21054 return-4.cfg c/nl_return_expr.c
|
|
|
|
- Reference output files (the reference against which the uncrustify
|
|
test run output is compared) are to be stored in the path
|
|
tests/expected/<X>/<NNN-TTTTT.TTT> where <X> is the 'language
|
|
directory' part of the input file, e.g. 'c' for input file
|
|
'c/nl_return_expr.c', <NNN> is the test name (number), e.g. '21051',
|
|
and <TTTTT.TTT> is the file name part of the input test filespec,
|
|
e.g. 'nl_return_expr.c' for input file spec 'c/nl_return_expr.c'.
|
|
|
|
This means that for test
|
|
|
|
21051 return-1.cfg c/nl_return_expr.c
|
|
|
|
the accompanying reference output is:
|
|
|
|
tests/expected/c/21051-nl_return_expr.c
|
|
|
|
- When starting out with a new test, you don't need to have a
|
|
'reference output' yet: the test will simply be reported as a
|
|
'fail'ed test until you do.
|
|
|
|
- Tip: the easiest way to produce 'reference output' is to copy the
|
|
test output (from tests/results/...etc... ) to
|
|
tests/expected/...etc... once you've ascertained that those tests
|
|
produce the desired (correct) output. The helper script
|
|
tests/fixtest.sh will copy the results file to the output folder:
|
|
|
|
$ fixtest.sh 30014 00110 # copy tests 30014 and 00110
|
|
|
|
- Tip: the helper script tests/difftest.sh runs 'diff' on the result
|
|
and output versions to point out what changed.
|
|
|
|
- Tip: There are also a few options to run_format_tests.py that can
|
|
help. Most interesting is '-d' will run 'diff' if the test fails.
|
|
|
|
- Tip: If some errors occur with Windows, set the macro variable
|
|
NO_MACRO_VARARG to 1 to test some more pointer under Linux.
|