Test Cases
What is a Test Case?
A test case is a definition of the conditions in which a ScientificSimulation is going to be simulated.
Not only should it include the parameters that configure the said simulation, but also the geometry definition, as a mesh or as CAD information.
In COOLFluiD, this information is stored in CFcase files which hold the parameters and configuration of the simulation. However it does not include the geometry information, which should be stored in separate file in some format, to which COOLFluiD is able to read, either via some framework functionality or via some external Plug-In.
Check Test Cases
How to check if the testcases still work?
Before committing any change in the COOLFluiD repository, the developer should ALWAYS check if the modifications he's going to submit are affecting an existing set of reference testcases. A script named "check-testcases.pl" does this automatically, by running all the CFcases having the keyword:
### Residual = VALUE
The script compares the given residual VALUE (corresponding to the status in the current COOLFluiD version in the repository) with the residual obtained by the same testcases after the changes one is going to submit.
A tolerance is fixed and allows to make the check less constraining, in order to take into account machine dependent behaviour (e.g. due to different versions of BLAS, LAPACK, etc.).
If the testcase fails, despite the tolerance, this means that the changes are introducing a bug, and, unless the developer has a good explanation for that (e.g. he fixed an already existing bug, affecting existing testcases), the developer should investigate the problem and fix it before submitting. Only if the difference in residual is due to something foreseen by the developer, because he actually fixed an existing problem, he should update the ### Residual field with the new right value.
How to handle a failing testcase?
It is allowed to check in changes with some failing testcases only if the reasons for all the failures are known and reported on the FailingTestCases page.
