From: Gennadiy Rozental (rogeeff_at_[hidden])
Date: 2008-05-05 02:19:53
Juergen Hunold wrote:
> Hi !
>
> I'm using Boost.Test for quite a while and now I wonder if the behavior
> of "--catch_system_errors" is correct.
[...]
> When looking at the output of trunk "with_catch.test", I see
>
> --snip---
> with_catch: assert_test.cpp:13: void test_assert::test_method():
> Assertion `false' failed.
> <TestLog><TestSuite name="Master Test Suite"><TestCase
> name="test_assert"><Message file="assert_test.cpp"
> line="11">one</Message><Info file="assert_test.cpp" line="12">check
> true
> passed</Info><TestingTime>0</TestingTime></TestCase></TestSuite></TestLog><TestResult><TestSuite
> name="Master Test Suite" result="passed" assertions_passed="1"
> assertions_failed="0" expected_failures="0" test_cases_passed="1"
> test_cases_failed="0" test_cases_skipped="0"
> test_cases_aborted="0"></TestSuite></TestResult>
> EXIT STATUS: 0
> --snap--
I've copied the content of your test into unit_test_example_01.cpp. My
local run with msvc produces different output (I've added 2>err.xml
1>out.xml).
out.xml:
<TestLog><TestSuite name="Master Test Suite"><TestCase
name="test_assert"><Message file="unit_test_example_01.cpp"
line="11">one</Message><Info file="unit_test_example_01.cpp"
line="12">check true passed</Info><Exception name="test_assert">
This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
<LastCheckpoint file="unit_test_example_01.cpp"
line="12"></LastCheckpoint></Exception><TestingTime>0</TestingTime></TestCase></TestSuite></TestLog>
err.xml
Assertion failed: false, file unit_test_example_01.cpp, line 13
<TestResult><TestSuite name="Master Test Suite" result="failed"
assertions_passed="1" assertions_failed="1" expected_failures="0"
test_cases_passed="0" test_cases_failed="1" test_cases_skipped="0"
test_cases_aborted="1"></TestSuite></TestResult>
>
> So, some observations:
> Obviously, the UTF succeds in catching the "assert".
> The resulting message is not in the XML-Log
It is. You need to split stdout from stderr. Leading message in stderr
is unfortunate. I don't believe I can fix it easily
> The return code is 0 (Success).
Not in my expirience.
> A quick look at the new docs prototype at
>
> http://www.patmedia.net/~rogeeff/html/utf/usage-recommendations/command-line-specific.html
>
> found
> "If you got fatal exception somewhere within test case, make program
> generate coredump by adding extra command line argument "
>
> This was not easy to spot since the real test cases succeeded with error
> code 0 and the message was hidden in the log file.
We need to figure out why you have zero result code first.
> I found this because msvc on windows fires up an annoying debug window
> instead. So it seems this is not working there.
>
> So, my questions/suggestions are:
>
> Is this a bug ? Or a feature ? And if its a feature, where can I find
> the rationale ?
>
> I think the caught assertions should lead to a non-zero exit code and a
> failing test suite.
>
> And it would be nice if they got properly XML-formatted (but after a
> quick scan through the source, this seems difficult...)
It seems to be well formed for the most part.
Gennadiy