From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2005-04-05 21:16:00
>> Will you be willing to comply to Boost.Test format or adopt your format
>> to
>> Boost.Test interfaces?
>
> Sure ... ? I mean, I didn't see the part of the documentation that
> specified
> what Boost.Test format was. Perhaps you could be so kind as to point it
> out.
> Thx.
It's not there yet. I just asking to make sure that what I am doing will
meet your needs.
>> I am not sure how list of tests help you here: are you planning to run
> test
>> cases by name and want to see what persentage of tests you run?
> Two reasons:
> 1) I need to collect a report of which tests worked, which failed, and
> which
> succeeded. Percentage is only useful if 100% pass, otherwise there has to
> be
> further investigation. Typically our team collects the entire list of
> failures and divvies them up every day, so that the failure reports need
> to
> list the names of the test cases that failed.
Run a test program (all test cases) - report will show all test cases that
fails (with names)
> 2) We need to inventory the testcases so that we know how many there are
> and
> what they do, in order to assess coverage and compare that to the test
> plan.
> The gap between the test plan and the current list of testcases is one
> measure of how far we have left - sort of a measure of the known unknowns.
Set a report level to "detailed". Report will show you every test case with
it's status (passes/failed)
Note that you could specify report format as XML and use some automation to
analize the test report.
Also you could write and register your own report formatter.
> 3) Being able to list which tests are available would support our current
> test infrastructure, which allows you to request a build and then to run a
> particular test.
That's the only valid point from what I see.
>> > - the ability to generate a better error message - ie, include a bit of
>> > text
>> > about what the test was about.
>>
>> Why BOOST_..._MESSAGE doesn't work for you? You could also use
> BOOST_MESSAGE
>> for standalone mesage in test log
>
> Because I'm stupid? I guess I looked at the parameters of BOOST_TEST and
> thought that everything about the reporting the results of the unit test
> should be right there. Currently, it looks to me like a message for
> reporting an error is formatted to look like a compile error, which is
> convenient if you happen to be running the ad-hoc tests from emacs, but
> not
> particular convenient for generating an overall report of successes or
> failures.
1. You could specify XML as an log format and use some automation to get any
information you need
2. You could write and register your own log formatter.
HTH,
Gennadiy