Re: [boost] CMake and Boost Build tests

Subject: Re: [boost] CMake and Boost Build tests
From: paul (pfultz2_at_[hidden])
Date: 2017-07-27 22:07:01

On Thu, 2017-07-27 at 17:53 -0400, Edward Diener via Boost wrote:
> On 7/27/2017 5:15 PM, Florent Castelli via Boost wrote:
> > 
> > 
> > > 
> > > On 27 Jul 2017, at 22:29, Edward Diener via Boost <boost_at_[hidden]
> > > > wrote:
> > > 
> > > On 7/27/2017 12:58 PM, Florent Castelli via Boost wrote:
> > > > 
> > > > On Jul 27, 2017 16:41, "Edward Diener via Boost" <boost_at_lists.boost.or
> > > > g>
> > > > wrote:
> > > > On 7/27/2017 8:43 AM, Florent Castelli via Boost wrote:
> > > > > 
> > > > > On 26/07/2017 20:49, Edward Diener via Boost wrote:
> > > > > 
> > > > > > 
> > > > > > Following John Maddock's appeal for practical solutions related to
> > > > > > the
> > > > > > move to CMake, I would like to know what the CMake equivalent is
> > > > > > to the
> > > > > > Boost Build unit test functionality.
> > > > > > 
> > > > > > In other words what do I write for CMake in order to do a Boost
> > > > > > Build
> > > > > > compile, compile-fail, link, link-fail, run, and run-fail unit
> > > > > > tests ?
> > > > > > 
> > > > > In my own Boost-CMake project, I have implemented regular "RUN" in a
> > > > > way
> > > > > that looks similar to the original Boost Build using functions:
> > > > > https://github.com/Orphis/boost-cmake/blob/master/libs/system.cmake
> > > > > 
> > > > What is RUN supposed to be in your link above ?
> > > > An equivalent of run from Boost Build. See
> > > > https://github.com/boostorg/system/blob/develop/test/Jamfile.v2
> > > Please show the code for RUN in CMake. To ease the transition from Boost
> > > Build to CMake we really need CMake equivalents to the unit testing
> > > rules of Boost Build at the very least.
> > Let’s start over. Basically, you end up with those equivalents.
> > Run is compiling, linking an executable and running it, checking the
> > return code is success. It could translate to:
> > 
> > add_executable(test_name test_file.cpp)
> > target_link_libraries(test_name PRIVATE boost_library)
> > add_test(NAME test_name COMMAND test_name)
> > 
> > But, if you do that, you need 2 steps to run the test. First you compile
> > the program (using regular "make test_name" or even “cmake —build .
> > —target test_name”) and then you run your test through ctest “ctest . -R
> > test_name” (or you run all the tests without the filter). The problem
> > there is you have 2 commands, and ctest won’t capture the compilation or
> > linking errors.
> > To work around that, you can use this instead:
> > 
> > add_test(NAME test_name COMMAND cmake —build . —target test_name &&
> > $<TARGET_FILE:test_name>)
> > 
> > This will for each test build it with the proper build tool (make, ninja,
> > msbuild, xcodebuild…) and then run the program. The whole output will be
> > captured by ctest and can be transfered to any test dashboard, you won’t
> > have to look for errors in 2 places.
> > The big issue I talk about later if that if you asked ctest to run several
> > tests in parallel, it will have several “cmake —build . —target …” in
> > parallel for those tests, which is not supported by some tools (mainly
> > Ninja).
> > 
> Understood.
> 
> > 
> > > 
> > > 
> > > > 
> > > > Other commands should be easy enough to implement as well, but I
> > > > haven't
> > > > > 
> > > > > had the interest in doing that in my project yet (which predates the
> > > > > SC
> > > > > decision).
> > > > > 
> > > > > It is possible to do "compile" by creating a regular static library
> > > > > target
> > > > > with that file and then invoking "cmake --build . --target <compile
> > > > > test
> > > > > name>". Compile failures can be done similarly and check they fail.
> > > > > Link and Link-fail can be done the same way using actual binaries
> > > > > that are
> > > > > built, but not run.
> > > > > 
> > > > This is a poor solution.
> > > > Could you please explain why? Using Cmake directly to build the
> > > > sources
> > > > with the same flags as set in the toolchain is not controversial I'd
> > > > say.
> > > > It's actually working better than other solutions I've seen trying to
> > > > reproduce the features.
> > > I said it was a poor solution because all the Boost Build 'compile' rule
> > > does is to try to compile one or more source files. Success is if there
> > > are no compilations errors, while failure is if there is at least one
> > > compilation error. Why should one have to create an actual library to do
> > > this, especially as the source(s) may contain a main() function, and in
> > > fact usually does ?
> > > 
> > > I think the right solution, as suggested to me by someone else, is to
> > > create a CMake OBJECT library, which as I understand it is not really a
> > > library but just OBJECT files, and then run a test based on that. But of
> > > course I do not know if CMake supports that.
> > The thing is, static libraries are guaranteed to work and fairly standard,
> > object libraries are a bit finicky and may not work depending on the
> > generator. Using one or the other is merely an optimisation though, static
> > libraries are just an archive of object files after all. Similar to the
> > code above, you could implement them with this code:
> > 
> > add_library(test_name STATIC (or OBJECT) test_file.cpp)
> > target_link_libraries(test_name PRIVATE boost_library)
> > add_test(NAME test_name COMMAND cmake —build . —target test_name)
> > 
> > In this code, ctest will just build it and capture any compilation error.
> > Easy enough. It doesn’t matter if there are functions called main or not
> > since it’s not linked, the compiler won’t care for the most part about the
> > name of the functions (well, main has some special semantics, but I doubt
> > any test would rely on that).
> But now you are filling up an end-user's build output with a static 
> library for each 'compile' tested. Ughhh ! That seems so silly to me, 
> where just adding object files on success seems normal.

Well actually it should be added with `EXCLUDE_FROM_ALL`:

add_library(test_name STATIC test_file.cpp EXCLUDE_FROM_ALL)

This way it only builds when it is invoked directly(which is when the test is
ran), and not in the general build.