Subject: Re: [boost] [thread] Memory usage of threads on 64bit linux
From: Phil Endecott (spam_from_boost_dev_at_[hidden])
Date: 2012-08-28 12:32:57
Conrad Mercer wrote:
> I have been using boost threads on 32bit linux for some time and am very
> happy with their performance so far. Recently the project was moved to a
> 64bit platform and we saw a huge increase in memory usage (from about
> 2.5gb to 16-17gb). I have done profiling and found that the boost
> threads are the source of the huge allocation. Each thread is allocating
> about 10x what it was doing on 32bit.
>
> I profiled using valgrind's massif and have confirmed the issue using only
> boost threads in a separate test application. I also tried using std::threads
> instead and these do not exhibit the large memory allocation issue.
I believe that 32-bit Linux sets the default stack size to 2 MB, and
much larger on 64-bit. So I would expect to see it using much more
_virtual_ memory - but if you don't actually use it, then no physical
memory is ever mapped to those pages and there is no performance or
other concern.
A deficiency of Boost.Thread and std::thread is that they provide no
way to change the stack size of new threads. This has been discussed a
few times here. It is possible that the libstdc++ implementation of
std::thread chooses to set a smaller default stack size.
If you need control over the stack size, and if you're not concerned
about portability to non-posix platforms, it is easy enough to write
your own thread class wrapping the pthread calls.
Regards, Phil.