Resizing vector in child thread leads to unexpected memory consumption

Question

I'm struggling with a (strange?) behavior when resizing vectors in subthreads.

The virtual memory consumption increases a lot when vector<T>.reserve is called in a subthread (or when a reallocation appends on adding elements) compared to the same code called in the main process.

I had a similar issue on vector.swap that can be solved thanks to std::swap.

In short, is there a way to safely resize a vector in a subthread, and what does cause such a behavior ?

I can put sample code if needed.

Thanks in advance ;)

EDIT

@david Haim : there is indeed a reference, and its extracted from a library with unitary tests and the original vector is modified as expected.

@user0042 : there is no synchronization but no concurrency either. Here is a small code.
The memory monitoring comes from this post

#include <iostream>
#include <vector>
#include <thread>

typedef std::vector< int > VectInt_t;

void init(VectInt_t &vec_p)
{
    vec_p.reserve(500);
}
void resize(VectInt_t &vec_p)
{
    vec_p.resize(2000);
}

// ==========================================================================
int main(int argc, char ** argv)
{
    int cas_l(0);
    if(argc > 1)
    {
        cas_l = atol(argv[1]);
    }

    VectInt_t myVec_l;
    init(myVec_l);

    //std::cout << "After init : " << std::endl
    //<< "\tVirtualMemoryUsedByCurrentProcess = " << getVirtualMemoryUsedByCurrentProcess() << std::endl
    //<< "\tPhysicalMemoryUsedByCurrentProcess = " << getPhysicalMemoryUsedByCurrentProcess() << std::endl;

    switch(cas_l)
    {
        case 0:
        {
            resize(myVec_l);
            break;
        }
        case 1:
        {
            std::thread thread_l(resize, std::ref(myVec_l));
            thread_l.join();
            std::cout << "thread done" << std::endl;
            break;
        }
    }

    //std::cout << "After resize : " << std::endl
    //<< "\tVirtualMemoryUsedByCurrentProcess = " << getVirtualMemoryUsedByCurrentProcess() << std::endl
    //<< "\tPhysicalMemoryUsedByCurrentProcess = " << getPhysicalMemoryUsedByCurrentProcess() << std::endl;
    return 0;
}

The result gives (on debian 9) :

$ ./minCase 0
After init : 
    VirtualMemoryUsedByCurrentProcess = 15252
    PhysicalMemoryUsedByCurrentProcess = 1724
After resize : 
    VirtualMemoryUsedByCurrentProcess = 15252
    PhysicalMemoryUsedByCurrentProcess = 1724


$ ./minCase 1
After init : 
    VirtualMemoryUsedByCurrentProcess = 15252
    PhysicalMemoryUsedByCurrentProcess = 1688
thread done
After resize : 
    VirtualMemoryUsedByCurrentProcess = 88984
    PhysicalMemoryUsedByCurrentProcess = 1700

my guess: you're copying the vector into the new thread rather than pass a reference to it. — David Haim, Oct 25 '17 at 10:21
My guess would be, it's the act of creating a thread rather than the act of resizing a vector that leads to memory increase. Probably when you create your first thread, some OS library (`pthreads`?) gets loaded into the process. Would be easy to test by giving `std::thread` some no-op function to call, in place of `resize` — Igor Tandetnik, Oct 25 '17 at 13:45
@IgorTandetnik actually on big instances my memory consumption goes from 4Gb to > 6Gb with only 4 threads created — Nael, Oct 25 '17 at 13:49
On big instances of what? I'm not sure I understand this sentence. — Igor Tandetnik, Oct 25 '17 at 13:51
@IgorTandetnik this code is a minimalist extract. The original issue comes from another project of mine, where tons of vectors are resized. When done in subthreads the memory increases a lot (roughly 50%), even when only one thread is used. — Nael, Oct 25 '17 at 14:01
tcmalloc and jemalloc improve thing a little bit, from 25720 to 38012.But I don't understand what happen here — kinnou02, Oct 26 '17 at 09:36
Are you short of virtual memory on your system? (ie, is this a 32 bit system)? Virtual memory being allocated is nearly free, it is just an address *range* reserved for use by your process in an address space that only your process consumes. Assuming this isn't some random artifact of your code, it could easily be that your memory allocator avoids contention by presuming threads tend to allocate memory and deallocate memory internally, so uses a different address range for memory allocated in a different thread. — Yakk - Adam Nevraumont, Oct 26 '17 at 14:41
@Yakk I'm currently doing the memory operations in the main thread (and it's fast enough), and only the expensive computation in the subthreads. So to be honest, I've got a workaround efficient enough. However, if I let everything in the subthreads, the memory consumption can lead to a lot of swap and a not negligible performance degradation. And most importantly, I'm eager to understand what's going on in there :D — Nael, Oct 26 '17 at 14:55
@Nael There being more swap can be related to virtual memory use, but you complained about virtual memory use not swap problems. Do you have a lot of swapping going on? How did you measure it? Can you generate a [MCVE] where you have 2 versions of code that has the same threads, but one does allocation in a subthread and you get a lot more swapping? Virtual memory can include address space marked as "reserved for use", but need not exist on disk or in physical memory. I suspect that may be what you are seeing. — Yakk - Adam Nevraumont, Oct 26 '17 at 14:57
@Yakk I'll try to make an example, and will investigate by myself tomorrow (I hope) The swap is seen thx to top, it goes from almost nothing to 1765500KiB (with a %MEM at 81.1%, ie. ~ 6Gb) — Nael, Oct 26 '17 at 16:05

Resizing vector in child thread leads to unexpected memory consumption

0 Answers0