1

The program below is repeatedly creating 0.1 MB vectors in shared memory. The shared memory segment grows by 1 MB each time there isn't enough space left to create or resize a vector. Eventually the program crashes on the vec->resize call (marked in the code) with a SIGBUS error. The stack trace is below. The crash usually happens when the shared memory segment is around 8G. I have 16G of RAM and only about 1G of it is being used by other processes besides this one.

Am I growing the memory or using the boost::interproces::vector incorrectly here or could there be some problem in the boost library itself?

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/sync/named_mutex.hpp>
#include <boost/interprocess/sync/scoped_lock.hpp>
#include <boost/interprocess/containers/string.hpp>
#include <boost/container/scoped_allocator.hpp>
#include <boost/interprocess/containers/vector.hpp>
#include <iostream>
#include <unistd.h>

namespace bip = boost::interprocess;  
template <typename T>
using Alloc   = bip::allocator<T, bip::managed_shared_memory::segment_manager>;
typedef bip::vector<uint8_t, boost::container::scoped_allocator_adaptor<Alloc<uint8_t> > > DataVector;


int main(int argc, char** argv){
  std::string shm_name = "test";
  int size_increment = 1000000; // 1 MB
  int vec_size = 100000; // 0.1 MB
  assert(vec_size < size_increment);

  bip::shared_memory_object::remove(shm_name.c_str());

  bip::managed_shared_memory* segment = new bip::managed_shared_memory(bip::open_or_create, shm_name.c_str(), size_increment);

  for(int i = 0; true; i++){
    std::cout << i << std::endl;
    std::string var_name = std::to_string(i);
    DataVector* vec = NULL;

    // create the vector
    try{
      vec = segment->find_or_construct<DataVector>(var_name.c_str())(segment->get_segment_manager());
    }
    catch(std::exception& ex){
      // grow
      std::cout << "1 growing: " << ex.what() << std::endl;
      delete segment;
      bip::managed_shared_memory::grow(shm_name.c_str(), size_increment);
      segment = new bip::managed_shared_memory(bip::open_only, shm_name.c_str());
      std::cout << "1 done growing" << std::endl;

      vec = segment->find_or_construct<DataVector>(var_name.c_str())(segment->get_segment_manager());
    }

    // make sure the vector isn't null
    if(vec == NULL){
      std::cout << "1 vec was NULL" << std::endl;
      return 1;
    }

    // grow the vector
    try{
      vec->resize(vec_size); // SIGBUS crash happens here
    }
    catch(std::exception& ex){
      // grow
      std::cout << "2 growing: " << ex.what() << std::endl;
      delete segment;
      bip::managed_shared_memory::grow(shm_name.c_str(), size_increment);
      segment = new bip::managed_shared_memory(bip::open_only, shm_name.c_str());
      std::cout << "2 done growing" << std::endl;

      vec = segment->find<DataVector>(var_name.c_str()).first;
      if(vec == NULL){
    std::cout << "2 vec was NULL" << std::endl;
    return 1;
      }
      vec->resize(vec_size);
    }

    usleep(1000); // sleep 1ms
  }


  return 0;
}

Program terminated with signal SIGBUS, Bus error.
#0  __memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:141
141 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: No such file or directory.

#0  __memset_avx2_erms () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:141
#1  0x0000562cec17143c in boost::container::uninitialized_value_init_alloc_n<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>(boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>&, boost::container::allocator_traits<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::size_type, unsigned char*) (n=100000, r=0x7fd0d0c1ce90 <error: Cannot access memory at address 0x7fd0d0c1ce90>)
    at /usr/include/boost/container/detail/copy_move_algo.hpp:588
#2  0x0000562cec16e9b5 in boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>::uninitialized_copy_n_and_update(boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>&, unsigned char*, unsigned long) const (this=0x7ffc3a961780, a=..., p=0x7fd0d0c1ce90 <error: Cannot access memory at address 0x7fd0d0c1ce90>, 
    n=100000) at /usr/include/boost/container/detail/advanced_insert_int.hpp:126
#3  0x0000562cec16e6a3 in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::priv_forward_range_insert_new_allocation<boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*> >(unsigned char*, unsigned long, unsigned char*, unsigned long, boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>) (this=0x7fd0d0c1ce58, new_start=0x7fd0d0c1ce90 <error: Cannot access memory at address 0x7fd0d0c1ce90>, new_cap=100000, pos=0x0, n=100000, insert_range_proxy=...) at /usr/include/boost/container/vector.hpp:2943
#4  0x0000562cec16c0ad in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::priv_forward_range_insert_no_capacity<boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*> >(boost::interprocess::offset_ptr<unsigned char, long, unsigned long, 0ul> const&, unsigned long, boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>, boost::move_detail::integral_constant<unsigned int, 1u>) (this=0x7fd0d0c1ce58, pos=..., n=100000, insert_range_proxy=...) at /usr/include/boost/container/vector.hpp:2682
#5  0x0000562cec16a6f8 in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::priv_forward_range_insert<boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*> >(boost::interprocess::offset_ptr<unsigned char, long, unsigned long, 0ul> const&, unsigned long, boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>) (this=0x7fd0d0c1ce58, pos=..., n=100000, insert_range_proxy=...) at /usr/include/boost/container/vector.hpp:2744
#6  0x0000562cec168ff4 in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::priv_forward_range_insert_at_end<boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>, boost::move_detail::integral_constant<unsigned int, 1u> >(unsigned long, boost::container::container_detail::insert_value_initialized_n_proxy<boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >>, unsigned char*>, boost::move_detail::integral_constant<unsigned int, 1u>) (this=0x7fd0d0c1ce58, n=100000, insert_range_proxy=...) at /usr/include/boost/container/vector.hpp:2774
#7  0x0000562cec167e43 in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::priv_resize<boost::container::value_init_t>(unsigned long, boost::container::value_init_t const&) (this=0x7fd0d0c1ce58, new_size=100000, u=...) at /usr/include/boost/container/vector.hpp:2602
#8  0x0000562cec167838 in boost::container::vector<unsigned char, boost::container::scoped_allocator_adaptor<boost::interprocess::allocator<unsigned char, boost::interprocess::segment_manager<char, boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, boost::interprocess::offset_ptr<void, long, unsigned long, 0ul>, 0ul>, boost::interprocess::iset_index> >> >::resize(unsigned long) (this=0x7fd0d0c1ce58, new_size=100000) at /usr/include/boost/container/vector.hpp:1460
#9  0x0000562cec164e4f in main (argc=2, argv=0x7ffc3a9620d8) at /home/username/subt/map_database_ws/src/map_database_demo/src/boost_test.cpp:53
Couch Tomato
  • 13
  • 2
  • 1
    1MB to 8GB is 8000 resize. Do you really need 8GB ? When does your program stop allocating new memory blocks ? – UmNyobe Apr 22 '20 at 23:05
  • 1
    What's the shared memory limit? `ulimit` can explain. – tadman Apr 22 '20 at 23:15
  • related: https://stackoverflow.com/questions/212466/what-is-a-bus-error – bolov Apr 22 '20 at 23:34
  • @UmNyobe This program will keep allocating until there is none left to allocate. If there is none left, boost should generate exceptions and not crash with SIGBUS. This program is just a small self contained example emulating what another program I am working on does. I will hopefully not need anywhere near 8G but I want to make sure there are no bugs with the boost interprocess library. – Couch Tomato Apr 22 '20 at 23:36
  • 2
    @tadman Boost allocates the shared memory in /dev/shm. According to `df` the max available in /dev/shm is around 8G so it looks like I am hitting this limit. Thanks for pointing that out. I still think boost should throw an exception and not crash with SIGBUS if I'm using it correctly and the boost library is implemented correctly. – Couch Tomato Apr 22 '20 at 23:44

1 Answers1

1

So, you're running into the limitations of your shm filesystem.

You can increase the size of course, but I suggest to look at the overhead of managed-memory segments. In particular the fragmentation happening with organic growth.

See e.g. Bad alloc is thrown

enter image description here

Maybe you can't reserve everything up-front, but you MAY be able to defragment when resizing the segment.

The simplest thing to do would be to create a new segment with a properly pre-allocated data-structure, and then copy the data over. When it's all done, remove the old one and rename the new one.

Obviously you may run out of virtual memory doing this, so it could be necessary/a good idea to use managed_mapped_file instead, or at least temporarily during the resize.

Other links:

sehe
  • 374,641
  • 47
  • 450
  • 633