C++ way of creating a structure with variable number of items

Question

I need to create a memory region with variable number of items at the end. Something that I can write like this:

#pragma pack(push,0)
struct MyData
{
    int noOfItems;
    int buffer[0];
};
#pragma pack(pop)

MyData * getData(int size)
{
     bufferSize = size* sizeof(int) + sizeof(MyData ::noOfItems);
     MyData * myData= (MyData *)new char[m_bufferSize];
     return myData;
}

This code works on VS 2015 with a warning that array of size of zero is not standard

A bit of search showed me that this is a C hack and is not supported on C++.

Array with size 0

Array of zero length

What happens if I define a 0-size array in C/C++?.

How can I do this in C++

Since C is essentially a subset of C++, it is often useful in low-level C++ to just use the C idioms. Especially in gcc/g++, which exists on almost any CPU platform on earth. It may not comply with C++ standard, but still extremely useful and ubiquitous. — Erik Alapää, May 29 '18 at 09:06
As an additional comment, the struct hack is well-known and can be done in C++ using a compiler extension. C FAQ: http://www.c-faq.com/struct/structhack.html C++ discussion: https://stackoverflow.com/questions/20290293/struct-hack-equivalent-in-c — Erik Alapää, May 29 '18 at 10:17

Daniel Langr · Accepted Answer · 2018-05-28T14:06:36.180

What about something like this? It works for trivially copyable types:

template <typename T, typename INT = size_t>
class packed_array {
  public:
    packed_array(INT size) {
      buf_ = new char[sizeof(INT) + size * sizeof(T)];
      memcpy(buf_, &size, sizeof(INT));
    }
    ~packed_array() { delete[] buf_; }

    void set(INT index, T val) {
      memcpy(buf_ + sizeof(INT) + index * sizeof(T), &val, sizeof(T));
    }
    T get(INT index) const {
        T temp;
        memcpy(&temp, buf_ + sizeof(INT) + index * sizeof(T), sizeof(T));
        return temp;
    }

    const char* data() const { return buf_; }

  private:
    char* buf_;

  static_assert(std::is_trivially_copyable<T>::value);
};

int main() {
  int n;
  std::cin >> n;
  packed_array<double, int> a(n);
  for (int i = 0; i < n; i++)
    a.set(i, pow(2.0, i)); 
  for (int i = 0; i < n; i++)
    std::cout << a.get(i) << std::endl;
}

Live demo: https://wandbox.org/permlink/Vc4ok756R1Sxieoj

Yakk - Adam Nevraumont · Answer 2 · 2018-05-28T15:01:30.763

template<class T, class D>
struct var_array_at_end {
  var_array_at_end( std::size_t N ) {
    ::new( (void*)data() ) std::aligned_storage_t<sizeof(T)*N, alignof(T)>;
    for (std::size_t i = 0; i < N; ++i) {
      ::new( (void*)( data()+sizeof(T)*i) ) ) T();
    }
  }
  char* data() { return reinterpret_cast<char*>(this)+sizeof(D); }
  char const* data() const { return reinterpret_cast<char*>(this)+sizeof(D); }
  T* ptr(std::size_t i = 0) { return reinterpret_cast<T*>( data()+sizeof(T)*i ); }
  T const* ptr(std::size_t i = 0) const { return reinterpret_cast<T*>( data()+sizeof(T)*i ); }
  T& operator[](std::size_t n) {
    return *ptr(n);
  }
  T const& operator[](std::size_t n) const {
    return *ptr(n);
  }
};

struct MyData:
  var_array_at_end<int, MyData>
{
private:
  explicit MyData( int count ):
    var_array_at_end<int, MyData>( count>0?(unsigned)count:0 ),
    noOfItems(count)
  {}
  struct cleanup {
    void operator()(MyData* ptr) {
      char* buff = reinterpret_cast<char*>(ptr);
      ptr->~MyData();
      delete[] buff;
    }
  };
public:
  using up = std::unique_ptr<MyData*, cleanup>;
  static up create( int count ) {
    if (count < 0) count = 0;
    std::unique_ptr<char> buff = std::make_unique<char[]>( sizeof(MyData)+sizeof(int)*count );
    auto* ptr = ::new( (void*)buff.get() ) MyData( count );
    (void)buff.release();
    return up( ptr, {} );
  }
  int noOfItems;
};

MyData * getData(int size)
{
   return MyData::create(size).release(); // dangerous, unmanaged memory
}

I believe this is standard compliant, assuming your implementation doesn't add padding on arrays of trivial types (like char). I am unware of any implementation thta does.

I didn't assume that MyData only contains plain old data; you could stuff a std::vector into it with the above. I might be able to simplify a few lines with that assumption.

It is more than a bit of a pain.

auto foo = MyData::create(100) creates a unique ptr to MyData that has a buffer of 100 ints after it. (*foo)[77] accesses the 77th element of the buffer.

Due to a defect in the standard, you there isn't an array after MyData but rather a buffer containing 100 distinct int objects in adjacent memory locations. There are vrey annoying differences between those two things; naive pointer arithimetic is guaranteed to work within arrays, but not between packed adjacent ints in a buffer. I am unaware of a compiler that enforces that difference.

I don't think it's _standard compliant_. You can `reinterpret_cast` a pointer of some type into a pointer of different type, but dereferencing of such pointer would be undefined behavior. See, e.g., https://stackoverflow.com/questions/39429476/reinterpret-cast-char-and-undefined-behavior. — Daniel Langr, May 28 '18 at 13:54
@DanielLangr Which reinterpret, exactly, are you talking about? I am pretty sure about most of them. You can do pointer arithmetic on `char*` even if there isn't any `char`s there. And if you look at the constructor, there is a `MyData` and a bunch of `int`s stuffed into a contiguous buffer asI create them using placement `new` in `MyData::create`. The only one I'm leery of is the one in `cleanup` where I destroy the `MyData` object, then feed the `char` array to `delete[]` -- I think the `char[]` array can persist while I create other objects in it, but I could be wrong. — Yakk - Adam Nevraumont, May 28 '18 at 14:58
Hmm, possibly `auto* ptr = ::new( (void*)buff.get() ) MyData( count );` accessing the rest of `buff` within `MyData::MyData` might have issues. It depends on if the `this` pointer in `MyData::MyData` is pointer-interconvertible with the `void*` passed to placement-`new`. — Yakk - Adam Nevraumont, May 28 '18 at 15:03
I see, I was kind of lost in your code, it's quite complicated. Why don't you simply allocate a plain char buffer and then use placement new to construct elements? The only thing you need care is proper alignment (and exception safety). You would be then able to destruct the elements simply by explicit destructor calls (the same way `std::vector` does). — Daniel Langr, May 29 '18 at 07:35
@DanielLangr I do allocate a plain char buffer -- in `create`. I then placement new `MyData` in that buffer. It then proceeds to placement new the elements *after* it. I guess I could create those elements in `create` instead of in `var_array_at_end`; I was trying to move more of that code into a parent class that handles those details. Not very successfully, as `MyData` remains complex. — Yakk - Adam Nevraumont, May 29 '18 at 13:45

C++ way of creating a structure with variable number of items

2 Answers2