C++ Flexible Array Members (sort of): is this UB?

Question

Does this code lead to Undefined Behavior if we want to use the character array returned by buf()? I know C99 allows flexible array members but ISO C++ doesn't. I just came across this 'trick' to basically do the same in C++. Just wondering if it's safe to use.

struct Foo {
  size_t len;
  size_t cap;
  char* buf() { return reinterpret_cast<char*>(this + 1); }
};

Foo* new_foo(size_t sz) {
  Foo* foo = reinterpret_cast<Foo*>(malloc(sz + sizeof(Foo)));
  foo->len = 0;
  foo->cap = sz;
  return foo;
}

What are you even trying to do, break C++ on purpose? Yes it is UB, reinterpret_cast will not start the lifetime of Foo (nor will malloc). You might be in luck with bit_cast but again. WHAT are you trying to do, this certainly is going nowhere (fast). Also in general don't write code that make any assumptions on memory layout — Pepijn Kramer, Jun 22 '23 at 17:06
The `foo` object needs to be created. See https://stackoverflow.com/questions/222557/what-uses-are-there-for-placement-new — nielsen, Jun 22 '23 at 17:06
I use a similar technique myself, it's UB, but it works on my platform. You can construct the object using placement new (and even the flexible array as well) but it's still UB. You should disable copying of `Foo`. — john, Jun 22 '23 at 17:07
There's no UB in this code by itself, but there's plenty of opportunity for UB. Constructing a `Foo` produces an object that returns an invalid pointer. Copying any `Foo` produces an object that returns an invalid pointer. — Drew Dormann, Jun 22 '23 at 17:08
you can reinterpret cast a lot without invoking ub, but then using the casted pointer can easily lead to ub. — 463035818_is_not_an_ai, Jun 22 '23 at 17:08
please recheck if you want this question tagged `language-lawyer`. The tag usually implies that you are asking for quotes from the standard directly. If you would be fine with layman terms answers the tag is better removed. — 463035818_is_not_an_ai, Jun 22 '23 at 17:11
@DrewDormann: `new_foo` writes to `foo->len` without constructing a `Foo` there, which is undefined behavior. I think `buf()` is valid though — Mooing Duck, Jun 22 '23 at 17:11
Do you want the language lawyer answer, or the practical answer? In practice, this is usually done by putting a length 1 array named `buf` at the end of the struct, to keep it plain data, no methods, no casting, no use of any C++ thing so it remains legal in the grey area of code that, in practice, works in both. As a bonus, for C-style strings, this means the `NUL` terminator (which you usually want for compatibility with C string APIs) is part of the `struct` size, and you only need to increase the size by the number of actual characters in the string. But it's still not true C++. — ShadowRanger, Jun 22 '23 at 17:12
@MooingDuck does C++20 not introduce "implicit object creation" for types such as this? — Drew Dormann, Jun 22 '23 at 17:14
I wonder how this is any better than having a bare `char*` member and dynamically allocate. I mean on the one hand you have `sz` decided at runtime, but then any chunck for such allocated `Foo` is different size. Afaik real flexible array members do not result in effecting `sizeof(Foo)` — 463035818_is_not_an_ai, Jun 22 '23 at 17:15
Oh, and one other benefit to the length 1 array approach if you design it a little differently: No wasted memory in padded structs. With your approach, if the `struct` ends with pad bytes due to the alignment of the named members, you allocate more than you need, and never use it. With a `char buf[1];` (and proper use of `offsetof` rather than `sizeof` for determining how much to allocate), the array can begin immediately after the last member, no padding, at all. This is how CPython handles stuff like their `bytes` type, and it's portable to everywhere CPython targets, which is pretty decent. — ShadowRanger, Jun 22 '23 at 17:22
FYI, this is an old C language pattern. The equivalent in C++ is to base class and inheritance for the variable type field. Although you could use a union, the preferred method is to use a pointer to a base class, then point to various child classes. — Thomas Matthews, Jun 22 '23 at 18:08

HolyBlackCat · Accepted Answer · 2023-06-22T17:37:11.847

Before C++20, you would need to substitute Foo* foo = reinterpret_cast<Foo*>(malloc(sz + sizeof(Foo))); for Foo* foo = ::new(malloc(sz + sizeof(Foo)) Foo;. In C++20 and newer your class is an implicit-lifetime class, and as such it doesn't need to be created manually in malloced memory. I wouldn't rely on this C++20 feature, as it's rather obscure.

Don't forget to call the destructor before freeing the memory. Read about std::launder and consider if you need to use it or not.

This is technically still UB, but only because the standard is defective in its description of reinterpret_cast. No compiler is going to enforce this.

Also the pointer reachibility rules for std::launder somewhat imply that it could be UB, but it's not stated explicitly, and again, no contemporary compiler enforces this.

Consider overloading operator new for your class to have a semblance of a good class allocation interface:

#include <new>
#include <iostream>

struct Foo
{
    size_t len = 0;
    size_t cap = 0;

    char *buf()
    {
        return reinterpret_cast<char*>(this + 1);
    }

    void *operator new(std::size_t size, std::size_t cap)
    {
        return ::operator new(size + cap);
    }

    // Only if you need custom cleanup in `delete`:
    // void operator delete(Foo *ptr, std::destroying_delete_t)
    // {
    //     // Destroy array elements here.
    //     ptr->~Foo();
    //     ::operator delete(ptr);
    // }
};

Foo *new_foo(size_t sz)
{
    Foo* foo = new(sz) Foo;
    foo->len = 0;
    foo->cap = sz;
    return foo;
}

Now new Foo causes a compilation error, while new(cap) Foo works and forces you to allocate enough memory for the element. Though ::new Foo still works, there is no way to disable it.

If you uncomment the destroying operator delete, you can hook into delete foo to insert the cleanup for the elements.

Note that one minor issue with using `new`/`::operator new` is that it's impossible to even try to reallocate the storage in place. If the `cap` is intended to be `const`, then that's fine (though `cap` should be marked as such and initialized properly), but I wouldn't be surprised if the use of `malloc` was, at least in part, to enable resizing (possibly in place if `realloc` cooperates) if the capacity gets exhausted. — ShadowRanger, Jun 22 '23 at 17:26
The issue here is different from the issue in the other question. Here, because the trailing buffer is indeed not reachable from the `Foo` object, I think CWG actually does intend for this to be UB: it is not a wording defect. If you could get from an object to an unrelated object using `reinterpret_cast` and pointer arithmetic, many reachability-based optimizations would be impossible. — Brian Bi, Jun 24 '23 at 01:10
The UB-free strategy is to pass around a pointer to the entire buffer, from which you can get to either the `Foo` object or the trailing buffer (with `std::launder` being required in the first case). — Brian Bi, Jun 24 '23 at 01:10
@BrianBi Yep, those are two unrelated issues, I've mentioned this one too (*"pointer reachibility rules for std::launder"*). I'm fairly sure no compiler optimizes based on this (reachibility of enclosing object from a subobject; I'm not talking about unrelated objects), but I'd be happy to be proven wrong. — HolyBlackCat, Jun 24 '23 at 09:21

C++ Flexible Array Members (sort of): is this UB?

1 Answers1