"On the heap" is, pedantically, a bit of a misnomer. C++ does not dictate the use of a heap, only that objects that are allocated using new are allocated on the unspecified "free-store". Anyway...
Yes, if B has a member variable of type A and you instantiate a B using new then all of that object is on the free-store.
It's important to note however that when you do:
B* b = new B;
b itself is not on the free-store, but is rather an automatic variable (ie "on the stack"). The thing that b points to is on the free-store. Automatic variables are destroyed when they go out of scope -- but the thing b points to in this case will not be. If you don't delete b;, you'll get a memory leak.
This may be confusing and seem unimportant. Maybe it is confusing, but it isn't unimportant. The fact that b is an automatic variable that points to something on the free-store makes possible a very important idiom known as RAII, which is used for things like smart pointers.
shared_ptr<B> b(new B);
b, here, is still an automatic variable. But when b goes out of scope and is destroyed, it takes the thing b points to with it. Using smart pointers here will help to eliminate the memory leak created when you didn't delete b above.