Because this is bound to the global object in that case. 
You're calling this function
function() {
        alert(this.msg);
    }
when it is not bound to an object.  So this will refer to the global object (which is window in the browser) and since that won't have a msg property, it will alert undefined.
When you call a = new A() you create a new object, add msg as a property, and set foo() on its prototype chain.  So when you call a.foo() foo is bound to a and this refers to a.
In general you probably want something that looks more like this.
function A() {
    this.msg = 'meuahah';
}
A.prototype.foo = function() {
    alert(this.msg);
}
function B() {
    A.call(this);
}
B.prototype = Object.create(A.prototype);
B.prototype.bar = function() {
    this.foo();
}
You can read more about how this works in javascript from this question