First off, hiding ToString is asking for trouble, dont do it; it is pretty confusing that string.Format("{0}", email) or $"{email} prints out something different than email.ToString().
Second, emailString1 and emailString2 can not be the same as you claim. The output of email.ToString() is "joe@bloggs.com", the output of $"{email}" is "{namespace}.EmailAddress".
Why the difference? It's because you are hiding ToString in ValueBase. A call of the type EmailAddress.ToString (a call through an EmailAddress typed reference) will call the new implementation of ToString but a call Object.ToString (a call through an object typed reference) will call the implementation in Object; $"{email}" is equivalent to string.Format("{0}", email) which is morally equivalent to string.Format("{0}", (object)mail) so you are really invoking ToString form an object typed reference;
What you need to do is override ToString and you will get the expected behavior:
public abstract class ValueBase
{
public override abstract string ToString();
}
Any call to an EmailAddress.ToString, no mater the type of the reference will call the overriden implementation.
Note that you seem to be conflating new and override:
Why does ... return a string of the type name (Namespace.EmailAddress), not the overridden ToString method (joe@bloggs.com)?
You are not overriding ToString, you are hiding it. To see the difference between new and override read this SO question.