The array constructor says:
also creates and initializes a new Array object when called as a function rather than as a constructor. Thus the function call Array(…) is equivalent to the object creation expression new Array(…) with the same arguments.
The object constructor says:
- If NewTarget is neither undefined nor the active function, then
a. Return ? OrdinaryCreateFromConstructor(NewTarget, "%Object.prototype%").
- If value is undefined or null, return ! OrdinaryObjectCreate(%Object.prototype%).
- Return ! ToObject(value).
So it's explicitly noted that new isn't required for either of those.
(though, you should probably never be using either Object or Array as constructors, whether with or without new - better to use object and array literals)
It'll be trivial to implement this sort of thing yourself in Custom, if you want - check if the function was called with new, and if not, explicitly return something:
function Custom(){
  if (!new.target) {
    return new Custom();
  }
}
console.log(Custom())
console.log(new Custom());