Because the specs say so:
enum-declaration:
    attributesopt
    enum-modifiersopt enum identifier enum-baseopt
    enum-body ;opt
enum-base:
    : integral-type
enum-body:
    { enum-member-declarationsopt }
    { enum-member-declarations , }
Each enum type has a corresponding integral type called the underlying type
of the enum type. This underlying type must be able to represent all the
enumerator values defined in the enumeration. An enum declaration may
explicitly declare an underlying type of byte, sbyte, short, ushort, int, uint,
long or ulong. Note that char cannot be used as an underlying type. An enum
declaration that does not explicitly declare an underlying type has an
underlying type of int.
...
integral-type is defined as,
integral-type:
    sbyte
    byte
    short
    ushort
    int
    uint
    long
    ulong
    char