I had a similar case with a "non-removable" English (Hong Kong SAR) keyboard, when my display language was English (United States).
And then I found where a setting where it was "hiding":
Language & Region > Administrative Language Settings > Administrative > Welcome screen and new user accounts > Copy settings...
(The welcome screen used that keyboard)
After applying "Copy your current settings to..." (tick both boxes) and restarting, it disappeared.
I also vaguely remember a case in Windows 10 where, if I choose "Regional Format" to be "English (Hong Kong SAR)", the keyboard will appear, but I can't reproduce it in my Windows 11 PC.