Did you delete it already from your "Keyboard language from Regional settings"? Or is it still visible under the "Installed services"? See picture at the bottom.
If it's not under there (or you deleted it) and after a restart it comes back as an "Installed service" it indicates another problem.
If you delete it from "Installed services" it should not come back.
But if you really want to delete the keyboard via the registry you can.
First make sure the keyboard is not in the "Installed services". (See picture below)
Then, in regedit.exe goto the key
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Keyboard Layouts\00010409
It should say Layout Text = United States-Dvorak on the right. You could delete the whole 00010409 key. (You can also rename the key to dddd_00010409. Windows doesn't recognize it again as keyboard layout.) No need to restart. You won't be able to add United States-Dvorak again.
BTW there is also a 00030409 United States-Dvorak for left hand and
0040409 United States-Dvorak for right hand in my Windows version.
