There are two ways because they are driver-specific commands (there is no generic "remove this device" command on Linux), and of course the two examples deal with different device types handled by different device drivers. (It's sort of how Windows has the StorAhci.sys driver for SATA and StorNvme.sys for NVMe.)
The point is that sd* doesn't just mean "disk", it specifically means "SCSI disk", i.e. a device that speaks the SCSI command set and is handled by the SCSI driver. It's used for USB disks because those are always SCSI under the hood.
On modern Linux, the SCSI driver also happens to be used for ATA disks because it turned out to be more robust to do it that way (SATA shares concepts with SCSI and there's an 1:1 translation for most SCSI-ATA commands). But until just a few years ago, ATA disks had their own driver and an internal HDD would've been named hd* instead (/dev/hda, etc) – and they would have had their own structure under /sys as well.
NVMe, on the other hand, is vastly different from SCSI and ATA, so while Linux did also have a similar sd* translation layer for a short while, it was ultimately found better to let NVMe disks have their own device type and to have tools directly speak NVMe commands. Therefore NVMe disks are named nvme*.
(Internally, both SCSI and NVMe also have more complex structure than just "cable goes to disk, disk holds sectors"; for example, SCSI has 'LUNs' and NVMe has 'namespaces', both of which allow a single disk to present as multiple but in different ways; and the corresponding Linux drivers end up representing them in very different ways for probably a combination of many reasons – so it is not surprising to see differences in the way their /sys paths are laid out.)
It also seems that for NVMe you're starting off with a partition device rather than a whole-disk device, so when you go "upwards" through the .../device symlinks in /sys, there are naturally two levels from 'partition' to 'whole disk' to 'controller' instead of just one.
As for delete vs remove, it's probably just a random inconsistency resulting from different people implementing it in different kernel subsystems at different times.