TL;DR: The short time format of your user locale on the system is stored under the registry key HKCU:\Control Panel\International in the value sShortTime. You can edit the data of this value by the following PowerShell command:
Set-ItemProperty -Path 'HKCU:\Control Panel\International' -Name 'sShortTime' -Value 'HH:mm'
The change will be visible after the UI updates, e. g. when a minute passes. You might want to change sTimeFormat, too, as it corresponds to the LongTimePattern property.
Set-Culture is designed to set/load predefined objects of the CultureInfo class. You normally use it like this:
Set-Culture -CultureInfo 'en-US'
If you retrieve a CultureInfo object and then change its properties, you do not create a new CultureInfo object. It especially still has the name of a predefined CultureInfo object (e. g. en-US). If you use such a modified CultureInfo object with Set-Culture, I think the behaviour is undefined (I'm not sure). You can see that it accepts to overwrite the LongTimePattern property, but not the ShortTimePattern property. I don't know if this is a bug or just undefined.
Nevertheless, you can define your own custom CultureInfo object, derive it from a predefined object, and use this on your system instead. You can read more about that here. An example implementation in C# is shown here. You can translate this to PowerShell like follows (execute it with administrative privileges):
Add-Type -AssemblyName sysglobl
$carib = New-Object -TypeName System.Globalization.CultureAndRegionInfoBuilder -ArgumentList @('en-US-Custom', [System.Globalization.CultureAndRegionModifiers]::None)
$carib.LoadDataFromCultureInfo([System.Globalization.CultureInfo]::GetCultureInfo('en-US'))
$carib.LoadDataFromRegionInfo((New-Object -TypeName System.Globalization.RegionInfo -ArgumentList 'en-US'))
$carib.GregorianDateTimeFormat.LongTimePattern = 'HH:mm:ss'
$carib.GregorianDateTimeFormat.ShortTimePattern = 'HH:mm'
$carib.Register()
After that, you have a custom CultureInfo object - named en-US-Custom - available on your system and you can set it with:
Set-Culture -CultureInfo 'en-US-Custom'
If you want to delete your custom CultureInfo object, you can do it like this:
Add-Type -AssemblyName sysglobl
[System.Globalization.CultureAndRegionInfoBuilder]::Unregister('en-US-Custom')
Now you can decide, if you want to define a custom CultureInfo object or just change one or two values in the registry. In your case, I recommend to change the registry, as this is exactly what happens, when you change the short time pattern in the GUI:
