Compo has provided the crucial pointer:
When calling PowerShell's CLI, powershell.exe, from the outside:
- escape as
\" those " chars. that you want to be retained as part of the PowerShell command to execute - see this answer for more information.
In short: replace "" with \"\":
powershell -command "echo kali | ssh -t kali:kali@192.168.158.129 'sudo -S bash -c \"\"echo 123 >> /etc/123\"\" '"
Note that the quoting you're using from inside PowerShell,
'sudo -S bash -c ""echo 123 >> /etc/123"" ',
only works due to a long-standing bug with respect to passing arguments with embedded " to external programs, such as ssh:
PowerShell places verbatim "sudo -S bash -c ""echo 123 >> /etc/123"" " on the process command line constructed behind the scenes, i.e. it neglects to escape the embedded " in the overall "..." string, which results in ssh parsing the argument as verbatim sudo -S bash -c "echo 123 >> /etc/123" (as desired), because it understands "" inside "..." as a single, escaped " - while many CLIs on Windows do, the most widely supported escape syntax is \".
See this answer for details.
In PowerShell (Core) 7.3.0 and above, if $PSNativeCommandArgumentPassing = 'Standard' is in effect, simply using " as-is inside your '...' string is enough:
'sudo -S bash -c "echo 123 >> /etc/123" ',
which then causes PowerShell to automatically escape the " as \", placing the following verbatim on the behind-the-scenes process command line:
"sudo -S bash -c \"echo 123 >> /etc/123\" "
However, if you were to use the latter from cmd.exe, you'd run afoul of the latter's parsing: because cmd.exe doesn't understand \" as an escaped ", it sees the >> as being outside a "..." string and therefore interpret its itself, up front.
In such a corner case, you tan cake advantage of the fact that pwsh, the PowerShell (Core) CLI, alternatively understands "" as escaped " chars. insider overall "...":
# PowerShell (Core) only: alternatively use "" to escape " on the command line.
# !! "" is enough if $PSNativeCommandArgumentPassing = 'Standard' is in effect
# !! in your case, which by default is only true
# !! for PowerShell 7.3.0 and at least up to 7.3.1, and may change again.
# !! Otherwise, use """" (2 escaped ") in your case.
pwsh -command "echo kali | ssh -t kali:kali@192.168.158.129 'sudo -S bash -c ""echo 123 >> /etc/123"" '"
With the Windows PowerShell CLI (powershell.exe), you'd have to use "^"" (sic) instead of "" - see this answer.