PowerShell has two types of terminating errors, which the documentation currently conflates, unfortunately:
Statement-terminating errors, as reported by compiled cmdlets (as opposed to written-in-PowerShell advanced functions)[1] and .NET methods, which only terminate the current statement by default; that is, by default execution of the script continues, with the next statement.
 
Script-terminating (runspace-terminating, fatal) errors, as raised by throw and when $ErrorActionPreference = 'Stop' is in effect or when -ErrorAction Stop is passed to cmdlets and advanced functions / scripts and a non-terminating error occurs.
 
The third error type - the most common one - is a non-terminating error, which doesn't affect the control flow at all: by default even processing of the current statement (with remaining pipeline input) continues. As for when cmdlets are expected to use non-terminating vs. terminating errors, see this answer.
Note:
Arguably, there should only ever have been one type of terminating error: script-terminating (fatal), which, if needed, could be handled as merely statement-terminating, with try / catch (see below), but it's too late to change that.
 
For a comprehensive overview of PowerShell's surprisingly complex error handling, see this GitHub docs issue.
 
ConvertFrom-Json, as a compiled cmdlet, emits a statement-terminating error when invalid JSON is provided as input, which explains why execution of your script continues.
You can use try / catch or trap to catch both statement- and script-terminating errors - but not non-terminating ones by default, unless you promote them to script-terminating ones with -ErrorAction Stop.
By contrast, $ErrorActionPreference = 'Stop' promotes all errors to script-terminating (fatal) ones, including non-terminating ones.
Confusingly, the seeming per-command equivalent, namely passing -ErrorAction Stop to an individual cmdlet, does not promote a statement-terminating error to a script-terminating one - you still need try / catch or trap for that - see below.
The upshot:
- To treat all errors occurring in your scope - including non-terminating ones - as fatal, use 
$ErrorActionPreference = 'Stop' 
# Treat *all* errors in this scope (and descendant scopes)
# as *script*-terminating (fatal).
$ErrorActionPreference = 'Stop'
# Trigger a *non*-terminating error, which is *also* promoted
# to a script-terminating one.
Get-Item NoSuchFile 
# Trigger a *statement*-terminating error, which is promoted
# to a script-terminating one.
# Note: Because of the Get-Item error, this statement isn't reached, 
#       Comment it out to see that this statement too causes a
#       script-terminating error.
$hashtableJson = "asdfg.kjhgf" | ConvertFrom-Json
'This statement is now never reached.'
# Trap both kinds of *terminating* errors and abort.
trap { break }
# Trigger a *non*-terminating error, which is left alone (not
# caught by the trap).
Get-Item NoSuchFile 
# Trigger a *statement*-terminating error, which triggers the trap.
$hashtableJson = "asdfg.kjhgf" | ConvertFrom-Json
'This statement is now never reached.'
- Alternatively, especially if you need you more fine-grained control, use 
try / catch around statements of interest and use throw from the catch block to trigger a script-terminating error. 
try {
  # Trigger a *non*-terminating error, which is left alone (does
  # not trigger the catch block).
  Get-Item NoSuchFile 
  # Trigger a *statement*-terminating error, which triggers the catch block.
  $hashtableJson = "asdfg.kjhgf" | ConvertFrom-Json
} catch {
  throw # Re-throw the *statement*-terminating error as *script*-terminating.
}
'This statement is now never reached.'
[1] It is possible to make an advanced function / script emit a statement-terminating error, namely via $PSCmdlet.ThrowTerminatingError(), but that is both obscure and cumbersome; in practice, it is far more common to use the script-terminating throw statement.