The easiest approach - and the one that is generally applicably - is to use the -Verbose common parameter:
Write-Verbose -Verbose "Deleting files:"
Get-ChildItem -Path $path -Recurse -Force |
Where-Object { ! $_.PSIsContainer -and $_.CreationTime -lt $limit } |
Remove-Item -Force -Verbose
Note that in PSv3+ you can simplify this somewhat by using the -Files switch with Get-ChildItem, which directly limits the output to files only, which then allows simplification of the Where-Object call to a so-called comparison statement:
Write-Verbose -Verbose "Deleting files:"
Get-ChildItem -File -Path $path -Recurse -Force |
Where-Object CreationTime -lt $limit |
Remove-Item -Force -Verbose
If you only want to echo what would be deleted - i.e., to perform a dry run - use the -WhatIf common parameter with Remove-Item, instead of -Verbose.
Also note that I've replaced echo "Deleting files: " with Write-Verbose -Verbose "Deleting files:".
In PowerShell, echo is an alias for Write-Output, which writes to the success stream, which is where output data is meant to go.
In fact, writing to that stream is the default action, so your command could be simplified to just:
"Deleting files: " # implicitly output the string to the success stream
That said, status messages such as the above do not belong in the success stream, and using the separate stream for verbose output is one option.
Write-Verbose -Verbose generates such verbose output explicitly, but the usual mechanism is to let a function / cmdlet's -Verbose common parameter or the $VerbosePreference preference variable drive whether verbose output should be shown or not.