Olaf's answer contains good pointers, but let me try to boil it down conceptually:
There are two unrelated problems with your attempt:
Inside expandable strings ("...") only simple variable references ($Installer) can be used as-is; expressions ($Installer.FullName) require $(), the subexpression operator: $($Installer.FullName) - see this answer for an overview of expandable strings (string interpolation) in PowerShell.
Since you're passing the arguments for msiexec via -ArgumentList as a single string, only embedded double quoting, "...", is supported, not '...' (single quoting).
Therefore, use the following (for brevity, the -FilePath and -ArgumentList arguments are passed positionally to Start-Process):
Get-ChildItem $Path.DirectoryName -Recurse -Filter *.MSI | ForEach-Object {
Start-Process -Wait C:\windows\system32\msiexec.exe "/i `"$($_.FullName)`""
}
Note:
The -ArgumentList parameter is array-typed ([string[]]). Ideally, you'd therefore pass the arguments individually, as the elements of an array: '/i', $_.FullName, which wouldn't require you to think about embedded quoting inside a single string.
Unfortunately, Start-Process doesn't handle such individually passed arguments properly if they contain embedded spaces, so the robust solution is to use a single -ArgumentList argument comprising all arguments, with embedded double-quoting, as necessary, as shown above.
See this answer for a more detailed discussion.