Powershell 'call' passing redirects to program instead of actioning it

Question

I am trying to get powershell to call an exe with a constructed list of parameters - and to pipe the response to a log file. My issue is the redirect characters seem to be passed to the exe. (I have also struggled for nearly a day to get the right quotes around the path names which are full of spaces!!!)

The syntax for the exe is

   filemeta  {-d|-i|-e} [-c] [-f=<directory name>] [-x=<file name>] [-v]
             [-p] [--] [--version] [-h] <file name> ...

as you can see repeats. The reason I think the redirects are being passed as another file name is firstly the error message Cannot find file " >>" and secondly the log file is not populated

The following command pasted into a command prompt does what is intended. Its just how to get it wrapped in a powershell file iteration

"C:\Program Files\File Metadata\FileMeta.exe" -i -f="M:\metadata_XML_tobe\ManagedCatalog\Business Records\Morse Bakery"  "M:\ManagedCatalog\Business Records\Morse Bakery\Morse Bakery Book 1-000.JPG"  >>"M:\metadata_XML_tobe\apply_doc_id_20230412-124532.log" 2>>&1 

the powershell commands so far are ...

$metaDataPath = 'M:\metadata_XML_tobe'
$timeStr = get-date -Format 'yyyyMMdd-HHmmss' 
$logFileName = $metaDataPath + "\apply_doc_id_" + $timeStr + ".log"
$files = Get-ChildItem -Path $metaDataPath -Recurse -Include *.xml
$quote = '"'
$hits  = 0
foreach ($f in $files){
    $hits++
    if ($hits -ge 5) {
        break
    }
    $directory = [System.IO.Path]::GetDirectoryName($f)
    $realFile_name = $f.FullName.Replace('.metadata.xml', '')
    $realFile_name = $realFile_name.Replace('\metadata_XML_tobe', '' )
    $parm1 = '-i'  
    $parm2 = '-f='+ $quote + $directory + $quote 
    $parm3 = $quote + $realFile_name + $quote
    $parm4 = ' >>'  
    $parm5 = $quote + $logFileName + $quote
    $parm6 = ' 2>>&1'  
    Write-Host '**************'
    Write-Host $f.FullName 
    Write-Host  '   Parm1   = ' $parm1
    Write-Host  '   Parm2   = ' $parm2
    Write-Host  '   Parm3   = ' $parm3
    Write-Host  '   Parm4   = ' $parm4
    Write-Host  '   Parm5   = ' $parm5
    Write-Host  '   Parm6   = ' $parm6
    echoargs 'C:\Program Files\File Metadata\FileMeta.exe' $parm1 $parm2 $parm3 $parm4 $parm5 $parm6
    # & 'C:\Program Files\File Metadata\FileMeta.exe' $parm1 $parm2 $parm3 $parm4 $parm5 $parm6
}

which gives output of -----

M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG.metadata.xml
   Parm1   =  -i
   Parm2   =  -f="M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948"
   Parm3   =  "M:\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG"
   Parm4   =   >>
   Parm5   =  "M:\metadata_XML_tobe\apply_doc_id_20230417-105352.log"
   Parm6   =   2>>&1
Arg 0 is <C:\Program Files\File Metadata\FileMeta.exe>
Arg 1 is <-i>
Arg 2 is <-f=M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948>
Arg 3 is <M:\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG>
Arg 4 is < >>>
Arg 5 is <M:\metadata_XML_tobe\apply_doc_id_20230417-105352.log>
Arg 6 is < 2>>&1>

or if the echoargs is swapped out

**************
M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG.metadata.xml
   Parm1   =  -i
   Parm2   =  -f="M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948"
   Parm3   =  "M:\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG"
   Parm4   =   >>
   Parm5   =  "M:\metadata_XML_tobe\apply_doc_id_20230417-105958.log"
   Parm6   =   2>>&1
Imported metadata to M:\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG from M:\metadata_XML_tobe\ManagedCatalog\Church Records\Catholic Church Euroa\Baptism Records\1904-1948\Baptism Records 1904-1948-003.JPG.metadata.xml
Cannot find file " >>"

Please help me understand this maze

JC

Answer 1

As in any shell, metacharacters such as > that aren't used literally, unquoted aren't treated as such:

# OK: > $null redirects the success-stream output to $null, i.e. 
#     *discards* it:
# NO OUTPUT, as expected.
Write-Output hi > $null

vs.

# !! Due to saving *string* value '> $null' in a variable
# !! and passing that variable, its *verbatim value* is passed.
# OUTPUTS 'hi' and '> $null'
$var = '> $null'
Write-Output hi $var

That is, using variables for specifying individual pass-through arguments for a given target executable is fundamentally different from trying to pass entire redirection expressions such as >> file via a variable.

To put it differently: You cannot use variables to pass elements of a command line that have syntactic function, such as > or |.

In order for PowerShell to recognize the redirection operator (> or >>) as such, it must be specified literally, unquoted, whereas you're free to pass the redirection target file (or $null to suppress output) as a variable; e.g.:

# Define the redirection target (output file) as a variable.
$file = 'temp file.txt'

# Now pass the variable to the - unquoted - > operator.
Write-Output hi > $file

However, if the redirection target is the number of another stream (e.g., 2>&1 in order to merge the error stream (2) into the success stream (1)), that number can not be specified via a variable.

$streamNumber = 1

# !! Does NOT work and triggers a *syntax error*:
# !! -> "Missing file specification after redirection operator."
Write-Output hi 2>&$streamNumber

Note, however, that PowerShell only ever supports 1 - i.e. the success output streams - as the redirection target.

Read on for a solution in case you really need to specify entire redirection expressions via variables that contain the redirection operator as part of their value too.

---

If you do need to specify arbitrary redirection expressions via variables, you can use Invoke-Expression (which is generally to be avoided, however):

# Define the command to invoke with all its arguments, but without redirections,
# as a script block ({ ... }), to be invoked later.
# The block may contain *multiple* statements.
$scriptBlock = { Write-Output hi }

# Define the desired redirection expression as a variable.
$redirection = '> $null'

# This is now the equivalent of:
#   & { Write-Output hi } > $null
# or, given that there's just *one* command in the script block:
#   Write-Output hi > $null
# Note the need to escape the $ in $scriptBlock as `$, 
# so as to prevent *up-front* expansion of the variable reference.
Invoke-Expression "& `$scriptBlock $redirection"

Answer 2

Thanks mklement0 (this isn not an answer but it is too long for a comment - sorry) I think I understand what you are saying about the redirection symbols; if they are not literally there then they are treated as variables. I seem to have ended up this way because I could not form '-f=directory' without concatenating it all together first so if I had to put that into a variable then why not '-i' and '>>' as well. Then they are all handled the same. But could you please help me understand some of the other differences as well From all these trials I have come up with some 'rules', it would really help me if you could confirm or correct my understanding Note: I am assuming that any path could have spaces in the name

     & "C:\Program Files\File Metadata\FileMeta.exe" -i $parm2 $parm3  *>> $logFileName
     & "C:\Program Files\File Metadata\FileMeta.exe" -i $parm2 $parm3  *>> $logFileName
     | _____________________________________________ ----------------- ___ -----------------
     1              2                                       3           4        5
1 = The command or verb
2 = what to call. Can use single or double quotes
3 = anything up to pipe or redirect characters. MUST use DOUBLE quotes around paths
    even though these are stripped out. If you use single quotes you get an error like <Cannot find directory for arg Argument: f>
4 = Pipe or redirect chracters. Must be in the clear (not part of a variable)
5 = file names etc after the redirects.  MUST NOT use either double or single quotes - even if there are spaces in the path

Then there are other confusions - ideas from different posts that just do not work here. For example

$logFileName 2>>&1 "Missing file specification after redirection operator." but *>> $logFileName does work.

No wonder I am confused. This thing is not homogeneous so I dont really know when to use which set of rules. Thanks in anticipation JC