Piping multilpe times using unary operator

Question

I have the following function:

Function Refine{
    [CmdletBinding()]
        param (
            [parameter(ValueFromPipeline = $true)]
          $data
        )
        begin {}
        process {
            For ($i=0; $i -le $data.count; $i++){
            if ($data[$i] -match 'nmap scan report' -and $data[$i+1] -match 'host is up' )  {
                $dat += $data[$i] + "`n"
                $counter=($i+4)
                while ($data[$counter] -match '^[0-9]') {
                    $dat += $data[$counter] + "`n"
                    $counter++
                }        
                }
            }
            
         }
        end { 
            if ($dat){#See that its not empty
            return  $dat.Split([Environment]::NewLine)
            }
        }
    }

$file1 = gc c:\test.txt

,gc ($file1) | Refine

I must pipe with the Unary operator since i want the whole object to be piped at one time to the Refine Function.

The output is something like this for example: Nmap scan report for 1.1.1.1 113/tcp closed ident 443/tcp open ssl/http Fortinet SSL VPN Nmap scan report for 2.2.2.2 21/tcp filtered ftp 22/tcp filtered ssh

I now would like to use the another function which takes this and create objects from text:

  Function BuildArray{
      [CmdletBinding()]
      Param(
          [Parameter(ValueFromPipeline = $true)]
          $data
      )
      Begin {}
      Process {
        $NMAPS=@()
        $nmap=@()
        $ports=@()
        For ($i=0; $i -lt $data.count; $i++){
            If ($data[$i] -match 'nmap scan report')  {
            $ip = ($data[$i] |  Select-String -Pattern "\d{1,3}(\.\d{1,3}){3}" -AllMatches).Matches.Value
            $k=$i
            If ($data[($i+1)] -notmatch 'nmap scan report' -and $i -le $data.count){
                DO{
                $ports+=$data[$i+1]
                $i++
                }
               While ($data[$i+1] -notmatch 'nmap scan report' -and $i -le $data.count)
               $nmap = @{IP = "$ip"; ports=@($ports)}
               $NMAPS+=$nmap
               $nmap=@()
               $ports=@()
               $i = $k
               }
              }
            }
      }
      End {
        $NMAPS
      }
  }

My problem here is again - i would like the data from the pipe to pass as a whole to the BuildArray function. but i cannot use the unary operator since i get an error saying i can use it only at the start of the piping,

 ,(gc $file1 ) | ,(refine) | BuildArray #No good

How Can i pass this second object in the pipline as a whole to the third object once at once?

`Write-Output (gc $file1) -NoEnumerate` as the initial expression, then use `Write-Output $output -NoEnumerate` instead of `return` inside your functions — Mathias R. Jessen, Aug 10 '20 at 11:27
So you want the first and second combined into one object and passed as a whole to BuildArray? — Doug Maurer, Aug 10 '20 at 13:03
I Actually want first to read from a file using Get-Content then pipe it to a function that changes the text to the way i like (Refine) and pipe it to a function that take the text and build an custom object from that info — Shahar, Aug 11 '20 at 07:12

iRon · Accepted Answer · 2020-08-10T19:06:31.180

With all respect, you are not on the right track with the general concept of the PowerShell pipeline. Cmdlets should implement for the Middle of a Pipeline, meaning that each cmdlet should process each item a.s.a.p. and stream it onto the next cmdlet to eventually release it from memory.
A PowerShell object (PSCustomObjects) is optimized for streaming as it contains all type information of its properties. This makes these PowerShell objects heavy and less suitable for storing them in a (large) collection.
Specific to your own functions, it makes your input processing methods (the Start, Process and End block structure) pointless which is especially developed to Process each specific object in the pipeline. If you collect all your objects in an array before passing it to your BuildArray function, you might as well just use the End block (or leave all the input blocks which is similar to just an End block) where the automated $Input variable holds all the objects supplied in the pipeline in a single array:

Function BuildArray {
    [CmdletBinding()]
    Param(
        [Parameter(ValueFromPipeline = $true)]$data
    )
    Process { Write-Host 'Process:' $Input }
}

1,2,3 | BuildArray

Process: 1
Process: 2
Process: 3

Function BuildArray {
    [CmdletBinding()]
    Param(
        [Parameter(ValueFromPipeline = $true)]$data
    )
    End { Write-Host 'End:' $Input } # or just: Write-Host $Input
}

1,2,3 | BuildArray

End: 1 2 3

Also:

Try I avoid using the increase assignment operator (+=) to create a collection (or a string) as it is exponential expensive.
- And if you do use it, you should properly initialize the $dat = '' value otherwise you might start with an unexpected copy inherited from one of it parents functions.

Piping multilpe times using unary operator

1 Answers1