Send ByteArrayOutputStream to Perl script, then read response

Question

I have a ByteArrayOuputStream I received from performing a diff. Java's parsing of this is too slow, so I decided to try to pass the parsing off to a Perl script. I'm having a little trouble getting the script to receive data from this output stream. When I run my code, the application hangs indefinitely. This is what I have so far:

public static Diff analyzeDiff(ByteArrayOutputStream baos) throws IOException {

    ProcessBuilder pb = new ProcessBuilder();
    pb.command("perl/path/perl", TEMP.getAbsolutePath());
    Process process = pb.start();
    OutputStream str = process.getOutputStream();
    baos.writeTo(str);
    str.flush();
    try {
        process.waitFor();
    } catch (InterruptedException e) {
        BufferedReader bf = new BufferedReader(new InputStreamReader(process.getInputStream()));
        String line;
        while ((line = bf.readLine()) != null) {
            System.out.println(line);
        }
    }

    return null;
}

@Test
public void testDiffParser() throws IOException {
    DiffParser.init();

    File test = new File("path/to/file/test.diff");

    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    baos.write(FileUtils.readFileToByteArray(test));
    //String output = baos.toString();
    //System.out.println(output);

    DiffParser.analyzeDiff(baos);
    //DiffParser.analyzeDiff(output);
}

And here is my Perl script:

#!/usr/bin/perl
use strict;
use warnings;

my $additions = 0;
my $deletions = 0;
my $filesChanged = 0;

my $fileAdded = 0;
my $line;

foreach $line ( <> ) {
    $_ = $line;
    chomp( $_ );
    print( $_ );
    if ( /^\-\-\-/m ) {
        $fileAdded = 1;
    } elsif ( /^\+\+\+/m && $fileAdded ) {
        $filesChanged++;
        $fileAdded = 0;
    } elsif ( /^\+/ ) {
        $additions++;
        $fileAdded = 0;
    } elsif ( /^\-/ ) {
        $deletions++;
        $fileAdded = 0;
    } else {
        $fileAdded = 0;
    }
}

print("$additions $deletions $filesChanged\n")

Is there a way to actually do what I am trying to do?

Edit: This is how I was doing it in Java:

private Diff parseDiff(final ByteArrayOutputStream baos) {

    final Diff diff = new Diff();

    int filesChanged = 0;
    int additions = 0;
    int deletions = 0;

    boolean fileAdded = false;

    final String[] lines = baos.toString().split("\n");

    for (final String line : lines) {

        if (line.startsWith("---")) {
            fileAdded = true;
        } else if (line.startsWith("+++") && fileAdded) {
            filesChanged++;
            fileAdded = false;
        } else if (line.startsWith("+")) {
            additions++;
            fileAdded = false;
        } else if (line.startsWith("-")) {
            deletions++;
            fileAdded = false;
        } else {
            fileAdded = false;
        }

    }

    diff.additions = additions;
    diff.deletions = deletions;
    diff.changedFiles = filesChanged;

    return diff;
}

Edit 2 If you want some context, you can refer to this Related question

Answer 1

I'm using a tablet at present so I can't help much, but your Perl needs some work.

You shouldn't use for $line ( <> ) as that will try to read all of the input into a list before starting to iterate. You also don't use $line so you should read straight into $_ with

while ( <> ) { ... }

There's also no need to chomp every line, and I dont understand why do you call print for every record? It's after the chomp so the output will be a copy of the input all on one very long line with the aggregate values at the very end.

I suspect the Perl script is receiving the data just fine, but having trouble fitting all of the input into memory at once together with a second copy of everything as output!

Answer 2

Using ByteArrayOutputStream means that the whole result of the diff needs to be stored in memory, all at once, instead of getting processed and garbage collected in chunks. Your Java program may have been slow due to running out of memory and performing garbage collection all the time.

Java will be much faster at whatever task you throw at it, compared to Perl. It's a just-in-time compiled language, as opposed to an interpreted language in case of Perl. See e.g. http://blog.carlesmateo.com/2014/10/13/performance-of-several-languages/, https://attractivechaos.github.io/plb/ or https://en.wikipedia.org/wiki/Java_performance (comparison to other langauges).

If you need performance, you should optimize your Java code instead of creating dependency on Perl.