Do grammar subparse on a file

Question

Let's say you only want to parse the start of a large file using Perl 6 grammar. In order to avoid reading the whole file into a string, and then call subparse on the string. Is it possible to do a subparse when reading the file?

I could not find any subparsefile() method in the Grammar class, so I guess this is difficult to implement. But it should be possible in theory, see for example How do I search a file for a multiline pattern without reading the whole file into memory?

@EugeneBarsky Yes it should be anchored to the beginning, or alternatively, anchored so close to the beginning that it would pay off not to read the whole file into memory. — Håkon Hægland, Dec 16 '17 at 19:44
So the `rotor` part of the solution isn't needed in that case? — Eugene Barsky, Dec 16 '17 at 19:47
In that case, `"textfile".IO.lines.head(N).join.subparse($grammar)` should do the truck, where `N` is the number of lines to be read to ensure a possible match. — Elizabeth Mattijsen, Dec 17 '17 at 10:25

score 7 · Accepted Answer · answered Dec 14 '17 at 19:00

7

Currently you can't. Parsing anything at the moment requires the entire string to exist in memory.

Having said that, if you know the maximum number of lines your pattern may expand over, you could do something like:

my $max = 3; # maximum number of lines
for "textfile".IO.lines(:!chomp).rotor( $max => -$max + 1 ) -> @lines {
    @lines.join.subparse( $grammar)
    # and whatever you would like to do
}

It wouldn't be the fastest way of doing it, but it would not have to read the whole file in memory.

answered Dec 14 '17 at 19:00

Elizabeth Mattijsen

25,654
3
75
105

2

Alternately, if you know how many characters your pattern can be maximally, you might want to try a `"textfile".IO.comb(halfmaxsize).rotor(2=>-1) -> @chunks { @chunks.join.subparse($grammar) }` – Elizabeth Mattijsen Dec 14 '17 at 21:54

Do grammar subparse on a file

1 Answers1