How do I get the original text that an antlr4 rule matched?

Question

Using the Java 7 grammar https://github.com/antlr/grammars-v4/blob/master/java7/Java7.g4 I want to find methods with a specific name and then just print out that method. I see that I can use the methodDeclaration rule when I match. So I subclass Java7BaseListener and override this listener method:

@Override public void enterMethodDeclaration(Java7Parser.MethodDeclarationContext ctx) { }

How do I get the original text out? ctx.getText() gives me a string with all the whitespace stripped out. I want the comments and original formatting.

See also [this question](https://stackoverflow.com/questions/7443860/get-original-text-of-an-antlr-rule) and [this question](https://stackoverflow.com/questions/50443728/context-gettext-excludes-spaces-in-antlr4). — Some Guy, Jan 25 '19 at 11:51

score 60 · Accepted Answer · answered May 02 '13 at 16:37

60

ANTLR's CharStream class has a method getText(Interval interval) which will return the original source in the give range. The Context object has methods to get the beginning and end. Assuming you have a field in your listener called input which has the CharStream being parsed, you can do this:

    int a = ctx.start.getStartIndex();
    int b = ctx.stop.getStopIndex();
    Interval interval = new Interval(a,b);
    input.getText(interval);

answered May 02 '13 at 16:37

monty0

1,759
1
14
22

28

If you don't have access to or don't want to keep track of the `CharStream`, use `ctx.start.getInputStream()` to retrieve it. – Peter Jul 03 '14 at 16:23
1

CharStream input = ctx.start.getInputStream(); input.getText(interval); Gives me runtime errors .checkBoundsOffCount(String.java:3101) – john k Feb 02 '18 at 02:30
2

And where it doesn't fail it still removes whitespace – john k Feb 02 '18 at 02:32
Your answer helped me fix a weird error where calling `getText()` on a `Token` failed because somehow my original `CharStream` got garbage collected. I kept a reference to it and `getText` works now. – Julius Naeumann Apr 19 '20 at 14:26
Totally nonintuitive as far as solutions go, this was a lifesaver for me. – raffian Mar 02 '23 at 23:35

score 12 · Answer 2 · answered Aug 28 '18 at 03:23

demo:

SqlBaseParser.QueryContext queryContext = context.query();
int a = queryContext.start.getStartIndex();
int b = queryContext.stop.getStopIndex();
Interval interval = new Interval(a,b);
String viewSql = context.start.getInputStream().getText(interval);

score 0 · Answer 3 · answered Jul 12 '21 at 08:58

0

Python implementation:

def extract_original_text(self, ctx):
    token_source = ctx.start.getTokenSource()
    input_stream = token_source.inputStream
    start, stop  = ctx.start.start, ctx.stop.stop
    return input_stream.getText(start, stop)

answered Jul 12 '21 at 08:58

Heinrich

340
1
4
12

How do I get the original text that an antlr4 rule matched?

3 Answers3

Linked