Antlr4 - Is there a simple example of using the ParseTree Walker?

john ktejik picture john ktejik · Feb 1, 2018 · Viewed 8k times · Source

Antlr4 has a new class ParseTreeWalker. But how do I use it? I am looking for a minimal working example. My grammar file is 'gram.g4' and I want to parse a file 'program.txt'

Here is my code so far. (This assumes ANTLR has run my grammar file and created all of the gramBaseListener, gramLexer, etc etc):

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
import static org.antlr.v4.runtime.CharStreams.fromFileName;

public class launch{
public static void main(String[] args) {

    CharStream cs = fromFileName("gram.g4");  //load the file
    gramLexer lexer = new gramLexer(cs);  //instantiate a lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer); //scan stream for tokens
    gramParser parser = new gramParser(tokens);  //parse the tokens

    // Now what??  How do I connect the above with the below? 

    ParseTreeWalker walker = new ParseTreeWalker();  // how do I use this to parse program.txt??
}}

I am using java but I assume it is similar in other languages.

The ANTLR documentation (http://www.antlr.org/api/Java/index.html) is short on examples. There are many tutorials on the internet but they are mostly for ANTLR version 3. The few using version 4 don't work or are outdated (for example, there is no parser.init() function, and classes like ANTLRInputStream are depreciated)

Thanks in advance for anyone who can help.

Answer

Raven picture Raven · Feb 1, 2018

For each of your parser rules in your grammar the generated parser will have a corresponding method with that name. Calling that method will start parsing at that rule.

Therefore if your "root-rule" is named start then you'd start parsing via gramParser.start() which returns a ParseTree. This tree can then be fed into the ParseTreeWalker alongside with the listener you want to be using.

All in all it could look something like this (EDITED BY OP):

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;
import static org.antlr.v4.runtime.CharStreams.fromFileName;

public class launch{
public static void main(String[] args) {

    CharStream cs = fromFileName("program.txt");  //load the file
    gramLexer lexer = new gramLexer(cs);  //instantiate a lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer); //scan stream for tokens
    gramParser parser = new gramParser(tokens);  //parse the tokens

    ParseTree tree = parser.start(); // parse the content and get the tree
    Mylistener listener = new Mylistener();

    ParseTreeWalker walker = new ParseTreeWalker();
    walker.walk(listener,tree);
}}

************ NEW FILE Mylistener.java ************

public class Mylistener extends gramBaseListener {
        @Override public void enterEveryRule(ParserRuleContext ctx) {  //see gramBaseListener for allowed functions
            System.out.println("rule entered: " + ctx.getText());      //code that executes per rule
        }
    }

Of course you have to replace <listener> with your implementation of BaseListener

And just one small sidenode: In Java it is convention to start classnames with capital letters and I'd advise you to stick to that in order for making the code more readable for other people.