以点文件的形式生成AST

Question

我正在使用 ANTLR4 生成 java 源代码的 AST，我不得不转向 ANTLR3，因为我没有得到太多帮助和文档，而且 proceed.I 管理起来真的很难生成 AST 但不是以可视格式。然后我遇到了一个很棒的 answer，我真的能够在 DOT 文件中生成 AST，但是有一个小问题。

我的代码：

import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.ANTLRFileStream;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.DOTTreeGenerator;
import org.antlr.stringtemplate.StringTemplate;

class Main {

        public static void main(String[] args) throws Exception {
            parseFile("/home/satnam-sandhu/Workstation/ASTGenerator/resource/java/Blabla.java");
        }

    public static void parseFile(String f)throws Exception {

            JavaLexer lexer = new JavaLexer(new ANTLRFileStream(f));
            CommonTokenStream tokens = new CommonTokenStream(lexer);
            JavaParser parser = new JavaParser(tokens);
            CommonTree tree = (CommonTree)parser.compilationUnit().getTree();
            DOTTreeGenerator gen = new DOTTreeGenerator();
            StringTemplate st = gen.toDOT(tree);
            System.out.println(st);

    }

}

我正在使用 gradle 所以我通过以下方式构建项目：

gradle clean build

然后运行它并通过以下方式将输出通过管道传输到点文件：

java -jar ASTGenerator.jar > ast.dot

现在我面临的问题是，对于源代码：

class example{
    public static void print(int a){
        int b = a +1;
        System.out.println(b);
    }

    public static void main(){
        print(15);
    }
}

我得到这个作为输出：

digraph {

    ordering=out;
    ranksep=.4;
    bgcolor="lightgrey"; node [shape=box, fixedsize=false, fontsize=12, fontname="Helvetica-bold", fontcolor="blue"
        width=.25, height=.25, color="black", fillcolor="white", style="filled, solid, bold"];
    edge [arrowsize=.5, color="black", style="bold"]

  n0 [label=""];
  n1 [label="class"];
  n2 [label="example"];
  n3 [label="{"];
  n4 [label="public"];
  n5 [label="static"];
  n6 [label="void"];
  n7 [label="print"];
  n8 [label="("];
  n9 [label="int"];
  n10 [label="a"];
  n11 [label=")"];
  n12 [label="{"];
  n13 [label="int"];
  n14 [label="b"];
  n15 [label="="];
  n16 [label="a"];
  n17 [label="+"];
  n18 [label="1"];
  n19 [label=";"];
  n20 [label="System"];
  n21 [label="."];
  n22 [label="out"];
  n23 [label="."];
  n24 [label="println"];
  n25 [label="("];
  n26 [label="b"];
  n27 [label=")"];
  n28 [label=";"];
  n29 [label="}"];
  n30 [label="public"];
  n31 [label="static"];
  n32 [label="void"];
  n33 [label="main"];
  n34 [label="("];
  n35 [label=")"];
  n36 [label="{"];
  n37 [label="print"];
  n38 [label="("];
  n39 [label="15"];
  n40 [label=")"];
  n41 [label=";"];
  n42 [label="}"];
  n43 [label="}"];

  n0 -> n1 // "" -> "class"
  n0 -> n2 // "" -> "example"
  n0 -> n3 // "" -> "{"
  n0 -> n4 // "" -> "public"
  n0 -> n5 // "" -> "static"
  n0 -> n6 // "" -> "void"
  n0 -> n7 // "" -> "print"
  n0 -> n8 // "" -> "("
  n0 -> n9 // "" -> "int"
  n0 -> n10 // "" -> "a"
  n0 -> n11 // "" -> ")"
  n0 -> n12 // "" -> "{"
  n0 -> n13 // "" -> "int"
  n0 -> n14 // "" -> "b"
  n0 -> n15 // "" -> "="
  n0 -> n16 // "" -> "a"
  n0 -> n17 // "" -> "+"
  n0 -> n18 // "" -> "1"
  n0 -> n19 // "" -> ";"
  n0 -> n20 // "" -> "System"
  n0 -> n21 // "" -> "."
  n0 -> n22 // "" -> "out"
  n0 -> n23 // "" -> "."
  n0 -> n24 // "" -> "println"
  n0 -> n25 // "" -> "("
  n0 -> n26 // "" -> "b"
  n0 -> n27 // "" -> ")"
  n0 -> n28 // "" -> ";"
  n0 -> n29 // "" -> "}"
  n0 -> n30 // "" -> "public"
  n0 -> n31 // "" -> "static"
  n0 -> n32 // "" -> "void"
  n0 -> n33 // "" -> "main"
  n0 -> n34 // "" -> "("
  n0 -> n35 // "" -> ")"
  n0 -> n36 // "" -> "{"
  n0 -> n37 // "" -> "print"
  n0 -> n38 // "" -> "("
  n0 -> n39 // "" -> "15"
  n0 -> n40 // "" -> ")"
  n0 -> n41 // "" -> ";"
  n0 -> n42 // "" -> "}"
  n0 -> n43 // "" -> "}"

}

当使用 http://viz-js.com/ 可视化输出时是这样的：

我到目前为止的所有工作都已上传 here 如果你们想更深入地了解我正在使用的语法文件。我认为语法文件中指定的选项也可能是原因。我是初学者，没有一点帮助就无法继续。提前致谢。 :)

Answer 1

我一直在寻找的答案得到了回答here by Bart Kiers but for people who want to generate DOT files without modifying the grammar can print a intended syntax tree by taking help from this repository。由于我没有找到太多关于 ANTLR4 中点生成的文档，除了修改 ANTLR3 语法文件之外没有其他选择，我使用 Federico Tomassetti 示例并稍微修改它以生成我们自己的 DOT 文件。

您可以通过以下方式打印点文件输出：

import java.io.File;
import java.io.IOException;

import java.util.ArrayList;

import java.nio.charset.Charset;
import java.nio.file.Files;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.ParserRuleContext;
import org.antlr.v4.runtime.RuleContext;
import org.antlr.v4.runtime.tree.ParseTree;

public class ASTGenerator {

    static ArrayList<String> LineNum = new ArrayList<String>();
    static ArrayList<String> Type = new ArrayList<String>();
    static ArrayList<String> Content = new ArrayList<String>();

    private static String readFile() throws IOException {
        File file = new File("resource/java/Blabla.java");
        byte[] encoded = Files.readAllBytes(file.toPath());
        return new String(encoded, Charset.forName("UTF-8"));
    }

    public static void main(String args[]) throws IOException{
        String inputString = readFile();
        ANTLRInputStream input = new ANTLRInputStream(inputString);
        Java8Lexer lexer = new Java8Lexer(input);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        Java8Parser parser = new Java8Parser(tokens);
        ParserRuleContext ctx = parser.compilationUnit();

        generateAST(ctx, false, 0);

    System.out.println("digraph G {");
        printDOT();
    System.out.println("}");

    }

    private static void generateAST(RuleContext ctx, boolean verbose, int indentation) {
        boolean toBeIgnored = !verbose && ctx.getChildCount() == 1 && ctx.getChild(0) instanceof ParserRuleContext;

        if (!toBeIgnored) {
            String ruleName = Java8Parser.ruleNames[ctx.getRuleIndex()];
        LineNum.add(Integer.toString(indentation));
            Type.add(ruleName);
            Content.add(ctx.getText());
    }
        for (int i = 0; i < ctx.getChildCount(); i++) {
            ParseTree element = ctx.getChild(i);
            if (element instanceof RuleContext) {
                generateAST((RuleContext) element, verbose, indentation + (toBeIgnored ? 0 : 1));
            }
        }
    }
    
    private static void printDOT(){
        printLabel();
        int pos = 0;
        for(int i=1; i<LineNum.size();i++){
            pos=getPos(Integer.parseInt(LineNum.get(i))-1, i);
            System.out.println((Integer.parseInt(LineNum.get(i))-1)+Integer.toString(pos)+"->"+LineNum.get(i)+i);
        }
    }
    
    private static void printLabel(){
        for(int i =0; i<LineNum.size(); i++){
            System.out.println(LineNum.get(i)+i+"[label=\""+Type.get(i)+"\n "+Content.get(i)+" \"]");
        }
    }
    
    private static int getPos(int n, int limit){
        int pos = 0;
        for(int i=0; i<limit;i++){
            if(Integer.parseInt(LineNum.get(i))==n){
                pos = i;
            }
        }
        return pos;
    }
}

对于这样的源代码：

class example{
  public static void main(){
    int a;
    a = 5;
  }
}

输出将是：

digraph G {
00[label="compilationUnit\n classexample{publicstaticvoidmain(){inta;a=5;}}<EOF> "]
11[label="normalClassDeclaration\n classexample{publicstaticvoidmain(){inta;a=5;}} "]
22[label="classBody\n {publicstaticvoidmain(){inta;a=5;}} "]
33[label="methodDeclaration\n publicstaticvoidmain(){inta;a=5;} "]
44[label="methodModifier\n public "]
45[label="methodModifier\n static "]
46[label="methodHeader\n voidmain() "]
57[label="result\n void "]
58[label="methodDeclarator\n main() "]
49[label="block\n {inta;a=5;} "]
510[label="blockStatements\n inta;a=5; "]
611[label="localVariableDeclarationStatement\n inta; "]
712[label="localVariableDeclaration\n inta "]
813[label="integralType\n int "]
814[label="variableDeclaratorId\n a "]
615[label="expressionStatement\n a=5; "]
716[label="assignment\n a=5 "]
817[label="expressionName\n a "]
818[label="assignmentOperator\n = "]
819[label="literal\n 5 "]
00->11
11->22
22->33
33->44
33->45
33->46
46->57
46->58
33->49
49->510
510->611
611->712
712->813
712->814
510->615
615->716
716->817
716->818
716->819
}

将这段输出插入http://viz-js.com/ 你会得到这个作为输出：

您还可以通过以下方式将输出通过管道传输到 ast.dot 文件：

java -jar path-to-jar-file.jar > ast.dot

现在这不是完美的方法，但对我来说已经足够了。 :)

希望对您有所帮助。

以点文件的形式生成AST

Generate AST in the form of a dot file

java

dot

abstract-syntax-tree

antlr3