如何使用 stringtokenizer 构建扫描仪

Question

我在这段代码中遇到了问题，我正在尝试为我在编译器课程中的项目构建一个扫描器，扫描器从用户那里获取任何输入并将其分成标记..输出将是：打印每个令牌及其类型（如：数字、标识符、关键字、加号...等），最后打印令牌的数量。

我尝试了更多输入，每次输出都是标识符，当我尝试输入数字或关键字或 + 或 - 时，输出是标识符..

这是我的代码：

import java.util.Scanner;
import java.util.StringTokenizer;

public class MyScanner
{
    public static void main(String[] args)
    {
        String reserved_Keywords[] = { "abstract", "assert", "boolean",
                "break", "byte", "case", "catch", "char", "class", "const",
                "continue", "default", "do", "double", "else", "extends", "false",
                "final", "finally", "float", "for", "goto", "if", "implements",
                "import", "instanceof", "int", "interface", "long", "native",
                "new", "null", "package", "private", "protected", "public",
                "return", "short", "static", "strictfp", "super", "switch",
                "synchronized", "this", "throw", "throws", "transient", "true",
                "try", "void", "volatile", "while" };
        Scanner sc = new Scanner(System.in);
        System.out.println("Enter Your Text: ");
        String str = sc.nextLine();
        StringTokenizer st = new StringTokenizer(str);
        int numofTokens = st.countTokens();
        while( st.hasMoreElements() )
        {
            for (int i = 0; i < reserved_Keywords.length; i++)
            {  
                if ( st.equals(reserved_Keywords[i]) )
                {  
                    System.out.print(st.nextElement() + "\t");
                    System.out.println("Is Reserved Keyword");
                }
            }  

            if ( st.equals("+") )
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Plus Sign");
            }

            else if ( st.equals("-") )
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Minus Sign");
            }

            else if ( st.equals("*") )
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Multiply Sign");
            }

            else if( st.equals("/") )
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Divide Sign");
            }

            else if ( st.equals("=") )
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Assignment Operator");
            }

            else
            {
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Identifier");
            }
        }
        sc.close(); 
        System.out.println("Number of Tokens = " + numofTokens);
    }
}

Answer 1

您总是比较（调用 equals(..)( 与 StringTokenizer，而不是与 StringTokenizer 返回的标记。

要解决这个问题，请在 while 循环的第一行添加

 String TOKEN = st.nextToken();

然后将所有比较（对 equals() 的调用）替换为 st 而不是 TOKEN。

（你当然应该用大写字母命名变量，我这样做只是为了便于阅读）

那么您的代码将如下所示：

 StringTokenizer st = new StringTokenizer(str);
    int numofTokens = st.countTokens();
    while( st.hasMoreElements() )
    {   
        String TOKEN = st.nextToken();
        for (int i = 0; i < reserved_Keywords.length; i++)
        {  
            if ( TOKEN.equals(reserved_Keywords[i]) )
            {  
                System.out.print(st.nextElement() + "\t");
                System.out.println("Is Reserved Keyword");
            }
        }

...

如何使用 stringtokenizer 构建扫描仪

How to build a scanner with stringtokenizer

java

eclipse

compiler-construction

stringtokenizer