忽略 PLY/Yacc Productions 中的换行符

Question

我正在尝试在我的语法中（在 PLY 中）实现一个结构。如果我将我的结构写在一行上，解析器不会产生语法错误，但如果我将它写在多行上，它会在包含大括号的行上产生语法错误。我的代码如下（没有 ast.py 因为我认为它没有帮助）：

main.py

from parser import *
import ply.yacc as yacc
yacc.yacc()

def main():
    with open('main.smp') as fp:
        for line in fp:
            try:
                yacc.parse(line)
            except EOFError:
                break           

main()

parser.py（也包含词法分析代码）

import sys
sys.path.insert(0,"../..")

from ast import *

tokens = (
    'NAME', 'NUMBER', 'STRUCT', 'LBRACE', 'RBRACE'
    )

literals = ['=','+','-','*','/', '(',')', '{', '}']

# Tokens
t_NAME   = r'[a-zA-Z_][a-zA-Z0-9_]*'
t_LBRACE = r'\{'
t_RBRACE = r'\}'

def t_NUMBER(t):
    r'\d+'
    try:
        t.value = int(t.value)
    except ValueError:
        print "Integer value too large", t.value
        t.value = 0
    return t

def t_STRUCT(t):
    r'struct '
    return t

t_ignore = " \t"

def t_newline(t):
    r'\n+'
    t.lexer.lineno += len(t.value)
    print "new line"

def t_error(t):
    print "Illegal character '%s'" % t.value[0]
    t.lexer.skip(1)

# Build the lexer
import ply.lex as lex
lexer = lex.lex()

lexer.brace_count = 0

# Parsing rules
precedence = (
    ('left','+','-'),
    ('left','*','/'),
    ('right','UMINUS'),
    )

start='statement'

def p_statement_assign(p):
    'statement : NAME "=" expression'

    names[p[1]] = p[3]

def p_statement_struct(p):
    "statement : STRUCT NAME LBRACE statement RBRACE" 
    print "parsed struct"

def p_statement_expr(p):
    'statement : expression'

    try:
        print p[1].codegen()
    except AttributeError:
        print p[1], "no attribute"

def p_expression_binop(p):
    '''expression : expression '+' expression
                  | expression '-' expression
                  | expression '*' expression
                  | expression '/' expression'''

    p[0] = BinOpNode(p[1], p[3], p[2])

def p_expression_uminus(p):
    "expression : '-' expression %prec UMINUS"

    p[0] = NumNode(-p[2])

def p_expression_group(p):
    "expression : '(' expression ')'"

    p[0] = NumNode(p[2])

def p_expression_number(p):
    "expression : NUMBER"

    p[0] = NumNode(p[1])

def p_expression_name(p):
    "expression : NAME"

    try:
        p[0] = names[p[1]]
    except LookupError:
        print "Undefined name '%s'" % p[1]
        p[0] = 0

def p_error(p):
    try:
        print "{1}: Syntax error on '{0}'".format(p.value, lexer.lineno)
    except AttributeError:
        print "{0}: Syntax error on line".format(lexer.lineno)

main.smp

struct test 
{
    9+9   
}

输出

new line
2: Syntax error on line
2: Syntax error on '{'
new line
new line
18
4: Syntax error on '}'

Answer 1

你的主循环：

def main():
    with open('main.smp') as fp:
        for line in fp:
            try:
                yacc.parse(line)
            except EOFError:
                break

故意分别解析每一行输入。因此，每一行都需要是完整的 statement 才能使解析成功（对于该行）。

忽略 PLY/Yacc Productions 中的换行符

Ignoring new line characters in PLY/Yacc Productions

python

yacc

lex

newline

ply

main.py

parser.py（也包含词法分析代码）

main.smp

输出