x3 语法中无用的编译器错误

Unhelpful compiler errors in x3 grammar

以下用于简单机器人命令语言的 Spirit x3 语法在 Windows Visual Studio 中生成编译器错误 17. 对于此项目，我需要将警告级别设置为 4 (/W4 ) 并将警告视为错误 (/WX)。

Warning C4127 conditional expression is constant SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\char\detail\cast_char.hpp 29
Error C2039 'insert': is not a member of 'boost::spirit::x3::unused_type' SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\core\detail\parse_into_container.hpp 259 Error C2039 'end': is not a member of 'boost::spirit::x3::unused_type' SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\core\detail\parse_into_container.hpp 259 Error C2039 'empty': is not a member of 'boost::spirit::x3::unused_type' SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\core\detail\parse_into_container.hpp 254 Error C2039 'begin': is not a member of 'boost::spirit::x3::unused_type' SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\core\detail\parse_into_container.hpp 259

很明显，我的语法有问题，但错误消息完全没有帮助。我发现如果我在语法的最后一行删除 Kleene 星号（*参数只是参数）错误就会消失，但是我会收到很多这样的警告：

Warning C4459 declaration of 'digit' hides global declaration SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\support\numeric_utils\detail\extract_int.hpp 174 Warning C4127 conditional expression is constant SpiritTest e:\data\boost\boost_1_65_1\boost\spirit\home\x3\char\detail\cast_char.hpp 29 

#include <string>
#include <iostream>

#include <boost/config/warning_disable.hpp>
#include <boost/spirit/home/x3.hpp>

namespace x3 = boost::spirit::x3;

//
// Grammar for simple command language
//

namespace scl
    {
    using boost::spirit::x3::char_;
    using boost::spirit::x3::double_;
    using boost::spirit::x3::int_;
    using boost::spirit::x3::lexeme;
    using boost::spirit::x3::lit;
    using boost::spirit::x3::no_case;

    auto    valid_identifier_chars = char_ ("a-zA-Z_");
    auto    quoted_string = '"' >> *(lexeme [~char_ ('"')]) >> '"';
    auto    keyword_value_chars = char_ ("a-zA-Z0-9$_.");

    auto    qual = lexeme [!(no_case [lit ("no")]) >> +valid_identifier_chars] >> -('=' >> (quoted_string | int_ | double_ | +keyword_value_chars));
    auto    neg_qual = lexeme [no_case [lit ("no")] >> +valid_identifier_chars];
    auto    qualifier = lexeme ['/' >> (qual | neg_qual)];

    auto    verb = +valid_identifier_chars >> *qualifier;
    auto    parameter = +keyword_value_chars >> *qualifier;
    auto    command = verb >> *parameter;
    };  // End namespace scl

using namespace std;                                    // Must be after Boost stuff!

int
main ()

{
vector <string> input = 
    {
    "show/out=\"somefile.txt\" motors/all cameras/full",
    "start/speed=5 motors arm1 arm2/speed=2.5/track arm3",
    "rotate camera1/notrack/axis=y/angle=45"
    };

    //
    // Parse each of the strings in the input vector
    //

    for (string str : input)
        {
        auto    b = str.begin ();
        auto    e = str.end ();


        cout << "Parsing: " << str << endl;
        x3::phrase_parse (b, e, scl::command, x3::space);

        if (b != e)
            {
            cout << "Error, only parsed to position: " << b - str.begin () << endl;
            }

        }   // End for

    return 0;
}                           // End main

自 Boost 1.65 以来出现回归，导致某些规则出现问题，这些规则可能会传播到容器类型属性中。

在没有实际绑定属性的情况下实例化时，它们分派到错误的重载。发生这种情况时，会出现一个名为 unused_type 的 "mock" 属性类型。您看到的错误表明 unused_type 被视为具体属性类型，显然它不会运行。

回归已修复 https://github.com/boostorg/spirit/commit/ee4943d5891bdae0706fb616b908e3bf528e0dfa

用Boost 1.64编译可以看出是回归：

现在，最新的开发应该修复它，但你可以简单地复制补丁文件，even just the 7-line patch。

All of the above was already available when I linked the duplicate question , which highlights the same regression

评论

  • using namespace std; // Must be after Boost stuff!

    实际上，除非在非常局部的范围内，否则它可能无处可去，在那里您可以看到任何潜在名称冲突的影响。

  • 考虑封装 skipper，因为它在逻辑上很可能是语法规范的一部分，而不是被调用者覆盖的东西。

  • 这是一个错误：

    auto quoted_string = '"' >> *(lexeme[~char_('"')]) >> '"';

    您可能想断言整个文字是词素，而不是单个字符（那是……没有实际意义，因为空格永远不会命中解析器，因为船长）。

    auto quoted_string = lexeme['"' >> *~char_('"') >> '"'];

  • 同样，您可能希望 +keyword_value_chars 成为词位，因为现在 one=two three four 会用 [=] 解析 "qualifier" one onethreefour 的 144=]，而不是 one three four¹

  • x3::space 跳过嵌入的换行符，如果这不是本意，请使用 x3::blank

  • 由于 PEG 语法被解析 left-to-right greedy，您可以订购 qualifier 生产而无需 !(no_case["no"]) 前瞻断言。这不仅消除了重复，而且使语法更简单、更高效：

    auto qual      = lexeme[+valid_identifier_chars] >>
    -('=' >> (quoted_string | int_ | double_ | +keyword_value_chars)); // TODO lexeme
auto neg_qual  = lexeme[no_case["no"] >> +valid_identifier_chars];
auto qualifier = lexeme['/' >> (neg_qual | qual)];

    ¹ Note (Post-Scriptum) now that we notice qualifier is, itself, already a lexeme, there's no need to lexeme[] things inside (unless, of course they're reused in contexts with skippers).

    However, this also gives rise to the question whether whitespace around the = operator should be accepted (currently, it is not), or whether qualifiers can be separated with whitespace (like id /a /b; currently they can).

  • 也许 verb 也需要一些 lexemes[]（除非你真的 确实 想将 "one two three" 解析为动词)

  • 如果no前缀为否定限定符，那么也许标识符本身也是？这可以简化语法

  • int_ 和 double_ 的顺序使得大多数双打在被识别之前都是 mis-parsed 和 int。考虑一些更明确的东西，比如 x3::strict_real_policies<double>>{} | int_

  • 如果您正在解析引用的结构，也许您也想识别转义（例如 '\"' 和 '\'）：

    auto quoted_string = lexeme['"' >> *('\' >> char_ | ~char_('"')) >> '"'];

  • 如果您需要 "keyword values"，请考虑在 x3::symbols<> 中列出已知值。这也可以用来直接解析成枚举类型。

这是一个解析为 AST 类型并将其打印出来以供演示的版本：

Live On Coliru

#include <boost/config/warning_disable.hpp>

#include <string>
#include <vector>
#include <boost/variant.hpp>

namespace Ast {
    struct Keyword : std::string { // needs to be strong-typed to distinguish from quoted values
        using std::string::string;
        using std::string::operator=;
    };

    struct Nil {};
    using Value = boost::variant<Nil, std::string, int, double, Keyword>;

    struct Qualifier {
        enum Kind { positive, negative } kind;
        std::string identifier;
        Value       value;
    };

    struct Param {
        Keyword                keyword;
        std::vector<Qualifier> qualifiers;
    };

    struct Command {
        std::string            verb;
        std::vector<Qualifier> qualifiers;
        std::vector<Param>     params;
    };
}

#include <boost/fusion/adapted/struct.hpp>
BOOST_FUSION_ADAPT_STRUCT(Ast::Qualifier, kind, identifier, value)
BOOST_FUSION_ADAPT_STRUCT(Ast::Param, keyword, qualifiers)
BOOST_FUSION_ADAPT_STRUCT(Ast::Command, verb, qualifiers, params)

#include <boost/spirit/home/x3.hpp>

namespace x3 = boost::spirit::x3;
namespace scl {
    //
    // Grammar for simple command language
    //
    using x3::char_;
    using x3::int_;
    using x3::lexeme;
    using x3::no_case;

    // lexeme tokens
    auto keyword       = x3::rule<struct _keyword, Ast::Keyword> { "keyword" }
                       = lexeme [ +char_("a-zA-Z0-9$_.") ];
    auto identifier    = lexeme [ +char_("a-zA-Z_") ];
    auto quoted_string = lexeme['"' >> *('\' >> x3::char_ | ~x3::char_('"')) >> '"'];

    auto value
        = quoted_string
        | x3::real_parser<double, x3::strict_real_policies<double>>{}
        | x3::int_
        | keyword;

    auto qual
        = x3::attr(Ast::Qualifier::positive) >> identifier >> -('=' >> value);
    auto neg_qual
        = x3::attr(Ast::Qualifier::negative) >> lexeme[no_case["no"] >> identifier] >> x3::attr(Ast::Nil{}); // never a value
    auto qualifier 
        = lexeme['/' >> (neg_qual | qual)];

    auto verb      
        = identifier;

    auto parameter = x3::rule<struct _parameter, Ast::Param> {"parameter"}
        = keyword >> *qualifier;

    auto command = x3::rule<struct _command, Ast::Command> {"command"} 
        = x3::skip(x3::space) [ verb >> *qualifier >> *parameter ];

} // End namespace scl

// For Demo, Debug: printing the Ast types back
#include <iostream>
#include <iomanip>

namespace Ast {
    static inline std::ostream& operator<<(std::ostream& os, Value const& v) {
        struct {
            std::ostream& _os;
            void operator()(std::string const& s) const { _os << std::quoted(s); }
            void operator()(int i)                const { _os << i; } 
            void operator()(double d)             const { _os << d; } 
            void operator()(Keyword const& kwv)   const { _os << kwv; } 
            void operator()(Nil)                  const { }
        } vis{os};

        boost::apply_visitor(vis, v);
        return os;
    }

    static inline std::ostream& operator<<(std::ostream& os, Qualifier const& q) {
        os << "/" << (q.kind==Qualifier::negative?"no":"") << q.identifier;
        if (q.value.which())
           os << "=" << q.value;
        return os;
    }

    static inline std::ostream& operator<<(std::ostream& os, std::vector<Qualifier> const& qualifiers) {
        for (auto& qualifier : qualifiers)
            os << qualifier;
        return os;
    }

    static inline std::ostream& operator<<(std::ostream& os, Param const& p) {
        return os << p.keyword << p.qualifiers;
    }

    static inline std::ostream& operator<<(std::ostream& os, Command const& cmd) {
        os << cmd.verb << cmd.qualifiers;
        for (auto& param : cmd.params)         os << " " << param;
        return os;
    }
}

int main() {
    for (std::string const str : {
            "show/out=\"somefile.txt\" motors/all cameras/full",
            "start/speed=5 motors arm1 arm2/speed=2.5/track arm3",
            "rotate camera1/notrack/axis=y/angle=45",
        })
    {
        auto b = str.begin(), e = str.end();

        Ast::Command cmd;
        bool ok = parse(b, e, scl::command, cmd);
        std::cout << (ok?"OK":"FAIL") << '\t' << std::quoted(str) << '\n';

        if (ok) {
            std::cout << " -- Full AST: " << cmd << "\n";
            std::cout << " -- Verb+Qualifiers: " << cmd.verb << cmd.qualifiers << "\n";
            for (auto& param : cmd.params)
                std::cout << "     -- Param+Qualifiers: " << param << "\n";
        }

        if (b != e) {
            std::cout << " -- Remaining unparsed: " << std::quoted(std::string(b,e)) << "\n";
        }
    }
}

版画

OK  "show/out=\"somefile.txt\" motors/all cameras/full"
 -- Full AST: show/out="somefile.txt" motors/all cameras/full
 -- Verb+Qualifiers: show/out="somefile.txt"
     -- Param+Qualifiers: motors/all
     -- Param+Qualifiers: cameras/full
OK  "start/speed=5 motors arm1 arm2/speed=2.5/track arm3"
 -- Full AST: start/speed=5 motors arm1 arm2/speed=2.5/track arm3
 -- Verb+Qualifiers: start/speed=5
     -- Param+Qualifiers: motors
     -- Param+Qualifiers: arm1
     -- Param+Qualifiers: arm2/speed=2.5/track
     -- Param+Qualifiers: arm3
OK  "rotate camera1/notrack/axis=y/angle=45"
 -- Full AST: rotate camera1/notrack/axis=y/angle=45
 -- Verb+Qualifiers: rotate
     -- Param+Qualifiers: camera1/notrack/axis=y/angle=45

For completeness

  • Demo also Live On MSVC (Rextester) - note that RexTester uses Boost 1.60
  • Coliru uses Boost 1.66 but the problem doesn't manifest itself because now, there are concrete attribute values bound to parsers