boost::spirit 递归命令式 c++ 语法:BOOST_FUSION_ADAPT_STRUCT 失败
boost::spirit recursive imperative c++ grammar: BOOST_FUSION_ADAPT_STRUCT fails
我一直在研究一种在 C++ 中解析命令式语句(if/else/do/while/for/switch 等)的语法。所有其他语句都保留为字符串。
我目前只使用 if/else 进行测试(尽管其他语句在变体中应该类似)。
不幸的是我得到一个编译时错误:
Error 1 error C2440: 'return' : cannot convert from 'std::vector<
someSeqNode,std::allocator<Ty>>' to 'boost::fusion::vector<
someSeqNode, boost::fusion::void, boost::fusion::void_,
boost::fusion::void_, boost::fusion::void_, boost::fusion::void_,
boost::fusion::void_, boost::fusion::void_, boost::fusion::void_,
boost::fusion::void_> &'
on line 80 (closing bracket of BOOST_FUSION_ADAPT_STRUCT for ifElseStruct)
我发现的关于 BOOST_FUSION_ADAPT_STRUCT 和属性传播的编译问题的唯一其他问题是关于只有一个成员的结构,或者适应的结构和规则之间不兼容的属性。
适应 ifElseStruct.ifContent 似乎失败了,但我不明白为什么。解析 typedef 它真的只是 vector< variant< ifElseStruct, string>>.
是递归的问题吗?如果是这样,我该如何解决?
#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#pragma region INCLUDE_STUFF
#include <vector>
#include <string>
#include <iostream>
//boost includes for parser and some collection types (e.g. tuple)
#include <boost/spirit/include/qi.hpp>
#include <boost/phoenix/fusion.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/fusion/include/boost_tuple.hpp>
#include <boost/optional.hpp>
#include <boost/optional/optional_io.hpp>
#include <boost/spirit/include/qi_raw.hpp>
#include <boost/variant.hpp>
#include "vectorStreamOp.h"//overload stream operator << for std::vector -> for BOOST_SPIRIT_DEBUG
#pragma endregion INCLUDE_STUFF
#pragma region NAMESPACE_STUFF
//to shorten calls fruther down
namespace phx = boost::phoenix;
namespace qi = boost::spirit::qi;
using std::string;
using std::cout;
using std::endl;
using std::vector;
using boost::spirit::qi::parse;
using boost::optional;
using phx::ref;
using phx::at_c;
using qi::char_;
using qi::lexeme;
using qi::_1;
using qi::lit;
using qi::alnum;
using qi::alpha;
using qi::space;
using qi::raw;
using qi::as_string;
#pragma endregion NAMESPACE_STUFF
#pragma region STRUCT_STUFF
/*later make this variant with all impStatementVariants -> either make this a vector to have sequences on all levels or make imperativeCpp derive from this
-> typedef variant<
recursive_wrapper<ifElseStruct>,
recursive_wrapper<switchStruct>,
recursive_wrapper<forStruct>,
recursive_wrapper<whileStruct>,
recursive_wrapper<doWhileStruct>
*/
struct ifElseStruct;
typedef boost::variant<ifElseStruct, string> someSeqNode;
struct ifElseStruct
{
string ifCond;
vector<someSeqNode> ifContent;
optional<vector<someSeqNode>> elseContent;
//for BOOST DEBUG
friend std::ostream& operator<< (std::ostream& stream, const ifElseStruct& var) {
stream << "ifCond: " << var.ifCond << " ifContent: " << var.ifContent << endl << "elseContent:" << var.elseContent;
return stream;
}
};
BOOST_FUSION_ADAPT_STRUCT(
ifElseStruct,
(string, ifCond)
(vector<someSeqNode>, ifContent)
(optional<vector<someSeqNode>>, elseContent)
)
#pragma endregion STRUCT_STUFF
#pragma region GRAMMAR_STUFF
//GRAMMAR for flowcontrol (branching and looping)
template<typename Iterator, typename Skipper> struct imperativeGrammar :qi::grammar<Iterator, vector<someSeqNode>(), Skipper>
{
imperativeGrammar() : imperativeGrammar::base_type(startRule)
{
startRule = *(recursiveImpCpp | nestedSomething); //vector<variant<ifElseStruct(), string>>
recursiveImpCpp = ifElseNormalRule.alias() /*| switchRule | whileRule | forRule ...*/;
//attr: ifElseStruct containing-> string, vector<someSeqNode>, optional<vector<someSeqNode>>
ifElseNormalRule = lit("if")>> '(' >> condition >> ')' >> ifElseContent >> -(lit("else") >> ifElseContent);
condition = *~char_(")");//TODO: replace with nestedSomething rule
ifElseContent = ('{' >> startRule >> '}') /*| singleStatement*/;
singleStatement = !recursiveImpCpp >> (qi::as_string[*~char_(";")] >> qi::as_string[char_(';')]);
nestedSomething = !recursiveImpCpp >> qi::as_string[*~char_("(){}")]
>> -(raw['(' >> nestedSomething >> ')']) >> -(raw['{' >> nestedSomething >> '}'])
>> !recursiveImpCpp >> qi::as_string[*~char_("(){}")];
BOOST_SPIRIT_DEBUG_NODES((startRule)(ifElseNormalRule)(ifElseContent))
}
qi::rule<Iterator, vector<someSeqNode>(), Skipper> startRule;
qi::rule<Iterator, ifElseStruct(), Skipper> recursiveImpCpp;
qi::rule<Iterator, ifElseStruct(), Skipper> ifElseNormalRule;
qi::rule<Iterator, string(), Skipper> condition;
qi::rule<Iterator, vector<someSeqNode>(), Skipper> ifElseContent;
qi::rule<Iterator, std::string(), Skipper> nestedSomething;
qi::rule<Iterator, std::string(), Skipper> singleStatement;
/*qi::rule<Iterator, Skipper> forRule;
qi::rule<Iterator, Skipper> switchCaseBreakRule;
qi::rule<Iterator, Skipper> whileRule;
qi::rule<Iterator, Skipper> doWhileRule;*/
};
#pragma endregion GRAMMAR_STUFF
有很多问题。
- 如评论所说,不要使用using-directives;他们让你陷入困境
如果你想在 nestedSomething
中连接所有源字符串,那么只需将其全部包装在 raw[]
(或 as_string[raw[...]]
中,但这甚至没有必要), 例如
nestedSomething = !recursiveImpCpp >> qi::raw[*~char_("(){}")
>> -('(' >> nestedSomething >> ')')
>> -('{' >> nestedSomething >> '}')
>> !recursiveImpCpp >> *~char_("(){}")];
该规则在匹配空字符串的意义上被打破。这使得语法永远不会结束(它将匹配 "infinite" 数量的空 nestedSomething
)。您将不得不决定一些非可选部分。这是一个蛮力修复:
qi::raw[...] [ qi::_pass = px::size(qi::_1) > 0 ];
即便如此,对于非平凡的程序,语法仍会陷入无限循环(尝试使用自己的源代码)。以下内容应该稍微清楚一点:
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}");
parenthesized = '(' >> -nestedSomething >> ')';
bracketed = '{' >> -nestedSomething >> '}';
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -bracketed
>> -identifier_soup] [ qi::_pass = px::size(qi::_1) > 0 ];
但这仍然无法解析,例如int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }
)。原因是 main(<parenthesized>){<bracketed>}
只接受括号内的 nestedSomething
;明确禁止 if-else
构造...
让我们将 ifElseContent
重命名为适当的名称(例如 statement
)
block = '{' >> startRule >> '}';
statement = block | singleStatement;
并用它代替 bracketed
:
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -block
>> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
一般各种注意事项
- 您可以简化包含,而不是将它们包裹在区域中以隐藏它们
- 你也可以简化其余部分(见我的演示)
- 所有负面的前瞻性语法将变得非常低效。考虑标记化或使用更细粒度的关键字检测方案(使用 Qi 存储库中的期望点和/或
distinct()[]
)
部分演示
应用上面的注释:
#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/include/adapted.hpp>
#include <boost/optional/optional_io.hpp>
#ifdef BOOST_SPIRIT_DEBUG
namespace std {
template <typename... T, typename... V>
auto& operator<<(basic_ostream<T...>& os, vector<V...> const& v) {
os << "{";
for (auto& el : v) os << el << " ";
return os;
}
}
#endif
namespace qi = boost::spirit::qi;
/* later make this variant with all impStatementVariants -> either make this a
* vector to have sequences on all levels or make imperativeCpp derive from
* this
-> typedef variant<
recursive_wrapper<ifElseStruct>,
recursive_wrapper<switchStruct>,
recursive_wrapper<forStruct>,
recursive_wrapper<whileStruct>,
recursive_wrapper<doWhileStruct>
*
*/
struct ifElseStruct;
typedef boost::variant<ifElseStruct, std::string> someSeqNode;
struct ifElseStruct
{
std::string ifCond;
std::vector<someSeqNode> ifContent;
boost::optional<std::vector<someSeqNode>> elseContent;
friend std::ostream& operator<< (std::ostream& stream, const ifElseStruct& var) {
stream << "ifCond: " << var.ifCond << " ifContent: " << var.ifContent << std::endl << "elseContent:" << var.elseContent;
return stream;
}
};
BOOST_FUSION_ADAPT_STRUCT(ifElseStruct, ifCond, ifContent, elseContent)
//GRAMMAR for flowcontrol (branching and looping)
template<typename Iterator, typename Skipper> struct imperativeGrammar :qi::grammar<Iterator, std::vector<someSeqNode>(), Skipper>
{
imperativeGrammar() : imperativeGrammar::base_type(startRule)
{
startRule = *(recursiveImpCpp | nestedSomething); //vector<variant<ifElseStruct(), string>>
recursiveImpCpp = ifElseNormalRule.alias() /*| switchRule | whileRule | forRule ...*/;
//attr: ifElseStruct containing-> string, vector<someSeqNode>, optional<vector<someSeqNode>>
ifElseNormalRule = qi::lit("if") >> '(' >> condition >> ')' >> statement >> -("else" >> statement);
condition = *~qi::char_(")");//TODO: replace with nestedSomething rule
block = '{' >> startRule >> '}';
statement = block | singleStatement;
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}");
parenthesized = '(' >> -nestedSomething >> ')';
bracketed = '{' >> -nestedSomething >> '}';
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -block
>> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
singleStatement = !recursiveImpCpp >> qi::raw[*~qi::char_(';') >> ';'];
BOOST_SPIRIT_DEBUG_NODES((startRule)(ifElseNormalRule)(statement)(block)(nestedSomething)(identifier_soup)(parenthesized)(bracketed)(singleStatement))
}
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> startRule;
qi::rule<Iterator, ifElseStruct(), Skipper> recursiveImpCpp;
qi::rule<Iterator, ifElseStruct(), Skipper> ifElseNormalRule;
qi::rule<Iterator, std::string(), Skipper> condition;
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> block, statement;
qi::rule<Iterator, std::string(), Skipper> nestedSomething;
qi::rule<Iterator, std::string(), Skipper> singleStatement;
qi::rule<Iterator, Skipper> identifier_soup, parenthesized, bracketed;
/*qi::rule<Iterator, Skipper> forRule;
qi::rule<Iterator, Skipper> switchCaseBreakRule;
qi::rule<Iterator, Skipper> whileRule;
qi::rule<Iterator, Skipper> doWhileRule;*/
};
#include <fstream>
int main() {
//std::string const input = { std::istreambuf_iterator<char>(std::ifstream("main.cpp").rdbuf()), {} };
std::string const input = "int main() { if(true) { std::cout << \"Yes\n\"; } else { std::cout << \"No\n\"; } }";
using It = std::string::const_iterator;
It f(input.begin()), l(input.end());
imperativeGrammar<It, qi::space_type> p;
std::vector<someSeqNode> rep;
bool ok = phrase_parse(f, l, p, qi::space, rep);
if (ok) {
std::cout << "Parse success: " << rep << "\n";
}
else
std::cout << "Parse failure\n";
if (f!=l)
std::cout << "Remaining unparsed input: '" << std::string(f,l) << "'\n";
}
打印:
Parse success: {int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }
有调试信息
<startRule>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>int main() { if(true</try>
<identifier_soup>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<success>() { if(true) { std:</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>() { if(true) { std:</try>
<nestedSomething>
<try>) { if(true) { std::</try>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>) { if(true) { std::</try>
<fail/>
</parenthesized>
<block>
<try>) { if(true) { std::</try>
<fail/>
</block>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> { if(true) { std::c</success>
<attributes>[]</attributes>
</parenthesized>
<block>
<try> { if(true) { std::c</try>
<startRule>
<try> if(true) { std::cou</try>
<ifElseNormalRule>
<try> if(true) { std::cou</try>
<statement>
<try> { std::cout << "Yes</try>
<block>
<try> { std::cout << "Yes</try>
<startRule>
<try> std::cout << "Yes\n</try>
<ifElseNormalRule>
<try> std::cout << "Yes\n</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "Yes\n</try>
<identifier_soup>
<try>std::cout << "Yes\n"</try>
<ifElseNormalRule>
<try>std::cout << "Yes\n"</try>
<fail/>
</ifElseNormalRule>
<success>} else { std::cout <</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} else { std::cout <</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} else { std::cout <</try>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} else { std::cout <</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</startRule>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</block>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</statement>
<statement>
<try> { std::cout << "No\</try>
<block>
<try> { std::cout << "No\</try>
<startRule>
<try> std::cout << "No\n"</try>
<ifElseNormalRule>
<try> std::cout << "No\n"</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "No\n"</try>
<identifier_soup>
<try>std::cout << "No\n";</try>
<ifElseNormalRule>
<try>std::cout << "No\n";</try>
<fail/>
</ifElseNormalRule>
<success>} }</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} }</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} }</try>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</startRule>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</block>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</statement>
<success> }</success>
<attributes>[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]</attributes>
</ifElseNormalRule>
<ifElseNormalRule>
<try> }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> }</try>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>}</try>
<fail/>
</parenthesized>
<block>
<try>}</try>
<fail/>
</block>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> }</success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</startRule>
<success></success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success></success>
<attributes>[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try></try>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try></try>
<fail/>
</parenthesized>
<block>
<try></try>
<fail/>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success></success>
<attributes>[[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]]</attributes>
</startRule>
Summary/advice
如果您 运行 上面的示例 on its own source 您会注意到它在第 9 行的命名空间上退出。尽管程序完成而不是崩溃是件好事,但这并不令人鼓舞。
你需要一个正确的语法。你需要思考什么是语句,什么是块,什么是关键字以及它们之间的关系。您的部分困惑似乎来自将 expressions 与 statements.
混为一谈
我会非常考虑提出一种语法,您 1. 可以工作 2. 您知道为什么它会工作,而不是...似乎只是尝试一些东西来通过解析编程语言来欺骗您。
您是否考虑过如何解析 std::cout << "int main() { return 0; }";
?
您是否考虑过带有行延续的宏?
我一直在研究一种在 C++ 中解析命令式语句(if/else/do/while/for/switch 等)的语法。所有其他语句都保留为字符串。 我目前只使用 if/else 进行测试(尽管其他语句在变体中应该类似)。 不幸的是我得到一个编译时错误:
Error 1 error C2440: 'return' : cannot convert from 'std::vector< someSeqNode,std::allocator<Ty>>' to 'boost::fusion::vector< someSeqNode, boost::fusion::void, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_> &'
on line 80 (closing bracket of BOOST_FUSION_ADAPT_STRUCT for ifElseStruct)
我发现的关于 BOOST_FUSION_ADAPT_STRUCT 和属性传播的编译问题的唯一其他问题是关于只有一个成员的结构,或者适应的结构和规则之间不兼容的属性。
适应 ifElseStruct.ifContent 似乎失败了,但我不明白为什么。解析 typedef 它真的只是 vector< variant< ifElseStruct, string>>.
是递归的问题吗?如果是这样,我该如何解决?
#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#pragma region INCLUDE_STUFF
#include <vector>
#include <string>
#include <iostream>
//boost includes for parser and some collection types (e.g. tuple)
#include <boost/spirit/include/qi.hpp>
#include <boost/phoenix/fusion.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/fusion/include/boost_tuple.hpp>
#include <boost/optional.hpp>
#include <boost/optional/optional_io.hpp>
#include <boost/spirit/include/qi_raw.hpp>
#include <boost/variant.hpp>
#include "vectorStreamOp.h"//overload stream operator << for std::vector -> for BOOST_SPIRIT_DEBUG
#pragma endregion INCLUDE_STUFF
#pragma region NAMESPACE_STUFF
//to shorten calls fruther down
namespace phx = boost::phoenix;
namespace qi = boost::spirit::qi;
using std::string;
using std::cout;
using std::endl;
using std::vector;
using boost::spirit::qi::parse;
using boost::optional;
using phx::ref;
using phx::at_c;
using qi::char_;
using qi::lexeme;
using qi::_1;
using qi::lit;
using qi::alnum;
using qi::alpha;
using qi::space;
using qi::raw;
using qi::as_string;
#pragma endregion NAMESPACE_STUFF
#pragma region STRUCT_STUFF
/*later make this variant with all impStatementVariants -> either make this a vector to have sequences on all levels or make imperativeCpp derive from this
-> typedef variant<
recursive_wrapper<ifElseStruct>,
recursive_wrapper<switchStruct>,
recursive_wrapper<forStruct>,
recursive_wrapper<whileStruct>,
recursive_wrapper<doWhileStruct>
*/
struct ifElseStruct;
typedef boost::variant<ifElseStruct, string> someSeqNode;
struct ifElseStruct
{
string ifCond;
vector<someSeqNode> ifContent;
optional<vector<someSeqNode>> elseContent;
//for BOOST DEBUG
friend std::ostream& operator<< (std::ostream& stream, const ifElseStruct& var) {
stream << "ifCond: " << var.ifCond << " ifContent: " << var.ifContent << endl << "elseContent:" << var.elseContent;
return stream;
}
};
BOOST_FUSION_ADAPT_STRUCT(
ifElseStruct,
(string, ifCond)
(vector<someSeqNode>, ifContent)
(optional<vector<someSeqNode>>, elseContent)
)
#pragma endregion STRUCT_STUFF
#pragma region GRAMMAR_STUFF
//GRAMMAR for flowcontrol (branching and looping)
template<typename Iterator, typename Skipper> struct imperativeGrammar :qi::grammar<Iterator, vector<someSeqNode>(), Skipper>
{
imperativeGrammar() : imperativeGrammar::base_type(startRule)
{
startRule = *(recursiveImpCpp | nestedSomething); //vector<variant<ifElseStruct(), string>>
recursiveImpCpp = ifElseNormalRule.alias() /*| switchRule | whileRule | forRule ...*/;
//attr: ifElseStruct containing-> string, vector<someSeqNode>, optional<vector<someSeqNode>>
ifElseNormalRule = lit("if")>> '(' >> condition >> ')' >> ifElseContent >> -(lit("else") >> ifElseContent);
condition = *~char_(")");//TODO: replace with nestedSomething rule
ifElseContent = ('{' >> startRule >> '}') /*| singleStatement*/;
singleStatement = !recursiveImpCpp >> (qi::as_string[*~char_(";")] >> qi::as_string[char_(';')]);
nestedSomething = !recursiveImpCpp >> qi::as_string[*~char_("(){}")]
>> -(raw['(' >> nestedSomething >> ')']) >> -(raw['{' >> nestedSomething >> '}'])
>> !recursiveImpCpp >> qi::as_string[*~char_("(){}")];
BOOST_SPIRIT_DEBUG_NODES((startRule)(ifElseNormalRule)(ifElseContent))
}
qi::rule<Iterator, vector<someSeqNode>(), Skipper> startRule;
qi::rule<Iterator, ifElseStruct(), Skipper> recursiveImpCpp;
qi::rule<Iterator, ifElseStruct(), Skipper> ifElseNormalRule;
qi::rule<Iterator, string(), Skipper> condition;
qi::rule<Iterator, vector<someSeqNode>(), Skipper> ifElseContent;
qi::rule<Iterator, std::string(), Skipper> nestedSomething;
qi::rule<Iterator, std::string(), Skipper> singleStatement;
/*qi::rule<Iterator, Skipper> forRule;
qi::rule<Iterator, Skipper> switchCaseBreakRule;
qi::rule<Iterator, Skipper> whileRule;
qi::rule<Iterator, Skipper> doWhileRule;*/
};
#pragma endregion GRAMMAR_STUFF
有很多问题。
- 如评论所说,不要使用using-directives;他们让你陷入困境
如果你想在
nestedSomething
中连接所有源字符串,那么只需将其全部包装在raw[]
(或as_string[raw[...]]
中,但这甚至没有必要), 例如nestedSomething = !recursiveImpCpp >> qi::raw[*~char_("(){}") >> -('(' >> nestedSomething >> ')') >> -('{' >> nestedSomething >> '}') >> !recursiveImpCpp >> *~char_("(){}")];
该规则在匹配空字符串的意义上被打破。这使得语法永远不会结束(它将匹配 "infinite" 数量的空
nestedSomething
)。您将不得不决定一些非可选部分。这是一个蛮力修复:qi::raw[...] [ qi::_pass = px::size(qi::_1) > 0 ];
即便如此,对于非平凡的程序,语法仍会陷入无限循环(尝试使用自己的源代码)。以下内容应该稍微清楚一点:
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}"); parenthesized = '(' >> -nestedSomething >> ')'; bracketed = '{' >> -nestedSomething >> '}'; nestedSomething %= qi::raw[ -identifier_soup >> -parenthesized >> -bracketed >> -identifier_soup] [ qi::_pass = px::size(qi::_1) > 0 ];
但这仍然无法解析,例如
int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }
)。原因是main(<parenthesized>){<bracketed>}
只接受括号内的nestedSomething
;明确禁止if-else
构造...让我们将
ifElseContent
重命名为适当的名称(例如statement
)block = '{' >> startRule >> '}'; statement = block | singleStatement;
并用它代替
bracketed
:nestedSomething %= qi::raw[ -identifier_soup >> -parenthesized >> -block >> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
一般各种注意事项
- 您可以简化包含,而不是将它们包裹在区域中以隐藏它们
- 你也可以简化其余部分(见我的演示)
- 所有负面的前瞻性语法将变得非常低效。考虑标记化或使用更细粒度的关键字检测方案(使用 Qi 存储库中的期望点和/或
distinct()[]
)
部分演示
应用上面的注释:
#define BOOST_SPIRIT_DEBUG
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/fusion/include/adapted.hpp>
#include <boost/optional/optional_io.hpp>
#ifdef BOOST_SPIRIT_DEBUG
namespace std {
template <typename... T, typename... V>
auto& operator<<(basic_ostream<T...>& os, vector<V...> const& v) {
os << "{";
for (auto& el : v) os << el << " ";
return os;
}
}
#endif
namespace qi = boost::spirit::qi;
/* later make this variant with all impStatementVariants -> either make this a
* vector to have sequences on all levels or make imperativeCpp derive from
* this
-> typedef variant<
recursive_wrapper<ifElseStruct>,
recursive_wrapper<switchStruct>,
recursive_wrapper<forStruct>,
recursive_wrapper<whileStruct>,
recursive_wrapper<doWhileStruct>
*
*/
struct ifElseStruct;
typedef boost::variant<ifElseStruct, std::string> someSeqNode;
struct ifElseStruct
{
std::string ifCond;
std::vector<someSeqNode> ifContent;
boost::optional<std::vector<someSeqNode>> elseContent;
friend std::ostream& operator<< (std::ostream& stream, const ifElseStruct& var) {
stream << "ifCond: " << var.ifCond << " ifContent: " << var.ifContent << std::endl << "elseContent:" << var.elseContent;
return stream;
}
};
BOOST_FUSION_ADAPT_STRUCT(ifElseStruct, ifCond, ifContent, elseContent)
//GRAMMAR for flowcontrol (branching and looping)
template<typename Iterator, typename Skipper> struct imperativeGrammar :qi::grammar<Iterator, std::vector<someSeqNode>(), Skipper>
{
imperativeGrammar() : imperativeGrammar::base_type(startRule)
{
startRule = *(recursiveImpCpp | nestedSomething); //vector<variant<ifElseStruct(), string>>
recursiveImpCpp = ifElseNormalRule.alias() /*| switchRule | whileRule | forRule ...*/;
//attr: ifElseStruct containing-> string, vector<someSeqNode>, optional<vector<someSeqNode>>
ifElseNormalRule = qi::lit("if") >> '(' >> condition >> ')' >> statement >> -("else" >> statement);
condition = *~qi::char_(")");//TODO: replace with nestedSomething rule
block = '{' >> startRule >> '}';
statement = block | singleStatement;
identifier_soup = !recursiveImpCpp >> +~qi::char_("(){}");
parenthesized = '(' >> -nestedSomething >> ')';
bracketed = '{' >> -nestedSomething >> '}';
nestedSomething %= qi::raw[ -identifier_soup
>> -parenthesized
>> -block
>> -identifier_soup] [ qi::_pass = boost::phoenix::size(qi::_1) > 0 ];
singleStatement = !recursiveImpCpp >> qi::raw[*~qi::char_(';') >> ';'];
BOOST_SPIRIT_DEBUG_NODES((startRule)(ifElseNormalRule)(statement)(block)(nestedSomething)(identifier_soup)(parenthesized)(bracketed)(singleStatement))
}
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> startRule;
qi::rule<Iterator, ifElseStruct(), Skipper> recursiveImpCpp;
qi::rule<Iterator, ifElseStruct(), Skipper> ifElseNormalRule;
qi::rule<Iterator, std::string(), Skipper> condition;
qi::rule<Iterator, std::vector<someSeqNode>(), Skipper> block, statement;
qi::rule<Iterator, std::string(), Skipper> nestedSomething;
qi::rule<Iterator, std::string(), Skipper> singleStatement;
qi::rule<Iterator, Skipper> identifier_soup, parenthesized, bracketed;
/*qi::rule<Iterator, Skipper> forRule;
qi::rule<Iterator, Skipper> switchCaseBreakRule;
qi::rule<Iterator, Skipper> whileRule;
qi::rule<Iterator, Skipper> doWhileRule;*/
};
#include <fstream>
int main() {
//std::string const input = { std::istreambuf_iterator<char>(std::ifstream("main.cpp").rdbuf()), {} };
std::string const input = "int main() { if(true) { std::cout << \"Yes\n\"; } else { std::cout << \"No\n\"; } }";
using It = std::string::const_iterator;
It f(input.begin()), l(input.end());
imperativeGrammar<It, qi::space_type> p;
std::vector<someSeqNode> rep;
bool ok = phrase_parse(f, l, p, qi::space, rep);
if (ok) {
std::cout << "Parse success: " << rep << "\n";
}
else
std::cout << "Parse failure\n";
if (f!=l)
std::cout << "Remaining unparsed input: '" << std::string(f,l) << "'\n";
}
打印:
Parse success: {int main() { if(true) { std::cout << "Yes\n"; } else { std::cout << "No\n"; } }
有调试信息
<startRule>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>int main() { if(true</try>
<identifier_soup>
<try>int main() { if(true</try>
<ifElseNormalRule>
<try>int main() { if(true</try>
<fail/>
</ifElseNormalRule>
<success>() { if(true) { std:</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>() { if(true) { std:</try>
<nestedSomething>
<try>) { if(true) { std::</try>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>) { if(true) { std::</try>
<fail/>
</parenthesized>
<block>
<try>) { if(true) { std::</try>
<fail/>
</block>
<identifier_soup>
<try>) { if(true) { std::</try>
<ifElseNormalRule>
<try>) { if(true) { std::</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> { if(true) { std::c</success>
<attributes>[]</attributes>
</parenthesized>
<block>
<try> { if(true) { std::c</try>
<startRule>
<try> if(true) { std::cou</try>
<ifElseNormalRule>
<try> if(true) { std::cou</try>
<statement>
<try> { std::cout << "Yes</try>
<block>
<try> { std::cout << "Yes</try>
<startRule>
<try> std::cout << "Yes\n</try>
<ifElseNormalRule>
<try> std::cout << "Yes\n</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "Yes\n</try>
<identifier_soup>
<try>std::cout << "Yes\n"</try>
<ifElseNormalRule>
<try>std::cout << "Yes\n"</try>
<fail/>
</ifElseNormalRule>
<success>} else { std::cout <</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} else { std::cout <</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} else { std::cout <</try>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} else { std::cout <</try>
<fail/>
</parenthesized>
<block>
<try>} else { std::cout <</try>
<fail/>
</block>
<identifier_soup>
<try>} else { std::cout <</try>
<ifElseNormalRule>
<try>} else { std::cout <</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} else { std::cout <</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</startRule>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</block>
<success> else { std::cout <<</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]]]</attributes>
</statement>
<statement>
<try> { std::cout << "No\</try>
<block>
<try> { std::cout << "No\</try>
<startRule>
<try> std::cout << "No\n"</try>
<ifElseNormalRule>
<try> std::cout << "No\n"</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> std::cout << "No\n"</try>
<identifier_soup>
<try>std::cout << "No\n";</try>
<ifElseNormalRule>
<try>std::cout << "No\n";</try>
<fail/>
</ifElseNormalRule>
<success>} }</success>
<attributes>[]</attributes>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success>} }</success>
<attributes>[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try>} }</try>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>} }</try>
<fail/>
</parenthesized>
<block>
<try>} }</try>
<fail/>
</block>
<identifier_soup>
<try>} }</try>
<ifElseNormalRule>
<try>} }</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success>} }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</startRule>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</block>
<success> }</success>
<attributes>[[[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]</attributes>
</statement>
<success> }</success>
<attributes>[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]</attributes>
</ifElseNormalRule>
<ifElseNormalRule>
<try> }</try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try> }</try>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try>}</try>
<fail/>
</parenthesized>
<block>
<try>}</try>
<fail/>
</block>
<identifier_soup>
<try>}</try>
<ifElseNormalRule>
<try>}</try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success> }</success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</startRule>
<success></success>
<attributes>[[[[t, r, u, e], [[s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, ]], [[s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, ]]]]]</attributes>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<success></success>
<attributes>[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]</attributes>
</nestedSomething>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<nestedSomething>
<try></try>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<parenthesized>
<try></try>
<fail/>
</parenthesized>
<block>
<try></try>
<fail/>
</block>
<identifier_soup>
<try></try>
<ifElseNormalRule>
<try></try>
<fail/>
</ifElseNormalRule>
<fail/>
</identifier_soup>
<fail/>
</nestedSomething>
<success></success>
<attributes>[[[i, n, t, , m, a, i, n, (, ), , {, , i, f, (, t, r, u, e, ), , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", Y, e, s, \, n, ", ;, , }, , e, l, s, e, , {, , s, t, d, :, :, c, o, u, t, , <, <, , ", N, o, \, n, ", ;, , }, , }]]]</attributes>
</startRule>
Summary/advice
如果您 运行 上面的示例 on its own source 您会注意到它在第 9 行的命名空间上退出。尽管程序完成而不是崩溃是件好事,但这并不令人鼓舞。
你需要一个正确的语法。你需要思考什么是语句,什么是块,什么是关键字以及它们之间的关系。您的部分困惑似乎来自将 expressions 与 statements.
混为一谈我会非常考虑提出一种语法,您 1. 可以工作 2. 您知道为什么它会工作,而不是...似乎只是尝试一些东西来通过解析编程语言来欺骗您。
您是否考虑过如何解析 std::cout << "int main() { return 0; }";
?
您是否考虑过带有行延续的宏?