使用 FParsec，如何在解析器之间使用 manyCharsTill 而不会在结束字符串上失败？

Question

我正在尝试使用 FParsec 来解析 TOML 多行字符串，但我在使用结束分隔符 (""") 时遇到了问题。我有以下解析器：

let controlChars = 
    ['\u0000'; '\u0001'; '\u0002'; '\u0003'; '\u0004'; '\u0005'; '\u0006'; '\u0007';
     '\u0008'; '\u0009'; '\u000a'; '\u000b'; '\u000c'; '\u000d'; '\u000e'; '\u000f';
     '\u0010'; '\u0011'; '\u0012'; '\u0013'; '\u0014'; '\u0015'; '\u0016'; '\u0017';
     '\u0018'; '\u0019'; '\u001a'; '\u001b'; '\u001c'; '\u001d'; '\u001e'; '\u001f';
     '\u007f']

let nonSpaceCtrlChars =
    Set.difference (Set.ofList controlChars) (Set.ofList ['\n';'\r';'\t'])

let multiLineStringContents : Parser<char,unit> =
    satisfy (isNoneOf nonSpaceCtrlChars)

let multiLineString         : Parser<string,unit> =
    optional newline >>. manyCharsTill multiLineStringContents (pstring "\"\"\"")
    |> between (pstring "\"\"\"") (pstring "\"\"\"") 

let test parser str =
    match run parser str with
    | Success (s1, s2, s3) -> printfn "Ok: %A %A %A" s1 s2 s3
    | Failure (f1, f2, f3) -> printfn "Fail: %A %A %A" f1 f2 f3

当我针对这样的输入测试 multiLineString 时：

test multiLineString "\"\"\"x\"\"\""

解析器失败并出现此错误：

Fail: "Error in Ln: 1 Col: 8 """x""" ^ Note: The error occurred at the end of the input stream. Expecting: '"""'

我对此感到困惑。 manyCharsTill multiLineStringContents (pstring "\"\"\"") 解析器不会在 """ 处停止让 between 解析器找到它吗？为什么解析器会吃掉所有输入然后使 between 解析器失败？

这似乎是相关的post：How to parse comments with FParsec

但我真的看不出那个问题的解决方案与我在这里所做的有何不同。

Answer 1

manyCharsTill documentation 说（强调我的）：

manyCharsTill cp endp parses chars with the char parser cp until the parser endp succeeds. It stops after endp and returns the parsed chars as a string.

因此您不想将 between 与 manyCharsTill 结合使用；你想做类似 pstring "\"\"\"" >>. manyCharsTill (pstring "\"\"\"").

的事情

但碰巧，我可以为你省去很多工作。在业余时间，我一直在使用 FParsec 开发 TOML 解析器。它远未完成，但字符串部分可以正常工作并正确处理反斜杠转义（据我所知：我已经彻底测试但并非详尽无遗）。我唯一缺少的是 "strip first newline if it appears right after the opening delimiter" 规则，您已使用 optional newline 处理过该规则。因此，只需将该位添加到我下面的代码中，您就应该拥有一个可用的 TOML 字符串解析器。

顺便说一句，我计划在 MIT 许可下许可我的代码（如果我完成了它）。所以我特此在 MIT 许可证下发布以下代码块。如果对你有用，欢迎在你的项目中使用。

let pShortCodepointInHex = // Anything from 0000 to FFFF, *except* the range D800-DFFF
    (anyOf "dD" >>. (anyOf "01234567" <?> "a Unicode scalar value (range D800-DFFF not allowed)") .>>. exactly 2 isHex |>> fun (c,s) -> sprintf "d%c%s" c s)
    <|> (exactly 4 isHex <?> "a Unicode scalar value")

let pLongCodepointInHex = // Anything from 00000000 to 0010FFFF, *except* the range D800-DFFF
        (pstring "0000" >>. pShortCodepointInHex)
        <|> (pstring "000"  >>. exactly 5 isHex)
        <|> (pstring "0010" >>. exactly 4 isHex |>> fun s -> "0010" + s)
        <?> "a Unicode scalar value (i.e., in range 00000000 to 0010FFFF)"

let toCharOrSurrogatePair p =
    p |> withSkippedString (fun codePoint _ -> System.Int32.Parse(codePoint, System.Globalization.NumberStyles.HexNumber) |> System.Char.ConvertFromUtf32)

let pStandardBackslashEscape =
    anyOf "\\"bfnrt"
    |>> function
        | 'b' -> "\b"      // U+0008 BACKSPACE
        | 'f' -> "\u000c"  // U+000C FORM FEED
        | 'n' -> "\n"      // U+000A LINE FEED
        | 'r' -> "\r"      // U+000D CARRIAGE RETURN
        | 't' -> "\t"      // U+0009 CHARACTER TABULATION a.k.a. Tab or Horizonal Tab
        | c   -> string c

let pUnicodeEscape =     (pchar 'u' >>. (pShortCodepointInHex |> toCharOrSurrogatePair))
                     <|> (pchar 'U' >>. ( pLongCodepointInHex |> toCharOrSurrogatePair))

let pEscapedChar = pstring "\" >>. (pStandardBackslashEscape <|> pUnicodeEscape)

let quote = pchar '"'
let isBasicStrChar c = c <> '\' && c <> '"' && c > '\u001f' && c <> '\u007f'
let pBasicStrChars = manySatisfy isBasicStrChar
let pBasicStr = stringsSepBy pBasicStrChars pEscapedChar |> between quote quote

let pEscapedNewline = skipChar '\' .>> skipNewline .>> spaces
let isMultilineStrChar c = c = '\n' || isBasicStrChar c
let pMultilineStrChars = manySatisfy isMultilineStrChar


let pTripleQuote = pstring "\"\"\""

let pMultilineStr = stringsSepBy pMultilineStrChars (pEscapedChar <|> (notFollowedByString "\"\"\"" >>. pstring "\"")) |> between pTripleQuote pTripleQuote

Answer 2

@rmunn 提供了正确答案，谢谢！在进一步使用 FParsec API 之后，我还以稍微不同的方式解决了这个问题。正如另一个答案中所解释的那样，manyCharTill 的 endp 参数正在吃掉结束 """，所以我需要切换到不会那样做的东西。使用 lookAhead 的简单修改就达到了目的：

let multiLineString         : Parser<string,unit> =
    optional newline >>. manyCharsTill multiLineStringContents (lookAhead (pstring "\"\"\""))
    |> between (pstring "\"\"\"") (pstring "\"\"\"")

使用 FParsec，如何在解析器之间使用 manyCharsTill 而不会在结束字符串上失败？

With FParsec, how does one use the manyCharsTill and between parsers and not fail on the closing string?

f#

parsing

fparsec