如何使用 QuickCheck 调试发散测试

Question

我有一些使用 Megaparsec 的解析代码，我写了一个简单的属性来测试（它生成一个运行dom 表达式树，漂亮地打印它，然后检查结果解析回原始树）。

不幸的是，似乎有一个错误，如果我运行测试没有任何限制，我会看到 GHC 进程分配越来越多的内存，直到我杀死它或 OOM 杀手先到达那里。

没问题，我想...但我终究无法弄清楚是什么导致了分歧。属性本身看起来像这样：（我已经删除了适当的测试和收缩代码以尽量减少实际运行s 的代码）

prop_parse_expr :: Property
prop_parse_expr =
  forAll arbitrary $
  (\ pe ->
     let str = prettyParExpr 0 pe in
       counterexample ("Rendered to: " ++ show str) $
       trace ("STARTING TEST: " ++ show str) $
       case parse (expr <* eof) "" str of
         Left _ -> trace "NOPE" $ False
         Right _ -> trace "GOOD" $ True)

如果我使用概要分析进行编译（使用 stack test --profile），我可以运行生成带有 RTS 选项的二进制文件。啊哈，我想运行和 -xc，认为当我向卡住的作业发送 SIGINT 时我会得到一个有用的堆栈跟踪。好像没有。运行

./long/path/to/foo-test -j1 --test-seed 1 +RTS -xc

我看到这个输出：

STARTING TEST: "0"
GOOD
STARTING TEST: "(x [( !0 )]) "
STARTING TEST: "({ 2 {( !0 )}} ) "
STARTING TEST: "{ 2{ ( x[0? {( 0) ,( x ) } :((0 )? (x ):0) -: ( -(^( x  )) ) ]), 0**( x )} } "
STARTING TEST: "| (0? (x[({ 1{ (0)? x : ( 0 ) }} ) ]) :(~&( 0) ?( x):( (x ) ^( x ) )))"
STARTING TEST: "(0 )"
STARTING TEST: "0"
^C*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main
<snip: 2 more identical backtraces>
*** Exception (reporting due to +RTS -xc): (THUNK_STATIC), stack trace: 
  Test.Framework.Runners.Console.Utilities.hideCursorDuring,
  called from Test.Framework.Runners.Console.Run.showRunTestsTop,
  called from Test.Framework.Improving.runImprovingIO,
  called from Test.Framework.Providers.QuickCheck2.runProperty,
  called from Test.Framework.Providers.QuickCheck2.runTest,
  called from Test.Framework.Runners.Core.runSimpleTest,
  called from Test.Framework.Runners.Core.runTestTree.go,
  called from Test.Framework.Runners.Core.runTestTree,
  called from Test.Framework.Runners.Core.runTests',
  called from Test.Framework.Runners.Core.runTests,
  called from Test.Framework.Runners.Console.defaultMainWithOpts,
  called from Test.Framework.Runners.Console.defaultMainWithArgs,
  called from Test.Framework.Runners.Console.defaultMain,
  called from Main.main

谁能告诉我：

为什么我看到多行 STARTING TEST 行之间没有 GOOD 或 NOPE，尽管有 -j1?
我如何获得显示测试分配所有内存的实际堆栈跟踪？

感谢任何想法！

Answer 1

对于发现此问题的任何人，我的代码的问题是我的 arbitrary 表达式实例没有正确限制大小，因此有时会尝试制作巨大的树。请参阅 QuickCheck manual 的 "Generating Recursive Data Types" 部分了解我应该做的事情！

我发现运行命令如下：

./long/path/to/foo-test -o3 +RTS -xc

帮助我弄清楚发生了什么。奇怪的是，回溯仍然显示了几个执行线程。我真的不明白为什么，但至少我可以看到我在我的“makeAnExpr”功能上花费了时间。诀窍是调整超时（上面 3 秒），这样它就不会终止测试，直到它真正卡住，但在它开始吃掉你所有的 RAM 之前停止它！

如何使用 QuickCheck 调试发散测试

How to debug diverging test using QuickCheck

haskell

ghc

quickcheck