这是使用 Python 3 unittest 测试 stdout 的正确方法吗？

Question

假设我有一个提交文件，fileFromStudent.py，里面只有：

print("hello world")

我想测试标准输出，看看学生是否正确地写出了打印语句。根据我所阅读的内容，我已经能够创建以下代码：

from io import StringIO
from unittest.mock import patch
import unittest, importlib, sys

class TestStringMethods(unittest.TestCase):
    def setUp(self):
        studentSubmission = 'fileFromStudent'

        ## Stores output from print() in fakeOutput
        with patch('sys.stdout', new=StringIO()) as self.fakeOutput:
            ## Loads submission on first test, reloads on subsequent tests
            if studentSubmission in sys.modules:
                importlib.reload(sys.modules[ studentSubmission ] )
            else:
                importlib.import_module( studentSubmission )

    ## Test Cases
    def test_print_passes(self):
        test_case = "Checking Output Statement - Will Pass"
        self.output = self.fakeOutput.getvalue().strip()
        self.assertEqual(self.output, 'hello world', msg=test_case)

    def test_print_fails(self):
        test_case = "Checking Output Statement - Will Fail"
        self.output = self.fakeOutput.getvalue().strip()
        self.assertEqual(self.output, 'hell world', msg=test_case)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestStringMethods)
    testResult = unittest.TextTestRunner(verbosity=2).run(suite)

以上方法有效，但我的处理方式是否正确？我添加的其中一件事是 import.reload() 调用以重新加载学生的程序。这是因为在最初几周，我会让学生使用 print() 作为他们的最终输出（直到我们进入函数）。

我知道它看起来模棱两可，或者为什么我应该打扰它，但上面的代码是构建它的正确方法还是我完全缺少使这一切变得简单的东西？

Answer 1

我花了几个星期的时间来解决这个问题，取得了一定的成功，但也有一些头痛，而且 Google。我没有走 Popen 路线的原因之一是我想捕获学生是否提交了立即崩溃的错误代码。信不信由你，入门课程的前几周就是这样。因为我找到的所有东西都是 2011-2012 年的，所以我想我会 post 这样未来 Google 的人就可以找到它。

扩展我上面写的内容，让我们假设下一个作业是获取输入并说 "Hi"

name = input("What's your name? ")
print("Hi " + name)

现在，我想自动化测试，看看我是否可以输入 "Adam" 并返回 "Hi Adam"。为此，我选择使用 StringIO 作为标准输入 (sys.stdin = StringIO("Adam"))。这使我能够控制文本流的来源和去向。另外，我不想看到一个学生可能发生的所有错误(sys.stderr = StringIO())。

正如我提到的，我选择使用 importlib 而不是 Popen。我想确保如果学生提交了伪造的代码，而不是破坏一切，只是让我的测试失败运行。我对 subprocess 和 py.test 进行了试验，虽然它们可能更好、更干净，但我找不到任何对我来说有意义的关于如何让它正确移动的东西。

下面是我最新版本的测试副本：

from io import StringIO
from unittest.mock import patch
import unittest, importlib, sys, os
from time import sleep

# setup the environment
backup = sys.stderr

class TestTypingExercise(unittest.TestCase):
    def __init__(self, test_name, filename, inputs):
        super(TestTypingExercise, self).__init__(test_name)
        self.library = filename.split('.')[0]
        self.inputs = inputs

    def setUp(self):
        sys.stdin = StringIO(self.inputs[0])
        try:
            ## Stores output from print() in fakeOutput
            with patch('sys.stdout', new=StringIO()) as self.fakeOutput:
                ## Loads submission on first test, reloads on subsequent tests
                if self.library in sys.modules:
                    importlib.reload(sys.modules[ self.library ] )
                else:
                    importlib.import_module( self.library )
        except Exception as e:
            self.fail("Failed to Load - {0}".format(str(e)))

    ## Test Cases
    def test_code_runs(self):
        test_case = "Checking to See if code can run"
        self.assertTrue(True, msg=test_case)

    def test_says_hello(self):
        test_case = "Checking to See if code said 'Hi Adam'"
        # Regex might be cleaner, but this typically solves most cases
        self.output = self.fakeOutput.getvalue().strip().lower()
        self.assertTrue('hi adam' in self.output, msg=test_case)

if __name__ == '__main__':
    ignore_list = ["grader.py"]

    # Run Through Each Submitted File
    directory = os.listdir('.')
    for filename in sorted(directory):
        if (filename.split('.')[-1] != 'py') or (filename in ignore_list):
            continue
        #print("*"*15, filename, "*"*15)

        # 'Disables' stderr, so I don't have to see all their errors
        sys.stderr = StringIO()     # capture output

        # Run Tests Across Student's Submission
        suite = unittest.TestSuite()
        suite.addTest(TestTypingExercise('test_code_runs', filename, 'Adam'))
        suite.addTest(TestTypingExercise('test_says_hello', filename, 'Adam'))
        results = unittest.TextTestRunner().run(suite)

        # Reset stderr
        out = sys.stderr.getvalue() # release output
        sys.stderr.close()  # close the stream 
        sys.stderr = backup # restore original stderr

        # Display Test Results
        print(filename,"Test Results - ", end='')
        if not results.wasSuccessful():
            print("Failed (test cases that failed):")
            for error in results.failures:
                print('\t',error[1].split('\n')[-2])
        else:
            print("Pass!")
        sleep(0.05)

这是最终结果：

StudentSubmission01.py Test Results - Failed (test cases that failed):
     AssertionError: Failed to Load - EOL while scanning string literal (StudentSubmission01.py, line 23)
     AssertionError: Failed to Load - EOL while scanning string literal (StudentSubmission01.py, line 23)
StudentSubmission02.py Test Results - Pass!
StudentSubmission03.py Test Results - Pass!
StudentSubmission04.py Test Results - Pass!
StudentSubmission05.py Test Results - Pass!
StudentSubmission06.py Test Results - Pass!
StudentSubmission07.py Test Results - Pass!
StudentSubmission08.py Test Results - Pass!
StudentSubmission09.py Test Results - Pass!
StudentSubmission10.py Test Results - Pass!
StudentSubmission11.py Test Results - Pass!
StudentSubmission12.py Test Results - Pass!
StudentSubmission13.py Test Results - Pass!
[Finished in 0.9s]

如果我想测试多个不同的输入，我可能需要四处移动，但现在这可行。

这是使用 Python 3 unittest 测试 stdout 的正确方法吗？

Is this a proper way to test stdout with Python 3 unittest?

python

unit-testing

python-unittest