Getting Typeerror: can't concat str to bytes, when writing a test case to enter request files in django rest framework?

Question

我正在做一个项目，我可以添加一个 csv 来将衬衫的详细信息上传到数据库。

我在 django rest 框架中编写了一个视图集，该视图集采用从 request.data 输入的衬衫 csv 文件，并且正在为此编写一个测试用例。虽然，在编写测试用例时，我没有创建单独的 csv 文件，而是从测试用例中的 pandas 数据帧创建了一个 csv，并将其转换为 TextIOWrapper 以作为请求数据传递。这样做会引发以下错误：

  File "/Users/prasoon/projects/shirt_details/post_shirt_csv/tests/test_post_shirt_dataset.py", line 175, in test_shirt_dataset_post
    response = self.api_client.post(reverse('post-shirt-csv-dataset', kwargs={'session_id': self.session.id}), {'csv_files': [fp1, fp2]})
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/rest_framework/test.py", line 300, in post
    path, data=data, format=format, content_type=content_type, **extra)
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/rest_framework/test.py", line 212, in post
    data, content_type = self._encode_data(data, format, content_type)
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/rest_framework/test.py", line 184, in _encode_data
    ret = renderer.render(data)
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/rest_framework/renderers.py", line 920, in render
    return encode_multipart(self.BOUNDARY, data)
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/django/test/client.py", line 198, in encode_multipart
    lines.extend(encode_file(boundary, key, item))
  File "/Users/prasoon/projects/venv/lib/python3.6/site-packages/django/test/client.py", line 247, in encode_file
    to_bytes(file.read())
  File "/Users/prasoon/projects/venv/bin/../lib/python3.6/codecs.py", line 320, in decode
    data = self.buffer + input
TypeError: can't concat str to bytes

当我打开一个文件并传递它而不是创建字符串时，它就起作用了。下面会更清楚：

测试文件是：

class TestPostNewDataset(TestCase):
    @classmethod
    def setUpTestData(cls):
        cls.user = CurrentUserFactory()

        cls.test_data1 = {"no":[1,2,3],"colour":["Red","Blue","Gray"],"Size":["XL","L","M"],"Brand":["Nike","Polo","Adidas"],"Price":[800,600,1000]}
        cls.test_data2 = {"no":[1,2,3],"colour":["Black","Blue","Red"],"Size":["S","M","L"],"Brand":["Nike","Levis","Nike"],"Price":[2000,900,1000]}

        cls.test_df1 = pd.DataFrame(cls.test_data1)
        cls.test_df2 = pd.DataFrame(cls.test_data2)
        cls.session = CurrentSessionFactory(user=cls.user)

    def test_shirt_dataset_post(self):
        "Test request to post shirt dataset"

        fp1 = TextIOWrapper(StringIO(self.test_df1))
        fp2 = TextIOWrapper(StringIO(self.test_df2))

        response = self.api_client.post(reverse("post-shirt-csv-dataset", kwargs={'session_id':self.session.id}),{'csv_files':[fp1,fp2]})

        self.assertEqual(response.status_code,status.HTTP_200_OK)

对应的views文件为：

class CsvReadViewset(viewsets.ModelViewSet):
    authentication_classes = (BasicAuthentication,)
    permission_classes = (IsAuthenticated,)
    serializer_class = ShirtSerializer
    queryset = Shirt.objects.all()

    def addShirt(self,request,*args,**kwargs):
        csvs = request.data["csv_files"]
        current_session = CurrentSession.objects.get(session_id=kwargs["session_id"])
        for shirt_csv in csvs:
            shirt_df = pd.read_csv(shirt_csv)
            for index,row in short_df.iterrows():
                s = Shirt(colour=row["colour"],
                          brand=row["Brand"],
                          size=row["Size"],
                          price=row["Price"],
                          created_in=current_session)
                s.save()
        return Response(status=status.HTTP_200_OK)

有趣的是，当测试用例将一个单独的文件作为输入时，它工作正常。也就是说，如果我将 test_shirt_dataset_post 替换为以下内容：

    def test_shirt_dataset_post(self):
        with open(os.path.join(os.path.abspath(__file__ + "/../"), 'shirtcsv1.csv')) as fp1, open(os.path.join(os.path.abspath(__file__ + "/../"), 'shirtcsv2.csv')) as fp2:
             response = self.api_client.post(reverse("post-shirt-csv-dataset",kwargs={'session_id':self.session.id}),{'csv_files':[fp1,fp2]})

        self.assertEqual(response.status_code,status.HTTP_200_OK)

预期结果是测试用例起作用。但是，这里它会抛出一个错误，如图所示。添加TextIOWrapper的原因是因为with open(....) as fp中fp的类型是'_io.TextIOWrapper'。虽然，然后它抛出这个错误。任何帮助将不胜感激。

Answer 1

找到可能的答案。因此，将字符串文件作为文件对象传递的一种方法是使用 NamedTemporaryFile。因此，test_shirt_dataset_post 的代码更改为：


from tempfile import NamedTemporaryFile

def test_shirt_dataset_post(self):
    self.test_df1.set_index("id",inplace=True)
    self.test_df2.set_index("id",inplace=True)

    df1_csv_str = self.test_df1.to_csv().encode('utf-8')
    df2_csv_str = self.test_df2.to_csv().encode('utf-8')

    tp1 = NamedTemporaryFile()
    tp1.write(df1_csv_str)
    tp1.flush()

    tp2 = NamedTemporaryFile()
    tp2.write(df2_csv_str)
    tp2.flush()

    with open(tp1.name,'r') as fp1, open(tp2.name,'r') as fp2:
        response = self.api_client.post(reverse("post-shirt-csv-dataset", kwargs={'session_id':self.session.id}),{'csv_files':[fp1,fp2]})

        self.assertEqual(response.status_code,status.HTTP_200_OK)

Answer 2

您可以使用io.BytesIO发送编码字节流：

def test_shirt_dataset_post(self):
    "Test request to post shirt dataset"

    fp1 = BytesIO(self.test_df1.to_csv().encode('utf-8'))
    fp2 = BytesIO(self.test_df2.to_csv().encode('utf-8'))

    response = self.api_client.post(reverse("post-shirt-csv-dataset", kwargs={'session_id':self.session.id}),{'csv_files':[fp1,fp2]})

    self.assertEqual(response.status_code,status.HTTP_200_OK)

Getting Typeerror: can't concat str to bytes, when writing a test case to enter request files in django rest framework?

Getting Typeerror: can't concat str to bytes, when writing a test case to enter request files in django rest framework?

django

pandas

io

django-rest-framework

django-tests