通过协议缓冲区将 0 字节值从 C++ 发送到 python

Question

我正在尝试将我构建的 C++ 库中的字节字段中的图像数据发送到 python 程序。问题是 0 字节值似乎会扰乱协议缓冲区解析。当使用 0 以外的值时，一切正常。

image_buffer.proto

syntax = "proto3";

package transport;

message Response{
    repeated bytes img_data = 1;
}

test.cpp

extern "C" void* calculate()
{
    transport::Response response;

    unsigned char* data = new unsigned char [3];


    data[0] = 11;
    data[1] = 0; // this breaks the code, if other than 0 everything works fine
    data[2] = 120;

    response.add_img_data(data, 3);

    size_t out_size = response.ByteSizeLong();
    out = malloc(out_size);
    response.SerializeToArray(out, out_size);

    return out;
}

test.py

lib = ctypes.util.find_library("myLib")
libc = ctypes.cdll.LoadLibrary(lib)

calculate = libc.calculate
calculate.restype = c_char_p

result = calculate()

response = img_buf.Response()
response.ParseFromString(result) # i get a failed parse error here when using 0 values

for idx,img in enumerate(response.img_data):
    print("num pixels:",len(img))
    for pix in img:
        print(int(pix))

我已经为此苦苦挣扎了几天，所以如果有人有提示，我将不胜感激！

Answer 1

不要对二进制数据使用 .restype = c_char_p。 c_char_p 默认情况下作为 null-terminated 字节字符串处理并转换为 Python bytes 对象。相反，使用 POINTER(c_char) 将 return 然后可以正确处理的数据指针。但是，您需要知道 returned 缓冲区的大小。

这是一个例子：

test.cpp

#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

extern "C" API char* calculate(size_t* pout_size)
{
    // hard-coded data with embedded nulls for demonstration
    static char* data = "some[=10=]test[=10=]data";
    *pout_size = 14;
    return data;
}

test.py

import ctypes as ct

dll = ct.CDLL('./test')
dll.calculate.argtypes = ct.POINTER(ct.c_size_t),
dll.calculate.restype = ct.POINTER(ct.c_char)

def calculate():
    out_size = ct.c_size_t()                    # ctypes storage for output parameter
    result = dll.calculate(ct.byref(out_size))  # pass by reference
    return result[:out_size.value]              # slice to correct size and return byte string

result = calculate()
print(result)

输出：

b'some\x00test\x00data'

通过协议缓冲区将 0 字节值从 C++ 发送到 python

sending 0 byte values from c++ to python via protocoll buffers

c++

ctypes

protocol-buffers