从 numpy 结构化数组中提取 python 'native' 值

Question

我有一个结构化的 numpy 数组。

numpy 结构匹配类型google.protobuf.Timestamp。

我需要从所述结构的每个元素中提取 seconds int64 和 nanos int32 并将其分配给 real Timestamp结构。

下面我列出了一个脚本，它以方便任何人测试的方式执行此操作（需要安装 numpy 和 protobuf Python 模块）。

如何获取 rid/circumvent 最后列出的 TypeError 并在 Timestamp 变量中获取 numpy 结构之外的值？

import numpy as np
from google.protobuf import timestamp_pb2

# numpy structure that mimics google.protobuf.Timestamp
Timestamp_t = np.dtype([('seconds', np.int64), ('nanos', np.int32)])

# populate numpy array with above structure
x_values_size = 3
x_values = np.empty((x_values_size,), dtype=Timestamp_t)
x_values['seconds'] = np.linspace(0, 100, num=x_values_size, dtype=np.int64)
x_values['nanos']   = np.linspace(0, 10, num=x_values_size, dtype=np.int32)

# copy data from numpy structured array to a descriptor-created Timestamp
for elem in np.nditer(x_values) :
    # destination protobuf structure (actually, part of some sequence)
    # try 1: this will actually change the type of 'ts'
    ts1 = timestamp_pb2.Timestamp()
    print(type(ts1)) # Timestamp as expected
    ts1 = elem
    print(ts1) # now a numpy.ndarray
    print(type(ts1))
    print(ts1.dtype)

    # try 2: assign member by member
    ts2 = timestamp_pb2.Timestamp()
    # fails with:
    # TypeError: array(0, dtype=int64) has type <class 'numpy.ndarray'>, but expected one of: (<class 'int'>,)
    ts2.seconds = elem['seconds']
    ts2.nanos = elem['nanos']
    print("-----")

免责声明：python 和 numpy 数组的铁杆新手。

Answer 1

所以

In [112]: x_values
Out[112]: 
array([(  0,  0), ( 50,  5), (100, 10)], 
      dtype=[('seconds', '<i8'), ('nanos', '<i4')])

我通常不建议使用 nditer 除非您需要特殊行为。数组上的简单迭代（如果是 2d，则为行）通常就是您所需要的。但是为了更好地理解发生了什么，让我们比较一下迭代方法：

In [114]: for elem in np.nditer(x_values):
     ...:     print(elem, elem.dtype)
     ...:     print(type(elem))   
(0, 0) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.ndarray'>
(50, 5) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.ndarray'>
(100, 10) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.ndarray'>

In [115]: for elem in x_values:
     ...:     print(elem, elem.dtype)
     ...:     print(type(elem))
(0, 0) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.void'>
(50, 5) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.void'>
(100, 10) [('seconds', '<i8'), ('nanos', '<i4')]
<class 'numpy.void'>

除 type 不同外，np.ndarray 与 np.void 相同。修改nditer变量更容易。

做同样的事情，但只看一个字段：

In [119]: for elem in np.nditer(x_values):
     ...:     print(elem['seconds'], type(elem['seconds']))   
0 <class 'numpy.ndarray'>
50 <class 'numpy.ndarray'>
100 <class 'numpy.ndarray'>

In [120]: for elem in x_values:
     ...:     print(elem['seconds'], type(elem['seconds']))
0 <class 'numpy.int64'>
50 <class 'numpy.int64'>
100 <class 'numpy.int64'>

我没有protobuf密码，但我怀疑

ts2.seconds = elem['seconds']

将在第二次迭代中更好地工作，即产生 np.int64 值的迭代。或添加 elem['seconds'].item().

从 numpy 结构化数组中提取 python 'native' 值

Extract python 'native' values from numpy structured array

python

numpy

structured-array