numpy.vstack 与 python 的 numba 的正确用法
Correct usage of numpy.vstack with python's numba
我目前正在尝试使用 numba
加速一些 python 代码。根据 documentation of numba,numpy.meshgrid
不受支持,但 numpy.vstack
受支持。所以我用 vstack
替换了 meshgrid
调用,这在不使用 numba
时工作正常。但是,当使用 numba
.
时,它不起作用
代码如下:
import numpy as np
import timeit
from numba import njit, prange
@njit(parallel=True)
def method1_par1(n):
# create some data
x = np.linspace(0,2*np.pi,n)
y = np.linspace(0,2*np.pi,n)
# np.meshgrid not supported by numba, therefore trying to use np.vstack
X = np.vstack( n*[x] )
Y = np.hstack( n*[y[:,np.newaxis]] ) # tranform row vector into column vector, then duplicate
Z = np.sin(X) * np.sin(Y)
# calculate centered finite difference using for loop
Z_diff1 = Z*.0
for ii in prange(1,Z.shape[0]-1,2):
for jj in range(1,Z.shape[1]-1,2):
Z_diff1[ii,jj] = Z[ii+1, jj] - Z[ii-1,jj]
runs = 1000
print( min(timeit.repeat( "method1_par1(50)", "from __main__ import method1_par1",
number=runs )) )
这是错误消息:
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array(float64, 1d, C), Tuple(slice<a:b>, none))
There are 22 candidate implementations:
- Of which 20 did not match due to:
Overload of function 'getitem': File: <numerous>: Line N/A.
With argument(s): '(array(float64, 1d, C), Tuple(slice<a:b>, none))':
No match.
- Of which 2 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array(float64, 1d, C), Tuple(slice<a:b>, none))':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type none in Tuple(slice<a:b>, none)
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/arraydecl.py:68
During: typing of intrinsic-call at forloop_slicing_parallel_q.py (12)
During: typing of static-get-item at forloop_slicing_parallel_q.py (12)
File "forloop_slicing_parallel_q.py", line 12:
def method1_par1(n):
<source elided>
# np.meshgrid not supported by numba, therefore trying to use np.vstack
Y = np.hstack( n*[y[:,np.newaxis]] )
^
在我看来,我使用 vstack
的方式不受支持,对吗?
我使用的版本:
numpy: 1.20.3
numba: 0.54.1
python: 3.8.10
Update1:使用np.concatenate( n*[[x]] )
会导致类似的错误:
>>> mul(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)
There are 12 candidate implementations:
- Of which 10 did not match due to:
Overload of function 'mul': File: <numerous>: Line N/A.
With argument(s): '(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)':
No match.
- Of which 2 did not match due to:
Operator Overload in function 'mul': File: unknown: Line unknown.
With argument(s): '(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)':
No match for registered cases:
* (int64, int64) -> int64
* (int64, uint64) -> int64
* (uint64, int64) -> int64
* (uint64, uint64) -> uint64
* (float32, float32) -> float32
* (float64, float64) -> float64
* (complex64, complex64) -> complex64
* (complex128, complex128) -> complex128
Update2:使用np.hstack( n*[y[:,np.newaxis]] )
导致与原始错误相同的错误。
使用 y.reshape(-1,1).repeat(n,1)
会导致以下错误:
No implementation of function Function(<function array_repeat at 0x7f0dac54ee50>) found for signature:
>>> array_repeat(array(float64, 2d, C), int64, Literal[int](1))
There are 2 candidate implementations:
- Of which 2 did not match due to:
Overload in function 'array_repeat': File: numba/np/arrayobj.py: Line 2066.
With argument(s): '(array(float64, 2d, C), int64, int64)':
Rejected as the implementation raised a specific error:
TypeError: array_repeat() takes 2 positional arguments but 3 were given
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/templates.py:775
- Resolution failure for non-literal arguments:
No implementation of function Function(<function array_repeat at 0x7f0dac54ee50>) found for signature:
>>> array_repeat(array(float64, 2d, C), int64, int64)
There are 2 candidate implementations:
- Of which 2 did not match due to:
Overload in function 'array_repeat': File: numba/np/arrayobj.py: Line 2066.
With argument(s): '(array(float64, 2d, C), int64, int64)':
Rejected as the implementation raised a specific error:
TypeError: array_repeat() takes 2 positional arguments but 3 were given
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/templates.py:775
During: resolving callee type: BoundFunction((<class 'numba.core.types.npytypes.Array'>, 'repeat') for array(float64, 2d, C))
如果您仔细查看错误消息,您会看到它说
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array(float64, 1d, C), Tuple(slice<a:b>, none))
getitem
是 numba 编译 []
运算符的方式。签名显示 numba 不支持像 array[slice, None]
这样的调用。具体问题是
y[:, np.newaxis]
Numba 确实支持重塑操作和重复,但没有 axis
参数,因此您可以将该行更改为
Y = y.repeat(n).reshape(n, n)
我目前正在尝试使用 numba
加速一些 python 代码。根据 documentation of numba,numpy.meshgrid
不受支持,但 numpy.vstack
受支持。所以我用 vstack
替换了 meshgrid
调用,这在不使用 numba
时工作正常。但是,当使用 numba
.
代码如下:
import numpy as np
import timeit
from numba import njit, prange
@njit(parallel=True)
def method1_par1(n):
# create some data
x = np.linspace(0,2*np.pi,n)
y = np.linspace(0,2*np.pi,n)
# np.meshgrid not supported by numba, therefore trying to use np.vstack
X = np.vstack( n*[x] )
Y = np.hstack( n*[y[:,np.newaxis]] ) # tranform row vector into column vector, then duplicate
Z = np.sin(X) * np.sin(Y)
# calculate centered finite difference using for loop
Z_diff1 = Z*.0
for ii in prange(1,Z.shape[0]-1,2):
for jj in range(1,Z.shape[1]-1,2):
Z_diff1[ii,jj] = Z[ii+1, jj] - Z[ii-1,jj]
runs = 1000
print( min(timeit.repeat( "method1_par1(50)", "from __main__ import method1_par1",
number=runs )) )
这是错误消息:
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array(float64, 1d, C), Tuple(slice<a:b>, none))
There are 22 candidate implementations:
- Of which 20 did not match due to:
Overload of function 'getitem': File: <numerous>: Line N/A.
With argument(s): '(array(float64, 1d, C), Tuple(slice<a:b>, none))':
No match.
- Of which 2 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array(float64, 1d, C), Tuple(slice<a:b>, none))':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type none in Tuple(slice<a:b>, none)
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/arraydecl.py:68
During: typing of intrinsic-call at forloop_slicing_parallel_q.py (12)
During: typing of static-get-item at forloop_slicing_parallel_q.py (12)
File "forloop_slicing_parallel_q.py", line 12:
def method1_par1(n):
<source elided>
# np.meshgrid not supported by numba, therefore trying to use np.vstack
Y = np.hstack( n*[y[:,np.newaxis]] )
^
在我看来,我使用 vstack
的方式不受支持,对吗?
我使用的版本:
numpy: 1.20.3
numba: 0.54.1
python: 3.8.10
Update1:使用np.concatenate( n*[[x]] )
会导致类似的错误:
>>> mul(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)
There are 12 candidate implementations:
- Of which 10 did not match due to:
Overload of function 'mul': File: <numerous>: Line N/A.
With argument(s): '(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)':
No match.
- Of which 2 did not match due to:
Operator Overload in function 'mul': File: unknown: Line unknown.
With argument(s): '(int64, list(list(array(float64, 1d, C))<iv=None>)<iv=None>)':
No match for registered cases:
* (int64, int64) -> int64
* (int64, uint64) -> int64
* (uint64, int64) -> int64
* (uint64, uint64) -> uint64
* (float32, float32) -> float32
* (float64, float64) -> float64
* (complex64, complex64) -> complex64
* (complex128, complex128) -> complex128
Update2:使用np.hstack( n*[y[:,np.newaxis]] )
导致与原始错误相同的错误。
使用 y.reshape(-1,1).repeat(n,1)
会导致以下错误:
No implementation of function Function(<function array_repeat at 0x7f0dac54ee50>) found for signature:
>>> array_repeat(array(float64, 2d, C), int64, Literal[int](1))
There are 2 candidate implementations:
- Of which 2 did not match due to:
Overload in function 'array_repeat': File: numba/np/arrayobj.py: Line 2066.
With argument(s): '(array(float64, 2d, C), int64, int64)':
Rejected as the implementation raised a specific error:
TypeError: array_repeat() takes 2 positional arguments but 3 were given
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/templates.py:775
- Resolution failure for non-literal arguments:
No implementation of function Function(<function array_repeat at 0x7f0dac54ee50>) found for signature:
>>> array_repeat(array(float64, 2d, C), int64, int64)
There are 2 candidate implementations:
- Of which 2 did not match due to:
Overload in function 'array_repeat': File: numba/np/arrayobj.py: Line 2066.
With argument(s): '(array(float64, 2d, C), int64, int64)':
Rejected as the implementation raised a specific error:
TypeError: array_repeat() takes 2 positional arguments but 3 were given
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/templates.py:775
During: resolving callee type: BoundFunction((<class 'numba.core.types.npytypes.Array'>, 'repeat') for array(float64, 2d, C))
如果您仔细查看错误消息,您会看到它说
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array(float64, 1d, C), Tuple(slice<a:b>, none))
getitem
是 numba 编译 []
运算符的方式。签名显示 numba 不支持像 array[slice, None]
这样的调用。具体问题是
y[:, np.newaxis]
Numba 确实支持重塑操作和重复,但没有 axis
参数,因此您可以将该行更改为
Y = y.repeat(n).reshape(n, n)