将 rbf 与 scipy 一起使用时出现内存错误
memory error by using rbf with scipy
我想像这里一样用 rbf 函数绘制一些点以获得点的密度分布:
如果我运行下面的代码,它工作正常:
from scipy.interpolate.rbf import Rbf # radial basis functions
import cv2
import matplotlib.pyplot as plt
import numpy as np
# import data
x = [1, 1, 2 ,3, 2, 7, 8, 6, 6, 7, 6.5, 7.5, 9, 8, 9, 8.5]
y = [0, 2, 5, 6, 1, 2, 9, 2, 3, 3, 2.5, 2, 8, 8, 9, 8.5]
d = np.ones(len(x))
print(d)
ti = np.linspace(-1,10)
xx, yy = np.meshgrid(ti, ti)
rbf = Rbf(x, y, d, function='gaussian')
jet = cm = plt.get_cmap('jet')
zz = rbf(xx, yy)
plt.pcolor(xx, yy, zz, cmap=jet)
plt.colorbar()
# Plotting the original points.
plot3 = plt.plot(x, y, 'ko', markersize=5) # the original points.
plt.show()
现在我想更改我的输入数据:
# import data
input = "testProbe.jpg"
image = cv2.imread(input) # load the image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # convert it to grayscale
# threshold the image to reveal light regions in the gray image
thresh = cv2.threshold(gray, 145, 200, cv2.THRESH_BINARY)[1]
x, y = np.where(thresh > 0)
d = np.ones(len(x))
我收到以下错误消息:
Traceback (most recent call last):
File "C:/Users/.../Python/pythonprojects/03a_test_rbfWithScipy.py", line 32, in <module>
rbf = Rbf(x, y, z, function='gaussian')
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 220, in __init__
self.nodes = linalg.solve(self.A, self.di)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 226, in A
r = self._call_norm(self.xi, self.xi)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 236, in _call_norm
return self.norm(x1, x2)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 118, in _euclidean_norm
return np.sqrt(((x1 - x2)**2).sum(axis=0))
MemoryError
错误消息出现得非常快,当我查看任务管理器时,PC 并未 运行 满负荷运行。他立即发出这条消息。那总有些事情是不对的,不是吗?
我尝试使用其他阈值 thresh = cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY)[1]
来获得更少的值并且它有效,但我收到以下错误消息:
C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py:220: LinAlgWarning:
scipy.linalg.solve Ill-conditioned matrix detected. Result is not guaranteed to be accurate.
Reciprocal condition number2.246772e-22 self.nodes = linalg.solve(self.A, self.di)
有什么想法吗?
如果我只剪掉一小部分图片,也可以。也许它对内存进行了一些初步计算并给我内存错误
在我看来 Rbf
插值计算量很大,会导致 O(N^2)
操作(如果我错了请纠正我)。因此,为了避免内存错误,您可以执行以下操作之一,而不是
zz = rbf(xx, yy)
1.用 nditer
迭代
慢,但适用于小型阵列:
for iz, ix, iy in np.nditer(
[zz, xx, yy],
flags=['external_loop', 'buffered'],
op_flags=['readwrite']
):
iz[...] = rbf(ix, iy)
2。使用 dask.array
更快的选项并使用线程
import dask.array as da
n1 = xx.shape[1]
ix = da.from_array(xx, chunks=(1, n1))
iy = da.from_array(yy, chunks=(1, n1))
iz = da.map_blocks(rbf, ix, iy)
zz = iz.compute()
希望这对你有用。
P.S:如 another question and scipy docs on Rbf, Rbf
also allows you to replace the norm
calculation function with a callable
(see LowLevelCallable and the Pythran blog 中所建议)。它有点复杂,但它也可能会解决您的问题,同时也会提高性能。
P.P.S:我通过这个修改设法加速了 50 倍:
import numpy as np
def euclidean_norm_numpy(x1, x2):
return np.linalg.norm(x1 - x2, axis=0)
rbf = Rbf(x, y, d, function='gaussian', norm=euclidean_norm_numpy)
我想像这里一样用 rbf 函数绘制一些点以获得点的密度分布:
如果我运行下面的代码,它工作正常:
from scipy.interpolate.rbf import Rbf # radial basis functions
import cv2
import matplotlib.pyplot as plt
import numpy as np
# import data
x = [1, 1, 2 ,3, 2, 7, 8, 6, 6, 7, 6.5, 7.5, 9, 8, 9, 8.5]
y = [0, 2, 5, 6, 1, 2, 9, 2, 3, 3, 2.5, 2, 8, 8, 9, 8.5]
d = np.ones(len(x))
print(d)
ti = np.linspace(-1,10)
xx, yy = np.meshgrid(ti, ti)
rbf = Rbf(x, y, d, function='gaussian')
jet = cm = plt.get_cmap('jet')
zz = rbf(xx, yy)
plt.pcolor(xx, yy, zz, cmap=jet)
plt.colorbar()
# Plotting the original points.
plot3 = plt.plot(x, y, 'ko', markersize=5) # the original points.
plt.show()
现在我想更改我的输入数据:
# import data
input = "testProbe.jpg"
image = cv2.imread(input) # load the image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # convert it to grayscale
# threshold the image to reveal light regions in the gray image
thresh = cv2.threshold(gray, 145, 200, cv2.THRESH_BINARY)[1]
x, y = np.where(thresh > 0)
d = np.ones(len(x))
我收到以下错误消息:
Traceback (most recent call last):
File "C:/Users/.../Python/pythonprojects/03a_test_rbfWithScipy.py", line 32, in <module>
rbf = Rbf(x, y, z, function='gaussian')
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 220, in __init__
self.nodes = linalg.solve(self.A, self.di)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 226, in A
r = self._call_norm(self.xi, self.xi)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 236, in _call_norm
return self.norm(x1, x2)
File "C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py", line 118, in _euclidean_norm
return np.sqrt(((x1 - x2)**2).sum(axis=0))
MemoryError
错误消息出现得非常快,当我查看任务管理器时,PC 并未 运行 满负荷运行。他立即发出这条消息。那总有些事情是不对的,不是吗?
我尝试使用其他阈值 thresh = cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY)[1]
来获得更少的值并且它有效,但我收到以下错误消息:
C:\Users\...\Python\Anaconda2\lib\site-packages\scipy\interpolate\rbf.py:220: LinAlgWarning:
scipy.linalg.solve Ill-conditioned matrix detected. Result is not guaranteed to be accurate.
Reciprocal condition number2.246772e-22 self.nodes = linalg.solve(self.A, self.di)
有什么想法吗?
如果我只剪掉一小部分图片,也可以。也许它对内存进行了一些初步计算并给我内存错误
在我看来 Rbf
插值计算量很大,会导致 O(N^2)
操作(如果我错了请纠正我)。因此,为了避免内存错误,您可以执行以下操作之一,而不是
zz = rbf(xx, yy)
1.用 nditer
迭代慢,但适用于小型阵列:
for iz, ix, iy in np.nditer(
[zz, xx, yy],
flags=['external_loop', 'buffered'],
op_flags=['readwrite']
):
iz[...] = rbf(ix, iy)
2。使用 dask.array
更快的选项并使用线程
import dask.array as da
n1 = xx.shape[1]
ix = da.from_array(xx, chunks=(1, n1))
iy = da.from_array(yy, chunks=(1, n1))
iz = da.map_blocks(rbf, ix, iy)
zz = iz.compute()
希望这对你有用。
P.S:如 another question and scipy docs on Rbf, Rbf
also allows you to replace the norm
calculation function with a callable
(see LowLevelCallable and the Pythran blog 中所建议)。它有点复杂,但它也可能会解决您的问题,同时也会提高性能。
P.P.S:我通过这个修改设法加速了 50 倍:
import numpy as np
def euclidean_norm_numpy(x1, x2):
return np.linalg.norm(x1 - x2, axis=0)
rbf = Rbf(x, y, d, function='gaussian', norm=euclidean_norm_numpy)