在具有透明度的 3D 图像上叠加 2D 热图
Superimposing 2D heat map on 3D Image with transparency
我有一个 3D python 图像及其 2D 特征向量。我用 imshow() 显示了每个，如下面的代码所示，我可以清楚地看到它。我现在想要的是 将 2D 特征向量作为热图叠加在其 3D 图像 之上。我尝试添加它们但出现了维度问题，我通过添加第三维扩展了 2D 特征向量，但叠加图像被弄乱了。尺寸在这里：-
功能 >> (32,96) ,
图片 >> (32,96,3)
img_feature = np.uint8(feature22 + img_raw_np_resize)
ax0.imshow(img,extent =extent)
ax1.imshow(feature,alpha = 0.75, interpolation = 'gaussian', cmap = plt.cm.jet,extent =extent)
ax2.imshow(img_feature,alpha = 0.75, interpolation = 'gaussian', cmap = plt.cm.jet,extent =extent)
extent指的是让所有的显示都以相同的大小显示，它已经定义好了。
这是我通过上面的代码尝试的结果
我正在寻找如下所示的显示...
这是执行此操作的伪代码（我正在尝试生成最少的代码来共享，但这需要时间 - 所以我分享它以防它有助于找出问题：-
def save_feature_to_img_2(self):
feature = (
get_feature()
) # get feature of size -- > torch.Size([1, 256, 32,96])
feature2 = features[
:, 0, :, :
] # I pick one from the 256 feature maps - [1,1,32,96]
feature2 = (
feature2.data.numpy()
) # convert it to numpy array for later fusion with image array - [1,1,32,96]
features2 = features2.view(
features2.shape[1], features2.shape[2]
) # reshape into 2D feature map [32,96]
img_raw1 = Image.open(self.img_path) # read raw image size [128,64,3]
img_raw_np = np.array(img_raw1) # imge to numpy array
newsize = (h, w) # shape to re-size image to same size a feature
img_raw_resize = img_raw1.resize(newsize)
img_raw_np_resize = np.array(img_raw_resize) # size is now [32,96,3]
# use sigmod to normalize feature to [0,1]
feature2 = 1.0 / (1 + np.exp(-1 * feature2))
# display setup
dx, dy = 0.05, 0.05
y = np.arange(-8, 8, dx) # Y - axis range
x = np.arange(-4.0, 4.0, dy) # X -axis range
X, Y = np.meshgrid(x, y) # meshgrid to enclose my display into
extent = (
np.min(x),
np.max(x),
np.min(y),
np.max(y),
) # extent - the X and Y range - just to make unifrom display
feature2 = (255.5 * feature2 / np.amax(feature2)).astype(
np.uint8
) # put feature into [0-255] range for colour image
# to display img, feature and img_feature
fig = plt.figure()
ax0 = fig.add_subplot(131, title="Image")
ax1 = fig.add_subplot(132, title="Heatmap")
ax2 = fig.add_subplot(133, title="overlayed")
# hereunder I used this code share in the answer to fuse
alpha = 0.5
img_heatmap = (
feature2[:, :, None].astype(np.float64) * alpha
+ img_raw_np_resize * (1 - alpha)
).astype(np.uint8)
ax0.imshow(img_raw1, alpha=1.0, interpolation="gaussian", cmap=plt.cm.jet)
ax1.imshow(feature2, alpha=1.0, interpolation="gaussian", cmap=plt.cm.jet)
ax2.imshow(
img_heatmap, alpha=0.7, interpolation="gaussian", cmap=plt.cm.jet
)
cv2.imwrite("./img_heatmap.jpg", img_heatmap)
这是我得到的新显示器
我使用以下方法来融合图像和特征...
alpha = 0.5
img_feature = ((plt.cm.jet(feature2)[:, :, :3] * 255) * alpha + (1-alpha)*img_raw_np_resize).astype(np.uint8) and displaying it with ax2.imshow(img_feature,alpha = 0.7, interpolation = 'gaussian', cmap = plt.cm.jet)
变体 1
如果您只想显示图像的加热区域，则必须乘以热图而不是相加。
您的公式为 img_feature = (img * (feature[:, :, None].astype(np.float64) / np.amax(feature))).astype(np.uint8)。
完整示例代码（带有我自己的图像和自动生成的示例热图）：
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
img_feature = (img * (feature[:, :, None].astype(np.float64) / np.amax(feature))).astype(np.uint8)
# show images
fig, (ax0, ax1, ax2) = plt.subplots(1, 3)
ax0.imshow(img)
ax1.imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
ax2.imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
plt.show()
输出：
变体 2
如果您只想使加热的区域变亮（变白）并使未加热的区域变暗（变黑），您只需按公式 alpha = 0.5; img_feature = (feature[:, :, None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8).
进行 alpha 混合
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = (feature[:, :, None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig, (ax0, ax1, ax2) = plt.subplots(1, 3)
ax0.imshow(img)
ax1.imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
ax2.imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
plt.show()
输出：
变体 3
与 Variant-2（带 alpha 混合）相同，但使用基于 matplotlib.pyplot.cm.jet 着色方案的 RGB 特征数组而不是黑白特征数组。
正如您从代码中看到的那样，您可以在表达式 plt.cm.jet(feature) 中使用 any coloring scheme 而不是 plt.cm.jet 颜色。
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = ((plt.cm.jet(feature)[:, :, :3] * 255) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig, axs = plt.subplots(2, 2)
axs[0, 0].imshow(img)
axs[0, 1].imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
axs[1, 0].imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
axs[1, 1].remove()
plt.show()
输出：
PS。我刚刚注意到 Matplotlib 在绘制热图时进行了归一化 (feature - min) / (max - min) 因此我决定在叠加公式中做同样的事情，最终公式变成这样：
alpha = 0.5; img_feature = ((
plt.cm.jet(
(feature - np.amin(feature)).astype(np.float32)
/ (np.amax(feature) - np.amin(feature)).astype(np.float32)
)[:, :, :3] * 255
) * alpha + img * (1 - alpha)).astype(np.uint8)
使用上述公式的示例代码is here (or here, and for your images). Resulting image of using formula above is here (and for your images)。最终叠加图像：
我有一个 3D python 图像及其 2D 特征向量。我用 imshow() 显示了每个，如下面的代码所示，我可以清楚地看到它。我现在想要的是 将 2D 特征向量作为热图叠加在其 3D 图像 之上。我尝试添加它们但出现了维度问题，我通过添加第三维扩展了 2D 特征向量，但叠加图像被弄乱了。尺寸在这里：- 功能 >> (32,96) , 图片 >> (32,96,3)
img_feature = np.uint8(feature22 + img_raw_np_resize)
ax0.imshow(img,extent =extent)
ax1.imshow(feature,alpha = 0.75, interpolation = 'gaussian', cmap = plt.cm.jet,extent =extent)
ax2.imshow(img_feature,alpha = 0.75, interpolation = 'gaussian', cmap = plt.cm.jet,extent =extent)
extent指的是让所有的显示都以相同的大小显示，它已经定义好了。
这是我通过上面的代码尝试的结果
我正在寻找如下所示的显示...
这是执行此操作的伪代码（我正在尝试生成最少的代码来共享，但这需要时间 - 所以我分享它以防它有助于找出问题：-
def save_feature_to_img_2(self):
feature = (
get_feature()
) # get feature of size -- > torch.Size([1, 256, 32,96])
feature2 = features[
:, 0, :, :
] # I pick one from the 256 feature maps - [1,1,32,96]
feature2 = (
feature2.data.numpy()
) # convert it to numpy array for later fusion with image array - [1,1,32,96]
features2 = features2.view(
features2.shape[1], features2.shape[2]
) # reshape into 2D feature map [32,96]
img_raw1 = Image.open(self.img_path) # read raw image size [128,64,3]
img_raw_np = np.array(img_raw1) # imge to numpy array
newsize = (h, w) # shape to re-size image to same size a feature
img_raw_resize = img_raw1.resize(newsize)
img_raw_np_resize = np.array(img_raw_resize) # size is now [32,96,3]
# use sigmod to normalize feature to [0,1]
feature2 = 1.0 / (1 + np.exp(-1 * feature2))
# display setup
dx, dy = 0.05, 0.05
y = np.arange(-8, 8, dx) # Y - axis range
x = np.arange(-4.0, 4.0, dy) # X -axis range
X, Y = np.meshgrid(x, y) # meshgrid to enclose my display into
extent = (
np.min(x),
np.max(x),
np.min(y),
np.max(y),
) # extent - the X and Y range - just to make unifrom display
feature2 = (255.5 * feature2 / np.amax(feature2)).astype(
np.uint8
) # put feature into [0-255] range for colour image
# to display img, feature and img_feature
fig = plt.figure()
ax0 = fig.add_subplot(131, title="Image")
ax1 = fig.add_subplot(132, title="Heatmap")
ax2 = fig.add_subplot(133, title="overlayed")
# hereunder I used this code share in the answer to fuse
alpha = 0.5
img_heatmap = (
feature2[:, :, None].astype(np.float64) * alpha
+ img_raw_np_resize * (1 - alpha)
).astype(np.uint8)
ax0.imshow(img_raw1, alpha=1.0, interpolation="gaussian", cmap=plt.cm.jet)
ax1.imshow(feature2, alpha=1.0, interpolation="gaussian", cmap=plt.cm.jet)
ax2.imshow(
img_heatmap, alpha=0.7, interpolation="gaussian", cmap=plt.cm.jet
)
cv2.imwrite("./img_heatmap.jpg", img_heatmap)
这是我得到的新显示器
我使用以下方法来融合图像和特征...
alpha = 0.5
img_feature = ((plt.cm.jet(feature2)[:, :, :3] * 255) * alpha + (1-alpha)*img_raw_np_resize).astype(np.uint8) and displaying it with ax2.imshow(img_feature,alpha = 0.7, interpolation = 'gaussian', cmap = plt.cm.jet)
变体 1
如果您只想显示图像的加热区域，则必须乘以热图而不是相加。
您的公式为 img_feature = (img * (feature[:, :, None].astype(np.float64) / np.amax(feature))).astype(np.uint8)。
完整示例代码（带有我自己的图像和自动生成的示例热图）：
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
img_feature = (img * (feature[:, :, None].astype(np.float64) / np.amax(feature))).astype(np.uint8)
# show images
fig, (ax0, ax1, ax2) = plt.subplots(1, 3)
ax0.imshow(img)
ax1.imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
ax2.imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
plt.show()
输出：
变体 2
如果您只想使加热的区域变亮（变白）并使未加热的区域变暗（变黑），您只需按公式 alpha = 0.5; img_feature = (feature[:, :, None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8).
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = (feature[:, :, None].astype(np.float64) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig, (ax0, ax1, ax2) = plt.subplots(1, 3)
ax0.imshow(img)
ax1.imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
ax2.imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
plt.show()
输出：
变体 3
与 Variant-2（带 alpha 混合）相同，但使用基于 matplotlib.pyplot.cm.jet 着色方案的 RGB 特征数组而不是黑白特征数组。
正如您从代码中看到的那样，您可以在表达式 plt.cm.jet(feature) 中使用 any coloring scheme 而不是 plt.cm.jet 颜色。
import requests, PIL.Image, io, numpy as np, matplotlib.pyplot as plt
# load some image
img = np.array(PIL.Image.open(io.BytesIO(requests.get('https://i.stack.imgur.com/vPlCG.jpg').content)))
# load or compute some features
h, w, _ = img.shape
mg = np.mgrid[:h, :w]
feature = mg[0].astype(np.float64) * mg[1].astype(np.float64)
feature = (255.5 * feature / np.amax(feature)).astype(np.uint8)
# compute heated image
alpha = 0.5; img_feature = ((plt.cm.jet(feature)[:, :, :3] * 255) * alpha + img * (1 - alpha)).astype(np.uint8)
# show images
fig, axs = plt.subplots(2, 2)
axs[0, 0].imshow(img)
axs[0, 1].imshow(feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
axs[1, 0].imshow(img_feature, alpha = 1., interpolation = 'gaussian', cmap = plt.cm.jet)
axs[1, 1].remove()
plt.show()
输出：
PS。我刚刚注意到 Matplotlib 在绘制热图时进行了归一化 (feature - min) / (max - min) 因此我决定在叠加公式中做同样的事情，最终公式变成这样：
alpha = 0.5; img_feature = ((
plt.cm.jet(
(feature - np.amin(feature)).astype(np.float32)
/ (np.amax(feature) - np.amin(feature)).astype(np.float32)
)[:, :, :3] * 255
) * alpha + img * (1 - alpha)).astype(np.uint8)
使用上述公式的示例代码is here (or here, and for your images). Resulting image of using formula above is here (and for your images)。最终叠加图像：