ios / CoreML - keras模型转CoreML时输入类型为MultiArray

Question

我正在尝试训练 keras 模型并使用 keras 1.2.2 和 TensorFlow 后端将其转换为 coreML 模型。这是一个分类任务。 CoreML 的输入显示为 MultiArray。我需要它是 Image <BGR, 32, 32> 或类似 CVPixelBuffer 的东西。我尝试添加 image_input_names='data'，如前所述 here。另外，我的 input shape 是 (height, width, depth)，我认为这是必需的。

请帮助解决此问题。我用 cifar10 dataset and the following code (Reference):

from keras.datasets import cifar10
from keras.models import Model
from keras.layers import Input, Convolution2D, MaxPooling2D, Dense, Dropout, Flatten
from keras.utils import np_utils
import numpy as np
import coremltools

np.random.seed(1234)

batch_size = 32
num_epochs = 1

kernel_size = 3 
pool_size = 2 
conv_depth_1 = 32 
conv_depth_2 = 64 
drop_prob_1 = 0.25
drop_prob_2 = 0.5
hidden_size = 512 

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
num_train, height, width, depth = X_train.shape
num_test = X_test.shape[0]
num_classes = 10

X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= np.max(X_train)
X_test /= np.max(X_test)

y_train = np_utils.to_categorical(y_train, num_classes)
y_test = np_utils.to_categorical(y_test, num_classes)

data = Input(shape=(height, width, depth))
conv_1 = Convolution2D(conv_depth_1, (kernel_size, kernel_size), padding='same', activation='relu')(data)
conv_2 = Convolution2D(conv_depth_1, (kernel_size, kernel_size), padding='same', activation='relu')(conv_1)
pool_1 = MaxPooling2D(pool_size=(pool_size, pool_size))(conv_2)
drop_1 = Dropout(drop_prob_1)(pool_1)

conv_3 = Convolution2D(conv_depth_2, (kernel_size, kernel_size), padding='same', activation='relu')(drop_1)
conv_4 = Convolution2D(conv_depth_2, (kernel_size, kernel_size), padding='same', activation='relu')(conv_3)
pool_2 = MaxPooling2D(pool_size=(pool_size, pool_size))(conv_4)
drop_2 = Dropout(drop_prob_1)(pool_2)

flat = Flatten()(drop_2)
hidden = Dense(hidden_size, activation='relu')(flat)
drop_3 = Dropout(drop_prob_2)(hidden)
out = Dense(num_classes, activation='softmax')(drop_3)

model = Model(inputs=data, outputs=out) 

model.compile(loss='categorical_crossentropy', 
              optimizer='adam', 
              metrics=['accuracy']) 

model.fit(X_train, y_train,                
          batch_size=batch_size, epochs=num_epochs,
          verbose=1, validation_split=0.1) 
loss, accuracy = model.evaluate(X_test, y_test, verbose=1)
print ("\nTest Loss: {loss} and Test Accuracy: {acc}\n".format(loss = loss, acc = accuracy))
coreml_model = coremltools.converters.keras.convert(model, input_names='data', image_input_names='data')
coreml_model.save('my_model.mlmodel')

Answer 1

我刚刚用 Keras 2 检查了这个，你的模型的输入是 Image<RGB,32,32>，而不是 MultiArray。也许这取决于 Keras 版本。

如果您需要它是 BGR，请将 is_bgr=True 添加到 coremltools.converters.keras.convert() 调用中。

Here is the documentation 对于这个转换器。

Answer 2

问题出在我的 tf 版本和 protobuf 版本上。我能够通过安装 coremltools `documentation.

中提到的版本来解决问题

ios / CoreML - keras模型转CoreML时输入类型为MultiArray

ios / CoreML - The input type is MultiArray when keras model is converted to CoreML

ios

conv-neural-network

cvpixelbuffer

keras

coreml