重新训练 MobileNet SSD V1 COCO 后,Tensorflow 的 pb 和 pbtxt 文件不适用于 OpenCV

Tensorflow' pb and pbtxt files don't work with OpenCV after retraining MobileNet SSD V1 COCO

我已经按照 this 教程使用 Tensorflow GPU 重新训练 MobileNet SSD V1,并在使用 GPU 训练后损失了 0.5(关于配置的更多信息) 和得到 model.ckpt.

这是我用于训练的命令:

python ../models/research/object_detection/legacy/train.py --logtostderr --train_dir=./data/ --pipeline_config_path=./ssd_mobilenet_v1_pets.config

这是冻结命令(生成pb文件):

python ../models/research/object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path ./ssd_mobilenet_v1_pets.config --trained_checkpoint_prefix ./data/model.ckpt-1407 --output_directory ./data/

这是我在使用 frozen pbpbtxt:

时得到的错误
Traceback (most recent call last):
File "Object_detection_image.py", line 29, in <module>
    cvOut = cvNet.forward()
cv2.error: OpenCV(3.4.3) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:565: error: (-215:Assertion failed) inputs.size() == requiredOutputs in function 'cv::dnn::experimental_dnn_34_v7::DataLayer::getMemoryShapes'

这是我使用的 Object_detection_image.py 文件:

import cv2 as cv
import os 
import time 
import logging

logger = logging.getLogger()
fh = logging.FileHandler('xyz.log')
fh.setLevel(logging.DEBUG)    
logger.addHandler(fh)

cvNet = cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'object_detection.pbtxt')
dir_x  = "C:\Users\Omen\Desktop\LP_dataset\anno"
for filename in os.listdir(dir_x):
    print(filename)
    if not (filename.endswith(".png") or filename.endswith(".jpg")):
        continue
    print('daz')
    img = cv.imread(os.path.join(dir_x,filename))
    img = cv.resize(img, (300,300))
    #cv.imshow('i',img)
    #cv.waitKey(0)
    img = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
    img = cv.cvtColor(img,cv.COLOR_GRAY2RGB)
    rows = img.shape[0]
    cols = img.shape[1]
    #cvNet.setInput(cv.dnn.blobFromImage(img, size=(cols,rows), swapRB=True, crop=False))
    cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), crop=False))
    t0  = time.time()
    cvOut = cvNet.forward()
    print(time.time() - t0)
    for detection in cvOut[0,0,:,:]:
        score = float(detection[2])
        #print(score)
        if score > 0.80:
            left = detection[3] * cols
            top = detection[4] * rows
            right = detection[5] * cols
            bottom = detection[6] * rows
            cv.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (23, 230, 210), thickness=2)

    cv.imshow('img', img)
    cv.waitKey(0)

这是 pbtxt 文件 (我也尝试了导出的 pbtxt 和从 pb 生成的 pbtxt 但没有用):

item {
  id: 1
  name: 'licenseplate'
}

配置:

您使用的模型的顶层目录是什么:object_detetion

是否编写了自定义代码:否

OS 平台和发行版:win10

TensorFlow 安装自:二进制

TensorFlow GPU 版本:1.13.0

CUDA/cuDNN版本:10

显卡型号:1050 GTX

我可以提供你要的任何文件,请帮助我。 在 tensorflow 的 github 他们告诉我在 Whosebug 中问...

更新:

感谢解答,问题解决了,这里是cvOut的内容:

  [[[[-0.00476191 -0.00361736  0.          0.25361738 -0.07576995
     0.03405379  0.40910327]
   [ 0.21594621  0.04544836  0.          0.28788495  0.30689242
    -0.13025634  0.05074273]
   [ 0.46358964  0.19925728  0.         -0.09778295  0.26563603
     0.34778297 -0.02014329]
   [-0.01515752  0.3534766   0.          0.32857144 -0.00361736
     0.67142856  0.25361738]
   [ 0.25756338  0.03405379  0.          0.21594621  0.3787817
    -0.05689242  0.6212183 ]
   [ 0.30689242  0.203077    0.          0.796923    0.19925728
     0.40103063 -0.09778295]
   [ 0.5989694   0.34778297  0.         -0.01515752  0.68680996
     0.26515752  0.66190475]
   [-0.00361736  1.0047619   0.          0.59089667  0.03405379
     1.0757699   0.21594621]
   [ 0.712115   -0.05689242  0.          0.30689242  0.53641033
     0.05074273  1.1302563 ]
   [ 0.19925728  0.7343639   0.          0.93230265  0.34778297
     0.64652336 -0.01515752]
   [ 1.0201433   0.26515752  0.          0.24638264  0.33809522
     0.50361735 -0.07576995]
   [ 0.2840538   0.40910327  0.          0.04544836  0.19310758
     0.28788495  0.5568924 ]
   [-0.13025634  0.30074272  0.          0.44925728  0.06769729
     0.15221705  0.26563603]
   [ 0.59778297 -0.02014329  0.          0.3534766   0.5151575
     0.32857144  0.24638264]
   [ 0.67142856  0.50361735  0.          0.2840538   0.7424366
     0.4659462   0.3787817 ]
   [ 0.19310758  0.6212183   0.          0.203077    0.30074272
     0.796923    0.44925728]
   [ 0.40103063  0.15221705  0.          0.59778297  0.31319004
     0.23484248  0.68680996]
   [ 0.5151575   0.66190475  0.          1.0047619   0.50361735
     0.59089667  0.2840538 ]
   [ 1.0757699   0.4659462   0.          0.19310758  0.95455164
     0.5568924   0.53641033]
   [ 0.30074272  1.1302563   0.          0.7343639   0.15221705
     0.93230265  0.59778297]
   [ 0.64652336  0.23484248  0.          0.5151575  -0.00476191
     0.49638262  0.33809522]
   [ 0.75361735 -0.07576995  0.          0.40910327  0.7159462
     0.04544836  0.44310758]
   [ 0.28788495  0.8068924   0.          0.55074275  0.46358964
     0.69925725  0.06769729]
   [ 0.40221703  0.26563603  0.         -0.02014329  0.48484248
     0.3534766   0.7651575 ]
   [ 0.32857144  0.49638262  0.          0.75361735  0.25756338
     0.5340538   0.7424366 ]
   [ 0.7159462   0.3787817   0.          0.6212183   0.8068924
     0.203077    0.55074275]
   [ 0.796923    0.69925725  0.          0.40221703  0.5989694
     0.84778297  0.31319004]
   [ 0.48484248  0.68680996  0.          0.66190475  0.49638262
     1.0047619   0.75361735]
   [ 0.59089667  0.5340538   0.          0.7159462   0.712115
     0.44310758  0.95455164]
   [ 0.8068924   0.53641033  0.          1.1302563   0.69925725
     0.7343639   0.40221703]
   [ 0.93230265  0.84778297  0.          0.48484248  1.0201433
     0.7651575  -0.00476191]
   [ 0.74638265  0.33809522  0.         -0.07576995  0.7840538
     0.40910327  0.9659462 ]
   [ 0.04544836  0.6931076   0.          1.0568924  -0.13025634
     0.80074275  0.46358964]
   [ 0.94925725  0.06769729  0.          0.26563603  1.0977829
    -0.02014329  0.7348425 ]
   [ 0.3534766   1.0151576   0.          0.74638265  0.67142856
     1.0036174   0.25756338]
   [ 0.7840538   0.7424366   0.          0.3787817   0.6931076
     0.6212183   1.0568924 ]
   [ 0.203077    0.80074275  0.          0.94925725  0.40103063
     0.65221703  0.5989694 ]
   [ 1.0977829   0.31319004  0.          0.68680996  1.0151576
     0.66190475  0.74638265]
   [ 1.0047619   1.0036174   0.          0.7840538   1.0757699
     0.9659462   0.712115  ]
   [ 0.6931076   0.95455164  0.          0.53641033  0.80074275
     1.1302563   0.94925725]
   [ 0.7343639   0.65221703  0.          1.0977829   0.64652336
     0.7348425   1.0201433 ]
   [ 1.0151576   0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.1         0.2
     0.2         0.1       ]
   [ 0.1         0.2         0.          0.1         0.1
     0.2         0.2       ]
   [ 0.1         0.1         0.          0.2         0.1
     0.1         0.2       ]
   [ 0.2         0.1         0.          0.2         0.2
     0.1         0.1       ]
   [ 0.2         0.2         0.          0.8479438   0.67317617
     0.5581815   0.1778345 ]
   [-0.9215721   1.5896183   0.          0.6099795   0.5955366
    -0.46569395 -0.8461083 ]
   [ 1.6129647   1.4244858   0.          0.5209342   0.17585325
    -0.8687666   1.7872683 ]
   [ 1.3389692   0.8533131   0.         -0.00590521 -0.7195761
     1.6236191   1.1828533 ]
   [ 1.1838211   0.6728102   0.         -0.785988    1.2751837
     1.1616383   0.933811  ]
   [ 0.4684658   0.2719049   0.          1.2093123   0.66612804
     0.66964823  0.55971766]
   [ 0.17104894 -1.0688283   0.          0.6494252   0.6844874
     0.66586125  0.01329695]
   [-1.2607187  -0.22749203  0.         -0.8741171  -0.9443728
    -0.9659323  -0.03422031]
   [-0.0364061   0.54829746  0.          0.6263525   0.66758543
     0.04167109 -0.11780822]
   [ 0.48400337  0.4685324   0.         -0.04594427  0.02469592
    -0.3487326   0.08831279]
   [ 0.4161314   0.23332608  0.         -0.13553022 -0.31008872
     0.04969648  0.5674252 ]
   [ 0.36492363 -0.07475745  0.         -0.03859219  0.2016789
    -0.39845943 -0.07058203]
   [-0.08173721  0.1720942   0.          0.02323131  0.07122216
     0.07469177  0.12792486]
   [-0.24689877  0.196296    0.          0.5564647   0.535513
     0.22528338 -0.37152448]
   [-1.7235181  -1.8204601   0.         -1.5040898  -1.8099409
    -1.8550183  -1.1855855 ]
   [-1.6341007  -1.3448519   0.         -1.6656716  -1.6564709
    -1.2735447  -1.3357594 ]
   [-1.2829769  -1.2869868   0.         -1.6657944  -1.4066424
    -1.4230443  -1.4196167 ]
   [-1.3691044  -1.656098    0.         -1.4339573  -1.5685135
    -1.633306   -1.4437945 ]]]]

错误是由错误的输入 .pbtxt 文件传递​​给函数 readNetFromTensorflow 引起的,因为 .pbtxt 必须由 tf_text_graph_ssd.py as describe here 生成:

Run this script to get a text graph of SSD model from TensorFlow Object Detection API. Then pass it with .pb file to cv::dnn::readNetFromTensorflow function.

其他型号如faster r-cnn and mask r-cnn也有对应的脚本

PS: 刚刚发现有一个很好的官方教程here.