如何使用 TF1 正确恢复 TensorFlow 对象检测 API 中的检查点？

Question

我正在 COCO 2017 数据集上微调 SSD-MobileNetV3 Large 和 SSD-MobileDet-CPU，但只有书 class。我为此创建了一个新数据集并检查了该数据集，它很好。我还根据需要修改了配置文件。当我开始训练时，它只是忽略配置文件中提供的 'fine_tune_checkpoint' 并从头开始。但是，如果我执行相同的过程但使用 'model_dir' 目录中的检查点，它会尝试恢复它，但由于我有不同数量的 classes，它会给出错误。如何使训练过程正确恢复检查点？我还尝试使用所有 90 个 classes 的普通 COCO 数据集，当我开始训练时，'fine_tune_checkpoint' 被忽略，但如果我将检查点放在 'model_dir' 中，它会正确恢复.

我的配置文件如下。

# SSDLite with MobileDet-CPU feature extractor.
# Reference: Xiong & Liu et al., https://arxiv.org/abs/2004.14525
# Trained on COCO, initialized from scratch.
#
# 0.45B MulAdds, 4.21M Parameters. Latency is 113ms on Pixel 1 CPU.
# Achieves 24.0 mAP on COCO14 minival dataset.
# Achieves 23.5 mAP on COCO17 val dataset.
#
# This config is TPU compatible.

model {
  ssd {
    inplace_batchnorm_update: true
    freeze_batchnorm: false
    num_classes: 1
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    matcher {
      argmax_matcher {
        matched_threshold: 0.5
        unmatched_threshold: 0.5
        ignore_thresholds: false
        negatives_lower_than_unmatched: true
        force_match_for_each_row: true
        use_matmul_gather: true
      }
    }
    similarity_calculator {
      iou_similarity {
      }
    }
    encode_background_as_zeros: true
    anchor_generator {
      ssd_anchor_generator {
        num_layers: 6
        min_scale: 0.2
        max_scale: 0.95
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        aspect_ratios: 3.0
        aspect_ratios: 0.3333
      }
    }
    image_resizer {
      fixed_shape_resizer {
        height: 320
        width: 320
      }
    }
    box_predictor {
      convolutional_box_predictor {
        min_depth: 0
        max_depth: 0
        num_layers_before_predictor: 0
        use_dropout: false
        dropout_keep_probability: 0.8
        kernel_size: 3
        use_depthwise: true
        box_code_size: 4
        apply_sigmoid_to_scores: false
        class_prediction_bias_init: -4.6
        conv_hyperparams {
          activation: RELU_6,
          regularizer {
            l2_regularizer {
              weight: 0.00004
            }
          }
          initializer {
            random_normal_initializer {
              stddev: 0.03
              mean: 0.0
            }
          }
          batch_norm {
            train: true,
            scale: true,
            center: true,
            decay: 0.97,
            epsilon: 0.001,
          }
        }
      }
    }
    feature_extractor {
      type: 'ssd_mobiledet_cpu'
      min_depth: 16
      depth_multiplier: 1.0
      use_depthwise: true
      conv_hyperparams {
        activation: RELU_6,
        regularizer {
          l2_regularizer {
            weight: 0.00004
          }
        }
        initializer {
          truncated_normal_initializer {
            stddev: 0.03
            mean: 0.0
          }
        }
        batch_norm {
          train: true,
          scale: true,
          center: true,
          decay: 0.97,
          epsilon: 0.001,
        }
      }
      override_base_feature_extractor_hyperparams: false
    }
    loss {
      classification_loss {
        weighted_sigmoid_focal {
          alpha: 0.75,
          gamma: 2.0
        }
      }
      localization_loss {
        weighted_smooth_l1 {
          delta: 1.0
        }
      }
      classification_weight: 1.0
      localization_weight: 1.0
    }
    normalize_loss_by_num_matches: true
    normalize_loc_loss_by_codesize: true
    post_processing {
      batch_non_max_suppression {
        score_threshold: 1e-8
        iou_threshold: 0.6
        max_detections_per_class: 100
        max_total_detections: 100
        use_static_shapes: true
      }
      score_converter: SIGMOID
    }
  }
}

train_config: {
  batch_size: 64
  sync_replicas: true
  startup_delay_steps: 0
  replicas_to_aggregate: 1
  num_steps: 800000
  data_augmentation_options {
    random_horizontal_flip {
    }
  }
  data_augmentation_options {
    ssd_random_crop {
    }
  }
  optimizer {
    momentum_optimizer: {
      learning_rate: {
        cosine_decay_learning_rate {
          learning_rate_base: 0.8
          total_steps: 800000
          warmup_learning_rate: 0.13333
          warmup_steps: 100
        }
      }
      momentum_optimizer_value: 0.9
    }
    use_moving_average: false
  }
  fine_tune_checkpoint: "./checkpoints/model.ckpt-400000"
  max_number_of_boxes: 100
  unpad_groundtruth_tensors: false
  fine_tune_checkpoint_type: "detection"
  fine_tune_checkpoint_version: V1
  load_all_detection_checkpoint_vars: true
}

train_input_reader: {
  tf_record_input_reader {
    input_path: "./tf_record_coco_books/coco_train.record"
  }
  label_map_path: "./tf_record_coco_books/label_map.pbtxt"
}

eval_config: {
  metrics_set: "coco_detection_metrics"
  use_moving_averages: false
}

eval_input_reader: {
  tf_record_input_reader {
    input_path: "./tf_record_coco_books/coco_val.record"
  }
  label_map_path: "./tf_record_coco_books/label_map.pbtxt"
  shuffle: false
  num_readers: 1
}

Answer 1

您必须指定一个 model_dir，它不同于您加载之前训练的检查点的目录。

在训练的最开始，Tensorflow 对象检测 API 训练脚本（当前的 model_main 或 legacy/train) 将在您的 model_dir 中创建一个与您的新配置相对应的新检查点，然后在该检查点上进行训练。如果你的目录已经包含 pre-trained 个检查点，它确实会引发一个与类数相对应的问题。

如果这不起作用，您还可以更改配置文件中的字段：

fine_tune_checkpoint_type = "detection"

至：

fine_tune_checkpoint_type = "fine_tune"

关于这是对象检测的当前问题API：https://github.com/tensorflow/models/issues/8892#issuecomment-680207038

Answer 2

问题从 model_lib.py

中的第 446 行开始

load_pretrained = hparams.load_pretrained if hparams else False;

因为之前的提交之一将 hparams 更改为 None，所以 load_pretrained 始终为 False。将其设置为 True，然后重新安装 object_detection 库可以解决问题。

我在相关的 github 问题中提到了这一点： https://github.com/tensorflow/models/issues/9284

如何使用 TF1 正确恢复 TensorFlow 对象检测 API 中的检查点？

How to properly restore a checkpoint in TensorFlow object detection API with TF1?

object-detection

tensorflow

object-detection-api