仅从 TFrecords 中提取单个图像

Question

作为标题状态，从我的 tfrecord 文件中只加载了 1 个图像+标签。每个 tfrecord 中有可变数量的 images/labels，但始终至少有 8 对。我使用的是 TF 版本：2.4.1

可能与我收到此警告有关：

WARNING:tensorflow:AutoGraph could not transform <function parse_tfr_element at 0x7fbb7db99160> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: module 'gast' has no attribute 'Index' To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert

下面是我用来加载测试数据的函数。感谢任何帮助。

def parse_tfr_element(element):

    data = {
      'height': tf.io.FixedLenFeature([], tf.int64),
      'width':tf.io.FixedLenFeature([], tf.int64),
      'depth':tf.io.FixedLenFeature([], tf.int64),
      'raw_label':tf.io.FixedLenFeature([], tf.string),#tf.string = bytestring (not text string)
      'raw_image' : tf.io.FixedLenFeature([], tf.string),#tf.string = bytestring (not text string)
    }


    content = tf.io.parse_single_example(element, data)

    height = content['height']
    width = content['width']
    depth = content['depth']
    raw_label = content['raw_label']
    raw_image = content['raw_image']


    #get our 'feature'-- our image -- and reshape it appropriately
    feature = tf.io.parse_tensor(raw_image, out_type=tf.float16)
    feature = tf.reshape(feature, shape=[height,width,depth])
    label = tf.io.parse_tensor(raw_label, out_type=tf.int8)
    label = tf.reshape(label, shape=[height,width])
    return (feature, label)

def get_batched_dataset(filenames):
    option_no_order = tf.data.Options()
    option_no_order.experimental_deterministic = False

    dataset = tf.data.TFRecordDataset(filenames)
    dataset = dataset.with_options(option_no_order)
    dataset = dataset.map(parse_tfr_element, num_parallel_calls=AUTO)

    dataset = dataset.shuffle(2048)
    dataset = dataset.batch(BATCH_SIZE, drop_remainder=True) 


    return dataset

Answer 1

原来这个问题很愚蠢，与我在问题中发布的功能无关。问题是我将 steps_per_epoch 的以下值输入到模型中。

steps_per_epoch = len(training_filenames)  // BATCH_SIZE

由于文件包含多个案例，len(training_filenames) 需要乘以每个文件中的案例数。

steps_per_epoch = len(training_filenames) * images_in_file  // BATCH_SIZE

仅从 TFrecords 中提取单个图像

Only a single image is extracted from TFrecords

python

image-processing

tensorflow2.0

tfrecord