tensorflow中自己图片的有效读取

Question

我浏览了所有 tensorflow 教程，其中所有数据集由于体积小而加载到 RAM 中。但是，我自己的数据（约 30 Gb 的图像）无法加载到内存中，因此我正在寻找读取图像以进行进一步处理的有效方法。任何人都可以提供我如何做到这一点的例子吗？

P.S。我有两个文件 train_images 和 validation_images，其中包含：

<path/to/img> <label>

Answer 1

udacity 教程在 https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/4_convolutions.ipynb 中解释了随机方法，您可以使用相同的方法进行一次更改，而不是将所有图像保存在单个 pickle 文件中，而是将它们保存在 batch_size 的块中正在使用。这样一次，您只能加载一批中使用的数据。

Answer 2

推荐的方法是将其放入分片的 protobuf 文件中，其中编码的 jpeg 和标签是 tf.Example 的特征。 build_image_data.py in the tensorflow/models repository shows how to create such a database of image/label pairs from a directory structure, you'll need to adapt it a bit to your case (it's straightforward). Then for training time you can look at image_processing.py 其中展示了如何从 tf.Example 原型到 image/label 张量（从示例记录中提取解码的 jpg 和标签，解码 jpg，调整大小，根据需要应用扩充，然后入队） .

Answer 3

这就是您要查找的内容：

具体的代码片段是这样的：

def read_labeled_image_list(image_list_file):
    """Reads a .txt file containing pathes and labeles
    Args:
       image_list_file: a .txt file with one /path/to/image per line
       label: optionally, if set label will be pasted after each line
    Returns:
       List with all filenames in file image_list_file
    """
    f = open(image_list_file, 'r')
    filenames = []
    labels = []
    for line in f:
        filename, label = line[:-1].split(' ')
        filenames.append(filename)
        labels.append(int(label))
    return filenames, labels

def read_images_from_disk(input_queue):
    """Consumes a single filename and label as a ' '-delimited string.
    Args:
      filename_and_label_tensor: A scalar string tensor.
    Returns:
      Two tensors: the decoded image, and the string label.
    """
    label = input_queue[1]
    file_contents = tf.read_file(input_queue[0])
    example = tf.image.decode_png(file_contents, channels=3)
    return example, label

# Reads pfathes of images together with their labels
image_list, label_list = read_labeled_image_list(filename)

images = ops.convert_to_tensor(image_list, dtype=dtypes.string)
labels = ops.convert_to_tensor(label_list, dtype=dtypes.int32)

# Makes an input queue
input_queue = tf.train.slice_input_producer([images, labels],
                                            num_epochs=num_epochs,
                                            shuffle=True)

image, label = read_images_from_disk(input_queue, num_labels=num_labels)

# Optional Preprocessing or Data Augmentation
# tf.image implements most of the standard image augmentation
image = preprocess_image(image)
label = preprocess_label(label)

# Optional Image and Label Batching
image_batch, label_batch = tf.train.batch([image, label],
                                          batch_size=batch_size)

tensorflow中自己图片的有效读取

Effective reading of own images in tensorflow

deep-learning

tensorflow