生成图像文件而不将整个图像保存在内存中

Question

我编写了一个程序，使用三个方程生成 RGB 颜色值并将它们写入数据文件：

from struct import pack

#User-set variables
size = [16384, 16384]
center = [0, 0]
r_equation = "x ^ y"
g_equation = "x - y"
b_equation = "x + y"
data_path = "data.dat"

#Calculate x and y offset
offset = [center[0] - (size[0] // 2), center[1] - (size[1] // 2)]

#Compile code for calculating RGB values
r_code = compile(r_equation, "r_equation", "eval")
g_code = compile(g_equation, "g_equation", "eval")
b_code = compile(b_equation, "b_equation", "eval")

data_file = open(data_path, "wb")

#Pack image size as first 4 bytes
data_file.write(pack("HH", *size))

#Iterate through the graph
for y in range(offset[1], size[1] + offset[1]):
    for x in range(offset[0], size[0] + offset[0]):
        #Calculate the RGB values
        rgb = [int(eval(r_code)) % 256,
               int(eval(g_code)) % 256,
               int(eval(b_code)) % 256]
        #Pack the RGB values as 3 bytes
        data_file.write(pack("BBB", *rgb))

data_file.close()

我想从数据文件创建图像，但我没有足够的 RAM 来加载整个图像。我写了这个程序来读取文件的片段：

from struct import unpack

#User-set variables
data_path = "data.dat"
image_path = "image.xxx"

data_file = open(data_path, "rb")

#Unpack first 4 bytes to get image size
size = unpack("HH", data_file.read(4))

#create_image(size, image_path)

for data in data_file.read(1048576 * 3): #Read 1M pixels at a time
    #write_to_image(data)

如果我尝试使用 PIL 写入整个图像文件，它很快就会耗尽内存。有没有办法将图像文件分段写入，以便一次在内存中写入一个？

Answer 1

您可以尝试使用这个 python 库： https://github.com/libvips/pyvips

Programs that use pyvips don't manipulate images directly, instead they create pipelines of image processing operations building on a source image. When the end of the pipe is connected to a destination, the whole pipeline executes at once, streaming the image in parallel from source to destination a section at a time.

Vips 使处理大图像成为可能，而且速度非常快。

Because pyvips is parallel, it's quick, and because it doesn't need to keep entire images in memory, it's light.

Answer 2

这是你的程序（我认为！）用 pyvips:

完成

#!/usr/bin/python3

import sys
import pyvips

# make a huge two-band image where each pixel has the value of its (x, y)
# coordinate
xy = pyvips.Image.xyz(16384, 16384)

# subtract the half way point, so each pixel is now -8192 to +8191
xy -= [8192, 8192]

# compute three one-band images from (x, y) ... image[n] gets the nth band
r = xy[0] ^ xy[1]
g = xy[0] + xy[1]
b = xy[0] - xy[1]

# join as an RGB image, modulo 256
rgb = r.bandjoin([g, b]).copy(interpretation="srgb") & 0xff

rgb.write_to_file(sys.argv[1])

在这台普通的笔记本电脑上，我看到：

$ /usr/bin/time -f %M:%e ./pattern.py x.jpg
136712:5.81s

6s 和 137mb 内存。

生成图像文件而不将整个图像保存在内存中

Generate image file without keeping the whole image in memory

python

image

image-processing

large-data

large-files