Matlab - Blockproc/tifflib 写入大 tiff 文件时失败
Matlab - Blockproc/tifflib failing while writing big tiff File
在 Matlab 中,我使用 blockproc 来处理仅包含一个 32 位浮点数据波段的大(地理)tiff 文件(大约 10Gb)。
我现在的问题是 blockproc 或更具体的 tifflib 突然无法写入磁贴。这独立于输入文件、Harddrive/file 位置或其他参数,例如处理、Matlab 版本,甚至 OS.
而且它通常总是围绕相同的图块编号(#3372for linux,#3176 for Windows 10)。
但是当使用大文件的较小版本时(大约 4Gb 而不是 10Gb),blockproc 会运行。
这是我收到的错误消息(Debian 64 位,Matlab 2014b):
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3684.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3685.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3686.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3687.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Error using tifflib
Unable to write tile #3372.
Error in Tiff/writeEncodedTile (line 1602)
tifflib('writeEncodedTile',obj.FileID,tileNumber-1,varargin{:});
Error in images.internal.TiffAdapter/writeBlock (line 420)
obj.TiffObj.writeEncodedTile(blockId, block);
Error in images.internal.TiffAdapter/writeRegion (line 377)
obj.writeBlock(blockId, data(ry - start(1) + 1, ...
Error in blockproc>putBlock (line 879)
dest.writeRegion(start_loc,...
Error in blockproc/serialLoop (line 459)
putBlock(options.Destination,row,col,output_block,output_size);
Error in blockproc (line 340)
serialLoop();
Error in removeEGM (line 13)
blockproc(in_dem, [3000, 3000], my_fun, 'Destination', out_dem);
感谢@haraldKs 的评论,我找到了解决方案:
Matlab 可以毫无问题地读取 BigTiffs (imread()),并写入它们,但是 blockproc 只能读取 BigTiffs,而不能写入它们。
作为一种解决方法,我基于这个编写了自己的 imageAdapter:
https://blogs.mathworks.com/steve/2013/08/07/tiff-bigtiff-and-blockproc/
基本上我只更改了有关 tiff 标签的行以适合我拥有的数据(32 位浮点数,只有一个通道,无压缩等),然后简单地从原始文件中复制地理配准,然后使用 gdal。
obj.TiffObject.setTag('ImageLength', obj.ImageSize(1));
obj.TiffObject.setTag('ImageWidth', obj.ImageSize(2));
obj.TiffObject.setTag('TileLength', obj.TileLength);
obj.TiffObject.setTag('TileWidth', obj.TileWidth);
obj.TiffObject.setTag('Photometric', Tiff.Photometric.MinIsBlack);
obj.TiffObject.setTag('BitsPerSample', 32);
obj.TiffObject.setTag('SampleFormat', Tiff.SampleFormat.IEEEFP);
obj.TiffObject.setTag('SamplesPerPixel', 1);
obj.TiffObject.setTag('ResolutionUnit', Tiff.ResolutionUnit.Inch);
obj.TiffObject.setTag('Orientation', 1);
obj.TiffObject.setTag('Compression', Tiff.ResolutionUnit.None);
obj.TiffObject.setTag('PlanarConfiguration', Tiff.PlanarConfiguration.Chunky);
在 Matlab 中,我使用 blockproc 来处理仅包含一个 32 位浮点数据波段的大(地理)tiff 文件(大约 10Gb)。
我现在的问题是 blockproc 或更具体的 tifflib 突然无法写入磁贴。这独立于输入文件、Harddrive/file 位置或其他参数,例如处理、Matlab 版本,甚至 OS.
而且它通常总是围绕相同的图块编号(#3372for linux,#3176 for Windows 10)。
但是当使用大文件的较小版本时(大约 4Gb 而不是 10Gb),blockproc 会运行。
这是我收到的错误消息(Debian 64 位,Matlab 2014b):
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3684.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3685.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3686.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Warning: The following error was caught while executing 'images.internal.TileHandle'
class destructor:
Unable to write tile #3687.
> In TileCache>TileCache.setTile at 71
In TiffAdapter>TiffAdapter.getTileHandle at 560
In TiffAdapter>TiffAdapter.writeRegion at 354
In blockproc>putBlock at 879
In blockproc>serialLoop at 459
In blockproc at 340
In removeEGM at 13
Error using tifflib
Unable to write tile #3372.
Error in Tiff/writeEncodedTile (line 1602)
tifflib('writeEncodedTile',obj.FileID,tileNumber-1,varargin{:});
Error in images.internal.TiffAdapter/writeBlock (line 420)
obj.TiffObj.writeEncodedTile(blockId, block);
Error in images.internal.TiffAdapter/writeRegion (line 377)
obj.writeBlock(blockId, data(ry - start(1) + 1, ...
Error in blockproc>putBlock (line 879)
dest.writeRegion(start_loc,...
Error in blockproc/serialLoop (line 459)
putBlock(options.Destination,row,col,output_block,output_size);
Error in blockproc (line 340)
serialLoop();
Error in removeEGM (line 13)
blockproc(in_dem, [3000, 3000], my_fun, 'Destination', out_dem);
感谢@haraldKs 的评论,我找到了解决方案:
Matlab 可以毫无问题地读取 BigTiffs (imread()),并写入它们,但是 blockproc 只能读取 BigTiffs,而不能写入它们。
作为一种解决方法,我基于这个编写了自己的 imageAdapter: https://blogs.mathworks.com/steve/2013/08/07/tiff-bigtiff-and-blockproc/
基本上我只更改了有关 tiff 标签的行以适合我拥有的数据(32 位浮点数,只有一个通道,无压缩等),然后简单地从原始文件中复制地理配准,然后使用 gdal。
obj.TiffObject.setTag('ImageLength', obj.ImageSize(1));
obj.TiffObject.setTag('ImageWidth', obj.ImageSize(2));
obj.TiffObject.setTag('TileLength', obj.TileLength);
obj.TiffObject.setTag('TileWidth', obj.TileWidth);
obj.TiffObject.setTag('Photometric', Tiff.Photometric.MinIsBlack);
obj.TiffObject.setTag('BitsPerSample', 32);
obj.TiffObject.setTag('SampleFormat', Tiff.SampleFormat.IEEEFP);
obj.TiffObject.setTag('SamplesPerPixel', 1);
obj.TiffObject.setTag('ResolutionUnit', Tiff.ResolutionUnit.Inch);
obj.TiffObject.setTag('Orientation', 1);
obj.TiffObject.setTag('Compression', Tiff.ResolutionUnit.None);
obj.TiffObject.setTag('PlanarConfiguration', Tiff.PlanarConfiguration.Chunky);