mmap：无法在不知道其大小的情况下附加到现有区域 (Windows)

Question

我正在尝试附加到另一个应用程序创建的现有共享内存区域，而不是用 Python 编写的（这是它的插件模块相互通信的方式）。在 Windows 上，它使用命名的内核对象而不是文件系统中的文件； Python 的 mmap 模块通过 tagname 参数支持这一点。问题是我无法提前知道共享区域的大小是多少——这是另一个应用程序的配置参数，它是根据预期的数据量进行调整的。对于基于文件的共享区域，将大小传递为零会使用文件的现有大小，但这显然不适用于标记区域。这是我正在尝试的简化版本：

import mmap, random

TAGNAME = 'SHM_1001'

# This is a simulation of what the other application does.
# The size isn't actually random, I simply don't know in advance what it is.
m1 = mmap.mmap(-1, random.randint(1e3, 1e6), TAGNAME)

# This is what I'm trying to do in my application, to attach to the same region.
m2 = mmap.mmap(-1, 0, TAGNAME)
# WindowsError: [Error 87] The parameter is incorrect

如果我指定一个小的非零大小，那么我可以成功附加到该区域 - 但当然我只能访问该区域开头的那么多字节。如果我指定的大小大于该区域的实际大小（可能等于它可以拥有的最大大小），我会收到访问错误。 Python 2.7 和 3.4 都存在这个问题。

将大小传递为零的方法绝对适用于系统调用级别 - 这正是该应用程序的每个现有 C/C++ 插件的工作方式 - 所以问题显然在 Python 对 mmap() 调用的包装器。关于如何让它工作的任何想法？

Answer 1

它应该像这样工作：

If length is larger than the current size of the file, the file is extended to contain length bytes. If length is 0, the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an exception (you cannot create an empty mapping on Windows).

但目前看来，这是一个已知错误：http://www.gossamer-threads.com/lists/python/bugs/571941?search_string=1733986;#571941

Answer 2

参数校验在CreateFileMapping is erroring out before the system service NtCreateSection gets called, which if called would find the existing section. Using 0 size when hFile is INVALID_HANDLE_VALUE (-1) is invalid because CreateFileMapping presumes (wrongly in this case) that the section needs to be allocated from the paging file. I assume C plugins are instead calling OpenFileMapping (i.e. NtOpenSection).

您可以使用 ctypes、PyWin32 或 C 扩展模块。调用 OpenFileMappingW 后，调用 MapViewOfFile，然后调用 VirtualQuery 以获取映射区域大小，四舍五入到页面边界。

这是一个使用 ctypes 的例子。

from ctypes import *
from ctypes.wintypes import *

kernel32 = WinDLL('kernel32', use_last_error=True)

FILE_MAP_COPY       = 0x0001
FILE_MAP_WRITE      = 0x0002
FILE_MAP_READ       = 0x0004
FILE_MAP_ALL_ACCESS = 0x001f
FILE_MAP_EXECUTE    = 0x0020

PVOID = LPVOID
SIZE_T = c_size_t

class MEMORY_BASIC_INFORMATION(Structure):
    _fields_ = (('BaseAddress',       PVOID),
                ('AllocationBase',    PVOID),
                ('AllocationProtect', DWORD),
                ('RegionSize',        SIZE_T),
                ('State',             DWORD),
                ('Protect',           DWORD),
                ('Type',              DWORD))

PMEMORY_BASIC_INFORMATION = POINTER(MEMORY_BASIC_INFORMATION)

def errcheck_bool(result, func, args):
    if not result:
        raise WinError(get_last_error())
    return args

kernel32.VirtualQuery.errcheck = errcheck_bool
kernel32.VirtualQuery.restype = SIZE_T
kernel32.VirtualQuery.argtypes = (
    LPCVOID,                   # _In_opt_ lpAddress
    PMEMORY_BASIC_INFORMATION, # _Out_    lpBuffer
    SIZE_T)                    # _In_     dwLength

kernel32.OpenFileMappingW.errcheck = errcheck_bool
kernel32.OpenFileMappingW.restype = HANDLE
kernel32.OpenFileMappingW.argtypes = (
    DWORD,   # _In_ dwDesiredAccess
    BOOL,    # _In_ bInheritHandle
    LPCWSTR) # _In_ lpName

kernel32.MapViewOfFile.errcheck = errcheck_bool
kernel32.MapViewOfFile.restype = LPVOID
kernel32.MapViewOfFile.argtypes = (
    HANDLE, # _In_ hFileMappingObject
    DWORD,  # _In_ dwDesiredAccess
    DWORD,  # _In_ dwFileOffsetHigh
    DWORD,  # _In_ dwFileOffsetLow
    SIZE_T) # _In_ dwNumberOfBytesToMap

kernel32.CloseHandle.errcheck = errcheck_bool
kernel32.CloseHandle.argtypes = (HANDLE,)

if __name__ == '__main__':
    import mmap

    NPAGES = 9
    PAGE_SIZE = 4096

    TAGNAME = 'SHM_1001'
    mm1 = mmap.mmap(-1, PAGE_SIZE * NPAGES, TAGNAME)

    hMap = kernel32.OpenFileMappingW(FILE_MAP_ALL_ACCESS, False, TAGNAME)
    pBuf = kernel32.MapViewOfFile(hMap, FILE_MAP_ALL_ACCESS, 0, 0, 0)
    kernel32.CloseHandle(hMap)

    mbi = MEMORY_BASIC_INFORMATION()
    kernel32.VirtualQuery(pBuf, byref(mbi), PAGE_SIZE)

    assert divmod(mbi.RegionSize, PAGE_SIZE) == (NPAGES, 0)
    mm2 = (c_char * mbi.RegionSize).from_address(pBuf)

    # write using the mmap object
    mm1.seek(100)
    mm1.write(b'Windows')

    # read using the ctypes array
    assert mm2[100:107] == b'Windows'

mmap：无法在不知道其大小的情况下附加到现有区域 (Windows)

mmap: can't attach to existing region without knowing its size (Windows)

python

windows

mmap