为什么 malloc(1) 给出了不止一个页面大小?
Why malloc(1) gives more than one page size?
我已经尝试在我的机器上使用 sbrk(1),然后故意越界写入以测试页面大小,即 4096 字节。但是当我调用 malloc(1) 时,我在访问 135152 字节后得到了 SEGV,这远远超过一个页面大小。我知道 malloc 是库函数,它依赖于实现,但考虑到它最终会调用 sbrk,为什么它会给出一个以上的页面大小。谁能告诉我它的内部工作原理?
我的操作系统是 ubuntu 14.04,我的架构是 x86
更新:现在我想知道是否是因为 malloc returns 空闲列表块的地址足够大以容纳我的数据。但是那个地址可能在堆的中间,这样我就可以一直写,直到达到堆的上限。
malloc
出于性能原因在大块中分配内存。对 malloc
的后续调用可以从大块中为您提供内存,而不必向操作系统请求大量小块。这减少了所需的系统调用次数。
来自this article:
When a process needs memory, some room is created by moving the upper bound of the heap forward, using the brk() or sbrk() system calls. Because a system call is expensive in terms of CPU usage, a better strategy is to call brk() to grab a large chunk of memory and then split it as needed to get smaller chunks. This is exactly what malloc() does. It aggregates a lot of smaller malloc() requests into fewer large brk() calls. Doing so yields a significant performance improvement.
请注意,malloc
的某些现代实现使用 mmap
而不是 brk
/sbrk
来分配内存,但除此之外,上述内容仍然适用。
旧的 malloc()
UNIX 实现使用了 sbrk()
/brk()
系统调用。但是现在,实现使用 mmap()
和 sbrk()
。 glibc 的 malloc()
实现(可能是您在 Ubuntu 14.04 上使用的那个)同时使用 sbrk()
and mmap()
并且在您请求时选择使用哪个来分配通常取决于大小glibc 动态执行的分配请求。
对于小的分配,glibc 使用 sbrk()
,对于较大的分配,它使用 mmap()
。宏M_MMAP_THRESHOLD
is used to decide this. Currently, it's default value is set to 128K。这解释了为什么您的代码设法分配了 135152 个字节,因为它大约是 ~128K。尽管您只请求了 1 个字节,但您的实现分配了 128K 以实现高效的内存分配。所以在你超过这个限制之前不会发生段错误。
您可以通过更改默认参数使用 mallopt()
来玩 M_MAP_THRESHOLD
。
M_MMAP_THRESHOLD
For allocations greater than or equal to the limit
specified (in bytes) by M_MMAP_THRESHOLD that can't be satisfied from
the free list, the memory-allocation functions employ mmap(2) instead
of increasing the program break using sbrk(2).
Allocating memory using mmap(2) has the significant advantage that
the allocated memory blocks can always be independently released back
to the system. (By contrast, the heap can be trimmed only if memory
is freed at the top end.) On the other hand, there are some
disadvantages to the use of mmap(2): deallocated space is not placed
on the free list for reuse by later allocations; memory may be wasted
because mmap(2) allocations must be page-aligned; and the kernel must
perform the expensive task of zeroing out memory allocated via
mmap(2). Balancing these factors leads to a default setting of
128*1024 for the M_MMAP_THRESHOLD parameter.
The lower limit for this parameter is 0. The upper limit is
DEFAULT_MMAP_THRESHOLD_MAX: 512*1024 on 32-bit systems or
4*1024*1024*sizeof(long) on 64-bit systems.
Note: Nowadays, glibc uses a dynamic mmap threshold by default.
The initial value of the threshold is 128*1024, but when blocks
larger than the current threshold and less than or equal to
DEFAULT_MMAP_THRESHOLD_MAX are freed, the threshold is adjusted
upward to the size of the freed block. When dynamic mmap
thresholding is in effect, the threshold for trimming the heap is also
dynamically adjusted to be twice the dynamic mmap threshold. Dynamic
adjustment of the mmap threshold is disabled if any of the
M_TRIM_THRESHOLD, M_TOP_PAD, M_MMAP_THRESHOLD, or M_MMAP_MAX
parameters is set.
例如,如果您这样做:
#include<malloc.h>
mallopt(M_MMAP_THRESHOLD, 0);
在调用 malloc()
之前,您可能会看到不同的限制。其中大部分是实现细节,C 标准说它是 undefined behaviour to write into memory that your process doesn't own. So do it at your risk -- otherwise, demons may fly out of your nose ;-)
我已经尝试在我的机器上使用 sbrk(1),然后故意越界写入以测试页面大小,即 4096 字节。但是当我调用 malloc(1) 时,我在访问 135152 字节后得到了 SEGV,这远远超过一个页面大小。我知道 malloc 是库函数,它依赖于实现,但考虑到它最终会调用 sbrk,为什么它会给出一个以上的页面大小。谁能告诉我它的内部工作原理?
我的操作系统是 ubuntu 14.04,我的架构是 x86
更新:现在我想知道是否是因为 malloc returns 空闲列表块的地址足够大以容纳我的数据。但是那个地址可能在堆的中间,这样我就可以一直写,直到达到堆的上限。
malloc
出于性能原因在大块中分配内存。对 malloc
的后续调用可以从大块中为您提供内存,而不必向操作系统请求大量小块。这减少了所需的系统调用次数。
来自this article:
When a process needs memory, some room is created by moving the upper bound of the heap forward, using the brk() or sbrk() system calls. Because a system call is expensive in terms of CPU usage, a better strategy is to call brk() to grab a large chunk of memory and then split it as needed to get smaller chunks. This is exactly what malloc() does. It aggregates a lot of smaller malloc() requests into fewer large brk() calls. Doing so yields a significant performance improvement.
请注意,malloc
的某些现代实现使用 mmap
而不是 brk
/sbrk
来分配内存,但除此之外,上述内容仍然适用。
旧的 malloc()
UNIX 实现使用了 sbrk()
/brk()
系统调用。但是现在,实现使用 mmap()
和 sbrk()
。 glibc 的 malloc()
实现(可能是您在 Ubuntu 14.04 上使用的那个)同时使用 sbrk()
and mmap()
并且在您请求时选择使用哪个来分配通常取决于大小glibc 动态执行的分配请求。
对于小的分配,glibc 使用 sbrk()
,对于较大的分配,它使用 mmap()
。宏M_MMAP_THRESHOLD
is used to decide this. Currently, it's default value is set to 128K。这解释了为什么您的代码设法分配了 135152 个字节,因为它大约是 ~128K。尽管您只请求了 1 个字节,但您的实现分配了 128K 以实现高效的内存分配。所以在你超过这个限制之前不会发生段错误。
您可以通过更改默认参数使用 mallopt()
来玩 M_MAP_THRESHOLD
。
M_MMAP_THRESHOLD
For allocations greater than or equal to the limit specified (in bytes) by M_MMAP_THRESHOLD that can't be satisfied from the free list, the memory-allocation functions employ mmap(2) instead of increasing the program break using sbrk(2).
Allocating memory using mmap(2) has the significant advantage that the allocated memory blocks can always be independently released back to the system. (By contrast, the heap can be trimmed only if memory is freed at the top end.) On the other hand, there are some disadvantages to the use of mmap(2): deallocated space is not placed on the free list for reuse by later allocations; memory may be wasted because mmap(2) allocations must be page-aligned; and the kernel must perform the expensive task of zeroing out memory allocated via mmap(2). Balancing these factors leads to a default setting of 128*1024 for the M_MMAP_THRESHOLD parameter.
The lower limit for this parameter is 0. The upper limit is DEFAULT_MMAP_THRESHOLD_MAX: 512*1024 on 32-bit systems or 4*1024*1024*sizeof(long) on 64-bit systems.
Note: Nowadays, glibc uses a dynamic mmap threshold by default. The initial value of the threshold is 128*1024, but when blocks larger than the current threshold and less than or equal to DEFAULT_MMAP_THRESHOLD_MAX are freed, the threshold is adjusted upward to the size of the freed block. When dynamic mmap thresholding is in effect, the threshold for trimming the heap is also dynamically adjusted to be twice the dynamic mmap threshold. Dynamic adjustment of the mmap threshold is disabled if any of the M_TRIM_THRESHOLD, M_TOP_PAD, M_MMAP_THRESHOLD, or M_MMAP_MAX parameters is set.
例如,如果您这样做:
#include<malloc.h>
mallopt(M_MMAP_THRESHOLD, 0);
在调用 malloc()
之前,您可能会看到不同的限制。其中大部分是实现细节,C 标准说它是 undefined behaviour to write into memory that your process doesn't own. So do it at your risk -- otherwise, demons may fly out of your nose ;-)