为什么在 32 位 armv7l 上 mmap 一个 4GB 的文件成功了?
Why mmap a 4GB file on 32-bit armv7l succeeded?
我从 mmap(2)
手册页和搜索结果中得到的印象是,mmap
只限于系统的可用地址空间,减去系统保留的地址空间。所以在 32 位 armv7l 上,我假设它大约是 3GB = (4GB - 1GB).
但似乎我实际上 mmap
一个 5 GB 的文件没有任何问题:
int main(int argc, char** argv) {
// stats
char * path = argv[1];
struct stat sb;
stat(path, &sb);
std::cout << "File size: " << sb.st_size << std::endl;
// open
int fd = open(path, O_RDONLY, S_IRWXU);
std::cout << "File descriptor: " << fd << std::endl;
int i;
for (i =0; i<10; ++i){
void *pa = mmap(
nullptr, sb.st_size, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, fd, 0);
std::cout << "PA: " << pa
<< ", MAP_FAILED: "
<< (pa == MAP_FAILED) << ", Status: "
<< strerror(errno) << std::endl;
}
}
使用 -D_FILE_OFFSET_BITS=64
标志编译:
g++ -D_FILE_OFFSET_BITS=64 test.cc
结果为:
File size: 5045966585
File descriptor: 3
PA: 0x89f80000, MAP_FAILED: 0, Status: Success
PA: 0x5d34a000, MAP_FAILED: 0, Status: Success
PA: 0x30714000, MAP_FAILED: 0, Status: Success
PA: 0x3ade000, MAP_FAILED: 0, Status: Success
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
从结果来看,mmap成功了4次才真正出事。但它不应该成功,因为文件是 ~5GB。
我的问题是:
mmap
是否会出现这种行为?
- 如果不是,我哪里做错了?
编辑:
With physical addres extension (PAE) 32-bit systems can addres much more than 2^32 bytes, if that is available.
没有对此的 PAE 支持CPU
$> cat /proc/cpuinfo
Processor : ARMv7 Processor rev 4 (v7l)
processor : 0
BogoMIPS : 1436.46
processor : 1
BogoMIPS : 1436.46
Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xc07
CPU revision : 4
Hardware : sun7i
Revision : 0000
Serial : 09c11b9d52544848804857831651664b
PAE 无关紧要。这与访问大量物理内存无关。
问题是您的 mmap
函数采用 32 位值作为映射大小。所以你的 64 位大小被截断了,你实际上分配了不到 1 GB 的虚拟内存。
我从 mmap(2)
手册页和搜索结果中得到的印象是,mmap
只限于系统的可用地址空间,减去系统保留的地址空间。所以在 32 位 armv7l 上,我假设它大约是 3GB = (4GB - 1GB).
但似乎我实际上 mmap
一个 5 GB 的文件没有任何问题:
int main(int argc, char** argv) {
// stats
char * path = argv[1];
struct stat sb;
stat(path, &sb);
std::cout << "File size: " << sb.st_size << std::endl;
// open
int fd = open(path, O_RDONLY, S_IRWXU);
std::cout << "File descriptor: " << fd << std::endl;
int i;
for (i =0; i<10; ++i){
void *pa = mmap(
nullptr, sb.st_size, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, fd, 0);
std::cout << "PA: " << pa
<< ", MAP_FAILED: "
<< (pa == MAP_FAILED) << ", Status: "
<< strerror(errno) << std::endl;
}
}
使用 -D_FILE_OFFSET_BITS=64
标志编译:
g++ -D_FILE_OFFSET_BITS=64 test.cc
结果为:
File size: 5045966585
File descriptor: 3
PA: 0x89f80000, MAP_FAILED: 0, Status: Success
PA: 0x5d34a000, MAP_FAILED: 0, Status: Success
PA: 0x30714000, MAP_FAILED: 0, Status: Success
PA: 0x3ade000, MAP_FAILED: 0, Status: Success
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
PA: 0xffffffff, MAP_FAILED: 1, Status: Cannot allocate memory
从结果来看,mmap成功了4次才真正出事。但它不应该成功,因为文件是 ~5GB。
我的问题是:
mmap
是否会出现这种行为?- 如果不是,我哪里做错了?
编辑:
With physical addres extension (PAE) 32-bit systems can addres much more than 2^32 bytes, if that is available.
没有对此的 PAE 支持CPU
$> cat /proc/cpuinfo
Processor : ARMv7 Processor rev 4 (v7l)
processor : 0
BogoMIPS : 1436.46
processor : 1
BogoMIPS : 1436.46
Features : swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xc07
CPU revision : 4
Hardware : sun7i
Revision : 0000
Serial : 09c11b9d52544848804857831651664b
PAE 无关紧要。这与访问大量物理内存无关。
问题是您的 mmap
函数采用 32 位值作为映射大小。所以你的 64 位大小被截断了,你实际上分配了不到 1 GB 的虚拟内存。