mongodump 错误 "Failed: error reading from db: EOF"(服务器日志中没有条目)
mongodump error "Failed: error reading from db: EOF" (no entries in server log)
我们的 mongodb.conf 版本 3.06 和数据文件只有 240 MB 大小。网络在此时间戳是可靠的。
# mongod.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/db/db32/mongodb/data/
# path to logfile
logpath=/db/db32/mongodb/logs/mongod.log
# add new entries to the end of the logfile
logappend=true
# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1
# enable operation journaling
#journal = true
#smallfiles = true
nojournal = true
# Enables periodic logging of CPU utilization and I/O wait
cpu = true
# enable database authentication for users connecting from remote hosts
auth = true
# Verbose logging output.
#verbose = true
# Enable db quota management
#quota = true
# Set oplogging level where n is
# 0=off (default)
# 1=W
# 2=R
# 3=both
# 7=W+some reads
#diaglog = 0
# Ignore query hints
#nohints = true
# Turns off server-side scripting. This will result in greatly limited
# functionality
noscripting = true
# Turns off table scans. Any query that would do a table scan fails.
#notablescan = true
# Disable data file preallocation.
#noprealloc = true
# Specify .ns file size for new databases.
# nssize = <size>
# Replication Options
# in replicated mongo databases, specify the replica set name here
#replSet=setname
# maximum size in megabytes for replication operation log
#oplogSize=1024
# path to a key file storing authentication info for connections
# between replica set members
#keyFile=/path/to/keyfile
# Forces the mongod to validate all requests from clients
objcheck = true
# Disable HTTP status interface
nohttpinterface = true
# disable REST interface
rest = false
# database profiling 1 = only includes slow operations
profile = 1
# logs slow queries to the log
slowms = 100
# maximum number of simultaneous connections
maxConns = 25
mongo带有详细标志的转储。在服务器日志中没有此时间戳的条目。
(...)
2016-03-09T16:51:17.378+0100 enqueued collection 'Tetzi005.xxx'
2016-03-09T16:51:17.384+0100 enqueued collection 'Tetzi005.xxxxxx'
2016-03-09T16:51:17.391+0100 enqueued collection 'Tetzi005.system.indexes'
2016-03-09T16:51:17.391+0100 finalizing intent manager with longest task first prioritizer
2016-03-09T16:51:17.391+0100 dumping with 8 job threads
2016-03-09T16:51:17.391+0100 starting dump routine with id=0
2016-03-09T16:51:17.391+0100 starting dump routine with id=4
2016-03-09T16:51:17.391+0100 starting dump routine with id=1
2016-03-09T16:51:17.391+0100 writing Tetzi005.DailyEmailUser to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.391+0100 starting dump routine with id=3
2016-03-09T16:51:17.391+0100 starting dump routine with id=6
2016-03-09T16:51:17.391+0100 writing Tetzi005.Prototype to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=7
2016-03-09T16:51:17.392+0100 writing Tetzi005.ProfileUser to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=2
2016-03-09T16:51:17.392+0100 writing Tetzi005.OrganizationDataSet to dbbackup/dump/Tetzi005/xxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.DailyUserCount to dbbackup/dump/Tetzi005/xxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.DailyEmailOrganization to dbbackup/dump/Tetzi005/xxxxxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=5
2016-03-09T16:51:17.392+0100 writing Tetzi005.OrganizationStatistics to dbbackup/dump/Tetzi005/xxxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.Organization to dbbackup/dump/Tetzi005/xxxx.bson
2016-03-09T16:51:17.398+0100 counted 112 documents in Tetzi005.xxxxxxxxxxxxx
2016-03-09T16:51:17.398+0100 counted 475 documents in Tetzi005.xxxxxxxx
2016-03-09T16:51:17.405+0100 Failed: error reading from db: EOF
谷歌搜索 Failed: error reading from db: EOF
时没有解决方案
我们只有 large
计划有这个问题。从技术上讲,配置没有什么不同(除了内存、磁盘和 maxConns)。 Docker 容器中的所有 mongo 服务器 运行。 Docker 在带有 RHEL7 的 OpenStack 虚拟机上运行。
small Maximum 10 concurrent connections, 1GB storage, 256MB memory paid
medium Maximum 15 concurrent connections, 8GB storage, 1GB memory paid
large Maximum 25 concurrent connections, 16GB storage, 4GB memory paid
"Failed: error reading from db: EOF"
是由于oplog写出时运行内存不足造成的
错误Failed: error reading from db: EOF
是由于运行oplog写出时内存不足造成的。
当您 运行 mongodump
添加 --quiet
选项时,您可以使用更少的内存。
mongodump --quiet
对于容器化设置,您必须记住设置 storage.wiredTiger.engineConfig.cacheSizeGB
。
The storage.wiredTiger.engineConfig.cacheSizeGB
limits the size of the WiredTiger internal cache. The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory. In addition, the operating system will use any free RAM to buffer file system blocks and file system cache.
To accommodate the additional consumers of RAM, you may have to decrease WiredTiger internal cache size.
我们的 mongodb.conf 版本 3.06 和数据文件只有 240 MB 大小。网络在此时间戳是可靠的。
# mongod.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/db/db32/mongodb/data/
# path to logfile
logpath=/db/db32/mongodb/logs/mongod.log
# add new entries to the end of the logfile
logappend=true
# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1
# enable operation journaling
#journal = true
#smallfiles = true
nojournal = true
# Enables periodic logging of CPU utilization and I/O wait
cpu = true
# enable database authentication for users connecting from remote hosts
auth = true
# Verbose logging output.
#verbose = true
# Enable db quota management
#quota = true
# Set oplogging level where n is
# 0=off (default)
# 1=W
# 2=R
# 3=both
# 7=W+some reads
#diaglog = 0
# Ignore query hints
#nohints = true
# Turns off server-side scripting. This will result in greatly limited
# functionality
noscripting = true
# Turns off table scans. Any query that would do a table scan fails.
#notablescan = true
# Disable data file preallocation.
#noprealloc = true
# Specify .ns file size for new databases.
# nssize = <size>
# Replication Options
# in replicated mongo databases, specify the replica set name here
#replSet=setname
# maximum size in megabytes for replication operation log
#oplogSize=1024
# path to a key file storing authentication info for connections
# between replica set members
#keyFile=/path/to/keyfile
# Forces the mongod to validate all requests from clients
objcheck = true
# Disable HTTP status interface
nohttpinterface = true
# disable REST interface
rest = false
# database profiling 1 = only includes slow operations
profile = 1
# logs slow queries to the log
slowms = 100
# maximum number of simultaneous connections
maxConns = 25
mongo带有详细标志的转储。在服务器日志中没有此时间戳的条目。
(...)
2016-03-09T16:51:17.378+0100 enqueued collection 'Tetzi005.xxx'
2016-03-09T16:51:17.384+0100 enqueued collection 'Tetzi005.xxxxxx'
2016-03-09T16:51:17.391+0100 enqueued collection 'Tetzi005.system.indexes'
2016-03-09T16:51:17.391+0100 finalizing intent manager with longest task first prioritizer
2016-03-09T16:51:17.391+0100 dumping with 8 job threads
2016-03-09T16:51:17.391+0100 starting dump routine with id=0
2016-03-09T16:51:17.391+0100 starting dump routine with id=4
2016-03-09T16:51:17.391+0100 starting dump routine with id=1
2016-03-09T16:51:17.391+0100 writing Tetzi005.DailyEmailUser to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.391+0100 starting dump routine with id=3
2016-03-09T16:51:17.391+0100 starting dump routine with id=6
2016-03-09T16:51:17.391+0100 writing Tetzi005.Prototype to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=7
2016-03-09T16:51:17.392+0100 writing Tetzi005.ProfileUser to dbbackup/dump/Tetzi005/xxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=2
2016-03-09T16:51:17.392+0100 writing Tetzi005.OrganizationDataSet to dbbackup/dump/Tetzi005/xxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.DailyUserCount to dbbackup/dump/Tetzi005/xxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.DailyEmailOrganization to dbbackup/dump/Tetzi005/xxxxxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 starting dump routine with id=5
2016-03-09T16:51:17.392+0100 writing Tetzi005.OrganizationStatistics to dbbackup/dump/Tetzi005/xxxxxxxxxxx.bson
2016-03-09T16:51:17.392+0100 writing Tetzi005.Organization to dbbackup/dump/Tetzi005/xxxx.bson
2016-03-09T16:51:17.398+0100 counted 112 documents in Tetzi005.xxxxxxxxxxxxx
2016-03-09T16:51:17.398+0100 counted 475 documents in Tetzi005.xxxxxxxx
2016-03-09T16:51:17.405+0100 Failed: error reading from db: EOF
谷歌搜索 Failed: error reading from db: EOF
我们只有 large
计划有这个问题。从技术上讲,配置没有什么不同(除了内存、磁盘和 maxConns)。 Docker 容器中的所有 mongo 服务器 运行。 Docker 在带有 RHEL7 的 OpenStack 虚拟机上运行。
small Maximum 10 concurrent connections, 1GB storage, 256MB memory paid
medium Maximum 15 concurrent connections, 8GB storage, 1GB memory paid
large Maximum 25 concurrent connections, 16GB storage, 4GB memory paid
"Failed: error reading from db: EOF"
是由于oplog写出时运行内存不足造成的
错误Failed: error reading from db: EOF
是由于运行oplog写出时内存不足造成的。
当您 运行 mongodump
添加 --quiet
选项时,您可以使用更少的内存。
mongodump --quiet
对于容器化设置,您必须记住设置 storage.wiredTiger.engineConfig.cacheSizeGB
。
The
storage.wiredTiger.engineConfig.cacheSizeGB
limits the size of the WiredTiger internal cache. The operating system will use the available free memory for filesystem cache, which allows the compressed MongoDB data files to stay in memory. In addition, the operating system will use any free RAM to buffer file system blocks and file system cache.To accommodate the additional consumers of RAM, you may have to decrease WiredTiger internal cache size.