将旧版本 mongo 的数据库转储恢复到新版本 mongo
Restoring the database dump of an older version of mongo to a new version of mongo
目前,我的系统上有旧版本的 mongo,即 2.6 运行。我的网站已经投入生产并且拥有大量客户数据。我正计划升级到 mongo 3.2.
所以,我的问题是 mongomongov3.2 的恢复是否与 v2.6[的数据转储一起工作?或者,它是否会产生问题?
任何答案都将是无价的!谢谢
因为您有来自 mongo 2.6 的数据,索引字段限制已经满足。
Mongo 3.2 将毫无问题地恢复此备份。
另一种升级数据库的方法(如果你有副本集)是用 3.2 替换一个 2.6 成员并等待同步,然后另一个......这会给你业务连续性:-)
我问这个same question on the official MongoDB mailing list。他们说一次不要升级超过 1 个主要版本。 (主要版本为:2.2、2.4、2.6、3.0、3.2、3.4)
我不想遵循安装每个版本的正常升级过程只是为了启动 mongod 然后将其关闭。在我看来,这会留下一些麻烦,我喜欢让我的基础架构构建脚本和版本受控。因此,我决定使用最新的 Ubuntu(因为我的 Mongo v2.4 服务器也落后 2 个 LTS 版本)和最新的 MongoDB 启动新的 EC2 实例。我使用 docker 中间版本的 MongoDB 图像来进行数据升级。
大部分解决方案是这样的:
# mongo.conf is using the default dbPath: /var/lib/mongodb
# this path is for temporary use by the mongo docker container
mkdir -p /data/db/dump
# see: https://hub.docker.com/_/mongo/ (search for /data/db)
# see: https://github.com/docker-library/mongo/blob/30d09dbd6343d3cbd1bbea2d6afde49f5d9a9295/3.4/Dockerfile#L59
cd /data/db
mongodump -h prodmongo.int
# Get major versions from https://hub.docker.com/r/library/mongo/tags/
step=0
for major_version in 2.6.12 3.0.14 3.2.11 3.4.1; do
sudo docker stop some-mongo || true
sudo docker rm some-mongo || true
sudo docker run --name some-mongo -v /data/db:/data/db -d mongo:$major_version
false; while [[ $? > 0 ]]; do
sleep 0.5
sudo docker exec -it some-mongo mongo --eval 'printjson((new Mongo()).getDBNames())'
done
if (( $step == 0 )); then
sudo docker exec -it some-mongo mongorestore /data/db/dump
fi
((step += 1))
done
# Finish up with docker
sudo rm -rf /data/db/dump/*
sudo docker exec -it some-mongo bash -c 'cd /data/db; mongodump'
sudo docker stop some-mongo
sudo docker rm some-mongo
# Load upgraded data into latest version of MongoDB (WiredTiger storage engine will be used)
mongorestore /data/db/dump
sudo rm -rf /data
为我工作到 3.4 版,但是每次更新都需要设置一个 功能兼容版本 ,然后在 4.2 上你不能开始 mongo 如果你还在使用 MMAPv1,你需要切换到 WiredTiger。
所以,非常感谢 Bruno,你让我完成了大部分工作。
以下是我更新后的脚本,希望对大家有所帮助!
#!/bin/bash
# Log commands to stdout
set -o xtrace
# Exit on error
set -o errexit
# Exit on use of unset variables
set -o nounset
# Exit and report on pipe failure
set -o pipefail
# mongo.conf is using the default dbPath: /var/lib/mongodb
# this path is for temporary use by the mongo docker container
mkdir -p /data/db/dump
# see: https://hub.docker.com/_/mongo/ (search for /data/db)
# see: https://github.com/docker-library/mongo/blob/30d09dbd6343d3cbd1bbea2d6afde49f5d9a9295/3.4/Dockerfile#L59
cd /data/db
# our docker host holding the database
mongodump -h $PROD_MONGO_IP
# Get major versions from https://hub.docker.com/r/library/mongo/tags/
step=0
prev_real_major=""
for major_version in 2.6.12 3.0.15 3.2.21 3.4.24 3.6.23 4.0.27 4.2.17 4.4.10 5.0.3; do
real_major=`echo $major_version | cut -f1,2 -d "."`
sudo docker stop some-mongo || true
sudo docker rm some-mongo || true
docker run --rm -d --name some-mongo -v /data/db:/data/db mongo:$major_version
set +o errexit
false; while [[ $? > 0 ]]; do
sleep 0.5
docker exec -it some-mongo mongo --eval 'printjson((new Mongo()).getDBNames())'
if [[ $real_major > 3.3 ]]; then
docker exec -it some-mongo mongo --eval "db.adminCommand( { setFeatureCompatibilityVersion: \"$real_major\" } )"
fi
done
set -o errexit
if (( $step == 0 )); then
docker exec -it some-mongo mongorestore /data/db/dump
fi
# upgrade to WiredTiger
if [[ $real_major == 4.0 ]]; then
# delete the database dump from earlier
rm -rf /data/db/dump/
# dump the database again
docker exec -w /data/db -it some-mongo mongodump
# stop the existing mongo container
docker stop some-mongo
# delete everything in /data/db except /data/db/dump
find /data/db -mindepth 1 ! -regex '^/data/db/dump\(/.*\)?' -delete
# run the 4.0 mongo container again
docker run --rm -d --name some-mongo -v /data/db:/data/db mongo:$major_version
# restore the database, which automatically makes it wiretiger.
docker exec -it some-mongo mongorestore /data/db/dump
fi
((step += 1))
done
# Finish up with docker
sudo rm -rf /data/db/dump/*
docker exec -it some-mongo bash -c 'cd /data/db; mongodump'
docker stop some-mongo
# Commented these out because I did them manually, you decide what you want to do here
# Load upgraded data into latest version of MongoDB (WiredTiger storage engine will be used)
# mongorestore /data/db/dump
# sudo rm -rf /data
目前,我的系统上有旧版本的 mongo,即 2.6 运行。我的网站已经投入生产并且拥有大量客户数据。我正计划升级到 mongo 3.2.
所以,我的问题是 mongomongov3.2 的恢复是否与 v2.6[的数据转储一起工作?或者,它是否会产生问题?
任何答案都将是无价的!谢谢
因为您有来自 mongo 2.6 的数据,索引字段限制已经满足。 Mongo 3.2 将毫无问题地恢复此备份。
另一种升级数据库的方法(如果你有副本集)是用 3.2 替换一个 2.6 成员并等待同步,然后另一个......这会给你业务连续性:-)
我问这个same question on the official MongoDB mailing list。他们说一次不要升级超过 1 个主要版本。 (主要版本为:2.2、2.4、2.6、3.0、3.2、3.4)
我不想遵循安装每个版本的正常升级过程只是为了启动 mongod 然后将其关闭。在我看来,这会留下一些麻烦,我喜欢让我的基础架构构建脚本和版本受控。因此,我决定使用最新的 Ubuntu(因为我的 Mongo v2.4 服务器也落后 2 个 LTS 版本)和最新的 MongoDB 启动新的 EC2 实例。我使用 docker 中间版本的 MongoDB 图像来进行数据升级。
大部分解决方案是这样的:
# mongo.conf is using the default dbPath: /var/lib/mongodb
# this path is for temporary use by the mongo docker container
mkdir -p /data/db/dump
# see: https://hub.docker.com/_/mongo/ (search for /data/db)
# see: https://github.com/docker-library/mongo/blob/30d09dbd6343d3cbd1bbea2d6afde49f5d9a9295/3.4/Dockerfile#L59
cd /data/db
mongodump -h prodmongo.int
# Get major versions from https://hub.docker.com/r/library/mongo/tags/
step=0
for major_version in 2.6.12 3.0.14 3.2.11 3.4.1; do
sudo docker stop some-mongo || true
sudo docker rm some-mongo || true
sudo docker run --name some-mongo -v /data/db:/data/db -d mongo:$major_version
false; while [[ $? > 0 ]]; do
sleep 0.5
sudo docker exec -it some-mongo mongo --eval 'printjson((new Mongo()).getDBNames())'
done
if (( $step == 0 )); then
sudo docker exec -it some-mongo mongorestore /data/db/dump
fi
((step += 1))
done
# Finish up with docker
sudo rm -rf /data/db/dump/*
sudo docker exec -it some-mongo bash -c 'cd /data/db; mongodump'
sudo docker stop some-mongo
sudo docker rm some-mongo
# Load upgraded data into latest version of MongoDB (WiredTiger storage engine will be used)
mongorestore /data/db/dump
sudo rm -rf /data
所以,非常感谢 Bruno,你让我完成了大部分工作。
以下是我更新后的脚本,希望对大家有所帮助!
#!/bin/bash
# Log commands to stdout
set -o xtrace
# Exit on error
set -o errexit
# Exit on use of unset variables
set -o nounset
# Exit and report on pipe failure
set -o pipefail
# mongo.conf is using the default dbPath: /var/lib/mongodb
# this path is for temporary use by the mongo docker container
mkdir -p /data/db/dump
# see: https://hub.docker.com/_/mongo/ (search for /data/db)
# see: https://github.com/docker-library/mongo/blob/30d09dbd6343d3cbd1bbea2d6afde49f5d9a9295/3.4/Dockerfile#L59
cd /data/db
# our docker host holding the database
mongodump -h $PROD_MONGO_IP
# Get major versions from https://hub.docker.com/r/library/mongo/tags/
step=0
prev_real_major=""
for major_version in 2.6.12 3.0.15 3.2.21 3.4.24 3.6.23 4.0.27 4.2.17 4.4.10 5.0.3; do
real_major=`echo $major_version | cut -f1,2 -d "."`
sudo docker stop some-mongo || true
sudo docker rm some-mongo || true
docker run --rm -d --name some-mongo -v /data/db:/data/db mongo:$major_version
set +o errexit
false; while [[ $? > 0 ]]; do
sleep 0.5
docker exec -it some-mongo mongo --eval 'printjson((new Mongo()).getDBNames())'
if [[ $real_major > 3.3 ]]; then
docker exec -it some-mongo mongo --eval "db.adminCommand( { setFeatureCompatibilityVersion: \"$real_major\" } )"
fi
done
set -o errexit
if (( $step == 0 )); then
docker exec -it some-mongo mongorestore /data/db/dump
fi
# upgrade to WiredTiger
if [[ $real_major == 4.0 ]]; then
# delete the database dump from earlier
rm -rf /data/db/dump/
# dump the database again
docker exec -w /data/db -it some-mongo mongodump
# stop the existing mongo container
docker stop some-mongo
# delete everything in /data/db except /data/db/dump
find /data/db -mindepth 1 ! -regex '^/data/db/dump\(/.*\)?' -delete
# run the 4.0 mongo container again
docker run --rm -d --name some-mongo -v /data/db:/data/db mongo:$major_version
# restore the database, which automatically makes it wiretiger.
docker exec -it some-mongo mongorestore /data/db/dump
fi
((step += 1))
done
# Finish up with docker
sudo rm -rf /data/db/dump/*
docker exec -it some-mongo bash -c 'cd /data/db; mongodump'
docker stop some-mongo
# Commented these out because I did them manually, you decide what you want to do here
# Load upgraded data into latest version of MongoDB (WiredTiger storage engine will be used)
# mongorestore /data/db/dump
# sudo rm -rf /data