WAL-E:备份获取后无法重新启动 postgresql

WAL-E: Cannot restart postgresql after backup-fetch

这是在 Ubuntu 14.04 LTS

编辑:

WAL-E 使用 pip 安装,密钥由 envdir 管理,根据说明 https://gist.github.com/elithrar/8682235 and https://github.com/wal-e/wal-e#dependencies

我正在尝试使用 WAL-E 恢复数据库,一开始一切似乎都很顺利,因为我安装了 Postgres 并且 运行,并且可以轻松地创建或恢复数据库并在本地访问它并且通过 pgadmin 远程。当我尝试使用 wal-e fetch-backup 从 S3 备份执行恢复时出现问题。在启动 postgres 之前,它似乎进展顺利。

出现了一些错误,似乎是缺少包或权限问题,如下所示:

* Starting PostgreSQL 9.3 database server 
    * The PostgreSQL server failed to start. Please check the log output:
    2015-07-11 00:41:11 EDT LOG:  database system was interrupted; last known up at 2015-06-30 05:00:02 EDT
        2015-07-11 00:41:11 EDT LOG:  starting archive recovery
        Traceback (most recent call last):
          File "/usr/local/bin/wal-e", line 5, in <module>
            from pkg_resources import load_entry_point
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3084, in <module>
            @_call_aside
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3070, in _call_aside
            f(*args, **kwargs)
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
            working_set = WorkingSet._build_master()
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 651, in _build_master
            ws.require(__requires__)
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 952, in require
            needed = self.resolve(parse_requirements(requirements))
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 847, in resolve
            new_requirements = dist.requires(req.extras)[::-1]
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2602, in requires
            dm = self._dep_map
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2803, in _dep_map
            self.__dep_map = self._compute_dependencies()
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2825, in _compute_dependencies
            for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2794, in _parsed_pkg_info
            metadata = self.get_metadata(self.PKG_INFO)
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1617, in get_metadata
            return self._get(self._fn(self.egg_info, name))
          File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1728, in _get
            with open(path, 'rb') as stream:
        IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/six-1.9.0.dist-info/METADATA'
        2015-07-11 00:41:11 EDT LOG:  invalid checkpoint record
        2015-07-11 00:41:11 EDT FATAL:  could not locate required checkpoint record
        2015-07-11 00:41:11 EDT HINT:  If you are not restoring from a backup, try removing the file "/var/lib/postgresql/9.3/main/backup_label".
        2015-07-11 00:41:11 EDT LOG:  startup process (PID 1693) exited with exit code 1
        2015-07-11 00:41:11 EDT LOG:  aborting startup due to startup process failure

我有几个这样的问题,并且能够通过更改组和修改注释文件的权限以匹配目录中的其他文件来解决它们,但我怀疑问题更多地与 and/or 这些软件包的安装位置。解决上述问题后,postgres仍然启动失败,返回如下:

* Starting PostgreSQL 9.3 database server
* The PostgreSQL server failed to start. Please check the log output:
2015-07-11 00:30:04 EDT LOG:  database system was interrupted; last known up at 2015-06-30 05:00:02 EDT
2015-07-11 00:30:04 EDT LOG:  starting archive recovery
Traceback (most recent call last):
  File "/usr/local/bin/wal-e", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3084, in <module>
    @_call_aside
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3070, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 3097, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 651, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 952, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 839, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'wal-e==0.8.1' distribution was not found and is required by the application
2015-07-11 00:30:04 EDT LOG:  invalid checkpoint record
2015-07-11 00:30:04 EDT FATAL:  could not locate required checkpoint record
2015-07-11 00:30:04 EDT HINT:  If you are not restoring from a backup, try removing the file "/var/lib/postgresql/9.3/main/backup_label".
2015-07-11 00:30:04 EDT LOG:  startup process (PID 1495) exited with exit code 1
2015-07-11 00:30:04 EDT LOG:  aborting startup due to startup process failure

它抱怨the 'wal-e==0.8.1' distribution was not found...,但它显然已安装并可执行:

ls -l /usr/local/lib/python2.7/dist-packages
total 608
drwxr-sr-x  2 root staff  4096 Jul 10 20:14 argparse-1.3.0.dist-info
-rw-r--r--  1 root staff 88400 Jul 10 20:14 argparse.py
-rw-r--r--  1 root staff 65659 Jul 10 20:14 argparse.pyc
drwxr-sr-x  6 root staff  4096 Jul  9 22:36 azure
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 azure-0.11.1.egg-info
drwxr-sr-x  5 root staff  4096 Jul  9 22:36 babel
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 Babel-1.3.egg-info
drwxr-sr-x 57 root staff  4096 Jul  9 22:36 boto
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 boto-2.38.0.dist-info
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 concurrent
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 dateutil
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 debtcollector
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 debtcollector-0.5.0.dist-info
-rw-r--r--  1 root staff   207 Jul 10 20:10 easy-install.pth
-rw-r--r--  1 root staff   126 Jul 10 20:33 easy_install.py
-rw-r--r--  1 root staff   315 Jul 10 20:33 easy_install.pyc
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 futures-3.0.3.dist-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 gevent
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 gevent-1.0.2.egg-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 greenlet-0.4.7.egg-info
-rwxr-xr-x  1 root staff 82869 Jul  9 22:36 greenlet.so
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 iso8601
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 iso8601-0.1.10.egg-info
drwxr-sr-x 15 root staff  4096 Jul  9 22:36 keystoneclient
drwxr-sr-x  2 root staff  4096 Jul 10 20:33 _markerlib
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 msgpack
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 msgpack_python-0.4.6.egg-info
drwxr-sr-x  5 root staff  4096 Jul  9 22:36 netaddr
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 netaddr-0.7.15.dist-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 netifaces-0.10.4.egg-info
-rwxr-xr-x  1 root staff 58386 Jul  9 22:36 netifaces.so
drwxr-sr-x  4 root staff  4096 Jul  9 22:36 oslo
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 oslo_config
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 oslo.config-1.14.0.dist-info
-rw-r--r--  1 root root    299 Jul  9 22:35 oslo.config-1.14.0-py2.7-nspkg.pth
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 oslo_i18n
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 oslo.i18n-2.1.0.dist-info
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 oslo_serialization
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 oslo.serialization-1.7.0.dist-info
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 oslo_utils
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 oslo.utils-1.8.0.dist-info
-rw-r--r--  1 root root    299 Jul  9 22:35 oslo.utils-1.8.0-py2.7-nspkg.pth
drwxr-sr-x  5 root staff  4096 Jul 10 20:14 pbr
drwxr-sr-x  2 root staff  4096 Jul 10 20:14 pbr-1.3.0.dist-info
drwxr-sr-x  4 root staff  4096 Jul 10 20:10 pip-7.1.0-py2.7.egg
drwxr-sr-x  3 root staff  4096 Jul 10 20:33 pkg_resources
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 python_dateutil-2.4.2.dist-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 python_keystoneclient-1.6.0.dist-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 python_swiftclient-2.4.0.dist-info
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 pytz
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 pytz-2015.4.dist-info
drwxr-sr--  3 root staff  4096 Jul  9 23:25 requests
drwxr-sr--  2 root staff  4096 Jul  9 23:25 requests-2.7.0.dist-info
drwxr-sr-x  3 root staff  4096 Jul 10 20:33 setuptools
drwxr-sr-x  2 root staff  4096 Jul 10 20:33 setuptools-18.0.1.dist-info
drwxr-sr-x  3 root staff  4096 Jul  9 22:36 simplejson
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 simplejson-3.7.3.egg-info
drwxr-sr--  2 root staff  4096 Jul  9 23:26 six-1.9.0.dist-info
-rw-r--r--  1 root root  29664 Jul  9 23:26 six.py
-rw-r--r--  1 root root  29006 Jul  9 23:26 six.pyc
drwxr-sr-x  4 root staff  4096 Jul  9 22:36 stevedore
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 stevedore-1.6.0.dist-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 swiftclient
drwxr-sr-x  7 root staff  4096 Jul  9 22:35 wal_e
drwxr-sr-x  2 root staff  4096 Jul  9 22:35 wal_e-0.8.1.egg-info
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 wrapt
drwxr-sr-x  2 root staff  4096 Jul  9 22:36 wrapt-1.10.5.egg-info

我进行了大量搜索,但没有找到任何有用的信息。对正确方向的任何建议或观点表示赞赏。

此外,解决此问题以确保我的主要安装在需要恢复的情况下正常运行将非常有意义。如果无法恢复,备份毫无意义。

我不完全确定是什么解决了 The 'wal-e==0.8.1' distribution was not found 错误,但在重新启动时我再也没有看到它。

除此之外,解决这个问题非常简单。

许多 python 发行版的可执行位未设置。

Chowning 这些修复了错误:

      chmod o+x /usr/local/lib/python2.7/dist-packages/requests-2.7.0.dist-info/
      chmod o+x /usr/local/lib/python2.7/dist-packages/requests
...

原因backup_label未正确创建

运行 Master 中的以下命令

  • su postgres

    psql -c "select pg_start_backup('initial_backup');"

    rsync -cva --inplace --exclude=pg_xlog /var/lib/postgresql/9.1/主要/ slave_IP_address:/var/lib/postgresql/9.1/main/

    psql -c "select pg_stop_backup();"

  • cd /var/lib/postgresql/9.1/main

创建文件名recovery.conf添加以下行

  • standby_mode = 'on' primary_conninfo = '主机=master_IP_address port=5432 user=rep password=你的密码' trigger_file = '/tmp/postgresql.trigger.5432'

服务 postgresql 重启