InnoDB：Header 页由数据文件中的零字节组成：./ibdata1
InnoDB: Header page consists of zero bytes in datafile: ./ibdata1
我正在 openshift (kubernetes) 上部署 MySQL 但出现以下错误
---> 10:01:34 Initializing database ...
---> 10:01:34 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
---> 10:01:35 Starting MySQL server with disabled networking ...
---> 10:01:35 Waiting for MySQL to start ...
2019-10-18T10:01:35.188509Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-10-18T10:01:35.188567Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
2019-10-18T10:01:35.325995Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 40 ...
2019-10-18T10:01:35.430574Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-10-18T10:01:35.430612Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-10-18T10:01:35.430621Z 0 [Note] InnoDB: Uses event mutexes
2019-10-18T10:01:35.430627Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-10-18T10:01:35.430634Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-10-18T10:01:35.430642Z 0 [Note] InnoDB: Using Linux native AIO
2019-10-18T10:01:35.431024Z 0 [Note] InnoDB: Number of pools: 1
2019-10-18T10:01:35.431174Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-10-18T10:01:35.433296Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2019-10-18T10:01:35.584269Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-10-18T10:01:35.610249Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-10-18T10:01:35.624480Z 0 [ERROR] InnoDB: Header page consists of zero bytes in datafile: ./ibdata1, Space ID:0, Flags: 0. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2019-10-18T10:01:35.624518Z 0 [ERROR] InnoDB: Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.
2019-10-18T10:01:35.624667Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
---> 10:01:36 Waiting for MySQL to start ...
2019-10-18T10:01:36.225356Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2019-10-18T10:01:36.225372Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-10-18T10:01:36.225377Z 0 [ERROR] Failed to initialize builtin plugins.
2019-10-18T10:01:36.225380Z 0 [ERROR] Aborting
2019-10-18T10:01:36.225393Z 0 [Note] Binlog end
2019-10-18T10:01:36.225454Z 0 [Note] Shutting down plugin 'CSV'
2019-10-18T10:01:36.233760Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: Shutdown complete
我在 google 上搜索，他们建议执行以下操作
rm -rf /var/lib/mysql/ib_logfile*
但是正如您所知，在 docker 和 kubernetes 世界中，一旦 pod 无法启动，我们就无法登录到 pod 并删除这些文件。由于这个错误 MySQL pod 没有出现。
我正在使用 glusterfs，我删除了 PVC 并重新创建，但仍然出现同样的错误。
更新1：
在@Crou 的推荐下我做了以下改动
- 我下载了新的 "centos/mysql-57-centos7:5.7" 图片并重试仍然是同样的问题
我使用了 mysql:5.7 图像，现在它抛出了以下错误
我删除了 PVC 并重新创建了不同的大小，但仍然出现同样的错误。
[ERROR] --initialize specified but the data directory has files in it. Aborting.
[ERROR] Aborting
在我看来，您的图像已损坏，或者至少您尝试加载的数据已损坏。
即使您确实删除了文件 /var/lib/mysql/ib_logfile*，这也会在重新安排 pod 时再次中断。
您可以尝试在 pod 仍在启动时使用 kubectl exec -it <pod_name> bash 进入并快速进行更改。如果为时已晚并且它在 crashloopbackoff 的几秒钟内就失败了，您可以尝试在用于创建图像的 dockerfile 中修复此问题。如果您无权访问 dockerfile，您可以创建自己的 dockerfile 并基于它构建镜像。
这是一份 Dockerfile reference 的文档，可能会有帮助。
Dockerfile 包含如下内容：
FROM oraclelinux:7-slim
RUN rm -rf /var/lib/mysql/ib_logfile*
EXPOSE 3306 6606 6446 6447 33060
CMD [""]
请记住这只是一个简短的示例，可能无法自行处理。
我找到了这个 github mattlord/Docker-InnoDB-Cluster，也许会有帮助。
经过多次重试，我能够解决问题，以下是发现
我观察到当我第一次重新创建 PVC 并部署时 mysql 在这一行花费了太多时间
Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
在此期间 LiveProbe 耗尽并重新启动容器，导致出现错误。
我把initialDelaySeconds在livenessProbe下的时间增加到240，这次是4分钟后mysql
4 分钟后显示 Starting MySQL server with disabled networking 并开始进一步操作。
我的应用程序可以连接它，但我不知道这会有什么影响？
---> 16:18:24 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
---> 16:22:15 Starting MySQL server with disabled networking ...
---> 16:22:15 Waiting for MySQL to start ...
2019-10-18T16:22:15.752499Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-10-18T16:22:15.772646Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 69 ...
2019-10-18T16:22:15.849580Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-10-18T16:22:15.849613Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-10-18T16:22:15.849617Z 0 [Note] InnoDB: Uses event mutexes
2019-10-18T16:22:15.849621Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-10-18T16:22:15.849624Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-10-18T16:22:15.849629Z 0 [Note] InnoDB: Using Linux native AIO
2019-10-18T16:22:15.850236Z 0 [Note] InnoDB: Number of pools: 1
2019-10-18T16:22:15.850459Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-10-18T16:22:15.852211Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2019-10-18T16:22:15.978704Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-10-18T16:22:16.005183Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-10-18T16:22:16.208237Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-10-18T16:22:16.402086Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-10-18T16:22:16.414708Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
---> 16:22:17 Waiting for MySQL to start ...
---> 16:22:18 Waiting for MySQL to start ...
2019-10-18T16:22:18.601639Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-10-18T16:22:18.608922Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2019-10-18T16:22:18.608943Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2019-10-18T16:22:18.609240Z 0 [Note] InnoDB: Waiting for purge to start
2019-10-18T16:22:18.659387Z 0 [Note] InnoDB: 5.7.24 started; log sequence number 2592555
2019-10-18T16:22:18.659667Z 0 [Note] Plugin 'FEDERATED' is disabled.
2019-10-18T16:22:18.663687Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/data/ib_buffer_pool
2019-10-18T16:22:18.808093Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2019-10-18T16:22:18.808117Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2019-10-18T16:22:18.825225Z 0 [Warning] CA certificate ca.pem is self signed.
2019-10-18T16:22:18.828435Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2019-10-18T16:22:18.885079Z 0 [Note] InnoDB: Buffer pool(s) load completed at 191018 16:22:18
2019-10-18T16:22:18.948358Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948397Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948404Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948428Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948435Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948448Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.034170Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.034191Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.633088Z 0 [Note] Event Scheduler: Loaded 0 events
2019-10-18T16:22:19.633217Z 1 [Note] Event Scheduler: scheduler thread started with id 1
2019-10-18T16:22:19.633316Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: ready for connections.
Version: '5.7.24' socket: '/tmp/mysql.sock' port: 0 MySQL Community Server (GPL)
2019-10-18T16:22:19.634316Z 3 [Note] Your password has expired. To log in you must change it using a client that supports expired passwords.
---> 16:22:19 MySQL started successfully
---> 16:22:19 Setting password for MySQL root user ...
我正在 openshift (kubernetes) 上部署 MySQL 但出现以下错误
---> 10:01:34 Initializing database ...
---> 10:01:34 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
---> 10:01:35 Starting MySQL server with disabled networking ...
---> 10:01:35 Waiting for MySQL to start ...
2019-10-18T10:01:35.188509Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-10-18T10:01:35.188567Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
2019-10-18T10:01:35.325995Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 40 ...
2019-10-18T10:01:35.430574Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-10-18T10:01:35.430612Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-10-18T10:01:35.430621Z 0 [Note] InnoDB: Uses event mutexes
2019-10-18T10:01:35.430627Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-10-18T10:01:35.430634Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-10-18T10:01:35.430642Z 0 [Note] InnoDB: Using Linux native AIO
2019-10-18T10:01:35.431024Z 0 [Note] InnoDB: Number of pools: 1
2019-10-18T10:01:35.431174Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-10-18T10:01:35.433296Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2019-10-18T10:01:35.584269Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-10-18T10:01:35.610249Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-10-18T10:01:35.624480Z 0 [ERROR] InnoDB: Header page consists of zero bytes in datafile: ./ibdata1, Space ID:0, Flags: 0. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting-datadict.html for how to resolve the issue.
2019-10-18T10:01:35.624518Z 0 [ERROR] InnoDB: Corrupted page [page id: space=0, page number=0] of datafile './ibdata1' could not be found in the doublewrite buffer.
2019-10-18T10:01:35.624667Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption
---> 10:01:36 Waiting for MySQL to start ...
2019-10-18T10:01:36.225356Z 0 [ERROR] Plugin 'InnoDB' init function returned error.
2019-10-18T10:01:36.225372Z 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-10-18T10:01:36.225377Z 0 [ERROR] Failed to initialize builtin plugins.
2019-10-18T10:01:36.225380Z 0 [ERROR] Aborting
2019-10-18T10:01:36.225393Z 0 [Note] Binlog end
2019-10-18T10:01:36.225454Z 0 [Note] Shutting down plugin 'CSV'
2019-10-18T10:01:36.233760Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: Shutdown complete
我在 google 上搜索，他们建议执行以下操作
rm -rf /var/lib/mysql/ib_logfile*
但是正如您所知，在 docker 和 kubernetes 世界中，一旦 pod 无法启动，我们就无法登录到 pod 并删除这些文件。由于这个错误 MySQL pod 没有出现。
我正在使用 glusterfs，我删除了 PVC 并重新创建，但仍然出现同样的错误。
更新1： 在@Crou 的推荐下我做了以下改动
- 我下载了新的 "centos/mysql-57-centos7:5.7" 图片并重试仍然是同样的问题
我使用了 mysql:5.7 图像，现在它抛出了以下错误
我删除了 PVC 并重新创建了不同的大小，但仍然出现同样的错误。
[ERROR] --initialize specified but the data directory has files in it. Aborting.
[ERROR] Aborting
在我看来，您的图像已损坏，或者至少您尝试加载的数据已损坏。
即使您确实删除了文件 /var/lib/mysql/ib_logfile*，这也会在重新安排 pod 时再次中断。
您可以尝试在 pod 仍在启动时使用 kubectl exec -it <pod_name> bash 进入并快速进行更改。如果为时已晚并且它在 crashloopbackoff 的几秒钟内就失败了，您可以尝试在用于创建图像的 dockerfile 中修复此问题。如果您无权访问 dockerfile，您可以创建自己的 dockerfile 并基于它构建镜像。
这是一份 Dockerfile reference 的文档，可能会有帮助。
Dockerfile 包含如下内容：
FROM oraclelinux:7-slim
RUN rm -rf /var/lib/mysql/ib_logfile*
EXPOSE 3306 6606 6446 6447 33060
CMD [""]
请记住这只是一个简短的示例，可能无法自行处理。
我找到了这个 github mattlord/Docker-InnoDB-Cluster，也许会有帮助。
经过多次重试，我能够解决问题，以下是发现
我观察到当我第一次重新创建 PVC 并部署时 mysql 在这一行花费了太多时间
Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found在此期间 LiveProbe 耗尽并重新启动容器，导致出现错误。
我把
initialDelaySeconds在livenessProbe下的时间增加到240，这次是4分钟后mysql4 分钟后显示
Starting MySQL server with disabled networking并开始进一步操作。
我的应用程序可以连接它，但我不知道这会有什么影响？
---> 16:18:24 Running /opt/rh/rh-mysql57/root/usr/libexec/mysqld --initialize --datadir=/var/lib/mysql/data --ignore-db-dir=lost+found
---> 16:22:15 Starting MySQL server with disabled networking ...
---> 16:22:15 Waiting for MySQL to start ...
2019-10-18T16:22:15.752499Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-10-18T16:22:15.772646Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld (mysqld 5.7.24) starting as process 69 ...
2019-10-18T16:22:15.849580Z 0 [Note] InnoDB: PUNCH HOLE support available
2019-10-18T16:22:15.849613Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-10-18T16:22:15.849617Z 0 [Note] InnoDB: Uses event mutexes
2019-10-18T16:22:15.849621Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-10-18T16:22:15.849624Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2019-10-18T16:22:15.849629Z 0 [Note] InnoDB: Using Linux native AIO
2019-10-18T16:22:15.850236Z 0 [Note] InnoDB: Number of pools: 1
2019-10-18T16:22:15.850459Z 0 [Note] InnoDB: Using CPU crc32 instructions
2019-10-18T16:22:15.852211Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2019-10-18T16:22:15.978704Z 0 [Note] InnoDB: Completed initialization of buffer pool
2019-10-18T16:22:16.005183Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-10-18T16:22:16.208237Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-10-18T16:22:16.402086Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-10-18T16:22:16.414708Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
---> 16:22:17 Waiting for MySQL to start ...
---> 16:22:18 Waiting for MySQL to start ...
2019-10-18T16:22:18.601639Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-10-18T16:22:18.608922Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2019-10-18T16:22:18.608943Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2019-10-18T16:22:18.609240Z 0 [Note] InnoDB: Waiting for purge to start
2019-10-18T16:22:18.659387Z 0 [Note] InnoDB: 5.7.24 started; log sequence number 2592555
2019-10-18T16:22:18.659667Z 0 [Note] Plugin 'FEDERATED' is disabled.
2019-10-18T16:22:18.663687Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/data/ib_buffer_pool
2019-10-18T16:22:18.808093Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2019-10-18T16:22:18.808117Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2019-10-18T16:22:18.825225Z 0 [Warning] CA certificate ca.pem is self signed.
2019-10-18T16:22:18.828435Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2019-10-18T16:22:18.885079Z 0 [Note] InnoDB: Buffer pool(s) load completed at 191018 16:22:18
2019-10-18T16:22:18.948358Z 0 [Warning] 'user' entry 'root@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948397Z 0 [Warning] 'user' entry 'mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948404Z 0 [Warning] 'user' entry 'mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948428Z 0 [Warning] 'db' entry 'performance_schema mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948435Z 0 [Warning] 'db' entry 'sys mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:18.948448Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.034170Z 0 [Warning] 'tables_priv' entry 'user mysql.session@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.034191Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2019-10-18T16:22:19.633088Z 0 [Note] Event Scheduler: Loaded 0 events
2019-10-18T16:22:19.633217Z 1 [Note] Event Scheduler: scheduler thread started with id 1
2019-10-18T16:22:19.633316Z 0 [Note] /opt/rh/rh-mysql57/root/usr/libexec/mysqld: ready for connections.
Version: '5.7.24' socket: '/tmp/mysql.sock' port: 0 MySQL Community Server (GPL)
2019-10-18T16:22:19.634316Z 3 [Note] Your password has expired. To log in you must change it using a client that supports expired passwords.
---> 16:22:19 MySQL started successfully
---> 16:22:19 Setting password for MySQL root user ...