Cloudera Docker 图片...一切都出错了

Question

我正在运行安装 16 GB Macbook pro 和 El Capitan OS。我使用

安装了 cloudera docker 图像

docker pull cloudera/quickstart:latest
docker run --privileged=true --hostname=quickstart.cloudera -t -i 9f3ab06c7554 /usr/bin/docker-quickstart

图像启动正常，我可以看到大多数服务正在启动

Started Hadoop historyserver:                              [  OK  ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out
Started Hadoop nodemanager:                                [  OK  ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out
Started Hadoop resourcemanager:                            [  OK  ]
starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out
Started HBase master daemon (hbase-master):                [  OK  ]
starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out
Started HBase rest daemon (hbase-rest):                    [  OK  ]
starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out
Started HBase thrift daemon (hbase-thrift):                [  OK  ]
Starting Hive Metastore (hive-metastore):                  [  OK  ]
Started Hive Server2 (hive-server2):                       [  OK  ]
Starting Sqoop Server:                                     [  OK  ]
Sqoop home directory: /usr/lib/sqoop2

也有一些失败

Failure to start Spark history-server (spark-history-server[FAILED]n value: 1
Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out
hbase-regionserver.
Starting hue:                                              [FAILED]

但是启动完成后，如果我尝试运行任何操作都会失败

例如尝试运行 spark-shell

[root@quickstart /]# spark-shell
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000b0000000, 357892096, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 357892096 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid3113.log

或尝试运行蜂巢 shell

[root@quickstart /]# hive
Unable to determine Hadoop version information.
'hadoop version' returned:
Hadoop 2.6.0-cdh5.5.0 Subversion http://github.com/cloudera/hadoop -r fd21232cef7b8c1f536965897ce20f50b83ee7b2 Compiled by jenkins on 2015-11-09T20:37Z Compiled with protoc 2.5.0 From source with checksum 98e07176d1787150a6a9c087627562c This command was run using /usr/jars/hadoop-common-2.6.0-cdh5.5.0.jar
[root@quickstart /]#

我的问题是我该怎么做才能运行 spark-shell 和 hive shell 成功？

Answer 1

因为你是运行ning Docker on a Mac, Docker 运行s under VirtualBox, not directly with the Mac的记忆。（同样的事情也会发生在 Windows 中）。

您可能不会在 Linux 主机上遇到这些错误，因为 Docker 没有在那里虚拟化。

Cloudera quickstart 虚拟机建议运行所有服务使用 8Gb 内存，我认为 docker 虚拟机只有 512Mb。

解决方案是停止 docker-machine 实例，打开 VirtualBox，并将 "default" VM 的内存大小增加到必要的数量。

Cloudera Docker 图片...一切都出错了

Cloudera Docker image ... everything errors out

hadoop

cloudera

docker