Apache Flume 在我 windows 上的第一次安装中没有卡在启动时
Apache Flume not stuck at start in my first installation on windows
我已经下载了最新版本的 Apache Flume "apache-flume-1.8.0-bin" 并更改了如下配置,因为它是第一个 start.I 我正在 Windows 中设置 apache flume ] 8.1
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'
na.sources = nas1
na.channels = nac1
na.sinks = nal1
# For each one of the sources, the type is defined
na.sources.nas1.type = netcat
na.sources.nas1.bind = localhost
na.sources.nas1.port = 44444
# The channel can be defined as follows.
na.sources.nas1.channels = nac1
# Each sink's type must be defined
na.sinks.nal1.type = logger
#Specify the channel the sink should use
na.sinks.nal1.channel = nac1
# Each channel's type is defined.
na.channels.nac1.type = memory
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
na.channels.nac1.capacity = 100
保存此配置后,我尝试使用以下命令启动我的 flume 代理
flume-ng agent -n na -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
之后它卡在 flume 代理的启动中,如下所示,
C:\apache-flume-1.8.0-bin\bin>flume-ng agent -n na -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
C:\apache-flume-1.8.0-bin\bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File C:\apache-flume-1.8.0-bin\bin\flume-ng.ps1 agent -n na -f C:/apache-fl
ume-1.8.0-bin/conf/flume-conf-log.properties
WARN: Config directory not set. Defaulting to C:\apache-flume-1.8.0-bin\conf
Sourcing environment configuration script C:\apache-flume-1.8.0-bin\conf\flume-env.ps1
Including Hadoop libraries found in (C:\hadoop-2.7.6) for DFS access
Including HBase libraries found via (C:\hbase-2.1.0) for HBase access
WARN: HIVE_HOME not found
Running FLUME agent :
class: org.apache.flume.node.Application
arguments: -n na -f "C:\apache-flume-1.8.0-bin\conf\flume-conf-log.properties"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/apache-flume-1.8.0-bin/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/hadoop-2.7.6/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
附上错误截图供参考。我需要处理服务器日志以捕获错误消息,我需要使用 Flume.
将日志流式传输到 HDFS
Flume 启动时卡住 - 屏幕截图
通过添加 Java 路径更改 "flume-env.sh" 后,它可以使用以下命令工作。
flume-ng agent -n na --conf ./conf/ -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
我也附上了工作截图。
Working Screenshot
我已经下载了最新版本的 Apache Flume "apache-flume-1.8.0-bin" 并更改了如下配置,因为它是第一个 start.I 我正在 Windows 中设置 apache flume ] 8.1
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# The configuration file needs to define the sources,
# the channels and the sinks.
# Sources, channels and sinks are defined per agent,
# in this case called 'agent'
na.sources = nas1
na.channels = nac1
na.sinks = nal1
# For each one of the sources, the type is defined
na.sources.nas1.type = netcat
na.sources.nas1.bind = localhost
na.sources.nas1.port = 44444
# The channel can be defined as follows.
na.sources.nas1.channels = nac1
# Each sink's type must be defined
na.sinks.nal1.type = logger
#Specify the channel the sink should use
na.sinks.nal1.channel = nac1
# Each channel's type is defined.
na.channels.nac1.type = memory
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
na.channels.nac1.capacity = 100
保存此配置后,我尝试使用以下命令启动我的 flume 代理
flume-ng agent -n na -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
之后它卡在 flume 代理的启动中,如下所示,
C:\apache-flume-1.8.0-bin\bin>flume-ng agent -n na -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
C:\apache-flume-1.8.0-bin\bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File C:\apache-flume-1.8.0-bin\bin\flume-ng.ps1 agent -n na -f C:/apache-fl
ume-1.8.0-bin/conf/flume-conf-log.properties
WARN: Config directory not set. Defaulting to C:\apache-flume-1.8.0-bin\conf
Sourcing environment configuration script C:\apache-flume-1.8.0-bin\conf\flume-env.ps1
Including Hadoop libraries found in (C:\hadoop-2.7.6) for DFS access
Including HBase libraries found via (C:\hbase-2.1.0) for HBase access
WARN: HIVE_HOME not found
Running FLUME agent :
class: org.apache.flume.node.Application
arguments: -n na -f "C:\apache-flume-1.8.0-bin\conf\flume-conf-log.properties"
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/apache-flume-1.8.0-bin/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/hadoop-2.7.6/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
附上错误截图供参考。我需要处理服务器日志以捕获错误消息,我需要使用 Flume.
将日志流式传输到 HDFSFlume 启动时卡住 - 屏幕截图
通过添加 Java 路径更改 "flume-env.sh" 后,它可以使用以下命令工作。
flume-ng agent -n na --conf ./conf/ -f C:/apache-flume-1.8.0-bin/conf/flume-conf-log.properties
我也附上了工作截图。 Working Screenshot