flume 未使用 SocialAgent 获取 Facebook 数据

Question

我正在尝试使用 flume SocialAgent 检索 Facebook 数据。我已经使用 TwitterAgent 成功检索了推特数据。

但是对于 Facebook，我在 hdfs 中什么也没有。

我的终端卡在这个阶段。当我终止这个过程时，我得到了这个

我的 HDFS facebook 文件夹是 empty.I 我正在使用以下 flume.conf

SocialAgent.sources = FacebookHttpSource Twitter
SocialAgent.channels = FBmemoryChannel MemChannel
SocialAgent.sinks = fbHDFS HDFS

# For each one of the sources, the type is defined
SocialAgent.sources.FacebookHttpSource.type = org.apache.flume.source.http.HTTPSource
SocialAgent.sources.FacebookHttpSource.port = 51400
SocialAgent.sources.FacebookHttpSource.interceptors = Ts
SocialAgent.sources.FacebookHttpSource.interceptors.Ts.type = org.apache.flume.interceptor.TimestampInterceptor$Builder

# The channel can be defined as follows.
SocialAgent.sources.FacebookHttpSource.channels = FBmemoryChannel

# Each sink's type must be defined
#Specify the channel the sink should use
SocialAgent.sinks.fbHDFS.channel = FBmemoryChannel
SocialAgent.sinks.fbHDFS.type = hdfs
SocialAgent.sinks.fbHDFS.hdfs.path = hdfs://localhost:9000/user/flume/facebook/%Y/%m/%d/%H/
SocialAgent.sinks.fbHDFS.hdfs.fileType = DataStream
SocialAgent.sinks.fbHDFS.hdfs.writeFormat = Text
SocialAgent.sinks.fbHDFS.hdfs.batchSize = 1000
SocialAgent.sinks.fbHDFS.hdfs.rollSize = 0
SocialAgent.sinks.fbHDFS.hdfs.rollCount = 10000

# Each channel's type is defined.
SocialAgent.channels.FBmemoryChannel.type = memory

# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
SocialAgent.channels.FBmemoryChannel.capacity = 10000
SocialAgent.channels.FBmemoryChannel.transactionCapacity = 100

是否有其他 flume 代理来获取 FaceBook 数据。

Answer 1

org.apache.flume.source.http.HTTPSource 在您的配置中不会向 Facebook 发送任何请求以获取数据。它只是监听一个端口（在你的情况下是 51400）并等待任何请求 (https://flume.apache.org/FlumeUserGuide.html#http-source)

据我所知还没有任何 Flume FBClient。