apache thrift 监控一个 hdfs 文件

apache thrift monitor a hdfs file

我计划使用 Apache Thrift 来监视我本地更改的任何更改并将该数据推送到客户端(更改后的数据)。

当我查看 thrift 文档时,我看到了多个传输层,但不明白我应该使用哪个传输层

http://thrift-tutorial.readthedocs.io/en/latest/thrift-stack.html

Tranport Layer
The transport layer is responsible for reading from and writing to the wire. Thrift supports the following:

TSocket - Uses blocking socket I/O for transport.
TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.

有两种传输方式:

  • 端点传输
  • 分层传输

前者是您需要(其中之一)向线路写入和读取数据的那些。例如,这可能是一个 TSocket。

后者是加法使用的,有时甚至是合并使用的。例如,TFramedTransport 为数据添加了一个特殊层,以提高内存分配和 I/O 的效率。 zlib 传输可用于压缩数据。

一个例子可以是:

+------------------------------------+
|  Application code                  |
+------------------------------------+
|  TBinaryProtocol                   |
+------------------------------------+
|  TZLibTransport                    |
+------------------------------------+
|  TFramedTransport                  |
+------------------------------------+
|  TSocket transport                 |
+------------------------------------+

另一个完全没有分层传输的:

+------------------------------------+
|  Application code                  |
+------------------------------------+
|  TBinaryProtocol                   |
+------------------------------------+
|  TSocket transport                 |
+------------------------------------+

PS:您链接的不是官方文档,这是一些第三方人员设置的,Apache Thrift 项目对该站点的质量没有影响。

强烈推荐the forthcoming Manning book from Randy Abernethy。他是一位 Thrift Committer,这本书提供了宝贵的见解。不,我没有得到任何推荐。