ZeroMQ - 模拟多个客户端到一台服务器的标准套接字

ZeroMQ - Emulating standard socket for multiple clients to one server

我希望利用 ZeroMQ 来处理同时进入认证服务器的大量请求的排队。

我处于 HPC 环境中,因此有大量计算节点,所有这些节点都 运行 与服务器证明相同的启动程序。此服务器代码在前端节点上运行,在成功证明客户端后,将释放一个密钥,以便客户端解密作业数据。

目前使用的是标准套接字。当客户端最初向服务器发送内容时,使用 sys/socket.h 中的 accept() 生成一个新套接字。这允许客户端和服务器在证明过程(校验和等)中相互发送多条消息,如果成功则最终返回密钥。

ZeroMQ 的问题是附加命令不是必需的,因此不会为该特定证明创建辅助套接字。所有消息都按照它们进入的顺序处理来自所有客户端的所有消息,从而导致多部分证明过程无法正常工作。我花了很长时间浏览指南并使用谷歌搜索来尝试找到类似的解决方案,但到目前为止还没有运气。

有没有一种方法可以利用 ZeroMQ 在此应用程序中提供与标准套接字相同的行为?

Q : Is there a way I can utilise ZeroMQ to give the same behaviour in this application as a standard socket?

ZeroMQ 是 signaling / messaging-platform for ,不是套接字。

鉴于您的意图对于 HPC 生态系统来说是值得的,一个解决方案假定/指示使用某种工具并且必须尽可能地改变它,以便它变得接近于类似于本机的行为,但对于其他工具,似乎不是典型的 HPC 级方法。

HPC 代码通常都经过精心设计,以实现计算成本效益(并祝福老板、首席财务官和 gov/mil-funding 今天被允许不设计 HPC 的所有人最终性能和硬件资源使用效率的代码 :o) ) - 在这里,如果有人为 ZeroMQ 实例化支付费用,那么实例化的这些非零成本和获得 "just"-a 似乎没有任何好处-socket-like 行为,在成本上,具有负面的性能收益,在智能、集群范围的 ZeroMQ 服务(无论是 N+1 还是 N+M 冗余、低延迟智能节点间集群)的某些未来收益中没有任何调整信号、密码学、廉价的安全驱动的白名单,或任何可能代表任何额外的 HPC 级项目的好处,这可能证明初始 ZeroMQ 实例化的成本是合理的)。


ZMQ_STREAM 的定义原型可能会提供一些工具,但请参考。以上

A socket of type ZMQ_STREAM is used to send and receive TCP data from a non-ØMQ peer, when using the tcp:// transport. A ZMQ_STREAM socket can act as client and/or server, sending and/or receiving TCP data asynchronously.

When receiving TCP data, a ZMQ_STREAM socket shall prepend a message part containing the identity of the originating peer to the message before passing it to the application. Messages received are fair-queued from among all connected peers.

When sending TCP data, a ZMQ_STREAM socket shall remove the first part of the message and use it to determine the identity of the peer the message shall be routed to, and unroutable messages shall cause an EHOSTUNREACH or EAGAIN error.

To open a connection to a server, use the zmq_connect call, and then fetch the socket identity using the ZMQ_IDENTITY zmq_getsockopt call.

To close a specific connection, send the identity frame followed by a zero-length message (see EXAMPLE section).

When a connection is made, a zero-length message will be received by the application. Similarly, when the peer disconnects (or the connection is lost), a zero-length message will be received by the application.

You must send one identity frame followed by one data frame. The ZMQ_SNDMORE flag is required for identity frames but is ignored on data frames.

ZMQ_STREAM 示例:

void *ctx = zmq_ctx_new ();                          assert (ctx     && "Context Instantiation Failed..." );
void *socket = zmq_socket (ctx, ZMQ_STREAM);         assert (socket  && "socket Instantiation Failed..." );
int rc = zmq_bind (socket, "tcp://*:8080");          assert (rc == 0 && "socket.bind() Failed..." );
uint8_t id [256];                                 /* Data structure to hold the ZMQ_STREAM ID */
size_t id_size = 256;
uint8_t raw [256];                                /* Data structure to hold the ZMQ_STREAM received data */
size_t raw_size = 256;
while (1) {
   id_size = zmq_recv (socket, id, 256, 0);          assert (id_size > 0 && "Get HTTP request; ID frame and then request; Failed..." )
   do {
        raw_size = zmq_recv (socket, raw, 256, 0);   assert (raw_size >= 0 && "socket.recv() Failed..." );
   } while (raw_size == 256);

   char http_response [] =                        /* Prepares the response */
            "HTTP/1.0 200 OK\r\n"
            "Content-Type: text/plain\r\n"
            "\r\n"
            "Hello, World!";
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Sends the ID frame followed by the response */
   zmq_send (socket, http_response, strlen (http_response), 0);
   zmq_send (socket, id, id_size, ZMQ_SNDMORE);   /* Closes the connection by sending the ID frame followed by a zero response */
   zmq_send (socket, 0, 0, 0);
}
zmq_close (socket);
zmq_ctx_destroy (ctx);

ZeroMQ zmq_getsockopt() 可以提供一个 POSIX/SOCKET 描述符,用于低级技巧

The ZMQ_FD option shall retrieve the file descriptor associated with the specified socket. The returned file descriptor can be used to integrate the socket into an existing event loop; the ØMQ library shall signal any pending events on the socket in an edge-triggered fashion by making the file descriptor become ready for reading.

The ability to read from the returned file descriptor does not necessarily indicate that messages are available to be read from, or can be written to, the underlying socket; applications must retrieve the actual event state with a subsequent retrieval of the ZMQ_EVENTS option.

The returned file descriptor is also used internally by the zmq_send and zmq_recv functions. As the descriptor is edge triggered, applications must update the state of ZMQ_EVENTS after each invocation of zmq_send or zmq_recv.

To be more explicit: after calling zmq_send the socket may become readable (and vice versa) without triggering a read event on the file descriptor.

The returned file descriptor is intended for use with a poll or similar system call only. Applications must never attempt to read or write data to it directly, neither should they try to close it.

Option value type: int on POSIX systems, SOCKET on Windows


有关 details on ZeroMQ tricks 的更多信息,您可能会喜欢阅读解决方案、性能基准、延迟削减细节和其他已在此处讨论的问题解决技巧。