libcurl C++：多路复用时打印响应的问题

Question

目标：

修改 HTTP/2 multiplexing 的 libcurl 示例 can be found here 以在负载响应到达某个缓冲区时将其保存，而不是像当前上述示例那样将其写入文件做。缓冲区内的有效负载随后可用于打印、搜索字符串等。

预期输出：

每当 callback 函数检测到一个有效负载已被传送时，程序应该打印出每个收到的有效负载响应给 stdout。

实际输出：

有时程序会按预期进行少量传输（请参阅下面 main() 中的代码行 int num_transfers = 3）。如果传输次数增加到 8 或 10，有时程序无法正常运行，程序仍会将输出打印到 stdout，但默认格式是 libcurl，如果代码中没有包含 CURL_WRITEFUNCTION/CURL_WRITEDATA，这可能表明 callback 函数没有接收到任何内容？同样在这种情况下，将打印错误数量的响应。

在 main() 内的主 do...while 循环中，我将 chunk.memory 和 chunk.size 打印出来后设置为 0。如果不这样做，每次收到新的响应时，这些响应都会继续增长。但是，我不确定这是否是正确的方法。

当前尝试：

使用 libcurl 示例 that can be found here，我尝试模仿将输出写入回调函数的功能，如下所示（而不是将每个响应负载写入文件）。

#include <iostream>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

/* somewhat unix-specific */
#include <sys/time.h>
#include <unistd.h>

/* curl stuff */
#include <curl/curl.h>
#include <curl/mprintf.h>

#ifndef CURLPIPE_MULTIPLEX
#define CURLPIPE_MULTIPLEX 0
#endif

struct CURLMsg *msg;

struct transfer {
    CURL *easy;
    unsigned int num;
    FILE *out;
};

struct MemoryStruct {
    char *memory;
    size_t size;
};

struct MemoryStruct chunk;

#define NUM_HANDLES 1000

static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
    size_t realsize = size * nmemb;
    struct MemoryStruct *mem = (struct MemoryStruct *)userp;

    char *ptr = (char*)realloc(mem->memory, mem->size + realsize + 1);
    if(!ptr) {
        /* out of memory! */
        std::cout << "not enough memory (realloc returned NULL)" << std::endl;
        return 0;
    }

    mem->memory = ptr;
    memcpy(&(mem->memory[mem->size]), contents, realsize);
    mem->size += realsize;
    mem->memory[mem->size] = 0;

    return realsize;
}

static void setup(struct transfer *t, int num)
{
    CURL *hnd;

    hnd = t->easy = curl_easy_init();

    curl_easy_setopt(hnd, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
    curl_easy_setopt(hnd, CURLOPT_WRITEDATA, (void *)&chunk);

    /* set the same URL */
    curl_easy_setopt(hnd, CURLOPT_URL, "https://someurl.xyz");

    /* HTTP/2 please */
    curl_easy_setopt(hnd, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_2_0);

    /* we use a self-signed test server, skip verification during debugging */
    curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYPEER, 0L);
    curl_easy_setopt(hnd, CURLOPT_SSL_VERIFYHOST, 0L);

    #if (CURLPIPE_MULTIPLEX > 0)
    /* wait for pipe connection to confirm */
    curl_easy_setopt(hnd, CURLOPT_PIPEWAIT, 1L);
    #endif
}

int main() {
    struct transfer trans[NUM_HANDLES];
    CURLM *multi_handle;
    int i;
    int still_running = 0; /* keep number of running handles */
    int num_transfers = 3;

    chunk.memory = (char*)malloc(1);
    chunk.size = 0;

    /* init a multi stack */
    multi_handle = curl_multi_init();

    for(i = 0; i < num_transfers; i++) {
        setup(&trans[i], i);

        /* add the individual transfer */
        curl_multi_add_handle(multi_handle, trans[i].easy);
    }

    curl_multi_setopt(multi_handle, CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);

    // Main loop
    do {
        CURLMcode mc = curl_multi_perform(multi_handle, &still_running);

        if(still_running) {
            /* wait for activity, timeout or "nothing" */
            mc = curl_multi_poll(multi_handle, NULL, 0, 1000, NULL);
        }

        if(mc) {
            break;
        }

        // Get response
        do {
            int queued;
            msg = curl_multi_info_read(multi_handle, &queued);
            if ((msg) && (msg->msg == CURLMSG_DONE) && (msg->data.result == CURLE_OK)) {

                    // Print the response payload
                    std::cout << "size: " << chunk.size << std::endl;
                    std::cout << chunk.memory << std::endl;
                    chunk.memory = 0;
                    chunk.size = 0;

            }
            
        } while (msg);

    } while (still_running);

    for(i = 0; i < num_transfers; i++) {
        curl_multi_remove_handle(multi_handle, trans[i].easy);
        curl_easy_cleanup(trans[i].easy);
    }

    free(chunk.memory);
    curl_multi_cleanup(multi_handle);
        
    return 0;
}

总结题：

Q1. 如何修改上述程序以将接收到的负载响应正确地异步保存到 struct 或 buffer 中，以便它可以可用于打印到 stdout 或搜索 strings?

等功能

Answer 1

首先，您应该将 transfer 结构与访问输出的方式相关联：

struct transfer {
    CURL *easy;
    unsigned int num;
    std::string contents;
};

并将CURLOPT_WRITEDATA与指针相关联：

curl_easy_setopt(hnd, CURLOPT_WRITEDATA, (void *)t);

然后，WriteMemoryCallback变成：

static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp) {
    transfer *t = (transfer *)userp;
    size_t realsize = size * nmemb;    
    t->contents.append((const char *)contents, realsize);
    return realsize;
}

之后就可以在trans[i].contents变量中找到内容了。

libcurl C++：多路复用时打印响应的问题

libcurl C++: issue with printing responses when multiplexing

c++

libcurl

http2

multiplexing