是否可以在 linux 中打开具有大量元素的消息队列？

Question

根据documentation，可以使用/proc/sys/fs/mqueue/msg_max来增加队列中消息的限制。文档还说，该限制不应超过 HARD_MSGMAX，自 Linux 3.5.

以来为 65,536

但是，即使有 500 个元素，尝试打开队列也会失败 EMFILE:

#include <stdio.h>
#include <mqueue.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <stddef.h>
#include <sys/stat.h>

int main(int argc, char** argv)
{
    int message_size = argc > 1 ? atoi(argv[1]) : 16;
    int queue_size = argc > 2 ? atoi(argv[2]) : 10000;

    printf("Trying to open queue with msgsize: %d, and maxmsg: %d\n", message_size, queue_size);

    struct mq_attr initial_attributes = (struct mq_attr){
            .mq_msgsize = message_size,
            .mq_maxmsg = queue_size
    }; 

    int open_flags = O_RDWR | O_CREAT | O_EXCL;
    int permissions = S_IWUSR | S_IRUSR;

    const char* name = "/message_queue_name;";
    mqd_t queue = mq_open(name, open_flags, permissions, &initial_attributes);

    if(queue == -1)
    {
        printf("Cannot open message queue\n");
        printf("Errno: %d [%s]\n", errno, strerror(errno));
        return 1;
    }
    else
    {
        printf("Queue has been opened successfully. Closing...\n");
        mq_close(queue);
        mq_unlink(name);
    }

    return 0;
}

测试：

$ cat /proc/sys/fs/mqueue/msg_max
65536

$ ./main 16 300
Trying to open queue with msgsize: 16, and maxmsg: 300
Queue has been opened successfully. Closing...

$ ./main 16 500
Trying to open queue with msgsize: 16, and maxmsg: 500
Cannot open message queue
Errno: 24 [Too many open files]

根据documentation，EMFILE错误代码意味着，该进程已经打开了最大数量的文件和消息队列，尽管该程序没有打开任何其他文件。

所以，我的问题是：如何打开包含大量元素的消息队列？

UPD [1]

这是我的系统限制：

$ ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-m: resident set size (kbytes)      unlimited
-u: processes                       62820
-n: file descriptors                1024
-l: locked-in-memory size (kbytes)  unlimited
-v: address space (kbytes)          unlimited
-x: file locks                      unlimited
-i: pending signals                 62820
-q: bytes in POSIX msg queues       819200
-e: max nice                        30
-r: max rt priority                 99
-N 15:                              unlimited

Answer 1

看起来您运行没有文件描述符。只需检查每个进程的最大文件描述符数 (ulimit)，如果它很低，那么您可以使用

来增加它以满足您的需求

ulimit -n new_maximim_number

Answer 2

你遇到过RLIMIT_MSGQUEUE limit, see mq_overview(7):

Resource limit

The RLIMIT_MSGQUEUE resource limit, which places a limit on the amount of space that can be consumed by all of the message queues belonging to a process's real user ID, is described in getrlimit(2).

在调用 mq_open() 之前增加它，例如像这样：

...
#include <sys/resource.h>

int main(int argc, char** argv)
{
    struct rlimit rlim;
    rlim.rlim_cur = RLIM_INFINITY;
    rlim.rlim_max = RLIM_INFINITY;

    if (setrlimit(RLIMIT_MSGQUEUE, &rlim) == -1) {
        perror("setrlimit");
        return 1;
    }

    ...

你需要 root 权限（或者 CAP_SYS_RESOURCE 能力，我猜）。

Linux kernel really returns EMFILE 对于这种情况，检查它 here:

if (u->mq_bytes + mq_bytes < u->mq_bytes ||
    u->mq_bytes + mq_bytes > rlimit(RLIMIT_MSGQUEUE)) {
        spin_unlock(&mq_lock);
        /* mqueue_evict_inode() releases info->messages */
        ret = -EMFILE;
        goto out_inode;
}

这里EMFILE的原因可能是它是POSIX中那些specified最接近的错误代码； EMFILE 是唯一反映达到 每个进程 限制的错误代码。

POSIX 没有为 RLIMIT_MSGQUEUE 指定更精确的错误代码，因为它是 Linux 特定的。

是否可以在 linux 中打开具有大量元素的消息队列？

Is it possible to open message queue in linux with huge number of elements?

c

linux

message-queue

UPD [1]