C++ Boost ASIO async_send_to 内存泄漏
C++ Boost ASIO async_send_to memory leak
我目前正在开发 UDP 套接字客户端。我目前注意到内存泄漏,我已经尝试了几种方法以期消除它,但它仍然存在。在我看来,我有一个 char*
已经 malloc
了。然后我调用下面的函数来发送数据:
void Send(const char* data, const int size) {
Socket.async_send_to(boost::asio::buffer(data, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
如果我运行这段代码,它总是会泄漏内存。但是,如果我注释掉 async_send_to
调用,内存将保持一致。
我已经尝试了几种变体(见下文),但它们似乎都只会加速内存泄漏。
请注意,传递给 Send 的 char*
有可能在调用完成之前得到 free
。但是,在我的变体中,我已经采取了预防措施来处理这个问题。
变体 1:
void Send(const char* data, const int size) {
char* buf = (char*)malloc(size);
memcpy(buf, data, size);
Socket.async_send_to(boost::asio::buffer(buf, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error, buf));
}
void HandleSendTo(const boost::system::error_code& ec, const char* buf) {
free(buf);
}
变体 2:
class MulticastSender {
char* Buffer;
public:
void Send(const char* data, const int size) {
Buffer = (char*)malloc(size);
memcpy(Buffer, data, size);
Socket.async_send_to(boost::asio::buffer(Buffer, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
void HandleSendTo(const boost::system::error_code& ec) {
free(Buffer);
}
}
然而,这两种变体似乎只会加速内存泄漏。我也试过删除 async_send_to
并只调用 boost::asio::buffer(data, size)
,但正如其他问题中所解释的那样,缓冲区不拥有内存,因此由用户安全地管理它。关于可能导致此问题的原因以及如何解决它的任何想法?
编辑 1:
正如评论中所建议的那样,我已经预先分配了一个缓冲区(用于测试目的)并且我从未取消分配它,但是,内存泄漏仍然存在。
class MulticastSender {
char* Buffer;
const int MaxSize = 16384;
public:
MulticastSender() {
Buffer = (char*)malloc(MaxSize);
}
void Send(const char* data, const int size) {
memcpy(Buffer, data, size);
Socket.async_send_to(boost::asio::buffer(Buffer, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
void HandleSendTo(const boost::system::error_code& ec) {
}
}
编辑 2:
正如此处所要求的那样,是问题的 MCVE。在做这个的过程中,我还观察到一个有趣的行为,我将在下面解释。
#include <string>
#include <iostream>
#include <functional>
#include <thread>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
class MulticastSender {
private:
boost::asio::io_service IOService;
const unsigned short Port;
const boost::asio::ip::address Address;
boost::asio::ip::udp::endpoint Endpoint;
boost::asio::ip::udp::socket Socket;
boost::asio::streambuf Buffer;
void HandleSendTo(const boost::system::error_code& ec) {
if(ec) {
std::cerr << "Error writing data to socket: " << ec.message() << '\n';
}
}
void Run() {
IOService.run();
}
public:
MulticastSender(const std::string& address,
const std::string& multicastaddress,
const unsigned short port) : Address(boost::asio::ip::address::from_string(address)),
Port(port),
Endpoint(Address, port),
Socket(IOService, Endpoint.protocol()) {
std::thread runthread(&MulticastSender::Run, this);
runthread.detach();
}
void Send(const char* data, const int size) {
std::ostreambuf_iterator<char> out(&Buffer);
std::copy(data, data + size, out);
Socket.async_send_to(Buffer.data(), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
};
const int SIZE = 8192;
int main() {
MulticastSender sender("127.0.0.1", "239.255.0.0", 30000);
while(true) {
char* data = (char*)malloc(SIZE);
std::memset(data, 0, SIZE);
sender.Send(data, SIZE);
usleep(250);
free(data);
}
}
以上代码仍然会产生内存泄漏。我应该提到我在 CentOS 6.6 with kernel Linux dev 2.6.32-504.el6.x86_64
和 运行 宁 Boost 1.55.0
上 运行宁这个。我只是通过观察 top
中的过程来观察这一点。
但是,如果我只是将 MulticastSender
的创建移动到 while
循环中,我就不会再观察到内存泄漏。不过我担心应用程序的速度,所以这不是一个有效的选项。
class 变体看起来更好,您可以使用 boost::asio::streambuf
作为网络 io 的缓冲区(它不会泄漏,也不需要太多维护)。
// The send function
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket.async_send_to(buffer_, endpoint,
std::bind( &multicast_sender,
this, std::placeholders::_1 ));
}
将套接字和端点移动到 class 内是个好主意。您还应该记住,当您的对象超出范围时,异步操作可以完成。我建议使用 enable_shared_from_this
(提升或标准口味)并将 shared_from_this()
而不是 this
传递给绑定函数。
整个解决方案如下所示:
#include <boost/asio.hpp>
class multicast_sender :
public std::enable_shared_from_this<multicast_sender> {
using boost::asio::ip::udp;
udp::socket socket_;
udp::endpoint endpoint_;
boost::asio::streambuf buffer_;
public:
multicast_sender(boost::asio::io_service& io_service, short port,
udp::endpoint const& remote) :
socket_(io_service, udp::endpoint(udp::v4(), port)),
endpoint_(remote)
{
}
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket_.async_send_to(buffer_, endpoint_,
std::bind( &multicast_sender,
shared_from_this(), std::placeholders::_1 ));
}
void
handle_send(boost::system::error_code const& ec)
{
}
};
编辑
只要您不必在写入处理程序中执行任何操作,就可以使用 lambda(需要 C++11)作为完成回调
// The send function
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket.async_send_to(buffer_, endpoint,
[](boost::system::error_code const& ec){
std::cerr << "Error sending :" << ec.message() << "\n";
});
}
内存没有泄漏,因为分配的内存仍有句柄。但是,将会持续增长,因为:
io_service
不是运行因为run()
is returning as there is no work. This results in completion handlers being allocated, queued into the io_service
, but neither executed nor freed. Additionally, any cleanup that is expected to occur within the completion handler is not occurring. It is worth noting that during the destruction of the io_service
, completion handlers will be destroyed and not invoked; hence, one cannot depend on only performing cleanup within the execution of the completion handler. For more details as to when io_service::run()
blocks or unblocks, consider reading this问题。
streambuf
的输入序列永远不会被消耗。主循环中的每次迭代都会附加到 streambuf
,然后发送先前的消息内容和新附加的数据。有关 streambuf
. 整体用法的更多详细信息,请参阅 答案
其他几点:
- 该程序未能满足
async_send_to()
, where ownership of the underlying buffer memory is retained by the caller, who must guarantee that it remains valid until the handler is called. In this case, when copying into the streambuf
via the ostreambuf_iterator
, the streambuf
's input sequence is modified and invalidates the buffer returned from streambuf.data()
的要求。
- 在关闭期间,需要针对 运行 和
io_service
的线程进行某种形式的同步。否则,可能会调用未定义的行为。
要解决这些问题,请考虑:
- 使用
boost::asio::io_service::work
确保 io_service
对象的 run()
在没有剩余工作时不会退出。
- 通过
std::shared_ptr
或另一个 class 将内存所有权传递给完成处理程序,后者将通过 resource acquisition is initialization (RAII) 习惯用法管理内存。这将允许进行适当的清理并满足 async_send_to()
. 的缓冲区有效性要求
- 不在工作线程上分离和加入。
这里是一个完整的例子,基于原来的 demonstrates 这些变化:
#include <string>
#include <iostream>
#include <thread>
#include <boost/asio.hpp>
class multicast_sender
{
public:
multicast_sender(
const std::string& address,
const std::string& multicast_address,
const unsigned short multicast_port)
: work_(io_service_),
multicast_endpoint_(
boost::asio::ip::address::from_string(multicast_address),
multicast_port),
socket_(io_service_, boost::asio::ip::udp::endpoint(
boost::asio::ip::address::from_string(address),
0 /* any port */))
{
// Start running the io_service. The work_ object will keep
// io_service::run() from returning even if there is no real work
// queued into the io_service.
auto self = this;
work_thread_ = std::thread([self]()
{
self->io_service_.run();
});
}
~multicast_sender()
{
// Explicitly stop the io_service. Queued handlers will not be ran.
io_service_.stop();
// Synchronize with the work thread.
work_thread_.join();
}
void send(const char* data, const int size)
{
// Caller may delete before the async operation finishes, so copy the
// buffer and associate it to the completion handler's lifetime. Note
// that the completion may not run in the event the io_servie is
// destroyed, but the the completion handler will be, so managing via
// a RAII object (std::shared_ptr) is ideal.
auto buffer = std::make_shared<std::string>(data, size);
socket_.async_send_to(boost::asio::buffer(*buffer), multicast_endpoint_,
[buffer](
const boost::system::error_code& error,
std::size_t bytes_transferred)
{
std::cout << "Wrote " << bytes_transferred << " bytes with " <<
error.message() << std::endl;
});
}
private:
boost::asio::io_service io_service_;
boost::asio::io_service::work work_;
boost::asio::ip::udp::endpoint multicast_endpoint_;
boost::asio::ip::udp::socket socket_;
std::thread work_thread_;
};
const int SIZE = 8192;
int main()
{
multicast_sender sender("127.0.0.1", "239.255.0.0", 30000);
char* data = (char*) malloc(SIZE);
std::memset(data, 0, SIZE);
sender.send(data, SIZE);
free(data);
// Give some time to allow for the async operation to complete
// before shutting down the io_service.
std::this_thread::sleep_for(std::chrono::seconds(2));
}
输出:
Wrote 8192 bytes with Success
我目前正在开发 UDP 套接字客户端。我目前注意到内存泄漏,我已经尝试了几种方法以期消除它,但它仍然存在。在我看来,我有一个 char*
已经 malloc
了。然后我调用下面的函数来发送数据:
void Send(const char* data, const int size) {
Socket.async_send_to(boost::asio::buffer(data, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
如果我运行这段代码,它总是会泄漏内存。但是,如果我注释掉 async_send_to
调用,内存将保持一致。
我已经尝试了几种变体(见下文),但它们似乎都只会加速内存泄漏。
请注意,传递给 Send 的 char*
有可能在调用完成之前得到 free
。但是,在我的变体中,我已经采取了预防措施来处理这个问题。
变体 1:
void Send(const char* data, const int size) {
char* buf = (char*)malloc(size);
memcpy(buf, data, size);
Socket.async_send_to(boost::asio::buffer(buf, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error, buf));
}
void HandleSendTo(const boost::system::error_code& ec, const char* buf) {
free(buf);
}
变体 2:
class MulticastSender {
char* Buffer;
public:
void Send(const char* data, const int size) {
Buffer = (char*)malloc(size);
memcpy(Buffer, data, size);
Socket.async_send_to(boost::asio::buffer(Buffer, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
void HandleSendTo(const boost::system::error_code& ec) {
free(Buffer);
}
}
然而,这两种变体似乎只会加速内存泄漏。我也试过删除 async_send_to
并只调用 boost::asio::buffer(data, size)
,但正如其他问题中所解释的那样,缓冲区不拥有内存,因此由用户安全地管理它。关于可能导致此问题的原因以及如何解决它的任何想法?
编辑 1:
正如评论中所建议的那样,我已经预先分配了一个缓冲区(用于测试目的)并且我从未取消分配它,但是,内存泄漏仍然存在。
class MulticastSender {
char* Buffer;
const int MaxSize = 16384;
public:
MulticastSender() {
Buffer = (char*)malloc(MaxSize);
}
void Send(const char* data, const int size) {
memcpy(Buffer, data, size);
Socket.async_send_to(boost::asio::buffer(Buffer, size), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
void HandleSendTo(const boost::system::error_code& ec) {
}
}
编辑 2: 正如此处所要求的那样,是问题的 MCVE。在做这个的过程中,我还观察到一个有趣的行为,我将在下面解释。
#include <string>
#include <iostream>
#include <functional>
#include <thread>
#include <boost/asio.hpp>
#include <boost/bind.hpp>
class MulticastSender {
private:
boost::asio::io_service IOService;
const unsigned short Port;
const boost::asio::ip::address Address;
boost::asio::ip::udp::endpoint Endpoint;
boost::asio::ip::udp::socket Socket;
boost::asio::streambuf Buffer;
void HandleSendTo(const boost::system::error_code& ec) {
if(ec) {
std::cerr << "Error writing data to socket: " << ec.message() << '\n';
}
}
void Run() {
IOService.run();
}
public:
MulticastSender(const std::string& address,
const std::string& multicastaddress,
const unsigned short port) : Address(boost::asio::ip::address::from_string(address)),
Port(port),
Endpoint(Address, port),
Socket(IOService, Endpoint.protocol()) {
std::thread runthread(&MulticastSender::Run, this);
runthread.detach();
}
void Send(const char* data, const int size) {
std::ostreambuf_iterator<char> out(&Buffer);
std::copy(data, data + size, out);
Socket.async_send_to(Buffer.data(), Endpoint, boost::bind(&MulticastSender::HandleSendTo, this, boost::asio::placeholders::error));
}
};
const int SIZE = 8192;
int main() {
MulticastSender sender("127.0.0.1", "239.255.0.0", 30000);
while(true) {
char* data = (char*)malloc(SIZE);
std::memset(data, 0, SIZE);
sender.Send(data, SIZE);
usleep(250);
free(data);
}
}
以上代码仍然会产生内存泄漏。我应该提到我在 CentOS 6.6 with kernel Linux dev 2.6.32-504.el6.x86_64
和 运行 宁 Boost 1.55.0
上 运行宁这个。我只是通过观察 top
中的过程来观察这一点。
但是,如果我只是将 MulticastSender
的创建移动到 while
循环中,我就不会再观察到内存泄漏。不过我担心应用程序的速度,所以这不是一个有效的选项。
class 变体看起来更好,您可以使用 boost::asio::streambuf
作为网络 io 的缓冲区(它不会泄漏,也不需要太多维护)。
// The send function
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket.async_send_to(buffer_, endpoint,
std::bind( &multicast_sender,
this, std::placeholders::_1 ));
}
将套接字和端点移动到 class 内是个好主意。您还应该记住,当您的对象超出范围时,异步操作可以完成。我建议使用 enable_shared_from_this
(提升或标准口味)并将 shared_from_this()
而不是 this
传递给绑定函数。
整个解决方案如下所示:
#include <boost/asio.hpp>
class multicast_sender :
public std::enable_shared_from_this<multicast_sender> {
using boost::asio::ip::udp;
udp::socket socket_;
udp::endpoint endpoint_;
boost::asio::streambuf buffer_;
public:
multicast_sender(boost::asio::io_service& io_service, short port,
udp::endpoint const& remote) :
socket_(io_service, udp::endpoint(udp::v4(), port)),
endpoint_(remote)
{
}
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket_.async_send_to(buffer_, endpoint_,
std::bind( &multicast_sender,
shared_from_this(), std::placeholders::_1 ));
}
void
handle_send(boost::system::error_code const& ec)
{
}
};
编辑 只要您不必在写入处理程序中执行任何操作,就可以使用 lambda(需要 C++11)作为完成回调
// The send function
void
send(char const* data, size_t size)
{
std::ostreambuf_iterator<char> out(&buffer_);
std::copy(data, data + size, out);
socket.async_send_to(buffer_, endpoint,
[](boost::system::error_code const& ec){
std::cerr << "Error sending :" << ec.message() << "\n";
});
}
内存没有泄漏,因为分配的内存仍有句柄。但是,将会持续增长,因为:
io_service
不是运行因为run()
is returning as there is no work. This results in completion handlers being allocated, queued into theio_service
, but neither executed nor freed. Additionally, any cleanup that is expected to occur within the completion handler is not occurring. It is worth noting that during the destruction of theio_service
, completion handlers will be destroyed and not invoked; hence, one cannot depend on only performing cleanup within the execution of the completion handler. For more details as to whenio_service::run()
blocks or unblocks, consider reading this问题。streambuf
的输入序列永远不会被消耗。主循环中的每次迭代都会附加到streambuf
,然后发送先前的消息内容和新附加的数据。有关streambuf
. 整体用法的更多详细信息,请参阅
其他几点:
- 该程序未能满足
async_send_to()
, where ownership of the underlying buffer memory is retained by the caller, who must guarantee that it remains valid until the handler is called. In this case, when copying into thestreambuf
via theostreambuf_iterator
, thestreambuf
's input sequence is modified and invalidates the buffer returned fromstreambuf.data()
的要求。 - 在关闭期间,需要针对 运行 和
io_service
的线程进行某种形式的同步。否则,可能会调用未定义的行为。
要解决这些问题,请考虑:
- 使用
boost::asio::io_service::work
确保io_service
对象的run()
在没有剩余工作时不会退出。 - 通过
std::shared_ptr
或另一个 class 将内存所有权传递给完成处理程序,后者将通过 resource acquisition is initialization (RAII) 习惯用法管理内存。这将允许进行适当的清理并满足async_send_to()
. 的缓冲区有效性要求
- 不在工作线程上分离和加入。
这里是一个完整的例子,基于原来的 demonstrates 这些变化:
#include <string>
#include <iostream>
#include <thread>
#include <boost/asio.hpp>
class multicast_sender
{
public:
multicast_sender(
const std::string& address,
const std::string& multicast_address,
const unsigned short multicast_port)
: work_(io_service_),
multicast_endpoint_(
boost::asio::ip::address::from_string(multicast_address),
multicast_port),
socket_(io_service_, boost::asio::ip::udp::endpoint(
boost::asio::ip::address::from_string(address),
0 /* any port */))
{
// Start running the io_service. The work_ object will keep
// io_service::run() from returning even if there is no real work
// queued into the io_service.
auto self = this;
work_thread_ = std::thread([self]()
{
self->io_service_.run();
});
}
~multicast_sender()
{
// Explicitly stop the io_service. Queued handlers will not be ran.
io_service_.stop();
// Synchronize with the work thread.
work_thread_.join();
}
void send(const char* data, const int size)
{
// Caller may delete before the async operation finishes, so copy the
// buffer and associate it to the completion handler's lifetime. Note
// that the completion may not run in the event the io_servie is
// destroyed, but the the completion handler will be, so managing via
// a RAII object (std::shared_ptr) is ideal.
auto buffer = std::make_shared<std::string>(data, size);
socket_.async_send_to(boost::asio::buffer(*buffer), multicast_endpoint_,
[buffer](
const boost::system::error_code& error,
std::size_t bytes_transferred)
{
std::cout << "Wrote " << bytes_transferred << " bytes with " <<
error.message() << std::endl;
});
}
private:
boost::asio::io_service io_service_;
boost::asio::io_service::work work_;
boost::asio::ip::udp::endpoint multicast_endpoint_;
boost::asio::ip::udp::socket socket_;
std::thread work_thread_;
};
const int SIZE = 8192;
int main()
{
multicast_sender sender("127.0.0.1", "239.255.0.0", 30000);
char* data = (char*) malloc(SIZE);
std::memset(data, 0, SIZE);
sender.send(data, SIZE);
free(data);
// Give some time to allow for the async operation to complete
// before shutting down the io_service.
std::this_thread::sleep_for(std::chrono::seconds(2));
}
输出:
Wrote 8192 bytes with Success