使用 Wt::Dbo 导致 MySQL 准备语句泄漏

Question

我有一个项目使用 Wt::Dbo 作为对象关系数据库管理和 MySQL。大约一周以来，我注意到数据库准备语句中存在泄漏。以前这个项目使用的是 SQLite。

我尝试了不同的 flush() 但没有成功，也无法弄清楚是什么导致了这种泄漏，但是可以肯定的是，当准备好的语句在某个点增长时，MySQL 停止回答。

以下是我如何监控泄漏的准备好的语句：

$ mysql -uroot -p -e "SHOW SESSION STATUS LIKE '%prepare%';" | grep stmt_count
Enter password: 
Prepared_stmt_count 260

程序重新启动时会清除泄漏的语句。

所有数据库操作都集中在一个名为 DataBase 的 class 中，这里有一些已知会泄漏的函数：

数据库::初始化()

void DataBase::initialize(void)
{
    m_oMutex.lock();

    if(!m_bInitialized)
    {
        m_bInitialized = true;

        try
        {
            m_oSessionConfigurations.setConnection(*new Wt::Dbo::backend::Sqlite3("databases/configuration.db"));
            m_oSessionConfigurations.mapClass<InformationSite>("informationsite");

            m_oSessionConfigurations.createTables();
        }
        catch(std::exception &e)
        {
            ajouterLog("error",std::string("DataBase::initialize() : ") + e.what());
        }

        try
        {
            #if defined(DATABASE_TYPE_SQLITE)
                Wt::Dbo::backend::Sqlite3 *pBackend = new Wt::Dbo::backend::Sqlite3("databases/dataBase.db");
            #elif defined(DATABASE_TYPE_MYSQL)
                Wt::Dbo::backend::MySQL *pBackend;

                try
                {
                    pBackend = new Wt::Dbo::backend::MySQL(DATABASE_NAME,DATABASE_USERNAME,DATABASE_PASSWORD,"localhost");
                }
                catch(Wt::Dbo::Exception &e)
                {
                    ajouterLog("error",std::string("DataBase::initialize() : ") + e.what());

                    // If MySQL is not available, this cause issue to program until restart.
                    exit(1);
                }
            #endif

            pBackend->setProperty("show-queries","true");

            m_oSession.setConnection(*pBackend);

            m_oSession.setFlushMode(Wt::Dbo::FlushMode::Auto);

            m_oSession.mapClass<RFNode>("rfnode");
            m_oSession.mapClass<NodeMeasure>("nodemeasure");

            // Override the default InnoDB from Wt, MyISAM is easier to repair in case of hardware failure with database corruption
            #if defined(DATABASE_TYPE_MYSQL)
                try
                {
                    Wt::Dbo::Transaction oTransaction(m_oSession);

                    m_oSession.execute("SET default_storage_engine=MYISAM;");

                    oTransaction.commit();
                }
                catch(Wt::Dbo::Exception &e)
                {
                    ajouterLog("error",std::string("DataBase::initialize() : ") + e.what());
                }
            #endif

            m_oSession.createTables();
        }
        catch(Wt::Dbo::Exception &e)
        {
            ajouterLog("error",std::string("DataBase::initialize() : ") + e.what());
        }
    }

    m_oMutex.unlock();
}

DataBase::addNodeMeasure()

void DataBase::addNodeMeasure(NodeMeasure *p_pItem)
{
    m_oMutex.lock();

    try
    {
        Wt::Dbo::Transaction oTransaction(m_oSession);

        Wt::Dbo::ptr<NodeMeasure> oItem = m_oSession.add(p_pItem);

        oItem.flush();

        oTransaction.commit();
    }
    catch(std::exception &e)
    {
        ajouterLog("error",std::string("Exception DataBase::addNodeMeasure() : ") + e.what());
    }

    m_oMutex.unlock();

    printPreparedStatementCount("DataBase::addNodeMeasure()");
}

DataBase::updateNode()

void DataBase::updateNode(RFNode *p_pItem)
{
    printPreparedStatementCount("DataBase::updateNode() Before");

    m_oMutex.lock();

    try
    {
        Wt::Dbo::Transaction oTransaction(m_oSession);

        Wt::Dbo::ptr<RFNode> oItem = m_oSession.find<RFNode>().where("mac = ?").bind(p_pItem->mac);

        oItem.modify()->zone                    = p_pItem->zone;
        oItem.modify()->subZone                 = p_pItem->subZone;
        oItem.modify()->unit                    = p_pItem->unit;
        oItem.modify()->pwm                     = p_pItem->pwm;
        oItem.modify()->led                     = p_pItem->led;
        oItem.modify()->network                 = p_pItem->network;
        oItem.modify()->lastContact             = p_pItem->lastContact;
        oItem.modify()->ioConfiguration         = p_pItem->ioConfiguration;
        oItem.modify()->networkAddress          = p_pItem->networkAddress;
        oItem.modify()->type                    = p_pItem->type;
        oItem.modify()->functionality           = p_pItem->functionality;
        oItem.modify()->transmitPowerLevel      = p_pItem->transmitPowerLevel;
        oItem.modify()->lastNetworkRoute        = p_pItem->lastNetworkRoute;
        oItem.modify()->lastNetworkJumpsCount   = p_pItem->lastNetworkJumpsCount;
        oItem.modify()->lastRequestDuration     = p_pItem->lastRequestDuration;
        oItem.modify()->hardwareVersion         = p_pItem->hardwareVersion;
        oItem.modify()->softwareVersion         = p_pItem->softwareVersion;

        oItem.flush();

        oTransaction.commit();
    }
    catch(std::exception &e)
    {
        ajouterLog("error",std::string("Exception DataBase::updateNode() : ") + e.what());
    }

    m_oMutex.unlock();

    printPreparedStatementCount("DataBase::updateNode() After");
}

DataBase::getNodeMeasures()

std::vector<NodeMeasure> DataBase::getNodeMeasures(std::string p_sMAC, int p_nType, Wt::WDateTime p_oStartDate, Wt::WDateTime p_oEndDate, std::string p_sOrder, int p_nLimit)
{
    std::vector<NodeMeasure> lNodeMeasures;

    m_oMutex.lock();

    try
    {
        Wt::Dbo::Transaction oTransaction(m_oSession);

        std::string sWhereClause = "", sOrderClause = "";

        if(!p_sMAC.empty())
        {
            if(!sWhereClause.empty())
            {
                sWhereClause += " AND ";
            }

            sWhereClause += "mac = '" + p_sMAC + "'";
        }

        if(p_nType != -1)
        {
            if(!sWhereClause.empty())
            {
                sWhereClause += " AND ";
            }

            sWhereClause += "type = " + std::to_string(p_nType);
        }

        if(p_oStartDate.isValid())
        {
            if(!sWhereClause.empty())
            {
                sWhereClause += " AND ";
            }

            // When not using type, we usually want nodes measures (not external temperature), so we want to find them using batchDate instead of date
            sWhereClause += (p_nType != -1 ? "date" : "batchDate");
            sWhereClause += " >= '";
            sWhereClause += p_oStartDate.toString("yyyy-MM-ddTHH:mm:ss").toUTF8();
            sWhereClause += "'";
        }

        if(p_oEndDate.isValid())
        {
            if(!sWhereClause.empty())
            {
                sWhereClause += " AND ";
            }

            // When not using type, we usually want nodes measures (not external temperature), so we want to find them using batchDate instead of date
            sWhereClause += (p_nType != -1 ? "date" : "batchDate");
            sWhereClause += " <= '";
            // Add one second because SQLite have microseconds, and we must include results no matter microseconds field
            sWhereClause += p_oEndDate.addSecs(1).toString("yyyy-MM-ddTHH:mm:ss").toUTF8();
            sWhereClause += "'";
        }

        if(!p_sOrder.empty())
        {
            sOrderClause = " ORDER BY " + p_sOrder;
        }

        std::string sQuery = "";

        if(!sWhereClause.empty())
        {
            sQuery += " WHERE ";
            sQuery += sWhereClause;
        }

        if(!sOrderClause.empty())
        {
            sQuery += sOrderClause;
        }

        //std::cout << "**************************************************************************" << std::endl;
        //std::cout << sQuery << std::endl;
        //Wt::WDateTime oStart = Wt::WDateTime::currentDateTime();

        if(Configuration::getParameter(Configuration::PARAMETER_DEBUG).getBooleanValue())
        {
            ajouterLog("debug","DataBase::getNodeMeasures() " + sQuery);
        }

        // TEST : find vs query
        Wt::Dbo::collection<Wt::Dbo::ptr<NodeMeasure>> lMeasures = m_oSession.find<NodeMeasure>(sQuery).limit(p_nLimit).resultList();

        // TODO : Get it cleaner... can't use Wt::Dbo::ptr outside transaction.
        for(Wt::Dbo::collection<Wt::Dbo::ptr<NodeMeasure>>::const_iterator pMeasure = lMeasures.begin();pMeasure != lMeasures.end();pMeasure++)
        {
            lNodeMeasures.push_back(
                    NodeMeasure(
                            (*pMeasure)->mac,
                            (*pMeasure)->type,
                            (*pMeasure)->date,
                            (*pMeasure)->batchDate,
                            (*pMeasure)->value
                    )
            );

            (*pMeasure).flush();
        }

        //lNodeMeasures = m_oSession.find<NodeMeasure>(sQuery).limit(p_nLimit).resultList();

        //std::cout << "Result : " << lNodeMeasures.size() << " in " << oStart.secsTo(Wt::WDateTime::currentDateTime()) << "s" << std::endl;
        //std::cout << "**************************************************************************" << std::endl;

        oTransaction.commit();
    }
    catch(std::exception &e)
    {
        ajouterLog("error",std::string("Exception DataBase::getNodeMeasures() : ") + e.what());
    }

    m_oMutex.unlock();

    printPreparedStatementCount("DataBase::getNodeMeasures()");

    return lNodeMeasures;
}

Answer 1

语句的大小是否失控？ Wt::Dbo 缓存 MySQL SqlConnection 对象中的所有语句，并在销毁时清除它们。由于应用程序经常使用例如连接池大小 10，准备语句的数量通常是您使用的语句数量的倍数。

由于您似乎将语句编写为 ascii 文本而不是使用参数绑定，因此您最终可能确实在缓存中有很多语句。

Dbo支持这种代码风格：

    auto measuresQuery = m_oSession.find<NodeMeasure>(sQuery).limit(p_nLimit);
    if(!p_sMAC.empty())
    {
        measureQuery.where("mac = ?").bind(p_sMAC);
    }

    if(p_nType != -1)
    {
        measureQuery.where("type = ?").bind(std::to_string(p_nType));
    }
    if(p_oStartDate.isValid())
    {
        // When not using type, we usually want nodes measures (not external temperature), so we want to find them using batchDate instead of date
        measureQuery.where(p_nType != -1 ? "date >= ?" : "batchDate >= ?").bind(p_oStartData);
    }
    if(!p_sOrder.empty())
    {
        measureQuery.orderBy(p_sOrder);
    }

    Wt::Dbo::collection<Wt::Dbo::ptr<NodeMeasure>> lMeasures = measureQuery.resultList();

这将产生更少量的准备好的语句，并且在我看来更短、更清晰、更安全的代码。

使用 Wt::Dbo 导致 MySQL 准备语句泄漏

Using Wt::Dbo cause MySQL prepared statement leak

mysql

c++11

wt

dbo