Java 堆 space 与 jpa 本机查询

Question

我在尝试通过 jpa/hibernate EM 运行本机 SQL 查询时遇到 OOME 问题。每包 50 个处理可插入数百万次。

这是我的算法代码：

private void createNewJetonsForIndividus(boolean isGlobal, List<String> entreprises, List<String> services,
                                         String user, Timestamp dateDB) {

    LocalDateTime timer = LocalDateTime.now();

    List<Object[]> MinMaxId = getMinMaxIdDroitsIndividusActifsForCreation(
            isGlobal, entreprises, services);

    if (null != MinMaxId.get(0)[0]) {

        int idStart =  ((BigInteger) MinMaxId.get(0)[0]).intValue();
        int idEnd = idStart + PAS;
        int idMax = ((BigInteger) MinMaxId.get(0)[1]).intValue();
        int nbRowsTotal = 0;
        Logger.debug("Droits Individus : ID Min {}  -  ID Max {}", idStart, idMax);


        do {
            int finalIdStart = idStart;
            int finalIdEnd = idEnd;
            callTransaction(() -> create(false, true,isGlobal, entreprises, services, finalIdStart,
                    finalIdEnd, user, dateDB));
            idStart = idEnd + 1;
            idEnd = idEnd + PAS;
        }
        while (idMax > idEnd);

    }
}

该方法用于计算我感兴趣的记录的 id 最小值和最大值。随后，我使用代码如下的创建方法：

int nbRowsFind;

    List<Object[]> listeDroitsIndividusActifsForCreation = getDroitsIndividusActifsForCreation(
            isGlobal, entreprises, services, idStart, idEnd);

    if (ValidationUtils.isNotEmpty(listeDroitsIndividusActifsForCreation)) {

        nbRowsFind = listeDroitsIndividusActifsForCreation.size();

        StringBuilder sbJeton = new StringBuilder();
        sbJeton.append("INSERT INTO sigs_nv_jeton VALUES ");


        StringBuilder sbDroitHasJeton = new StringBuilder();

        if (isCreateForIndiv) {
            sbDroitHasJeton.append("INSERT INTO sigs_droits_individu_has_nv_jeton VALUES ");
        }

        listeDroitsIndividusActifsForCreation.stream().forEach(object -> {
            sbJeton.append("(");
            sbDroitHasJeton.append("(");

            BigInteger idDroit = (BigInteger) object[0];

            String jetonGenerated = IdJetonGenerator.codeGenerator(idDroit.toString(), DateUtils.now());
            sbJeton.append("'").append(jetonGenerated).append("', ");

            appendDate(sbJeton, object[1]);
            appendDate(sbJeton, object[2]);
            sbJeton.append(0).append(", ");
            sbJeton.append(0).append(", ");
            sbJeton.append("'").append(dateDB).append("', ");
            sbJeton.append("'").append(user).append("'");

            sbDroitHasJeton.append(idDroit).append(",'").append(jetonGenerated).append("'");

            sbJeton.append("),");
            sbDroitHasJeton.append("),");
        });


        String requestJeton = sbJeton.toString();
        sbJeton.delete(33, sbJeton.length());
        requestJeton = requestJeton.substring(0, requestJeton.length() - 1);
        jpaApi.em().createNativeQuery(requestJeton).executeUpdate();

        String requestDroitHasJeton = sbDroitHasJeton.toString();
        sbDroitHasJeton.delete(54, sbDroitHasJeton.length());
        requestDroitHasJeton = requestDroitHasJeton.substring(0, requestDroitHasJeton.length() - 1);
        **jpaApi.em().createNativeQuery(requestDroitHasJeton).executeUpdate();**


        **jpaApi.em().flush();
        jpaApi.em().clear();**
    }

当我分析 Heap Dump 时，我注意到尽管刷新和清除，查询仍然在 SessionFactory 中引用，这是否正常？ enter image description here

Answer 1

flush 正在执行工作单元中所做的更改，clear 从持久性上下文中删除实体。

两者都与本机 SQL 查询无关。我假设本机查询缓存在其他任何地方（Hibernate，JDBC...）

我建议您使用准备好的语句而不是动态插入语句。

Answer 2

这是我的新实现：

    Statement statement = null;
    ResultSet resultSet = null;
    int nbRows = 0;

    try {
        statement = connection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,
                ResultSet.CONCUR_READ_ONLY);
        try {
            resultSet = statement.executeQuery(queryGetDroitsEntreprisesActifsForCreation(isGlobal, entreprises, services));
            resultSet.beforeFirst();
            while (resultSet.next()) {
                nbRows += 1;
                queryInsertJetonsAndLinkDroitsJeton(isCreateForEntreprise, isCreateForIndiv, connection, resultSet,
                        user, dateDB);
            }
            System.out.println(nbRows);
        } finally {
            if (resultSet != null) {
                resultSet.close();
            }
        }
    } finally {
        if (statement != null) {
            statement.close();
        }
    }

方法 "queryGetDroitsEntreprisesActifsForCreation" returns SQL 字符串查询。方法 "queryInsertJetonsAndLinkDroitsJeton" 使用著名的 PreparedStatement :

PreparedStatement psJeton = null;
    PreparedStatement psDroitJeton = null;
    try {
        String sbJeton = "INSERT INTO TABLE1 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) ";

        psJeton = connection.prepareStatement(sbJeton);

        String jetonGenerated;
        if (isCreateForEntreprise) {
            jetonGenerated = IdJetonGenerator.codeGenerator(String.valueOf(resultSet.getInt("id_e")), DateUtils.now());
        } else {
            jetonGenerated = IdJetonGenerator.codeGenerator(String.valueOf(resultSet.getInt("id_i")), DateUtils.now());
        }
        psJeton.setString(1, jetonGenerated);
        psJeton.setString(2, ContextType.GD.name());
        psJeton.setString(3, JetonType.ANONYME.name());
        psJeton.setInt(4, 1);
        psJeton.setTimestamp(5, resultSet.getTimestamp("dt_debut"));
        psJeton.setTimestamp(6, resultSet.getTimestamp("dt_fin"));
        psJeton.setInt(7, 0);
        psJeton.setInt(8, 0);
        psJeton.setInt(9, 0);
        psJeton.setInt(10, 0);
        psJeton.setTimestamp(11, dateDB);
        psJeton.setString(12, user);
        psJeton.setTimestamp(13, dateDB);
        psJeton.setString(14, user);
        psJeton.executeUpdate();


        String sbDroitHasJeton = null;
        if (isCreateForEntreprise) {
            sbDroitHasJeton = "INSERT INTO sigs_dej VALUES (?, ?)";
        }
        if (isCreateForIndiv) {
            sbDroitHasJeton = "INSERT INTO sigs_dij VALUES (?, ?)";
        }

        psDroitJeton = connection.prepareStatement(sbDroitHasJeton);
        if(isCreateForEntreprise) {
            psDroitJeton.setInt(1, resultSet.getInt("id_e"));
        } else {
            psDroitJeton.setInt(1, resultSet.getInt("id_i"));
        }

        psDroitJeton.setString(2, jetonGenerated);
        psDroitJeton.executeUpdate();


    } catch (SQLException e) {
        e.printStackTrace();

    } finally {
        if (psJeton != null) {
            psJeton.close();
        }
        if (psDroitJeton != null) {
            psDroitJeton.close();
        }

    }

我希望这是 PreparedStatement 和 Scrollable ResultSet 的最佳实现

Java 堆 space 与 jpa 本机查询

Java Heap space with jpa native Query

java

jpa

heap-memory

nativequery