通过 JNA 将指向指针的指针从 Java 传递到 C 动态库

Question

Catboost 提供了一个 dynamic C library，理论上可以在任何编程语言中使用。

我正在尝试使用 JNA 通过 Java 调用它。

我在 CalcModelPrediction 函数中遇到问题，在 header file 中定义如下：

EXPORT bool CalcModelPrediction(
    ModelCalcerHandle* calcer,
    size_t docCount,
    const float** floatFeatures, size_t floatFeaturesSize,
    const char*** catFeatures, size_t catFeaturesSize,
    double* result, size_t resultSize);

在Java中，我定义了接口函数如下：

public interface CatboostModel extends Library {
        public Pointer ModelCalcerCreate();
        public String GetErrorString();
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, int docCount,
                PointerByReference floatFeatures, int floatFeaturesSize,
                PointerByReference catFeatures, int catFeaturesSize,
                Pointer result, int resultSize);
        public int GetFloatFeaturesCount(Pointer calcer);
        public int GetCatFeaturesCount(Pointer calcer);
    }

然后我这样称呼它：

CatboostModel catboost;
Pointer modelHandle;

catboost = Native.loadLibrary("catboostmodel", CatboostModel.class);
            modelHandle = catboost.ModelCalcerCreate();
if (!catboost.LoadFullModelFromFile(modelHandle, "catboost_test.model"))
{
    throw new RuntimeException("Cannot load Catboost model.");
}

final PointerByReference ppFloatFeatures = new PointerByReference();
final PointerByReference ppCatFeatures = new PointerByReference();
final Pointer pResult = new Memory(Native.getNativeSize(Double.TYPE));

float[] floatFeatures = {0.5f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};
int catFeaturesLength = 0;
for (String s : catFeatures)
{
    catFeaturesLength += s.length() + 1;
}

try
{
    final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
    for (int dloop=0; dloop<floatFeatures.length; dloop++) {
        pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
    }
    ppFloatFeatures.setValue(pFloatFeatures);

    final Pointer pCatFeatures = new Memory(catFeaturesLength * Native.getNativeSize(Character.TYPE));
    long offset = 0;
    for (final String s : catFeatures) {
        pCatFeatures.setString(offset, s);
        pCatFeatures.setMemory(offset + s.length(), 1, (byte)(0));
        offset += s.length() + 1;
    }
    ppCatFeatures.setValue(pCatFeatures);

}
catch (Exception e)
{
    throw new RuntimeException("Couldn't initialize parameters for catboost");
}

try
{
    if (!catboost.CalcModelPrediction(
                modelHandle,
                1,
                ppFloatFeatures, 10,
                ppCatFeatures, 4,
                pResult, 1
                ))
    {
        throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
    }
    else
    {
        double[] result = pResult.getDoubleArray(0, 1);
        log.info("Catboost prediction: " + String.valueOf(result[0]));
        Assert.assertFalse("ERROR: Result empty", result.length == 0);
    }
}
catch (Exception e)
{
    throw new RuntimeException("Prediction failed: " + e);
}

我尝试将 Pointer、PointerByReference 和 Pointer[] 传递给 CalcModelPrediction 函数来代替 float **floatFeatures 和 char ***catFeatures 但是没有任何效果。我总是遇到分段错误，大概是在 CalcModelPrediction 函数试图通过调用 floatFeatures[0][0] 和 catFeatures[0][0].[=48= 来获取 floatFeatures 和 catFeatures 的元素时]

所以问题是，将多维数组从 Java 通过 JNA 传递到 C 中的正确方法是什么，在那里它可以被视为指向值指针的指针？

有趣的是 CalcModelPredictionFlat 函数只接受 float **floatFeatures 然后简单地调用 *floatFeatures，在传递 PointerByReference.[=48= 时工作得很好]

更新 - 5.5.2018

第 1 部分

在尝试通过稍微修改原始 Catboost .cpp 和 .h 文件并重新编译 libcatboost.so 库来调试段错误后，我发现段错误是由于我将 size_t 映射到C 到 int 在 Java。解决这个问题后，我在 Java 中的接口函数如下所示：

public interface CatboostModel extends Library {
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
                Pointer[] floatFeatures, size_t floatFeaturesSize,
                String[] catFeatures, size_t catFeaturesSize,
                Pointer result, size_t resultSize);
    }

其中size_tClass定义如下：

public static class size_t extends IntegerType {                                           
    public size_t() { this(0); }                                                           
    public size_t(long value) { super(Native.SIZE_T_SIZE, value); }                       
}

第 2 部分 更多地查看 Catboost 代码，我注意到 **floatFeatures 被行访问，如 floatFeatures[i] 而 ***catFeatures 被行和列访问，如 catFetures[i][catFeatureIdx].

将 Java 中的 floatFeatures 更改为 Pointer 的数组后，我的代码开始使用没有分类特征训练的模型，即 catFeatures 长度为零.

然而，这个技巧不适用于通过双下标运算符 [i][catFeatureidx] 访问的 catFeatures。所以现在，我修改了原始的 Catboost 代码，以便它可以接受 char **catFeatures - 一个字符串数组。在Java接口函数中，我设置了String[] catFeatures。现在我一次可以预测一个元素，这并不理想。

Answer 1

我已经设法使用原始的 Catboost 代码和 libcatboost.so。

Java接口函数是这样定义的。请注意，为了模拟浮点值和字符串的二维数组（或指向指针的指针），我使用了 Pointer[] 类型。

public interface CatboostModel extends Library {
        public boolean LoadFullModelFromFile(Pointer calcer, String filename);
        public boolean CalcModelPrediction(Pointer calcer, size_t docCount,
                Pointer[] floatFeatures, size_t floatFeaturesSize,
                Pointer[] catFeatures, size_t catFeaturesSize,
                Pointer result, size_t resultSize);
    }

之后，我像这样填充 floatFeatures 和 catFeatures 参数（此处为一些虚拟数据）。请注意，对于字符串，我使用的是 JNA 的 StringArray.

float[] floatFeatures = {0.4f, 0.8f, 0.3f, 0.3f, 0.1f, 0.5f, 0.4f, 0.8f, 0.3f, 0.3f} ;
String[] catFeatures = {"1", "2", "3", "4"};

final Pointer pFloatFeatures = new Memory(floatFeatures.length * Native.getNativeSize(Float.TYPE));
final Pointer[] ppFloatFeatures = new Pointer[2];
for (int dloop=0; dloop<10; dloop++) {
    pFloatFeatures.setFloat(dloop * Native.getNativeSize(Float.TYPE), floatFeatures[dloop]);
}
ppFloatFeatures[0] = pFloatFeatures;
ppFloatFeatures[1] = pFloatFeatures;

final Pointer[] ppCatFeatures = new Pointer[catFeatures.length];
final Pointer pCatFeatures = new StringArray(catFeatures);
ppCatFeatures[0] = pCatFeatures;
ppCatFeatures[1] = pCatFeatures;

最后，我将这些参数传递给 Catboost：

if (!catboost.CalcModelPrediction(
                modelHandle,
                new size_t(2L),
                ppFloatFeatures, new size_t((long)floatFeatures.length),
                ppCatFeatures, new size_t((long)catFeatures.length),
                pResult, new size_t(2L)
                ))
{
    throw new RuntimeException("No prediction made: " + catboost.GetErrorString());
}

要获得预测，我们可以这样做：

double[] result = pResult.getDoubleArray(0, 2);

通过 JNA 将指向指针的指针从 Java 传递到 C 动态库

Passing pointer to pointer to float from Java through JNA to a C dynamic library

java

c

jna

catboost