MRTK 2.4 - 在运行时保存空间网格

Question

现在我正在尝试将全息镜头上的空间网格保存为 obj 文件。我的问题是，当试图打开 obj 文件时，例如在 Blender（或 3D 查看器）中，我收到一条错误消息 IndexError: List index out of range。所以我想我给了我的三角形错误的索引（在 MeshToString() 的最后一部分），但我不知道如何以正确的方式做到这一点。

有效方法： 例如，如果我只从第一个网格过滤器中取出一个网格并在 MeshToString() 中剪切掉 + lastFaceIndex，它就可以正常工作。但这只是我房间的一部分。我想要我房间里的整个网格，所以我必须通过所有网格过滤器并获取我的网格，然后将它们写入 obj 文件，但我不知道如何为三角形提供正确的索引。

我还注意到，如果我通过设备门户下载网格它有 ~4.8MB，但我通过我的统一应用程序下载的网格只有 1.4MB。

我的设置：
统一 2019.3.14 捷运 2.4 第二代全息透镜

GetSpatialMesh():

private void GetSpatialMesh()
{
    if (_observer == null)
        return;

    List<Mesh> meshes = new List<Mesh>();
    // Loop through all known Meshes
    foreach (SpatialAwarenessMeshObject meshObject in _observer.Meshes.Values)
        meshes.Add(meshObject.Filter.mesh);
    WriteMeshToFile("MyMesh.obj", meshes);
}

WriteMeshToFile():

public static void WriteMeshToFile(string fileName, IEnumerable<Mesh> meshes)
{
    string path = Path.Combine(Application.persistentDataPath, fileName);
    using (var file = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.Write))
    {
        faceCount = 0;
        using (var writer = new StreamWriter(file, Encoding.UTF8))
        {
            int o = 0;
            foreach (Mesh mesh in meshes)
            {
                o++;
                writer.WriteLine("o Object." + o);
                writer.Write(MeshToString(mesh, faceCount));
                writer.WriteLine("");
            }
        }
    }
}

MeshToString():

public static string MeshToString(Mesh m, int lastFaceIndex = 0)
{
    StringBuilder sb = new StringBuilder();

    foreach (Vector3 v in m.vertices)
    {
        sb.Append(string.Format("v {0} {1} {2}\n", v.x, v.y, v.z));
    }
    sb.Append("\n");
    foreach (Vector3 v in m.normals)
    {
        sb.Append(string.Format("vn {0} {1} {2}\n", v.x, v.y, v.z));
    }
    sb.Append("\n");
    foreach (Vector3 v in m.uv)
    {
        sb.Append(string.Format("vt {0} {1}\n", v.x, v.y));
    }
    for (int material = 0; material < m.subMeshCount; material++)
    {
        int[] triangles = m.GetTriangles(material);
        for (int i = 0; i < triangles.Length; i += 3)
        {
            faceCount += 3;
            sb.Append(string.Format("f {0}/{0}/{0} {1}/{1}/{1} {2}/{2}/{2}\n",
                triangles[i] + 1 + lastFaceIndex, triangles[i + 1] + 1 + lastFaceIndex, triangles[i + 2] + 1 + lastFaceIndex));
        }
    }
    return sb.ToString();
}

解法：
场景理解 sample project didnt work for me, so I just got the sdk via nuget manager in unity. I used the code from SceneUnderstandingManager（这是示例项目的一部分），由 Hernando-MSFT 在下面的回答中引用。

Answer 1

我注意到您发布的代码是基于 Legacy HoloToolkit but all support will now be with the newer Scene understanding SDK。由于您使用的是 HoloLens2，我们建议您使用场景理解 SDK 来查询静态版本的空间映射数据并将序列化的场景字节保存到磁盘。扫描房间后，调用SceneObserver.ComputeSerializedAsync将场景序列化为字节数组。

Microsoft.MixedReality.SceneUnderstanding.Samples is a Unity-based sample application that showcases Scene Understanding on HoloLens 2. And it shows up how to save any scene you've captured by saving the output of ComputeSerializedAsync to file: Line1154

此外，SaveObjsToDiskAsync函数展示了如何将场景理解中的Unity对象保存为Obj文件：Line1206

Answer 2

嗨，我知道这有点老了，Perazim 已经解决了他的问题，但我还需要将 hololens 2 Mesh 转换为 obj，所以...也许它将来对其他人有用:) ，这就是我提出解决方案的原因。

不幸的是，我没有设法使用场景理解的代码，正如 Hernando 所说（免责声明这是我第一次尝试为 Hololens 编程，所以......我很确定这是我的错).当我尝试调用

SceneBuffer serializedScene = SceneObserver.ComputeSerializedAsync(querySettings, 10.0f).GetAwaiter().GetResult();

我收到一个错误，实际上我的应用程序不再工作了！

我所做的工作是“手动”创建 obj，对我有用的函数如下：

    private void exportScene(String targetFileName)
    {
        var spatialAwarenessSystem = CoreServices.SpatialAwarenessSystem;
        spatialAwarenessSystem.SuspendObservers();

        var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
        System.Text.StringBuilder objFileContent = new System.Text.StringBuilder(4096);
        objFileContent.Append("# Automatic export of Hololens 2 scanned mesh scan. \r\n\r\n");

        // Collects the spread objects
        foreach (SpatialAwarenessMeshObject meshObject in observer.Meshes.Values)
        {
            meshes.Add(meshObject.Filter);
        }

        int adjust = 1;
        MeshFilter[] mf = meshes.ToArray();
        int countVertex = 0;

        // for each object gets the vertexes normals and faces
        for (int i = 0; i < mf.Length; i++)
        {
            Mesh m = mf[i].sharedMesh;
            objFileContent.Append("o Object." + (i + 1) + "\r\n");
            
            // inverts x as the coordinates are different from the ones of regular obj
            foreach (Vector3 v in m.vertices)
            {
                float x = -v.x;
                float z = v.z;
                objFileContent.Append("v " + x.ToString("0.000000", CultureInfo.InvariantCulture) + " " + v.y.ToString("0.000000", CultureInfo.InvariantCulture) + " " + z.ToString("0.000000", CultureInfo.InvariantCulture) + "\r\n");
            }

            objFileContent.Append("\r\n\r\n");

            foreach (Vector3 n in m.normals)
            {
                float x = n.x;
                float z = n.z;
                objFileContent.Append("vn " + x.ToString("0.000000", CultureInfo.InvariantCulture) + " " + n.y.ToString("0.000000", CultureInfo.InvariantCulture) + " " + z.ToString("0.000000", CultureInfo.InvariantCulture) + "\r\n");
            }
            objFileContent.Append("\r\n\r\n");

            // the count of the faces starts with 1 and is cumulative for 
            // all objects on the scene this is why it is add 
            // adjust + countVertex as the id of the vertexes
            for (int ti = 0; ti < m.triangles.Length; ti += 3)
            {
                objFileContent.Append("f " + (m.triangles[ti] + adjust + countVertex ) + "//" + (m.triangles[ti] + adjust + countVertex) + " " + (m.triangles[ti + 1] + adjust + countVertex) + "//" + (m.triangles[ti + 1] + adjust + countVertex) + " " + (m.triangles[ti + 2] + adjust + countVertex) + "//" + (m.triangles[ti + 2] + adjust + countVertex) + "\r\n");
            }
            objFileContent.Append("\r\n\r\n");
            countVertex += m.vertexCount;
      }
       // string objPath = Path.Combine(sceneFolderPath, sceneName + ".obj"); 
        using (StreamWriter sw = File.AppendText(targetFileName))
        {
            sw.WriteLine(objFileContent.ToString());
        }
    }

MRTK 2.4 - 在运行时保存空间网格

MRTK 2.4 - Save Spatial Mesh on Runtime

unity3d

spatial

mesh

hololens

mrtk