StartCoroutine() 修复 targetTexture.ReadPixels 错误
StartCoroutine() to fix targetTexture.ReadPixels error
如标题所示，我对第
行发生的错误有疑问
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
错误：
ReadPixels was called to read pixels from system frame buffer, while
not inside drawing frame. UnityEngine.Texture2D:ReadPixels(Rect,
Int32, Int32)
正如我从其他帖子中了解到的，解决此问题的一种方法是制作一个 Ienumerator 方法，该方法产生 return 新的 WaitForSeconds 或其他东西，并将其称为：StartCoroutine(methodname) 以便帧及时加载，以便有像素可读。
我不明白的是在下面的代码中这个方法最有意义。哪个部分没有及时加载？
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;
public string path = "";
CameraParameters cameraParameters = new CameraParameters();
private void Awake()
{
var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);
// Create a PhotoCapture object
PhotoCapture.CreateAsync(false, captureObject =>
{
photoCaptureObject = captureObject;
cameraParameters.hologramOpacity = 0.0f;
cameraParameters.cameraResolutionWidth = cameraResolution.width;
cameraParameters.cameraResolutionHeight = cameraResolution.height;
cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
});
}
private void Update()
{
// if not initialized yet don't take input
if (photoCaptureObject == null) return;
if (Input.GetKey("k") || Input.GetKey("k"))
{
Debug.Log("k was pressed");
VuforiaBehaviour.Instance.gameObject.SetActive(false);
// Activate the camera
photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
{
if (result.success)
{
// Take a picture
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
else
{
Debug.LogError("Couldn't start photo mode!", this);
}
});
}
}
private static string FileName(int width, int height)
{
return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
// Shutdown the photo capture resource
VuforiaBehaviour.Instance.gameObject.SetActive(true);
photoCaptureObject.Dispose();
photoCaptureObject = null;
}
抱歉，如果这算作 的重复。
编辑
和 this 到那个时候可能会有用。
难道我根本不需要这三行吗？
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
正如评论中所写，使用和不使用这三行的区别在于保存的照片具有黑色背景+ AR-GUI。上面没有第二行代码的是带有 AR-GUI 的照片，但背景是我的计算机网络摄像头的实时流。我真的不想看到计算机网络摄像头，而是 HoloLens 看到的东西。
你的三行
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
对我来说意义不大。 Texture2D.ReadPixels 用于创建 Screenshot 这样您就可以用屏幕截图覆盖刚刚从 PhotoCapture 收到的纹理？ （也有不正确的尺寸，因为相机分辨率很可能！=屏幕分辨率。）
这也是
的原因
As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI.
做完后
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
您已经从 targetTexture 中的 PhotoCapture 收到了 Texture2D。
我认为您可能将它与用于获取给定 Texture2D.
的像素数据的 Texture2D.GetPixels 混淆了
I would like to crop the captured photo from the center in the end and am thinking that maybe that is possible with this code row? Beginning the new rect at other pixels than 0, 0)
您真正想要的是像您在评论中提到的那样从中心裁剪收到的 Texture2D。您可以使用 GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) 来做到这一点，它用于剪切给定 Texture2D
的特定区域
public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
if(input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
return null;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Create a new texture with newSize
var output = new Texture2D(newSize.x, newSize.y);
output.SetPixels(pixels);
output.Apply();
return output;
}
为了（希望）更好地理解这是一个说明 GetPixels 重载给定值的作用：
然后在
中使用它
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
// for example take only half of the textures width and height
targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
或者你可以把它变成 extension method 分开的 static class 比如
public static class Texture2DExtensions
{
public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
{
if (input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
return;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Resize the texture (creating a new one didn't work)
input.Resize(newSize.x, newSize.y);
input.SetPixels(pixels);
input.Apply(true);
}
}
并像
那样使用它
targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));
注：
UploadImageDataToTexture: You may only use this method if you specified the BGRA32 format in your CameraParameters.
幸运的是你无论如何都在使用它 ;)
Keep in mind that this operation will happen on the main thread and therefore be slow.
然而，唯一的选择是 CopyRawImageDataIntoBuffer 并在另一个线程或外部生成纹理，所以我认为留在 UploadImageDataToTexture 是可以的 ;)
和
The captured image will also appear flipped on the HoloLens. You can reorient the image by using a custom shader.
翻转实际上意味着纹理的 Y-Axis 颠倒了。 X-Axis 是正确的。
要垂直翻转纹理，您可以使用第二种扩展方法：
public static class Texture2DExtensions
{
public static void CropAroundCenter(){....}
public static void FlipVertically(this Texture2D texture)
{
var pixels = texture.GetPixels();
var flippedPixels = new Color[pixels.Length];
// These for loops are for running through each individual pixel and
// write them with inverted Y coordinates into the flippedPixels
for (var x = 0; x < texture.width; x++)
{
for (var y = 0; y < texture.height; y++)
{
var pixelIndex = x + y * texture.width;
var flippedIndex = x + (texture.height - 1 - y) * texture.width;
flippedPixels[flippedIndex] = pixels[pixelIndex];
}
}
texture.SetPixels(flippedPixels);
texture.Apply();
}
}
并像
一样使用它
targetTexture.FlipVertically();
结果：（对于此示例和给定的纹理，我每秒使用 FlipVertically 和 cropp 到一半大小，但对于拍摄的照片应该同样有效。）
图片来源：http://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf
更新
关于你的按钮问题：
不要使用
if (Input.GetKey("k") || Input.GetKey("k"))
首先，您要检查完全相同的条件两次。并且 GetKey 在按键保持按下时每帧 触发 。而是使用
if (Input.GetKeyDown("k"))
只触发一次。我猜 Vuforia 和 PhotoCapture 有问题，因为您的原始版本经常触发，也许您有一些并发的 PhotoCapture 进程...
如标题所示，我对第
行发生的错误有疑问targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
错误：
ReadPixels was called to read pixels from system frame buffer, while not inside drawing frame. UnityEngine.Texture2D:ReadPixels(Rect, Int32, Int32)
正如我从其他帖子中了解到的，解决此问题的一种方法是制作一个 Ienumerator 方法，该方法产生 return 新的 WaitForSeconds 或其他东西，并将其称为：StartCoroutine(methodname) 以便帧及时加载，以便有像素可读。
我不明白的是在下面的代码中这个方法最有意义。哪个部分没有及时加载？
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;
public string path = "";
CameraParameters cameraParameters = new CameraParameters();
private void Awake()
{
var cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);
// Create a PhotoCapture object
PhotoCapture.CreateAsync(false, captureObject =>
{
photoCaptureObject = captureObject;
cameraParameters.hologramOpacity = 0.0f;
cameraParameters.cameraResolutionWidth = cameraResolution.width;
cameraParameters.cameraResolutionHeight = cameraResolution.height;
cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;
});
}
private void Update()
{
// if not initialized yet don't take input
if (photoCaptureObject == null) return;
if (Input.GetKey("k") || Input.GetKey("k"))
{
Debug.Log("k was pressed");
VuforiaBehaviour.Instance.gameObject.SetActive(false);
// Activate the camera
photoCaptureObject.StartPhotoModeAsync(cameraParameters, result =>
{
if (result.success)
{
// Take a picture
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
else
{
Debug.LogError("Couldn't start photo mode!", this);
}
});
}
}
private static string FileName(int width, int height)
{
return $"screen_{width}x{height}_{DateTime.Now:yyyy-MM-dd_HH-mm-ss}.png";
}
private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
private void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result)
{
// Shutdown the photo capture resource
VuforiaBehaviour.Instance.gameObject.SetActive(true);
photoCaptureObject.Dispose();
photoCaptureObject = null;
}
抱歉，如果这算作
编辑
和 this 到那个时候可能会有用。
难道我根本不需要这三行吗？
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
正如评论中所写，使用和不使用这三行的区别在于保存的照片具有黑色背景+ AR-GUI。上面没有第二行代码的是带有 AR-GUI 的照片，但背景是我的计算机网络摄像头的实时流。我真的不想看到计算机网络摄像头，而是 HoloLens 看到的东西。
你的三行
Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
targetTexture.ReadPixels(new Rect(0, 0, cameraResolution.width, cameraResolution.height), 0, 0);
targetTexture.Apply();
对我来说意义不大。 Texture2D.ReadPixels 用于创建 Screenshot 这样您就可以用屏幕截图覆盖刚刚从 PhotoCapture 收到的纹理？ （也有不正确的尺寸，因为相机分辨率很可能！=屏幕分辨率。）
这也是
的原因As written in the comments the difference between using these three lines and not is that the photo saved has a black background + the AR-GUI.
做完后
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
您已经从 targetTexture 中的 PhotoCapture 收到了 Texture2D。
我认为您可能将它与用于获取给定 Texture2D.
I would like to crop the captured photo from the center in the end and am thinking that maybe that is possible with this code row? Beginning the new rect at other pixels than 0, 0)
您真正想要的是像您在评论中提到的那样从中心裁剪收到的 Texture2D。您可以使用 GetPixels(int x, int y, int blockWidth, int blockHeight, int miplevel) 来做到这一点，它用于剪切给定 Texture2D
public static Texture2D CropAroundCenter(Texture2D input, Vector2Int newSize)
{
if(input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!", this);
return null;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Create a new texture with newSize
var output = new Texture2D(newSize.x, newSize.y);
output.SetPixels(pixels);
output.Apply();
return output;
}
为了（希望）更好地理解这是一个说明 GetPixels 重载给定值的作用：
然后在
中使用它private void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
// Copy the raw image data into the target texture
photoCaptureFrame.UploadImageDataToTexture(targetTexture);
// for example take only half of the textures width and height
targetTexture = CropAroundCenter(targetTexture, new Vector2Int(targetTexture.width / 2, targetTexture.height / 2);
byte[] bytes = targetTexture.EncodeToPNG();
string filename = FileName(Convert.ToInt32(targetTexture.width), Convert.ToInt32(targetTexture.height));
//save to folder under assets
File.WriteAllBytes(Application.streamingAssetsPath + "/Snapshots/" + filename, bytes);
Debug.Log("The picture was uploaded");
// Deactivate the camera
photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}
或者你可以把它变成 extension method 分开的 static class 比如
public static class Texture2DExtensions
{
public static void CropAroundCenter(this Texture2D input, Vector2Int newSize)
{
if (input.width < newSize.x || input.height < newSize.y)
{
Debug.LogError("You can't cut out an area of an image which is bigger than the image itself!");
return;
}
// get the pixel coordinate of the center of the input texture
var center = new Vector2Int(input.width / 2, input.height / 2);
// Get pixels around center
// Get Pixels starts width 0,0 in the bottom left corner
// so as the name says, center.x,center.y would get the pixel in the center
// we want to start getting pixels from center - half of the newSize
//
// than from starting there we want to read newSize pixels in both dimensions
var pixels = input.GetPixels(center.x - newSize.x / 2, center.y - newSize.y / 2, newSize.x, newSize.y, 0);
// Resize the texture (creating a new one didn't work)
input.Resize(newSize.x, newSize.y);
input.SetPixels(pixels);
input.Apply(true);
}
}
并像
那样使用它targetTexture.CropAroundCenter(new Vector2Int(targetTexture.width / 2, targetTexture.height / 2));
注：
UploadImageDataToTexture: You may only use this method if you specified the BGRA32 format in your CameraParameters.
幸运的是你无论如何都在使用它 ;)
Keep in mind that this operation will happen on the main thread and therefore be slow.
然而，唯一的选择是 CopyRawImageDataIntoBuffer 并在另一个线程或外部生成纹理，所以我认为留在 UploadImageDataToTexture 是可以的 ;)
和
The captured image will also appear flipped on the HoloLens. You can reorient the image by using a custom shader.
翻转实际上意味着纹理的 Y-Axis 颠倒了。 X-Axis 是正确的。
要垂直翻转纹理，您可以使用第二种扩展方法：
public static class Texture2DExtensions
{
public static void CropAroundCenter(){....}
public static void FlipVertically(this Texture2D texture)
{
var pixels = texture.GetPixels();
var flippedPixels = new Color[pixels.Length];
// These for loops are for running through each individual pixel and
// write them with inverted Y coordinates into the flippedPixels
for (var x = 0; x < texture.width; x++)
{
for (var y = 0; y < texture.height; y++)
{
var pixelIndex = x + y * texture.width;
var flippedIndex = x + (texture.height - 1 - y) * texture.width;
flippedPixels[flippedIndex] = pixels[pixelIndex];
}
}
texture.SetPixels(flippedPixels);
texture.Apply();
}
}
并像
一样使用它targetTexture.FlipVertically();
结果：（对于此示例和给定的纹理，我每秒使用 FlipVertically 和 cropp 到一半大小，但对于拍摄的照片应该同样有效。）
图片来源：http://developer.vuforia.com/sites/default/files/sample-apps/targets/imagetargets_targets.pdf
更新
关于你的按钮问题：
不要使用
if (Input.GetKey("k") || Input.GetKey("k"))
首先，您要检查完全相同的条件两次。并且 GetKey 在按键保持按下时每帧 触发 。而是使用
if (Input.GetKeyDown("k"))
只触发一次。我猜 Vuforia 和 PhotoCapture 有问题，因为您的原始版本经常触发，也许您有一些并发的 PhotoCapture 进程...