WebGL 3d 用于深度排序 2d 对象

Question

这个问题和我的另一个问题有很强的联系： Isometric rendering without tiles, is that goal reachable?

我想对等距世界中的对象进行深度排序 (html5 canvas)。世界不是平铺的，因此世界中的每个项目都可以放置在每个 x、y、z 坐标上。由于它不是一个平铺的世界，深度排序很难做到。我什至希望如果项目相交，则绘制可见部分就好像它是全 3d 世界中的相交部分一样。正如人们在我的另一个问题中回答的那样，这可以通过将每个 2d 图像表示为 3d 模型来完成。我想继续使用以下对该问题的评论中给出的解决方案：

You don't have to work in 3D when you use webGL. WebGL draws polygons and is very quick at drawing 2D images as 4 verts making a small fan of triangles. You can still use the zbuffer and set corners (verts) to the z distance. Most of the 2D game libraries use webGL to render 2D and fallback to canvas if webGL is not there. There is also a webGL implementation of the canvas API on github that you could modify to meet your needs. (comment link)

因此，您可以将 'logic' 视为 3d 模型。 webGL 的 z-buffer 提供了正确的渲染。渲染像素本身是二维图像的像素。但我不知道该怎么做。有人可以进一步解释如何完成这项工作吗？我看了很多资料，但都是关于真正的 3d 的。

Answer 1

可以像您在其他问题中指出的那样使用深度精灵（ps，您真的应该将这些图像放在这个问题中）

要使用深度精灵，您需要启用 EXT_frag_depth 扩展（如果存在）。然后你可以在片段着色器中写入 gl_fragDepthEXT 。制作深度精灵对我来说听起来比制作 3D 模型更费工夫。

在那种情况下，您只需为每个精灵加载 2 个纹理，一个用于颜色，一个用于深度，然后执行类似

的操作

 #extension GL_EXT_frag_depth : require

 varying vec2 texcoord;

 uniform sampler2D colorTexture;
 uniform sampler2D depthTexture;
 uniform float depthScale;
 uniform float depthOffset;

 void main() {
   vec4 color = texture2D(colorTexture, texcoord);

   // don't draw if transparent
   if (color.a <= 0.01) {
     discard;
   }

   gl_FragColor = color;

   float depth = texture2D(depthTexture, texcoord).r;
   gl_FragDepthEXT = depthOffset - depth * depthScale;
 }

您可以将 depthOffset 和 depthScale 设置为

 var yTemp = yPosOfSpriteInPixelsFromTopOfScreen + tallestSpriteHeight;
 var depthOffset = 1. - yTemp / 65536;
 var depthScale = 1 / 256;

假设深度纹理中的每个值随着深度变化而变小。

至于如何在 WebGL 中绘制 2D see this article。

这是一个似乎可行的示例。我生成图像是因为我懒得在 photoshop 中绘制它。手动绘制深度值非常乏味。它假定图像中最远的像素的深度值为 1，下一个最近的像素的深度值为 2，依此类推。

换句话说，如果你有一个 3x3 的小立方体，深度值将类似于

+---+---+---+---+---+---+---+---+---+---+
|   |   |   |   | 1 | 1 |   |   |   |   |
+---+---+---+---+---+---+---+---+---+---+
|   |   | 2 | 2 | 2 | 2 | 2 | 2 |   |   |
+---+---+---+---+---+---+---+---+---+---+
| 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 |
+---+---+---+---+---+---+---+---+---+---+
| 3 | 3 | 4 | 4 | 4 | 4 | 4 | 4 | 3 | 3 |
+---+---+---+---+---+---+---+---+---+---+
| 3 | 3 | 4 | 4 | 5 | 5 | 4 | 4 | 3 | 3 |
+---+---+---+---+---+---+---+---+---+---+
| 3 | 3 | 4 | 4 | 5 | 5 | 4 | 4 | 3 | 3 |
+---+---+---+---+---+---+---+---+---+---+
| 3 | 3 | 4 | 4 | 5 | 5 | 4 | 4 | 3 | 3 |
+---+---+---+---+---+---+---+---+---+---+
|   |   | 4 | 4 | 5 | 5 | 4 | 4 |   |   |
+---+---+---+---+---+---+---+---+---+---+
|   |   |   |   | 5 | 5 |   |   |   |   |
+---+---+---+---+---+---+---+---+---+---+

function makeDepthColor(depth) {
  return "rgb(" + depth + "," + depth + "," + depth + ")";
}

function makeSprite(ctx, depth) {
  // make an image (these would be made in photoshop ro
  // some other paint program but that's too much work for me
  ctx.canvas.width = 64;
  ctx.canvas.height = 64;
  for (y = 0; y <= 32; ++y) {
    var halfWidth = (y < 16 ? 1 + y : 33 - y) * 2;
    var width = halfWidth * 2;
    var cy = (16 - y);
    var cw = Math.max(0, 12 - Math.abs(cy) * 2) | 0;
    for (var x = 0; x < width; ++x) {
      var cx = x - halfWidth;
      var inCenter = Math.abs(cy) < 6 && Math.abs(cx) <= cw;
      var onEdge = x < 2 || x >= width - 2 || (inCenter && (Math.abs(cx / 2) | 0) === (cw / 2 | 0));
      var height = onEdge ? 12 : (inCenter ? 30 : 10);
      var color = inCenter ? (cx < 0 ? "#F44" : "#F66") : (cx < 0 ? "#44F" : "#66F");
      ctx.fillStyle = depth ? makeDepthColor(y + 1) : color;
      var xx = 32 - halfWidth + x;
      var yy = y;
      ctx.fillRect(xx, yy + 32 - height, 1, height);
      if (!depth) {
        ctx.fillStyle = onEdge ? "black" : "#CCF";
        ctx.fillRect(xx, yy + 32 - height, 1, 1);
      }
    }
  }
}

function main() {
  var m4 = twgl.m4;
  var gl = document.querySelector("canvas").getContext(
    "webgl", {preserveDrawingBuffer: true});
  var ext = gl.getExtension("EXT_frag_depth");
  if (!ext) {
    alert("need EXT_frag_depth");
    return;
  }

  var vs = `
    attribute vec4 position;
    attribute vec2 texcoord;

    varying vec2 v_texcoord;

    uniform mat4 u_matrix;
    uniform mat4 u_textureMatrix;

    void main() {
      v_texcoord = (u_textureMatrix * vec4(texcoord, 0, 1)).xy;
      gl_Position = u_matrix * position;
    }
  `;

  var fs = `
    #extension GL_EXT_frag_depth : require

    precision mediump float;

    varying vec2 v_texcoord;

    uniform sampler2D u_colorTexture;
    uniform sampler2D u_depthTexture;
    uniform float u_depthScale;
    uniform float u_depthOffset;

    void main() {
      vec4 color = texture2D(u_colorTexture, v_texcoord);
      if (color.a < 0.01) {
        discard;
      }

      float depth = texture2D(u_depthTexture, v_texcoord).r;
      gl_FragDepthEXT = u_depthOffset - depth * u_depthScale;
      gl_FragColor = color;
    }
  `;

  var programInfo = twgl.createProgramInfo(gl, [vs, fs]);
  var quadBufferInfo = twgl.createBufferInfoFromArrays(gl, {
    position: {
      numComponents: 2,
      data: [
        0, 0,
        0, 1,
        1, 0,
        1, 0,
        0, 1,
        1, 1,
      ],
    },
    texcoord: [
      0, 0,
      0, 1,
      1, 0,
      1, 0,
      0, 1,
      1, 1,
    ],
  });

  var ctx = document.createElement("canvas").getContext("2d");

  // make the color texture
  makeSprite(ctx, false);
  var colorTexture = twgl.createTexture(gl, {
    src: ctx.canvas,
    min: gl.NEAREST,
    mag: gl.NEAREST,
  });

  // make the depth texture
  makeSprite(ctx, true);
  var depthTexture = twgl.createTexture(gl, {
    src: ctx.canvas,
    format: gl.LUMINANCE,  // because depth is only 1 channel
    min: gl.NEAREST,
    mag: gl.NEAREST,
  });

  function drawDepthImage(
      colorTex, depthTex, texWidth, texHeight,
      x, y, z) {
    var dstY = y + z;
    var dstX = x;
    var dstWidth = texWidth;
    var dstHeight = texHeight;

    var srcX = 0;
    var srcY = 0;
    var srcWidth = texWidth;
    var srcHeight = texHeight;

    gl.useProgram(programInfo.program);

    twgl.setBuffersAndAttributes(gl, programInfo, quadBufferInfo);

    // this matirx will convert from pixels to clip space
    var matrix = m4.ortho(0, gl.canvas.width, gl.canvas.height, 0, -1, 1);

    // this matrix will translate our quad to dstX, dstY
    matrix = m4.translate(matrix, [dstX, dstY, 0]);

    // this matrix will scale our 1 unit quad
    // from 1 unit to texWidth, texHeight units
    matrix = m4.scale(matrix, [dstWidth, dstHeight, 1]);

    // just like a 2d projection matrix except in texture space (0 to 1)
    // instead of clip space. This matrix puts us in pixel space.
    var texMatrix = m4.scaling([1 / texWidth, 1 / texHeight, 1]);

    // because were in pixel space
    // the scale and translation are now in pixels
    var texMatrix = m4.translate(texMatrix, [srcX, srcY, 0]);
    var texMatrix = m4.scale(texMatrix, [srcWidth, srcHeight, 1]);

    twgl.setUniforms(programInfo, {
      u_colorTexture: colorTex,
      u_depthTexture: depthTex,
      u_matrix: matrix,
      u_textureMatrix: texMatrix,
      u_depthOffset: 1 - (dstY - z) / 65536,
      u_depthScale: 1 / 256,
    });

    twgl.drawBufferInfo(gl, quadBufferInfo);
  }

  // test render
  gl.enable(gl.DEPTH_TEST);

  var texWidth = 64;
  var texHeight = 64;

  // z is how much above/below ground
  function draw(x, y, z) {
    drawDepthImage(colorTexture, depthTexture, texWidth, texHeight , x, y, z);
  }

  draw(  0, 0, 0);  // draw on left

  draw(100, 0, 0);  // draw near center
  draw(113, 0, 0);  // draw overlapping

  draw(200, 0, 0);  // draw on right
  draw(200, 8, 0);  // draw on more forward

  draw(0, 60,  0);  // draw on left
  draw(0, 60, 10);  // draw on below

  draw(100, 60,  0);  // draw near center
  draw(100, 60, 20);  // draw below

  draw(200, 60, 20);  // draw on right
  draw(200, 60,  0);  // draw above
}

main();

<script src="https://twgljs.org/dist/2.x/twgl-full.min.js"></script>
<canvas></canvas>

左上角是图片的样子。顶部中间是并排绘制的 2 张图像。右上角是 2 个图像，在 y 中进一步向下绘制（x，y 是等平面）。左下方是两个图像，一个绘制在另一个下方（在平面下方）。底部中间是同一件事，只是分开更多。右下角是相同的东西，只是以相反的顺序绘制（只是为了检查它是否有效）

为了节省内存，您可以将深度值放在颜色纹理的 Alpha 通道中。如果是0则丢弃。

不幸的是，根据 webglstats.com 只有 75% 的 desktops 和 0% 的手机支持 EXT_frag_depth。虽然 WebGL2 需要支持 gl_FragDepth 和 AFAIK 大多数手机支持 OpenGL ES 3.0，WebGL2 是基于它的，所以在接下来的几个月里，大多数 Android 手机和大多数 PC 将获得 WebGL2。 iOS 另一方面，和往常一样，Apple 对何时在 iOS 上发布 WebGL2 保密。很明显，他们从未计划发布 WebGL2，因为 2 年多来还没有针对 WebGL2 的 WebKit 提交。

对于不支持 WebGL2 或 EXT_frag_depth WebGL1 的系统，您可以使用顶点着色器模拟 EXT_frag_depth。您会将深度纹理传递给顶点着色器并使用 gl.POINTS 绘制，每个像素一个点。这样你就可以选择每个点的深度。

它可以工作，但最终可能会很慢。可能比 JavaScript 直接写入数组并使用 Canvas2DRenderingContext.putImageData

慢

这是一个例子

function makeDepthColor(depth) {
  return "rgb(" + depth + "," + depth + "," + depth + ")";
}

function makeSprite(ctx, depth) {
  // make an image (these would be made in photoshop ro
  // some other paint program but that's too much work for me
  ctx.canvas.width = 64;
  ctx.canvas.height = 64;
  for (y = 0; y <= 32; ++y) {
    var halfWidth = (y < 16 ? 1 + y : 33 - y) * 2;
    var width = halfWidth * 2;
    var cy = (16 - y);
    var cw = Math.max(0, 12 - Math.abs(cy) * 2) | 0;
    for (var x = 0; x < width; ++x) {
      var cx = x - halfWidth;
      var inCenter = Math.abs(cy) < 6 && Math.abs(cx) <= cw;
      var onEdge = x < 2 || x >= width - 2 || (inCenter && (Math.abs(cx / 2) | 0) === (cw / 2 | 0));
      var height = onEdge ? 12 : (inCenter ? 30 : 10);
      var color = inCenter ? (cx < 0 ? "#F44" : "#F66") : (cx < 0 ? "#44F" : "#66F");
      ctx.fillStyle = depth ? makeDepthColor(y + 1) : color;
      var xx = 32 - halfWidth + x;
      var yy = y;
      ctx.fillRect(xx, yy + 32 - height, 1, height);
      if (!depth) {
        ctx.fillStyle = onEdge ? "black" : "#CCF";
        ctx.fillRect(xx, yy + 32 - height, 1, 1);
      }
    }
  }
}

function main() {
  var m4 = twgl.m4;
  var gl = document.querySelector("canvas").getContext(
    "webgl", {preserveDrawingBuffer: true});
  var numVertexTextures = gl.getParameter(gl.MAX_VERTEX_TEXTURE_IMAGE_UNITS);
  if (numVertexTextures < 2) {
    alert("GPU doesn't support textures in vertex shaders");
    return;
  }

  var vs = `
    attribute float count;

    uniform vec2 u_dstSize;
    uniform mat4 u_matrix;
    uniform mat4 u_textureMatrix;
    uniform sampler2D u_colorTexture;
    uniform sampler2D u_depthTexture;
    uniform float u_depthScale;
    uniform float u_depthOffset;

    varying vec4 v_color;

    void main() {
      float px = mod(count, u_dstSize.x);
      float py = floor(count / u_dstSize.x);

      vec4 position = vec4((vec2(px, py) + 0.5) / u_dstSize, 0, 1);
      vec2 texcoord = (u_textureMatrix * position).xy;

      float depth = texture2D(u_depthTexture, texcoord).r;

      gl_Position = u_matrix * position;
      gl_Position.z = u_depthOffset - depth * u_depthScale;
      v_color = texture2D(u_colorTexture, texcoord);
    }
  `;

  var fs = `
    precision mediump float;

    varying vec4 v_color;

    void main() {
      if (v_color.a < 0.01) {
        discard;
      }
      gl_FragColor = v_color;
    }
  `;
  
  // make a count
  var maxImageWidth = 256;
  var maxImageHeight = 256;
  var maxPixelsInImage = maxImageWidth * maxImageHeight
  var count = new Float32Array(maxPixelsInImage);
  for (var ii = 0; ii < count.length; ++ii) {
    count[ii] = ii;
  }

  var programInfo = twgl.createProgramInfo(gl, [vs, fs]);
  var quadBufferInfo = twgl.createBufferInfoFromArrays(gl, {
    count: { numComponents: 1, data: count, }
  });

  var ctx = document.createElement("canvas").getContext("2d");

  // make the color texture
  makeSprite(ctx, false);
  var colorTexture = twgl.createTexture(gl, {
    src: ctx.canvas,
    min: gl.NEAREST,
    mag: gl.NEAREST,
  });

  // make the depth texture
  makeSprite(ctx, true);
  var depthTexture = twgl.createTexture(gl, {
    src: ctx.canvas,
    format: gl.LUMINANCE,  // because depth is only 1 channel
    min: gl.NEAREST,
    mag: gl.NEAREST,
  });

  function drawDepthImage(
      colorTex, depthTex, texWidth, texHeight,
      x, y, z) {
    var dstY = y + z;
    var dstX = x;
    var dstWidth = texWidth;
    var dstHeight = texHeight;

    var srcX = 0;
    var srcY = 0;
    var srcWidth = texWidth;
    var srcHeight = texHeight;

    gl.useProgram(programInfo.program);

    twgl.setBuffersAndAttributes(gl, programInfo, quadBufferInfo);

    // this matirx will convert from pixels to clip space
    var matrix = m4.ortho(0, gl.canvas.width, gl.canvas.height, 0, -1, 1);

    // this matrix will translate our quad to dstX, dstY
    matrix = m4.translate(matrix, [dstX, dstY, 0]);

    // this matrix will scale our 1 unit quad
    // from 1 unit to texWidth, texHeight units
    matrix = m4.scale(matrix, [dstWidth, dstHeight, 1]);

    // just like a 2d projection matrix except in texture space (0 to 1)
    // instead of clip space. This matrix puts us in pixel space.
    var texMatrix = m4.scaling([1 / texWidth, 1 / texHeight, 1]);

    // because were in pixel space
    // the scale and translation are now in pixels
    var texMatrix = m4.translate(texMatrix, [srcX, srcY, 0]);
    var texMatrix = m4.scale(texMatrix, [srcWidth, srcHeight, 1]);

    twgl.setUniforms(programInfo, {
      u_colorTexture: colorTex,
      u_depthTexture: depthTex,
      u_matrix: matrix,
      u_textureMatrix: texMatrix,
      u_depthOffset: 1 - (dstY - z) / 65536,
      u_depthScale: 1 / 256,
      u_dstSize: [dstWidth, dstHeight],
    });

    var numDstPixels = dstWidth * dstHeight;
    twgl.drawBufferInfo(gl, quadBufferInfo, gl.POINTS, numDstPixels);
  }

  // test render
  gl.enable(gl.DEPTH_TEST);

  var texWidth = 64;
  var texHeight = 64;

  // z is how much above/below ground
  function draw(x, y, z) {
    drawDepthImage(colorTexture, depthTexture, texWidth, texHeight , x, y, z);
  }

  draw(  0, 0, 0);  // draw on left

  draw(100, 0, 0);  // draw near center
  draw(113, 0, 0);  // draw overlapping

  draw(200, 0, 0);  // draw on right
  draw(200, 8, 0);  // draw on more forward

  draw(0, 60,  0);  // draw on left
  draw(0, 60, 10);  // draw on below

  draw(100, 60,  0);  // draw near center
  draw(100, 60, 20);  // draw below

  draw(200, 60, 20);  // draw on right
  draw(200, 60,  0);  // draw above
}

main();

<script src="https://twgljs.org/dist/2.x/twgl-full.min.js"></script>
<canvas></canvas>

请注意，如果它太慢，我实际上并不认为在 JavaScript 中在软件中执行它一定会太慢。您可以使用 asm.js 制作渲染器。您设置和操作 JavaScript 中的数据，然后调用 asm.js 例程进行软件渲染。

举个例子this demo is entirely software rendered in asm.js as is this one

如果最终速度太慢，另一种方法将需要某种 3D 数据用于 2D 图像。如果 2D 图像总是立方体，你可以只使用立方体，但我已经从你的示例图片中看到这 2 个柜子需要 3D 模型，因为顶部比主体宽几个像素，背面有一个支撑梁。

无论如何，假设您为对象制作 3D 模型，您将使用模板缓冲区 + 深度缓冲区。

对于每个对象

打开STENCIL_TEST和DEPTH_TEST

gl.enable(gl.STENCIL_TEST);
gl.enable(gl.DEPTH_TEST);

将模板函数设置为ALWAYS，对迭代计数的引用，以及掩码为 255

var test = gl.ALWAYS;
var ref = ndx;  // 1 for object 1, 2 for object 2, etc.
var mask = 255;
gl.stencilFunc(test, ref, mask);

如果深度测试通过，则将模板操作设置为REPLACE 和 KEEP 否则

var stencilTestFailOp = gl.KEEP;
var depthTestFailOp = gl.KEEP;
var bothPassOp = gl.REPLACE;
gl.stencilOp(stencilTestFailOp, depthTestFailOp, bothPassOp);

现在绘制你的立方体（或任何代表你的 2D 图像的 3d 模型）

此时，模板缓冲区将在立方体绘制的任何地方都有一个带有 ref 的 2D 蒙版。所以现在使用模板绘制 2D 图像，只在成功绘制立方体的地方绘制

绘制图像

关闭 DEPTH_TEST
```
gl.disable(gl.DEPTH_TEST);
```

设置模板函数，这样我们只在模板等于ref

的地方绘制

var test = gl.EQUAL;
var mask = 255;
gl.stencilFunc(test, ref, mask);

将模板操作设置为 KEEP 对于所有情况

var stencilTestFailOp = gl.KEEP;
var depthTestFailOp = gl.KEEP;
var bothPassOp = gl.KEEP;
gl.stencilOp(stencilTestFailOp, depthTestFailOp, bothPassOp);

绘制二维图像

这最终只会在立方体绘制的地方绘制。

对每个对象重复。

您可能希望在每个对象之后或每 254 个对象之后清除模板缓冲区，并确保 ref 始终介于 1 和 255 之间，因为模板缓冲区只有 8 位，这意味着当您绘制对象 256它将使用与对象 #1 相同的值，因此如果模板缓冲区中有任何这些值，您可能会不小心在那里绘制。

 objects.forEach(object, ndx) {
    if (ndx % 255 === 0) {
      gl.clear(gl.STENCIL_BUFFER_BIT);
    }

    var ref = ndx % 255 + 1;  // 1 to 255

    ... do as above ...

Answer 2

您可以为每个对象单独 pre-rendered Depth-Maps 来做到这一点。使用额外的 Normal-Maps，您也可以模拟延迟光照。但是这种技术需要为每个对象创建 3D 以渲染漫反射、法线和深度图以正确相交对象。

在 https://www.youtube.com/watch?v=-Q6ISVaM5Ww

查看 XNA-Demo

此演示中的每个对象都只是一个具有漫反射、法线和深度贴图的渲染精灵。

在视频的最后你会看到它是如何工作的。作者在他的博客 https://infictitious.blogspot.de/2012/09/25d-xna-rpg-engine-some-technical.html

中有一个额外的解释和着色器 code-examples

我认为 WebGL 也可以做到这一点。

WebGL 3d 用于深度排序 2d 对象

WebGL 3d usage for depth sorting 2d objects

html

3d

canvas

depth-buffer

webgl