gpu picking - 精灵周围的不可见像素

Question

我正在渲染一个包含精灵的采摘场景。当我的光标靠近精灵时，它注册为一种颜色并获得 "picked"。当您放大精灵时，这个不可见的边框会变大。

打开控制台查看实时打印的 ID。将光标移近或远离大精灵和小精灵。您会看到精灵在不可见的边框上被选中。这种行为不会发生在常规几何体上，只会发生在 sprite 上。

这很奇怪，因为我正在渲染 renderer.readRenderTargetPixels 实际看到的内容。

如何去除不可见的边界以更准确地挑选？

var renderer, scene, camera, controls;

var particles, uniforms;

var PARTICLE_SIZE = 50;

var raycaster, intersects;
var mouse, INTERSECTED;

var pickingTexture;

var numOfVertices;

init();
animate();

function init() {

    container = document.getElementById('container');

    scene = new THREE.Scene();

    camera = new THREE.PerspectiveCamera(45, window.innerWidth / window.innerHeight, 1, 10000);
    camera.position.z = 150;

    //

    var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
    var vertices = geometry1.vertices;
    numOfVertices = vertices.length;

    var positions = new Float32Array(vertices.length * 3);
    var colors = new Float32Array(vertices.length * 3);
    var sizes = new Float32Array(vertices.length);

    var vertex;
    var color = new THREE.Color();

    for (var i = 0, l = vertices.length; i < l; i++) {

        vertex = vertices[i];
        vertex.toArray(positions, i * 3);

        color.setHex(i + 1);
        color.toArray(colors, i * 3);

        sizes[i] = PARTICLE_SIZE * 0.5;

    }

    var geometry = new THREE.BufferGeometry();
    geometry.addAttribute('position', new THREE.BufferAttribute(positions, 3));
    geometry.addAttribute('customColor', new THREE.BufferAttribute(colors, 3));
    geometry.addAttribute('size', new THREE.BufferAttribute(sizes, 1));

    //

    var material = new THREE.ShaderMaterial({

        uniforms: {
//                texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
            texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/disc.png")}
        },
        vertexShader: document.getElementById('vertexshader').textContent,
        fragmentShader: document.getElementById('fragmentshader').textContent,
        depthTest: false,
        transparent: false
//            alphaTest: 0.9

    });

    //

    particles = new THREE.Points(geometry, material);
    scene.add(particles);

    //

    renderer = new THREE.WebGLRenderer({
        antialias: true,
        alpha: true
    });
    renderer.setPixelRatio(window.devicePixelRatio);
    renderer.setSize(window.innerWidth, window.innerHeight);
    renderer.setClearColor(0xffffff);
    container.appendChild(renderer.domElement);

    //

    raycaster = new THREE.Raycaster();
    mouse = new THREE.Vector2();

    //


    //

    window.addEventListener('resize', onWindowResize, false);
    document.addEventListener('mousemove', onDocumentMouseMove, false);

    // defaults are on the right (except minFilter)
    var options = {
        format: THREE.RGBAFormat,       // THREE.RGBAFormat
        type: THREE.UnsignedByteType,   // THREE.UnsignedByteType
        anisotropy: 1,                  // 1
        magFilter: THREE.LinearFilter,  // THREE.LinearFilter
        minFilter: THREE.LinearFilter,  // THREE.LinearFilter
        depthBuffer: true,              // true
        stencilBuffer: true             // true
    };

    pickingTexture = new THREE.WebGLRenderTarget(window.innerWidth, window.innerHeight, options);
    pickingTexture.texture.generateMipmaps = false;

    controls = new THREE.OrbitControls(camera, container);
    controls.damping = 0.2;
    controls.enableDamping = false;

}

function onDocumentMouseMove(e) {

//        event.preventDefault();

    mouse.x = e.clientX;
    mouse.y = e.clientY;

}

function onWindowResize() {

    camera.aspect = window.innerWidth / window.innerHeight;
    camera.updateProjectionMatrix();

    renderer.setSize(window.innerWidth, window.innerHeight);

}

function animate() {

    requestAnimationFrame(animate);


    controls.update();

    render();

}

function render() {

    pick();
    renderer.render(scene, camera);


}

function pick() {

    renderer.render(scene, camera, pickingTexture);

    //create buffer for reading single pixel
    var pixelBuffer = new Uint8Array(4);

    //read the pixel under the mouse from the texture
    renderer.readRenderTargetPixels(pickingTexture, mouse.x, pickingTexture.height - mouse.y, 1, 1, pixelBuffer);

    //interpret the pixel as an ID

    var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
    if (id <= numOfVertices) console.log(id);

}

body {
    color: #ffffff;
    background-color: #000000;
    margin: 0px;
    overflow: hidden;
}

<script src="http://threejs.org/build/three.min.js"></script>
<script src="http://threejs.org/examples/js/controls/OrbitControls.js"></script>



<script type="x-shader/x-fragment" id="fragmentshader">

uniform sampler2D texture;
varying vec3 vColor;

void main() {

    // solid squares of color
    gl_FragColor = vec4( vColor, 1.0 );

}

</script>

<script type="x-shader/x-vertex" id="vertexshader">

attribute float size;
attribute vec3 customColor;
varying vec3 vColor;

void main() {

    vColor = customColor;

    vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );

    gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );

    gl_Position = projectionMatrix * mvPosition;

}

</script>
<div id="container"></div>

Answer 1

问题是您使用的设备的 devicePixelRatio != 1.0 和 three.js 大小不符。

因为您调用了 renderer.setPixelRatio，现在幕后发生了奇迹。您的 canvas 不是您要求的尺寸，它是根据隐藏在 three.js 代码中的某些公式得出的其他尺寸。

那么，会发生什么。您的 canvas 是一种尺寸，但您的渲染目标是不同的尺寸。您的着色器使用 gl_PointSize 来绘制它的点。该大小以设备像素为单位。因为您的渲染目标大小不同，所以渲染目标中的点大小与屏幕上的点大小不同。

删除对 render.setPixelRatio 的调用，它将开始工作。

IMO 解决此问题的正确方法是使用 devicePixelRatio 自己，因为这样发生的一切对您都是 100% 可见的。幕后没有魔法发生。

所以，

去掉容器直接用一个canvas
```
<canvas id="c"></canvas>
```
设置 canvas 宽度使用 100vw，高度使用 100vh 并使正文 margin: 0;
```
canvas { width: 100vw; height: 100vh; display: block; }
body { margin: 0; }
```
这将使您的 canvas 自动伸展以填充 window。

使用浏览器拉伸 canvas 的大小来选择其 drawingBuffer 应该的大小并乘以 devicePixelRatio。这假设您实际上想要支持设备像素比。无需在 D.R.Y. 之后执行两次，因此只需在 onWindowResize 中执行即可。

    canvas = document.getElementById("c");
    renderer = new THREE.WebGLRenderer({
        antialias: true,
        alpha: true,
        canvas: canvas,
    });
    pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);

    onWindowResize(); 

...

function onWindowResize() {

    var width = canvas.clientWidth * window.devicePixelRatio;
    var height = canvas.clientHeight * window.devicePixelRatio;

    camera.aspect = width / height;
    camera.updateProjectionMatrix();

    renderer.setSize(width, height, false);  // YOU MUST PASS FALSE HERE otherwise three.js will muck with the CSS
    pickingTexture.setSize(width, height);  
}

将鼠标坐标转换为设备坐标

    renderer.readRenderTargetPixels(
        pickingTexture, 
        mouse.x * window.devicePixelRatio, 
        pickingTexture.height - mouse.y * window.devicePixelRatio,
        1, 1, pixelBuffer);

这是解决方案

var renderer, scene, camera, controls;

var particles, uniforms;

var PARTICLE_SIZE = 50;

var raycaster, intersects;
var mouse, INTERSECTED;

var pickingTexture;

var numOfVertices;
var info = document.querySelector('#info');

init();
animate();

function init() {

    canvas = document.getElementById('c');

    scene = new THREE.Scene();

    camera = new THREE.PerspectiveCamera(45, 1, 1, 10000);
    camera.position.z = 150;

    //

    var geometry1 = new THREE.BoxGeometry(200, 200, 200, 4, 4, 4);
    var vertices = geometry1.vertices;
    numOfVertices = vertices.length;

    var positions = new Float32Array(vertices.length * 3);
    var colors = new Float32Array(vertices.length * 3);
    var sizes = new Float32Array(vertices.length);

    var vertex;
    var color = new THREE.Color();

    for (var i = 0, l = vertices.length; i < l; i++) {

        vertex = vertices[i];
        vertex.toArray(positions, i * 3);

        color.setHex(i + 1);
        color.toArray(colors, i * 3);

        sizes[i] = PARTICLE_SIZE * 0.5;

    }

    var geometry = new THREE.BufferGeometry();
    geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
    geometry.setAttribute('customColor', new THREE.BufferAttribute(colors, 3));
    geometry.setAttribute('size', new THREE.BufferAttribute(sizes, 1));

    //

    var loader = new THREE.TextureLoader();
    var material = new THREE.ShaderMaterial({

        uniforms: {
//                texture: {type: "t", value: THREE.ImageUtils.loadTexture("../textures/circle.png")}
            texture: {value: loader.load("https://i.imgur.com/iXT97XR.png")}
        },
        vertexShader: document.getElementById('vertexshader').textContent,
        fragmentShader: document.getElementById('fragmentshader').textContent,
        depthTest: false,
        transparent: false
//            alphaTest: 0.9

    });

    //

    particles = new THREE.Points(geometry, material);
    scene.add(particles);

    //

    renderer = new THREE.WebGLRenderer({
        antialias: true,
        alpha: true,
        canvas: canvas,
    });
    renderer.setClearColor(0xffffff);
    //

    raycaster = new THREE.Raycaster();
    mouse = new THREE.Vector2();

    //


    //

    window.addEventListener('resize', onWindowResize, false);
    document.addEventListener('mousemove', onDocumentMouseMove, false);

    // defaults are on the right (except minFilter)
    var options = {
        format: THREE.RGBAFormat,       // THREE.RGBAFormat
        type: THREE.UnsignedByteType,   // THREE.UnsignedByteType
        anisotropy: 1,                  // 1
        magFilter: THREE.LinearFilter,  // THREE.LinearFilter
        minFilter: THREE.LinearFilter,  // THREE.LinearFilter
        depthBuffer: true,              // true
        stencilBuffer: true             // true
    };

    pickingTexture = new THREE.WebGLRenderTarget(1, 1, options);
    pickingTexture.texture.generateMipmaps = false;

    controls = new THREE.OrbitControls(camera, canvas);
    controls.damping = 0.2;
    controls.enableDamping = false;

    onWindowResize();

}

function onDocumentMouseMove(e) {

//        event.preventDefault();

    mouse.x = e.clientX;
    mouse.y = e.clientY;

}

function onWindowResize() {

    var width = canvas.clientWidth * window.devicePixelRatio;
    var height = canvas.clientHeight * window.devicePixelRatio;

    camera.aspect = width / height;
    camera.updateProjectionMatrix();

    renderer.setSize(width, height, false);  // YOU MUST PASS FALSE HERE!
    pickingTexture.setSize(width, height);  
}

function animate() {

    requestAnimationFrame(animate);


    controls.update();

    render();

}

function render() {

    pick();
    renderer.render(scene, camera);


}

function pick() {
    renderer.setRenderTarget(pickingTexture);
    renderer.setClearColor(0);
    renderer.render(scene, camera);
    renderer.setClearColor(0xFFFFFF);
    renderer.setRenderTarget(null)

    //create buffer for reading single pixel
    var pixelBuffer = new Uint8Array(4);

    //read the pixel under the mouse from the texture
    renderer.readRenderTargetPixels(pickingTexture, mouse.x * window.devicePixelRatio, pickingTexture.height - mouse.y * window.devicePixelRatio, 1, 1, pixelBuffer);

    //interpret the pixel as an ID

    var id = ( pixelBuffer[0] << 16 ) | ( pixelBuffer[1] << 8 ) | ( pixelBuffer[2] );
    //if (id > 0) console.log(id);
    info.textContent = id;

}

body {
    color: #ffffff;
    background-color: #000000;
    margin: 0;
}
canvas { width: 100vw; height: 100vh; display: block; }
#info { position: absolute; left: 0; top: 0; color: red; background: black; padding: 0.5em; font-family: monospace; }

<script src="https://threejs.org/build/three.min.js"></script>
<script src="https://threejs.org/examples/js/controls/OrbitControls.js"></script>



<script type="x-shader/x-fragment" id="fragmentshader">

uniform sampler2D texture;
varying vec3 vColor;

void main() {

    // solid squares of color
    gl_FragColor = vec4( vColor, 1.0 );

}

</script>

<script type="x-shader/x-vertex" id="vertexshader">

attribute float size;
attribute vec3 customColor;
varying vec3 vColor;

void main() {

    vColor = customColor;

    vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );

    gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) );

    gl_Position = projectionMatrix * mvPosition;

}

</script>
<canvas id="c"></canvas>
<div id="info"></div>

注意其他一些事情。

我猜你真的想将拾取纹理清除为零而不是白色。这样 0 = 那里什么都没有，其他 = 那里有东西。
```
renderer.setClearColor(0);
renderer.render(scene, camera, pickingTexture);
renderer.setClearColor(0xFFFFFF);
```
不知道 id <= numOfVertices 是什么意思

鉴于现在它正在清零，代码只是
```
if (id) console.log(id);
```
我没有在初始化时设置渲染器大小、pickingTexture 大小和相机方面。

为什么要重复我自己。 onWindowResize已经设置了
当 canvas 调整大小时，您需要调整 pickingTexture 渲染目标的大小以匹配大小。
我删除了对 window.innerWidth 和 window.innerHeight
的大部分引用
我会删除所有这些，但我不想为此示例更改更多代码。使用 window.innerWidth 将代码绑定到 window。如果您想要在不是 window 的全尺寸的东西中使用代码，例如假设您制作了 an editor。您必须更改代码。

以适用于更多情况的方式编写代码并不难，所以为什么以后要为自己做更多的工作。

其他我没有选择的方案

您可以调用 render.setPixelRatio，然后使用 window.devicePixelRatio
设置 pickingTexture 渲染目标的大小
我没有选择这个解决方案，因为你必须猜测 three.js 在幕后做了什么。您的猜测今天可能是正确的，但明天可能是错误的。如果你告诉 three.js 制作一些东西 width by height 似乎更好，它应该只制作它 width by height 而不是制作其他东西。同样，您必须猜测 three.js 何时应用 pixelRatio 以及何时不应用。正如您在上面注意到的那样，它不会将其应用于渲染目标的大小，而且它不能，因为它不知道您的目的是什么。你正在制作一个渲染目标来挑选吗？为了全屏效果？为了抓捕？对于非全屏效果？因为它不知道它不能为你应用 pixelRatio。这发生在整个 three.js 代码中。有些地方它应用 pixelRatio，有些地方不应用。你只能猜测。如果您从未设置 pixelRatio，那么问题就会消失。
您可以将 devicePixelRatio 传入您的着色器
```
<script type="x-shader/x-vertex" id="vertexshader">

attribute float size;
attribute vec3 customColor;
varying vec3 vColor;
uniform float devicePixelRatio;  // added

void main() {
    vColor = customColor;
    vec4 mvPosition = modelViewMatrix * vec4( position, 1.0 );
    gl_PointSize = size * ( 300.0 / length( mvPosition.xyz ) ) * devicePixelRatio;
    gl_Position = projectionMatrix * mvPosition;

}
</script>
```
当然你需要在制服上设置devicePixelRatio。

我可能选择这个解决方案。小问题是，如果 pickingTexture 与 canvas 的后备缓冲区的分辨率不同，您可能会遇到 1 个错误。在这种情况下，如果 canvas 是 pickingTexture 的 2 倍，那么 canvas 中每 4 个像素中的 3 个像素不存在于 pickingTexture 中。取决于您的应用程序，这可能没问题。你不能选择 1/2 像素，至少不能用鼠标。

我可能不会选择此解决方案的另一个原因是它只会让问题在其他地方弹出。 lineWidth 是一个，gl_FragCoord 是另一个。视口和剪刀设置也是如此。使渲染目标大小与 canvas 匹配似乎更好，这样一切都是一样的，而不是做出越来越多的变通办法，并且必须记住在哪里使用一种尺寸与另一种尺寸。明天我开始使用PointsMaterial。它还存在与 devicePixelRatio 相关的问题。通过不调用 renderer.setPixelRatio，这些问题就会消失。

gpu picking - 精灵周围的不可见像素

gpu picking - invisible pixels around sprites

three.js

picking

mouse-picking