如何使 webworker 的 onmessage 阶段异步？

Question

我正在使用 webworker 来计算属于这些地方的坐标和值。计算完美地在后台进行，保持 DOM 响应。但是，当我将数据从 webworker 发送回主线程时，DOM 在部分传输时间内变得无响应。

我的webworker（发送部分）：

//calculates happen before; this is the final step to give the calculated data back to the mainthread.
var processProgressGEO = {'cmd':'geoReport', 'name': 'starting transfer to main', 'current': c, 'total': polys}
postMessage(processProgressGEO);
postMessage({
  'cmd':'heatmapCompleted',
  'heatdata': rehashedMap,
  'heatdatacount': p,
  'current': c,
  'total': polys,
  'heatmapPeak': peakHM,
});
self.close();

上面代码片段中的变量rehashedMap是一个带有数字键的对象。每个键包含一个 array 和另一个 object in.

我的主线程（只有相关的部分：）

var heatMaxforAuto = 1000000;  //maximum amount of datapoints allowed in the texdata. This takes into account the spread of a singel datapoint.
async function fetchHeatData(){
  return new Promise((resolve, reject) => {
    var numbercruncher = new Worker('calculator.js');
    console.log("Performing Second XHR request:");
    var url2 = 'backend.php?datarequest=geodata'
    $.ajax({
      type: "GET",
      url: url2,
    }).then(async function(RAWGEOdata) {
      data.georaw = RAWGEOdata;
      numbercruncher.onmessage = async function(e){
        var w = (e.data.current/e.data.total)*100+'%';
        if (e.data.cmd === 'geoReport'){
          console.log("HEAT: ", e.data.name, end(),'Sec.' );
        }else if (e.data.cmd === 'heatmapCompleted') {
          console.log("received Full heatmap data: "+end());
          data.heatmap = e.data.heatdata;
          console.log("heatData transfered", end());
          data.heatmapMaxValue = e.data.heatmapPeak;
          data.pointsInHeatmap = e.data.heatdatacount;
          console.log("killing worker");
          numbercruncher.terminate();
          resolve(1);
        }else{
          throw "Unexpected command received by worker: "+ e.data.cmd;
        }
      }
      console.log('send to worker')
      numbercruncher.postMessage({'mode':'geo', 'data':data});
    }).catch(function(error) {
      reject(0);
      throw error;
    })
  });
}

async function makemap(){
  let heatDone = false;
      if (data.texdatapoints<= heatMaxforAuto){
      heatDone = await fetchHeatData();
    }else{
      var manualHeatMapFetcher = document.createElement("BUTTON");
      var manualHeatMapFetcherText = document.createTextNode('Fetch records');
      manualHeatMapFetcher.appendChild(manualHeatMapFetcherText);
      manualHeatMapFetcher.id='manualHeatTriggerButton';
      manualHeatMapFetcher.addEventListener("click", async function(){
        $(this).toggleClass('hidden');
        heatDone = await fetchHeatData();
        console.log(heatDone, 'allIsDone', end());
      });
      document.getElementById("toggleIDheatmap").appendChild(manualHeatMapFetcher);
    }


}

makemap();

需要调用end()函数来计算webworker启动后的秒数。它 return 是全局设置开始时间和调用时间之间的差异。

我的控制台中显示的内容：

HEAT:  starting transfer to main 35 Sec.   animator.js:44:19
received Full heatmap data: 51             animator.js:47:19
heatData transfered 51                     animator.js:49:19
killing worker                             animator.js:52:19

1 allIsDone 51

问题： 我的 DOM 在数据传输开始和收到完整热图数据后的消息之间冻结。这是我的控制台中第一条消息和第二条消息之间的阶段。传输需要 16 秒，但 DOM 仅在一段时间内无响应一次。 Webworkers 无法与主线程共享数据，因此需要进行传输。

问题： 首先，如何防止在webworker的onmessage阶段冻结DOM？其次，更多的是出于好奇：这种冻结怎么可能只发生在那个阶段的一部分，因为这些是由两个连续的步骤触发的，中间没有任何事情发生？

到目前为止我尝试了什么：

对 rehashedMap 和 return 按键进行 for 循环。这仍然会触发 DOM 冻结；更短，但不止一次。在极少数情况下，它会取下标签。
寻找缓冲 onmessage 阶段的方法；但是，文档中没有指定此类选项 (https://developer.mozilla.org/en-US/docs/Web/API/Worker/onmessage) as compared to the postMessage phase (https://developer.mozilla.org/en-US/docs/Web/API/Worker/postMessage)。我在这里遗漏了什么吗？
作为测试，我用空对象替换了 rehashedMap；这并没有导致 DOM 中的任何冻结。当然，这让我无法访问计算数据。
我在 SO 上查看了这个线程：Javascript WebWorker - Async/Await 但我不确定如何将该上下文与我的进行比较。

Answer 1

选项

~~将此与网络工作者联系起来是可以理解的，但它可能与它没有任何关系。~~我错了，确实如此。我看到这个问题有两个可能的原因：

（我们知道这对 OP 来说不是真的，但可能对其他人仍然有意义。） 问题可能是你有很多DOM 收到热图后要进行的操作。如果您在一个从不让主线程做任何其他事情的紧密循环中执行此操作，则页面在此期间将没有响应。

如果发生这种情况，您必须找到一种方法来更快地进行 DOM 操作（有时可能，有时则不行），或者找到一种方法将其分割成块并进行处理每个块分开，在块之间返回给浏览器，以便浏览器可以处理任何未决的 UI 工作（包括渲染新元素）。

您还没有包括 DOM 正在使用热图完成的工作，因此实际上不可能给您代码来解决问题，但是“分割”将通过处理数据的一个子集然后使用 setTimeout(fn, 0)（可能与 requestAnimationFrame 结合以确保重绘已经发生）来安排继续工作（使用 fn ) 在短暂地屈服于浏览器之后。
如果确实是在worker和主线程之间传输数据所花费的时间，你也许可以使用transferable object for your heat map data rather than your current object, although doing so may require significantly changing your data structure. With a transferable object, you avoid copying the data from the worker to the main thread; instead, the worker transfers the actual memory to the main thread (the worker loses access to the transferable object, and the main thread gains access to it — all without copying it). For instance, the ArrayBuffer通过类型数组（Int32Array 等）是可转移的。
如果这真的是花在从工作人员那里接收数据的时间（从你的实验来看，它听起来是这样），并且使用可转让的不是一个选项（例如，因为你需要数据采用与可传输文件不兼容的格式），我能看到的唯一剩余选项是让工作人员向主脚本发送较小的数据块，这些数据块的间隔足以让主线程保持响应。（甚至可能在数据可用时发送数据。）

仔细看看#3

您描述了一个包含 1,600 个条目的数组，其中每个条目都是一个包含 0 到“远远超过 7,000”个对象的数组，每个对象具有三个属性（具有数值）。超过 5.6 百万个对象。克隆该数据需要相当多的时间也就不足为奇了。

这是您描述的问题的示例：

const workerCode = document.getElementById("worker").textContent;
const workerBlob = new Blob([workerCode], { type: "text/javascript" });
const workerUrl = (window.webkitURL || window.URL).createObjectURL(workerBlob);
const worker = new Worker(workerUrl);
worker.addEventListener("message", ({data}) => {
    if ((data && data.action) === "data") {
        console.log(Date.now(), `Received ${data.array.length} rows`);
        if (data.done) {
            stopSpinning();
        }
    }
});
document.getElementById("btn-go").addEventListener("click", () => {
    console.log(Date.now(), "requesting data");
    startSpinning();
    worker.postMessage({action: "go"});
});
const spinner = document.getElementById("spinner");
const states = [..."▁▂▃▄▅▆▇█▇▆▅▄▃▂▁"];
let stateIndex = 0;
let spinHandle = 0;
let maxDelay = 0;
let intervalStart = 0;
function startSpinning() {
    if (spinner) {
        cancelAnimationFrame(spinHandle);
        maxDelay = 0;
        queueUpdate();
    }
}
function queueUpdate() {
    intervalStart = Date.now();
    spinHandle = requestAnimationFrame(() => {
        updateMax();
        spinner.textContent = states[stateIndex];
        stateIndex = (stateIndex + 1) % states.length;
        if (spinHandle) {
            queueUpdate();
        }
    });
}
function stopSpinning() {
    updateMax();
    cancelAnimationFrame(spinHandle);
    spinHandle = 0;
    if (spinner) {
        spinner.textContent = "Done";
        console.log(`Max delay between frames: ${maxDelay}ms`);
    }
}
function updateMax() {
    if (intervalStart !== 0) {
        const elapsed = Date.now() - intervalStart;
        if (elapsed > maxDelay) {
            maxDelay = elapsed;
        }
    }
}

<div>(Look in the real browser console.)</div>
<input type="button" id="btn-go" value="Go">
<div id="spinner"></div>
<script type="worker" id="worker">
const r = Math.random;
self.addEventListener("message", ({data}) => {
    if ((data && data.action) === "go") {
        console.log(Date.now(), "building data");
        const array = Array.from({length: 1600}, () =>
            Array.from({length: Math.floor(r() * 7000)}, () => ({lat: r(), lng: r(), value: r()}))
        );
        console.log(Date.now(), "data built");
        console.log(Date.now(), "sending data");
        postMessage({
            action: "data",
            array,
            done: true
        });
        console.log(Date.now(), "data sent");
    }
});
</script>

这是工作人员尽可能快地分块发送数据但在单独的消息中的示例。它使页面在接收数据时响应（尽管仍然很紧张）：

const workerCode = document.getElementById("worker").textContent;
const workerBlob = new Blob([workerCode], { type: "text/javascript" });
const workerUrl = (window.webkitURL || window.URL).createObjectURL(workerBlob);
const worker = new Worker(workerUrl);
let array = null;
let clockTimeStart = 0;
worker.addEventListener("message", ({data}) => {
    if ((data && data.action) === "data") {
        if (clockTimeStart === 0) {
            clockTimeStart = Date.now();
            console.log(Date.now(), "Receiving data");
        }
        array.push(...data.array);
        if (data.done) {
            console.log(Date.now(), `Received ${array.length} row(s) in total, clock time to receive data: ${Date.now() - clockTimeStart}ms`);
            stopSpinning();
        }
    }
});
document.getElementById("btn-go").addEventListener("click", () => {
    console.log(Date.now(), "requesting data");
    array = [];
    clockTimeStart = 0;
    startSpinning();
    worker.postMessage({action: "go"});
});
const spinner = document.getElementById("spinner");
const states = [..."▁▂▃▄▅▆▇█▇▆▅▄▃▂▁"];
let stateIndex = 0;
let spinHandle = 0;
let maxDelay = 0;
let intervalStart = 0;
function startSpinning() {
    if (spinner) {
        cancelAnimationFrame(spinHandle);
        maxDelay = 0;
        queueUpdate();
    }
}
function queueUpdate() {
    intervalStart = Date.now();
    spinHandle = requestAnimationFrame(() => {
        updateMax();
        spinner.textContent = states[stateIndex];
        stateIndex = (stateIndex + 1) % states.length;
        if (spinHandle) {
            queueUpdate();
        }
    });
}
function stopSpinning() {
    updateMax();
    cancelAnimationFrame(spinHandle);
    spinHandle = 0;
    if (spinner) {
        spinner.textContent = "Done";
        console.log(`Max delay between frames: ${maxDelay}ms`);
    }
}
function updateMax() {
    if (intervalStart !== 0) {
        const elapsed = Date.now() - intervalStart;
        if (elapsed > maxDelay) {
            maxDelay = elapsed;
        }
    }
}

<div>(Look in the real browser console.)</div>
<input type="button" id="btn-go" value="Go">
<div id="spinner"></div>
<script type="worker" id="worker">
const r = Math.random;
self.addEventListener("message", ({data}) => {
    if ((data && data.action) === "go") {
        console.log(Date.now(), "building data");
        const array = Array.from({length: 1600}, () =>
            Array.from({length: Math.floor(r() * 7000)}, () => ({lat: r(), lng: r(), value: r()}))
        );
        console.log(Date.now(), "data built");
        const total = 1600;
        const chunks = 100;
        const perChunk = total / chunks;
        if (perChunk !== Math.floor(perChunk)) {
            throw new Error(`total = ${total}, chunks = ${chunks}, total / chunks has remainder`);
        }
        for (let n = 0; n < chunks; ++n) {
            postMessage({
                action: "data",
                array: array.slice(n * perChunk, (n + 1) * perChunk),
                done: n === chunks - 1
            });
        }
    }
});
</script>

自然是权衡取舍。块越小，接收数据所花费的总时钟时间越长；块越小，页面的抖动就越少。这是非常小的块（分别发送 1,600 个数组中的每一个）：

const workerCode = document.getElementById("worker").textContent;
const workerBlob = new Blob([workerCode], { type: "text/javascript" });
const workerUrl = (window.webkitURL || window.URL).createObjectURL(workerBlob);
const worker = new Worker(workerUrl);
let array = null;
let clockTimeStart = 0;
worker.addEventListener("message", ({data}) => {
    if ((data && data.action) === "data") {
        if (clockTimeStart === 0) {
            clockTimeStart = Date.now();
        }
        array.push(data.array);
        if (data.done) {
            console.log(`Received ${array.length} row(s) in total, clock time to receive data: ${Date.now() - clockTimeStart}ms`);
            stopSpinning();
        }
    }
});
document.getElementById("btn-go").addEventListener("click", () => {
    console.log(Date.now(), "requesting data");
    array = [];
    clockTimeStart = 0;
    startSpinning();
    worker.postMessage({action: "go"});
});
const spinner = document.getElementById("spinner");
const states = [..."▁▂▃▄▅▆▇█▇▆▅▄▃▂▁"];
let stateIndex = 0;
let spinHandle = 0;
let maxDelay = 0;
let intervalStart = 0;
function startSpinning() {
    if (spinner) {
        cancelAnimationFrame(spinHandle);
        maxDelay = 0;
        queueUpdate();
    }
}
function queueUpdate() {
    intervalStart = Date.now();
    spinHandle = requestAnimationFrame(() => {
        updateMax();
        spinner.textContent = states[stateIndex];
        stateIndex = (stateIndex + 1) % states.length;
        if (spinHandle) {
            queueUpdate();
        }
    });
}
function stopSpinning() {
    updateMax();
    cancelAnimationFrame(spinHandle);
    spinHandle = 0;
    if (spinner) {
        spinner.textContent = "Done";
        console.log(`Max delay between frames: ${maxDelay}ms`);
    }
}
function updateMax() {
    if (intervalStart !== 0) {
        const elapsed = Date.now() - intervalStart;
        if (elapsed > maxDelay) {
            maxDelay = elapsed;
        }
    }
}

<div>(Look in the real browser console.)</div>
<input type="button" id="btn-go" value="Go">
<div id="spinner"></div>
<script type="worker" id="worker">
const r = Math.random;
self.addEventListener("message", ({data}) => {
    if ((data && data.action) === "go") {
        console.log(Date.now(), "building data");
        const array = Array.from({length: 1600}, () =>
            Array.from({length: Math.floor(r() * 7000)}, () => ({lat: r(), lng: r(), value: r()}))
        );
        console.log(Date.now(), "data built");
        array.forEach((chunk, index) => {
            postMessage({
                action: "data",
                array: chunk,
                done: index === array.length - 1
            });
        });
    }
});
</script>

这是构建所有数据然后发送它，但是如果构建数据时间，穿插构建和发送它可能会使页面响应更流畅，特别是如果你可以将内部数组分成更小的部分发送（甚至发送~7,000 个对象仍然会导致抖动，正如我们在上面最后一个示例中看到的那样。

合并 #2 和 #3

主数组中的每个条目都是具有三个数字属性的对象数组。我们可以使用 lat/lng/value 顺序发送带有这些值的 Float64Arrays，因为它们是可转移的：

const workerCode = document.getElementById("worker").textContent;
const workerBlob = new Blob([workerCode], { type: "text/javascript" });
const workerUrl = (window.webkitURL || window.URL).createObjectURL(workerBlob);
const worker = new Worker(workerUrl);
let array = null;
let clockTimeStart = 0;
worker.addEventListener("message", ({data}) => {
    if ((data && data.action) === "data") {
        if (clockTimeStart === 0) {
            clockTimeStart = Date.now();
        }
        const nums = data.array;
        let n = 0;
        const entry = [];
        while (n < nums.length) {
            entry.push({
                lat: nums[n++],
                lng: nums[n++],
                value: nums[n++]
            });
        }
        array.push(entry);
        if (data.done) {
            console.log(Date.now(), `Received ${array.length} row(s) in total, clock time to receive data: ${Date.now() - clockTimeStart}ms`);
            stopSpinning();
        }
    }
});
document.getElementById("btn-go").addEventListener("click", () => {
    console.log(Date.now(), "requesting data");
    array = [];
    clockTimeStart = 0;
    startSpinning();
    worker.postMessage({action: "go"});
});
const spinner = document.getElementById("spinner");
const states = [..."▁▂▃▄▅▆▇█▇▆▅▄▃▂▁"];
let stateIndex = 0;
let spinHandle = 0;
let maxDelay = 0;
let intervalStart = 0;
function startSpinning() {
    if (spinner) {
        cancelAnimationFrame(spinHandle);
        maxDelay = 0;
        queueUpdate();
    }
}
function queueUpdate() {
    intervalStart = Date.now();
    spinHandle = requestAnimationFrame(() => {
        updateMax();
        spinner.textContent = states[stateIndex];
        stateIndex = (stateIndex + 1) % states.length;
        if (spinHandle) {
            queueUpdate();
        }
    });
}
function stopSpinning() {
    updateMax();
    cancelAnimationFrame(spinHandle);
    spinHandle = 0;
    if (spinner) {
        spinner.textContent = "Done";
        console.log(`Max delay between frames: ${maxDelay}ms`);
    }
}
function updateMax() {
    if (intervalStart !== 0) {
        const elapsed = Date.now() - intervalStart;
        if (elapsed > maxDelay) {
            maxDelay = elapsed;
        }
    }
}

<div>(Look in the real browser console.)</div>
<input type="button" id="btn-go" value="Go">
<div id="spinner"></div>
<script type="worker" id="worker">
const r = Math.random;
self.addEventListener("message", ({data}) => {
    if ((data && data.action) === "go") {
        for (let n = 0; n < 1600; ++n) {
            const nums = Float64Array.from(
                {length: Math.floor(r() * 7000) * 3},
                () => r()
            );
            postMessage({
                action: "data",
                array: nums,
                done: n === 1600 - 1
            }, [nums.buffer]);
        }
    }
});
</script>

这大大减少了接收数据的时钟时间，同时保持 UI 相当灵敏。

如何使 webworker 的 onmessage 阶段异步？

How to make the onmessage phase of a webworker asynchronous?

javascript

asynchronous

web-worker

选项

仔细看看#3

合并 #2 和 #3