如何让Pix2Pix 跟p5.js 运行？

Question

我想试试ml5.js Pix2Pix example for p5.js。如果我只是复制代码，更新路径，并尝试让它运行在我的本地服务器上，它不起作用。

这里也一样：

// Copyright (c) 2019 ml5
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

/* ===
ml5 Example
Pix2pix Edges2Pikachu example with p5.js using callback functions
This uses a pre-trained model on Pikachu images
For more models see: https://github.com/ml5js/ml5-data-and-training/tree/master/models/pix2pix
=== */

// The pre-trained Edges2Pikachu model is trained on 256x256 images
// So the input images can only be 256x256 or 512x512, or multiple of 256
const SIZE = 256;
let inputImg, inputCanvas, outputContainer, statusMsg, pix2pix, clearBtn, transferBtn, modelReady = false,
  isTransfering = false;

function setup() {
  // Create a canvas
  inputCanvas = createCanvas(SIZE, SIZE);
  inputCanvas.class('border-box').parent('canvasContainer');

  // Display initial input image
  inputImg = loadImage('https://ml5js.github.io/ml5-examples/javascript/Pix2Pix/Pix2Pix_promise/images/input.png', drawImage);

  // Selcect output div container
  outputContainer = select('#output');
  statusMsg = select('#status');

  // Select 'transfer' button html element
  transferBtn = select('#transferBtn');

  // Select 'clear' button html element
  clearBtn = select('#clearBtn');
  // Attach a mousePressed event to the 'clear' button
  clearBtn.mousePressed(function() {
    clearCanvas();
  });

  // Set stroke to black
  stroke(0);
  pixelDensity(1);

  // Create a pix2pix method with a pre-trained model
  pix2pix = ml5.pix2pix('https://github.com/ml5js/ml5-library/blob/main/examples/p5js/Pix2Pix/Pix2Pix_callback/models/edges2pikachu.pict', modelLoaded);
}

// Draw on the canvas when mouse is pressed
function draw() {
  if (mouseIsPressed) {
    line(mouseX, mouseY, pmouseX, pmouseY);
  }
}

// Whenever mouse is released, transfer the current image if the model is loaded and it's not in the process of another transformation
function mouseReleased() {
  if (modelReady && !isTransfering) {
    transfer()
  }
}

// A function to be called when the models have loaded
function modelLoaded() {
  // Show 'Model Loaded!' message
  statusMsg.html('Model Loaded!');

  // Set modelReady to true
  modelReady = true;

  // Call transfer function after the model is loaded
  transfer();

  // Attach a mousePressed event to the transfer button
  transferBtn.mousePressed(function() {
    transfer();
  });
}

// Draw the input image to the canvas
function drawImage() {
  image(inputImg, 0, 0);
}

// Clear the canvas
function clearCanvas() {
  background(255);
}

function transfer() {
  // Set isTransfering to true
  isTransfering = true;

  // Update status message
  statusMsg.html('Applying Style Transfer...!');

  // Select canvas DOM element
  const canvasElement = select('canvas').elt;

  // Apply pix2pix transformation
  pix2pix.transfer(canvasElement, function(err, result) {
    if (err) {
      console.log(err);
    }
    if (result && result.src) {
      // Set isTransfering back to false
      isTransfering = false;
      // Clear output container
      outputContainer.html('');
      // Create an image based result
      createImg(result.src).class('border-box').parent('output');
      // Show 'Done!' message
      statusMsg.html('Done!');
    }
  });
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js"></script>
<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js" type="text/javascript"></script>

<h1>Pix2Pix Edges2Pichaku Example</h1>
<p>1. Wait until the model is loaded</p>
<p>2. Press your mouse to draw a Pikachu on the left side of the canvas.</p>
<p>3. A colored Pikachu image will automatically appear on the right side of the canvas in ~2 seconds. You could also click the "Transfer" button to generate an new image.</p>
<p>4. You could click the "Clear" button to clear the canvas and draw again.</p>
<p id="status">Loading Model... Please wait...</p>
<div class="flex">
  <div>
    <div id="canvasContainer"></div>
    <div id="btnContainer" class="flex flex-space-between">
      <button id="clearBtn">Clear</button><br />
      <button id="transferBtn" class="btn">Transfer</button>
    </div>
  </div>
  <div id="transferContainer">
  </div>
  <div id="output"></div>
</div>

这也是一个 jsFiddle： https://jsfiddle.net/L6oaydrm/

有谁知道如何让它运行？将不胜感激。

Answer 1

我想我能够通过一些修补让 'callback' 示例在本地工作：

从示例中下载文件：https://github.com/ml5js/ml5-library/tree/main/examples/p5js/Pix2Pix/Pix2Pix_callback
调整 index.html 以从代码中的 unpkg.com URL 加载 ml5.min.js。
创建一个新函数：

function startTransfer(){
  // Create a pix2pix method with a pre-trained model
  pix2pix = ml5.pix2pix('./models/edges2pikachu.pict', modelLoaded);
}

将对 transfer() 的所有调用替换为 modelLoaded() 中第一个调用除外 startTransfer()。
启动一个简单的本地网络服务器；对我来说：python -m http.server 有效。

该示例似乎有效。我可以在 canvas 上绘图，ML 模型会根据我添加的新行重新绘制皮卡丘图像。注意，有时初始传输是运行在加载模板图像 (input.png) 之前，结果是乱码的黄色/红色像素；单击 'Transfer' 修复此问题。

基本上，它总是会将模型重新加载到 ml5 库中；我不知道这对性能有何影响，但它在我的浏览器中重新绘制的速度相对较快。该文件将缓存在浏览器中，所以这不是问题，但我不确定 ml5.js 库的内部结构以及 ml5.pix2pix(...) 的作用。

我已将修改后的代码（包括对 JS 的一些其他调整）放在 https://jsfiddle.net/lecrte/jvohcw8r/16/ 上......但它不会在那里工作，因为资产相对于 HTML 文件，由于 CORS 问题，我们无法直接从 github.com 加载 edges2pikachu.pict。

如何让Pix2Pix 跟p5.js 运行？

How to let Pix2Pix with p5.js run?

javascript

p5.js

ml5.js