如何使用 FastAPI return 视频帧和文本到 HTML 页面？

Question

以目前的代码，我只能将视频帧发送到网页。我怎样才能将一些文本与每一帧一起发送并显示出来。

FastAPI 代码

def generate_frames(cap,i):
    while True:
        success,frame = cap.read()
        if not success:
            break
        else:
                # Reshape image
                im_arr = cv2.imencode('.jpg', frame)[1]
                cv2.waitKey(50)
                print(loa[i]) //text to be displayed along with image
                i = i + 1
                yield (b'--frame\r\n'
                       b'Content-Type: image/jpeg\r\n\r\n' + bytearray(im_arr) + b'\r\n')


@app.get('/video')
def video():
    i = 0
    cap = cv2.VideoCapture('C:\Users\ryanv_k78mbsh\Desktop\FINAL\MovenetTest\animation.gif')
    return StreamingResponse(generate_frames(cap,i),media_type = 'multipart/x-mixed-replace; boundary=frame')

HTML接收并显示视频帧的代码

<div style= "height:50px"></div>
<img src ="{{ url_for('video') }}" width="50%" />
</div>

Answer 1

您可以使用 WebSockets instead, as described in this answer（选项 2），并将文本和图像字节都发送到前端。示例如下：

app.py

from fastapi import FastAPI, Request, WebSocket, WebSocketDisconnect
from fastapi.templating import Jinja2Templates
import uvicorn
import cv2

app = FastAPI()
camera = cv2.VideoCapture(0,cv2.CAP_DSHOW)
templates = Jinja2Templates(directory="templates")

@app.get('/')
def index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

@app.websocket("/ws")
async def get_stream(websocket: WebSocket):
    await websocket.accept()
    try:
        while True:
            success, frame = camera.read()
            if not success:
                break
            else:
                ret, buffer = cv2.imencode('.jpg', frame)
                await websocket.send_text("some text")
                await websocket.send_bytes(buffer.tobytes())
    except WebSocketDisconnect:
        print("Client disconnected")   
 
if __name__ == '__main__':
    uvicorn.run(app, host='127.0.0.1', port=8000)

模板/index.html

<!DOCTYPE html>
<html>
    <head>
        <title>Live Streaming</title>
    </head>
    <body>
        <img id="frame" src="">
        <div id="textArea"></div>
        <script>
            let ws = new WebSocket("ws://localhost:8000/ws");
            let image = document.getElementById("frame");
            image.onload = function(){
                URL.revokeObjectURL(this.src); // release the blob URL once the image is loaded
            } 
            ws.onmessage = function(event) {
                if (typeof event.data === 'string') 
                    document.getElementById("textArea").innerHTML = event.data;
                else
                    image.src = URL.createObjectURL(event.data);
            };
        </script>
    </body>
</html>

如何使用 FastAPI return 视频帧和文本到 HTML 页面？

How to return video frame and text to HTML page using FastAPI?

html

python

opencv

computer-vision

fastapi