sciptprocessor 在 Angular 10 中增加了 CPU 的使用,并更改了输入流。还有怎么销毁
sciptprocessor increasing CPU usage in Angular 10 with changes to input stream. Also how to destroy it
希望从我在 device-access 组件路由上实现的流量计释放 CPU 资源。此 volume-meter 组件采用在 parent 组件 (device-access) 中设置的现有流,使用 WebRTC navigator.mediaDevices.getUserMedia
调用。 parent 组件允许用户切换音频输入设备并让音量计反映输入的 volume/gain/feedback 反映在音量计中。
目前存在一个问题,在多次切换设备时,CPU 级别会逐渐上升。此外,此 device-access 页面是视频会议组件的入口。一个常见的场景是让用户返回到这个 deice-access 页面。当用户返回时,onaudioprocess 仍然是 运行ning 并复合了 CPU 用法。
代码如下。我在 child 组件 (volume-meter) 中实现了一个 ngOnDestroy,但它似乎并没有影响仍在 运行ning 的进程。当我切换输入音频设备(在订阅中)时,我想终止脚本处理器并重新启动它。我该怎么做?
export class VolumeMeterComponent implements OnInit, AfterViewInit, OnDestroy {
private stream: MediaStream = null;
private audioContext: AudioContext = null;
private meter: any = null;
private canvasContext: any = null;
// height and width of the volume meter
private WIDTH: number = 146;
private HEIGHT: number = 9;
private rafID: number = null;
private mediaStreamSource: any = null;
private clipping: boolean = null;
private lastClip: number = null;
private volume:number = null;
// averaging: how "smoothed" you would like the meter to be over time.
// Should be between 0 and less than 1.
private averaging: number = .95;
// the level (0 to 1) that you would consider "clipping"
private clipLevel: number = .98;
// clipLag: how long you would like the "clipping" indicator to show after clipping has occurred, in milliseconds.
private clipLag: number = 750;
private loopInstance: any = null;
private processHandle: any = null;
// @ts-ignore
@ViewChild('meterElement', {read: ElementRef, static: false}) meterElement: ElementRef;
constructor(private streamService: StreamService) {}
ngOnInit(): void {
// nothing here for now
}
ngAfterViewInit(): void {
this.streamService.stream$.subscribe(stream =>{
this.stream = stream;
if (this.loopInstance) {
this.processHandle.stop();
this.cleanupMeterOnChange();
}
this.initializeMeter();
});
}
ngOnDestroy(): void {
this.cleanupMeterOnChange();
}
cleanupMeterOnChange():void {
// cleanup the canvasContext and meter onDestroy
// best practice is to cleanup the the canvasContext as it has processes
this.meter = null;
this.canvasContext = null;
this.drawLoop = null;
}
initializeMeter():void {
this.canvasContext = this.meterElement.nativeElement.getContext('2d');
// update audioContext to whatever is available from browser
try {
(window as any).AudioContext = (window as any).AudioContext || (window as any).webkitAudioContext;
this.audioContext = new AudioContext();
this.mediaStreamSource = this.audioContext.createMediaStreamSource(this.stream);
// Create a new volume meter and connect it.
this.meter = this.createAudioMeter(this.audioContext);
this.mediaStreamSource.connect(this.meter);
this.loopInstance = this.drawLoop();
} catch(error) {
console.log('Error setting up the volume meter. ' + error);
}
}
drawLoop = () => {
// clear the background
this.canvasContext.clearRect(0, 0, this.WIDTH, this.HEIGHT);
// check if we're currently clipping
if (this.meter.checkClipping()) {
this.canvasContext.fillStyle = 'red';
} else {
this.canvasContext.fillStyle = 'green';
}
// draw a bar based on the current volume
this.canvasContext.fillRect(0, 0, this.meter.volume * this.WIDTH * 1.4, this.HEIGHT);
// set up the next visual callback
this.rafID = window.requestAnimationFrame( this.drawLoop );
}
createAudioMeter = audioContext => {
const processor = audioContext.createScriptProcessor(2048, 1, 1);
processor.onaudioprocess = this.volumeAudioProcess;
this.processHandle = processor;
processor.clipping = false;
processor.lastClip = 0;
processor.volume = 0;
processor.clipLevel = this.clipLevel;
processor.averaging = this.averaging;
processor.clipLag = this.clipLag;
// this will have no effect, since we don't copy the input to the output,
// but works around a current Chrome bug.
processor.connect(audioContext.destination);
processor.checkClipping =
checkClipping;
// tslint:disable-next-line:typedef
function checkClipping() {
const that = this;
if (!that.clipping) {
return false;
}
if ((that.lastClip + that.clipLag) < window.performance.now()) {
that.clipping = false;
}
return that.clipping;
}
processor.shutdown =
function(): void {
this.disconnect();
this.onaudioprocess = null;
};
return processor;
}
volumeAudioProcess( event ): void {
this.clipping = false;
const buf = event.inputBuffer.getChannelData(0);
const bufLength = buf.length;
let sum = 0;
let x;
// Do a root-mean-square on the samples: sum up the squares...
for (let i = 0; i < bufLength; i++) {
x = buf[i];
if (Math.abs(x) >= this.clipLevel) {
this.clipping = true;
this.lastClip = window.performance.now();
}
sum += x * x;
}
// ... then take the square root of the sum.
const rms = Math.sqrt(sum / bufLength);
// Now smooth this out with the averaging factor applied
// to the previous sample - take the max here because we
// want "fast attack, slow release."
this.volume = Math.max(rms, this.volume * this.averaging);
}
}
组件标记:
<canvas id="meterElement" #meterElement width="146" height="8"></canvas>
<p class="level-label">Microphone volume level</p>
我曾尝试使用 ViewChild 订阅 canvas 并取消订阅,但运气不佳。任何人都对 运行 更有效的策略有所了解。订阅 drawLoop(并将其提取到服务中)是最佳答案吗?
我知道 WebRTC 推荐 audioWorklets:
https://alvestrand.github.io/audio-worklet/
- 这是草稿,Safari 未采用。从长远来看,这似乎是一个更好的解决方案。
您可以在 AudioContext 上调用 close()
以停止其所有节点并使其释放执行其操作所需的任何系统资源。我认为将此添加到您的 cleanupMeterOnChange()
方法应该会提高性能。
cleanupMeterOnChange():void {
// ...
this.audioContext.close();
}
希望从我在 device-access 组件路由上实现的流量计释放 CPU 资源。此 volume-meter 组件采用在 parent 组件 (device-access) 中设置的现有流,使用 WebRTC navigator.mediaDevices.getUserMedia
调用。 parent 组件允许用户切换音频输入设备并让音量计反映输入的 volume/gain/feedback 反映在音量计中。
目前存在一个问题,在多次切换设备时,CPU 级别会逐渐上升。此外,此 device-access 页面是视频会议组件的入口。一个常见的场景是让用户返回到这个 deice-access 页面。当用户返回时,onaudioprocess 仍然是 运行ning 并复合了 CPU 用法。
代码如下。我在 child 组件 (volume-meter) 中实现了一个 ngOnDestroy,但它似乎并没有影响仍在 运行ning 的进程。当我切换输入音频设备(在订阅中)时,我想终止脚本处理器并重新启动它。我该怎么做?
export class VolumeMeterComponent implements OnInit, AfterViewInit, OnDestroy {
private stream: MediaStream = null;
private audioContext: AudioContext = null;
private meter: any = null;
private canvasContext: any = null;
// height and width of the volume meter
private WIDTH: number = 146;
private HEIGHT: number = 9;
private rafID: number = null;
private mediaStreamSource: any = null;
private clipping: boolean = null;
private lastClip: number = null;
private volume:number = null;
// averaging: how "smoothed" you would like the meter to be over time.
// Should be between 0 and less than 1.
private averaging: number = .95;
// the level (0 to 1) that you would consider "clipping"
private clipLevel: number = .98;
// clipLag: how long you would like the "clipping" indicator to show after clipping has occurred, in milliseconds.
private clipLag: number = 750;
private loopInstance: any = null;
private processHandle: any = null;
// @ts-ignore
@ViewChild('meterElement', {read: ElementRef, static: false}) meterElement: ElementRef;
constructor(private streamService: StreamService) {}
ngOnInit(): void {
// nothing here for now
}
ngAfterViewInit(): void {
this.streamService.stream$.subscribe(stream =>{
this.stream = stream;
if (this.loopInstance) {
this.processHandle.stop();
this.cleanupMeterOnChange();
}
this.initializeMeter();
});
}
ngOnDestroy(): void {
this.cleanupMeterOnChange();
}
cleanupMeterOnChange():void {
// cleanup the canvasContext and meter onDestroy
// best practice is to cleanup the the canvasContext as it has processes
this.meter = null;
this.canvasContext = null;
this.drawLoop = null;
}
initializeMeter():void {
this.canvasContext = this.meterElement.nativeElement.getContext('2d');
// update audioContext to whatever is available from browser
try {
(window as any).AudioContext = (window as any).AudioContext || (window as any).webkitAudioContext;
this.audioContext = new AudioContext();
this.mediaStreamSource = this.audioContext.createMediaStreamSource(this.stream);
// Create a new volume meter and connect it.
this.meter = this.createAudioMeter(this.audioContext);
this.mediaStreamSource.connect(this.meter);
this.loopInstance = this.drawLoop();
} catch(error) {
console.log('Error setting up the volume meter. ' + error);
}
}
drawLoop = () => {
// clear the background
this.canvasContext.clearRect(0, 0, this.WIDTH, this.HEIGHT);
// check if we're currently clipping
if (this.meter.checkClipping()) {
this.canvasContext.fillStyle = 'red';
} else {
this.canvasContext.fillStyle = 'green';
}
// draw a bar based on the current volume
this.canvasContext.fillRect(0, 0, this.meter.volume * this.WIDTH * 1.4, this.HEIGHT);
// set up the next visual callback
this.rafID = window.requestAnimationFrame( this.drawLoop );
}
createAudioMeter = audioContext => {
const processor = audioContext.createScriptProcessor(2048, 1, 1);
processor.onaudioprocess = this.volumeAudioProcess;
this.processHandle = processor;
processor.clipping = false;
processor.lastClip = 0;
processor.volume = 0;
processor.clipLevel = this.clipLevel;
processor.averaging = this.averaging;
processor.clipLag = this.clipLag;
// this will have no effect, since we don't copy the input to the output,
// but works around a current Chrome bug.
processor.connect(audioContext.destination);
processor.checkClipping =
checkClipping;
// tslint:disable-next-line:typedef
function checkClipping() {
const that = this;
if (!that.clipping) {
return false;
}
if ((that.lastClip + that.clipLag) < window.performance.now()) {
that.clipping = false;
}
return that.clipping;
}
processor.shutdown =
function(): void {
this.disconnect();
this.onaudioprocess = null;
};
return processor;
}
volumeAudioProcess( event ): void {
this.clipping = false;
const buf = event.inputBuffer.getChannelData(0);
const bufLength = buf.length;
let sum = 0;
let x;
// Do a root-mean-square on the samples: sum up the squares...
for (let i = 0; i < bufLength; i++) {
x = buf[i];
if (Math.abs(x) >= this.clipLevel) {
this.clipping = true;
this.lastClip = window.performance.now();
}
sum += x * x;
}
// ... then take the square root of the sum.
const rms = Math.sqrt(sum / bufLength);
// Now smooth this out with the averaging factor applied
// to the previous sample - take the max here because we
// want "fast attack, slow release."
this.volume = Math.max(rms, this.volume * this.averaging);
}
}
组件标记:
<canvas id="meterElement" #meterElement width="146" height="8"></canvas>
<p class="level-label">Microphone volume level</p>
我曾尝试使用 ViewChild 订阅 canvas 并取消订阅,但运气不佳。任何人都对 运行 更有效的策略有所了解。订阅 drawLoop(并将其提取到服务中)是最佳答案吗?
我知道 WebRTC 推荐 audioWorklets: https://alvestrand.github.io/audio-worklet/
- 这是草稿,Safari 未采用。从长远来看,这似乎是一个更好的解决方案。
您可以在 AudioContext 上调用 close()
以停止其所有节点并使其释放执行其操作所需的任何系统资源。我认为将此添加到您的 cleanupMeterOnChange()
方法应该会提高性能。
cleanupMeterOnChange():void {
// ...
this.audioContext.close();
}