如何叠加 2 个视频，一个是主要的，另一个是叠加它，并同时播放声音。在 ANDROID STUDIO 中使用 FFMPEG
How to overlay 2 videos, one is main second one is overlaying it, and play sound simultaneously. using FFMPEG in ANDROID STUDIO
正如标题所说，我正在尝试叠加 2 个视频并同时播放声音。到目前为止，我设法使用以下命令将 1 个视频放在另一个视频上：
String[] command = {"-i", mainVideoPath, "-vf",
"movie=" + overlayVideo + ", scale=300:-1[inner]; [in][inner]overlay=10:10[out]" ,combinedVideoOutput};
这行得通，但我这里有 3 个问题。
首先，视频旋转 90 度（叠加视频），第二个音频仅从主视频播放（我想同时播放两个视频的声音），第三个叠加视频更长（例如：overlayVideo 持续时间为 10 秒，主视频video last 7 seconds) 然后是 mainVideo，所以我想最后的视频和 mainVideo 一样长，一旦 mainVideo 结束，overlayVideo 也应该停止（需要剪掉它吗？）
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,
"-filter_complex", "[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];[base][wm]overlay=10:10" ,combinedVideoOutput};
使用这个命令我遇到了 2 个与上面相同的问题，只是这里的视频没有旋转。
我不得不说我对 ffmpeg 命令不是很熟悉。我试图从文档 link to documentation 中找出答案，但没有成功。
我知道我缺少一些过滤器，例如 -map merge 或其他东西，但无法弄清楚。
提前致谢！
编辑 1：
这是 logcat 来自第二个命令
D/LISKO: ffmpeg version n4.0-39-gda39990 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.9.x (GCC) 20150123 (prerelease)
D/LISKO: configuration: --target-os=linux --cross-prefix=/root/bravobit/ffmpeg-android/toolchain-android/bin/arm-linux-androideabi- --arch=arm --cpu=cortex-a8 --enable-runtime-cpudetect --sysroot=/root/bravobit/ffmpeg-android/toolchain-android/sysroot --enable-pic --enable-libx264 --enable-ffprobe --enable-libopus --enable-libvorbis --enable-libfdk-aac --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-fontconfig --enable-libvpx --enable-libass --enable-yasm --enable-pthreads --disable-debug --enable-version3 --enable-hardcoded-tables --disable-ffplay --disable-linux-perf --disable-doc --disable-shared --enable-static --enable-runtime-cpudetect --enable-nonfree --enable-network --enable-avresample --enable-avformat --enable-avcodec --enable-indev=lavfi --enable-hwaccels --enable-ffmpeg --enable-zlib --enable-gpl --enable-small --enable-nonfree --pkg-config=pkg-config --pkg-config-flags=--static --prefix=/root/bravobit/ffmpeg-android/build/armeabi-v7a --extra-cflags='-I/root/bravobit/ffmpeg-android/toolchain-android/include -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-overflow -fstack-protector-all' --extra-ldflags='-L/root/bravobit/ffmpeg-android/toolchain-android/lib -Wl,-z,relro -Wl,-z,now -pie' --extra-cxxflags=
D/LISKO: libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
D/LISKO: libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
D/LISKO: libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
D/LISKO: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/mainVideo.mp4':
Metadata:
major_brand : iso6
minor_version : 1
compatible_brands: mp42iso6avc1isom
creation_time : 2020-08-03T13:20:11.000000Z
Duration: 00:00:07.04, start: 0.000000, bitrate: 1380 kb/s
Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 140 kb/s (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
Stream #0:1(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 81:256 DAR 9:16], 1264 kb/s, 30 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
D/LISKO: Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/overlayVideo.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2020-08-04T07:27:47.000000Z
com.android.version: 10
Duration: 00:00:11.19, start: 0.000000, bitrate: 9993 kb/s
D/LISKO: Stream #1:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 9238 kb/s, SAR 1:1 DAR 16:9, 28.38 fps, 29.75 tbr, 90k tbn, 180k tbc (default)
Metadata:
rotate : 270
creation_time : 2020-08-04T07:27:47.000000Z
handler_name : VideoHandle
Side data:
displaymatrix: rotation of 90.00 degrees
Stream #1:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
Metadata:
creation_time : 2020-08-04T07:27:47.000000Z
handler_name : SoundHandle
Stream mapping:
Stream #0:1 (h264) -> scale2ref:ref (graph 0)
Stream #1:0 (h264) -> scale2ref:default (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
D/LISKO: [libx264 @ 0xee986100] using SAR=81/256
D/LISKO: [libx264 @ 0xee986100] using cpu capabilities: ARMv6 NEON
[libx264 @ 0xee986100] profile High, level 3.1
D/LISKO: [libx264 @ 0xee986100] 264 - core 152 r2851M ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
D/LISKO: Output #0, mp4, to '/storage/emulated/0/outputVideo.mp4':
D/LISKO: Metadata:
major_brand : iso6
D/LISKO: minor_version : 1
D/LISKO: compatible_brands: mp42iso6avc1isom
D/LISKO: encoder : Lavf58.12.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 81:256 DAR 9:16], q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
D/LISKO: Metadata:
encoder : Lavc58.18.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
D/LISKO: Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
D/LISKO: encoder : Lavc58.18.100 aac
D/LISKO: frame= 26 fps=0.0 q=0.0 size= 0kB time=00:00:00.09 bitrate= 4.1kbits/s dup=2 drop=0 speed=0.185x
D/LISKO: frame= 41 fps= 41 q=0.0 size= 0kB time=00:00:00.58 bitrate= 0.7kbits/s dup=2 drop=0 speed=0.574x
D/LISKO: frame= 49 fps= 32 q=0.0 size= 0kB time=00:00:00.92 bitrate= 0.4kbits/s dup=2 drop=0 speed=0.613x
D/LISKO: frame= 59 fps= 29 q=29.0 size= 0kB time=00:00:01.97 bitrate= 0.2kbits/s dup=2 drop=0 speed=0.974x
D/LISKO: frame= 75 fps= 29 q=29.0 size= 0kB time=00:00:01.97 bitrate= 0.2kbits/s dup=2 drop=0 speed=0.762x
编辑2：
在命令中添加“-shortest”后，我设法将覆盖视频剪切为与主视频相同的长度（因为覆盖视频总是比 mainVideo 长，“-shortest”持续时间较短。所以现在，命令如下所示：
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];[base][wm]overlay=10:10", "-shortest", combinedVideoOutput};
旋转很好所以只需要合并他们的音频。目前，仅播放 mainVideo 音频，不播放叠加视频音频
编辑 3：
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,
"-strict", "experimental",
"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];" +
"[base][wm]overlay=10:10; " +
"pan=stereo|c0=2*c0|c1=3*c0[a0];[1:a]pan=stereo|c0=1*c0|c1=4*c0[a1];[a0][a1]amix=inputs=2:duration=first:dropout_transition=2",
"-shortest" ,combinedVideoOutput};
通过这个命令，我成功地叠加了视频，并播放了两个视频的声音，旋转很好，但是 -shortest 现在不起作用。现在唯一存在的问题是让它们持续更短（主视频总是更短）？？？
编辑 4：
这终于是工作命令
String[] command = {"-i", mainVideoPath, "-i", overlayVideo,
"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];" +
"[base][wm]overlay=10:10:shortest=1;" +
"pan=stereo|c0=2*c0|c1=3*c0[a0];[1:a]pan=stereo|c0=1*c0|c1=4*c0[a1];" +
"[a0][a1]amix=inputs=2:duration=first:dropout_transition=2",
combinedVideoOutput};
谢谢
使用 overlay filter 中的 shortest 选项：
overlay=10:10:shortest=1
正如标题所说，我正在尝试叠加 2 个视频并同时播放声音。到目前为止，我设法使用以下命令将 1 个视频放在另一个视频上：
String[] command = {"-i", mainVideoPath, "-vf",
"movie=" + overlayVideo + ", scale=300:-1[inner]; [in][inner]overlay=10:10[out]" ,combinedVideoOutput};
这行得通，但我这里有 3 个问题。 首先，视频旋转 90 度（叠加视频），第二个音频仅从主视频播放（我想同时播放两个视频的声音），第三个叠加视频更长（例如：overlayVideo 持续时间为 10 秒，主视频video last 7 seconds) 然后是 mainVideo，所以我想最后的视频和 mainVideo 一样长，一旦 mainVideo 结束，overlayVideo 也应该停止（需要剪掉它吗？）
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,
"-filter_complex", "[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];[base][wm]overlay=10:10" ,combinedVideoOutput};
使用这个命令我遇到了 2 个与上面相同的问题，只是这里的视频没有旋转。 我不得不说我对 ffmpeg 命令不是很熟悉。我试图从文档 link to documentation 中找出答案，但没有成功。 我知道我缺少一些过滤器，例如 -map merge 或其他东西，但无法弄清楚。 提前致谢！
编辑 1： 这是 logcat 来自第二个命令
D/LISKO: ffmpeg version n4.0-39-gda39990 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 4.9.x (GCC) 20150123 (prerelease)
D/LISKO: configuration: --target-os=linux --cross-prefix=/root/bravobit/ffmpeg-android/toolchain-android/bin/arm-linux-androideabi- --arch=arm --cpu=cortex-a8 --enable-runtime-cpudetect --sysroot=/root/bravobit/ffmpeg-android/toolchain-android/sysroot --enable-pic --enable-libx264 --enable-ffprobe --enable-libopus --enable-libvorbis --enable-libfdk-aac --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-fontconfig --enable-libvpx --enable-libass --enable-yasm --enable-pthreads --disable-debug --enable-version3 --enable-hardcoded-tables --disable-ffplay --disable-linux-perf --disable-doc --disable-shared --enable-static --enable-runtime-cpudetect --enable-nonfree --enable-network --enable-avresample --enable-avformat --enable-avcodec --enable-indev=lavfi --enable-hwaccels --enable-ffmpeg --enable-zlib --enable-gpl --enable-small --enable-nonfree --pkg-config=pkg-config --pkg-config-flags=--static --prefix=/root/bravobit/ffmpeg-android/build/armeabi-v7a --extra-cflags='-I/root/bravobit/ffmpeg-android/toolchain-android/include -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-overflow -fstack-protector-all' --extra-ldflags='-L/root/bravobit/ffmpeg-android/toolchain-android/lib -Wl,-z,relro -Wl,-z,now -pie' --extra-cxxflags=
D/LISKO: libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
D/LISKO: libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
D/LISKO: libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
D/LISKO: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/mainVideo.mp4':
Metadata:
major_brand : iso6
minor_version : 1
compatible_brands: mp42iso6avc1isom
creation_time : 2020-08-03T13:20:11.000000Z
Duration: 00:00:07.04, start: 0.000000, bitrate: 1380 kb/s
Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 140 kb/s (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
Stream #0:1(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 81:256 DAR 9:16], 1264 kb/s, 30 fps, 30 tbr, 90k tbn, 180k tbc (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
D/LISKO: Input #1, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/overlayVideo.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2020-08-04T07:27:47.000000Z
com.android.version: 10
Duration: 00:00:11.19, start: 0.000000, bitrate: 9993 kb/s
D/LISKO: Stream #1:0(eng): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 9238 kb/s, SAR 1:1 DAR 16:9, 28.38 fps, 29.75 tbr, 90k tbn, 180k tbc (default)
Metadata:
rotate : 270
creation_time : 2020-08-04T07:27:47.000000Z
handler_name : VideoHandle
Side data:
displaymatrix: rotation of 90.00 degrees
Stream #1:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
Metadata:
creation_time : 2020-08-04T07:27:47.000000Z
handler_name : SoundHandle
Stream mapping:
Stream #0:1 (h264) -> scale2ref:ref (graph 0)
Stream #1:0 (h264) -> scale2ref:default (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
D/LISKO: [libx264 @ 0xee986100] using SAR=81/256
D/LISKO: [libx264 @ 0xee986100] using cpu capabilities: ARMv6 NEON
[libx264 @ 0xee986100] profile High, level 3.1
D/LISKO: [libx264 @ 0xee986100] 264 - core 152 r2851M ba24899 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
D/LISKO: Output #0, mp4, to '/storage/emulated/0/outputVideo.mp4':
D/LISKO: Metadata:
major_brand : iso6
D/LISKO: minor_version : 1
D/LISKO: compatible_brands: mp42iso6avc1isom
D/LISKO: encoder : Lavf58.12.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 81:256 DAR 9:16], q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
D/LISKO: Metadata:
encoder : Lavc58.18.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
D/LISKO: Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
creation_time : 2020-07-28T08:11:36.000000Z
D/LISKO: encoder : Lavc58.18.100 aac
D/LISKO: frame= 26 fps=0.0 q=0.0 size= 0kB time=00:00:00.09 bitrate= 4.1kbits/s dup=2 drop=0 speed=0.185x
D/LISKO: frame= 41 fps= 41 q=0.0 size= 0kB time=00:00:00.58 bitrate= 0.7kbits/s dup=2 drop=0 speed=0.574x
D/LISKO: frame= 49 fps= 32 q=0.0 size= 0kB time=00:00:00.92 bitrate= 0.4kbits/s dup=2 drop=0 speed=0.613x
D/LISKO: frame= 59 fps= 29 q=29.0 size= 0kB time=00:00:01.97 bitrate= 0.2kbits/s dup=2 drop=0 speed=0.974x
D/LISKO: frame= 75 fps= 29 q=29.0 size= 0kB time=00:00:01.97 bitrate= 0.2kbits/s dup=2 drop=0 speed=0.762x
编辑2： 在命令中添加“-shortest”后，我设法将覆盖视频剪切为与主视频相同的长度（因为覆盖视频总是比 mainVideo 长，“-shortest”持续时间较短。所以现在，命令如下所示：
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];[base][wm]overlay=10:10", "-shortest", combinedVideoOutput};
旋转很好所以只需要合并他们的音频。目前，仅播放 mainVideo 音频，不播放叠加视频音频
编辑 3：
String[] command = {"-i", mainVideoPath, "-i", overlayVideo ,
"-strict", "experimental",
"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];" +
"[base][wm]overlay=10:10; " +
"pan=stereo|c0=2*c0|c1=3*c0[a0];[1:a]pan=stereo|c0=1*c0|c1=4*c0[a1];[a0][a1]amix=inputs=2:duration=first:dropout_transition=2",
"-shortest" ,combinedVideoOutput};
通过这个命令，我成功地叠加了视频，并播放了两个视频的声音，旋转很好，但是 -shortest 现在不起作用。现在唯一存在的问题是让它们持续更短（主视频总是更短）？？？
编辑 4：
这终于是工作命令
String[] command = {"-i", mainVideoPath, "-i", overlayVideo,
"-filter_complex",
"[1:v][0:v]scale2ref=(256/256)*ih/8/sar:ih/8[wm][base];" +
"[base][wm]overlay=10:10:shortest=1;" +
"pan=stereo|c0=2*c0|c1=3*c0[a0];[1:a]pan=stereo|c0=1*c0|c1=4*c0[a1];" +
"[a0][a1]amix=inputs=2:duration=first:dropout_transition=2",
combinedVideoOutput};
谢谢
使用 overlay filter 中的 shortest 选项：
overlay=10:10:shortest=1