ffmpeg使用说明

ffmpeg与opencv是视频图像处理的两大利器，本文对ffmpeg的使用方法做一个介绍。

ffmpeg可以用来做什么

• 转封装(demux-mux)：把一种格式的封装转为其他格式，譬如 .mp4 转化为 .ts
• 转码(transcoding)：把一种编码转换为另一种编码，譬如 h264 转码为 hevc
• 添加特效(filter): 对音视频的每一帧进行处理， 譬如添加logo
• 拆分合并视频: 把长视频拆分为短视频或者一帧帧的图片

ffmpeg基本用法

ffmpeg处理流程

 _______              ______________                __________
|       |            |              |              |          |  
| input |  demuxer   | encoded data |    decoder   |  decoded |
| file  | ---------> | packets      | -----------> |  frames  |
|_______|            |______________|              |__________|
                                                        |
                                                        v
                                                    __________  
                                                   | (optinal)|
                                                   | filter   |
                                                   | frames   |
 ________             ______________               |__________|  
|        |           |              |                   |
| output | <-------- | encoded data | <-----------------+
| file   |   muxer   | packets      |     encoder
|________|           |______________|

命令格式：ffmpeg [global_options] {[input_file_options] -i input_url} ... {[output_file_options] output_url} ...

命令示例

把一个任意编码格式的flv文件，转码为1080P的H.264编码的mp4文件。

1
./ffmpeg -y -i ./clip_12.flv -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease:flags=lanczos,setpts=PTS-STARTPTS -vcodec libx264 -x264-params psnr=1:ssim=1:stitchable=1:keyint=150:crf=21 -acodec copy ./clip_12_crf21.mp4

Demux and Mux

mp4文件转为ts文件，并输出m3u8

1
2
3
4
5
./ffmpeg -i input.mp4 -codec copy -bsf:v h264_mp4toannexb output.ts

./ffmpeg -i output.ts -c copy -map 0 -f segment -segment_list playlist.m3u8 -segment_time 10 ts_output%03d.ts
or
./ffmpeg -i ./output.ts -codec copy -f hls -hls_list_size 0 -hls_time 10 ./ts2_output.m3u8

• -bsf : Bitstream Filters https://ffmpeg.org/ffmpeg-bitstream-filters.html
• -f : Force input or output file format. The format is normally auto detected for input files and guessed from the file extension for output files, so this option is not needed in most cases. https://ffmpeg.org/ffmpeg-formats.html
• hls format: 此处使用的是mux的hls, Apple HTTP Live Streaming muxer that segments MPEG-TS according to the HTTP Live Streaming (HLS) specification.
• segment format: Basic stream segmenter, this muxer outputs streams to a number of separate files of nearly fixed duration.

参考文章：

1. http://blog.p2hp.com/archives/4824

Transcoding

任意编码格式的文件转码为hevc编码格式

1
./ffmpeg -y -i ./section_20.mp4 -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease:flags=lanczos,setpts=PTS-STARTPTS -vcodec libx265 -x265-params crf=22:keyint=150 -acodec aac ./out_265.mp4

Extract video

提取出一个视频的第30s–50s视频

1
./ffmpeg -ss 30 -t 20 -i input.mp4 -codec copy section_20.mp4

参考文章：

1. https://zhuanlan.zhihu.com/p/27366331

ffmpeg filter

简单filter：单输入单输出

 _________                        ______________
|         |                      |              |
| decoded |                      | encoded data |
| frames  |\                   _ | packets      |
|_________| \                  /||______________|
             \   __________   /
  simple     _\||          | /  encoder
  filtergraph   | filtered |/
                | frames   |
                |__________|

命令参数： With -vf and -af aliases for video and audio respectively

复杂filter：多输入多输出

 _________
|         |
| input 0 |\                    __________
|_________| \                  |          |
             \   _________    /| output 0 |
              \ |         |  / |__________|
 _________     \| complex | /
|         |     |         |/
| input 1 |---->| filter  |\
|_________|     |         | \   __________
               /| graph   |  \ |          |
              / |         |   \| output 1 |
 _________   /  |_________|    |__________|
|         | /
| input 2 |/
|_________|

命令参数： Complex filtergraphs are configured with the -filter_complex option. The -lavfi option is equivalent to -filter_complex

filter标签说明

• A filterchain consists of a sequence of connected filters, each one connected to the previous one in the sequence. A filterchain is represented by a list of “,”-separated filter descriptions.
• A filtergraph consists of a sequence of filterchains. A sequence of filterchains is represented by a list of “;”-separated filterchain descriptions.
• A filter is represented by a string of the form: [in_link_1]…[in_link_N]filter_name@id=arguments[out_link_1]…[out_link_M]

ffmpeg常见功能举例

把2个视频合并到同一个画面，用以效果对比

命令：把2个视频左右合并为1个，并在中间设置一条均匀来回滑动的线

1
./ffmpeg -i ./section_20.mp4 -i ./section_20.mp4 -filter_complex "[0:v] setpts=PTS-STARTPTS, scale=w=1920:h=1080:force_original_aspect_ratio=decrease:flags=lanczos [left];[1:v] setpts=PTS-STARTPTS, scale=w=1920:h=1080:force_original_aspect_ratio=decrease:flags=lanczos,curves=preset=color_negative [right];[left][right] blend=shortest=1:all_expr='if(lte(X,W*(1-abs((mod(T/10\,2)-1)))),A,B)' [scene]; color=c=black:size=2x1080 [line];[scene][line] overlay=shortest=1:format=yuv420:x=W*(1-abs((mod(t/10\,2)-1))) [out]" -map "[out]" -map "0:a" -movflags faststart -vcodec libx264 -x264opts crf=16 -y merge2.mp4

• blend:逐点改变XY，逐帧改变T, A与B是逐像素点处理。
• overlay:叠加一个画面在原始画面上。

效果图:

画出当前视频的cie坐标，并列出色彩分布

命令：画出视频每一帧的色彩分布与直方图，用以图像处理指导

1
./ffmpeg -ss 0 -t 20 -i ./test.mp4 -lavfi "scale=w=1280:h=720:force_original_aspect_ratio=decrease:flags=lanczos,split=3 [src][hist_src][cie_src];[src]pad=iw+640:ih[background];[cie_src]ciescope=system=rec709:cie=xyy:gamuts=rec709:showwhite=1,format=pix_fmts=yuv420p,scale=640:360[cie];[hist_src]format=pix_fmts=yuv420p,histogram,scale=640:360[hist];[background][hist] overlay=shortest=1:x=1280[back_hist];[back_hist][cie]overlay=shortest=1:x=1280:y=360[out]" -map "[out]" -movflags faststart -vcodec libx264 -x264-params crf=20 -v info -y ./test_cie_hist.mp4

• ciescope:统计cie图
• histogram:统计直方图

效果图：

淡入淡出功能

https://blog.csdn.net/yu540135101/article/details/84503451

画中画功能（添加logo）

1. 添加logo
   ffmpeg -i ./test.mp4 -vf "movie=./ico.png[logo];[0:v][logo]overlay=main_w-overlay_w-100:100" -movflags faststart -vcodec libx264 -x264-params crf=20 -v info -y ./test_logo.mp4

   

2. 去除logo
   https://blog.csdn.net/u013699869/article/details/48264071

音频增强功能

音量均衡：

./ffmpeg -i ./test.mp4 -af loudnorm=i=-30:print_format=summary -movflags faststart -vcodec copy -acodec aac -v info -y ./test_norm.mp4

参考文章：https://medium.com/kkstream/有關audio-normalization兩三事-dca62497e197

获取音视频的pts、dts信息，用以音视频同步问题定位

1
2
./ffmpeg -i ./test.mp4 -vf showinfo -f null -
./ffmpeg -i ./test.mp4 -af ashowinfo -f null -

[Parsed_showinfo_0 @ 0x1e95900] config in time_base: 1/2500, frame_rate: 25/1
[Parsed_showinfo_0 @ 0x1e95900] config out time_base: 0/0, frame_rate: 0/0
[Parsed_showinfo_0 @ 0x1e95900] n:   0 pts:      0 pts_time:0       pos:   104995 fmt:yuv420p sar:0/1 s:1920x1080 i:P iskey:1 type:I checksum:EB53D44D plane_checksum:[2F205DA6 DDC5BB4C DDC5BB4C] mean:[16 128 128] stdev:[0.0 0.0 0.0]
[Parsed_showinfo_0 @ 0x1e95900] n:   1 pts:    100 pts_time:0.04    pos:   111483 fmt:yuv420p sar:0/1 s:1920x1080 i:P iskey:0 type:B checksum:EB53D44D plane_checksum:[2F205DA6 DDC5BB4C DDC5BB4C] mean:[16 128 128] stdev:[0.0 0.0 0.0]
[Parsed_showinfo_0 @ 0x1e95900] n:   2 pts:    200 pts_time:0.08    pos:   111349 fmt:yuv420p sar:0/1 s:1920x1080 i:P iskey:0 type:P checksum:EB53D44D plane_checksum:[2F205DA6 DDC5BB4C DDC5BB4C] mean:[16 128 128] stdev:[0.0 0.0 0.0]


[Parsed_ashowinfo_0 @ 0x20cd360] n:0 pts:0 pts_time:0 pos:98125 fmt:fltp channels:2 chlayout:stereo rate:44100 nb_samples:1024 checksum:00000000 plane_checksums: [ 00000000 00000000 ]
[Parsed_ashowinfo_0 @ 0x20cd360] n:1 pts:1024 pts_time:0.02322 pos:98131 fmt:fltp channels:2 chlayout:stereo rate:44100 nb_samples:1024 checksum:744E349C plane_checksums: [ 5F359F90 7F7394FD ]
[Parsed_ashowinfo_0 @ 0x20cd360] n:2 pts:2048 pts_time:0.0464399 pos:98173 fmt:fltp channels:2 chlayout:stereo rate:44100 nb_samples:1024 checksum:71715EED plane_checksums: [ 377194F8 2E50C9E6 ]

• 音视频更多信息的查询，可以采用ffprobe获取。

计算2个视频的psnr,并dump出src和dst用以其他方式验证

1
ffmpeg -y -i ./testA.mp4 -i ./testB.mkv -vsync 0 -shortest -filter_complex "[0:v][1:v]scale2ref=flags=lanczos[s0][d0];[s0]setpts=0,showinfo,split=2[s][s1];[d0]setpts=0,showinfo,split=2[d][d1];[s][d]psnr="stats_file=./psnr_q.info:shortest=1[s3]"" -map [s1] src.yuv -map [d1] dst.yuv -map [s3] -f null -

• -vsync : 视频帧是否根据指定的fps进行复制或者丢失帧，设为0改变，默认是-1
• -shortest : (output)Finish encoding when the shortest input stream ends.
• -f null - : This muxer does not generate any output file, it is mainly useful for testing or benchmarking purposes. Specifying the output file - is required by the ffmpeg syntax.

ffmpeg部分参数解析

ffmpeg -map介绍

-map [-]input_file_id[:stream_specifier][?][,sync_file_id[:stream_specifier]] | [linklabel] (output)

Designate one or more input streams as a source for the output file. Each input stream is identified by the input file index input_file_id and the input stream index input_stream_id within the input file. Both indices start at 0. If specified, sync_file_id:stream_specifier sets which input stream is used as a presentation sync reference.

• A - character before the stream identifier creates a “negative” mapping. It disables matching streams from already created mappings.
• A trailing ? after the stream index will allow the map to be optional: if the map matches no streams the map will be ignored instead of failing. Note the map will still fail if an invalid input file index is used; such as if the map refers to a non-existent input.
• An alternative [linklabel] form will map outputs from complex filter graphs (see the -filter_complex option) to the output file. linklabel must correspond to a defined output link label in the graph.

1. ffmpeg provides the -map option for manual control of stream selection in each output file. Users can skip -map and let ffmpeg perform automatic stream selection as described below. The -vn / -an / -sn / -dn options can be used to skip inclusion of video, audio, subtitle and data streams respectively, whether manually mapped or automatically selected, except for those streams which are outputs of complex filtergraphs.
2. map可以选择输入文件中的特定stream，也可以选择filter_complex指定的特定输出。

示例：从多个输入流中选择合适的转码

Assume the following two input files:

input file 'A.avi'
      stream 0: video 640x360
      stream 1: audio 2 channels

input file 'B.mp4'
      stream 0: video 1920x1080
      stream 1: audio 2 channels
      stream 2: subtitles (text)
      stream 3: audio 5.1 channels
      stream 4: subtitles (text)

ffmpeg -i A.avi -i B.mp4 out1.mkv out2.wav -map 1:a -c:a copy out3.mov

按照规则，上述的输出是：
There are three output files specified, and for the first two, no -map options are set, so ffmpeg will select streams for these two files automatically.

1. out1.mkv is a Matroska container file and accepts video, audio and subtitle streams, so ffmpeg will try to select one of each type. For video, it will select stream 0 from B.mp4, which has the highest resolution among all the input video streams. For audio, it will select stream 3 from B.mp4, since it has the greatest number of channels. For subtitles, it will select stream 2 from B.mp4, which is the first subtitle stream from among A.avi and B.mp4.
2. out2.wav accepts only audio streams, so only stream 3 from B.mp4 is selected.
3. For out3.mov, since a -map option is set, no automatic stream selection will occur. The -map 1:a option will select all audio streams from the second input B.mp4. No other streams will be included in this output file.

• For the first two outputs, all included streams will be transcoded. The encoders chosen will be the default ones registered by each output format, which may not match the codec of the selected input streams.
• For the third output, codec option for audio streams has been set to copy, so no decoding-filtering-encoding operations will occur, or can occur. Packets of selected streams shall be conveyed from the input file and muxed within the output file.

日志级别

-loglevel [flags+]loglevel or -v [flags+]loglevel

For example to enable repeated log output, add the level prefix, and set loglevel to info:
ffmpeg -loglevel repeat+level+info -i input output ...

codec选择

-c[:stream_specifier] codec (input/output,per-stream)
-codec[:stream_specifier] codec (input/output,per-stream)
Select an encoder (when used before an output file) or a decoder (when used before an input file) for one or more streams. codec is the name of a decoder/encoder or a special value copy (output only) to indicate that the stream is not to be re-encoded.

示例：Encodes all video streams with libx264 and copies all audio streams.
ffmpeg -i INPUT -map 0 -c:v libx264 -c:a copy OUTPUT

帧选择（流选择）

-discard (input)
Allows discarding specific streams or frames from streams. Any input stream can be fully discarded, using value all whereas selective discarding of frames from a stream occurs at the demuxer and is not supported by all demuxers. 在transcoding时舍弃帧，参考 filter:select

• none
  Discard no frame.
• default
  Default, which discards no frames.
• noref
  Discard all non-reference frames.
• bidir
  Discard all bidirectional frames.
• nokey
  Discard all frames excepts keyframes.
• all
  Discard all frames.

其他常用的一些参数

偶尔用到的一些参数

• -copyts : Do not process input timestamps, but keep their values without trying to sanitize them. In particular, do not remove the initial start time offset value.
  Note that, depending on the vsync option or on specific muxer processing (e.g. in case the format option avoid_negative_ts is enabled) the output timestamps may mismatch with the input timestamps even when this option is selected.
• -start_at_zero : When used with copyts, shift input timestamps so they start at zero.
• -tag[:stream_specifier] : codec_tag (input/output,per-stream) Force a tag/fourcc for matching streams.
• -accurate_seek (input) : This option enables or disables accurate seeking in input files with the -ss option. It is enabled by default, so seeking is accurate when transcoding. Use -noaccurate_seek to disable it, which may be useful e.g. when copying some streams and transcoding the others.
• -re (input) : Read input at native frame rate. Mainly used to simulate a grab device, or live input stream (e.g. when reading from a file).

参考文献

1. https://ffmpeg.org/ffmpeg.html
2. http://ffmpeg.org/ffmpeg-filters.html

总阅读量：

• Previous
  图像处理领域顶级期刊
• Next
  Optimizing Transcoder Quality Targets Using a Neural Network with an Embedded Bitrate Model