I have some 4K HDR video, which looks very dark (and redish) if directly played on a non-HDR device (in practice with VLC), which is a known issue. I'm trying to convert it to SDR using ffmpeg, and when browsing the web there is a filter set that appears on many pages, so I went with it (just added the scale and unsharp filters):
ffmpeg -ss 75 -i in0.mkv -map 0:0 \
-vf scale=1280x720,unsharp,zscale=t=linear:npl=100,format=gbrpf32le,\
zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv,\
format=yuv420p -t 60 -y ou0.mkv
The problem is, the brightness is only a bit higher, and color tones are the same as on the input video. I'm not unfamiliar with colorspaces and so on, but I don't quite understand what is supposed to do this filter set, and what should be done to get the expected result.
Here is the terminal output from ffmpeg:
*****$ ffmpeg -ss 75 -i in0.mkv -map 0:0 -vf scale=1280x720,unsharp,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -t 60 -y ou0.mkv
ffmpeg version 4.4.4 Copyright (c) 2000-2023 the FFmpeg developers
built with Apple LLVM version 9.1.0 (clang-902.0.39.2)
configuration: --prefix=/opt/local --cc=/usr/bin/clang --mandir=/opt/local/share/man --enable-audiotoolbox --disable-indev=jack --disable-libjack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --enable-opencl --disable-outdev=xv --enable-sdl2 --disable-securetransport --enable-videotoolbox --enable-avfilter --enable-avresample --enable-fontconfig --enable-gnutls --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libfribidi --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libzimg --enable-libzvbi --enable-lzma --enable-pthreads --enable-shared --enable-swscale --enable-zlib --enable-libaom --enable-libsvtav1 --arch=x86_64 --enable-x86asm --enable-gpl --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxvid --enable-postproc
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, matroska,webm, from 'in0.mkv':
Metadata:
encoder : libebml v1.4.0 + libmatroska v1.6.2
creation_time : 2021-03-09T20:53:48.000000Z
Writing frontend: StaxRip v1.7.0.3
Duration: 00:49:10.18, start: 0.000000, bitrate: 4963 kb/s
Chapters:
[...]
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc (default)
Metadata:
BPS-eng : 4003363
DURATION-eng : 00:49:09.197000000
NUMBER_OF_FRAMES-eng: 70710
NUMBER_OF_BYTES-eng: 1475838552
_STATISTICS_WRITING_APP-eng: mkvmerge v50.0.0 ('Awakenings') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2021-03-09 20:53:48
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
[...]
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x7f9566801200] using SAR=1/1
[libx264 @ 0x7f9566801200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7f9566801200] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x7f9566801200] 264 - core 157 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=23 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'ou0.mkv':
Metadata:
Writing frontend: StaxRip v1.7.0.3
encoder : Lavf58.76.100
Chapters:
[...]
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 1k tbn (default)
Metadata:
BPS-eng : 4003363
DURATION-eng : 00:49:09.197000000
NUMBER_OF_FRAMES-eng: 70710
NUMBER_OF_BYTES-eng: 1475838552
_STATISTICS_WRITING_APP-eng: mkvmerge v50.0.0 ('Awakenings') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2021-03-09 20:53:48
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
encoder : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 1431 fps= 13 q=-1.0 Lsize= 6532kB time=00:00:59.89 bitrate= 893.4kbits/s speed=0.551x
video:6520kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.177061%
[libx264 @ 0x7f9566801200] frame I:12 Avg QP:17.09 size: 45016
[libx264 @ 0x7f9566801200] frame P:365 Avg QP:20.47 size: 9268
[libx264 @ 0x7f9566801200] frame B:1054 Avg QP:23.93 size: 2612
[libx264 @ 0x7f9566801200] consecutive B-frames: 1.5% 0.7% 0.2% 97.6%
[libx264 @ 0x7f9566801200] mb I I16..4: 13.1% 68.3% 18.6%
[libx264 @ 0x7f9566801200] mb P I16..4: 1.6% 5.9% 0.6% P16..4: 30.9% 7.0% 3.9% 0.0% 0.0% skip:50.1%
[libx264 @ 0x7f9566801200] mb B I16..4: 0.1% 0.3% 0.0% B16..8: 27.7% 1.8% 0.3% direct: 0.7% skip:69.0% L0:42.9% L1:54.3% BI: 2.9%
[libx264 @ 0x7f9566801200] 8x8 transform intra:70.6% inter:84.0%
[libx264 @ 0x7f9566801200] coded y,uvDC,uvAC intra: 41.5% 51.8% 27.2% inter: 5.2% 6.5% 0.7%
[libx264 @ 0x7f9566801200] i16 v,h,dc,p: 32% 21% 8% 39%
[libx264 @ 0x7f9566801200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 16% 27% 5% 5% 6% 5% 5% 5%
[libx264 @ 0x7f9566801200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 23% 14% 5% 7% 6% 7% 5% 4%
[libx264 @ 0x7f9566801200] i8c dc,h,v,p: 59% 16% 18% 7%
[libx264 @ 0x7f9566801200] Weighted P-Frames: Y:0.5% UV:0.3%
[libx264 @ 0x7f9566801200] ref P L0: 63.9% 8.5% 18.4% 9.2% 0.1%
[libx264 @ 0x7f9566801200] ref B L0: 89.3% 8.5% 2.3%
[libx264 @ 0x7f9566801200] ref B L1: 95.5% 4.5%
[libx264 @ 0x7f9566801200] kb/s:890.50
I also tried correcting the luminosity in the shadows with a simple curve filter (-vf curves=m='0/0 0.1/0.18 0.25/0.33 1/1'): it's not too bad, but the color tones are very difficult/almost impossible to correct like this. So I'd like to do that properly...
Any help appreciated, thanks
EDIT
I tried the suggestion by @Rotem in the comments to embed the resizing in the zscale filter. It's mucher better for the luminosity, but still the colors are out of tone (much too reddish).
Below are some screenshots of what I see on the screen:
Original file; this is initially a dark shot, but it appears even darker than it should be:
My initial correction; contrary to what I first wrote in the question it changes a bit, but this is very subtle:
The correction with @Rotem suggestion; it's much better for the brigthness (this is actually close to what it should be), but the color tone is still much too reddish:
Interestingly, this is the result of the VLC built-in capture, which is rather a frame extraction and and a screen capture. It shows the scene as it should be (it's still dark, but at least there are some details in the hair, and the color tones are very natural). I guess that VLC applies a proper color profile conversion when outputing the png file (which appears to have the sRGB profile):
