Opened 16 months ago

Closed 15 months ago

#13311 closed Developer Task (fixed)

VAAPI rewrite

Reported by: Peter Bennett Owned by: Peter Bennett
Priority: minor Milestone: 30.0
Component: MythTV - Video Decoding Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description (last modified by Peter Bennett)

Support the latest vaapi API from FFmpeg.

  1. Current vaapi support fails when there is a resolution change during playback. Fix this.
  2. Add support for vaapi hardware deinterlacing.
  3. Fix vaapi broken under wayland.

Attachments (18)

20180819_1111_vaaapi.patch (25.0 KB) - added by Peter Bennett 16 months ago.
Preview very lightly tested first phase of new vaapi support
20180819_1258_vaapi_add_xvideo.patch (26.0 KB) - added by Peter Bennett 16 months ago.
Another preview - fixes segfault on jellyfish, also adds the ability to use xvideo display with vaapi2 decoding in case opengl is not working.
20180820_1400_vaapi_device_option.patch (27.6 KB) - added by Peter Bennett 16 months ago.
Adds VAAPIDevice option. Try "-O VAAPIDevice=/dev/dri/renderD128" on mythfrontend. If this works I will add the option to the setup screens.
20180829_working_vaapi2.patch (66.5 KB) - added by Peter Bennett 15 months ago.
VAAPI hardware deinterlace is now working. If you already had installed a prior patch, please delete the VAAPI2 Normal Profile, exit setup and go in again and reselect it. This will re-add it with the new defaults.
20180830_1146_latest_master.patch (67.2 KB) - added by Peter Bennett 15 months ago.
Updated to work with latest master.
DI-issue.log.txt (290.6 KB) - added by warpme 15 months ago.
FE log for DI 1x 2x toggle issue
20180902_1519_vaapi2_working.patch (83.2 KB) - added by Peter Bennett 15 months ago.
Improved version of vaapi2. This adds the deinterlacers to the OSD menu. Also supports fallback deinterlacres for speedup and slow scan rate monitors.
gdb.txt (234.6 KB) - added by Mike Bibbings 15 months ago.
frontend seq fault gdb backtrace
mythfrontend.20180903114717.3379.log (26.2 KB) - added by Mike Bibbings 15 months ago.
frontend log associated with gdb backtrace
20180903_1555_livetv_fix.patch (86.1 KB) - added by Peter Bennett 15 months ago.
Fixes Live TV. Also fixes plugins build by removing requirement for mythcodecid.h
20180903_1849_fix_configure.patch (86.1 KB) - added by Peter Bennett 15 months ago.
Fix configure error that crept into the prior version
gdb.2.txt (139.0 KB) - added by Mike Bibbings 15 months ago.
gdb backtrace SD to HD switch
mythfrontend.20180904061030.26369.log (51.5 KB) - added by Mike Bibbings 15 months ago.
mythfrontend log SD to HD switch
trace2.txt (125.0 KB) - added by warpme 15 months ago.
fe segfault when LiveTV starts
gdb-audio-playback.txt (152.9 KB) - added by Mike Bibbings 15 months ago.
gdb backtrace seg fault audio only
mythfrontend.20180905112602.4783.log (35.6 KB) - added by Mike Bibbings 15 months ago.
mythfrontend log audio only seg fault
20180904_1601_fixed.patch (86.5 KB) - added by Peter Bennett 15 months ago.
Rolls back the previous patch. Instead, changes to make sure there is always a codec context available. This will prevent the seg faults that happened at various places.
20180909_1639_fix_configure.patch (77.9 KB) - added by Peter Bennett 15 months ago.
Cleaned up and ready for commit to master

Download all attachments as: .zip

Change History (48)

comment:1 Changed 16 months ago by Peter Bennett

Description: modified (diff)

The new decoding method is named "vaapi2" in the code and exists alongside the existing vaapi or glvaapi method. It uses very little of the existing vaapi code. If this all works we can compile with disable-vaapi and enable-vaapi2 to get rid of the obsolete code.

Changed 16 months ago by Peter Bennett

Attachment: 20180819_1111_vaaapi.patch added

Preview very lightly tested first phase of new vaapi support

comment:2 Changed 16 months ago by Peter Bennett

The first preview. 20180819_1111_vaaapi.patch, adds the new vaapi2 decoding method. It uses none of the code in vaapicontext.cpp and videoout_openglvaapi.cpp, instead using the standard videoout_opengl.cpp

It is still somewhat inefficient because it is doing scaling and conversion in software. It does fix the problem of failing when resolution changes. It plays 4K videos but crashes on the jellyfish 4K high bitrate video.

Changed 16 months ago by Peter Bennett

Another preview - fixes segfault on jellyfish, also adds the ability to use xvideo display with vaapi2 decoding in case opengl is not working.

comment:3 Changed 16 months ago by Stuart Auchterlonie

Milestone: needs_triage30.0

See also https://code.mythtv.org/trac/ticket/13104 "vaapi fails to initialize under wayland"

comment:4 Changed 16 months ago by Stuart Auchterlonie

Testing results:

  • f25 system, works with both vaapi and vaapi2, mpeg and h264 content
  • f28 system (the one with wayland) doesn't initialize properly with

20180819_1258_vaapi_add_xvideo.patch, however after applying

Index: mythtv/mythtv/libs/libmythtv/vaapi2context.cpp
===================================================================
--- mythtv.orig/mythtv/libs/libmythtv/vaapi2context.cpp
+++ mythtv/mythtv/libs/libmythtv/vaapi2context.cpp
@@ -78,7 +78,7 @@ int Vaapi2Context::HwDecoderInit(AVCodec
     AVBufferRef *hw_device_ctx = NULL;
 
     ret = av_hwdevice_ctx_create(&hw_device_ctx, AV_HWDEVICE_TYPE_VAAPI,
-                                      NULL, NULL, 0);
+                                      "/dev/dri/renderD128", NULL, 0);
     if (ret < 0)
     {
         char error[AV_ERROR_MAX_STRING_SIZE];

Then VAAPI successfully initializes. I can also playback MPEG2 and H264 content, although I have 1 H264 recording which seems to wedge the decoder.

Regards Stuart

Changed 16 months ago by Peter Bennett

Adds VAAPIDevice option. Try "-O VAAPIDevice=/dev/dri/renderD128" on mythfrontend. If this works I will add the option to the setup screens.

comment:5 Changed 16 months ago by Stuart Auchterlonie

The override for the VAAPIDevice works correctly.

Changed 15 months ago by Peter Bennett

VAAPI hardware deinterlace is now working. If you already had installed a prior patch, please delete the VAAPI2 Normal Profile, exit setup and go in again and reselect it. This will re-add it with the new defaults.

comment:6 Changed 15 months ago by Peter Bennett

Speedup is jerky with 2x deinterlace and with 60 fps content, because MythTV is using the minitor refresh to sync and my monitor has a 59.95 hz refresh rate. We need to do something like drop every second frame when speeded up or switch to another sync method. Any suggestions? For now, selecting a 1x deinterlacer allows speedup, but nothing helps for 60fps content.

NOTE: xvideo output methods are no longer enabled with vaapi2. They were causing seg faults and it did not seem worth while pursuing it.

Last edited 15 months ago by Peter Bennett (previous) (diff)

comment:7 Changed 15 months ago by warpme

Peter, quick Q: I just applied 20180829-working-vaapi2on current master and receiving:

avformatdecoder.cpp: In member function ‘void AvFormatDecoder::InitVideoCodec(AVStream*, AVCodecContext*, bool)’:
avformatdecoder.cpp:1711:13: error: invalid use of incomplete type ‘class MythCodecContext’
         mctx->setPlayer(m_parent);
             ^~
In file included from decoderbase.h:15:0,
                 from avformatdecoder.h:13,
                 from avformatdecoder.cpp:18:

this is from incremental compile.

Probably I miss something? pls advice

comment:8 Changed 15 months ago by Peter Bennett

You will need to run configure. Without running configure VAAPI2 is disabled. I need to fix it to prevent compile errors when compiling with VAAPI2 disabled.

Some clean up and code fixes still have to be done.

Thank you for testing it. Let me know your results.

Changed 15 months ago by Peter Bennett

Updated to work with latest master.

comment:9 Changed 15 months ago by warpme

Peter, IMHO very good work!.

Short feedback:

  1. Install process is missing mythcodecid.h so my build is failing on mythmusic plugin build. Manual copy fixes issue.

2.Playback OSD shows for me full list of deinterlacers - but there is no VA specific deinterlacers on list. Any selection has no impact on playback.

3.VA deinterlacers are working really nice for me. Playback 25i on 50p is silky smooth (IMHO comparable with nvidia 2x HW DI which is reference for me). This is very good!

4.Briefly tested on some H264 and MPEG2 on SD & HD content. So far all plays perfectly!

If I'll discover any playback issues - ill report ASAP.

Great work!

comment:10 Changed 15 months ago by Peter Bennett

I did not realize you can change deinterlacer via the OSD menu. I need to fix that.

The VAAPI deinterlace is decoder based and the others are video output based. Selecting one of those others seems to add a second deinterlacer without stopping the VAAPI deinterlacer so it is not good.

Thanks for pointing this out.

Do people really need to change deinterlacer while playing? Perhaps if a VAAPI deinterlacer is currently running I will prevent that.

comment:11 Changed 15 months ago by warpme

Peter,

"Do people really need to change deinterlacer while playing? Perhaps if a VAAPI deinterlacer is currently running I will prevent that."

I think ability to change DI in OSD isn't critical but nice to have. IMHO generally: having relevant (per current decode type) list of DIs in OSD is one from these advanced aspects making mythtv one from best players on market. To keep this crown IMHO we should try keep these advanced functions - despite additional work it needs.

So I would suggest: if about of work to having relevant (per current decode type) list of DIs in OSD is worth for You - pls implement this. If not - all perfect. You are already achieving incredible things here....

comment:12 Changed 15 months ago by ijc

Do people really need to change deinterlacer while playing?

My guess is that the major use case is when experimenting to find the best settings for your setup, where leaping back and forth between the settings and playback is a faff. In normal use it seems like it would be much less useful.

Maybe that sort of setup tweaking is not something which should happen when VAAPI deinterlacing is active though?

Are there circumstances where the selection of deinterlacer can depend on the content being played back (as opposed to the hardware in use etc), e.g. perhaps one channel is better with one deint and another with a different one?

comment:13 Changed 15 months ago by Peter Bennett

I have enough comments in support of it that I will make sure the deinterlacer can be changed during playback.

comment:14 Changed 15 months ago by warpme

Peter,

I do a bit more playback tests and discover interesting issue:

-star playback. DI is silk smooth (like nvidia HW DI 2x frame rate). I see this on news channel, scrolling band with news.

-press jump forward. DI is not silk smooth ((like nvidia HW DI 1x frame rate)

-press jump forward. DI is again silk smooth ((like nvidia HW DI 2x frame rate)

-press jump forward. DI is not silk smooth ((like nvidia HW DI 1x frame rate)

-press jump forward. DI is again silk smooth ((like nvidia HW DI 2x frame rate)

I see issue: jump in playback has exact toggle effect on DI: smooth/not smooth (or 2x frame rate vs 1x frame rate).

Looking on log I dno't see anything. Log from FE attached.

Changed 15 months ago by warpme

Attachment: DI-issue.log.txt added

FE log for DI 1x 2x toggle issue

Changed 15 months ago by Peter Bennett

Improved version of vaapi2. This adds the deinterlacers to the OSD menu. Also supports fallback deinterlacres for speedup and slow scan rate monitors.

comment:15 Changed 15 months ago by Peter Bennett

Piotr:

Please try the jump-forward issue with latest patch. I cannot recreate the problem. I have made many improvements and it may have been fixed.

If you still have the problem please use -v playback. I have added messages to the log to tell which deinterlacer is being used. That may tell us what is happening.

Note that if you use time stretch to speed up or slow down it switches to the fallback deinterlacer and this is not smooth.

Please also check the refresh rate of your screen if it still happens.

Last edited 15 months ago by Peter Bennett (previous) (diff)

comment:16 Changed 15 months ago by Mike Bibbings

Peter,

I applied your latest patch 20180902_1519_vaapi2_working.patch

Playback of previous recordings works with vappi2, not tried different interlacers

LiveTV seg faults, irrespective of which Video profile is being used, gdb backtrace attached, as is frontend log

Build environment setup using ansible on Xubuntu 18.04.1

backend build is current master, built using default ./configure mike@1804-test:~$ mythbackend --version Please attach all output as a file in bug reports. MythTV Version : v30-Pre-767-gea500ae374 MythTV Branch : master Network Protocol : 91 Library API : 30.20180319-1 QT Version : 5.9.5 Options compiled in:

linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_crystalhd using_dvb using_firewire using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr using_ivtv using_joystick_menu using_libcec using_libcrypto using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg using_opengl using_opengl_video using_opengl_themepainter using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_x11 using_xnvctrl using_xnvctrl_external using_libbluray_external using_xrandr using_xv using_profiletype using_systemd_notify using_systemd_journal using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_vaapi using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2

Backend Machine Information: motherboard M5A78L-M/USB3, BIOS 2101 12/02/2014 CPU AMD Phenom(tm) II X4 965 Processor (family: 0x10, model: 0x4, stepping: 0x3) RAM 8GB Capture Cards:

TBS 6981 Dual DVB-S/S2 TBS 6280 Dual DVB-T/T2

Frontend built using default ./configure and patch applied. mike@mike-Aspire-V3-571:~$ mythfrontend --version Please attach all output as a file in bug reports. MythTV Version : v30-Pre-767-gea500ae374-dirty MythTV Branch : master Network Protocol : 91 Library API : 30.20180319-1 QT Version : 5.9.5 Options compiled in:

linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_crystalhd using_dvb using_firewire using_frontend using_hdhomerun using_vbox using_ceton using_hdpvr using_ivtv using_joystick_menu using_libcec using_libcrypto using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg using_opengl using_opengl_video using_opengl_themepainter using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_x11 using_xnvctrl using_xnvctrl_external using_libbluray_external using_xrandr using_xv using_profiletype using_systemd_notify using_systemd_journal using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_vaapi using_vaapi2 using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2

Frontend Machine Information: Laptop Acer Aspire V3-571/VA50_HC_CR, BIOS V2.15 03/11/2013 CPU Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz (family: 0x6, model: 0x3a, stepping: 0x9) RAM 8GB

Mike

Changed 15 months ago by Mike Bibbings

Attachment: gdb.txt added

frontend seq fault gdb backtrace

Changed 15 months ago by Mike Bibbings

frontend log associated with gdb backtrace

comment:17 Changed 15 months ago by ijc

Some tiny things I noticed in 20180902_1519_vaapi2_working.patch:

--disable-vaapi2 disable VAAPI@ hardware accelerated video decoding

I suspect that "@" is meant to be a "2".

In mythtv/libs/libmythtv/avformatdecoder.cpp line 2625 (the if (!foundgpudecoder) block) you have an #endif // USING_MEDIACODEC which I think should be saying USING_VAAPI2.

comment:18 Changed 15 months ago by warpme

Peter, I give try for 20180902_1519_vaapi2_working.patch

  1. issue with missing mythcodecid.h in <include> (so myth music plugin build fails) is still present for me.
  1. Also I have segfault at LiveTV start (like in #comment16)
  1. Issue described by me in #comment14 is still present.

I'm not seeing any special in log. Maybe You will be able try to reproduce in Your system with my content? Short sample is here: http://warped.inet2.org/sample7.ts it is HD 50i. I'm playing it on 50Hz refresh monitor to have 2x HW DI ​

Changed 15 months ago by Peter Bennett

Fixes Live TV. Also fixes plugins build by removing requirement for mythcodecid.h

comment:19 Changed 15 months ago by Peter Bennett

Code is still messy with lots of commented out stuff. I will fix that.

I cannot reproduce the issue with forward jump. I tried the supplied sample with 60hz and 50hz refresh rate. After jump, sometimes there are a few seconds where the ticker speeds up, slows down or jumps to a different place, but after that it seems to be as smooth as before.

The log should tell which deinterlacer is being used, and if it switches to a different one.

Changed 15 months ago by Peter Bennett

Fix configure error that crept into the prior version

comment:20 Changed 15 months ago by Mike Bibbings

Peter,

Applied latest patch 20180903_1849_fix_configure.patch​.

LiveTV now working with vaapi2. However changing channel in LiveTV from SD broadcast to HD broadcast results in mythfrontend seg fault. Attached are gdb backtrace and mythfrontend log

Mike

Changed 15 months ago by Mike Bibbings

Attachment: gdb.2.txt added

gdb backtrace SD to HD switch

Changed 15 months ago by Mike Bibbings

mythfrontend log SD to HD switch

comment:21 Changed 15 months ago by warpme

Peter, Unfortunately also I'm getting as well segfault with starting LiveTV. Pls see attached gdb trace. This is from current master with latest patch applied.

Changed 15 months ago by warpme

Attachment: trace2.txt added

fe segfault when LiveTV starts

comment:22 Changed 15 months ago by Mike Bibbings

Peter,

Still with 20180903_1849_fix_configure.patch​

Pressing "m" (Menu)" key when LiveTV is playing a radio only channel results in seg fault. gdb backtrace and frontend log attached

Changed 15 months ago by Mike Bibbings

Attachment: gdb-audio-playback.txt added

gdb backtrace seg fault audio only

Changed 15 months ago by Mike Bibbings

mythfrontend log audio only seg fault

comment:23 Changed 15 months ago by Peter Bennett

All of these seg faults are a result of the new concept where the deinterlacer can be in the decoder or the renderer. In many cases the code was looking for the deinterlacer before the decoder was set up (or in the case of radio, where no decoder was set up) and that resulted in a seg fault. I am rolling back the latest abortive fix and redesigning how the decoder based deinterlacer is set up so as to make sure this cannot happen.

Changed 15 months ago by Peter Bennett

Attachment: 20180904_1601_fixed.patch added

Rolls back the previous patch. Instead, changes to make sure there is always a codec context available. This will prevent the seg faults that happened at various places.

comment:24 Changed 15 months ago by Peter Bennett

Hopefully this will fix all of the seg faults. Code still needs to be cleaned up.

comment:25 Changed 15 months ago by Mike Bibbings

Peter,

Latest patch has resolved my seg fault issues.

However, I did uncover an existing mythfrontend hang problem on Audio only channel exit from LiveTV (nothing to do with your patch), I have raised a ticket on current master see https://code.mythtv.org/trac/ticket/13319

Mike

Changed 15 months ago by Peter Bennett

Cleaned up and ready for commit to master

comment:26 Changed 15 months ago by Peter Bennett

Built and tested on several environments, and tested other playback methods to make sure they are not affected.

comment:27 Changed 15 months ago by warpme

Peter,

I applied 20180909_1639_fix_configure.patch. So far it works well except:

1.issud in comment #14

2.CPU load is noticeable higher compared to original VAAPI

on my i5 NUC CPU load is: 2-5% on VAAPI 15-25% no VAAPI2

CPU load is of course very minor thing - but I'm just wonder why there is so high difference?

maybe issue at comment #14 is specific to my hardware or content - I don't know. Let's see what userbase will see.

comment:28 Changed 15 months ago by Peter Bennett

Issue comment 14 : Does it not settle down a few seconds after the jump forward? I tried your clip. I do not see the problem, it is slightly uneven on my system because I have a refresh rate of 60, but it is the same before and after a jump. Perhaps you can try different interlacers, also look at the cpu usage, does that change? Run with -v playback and look to see if it is dropping frames when it is uneven.

CPU Load : I looked at this. It is because of the 2x VA deinterlacers. If you change to a VA deinterlacer that is not 2x you will see less CPU usage. The 2x VA deinterlacers produce double the frame rate (50 fps for a 25 fps video). The whole mythtv frame cycle is repeated 50 times a second. With the 2x GL deinterlacers, only the video display part of the processing is repeated 50 times a second, the rest of the frame cycle is repeated only 25 times a second. There is a lot of processing in the mythtv frame cycle, it checks for keyboard input, synchronizes audio/video, checks for speedup, updates OSD, plus other things.

comment:29 Changed 15 months ago by Peter Bennett

Description: modified (diff)

comment:30 Changed 15 months ago by Peter Bennett <pbennett@…>

Resolution: fixed
Status: newclosed

In ff7a1478f/mythtv:

VAAPI Rewrite: Support the latest VAAPI from FFmpeg.

The new vaapi code is called vaapi2 in the profile selection.
It adds the following:

  • Fixes problems where vaapi fails when there is a resolution change during playback.
  • Adds support for vaapi hardware deinterlacing.
  • Fixes vaapi broken under wayland.
  • Adds support for decoder based deinterlacing. Previously deinterlacing was always part of rendering. Now both methods are possible.

It uses the standard OpenGL rendering in the code, unlike the prior vaapi
support, which created a special render class called OpenGLVAAPI.

It does not share code with the old vaapi. The old vaapi can be disabled
and eventually removed.

Fixes #13311

Note: See TracTickets for help on using tickets.